Data importing method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses a data importing method and device, computer equipment and a storage medium. The data import method comprises the following steps: acquiring a data import request and an import request type of the data import request; analyzing the data import request to obtain data address information and a data import condition corresponding to the type of the import request; determining a target data table according to the data address information, performing data editing processing on the target data table according to the data import condition to obtain to-be-imported data corresponding to the data import request, and sending the to-be-imported data to the client; and when import confirmation information corresponding to the to-be-imported data sent by the client is received, importing the to-be-imported data into the target data table according to the data import condition to obtain an updated target data table. According to the invention, the efficiency of data import is improved, the flexibility of data processing is enhanced, the accuracy of data import is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a data import method, apparatus, computer device, and storage medium. Background Art

[0002] During the data testing process, it is often necessary to import a large amount of data into a database table. When the test environment changes with the date, in order to meet the test requirements for a specific time period, it is also necessary to generate data within different date ranges.

[0003] The existing data import uses a manual operation method, which has some defects. First, the process of manually importing data is cumbersome, resulting in low efficiency. Second, when the database table lacks data for a specific time period or under specific conditions, it is impossible to flexibly implement the replacement and insertion of data, and data redundancy may also be caused by repeated imports. Finally, users cannot verify the correctness of the data before actual import, and it is easy to affect the test effect due to data import errors. Therefore, there is an urgent need for an automated data import method to improve the efficiency and accuracy of data import. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a data import method, apparatus, computer device, and storage medium to solve the problems of low efficiency, poor flexibility, and easy errors in the existing data import operation method.

[0005] A data import method includes: Obtaining a data import request and the import request type of the data import request; Parsing the data import request to obtain data address information and data import conditions corresponding to the import request type; Determining a target data table according to the data address information, and performing data editing processing on the target data table according to the data import conditions to obtain data to be imported corresponding to the data import request, and sending the data to be imported to the client; When receiving the import confirmation information corresponding to the data to be imported sent by the client, importing the data to be imported into the target data table according to the data import conditions to obtain an updated target data table.

[0006] A data import apparatus includes: A request acquisition module for obtaining a data import request and the import request type of the data import request; A request parsing module for parsing the data import request to obtain data address information and data import conditions corresponding to the import request type; An imported data determination module, configured to determine a target data table according to the data address information, perform data editing processing on the target data table according to the data import condition, obtain imported data corresponding to the data import request, and send the imported data to a client; A data import module, configured to, when receiving import confirmation information corresponding to the imported data sent by the client, import the imported data into the target data table according to the data import condition, and obtain an updated target data table.

[0007] A computer device includes a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the above data import method is implemented.

[0008] A computer-readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the data import method as described above.

[0009] In the above data import method, apparatus, computer device, and storage medium, the data import method obtains a data import request and the import request type of the data import request; parses the data import request to obtain data address information and a data import condition corresponding to the import request type; determines a target data table according to the data address information, performs data editing processing on the target data table according to the data import condition, obtains imported data corresponding to the data import request, and sends the imported data to a client; when receiving import confirmation information corresponding to the imported data sent by the client, imports the imported data into the target data table according to the data import condition, and obtains an updated target data table. By parsing the data import request, the present invention obtains the data import condition. Based on the data import condition, automated batch import and replacement insertion functions can be implemented, improving the efficiency of data import. At the same time, different import request types correspond to different data import conditions, enabling batch import processing of data within different time ranges and flexible replacement insertion processing of specific data, enhancing the flexibility of data processing. In addition, the present invention implements a data preview function by sending the imported data to the client for import confirmation, avoiding errors during the actual data import process, improving the accuracy of data import, and enhancing the user experience. Description of the Drawings

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0011] Figure 1 It is a schematic flowchart of a data import method in an embodiment of the present invention; Figure 2 It is a schematic structural diagram of a data import device in an embodiment of the present invention; Figure 3 It is a schematic diagram of a computer device in an embodiment of the present invention. Detailed implementation manners

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0013] The data import method provided in this embodiment can be applied to the application scenario of generating and amplifying test data through data import during the data testing process. The scenario includes a client and a server, and the client communicates with the server. Among them, the client includes, but is not limited to, various personal computers, laptop computers, smart phones, and tablet computers. The server can be implemented by an independent server or a server cluster composed of multiple servers. The user configures the data import requirements through the client. The client generates a data import request according to the configured requirement information and sends the data import request to the server. The server obtains the data import request and the import request type of the data import request; parses the data import request to obtain the data address information and the data import conditions corresponding to the import request type; determines the target data table according to the data address information, and performs data editing processing on the target data table according to the data import conditions to obtain the data to be imported corresponding to the data import request, and sends the data to be imported to the client; when receiving the import confirmation information corresponding to the data to be imported sent by the client, imports the data to be imported into the target data table according to the data import conditions to obtain the updated target data table.

[0014] In one embodiment, as Figure 1 shown, a data import method is provided, including the following steps S10 - S40: S10. Obtain a data import request and the import request type of the data import request.

[0015] Understandably, a data import request refers to the request information sent by the client to the server for importing specific data into a specified database. The data import request contains a series of key information pre-configured by the user according to the data import requirements, so that the server can accurately understand and execute the data import task, such as the import request type. Among them, the import request type is used to represent the category of import operations required when importing data, including the batch import type and the replace and insert type. The batch import type refers to the category of operations for importing a large amount of data into the database table on the server at one time, such as data update or data overwrite scenarios. The replace and insert type refers to the category of operations for importing partial data into the database table on the server and replacing the existing data in the database table, such as scenarios where only new data or changed data is imported.

[0016] S20. Parse the data import request to obtain the data address information and the data import conditions corresponding to the import request type.

[0017] Understandably, when the server parses the data import request, it can extract the data address information and the data import conditions corresponding to the import request type. The data address information refers to the address information of the database table for data import operations configured by the user in the data import request, such as file paths or address links. The data import conditions refer to a series of rules and parameter settings that need to be met when performing data import operations, and the data import conditions can ensure that data can be accurately and efficiently imported into the database table.

[0018] S30. Determine the target data table according to the data address information, perform data editing processing on the target data table according to the data import conditions to obtain the data to be imported corresponding to the data import request, and send the data to be imported to the client.

[0019] Understandably, the server can determine the target data table according to the data address information. The target data table refers to the data table that needs to perform data import operations. The data import operation process in this embodiment is based on a part of the data stored in the target data table for transformation processing, and then re-imported into the target data table. The target data table is both the data table for reading the data source and the data table for receiving data. Different import request types correspond to different data import conditions. The server performs data editing processing on the target data table according to the data import conditions to obtain the data to be imported corresponding to the data import request. Data editing includes processes such as data copying, transformation, and screening. The data to be imported refers to the data that needs to be imported into the target data table after processing. After obtaining the data to be imported, the server sends the data to be imported to the client so that the user can preview and review the data to be imported through the visual operation interface of the client. After the user confirms that the data to be imported is correct, the user sends an import confirmation message to the server through the client.

[0020] S40. When receiving the import confirmation message corresponding to the data to be imported sent by the client, import the data to be imported into the target data table according to the data import conditions to obtain an updated target data table.

[0021] Understandably, the server needs to import the data to be imported into the target data table according to the data import conditions after receiving the import confirmation message corresponding to the data to be imported sent by the client. The import confirmation message refers to the information sent by the client to the server to indicate that the server starts to perform the data import operation. Information such as the data storage location and import method when performing the data import operation is recorded in the data import conditions. According to the data import conditions, the data to be imported can be accurately written into the target data table to obtain an updated target data table. After the server completes the data import operation, it generates a data import record corresponding to the data import request and the data to be imported, realizing the record management of the data import operation. The user can view the records of the imported data through the client in the record management interface according to conditions such as data address information and import request type to ensure the traceability of the data import operation and avoid duplicate operations.

[0022] In this embodiment, a data import request and the import request type of the data import request are obtained; the data import request is parsed to obtain data address information and data import conditions corresponding to the import request type; a target data table is determined according to the data address information, and data editing processing is performed on the target data table according to the data import conditions to obtain data to be imported corresponding to the data import request, and the data to be imported is sent to the client; when the import confirmation information corresponding to the data to be imported sent by the client is received, the data to be imported is imported into the target data table according to the data import conditions to obtain an updated target data table. In this embodiment, the data import conditions are obtained by parsing the data import request. Based on the data import conditions, automated batch import and replacement insertion functions can be realized, improving the efficiency of data import. At the same time, different import request types correspond to different data import conditions, enabling batch import processing of data in different time ranges and flexible replacement insertion processing of specific data, enhancing the flexibility of data processing. In addition, in this embodiment, the data preview function is realized by sending the data to be imported to the client for import confirmation, avoiding errors during the actual data import process, improving the accuracy of data import, and enhancing the user experience.

[0023] In one embodiment, the data import request includes a batch import request; before step S10, that is, before obtaining the data import request, it includes: S101. Receive import type selection information. When it is confirmed according to the import type selection information that the import request type is the batch import type, send a batch import configuration request corresponding to the batch import type to the client; S102. When the data address configuration information, query time configuration information, import time configuration information, and import method configuration information corresponding to the batch import configuration request sent by the client are received, generate the batch import request according to the data address configuration information, query time configuration information, import time configuration information, and import method configuration information.

[0024] Understandably, the data import request includes a batch import request. When a user needs to batch import data, the user can enter the requirement configuration information through the visual operation interface of the client. When the user enters the batch import configuration information through the batch import configuration page, the user can input it by selecting content from a drop-down list or by manually entering content. Before the server obtains the data import request, it is necessary to generate a batch import request according to the requirement configuration information entered through the client. First, the visual operation interface of the client displays import type options, such as two options: batch import type and replace insert type. The user can select one of the options according to the need. When the user selects the batch import type, the server receives the import type selection information, confirms that the import request type is the batch import type according to the import type selection information, and sends a batch import configuration request corresponding to the batch import type to the client. The import type selection information refers to the information used to represent the category of the data import operation selected by the user. The batch import configuration request refers to the request information used to enable the user to configure the batch import operation rules according to the requirements. Then, the client displays a batch import configuration page for configuring the batch import operation rules in the visual operation interface according to the batch import configuration request, so that after the user enters the batch import configuration information through the batch import configuration page, the client sends the batch import configuration information to the server. Finally, when the server receives the batch import configuration information sent by the client, it generates a batch import request according to the batch import configuration information.

[0025] Among them, the batch import configuration information includes data address configuration information, query time configuration information, import time configuration information, and import method configuration information corresponding to the batch import configuration request. The data address configuration information refers to the data table address where the data import operation needs to be performed entered by the user, that is, the address in the target data table. For example, the mysql or click house data source and the corresponding library table address information configured in the database (data_import_source). The data in the target data table is stored in a structured data manner associated with the time field. The query time configuration information refers to the time corresponding to when the user needs to read data from the target data table, such as the time field. The time field types of the query time configuration information include four types: day, week, month, and year. The user can select one of them according to the need. The import time configuration information refers to the time corresponding to when the user needs to write data to the target data table, such as the insert time range. The time field types of the import time configuration information and the query time configuration information are the same. For example, if the query time configuration information is accurate to the date, the corresponding import time configuration information is the number of days in the time range from "a certain day to a certain day". The import method configuration information refers to the writing method when writing data in the target data table entered by the user, such as the skip import method and the overwrite import method.

[0026] In one embodiment, first, the user selects the batch import type through the visual operation interface of the client. Then, on the batch import configuration page, select the "wide_ed_day_d_all" data table under the "goldeneye" database in the "CK goldeneye" data source. Then the data address configuration information includes "CK goldeneye", "goldeneye", and "wide_ed_day_d_all". Then, select the time field "biz_date", set the data date to "2024-01-01", and determine that the time field is of the "day" type. Then the query time configuration information includes "biz_date", "2024-01-01", and "day". Next, set the time range to "2024-01-01" to "2024-01-31". Then the import time configuration information includes "2024-01-01" and "2024-01-31". Finally, select the insertion method as "skip". Then the import method configuration information is "skip import method". At this time, the batch import request means to batch import the entire month's data in January 2024 in the "wide_ed_day_d_all" data table based on the data on "2024-01-01" in a skip import method.

[0027] This embodiment simplifies the user configuration operation and improves the user experience through an intuitive user interface and operation process. At the same time, based on the data address configuration information, query time configuration information, import time configuration information, and import method configuration information, batch processing of data for different time ranges can be configured, meeting the flexible requirements of users for batch data import. In addition, through intelligent time field processing and configuration, the time consumption of users in the preliminary preparation work of data import is reduced, and the accuracy and efficiency of data import are improved.

[0028] In one embodiment, the data import request includes a replace-insert request; before step S10, that is, before obtaining the data import request, it includes: S103. Receive the import type selection information. When it is confirmed that the import request type is the replace-insert type according to the import type selection information, send a replace-insert configuration request corresponding to the replace-insert type to the client; S104. When receiving the data address configuration information, and at least one set of replace field configuration information, replace query configuration information, and replace import configuration information sent by the client corresponding to the replace-insert configuration request, generate the replace-insert request according to the data address configuration information, and at least one set of replace field configuration information, replace query configuration information, and replace import configuration information.

[0029] Understandably, the data import request includes a replacement insertion request. When the user needs to replace and insert data, the user can enter the required configuration information through the visual operation interface of the client. First, when the user selects the replacement insertion type, the server receives the import type selection information, confirms that the import request type is the replacement insertion type according to the import type selection information, and sends a replacement insertion configuration request corresponding to the replacement insertion type to the client. The replacement insertion configuration request refers to the request information used to enable the user to configure the replacement insertion operation rules according to the requirements. Then, the client displays the replacement insertion configuration page on the visual operation interface according to the replacement insertion configuration request, so that after the user enters the replacement insertion configuration information through the replacement insertion configuration page, the client sends the replacement insertion configuration information to the server. Finally, when the server receives the replacement insertion configuration information sent by the client, it generates a replacement insertion request according to the replacement insertion configuration information.

[0030] Among them, the replacement insertion configuration information includes data address configuration information and at least one set of replacement field configuration information, replacement query configuration information, and replacement import configuration information. The replacement field configuration information, replacement query configuration information, and replacement import configuration information constitute a set of replacement operation conditions. The replacement insertion configuration information can contain only one set of replacement operation conditions or multiple sets of replacement operation conditions at the same time. The replacement field configuration information refers to the fields corresponding to the data that needs to be replaced in the target data table entered by the user, such as the time field. The replacement query configuration information refers to the field values in the target data table that need to be replaced entered by the user, that is, the original data before replacement. The replacement import configuration information refers to the field values that need to be written into the target data table entered by the user, that is, the new data after replacement.

[0031] In one embodiment, first, the user selects a replacement and insertion type through the visual operation interface of the client. Then, on the replacement and insertion configuration page, the user selects the data table "kpi_ed_box_day_16" under the "dw_app" database in the "mysql" data source. At this time, the data address configuration information includes "mysql", "dw_app", and "kpi_ed_box_day_16". Then, the user selects the replacement field as the cabinet code "ed_code", sets the original value to "FC000001", and sets the inserted value to "FC000002", thus forming a set of replacement field configuration information "ed_code", replacement query configuration information "FC000001", and replacement import configuration information "FC000002". Finally, the user selects the replacement field as the operation position employee number "charge_emp_no", sets the original value to "001149", and sets the inserted value to "004219", thus forming another set of replacement field configuration information "charge_emp_no", replacement query configuration information "001149", and replacement import configuration information "004219". At this time, the replacement and insertion request indicates that, based on the data in the "kpi_ed_box_day_16" data table where the cabinet code is "FC000001" and the operation position employee number is "001149", the data is replaced and inserted with the data where the cabinet code is "FC000002" and the operation position employee number is "004219".

[0032] This embodiment simplifies the user configuration operation and improves the user experience through an intuitive user interface and operation process. At the same time, based on the data address configuration information, replacement field configuration information, replacement query configuration information, and replacement import configuration information, the replacement and insertion of different conditional data can be configured, meeting the flexible requirements of users for data replacement and insertion. In addition, through intelligent replacement field processing and configuration, the accuracy and efficiency of replacement and insertion are improved.

[0033] In one embodiment, in step S101, that is, after sending the batch import configuration request corresponding to the batch import type to the client, it includes: S1011. When receiving the data address configuration information, query time configuration information, import time configuration information, import method configuration information, and additional condition configuration information sent by the client corresponding to the batch import configuration request, generate the batch import request according to the data address configuration information, query time configuration information, import time configuration information, import method configuration information, and additional condition configuration information. The additional condition configuration information includes import screening configuration information and / or import random number configuration information.

[0034] Understandably, during the process of the user entering the batch import configuration information through the batch import configuration page, in addition to the data address configuration information, query time configuration information, import time configuration information, and import method configuration information, additional condition configuration information can also be entered as needed. The additional condition configuration information refers to additional condition or rule information used to further refine or limit the batch import process. The additional condition configuration information includes import screening configuration information and / or import random number configuration information. The import screening configuration information refers to the information entered by the user for screening data based on one or more field names other than the time field. The import random number configuration information refers to the information entered by the user for generating random data based on the specified field name and random number range. Similarly, during the process of the user entering the replacement and insertion configuration information through the replacement and insertion configuration page, in addition to the data address configuration information, replacement field configuration information, replacement query configuration information, and replacement import configuration information, import screening configuration information and / or import random number configuration information can also be entered as needed.

[0035] In one embodiment, the batch import configuration information includes import screening configuration information and import random number configuration information. Among them, the screening field in the import screening configuration information is "responsibility area", and the set content is "Hunan-Hubei region". That is, when the user does not need the full volume of data associated with the date, based on the time range condition, data associated with the Hunan-Hubei region on a specific date is further screened to generate the data to be imported for subsequent import. The random number field in the import random number configuration information is the delivery volume "post_cnt", and the set random number range is "20-40", that is, a random delivery volume within the range of "20-40" is generated based on the time condition and added to the data to be imported.

[0036] In this embodiment, during the batch import and replacement and insertion configuration requirements process, flexible random field configuration and screening configuration functions are realized to meet specific data generation requirements and data insertion requirements in different scenarios. Among them, the random field processing function can generate random numbers according to user needs, which is applicable to scenarios such as test data generation, enriches the diversity of data, and enhances the flexibility of data processing.

[0037] In one embodiment, the data import conditions include query time information, import time information, and import method information; in step S30, that is, the data editing process is performed on the target data table according to the data import conditions to obtain the data to be imported corresponding to the data import request, including: S301. Search for reference data from the target data table according to the query time information; S302. Perform data copying process on the reference data according to the import time information to obtain the data to be imported; In step S40, that is, importing the data to be imported into the target data table according to the data import conditions to obtain an updated target data table, includes: S401. Import the data to be imported into the target data table according to the import method information to obtain an updated target data table.

[0038] Understandably, when the data import request is a batch import request, after parsing the data import request, the obtained data import conditions include query time information corresponding to the query time configuration information, import time information corresponding to the import time configuration information, and import method information corresponding to the import method configuration information. In the process of performing data editing processing on the target data table according to the data import conditions, first, find the reference data from the target data table according to the query time information. The reference data refers to the original data read from the target data table. Then, perform data replication processing on the reference data according to the time range corresponding to the import time information to obtain the data to be imported. For example, if the reference data is one day's data and the import time information is 30 days, then the data to be imported is the data obtained by replicating the reference data 30 times for 30 days. Finally, use the operation process corresponding to the import method information to import the data to be imported into the target data table to obtain an updated target data table.

[0039] In one embodiment, first, the query time configuration information includes "biz_date", "2024-01-01", and "day", then the reference data is the data for the day of 2024-01-01. Second, the import time configuration information includes "2024-01-01" and "2024-01-31", then the data to be imported is the data for 31 days from 2024-01-01 to 2024-01-31, and the data for each day is the same as the data for the day of 2024-01-01. Finally, the import method configuration information is "overwrite import method", then the import method information is the overwrite import method, that is, after clearing the existing content in the data cells corresponding to the data from 2024-01-01 to 2024-01-31 in the data table, batch import the newly generated 31-day data from 2024-01-01 to 2024-01-31.

[0040] This embodiment realizes an automated and complete process of searching, copying, and importing the target data table under specific conditions based on the query time information, import time information, and import method information, which can quickly complete the batch import operation for different time ranges, and at the same time helps to ensure the accuracy and consistency of the data.

[0041] In one embodiment, in step S401, that is, importing the data to be imported into the target data table according to the import method information to obtain an updated target data table, includes: S4011. When it is confirmed that the import method information is the skip import method, determine the data cells in the target data table that are associated with the import time information and are in a blank state as the data cells to be updated, and import the data to be imported into the data cells to be updated to obtain an updated target data table. S4012. When it is confirmed that the import method information is the append import method, create data cells to be updated that are associated with the import time information in the target data table, and import the data to be imported into the data cells to be updated to obtain an updated target data table. S4013. When it is confirmed that the import method information is the overwrite import method, clear the data cells to be updated in the target data table that are associated with the import time information, and import the data to be imported into the data cells to be updated to obtain an updated target data table.

[0042] Understandably, in one embodiment, the reference data is the data of January 1, 2024, the data to be imported is the data of 31 days from January 1, 2024 to January 31, 2024, and there is existing content in the data cells corresponding to January 1, 2024 and January 10, 2024 in the target data table. The target data table consists of multiple data cells associated with time. The data cells in a blank state represent the data cells without existing content, and the data cells to be updated refer to the data cells that need to be written with data. When the import method information is the skip import method, determine the data cells in the target data table except for the data cells corresponding to January 1, 2024 and January 10, 2024 as the data cells to be updated, and import the data to be imported into the data cells to be updated according to the date association relationship to obtain an updated target data table. When the import method information is the append import method, create data cells to be updated corresponding to January 1, 2024 to January 31, 2024 in the target data table, and import the data to be imported into the data cells to be updated to obtain an updated target data table. When the import method information is the overwrite import method, clear the data cells to be updated in the target data table corresponding to January 1, 2024 to January 31, 2024, and import the data to be imported into the data cells to be updated to obtain an updated target data table.

[0043] In this embodiment, during the automated batch import operation, determine which data cells need to be updated or modified according to different import methods, ensuring the accuracy and flexibility of data import and meeting the data import requirements in different scenarios.

[0044] In one embodiment, the data import conditions include at least one set of replacement field information, replacement query information, and replacement import information; in step S30, that is, performing data editing processing on the target data table according to the data import conditions to obtain the data to be imported corresponding to the data import request, including: S303. Search for the data to be replaced corresponding to each replacement import information from the target data table according to each set of the replacement field information and the replacement query information; S304. Obtain the data to be imported according to all the data to be replaced; In step S40, that is, importing the data to be imported into the target data table according to the data import conditions to obtain the updated target data table, including: S402. Search for the data cells to be replaced from the target data table according to all the replacement field information and the replacement import information, and add the data to be imported to the data cells to be replaced to obtain the updated target data table.

[0045] Understandably, when the data import request is a replacement insertion request, after parsing the data import request, the obtained data import conditions include the replacement field information corresponding to the replacement field configuration information, the replacement query information corresponding to the replacement query configuration information, and the replacement import information corresponding to the replacement import configuration information. In the process of performing data editing processing on the target data table according to the data import conditions, first, search for the data to be replaced corresponding to each replacement import information from the target data table according to each set of the replacement field information and the replacement query information. The data to be replaced refers to the data that needs to replace the data cell corresponding to the replacement import information, for example, the data that needs to replace the data cell corresponding to the date 20241231. Each set of the replacement field information, the replacement query information, and the replacement import information corresponds to a set of the data to be replaced. Then, perform an intersection operation on all the data to be replaced to obtain the data to be imported. Finally, perform an intersection operation on all sets of the replacement field information, search for the data cells to be replaced corresponding to the intersection-processed replacement field information from the target data table according to the replacement import information, and add the data to be imported to the data cells to be replaced to obtain the updated target data table.

[0046] In one embodiment, the replacement insertion configuration information includes two sets of replacement operation conditions: for the first set, the replacement field configuration information is "date", the replacement query configuration information is "20240215", and the replacement import configuration information is "20241231"; for the second set, the replacement field configuration information is "cabinet code", the replacement query configuration information is "FC00001", and the replacement import configuration information is "FC00002". The data to be replaced in the first set is the existing data content in the target data table associated with the date "20240215", and the data to be replaced in the second set is the existing data content in the target data table associated with the cabinet code "FC00001". Take the intersection of the data to be replaced in the first set and the second set, and determine the data to be imported as the existing data in the target data table that is simultaneously associated with the date "20240215" and the cabinet code "FC00001". Perform an intersection process on the data cells in the target data table according to the replacement field information and the replacement import information to obtain the data cells to be replaced that correspond to both the date "20241231" and the cabinet code "FC00002". That is, copy the existing data with the cabinet code FC00001 and the date 20240215 as the data to be imported, and use the data cell with the cabinet code FC00002 and the date 20241231 as the data cell to be replaced, and add the data to be imported to the data cell to be replaced to obtain the updated target data table.

[0047] This embodiment realizes an automated and complete process of searching, matching, and replacing and updating the target data table under specific conditions based on the replacement field information, replacement query information, and replacement import information, improving the efficiency of replacement insertion. At the same time, data replacement is achieved by accurately positioning the data to be replaced and the data cells in the target data table, thus ensuring the accuracy and consistency of replacement insertion.

[0048] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0049] In one embodiment, a data import device is provided, and this data import device corresponds one-to-one with the data import method in the above embodiment. As Figure 2 shown, this data import device includes a request acquisition module 10, a request parsing module 20, a data-to-be-imported determination module 30, and a data import module 40. The detailed descriptions of each functional module are as follows: The request acquisition module 10 is used to acquire a data import request and the import request type of the data import request; The request parsing module 20 is used to parse the data import request to obtain data address information and data import conditions corresponding to the import request type; The to-be-imported data determination module 30 is configured to determine a target data table according to the data address information, perform data editing processing on the target data table according to the data import condition, obtain the to-be-imported data corresponding to the data import request, and send the to-be-imported data to the client; The data import module 40 is configured to, when receiving the import confirmation information corresponding to the to-be-imported data sent by the client, import the to-be-imported data into the target data table according to the data import condition, and obtain the updated target data table.

[0050] In one embodiment, the request acquisition module 10 includes: The batch import type determination unit is configured to receive import type selection information, and when it is confirmed according to the import type selection information that the import request type is the batch import type, send a batch import configuration request corresponding to the batch import type to the client; The first batch import request generation unit is configured to, when receiving the data address configuration information, query time configuration information, import time configuration information, and import method configuration information corresponding to the batch import configuration request sent by the client, generate the batch import request according to the data address configuration information, query time configuration information, import time configuration information, and import method configuration information.

[0051] In one embodiment, the request acquisition module 10 further includes: The replacement and insertion type determination unit is configured to receive import type selection information, and when it is confirmed according to the import type selection information that the import request type is the replacement and insertion type, send a replacement and insertion configuration request corresponding to the replacement and insertion type to the client; The replacement and insertion request generation unit is configured to, when receiving the data address configuration information, and at least one set of replacement field configuration information, replacement query configuration information, and replacement import configuration information corresponding to the replacement and insertion configuration request sent by the client, generate the replacement and insertion request according to the data address configuration information, and at least one set of replacement field configuration information, replacement query configuration information, and replacement import configuration information.

[0052] In one embodiment, the request acquisition module 10 further includes: A second batch import request generation unit, configured to generate the batch import request according to the data address configuration information, query time configuration information, import time configuration information, import method configuration information, and additional condition configuration information corresponding to the batch import configuration request sent by the client, where the additional condition configuration information includes import screening configuration information and / or import random number configuration information.

[0053] In one embodiment, the to-be-imported data determination module 30 includes: A reference data search unit, configured to search for reference data from the target data table according to the query time information; A first to-be-imported data determination unit, configured to perform data copying processing on the reference data according to the import time information to obtain the to-be-imported data; The data import module 40 includes: A data batch import unit, configured to import the to-be-imported data into the target data table according to the import method information to obtain an updated target data table.

[0054] In one embodiment, the data import module 40 further includes: A skip import processing unit, configured to, when it is confirmed that the import method information is the skip import method, determine the data cells in the target data table that are associated with the import time information and are in a blank state as the to-be-updated data cells, and import the to-be-imported data into the to-be-updated data cells to obtain an updated target data table; An append import processing unit, configured to, when it is confirmed that the import method information is the append import method, create to-be-updated data cells associated with the import time information in the target data table, and import the to-be-imported data into the to-be-updated data cells to obtain an updated target data table; An overwrite import processing unit, configured to, when it is confirmed that the import method information is the overwrite import method, clear the to-be-updated data cells in the target data table that are associated with the import time information, and import the to-be-imported data into the to-be-updated data cells to obtain an updated target data table.

[0055] In one embodiment, the to-be-imported data determination module 30 includes: A to-be-replaced data determination unit, configured to search for to-be-replaced data corresponding to each group of the replacement field information and replacement query information from the target data table; A second to-be-imported data determination unit, configured to obtain the to-be-imported data according to all the to-be-replaced data; The data import module 40 further includes: A data replacement import unit, configured to find data cells to be replaced from the target data table according to all the replacement field information and replacement import information, and add the data to be imported to the data cells to be replaced, so as to obtain an updated target data table.

[0056] For the specific limitations on the data import device, reference may be made to the limitations on the data import method in the foregoing text, which will not be elaborated here. Each module in the foregoing data import device may be implemented in whole or in part by software, hardware, and their combination. The foregoing modules may be embedded in the processor in the computer device in hardware form or independent of the processor, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the foregoing modules.

[0057] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as Figure 3 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer-readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The database of the computer device is used to store the data involved in the data import method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a data import method is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0058] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored on the memory and executable on the processor. When the processor executes the computer-readable instructions, the following steps are implemented: Obtain a data import request and the import request type of the data import request; Parse the data import request to obtain data address information and data import conditions corresponding to the import request type; Determine a target data table according to the data address information, perform data editing processing on the target data table according to the data import conditions, obtain data to be imported corresponding to the data import request, and send the data to be imported to the client; When receiving the import confirmation information corresponding to the data to be imported sent by the client, import the data to be imported into the target data table according to the data import condition to obtain an updated target data table.

[0059] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. Computer-readable instructions are stored on the readable storage media. When the computer-readable instructions are executed by one or more processors, the following steps are implemented: Obtain a data import request and the import request type of the data import request; Parse the data import request to obtain data address information and data import conditions corresponding to the import request type; Determine the target data table according to the data address information, perform data editing processing on the target data table according to the data import condition to obtain the data to be imported corresponding to the data import request, and send the data to be imported to the client; When receiving the import confirmation information corresponding to the data to be imported sent by the client, import the data to be imported into the target data table according to the data import condition to obtain an updated target data table.

[0060] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or an external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0061] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be assigned to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0062] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A data import method, characterized in that, Including: Obtaining a data import request and the import request type of the data import request; Parsing the data import request to obtain data address information and data import conditions corresponding to the import request type; Determining a target data table according to the data address information, performing data editing processing on the target data table according to the data import conditions to obtain to-be-imported data corresponding to the data import request, and sending the to-be-imported data to the client; When receiving the import confirmation information corresponding to the to-be-imported data sent by the client, importing the to-be-imported data into the target data table according to the data import conditions to obtain an updated target data table.

2. The data import method according to claim 1, wherein The data import request includes a batch import request; Before obtaining the data import request, including: Receiving import type selection information, and when confirming that the import request type is the batch import type according to the import type selection information, sending a batch import configuration request corresponding to the batch import type to the client; When receiving the data address configuration information, query time configuration information, import time configuration information, and import method configuration information corresponding to the batch import configuration request sent by the client, generating the batch import request according to the data address configuration information, query time configuration information, import time configuration information, and import method configuration information.

3. The data import method according to claim 1, wherein The data import request includes a replace and insert request; Before obtaining the data import request, including: Receiving import type selection information, and when confirming that the import request type is the replace and insert type according to the import type selection information, sending a replace and insert configuration request corresponding to the replace and insert type to the client; When receiving the data address configuration information, and at least one set of replacement field configuration information, replacement query configuration information, and replacement import configuration information corresponding to the replace and insert configuration request sent by the client, generating the replace and insert request according to the data address configuration information, and at least one set of replacement field configuration information, replacement query configuration information, and replacement import configuration information.

4. The data import method according to claim 2, characterized in that After sending the batch import configuration request corresponding to the batch import type to the client, including: When receiving the data address configuration information, query time configuration information, import time configuration information, import method configuration information, and additional condition configuration information corresponding to the batch import configuration request sent by the client, generating the batch import request according to the data address configuration information, query time configuration information, import time configuration information, import method configuration information, and additional condition configuration information, where the additional condition configuration information includes import screening configuration information and / or import random number configuration information.

5. The data import method according to claim 2, wherein The data import conditions include query time information, import time information, and import method information; Performing data editing processing on the target data table according to the data import conditions to obtain to-be-imported data corresponding to the data import request, including: Searching for reference data from the target data table according to the query time information; Perform data replication processing on the reference data according to the import time information to obtain the data to be imported; Importing the data to be imported into the target data table according to the data import condition to obtain the updated target data table includes: Import the data to be imported into the target data table according to the import method information to obtain the updated target data table.

6. The data import method according to claim 5, wherein Importing the data to be imported into the target data table according to the import method information to obtain the updated target data table includes: When it is confirmed that the import method information is the skip import method, determine the data cells in the target data table that are associated with the import time information and are in a blank state as the data cells to be updated, and import the data to be imported into the data cells to be updated to obtain the updated target data table; When it is confirmed that the import method information is the append import method, create a data cell to be updated associated with the import time information in the target data table, and import the data to be imported into the data cell to be updated to obtain the updated target data table; When it is confirmed that the import method information is the overwrite import method, clear the data cells to be updated in the target data table that are associated with the import time information, and import the data to be imported into the data cells to be updated to obtain the updated target data table.

7. The data import method according to claim 3, wherein The data import condition includes at least one set of replacement field information, replacement query information, and replacement import information; Performing data editing processing on the target data table according to the data import condition to obtain the data to be imported corresponding to the data import request includes: Find the data to be replaced corresponding to each replacement import information from the target data table according to each set of replacement field information and replacement query information; Obtain the data to be imported according to all the data to be replaced; Importing the data to be imported into the target data table according to the data import condition to obtain the updated target data table includes: Find the data cells to be replaced from the target data table according to all the replacement field information and replacement import information, and add the data to be imported to the data cells to be replaced to obtain the updated target data table.

8. A data import device, characterized in that, Includes: A request acquisition module, configured to acquire a data import request and the import request type of the data import request; A request parsing module, configured to parse the data import request to obtain data address information and a data import condition corresponding to the import request type; A data-to-be-imported determination module, configured to determine a target data table according to the data address information, perform data editing processing on the target data table according to the data import condition to obtain the data to be imported corresponding to the data import request, and send the data to be imported to the client; A data import module, configured to, when receiving the import confirmation information corresponding to the data to be imported sent by the client, import the data to be imported into the target data table according to the data import condition to obtain the updated target data table.

9. A computer device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the computer-readable instructions, it implements the data import method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the data import method according to any one of claims 1 to 7.

Images