A method and system for automated batch file loading into a repository

Abstract

The application discloses a kind of methods and systems for automatically processing file loading into repository, it is related to computer technical field, especially to a kind of methods and systems for automatically processing file loading into repository.The method comprises: obtaining the set of files to be received;For each of the set of files to be received, configure the corresponding file path, generate the set of file paths to be received;By monitoring whether a single file path to be received in the set of file paths to be received exists corresponding file to be received;If a single file path to be received in the set of file paths to be received exists corresponding file to be received, generate the first synchronization instruction;The set of files to be received is synchronized to temporary table;The set of files to be received in temporary table is synchronized to target table.The technical problem of low efficiency and accuracy of file loading into repository in the prior art is solved.The technical goal of efficient and accurate loading of each file to be received is finally achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and system for automating batch file loading into a database. Background Technology

[0002] With the development of computer technology, all industries inevitably use computers for office work and learning. When using computers to handle various problems, it is often necessary to synchronize file data from different components and platforms, and file transfer is an important synchronization method. However, in existing technologies, when transferring large numbers of files, the efficiency is often low due to the large number of files, unpredictable arrival times, or errors that can easily occur when manually loading files into the database. Furthermore, the large number of files and their unpredictable arrival times lead to long manual processing and waiting times, resulting in high costs and significant resource waste. Thirdly, incorrect script execution order can affect file data loading into the database, leading to a high error rate. Therefore, it is essential to research an efficient and accurate method for loading files into the database.

[0003] However, existing technologies suffer from low efficiency and accuracy in loading files into the database. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for automated batch file loading into a database, in order to solve the technical problems of low efficiency and accuracy in file loading into a database in the prior art.

[0005] In view of the above problems, this application provides a method and system for automating batch file loading into a library.

[0006] In a first aspect, this application provides a method for automatically loading batch files into a database. The method is implemented using a system for automatically loading batch files into a database. The method includes: obtaining a set of files to be received; configuring a corresponding file path for each file in the set of files to be received, generating a set of file paths to be received; monitoring the set of files to be received, and determining whether a single file path in the set of file paths corresponds to a file to be received; if all single file paths in the set of file paths correspond to files to be received, generating a first synchronization instruction; synchronizing the set of files to be received to a temporary table according to the first synchronization instruction; and synchronizing the set of files to be received from the temporary table to a target table.

[0007] On the other hand, this application also provides a system for automatically loading batch files into a database, used to execute the method for automatically loading batch files into a database as described in the first aspect, wherein the system includes: a first obtaining unit: the first obtaining unit is used to obtain a set of files to be received; a first generating unit: the first generating unit is used to configure a corresponding file path for each file to be received in the set of files to be received, and generate a set of file paths to be received; a first judging unit: the first judging unit is used to monitor the set of files to be received and judge whether a single file path in the set of file paths to be received has a corresponding file to be received; a second generating unit: the second generating unit is used to generate a first synchronization instruction if a single file path in the set of file paths to be received has a corresponding file to be received; a first execution unit: the first execution unit is used to synchronize the set of files to be received to a temporary table according to the first synchronization instruction; and a second execution unit: the second execution unit is used to synchronize the set of files to be received in the temporary table to a target table.

[0008] Thirdly, embodiments of this application also provide a system for automatically loading batch files into a library, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the first aspect above.

[0009] Fourthly, a computer-readable storage medium, wherein a computer program is stored on the storage medium, and when executed by a processor, the computer program implements the method described in any one of the first aspects.

[0010] Fifthly, an electronic device, comprising a processor and a memory;

[0011] The memory is used for storage;

[0012] The processor is configured to execute the method described in any one of the first aspects above by means of a call.

[0013] A sixth aspect is a computer program product comprising a computer program and / or instructions, characterized in that, when executed by a processor, the computer program and / or instructions implement the steps of the method described in any of the first aspects above.

[0014] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0015] 1. By obtaining a set of files to be received; configuring a corresponding file path for each file in the set to be received, generating a set of file paths to be received; monitoring the set of files to be received, determining whether a single file path in the set of file paths corresponds to a file to be received; if a single file path in the set of file paths corresponds to a file to be received, generating a first synchronization instruction; synchronizing the set of files to be received to a temporary table according to the first synchronization instruction; synchronizing the set of files to be received in the temporary table to a target table. This achieves the technical goal of using configuration tools to automatically monitor each file to be received, thereby determining the arrival time of each file to be received. When all files to be received arrive at their corresponding paths, each file to be received is first synchronized to a temporary table. After confirming that the loading is correct, the configuration tool is further invoked to merge the files in the temporary table into the target table, ultimately achieving the technical goal of efficient and highly accurate loading of each file to be received.

[0016] 2. Through the first marking instruction, the date of a single file to be received that has not been transmitted is intelligently identified, and the corresponding file to be received is marked. This allows for a direct observation of the files in the set of files to be received that have not been transmitted, further enabling a quick search for the reasons for non-transmission and the development of targeted solutions for transmission failures. Ultimately, this achieves the technical effect of quickly locating a single file in the set of files to be received that has failed to be transmitted, and then carrying out targeted supplementary transmissions to avoid duplicate or missed file transmissions.

[0017] 3. By constructing a temporary table and clearing the data in the temporary table before each synchronization, it is ensured that each file transfer is not affected by the previous file transfer, thus avoiding file corruption issues. At the same time, the probability of table locking during updates is reduced, ultimately achieving the technical effect of improving the efficiency of file synchronization to the temporary table.

[0018] 4. By using the merge into method for updating or inserting, the values ​​of all target primary key fields in the target table are synchronized and updated, thereby enabling the complete processing of all individual files in the file set to be received, improving the efficiency of batch file processing and transmission.

[0019] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating an automated batch file loading method according to an embodiment of this application.

[0022] Figure 2 This is a flowchart illustrating the process of marking a non-existent first target file to be received in an automated batch file loading method according to an embodiment of this application.

[0023] Figure 3 This is a flowchart illustrating the synchronization process of exiting the set of files to be received in an automated batch file loading method according to an embodiment of this application.

[0024] Figure 4 This is a schematic diagram illustrating the process of updating the target table in an automated batch file loading method according to an embodiment of this application.

[0025] Figure 5 This is a schematic diagram of the structure of a system for automatically loading batch files into a database, according to an embodiment of this application.

[0026] Figure 6 This is a schematic diagram of the structure of an exemplary electronic device according to an embodiment of this application.

[0027] Explanation of reference numerals in the attached figures:

[0028] The system comprises a first acquisition unit 11, a first generation unit 12, a first judgment unit 13, a second generation unit 14, a first execution unit 15, a second execution unit 16, a bus 300, a receiver 301, a processor 302, a transmitter 303, a memory 304, and a bus interface 305. Detailed Implementation

[0029] This application provides an automated batch file loading and storage method and system, solving the technical problems of low efficiency and accuracy in file loading and storage in existing technologies. It achieves the technical goal of efficiently and accurately loading each file by utilizing a configuration tool to automatically monitor each file to be received, thereby determining the arrival time of each file. When all files arrive at their corresponding paths, they are first synchronized to a temporary table. After confirming that the loading is correct, the configuration tool is further invoked to merge the files in the temporary table into the target table, ultimately achieving the technical goal of efficient and highly accurate loading of each file to be received.

[0030] The technical solutions in the embodiments of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. It should be understood that this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should also be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all of them.

[0031] This application provides a method for automating batch file loading into a database. The method is applied to a system for automating batch file loading into a database. The method includes: obtaining a set of files to be received; configuring a corresponding file path for each file in the set of files to be received, generating a set of file paths to be received; monitoring the set of files to be received and determining whether a single file path in the set of file paths corresponds to a file to be received; if all single file paths in the set of file paths correspond to files to be received, generating a first synchronization instruction; synchronizing the set of files to be received to a temporary table according to the first synchronization instruction; and synchronizing the set of files to be received from the temporary table to a target table.

[0032] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0033] Example 1

[0034] Please see the appendix Figure 1 This application provides a method for automatically loading batch files into a database. The method is applied to a system for automatically loading batch files into a database, and specifically includes the following steps:

[0035] Step S100: Obtain the set of files to be received;

[0036] Specifically, the method for automatically loading batch files into a database is applied to the system for automatically loading batch files into a database. It utilizes various configuration tools to automatically monitor each file to be received, thereby determining the arrival time of each file. When all files arrive at their corresponding paths, they are first synchronized to a temporary table. After confirming successful loading, the configuration tools are invoked to merge the files from the temporary table into the target table, ultimately achieving the technical goal of high-efficiency and high-accuracy loading of the file set. The file set to be received consists of multiple files to be processed. By obtaining this file set, the system effectively identifies all files to be automatically processed using the automated batch file loading system.

[0037] Step S200: For each file to be received in the set of files to be received, configure the corresponding file path to generate a set of file paths to be received;

[0038] Specifically, each file in the set of files to be received is analyzed to configure a corresponding file path for each file. All file paths corresponding to the files to be received constitute the set of file paths to be received. Here, a file path refers to the storage location of a file on an electronic device. For example, "E:\work\Market Data\Market Plan.do" is a file path that refers to a Word file named "Market Plan" stored in the "Market Plan" folder within the "work" folder on drive E. To open this file, one must follow the file path step by step to find it and perform the corresponding operation. The set of file paths to be received refers to the target storage location of all files to be received, i.e., the location after transmission. By obtaining the set of file paths to be received, the target path corresponding to each file to be received is clearly defined, providing direction for the system that loads the automated batch processing files into the database to execute the corresponding batch processing operations.

[0039] Step S300: Monitor the set of files to be received and determine whether a single file path in the set of file paths to be received has a corresponding file to be received;

[0040] Specifically, by comparing the set of files to be received with the set of file paths to be received, it is determined whether a corresponding file from the set of file paths exists, thereby achieving the goal of monitoring the transmission of the set of files to be received. When a corresponding file exists for a single file path in the set of file paths to be received, it indicates that the corresponding file has been successfully transmitted; when no corresponding file exists for a single file path in the set of file paths to be received, it indicates that the corresponding file has not been successfully transmitted. This failure to transmit may be due to the file being in the process of transmission or a transmission failure. By comparing whether each individual file path has a corresponding file, the system effectively monitors each file in the set of files to be received, thereby controlling the arrival time of file transmissions.

[0041] Step S400: If each individual file path in the set of file paths to be received has a corresponding file to be received, generate a first synchronization instruction;

[0042] Step S500: Synchronize the set of files to be received to a temporary table according to the first synchronization instruction;

[0043] Specifically, when the judgment result shows that each individual file path in the set of file paths to be received has a corresponding file to be received, it indicates that all files in the set of file paths to be received have been successfully transmitted. At this time, the system for automatically batch-processing files and loading them into the database receives a signal indicating that the transmission is complete, and then issues the first synchronization instruction. The first synchronization instruction is used to synchronize all transmitted files to a temporary table. The temporary table has the same structure as the target table and is used to completely store the set of file paths to be received, but the temporary table does not have a primary key. By using the first synchronization instruction, the set of file paths to be received is synchronized to the temporary table, thus avoiding unexpected errors and resource waste that would occur if directly loaded into the target table.

[0044] Step S600: Synchronize the set of files to be received in the temporary table to the target table.

[0045] Specifically, by analyzing the set of files to be received in the temporary table and confirming that all files are being transmitted normally, the set of files to be received in the temporary table is further synchronized to the target table. The target table refers to the target storage location of all individual files to be received in the set of files to be received. By using a configuration tool to automatically monitor each file to be received, the arrival time of each file is determined. When all files to be received arrive at their corresponding paths, they are first synchronized to a temporary table. After confirming that the loading is correct, the configuration tool is further invoked to merge the files in the temporary table into the target table, ultimately achieving the technical goal of efficient and highly accurate loading of each file to be received.

[0046] Further details are attached. Figure 2 As shown, step S300 in this embodiment further includes:

[0047] Step S310: Obtain the configuration parameter information of the set of files to be received, wherein the configuration parameter information includes different file dates;

[0048] Step S320: Call the Java package to transmit the configuration parameter information and determine whether the files with different file dates have arrived;

[0049] Step S330: If all files from different file dates have arrived, obtain the set of file paths of the file set to be received based on the configuration parameter information;

[0050] Step S340: Perform a loop check on the set of file paths to determine whether a single file path in the set of file paths corresponds to a file to be received;

[0051] Step S350: If a single file path in the set of paths containing the file does not have a corresponding file to be received, generate a first marker instruction;

[0052] Step S360: Mark the non-existent first target file to be received according to the first marking instruction.

[0053] Specifically, by invoking a Java package, the transmission process of the set of files to be received is monitored. Simultaneously, relevant configuration parameters for each individual file in the set are extracted and recorded, including the filename, date, time, and target path of each file. The Java package is a Java class library, composed of a set of classes supporting program development; a compiler or development environment is based on one class library. Furthermore, by invoking the Java package to transmit the configuration parameter information for each individual file in the set of files to be received, it is determined whether individual files from different dates and times have been successfully transmitted and reached their target paths. If files from different dates have all reached their corresponding target paths, the automated batch processing system for loading files into the library can obtain the set of file paths for each individual file in the set of files to be received based on the configuration parameter information.

[0054] Furthermore, the system iteratively checks the set of file paths to determine if a corresponding file exists for a single path in the set of file paths to be received. When no corresponding file exists for a single path in the set of file paths to be received, the system automatically generates a first marking instruction. This first marking instruction is used to mark a non-existent first target file to be received. The first target file to be received refers to a single file in the set of files to be received that has not yet been transmitted.

[0055] Through the first marking instruction, the date of a single file to be received that has not been transmitted is intelligently identified, and the corresponding file to be received is marked. This allows for a direct observation of the files in the set of files to be received that have not been transmitted, and further, the reasons for the failure to transmit can be quickly found. A corresponding solution for transmission failure can be developed, and the technical effect of quickly locating a single file in the set of files to be received that has failed to be transmitted can be supplemented for transmission, thus avoiding duplicate or missed file transmissions.

[0056] Furthermore, step S340 in this embodiment of the application also includes:

[0057] Step S341: If a single file path in the set of paths containing the file contains a corresponding file to be received, generate a second marker instruction;

[0058] Step S342: Mark the existing second target file to be received according to the second marking instruction;

[0059] Step S343: Perform loop detection on the set of file paths until the detection of the files to be received for each file path in the set of file paths is completed.

[0060] Specifically, it is determined whether a corresponding file exists in a single path of a file to be received within the set of file paths. When a corresponding file exists in a single path of a file to be received within the set of file paths, the system for loading the automated batch file into the database automatically generates the second marking instruction. This second marking instruction is used to mark the existing second target file to be received. Further, the set of file paths is iteratively checked until the detection of files to be received for each path in the set of file paths is completed. Based on the configuration parameters of each file to be received, all individual files in the set of files to be received are intelligently marked accordingly. Successfully transmitted files are marked with a second mark, for example, a √; failed transmission files are marked with a first mark, for example, an ×. By marking each individual file in the set of files to be received accordingly, the technical effect of intuitively observing the transmission status of each individual file to be received is achieved.

[0061] Further details are attached. Figure 3 As shown, step S500 in this embodiment further includes:

[0062] Step S510: Create the fld configuration file;

[0063] Step S520: Based on the fld configuration file and splldr method, synchronize the set of files to be received to the temporary table to generate a first synchronization result;

[0064] Step S530: Determine whether the first synchronization result was successfully synchronized;

[0065] Step S540: If the first synchronization result is successful, exit the synchronization process of the set of files to be received.

[0066] Specifically, firstly, an FLD configuration file is created in a specified directory. Then, the splldr method is used to load each individual file from the set of files to be received into the database. The FLD configuration file is used to push floating-point data onto the coprocessor's stack. Its instruction format is FLD STReg / MemReal. When pushing memory unit contents onto the stack, the system automatically determines the precision of the transmitted data. The SPL (Secondary program loader) is the code executed in the first stage of U-Boot, mainly responsible for moving the code from the second stage of U-Boot to system memory (System RAM, also called off-chip memory) for execution. The splldr method loads each individual file from the set of files to be received into the database, generating the first synchronization result. Key parameters include database information, control files, and log files. The database information describes the libraries to be loaded; the control file mainly includes the file path, temporary table name, .dat file field separators, and FLD configuration file information; the log file mainly stores log information from the execution of this script.

[0067] Furthermore, it is determined whether the splldr method of data entry, i.e., the first synchronization result, was successfully synchronized. If the first synchronization result is successful, the synchronization process for the set of files to be received is exited. Through the configuration file, the status of each individual file to be received synchronized to the temporary table is accurately monitored, thus avoiding unexpected errors.

[0068] Furthermore, step S520 in this embodiment of the application also includes:

[0069] Step S521: Determine whether the temporary table has been formatted and cleared;

[0070] Step S522: If the temporary table does not exist, format and clear it, and synchronize the set of files to be received to the temporary table.

[0071] Specifically, when synchronizing the set of files to be received to the temporary table via the fld configuration file and splldr method, the system that automatically loads the batch files into the database first checks the temporary table to determine whether it has been formatted and cleared. If the check shows that the temporary table has not been formatted and cleared, it is formatted and cleared first, and then the set of files to be received is synchronized to the temporary table. By clearing the temporary table data before each synchronization, it ensures that each file transfer is not affected by the previous file transfer, avoiding file corruption issues, and reducing the probability of table locking during updates, ultimately achieving the technical effect of improving the efficiency of synchronization to the temporary table.

[0072] Further details are attached. Figure 4 As shown, step S600 in this embodiment further includes:

[0073] Step S610: Based on the merge into method, determine whether the value of the temporary primary key field in the temporary table matches the value of the target primary key field in the target table;

[0074] Step S611: If the temporary primary key field value in the temporary table does not match the target primary key field value in the target table, insert the data of the temporary primary key field value in the temporary table into the target table.

[0075] Step S620: If the value of the temporary primary key field in the temporary table matches the value of the target primary key field in the target table, update the data in the target table based on the value of the temporary primary key field in the temporary table.

[0076] Specifically, the process first checks whether the values ​​of each temporary primary key field in the temporary table match the values ​​of each target primary key field in the target table. If the target primary key field values ​​in the target table and the temporary primary key field values ​​in the temporary table are the same, then the temporary primary key field values ​​in the temporary table are updated to the corresponding target primary key field values ​​in the target table using the `merge into` method. If the target primary key field values ​​in the target table and the temporary primary key field values ​​in the temporary table are different, then the temporary primary key field values ​​in the temporary table are inserted into the corresponding target primary key field values ​​in the target table using the `merge into` method. The `merge into` method combines the `insert` and `update` operations, and its execution efficiency is higher than executing `insert` and `update` statements separately. Simply put, by checking if there is data in the table that meets the `on()` condition, if so, the data is updated; otherwise, data is inserted. For example, given a table T with two fields a and b, and the goal is to perform an `insert / update` operation, if the condition is met, the value of b in T is updated; otherwise, a record is inserted into T.

[0077] By using the MERGE INTO method for updates or inserts, the system synchronizes and updates all primary key field values ​​in the target table, thereby enabling complete processing of all individual files in the file set to be received, and improving the efficiency of file batch processing and transmission.

[0078] In summary, the automated batch file loading method provided in this application has the following technical effects:

[0079] 1. By obtaining a set of files to be received; configuring a corresponding file path for each file in the set to be received, generating a set of file paths to be received; monitoring the set of files to be received, determining whether a single file path in the set of file paths corresponds to a file to be received; if a single file path in the set of file paths corresponds to a file to be received, generating a first synchronization instruction; synchronizing the set of files to be received to a temporary table according to the first synchronization instruction; synchronizing the set of files to be received in the temporary table to a target table. This achieves the technical goal of using configuration tools to automatically monitor each file to be received, thereby determining the arrival time of each file to be received. When all files to be received arrive at their corresponding paths, each file to be received is first synchronized to a temporary table. After confirming that the loading is correct, the configuration tool is further invoked to merge the files in the temporary table into the target table, ultimately achieving the technical goal of efficient and highly accurate loading of each file to be received.

[0080] 2. Through the first marking instruction, the date of a single file to be received that has not been transmitted is intelligently identified, and the corresponding file to be received is marked. This allows for a direct observation of the files in the set of files to be received that have not been transmitted, further enabling a quick search for the reasons for non-transmission and the development of targeted solutions for transmission failures. Ultimately, this achieves the technical effect of quickly locating a single file in the set of files to be received that has failed to be transmitted, and then carrying out targeted supplementary transmissions to avoid duplicate or missed file transmissions.

[0081] 3. By constructing a temporary table and clearing the data in the temporary table before each synchronization, it is ensured that each file transfer is not affected by the previous file transfer, thus avoiding file corruption issues. At the same time, the probability of table locking during updates is reduced, ultimately achieving the technical effect of improving the efficiency of file synchronization to the temporary table.

[0082] 4. By using the merge into method for updating or inserting, the values ​​of all target primary key fields in the target table are synchronized and updated, thereby enabling the complete processing of all individual files in the file set to be received, improving the efficiency of batch file processing and transmission.

[0083] Example 2

[0084] Based on the aforementioned method for automating batch file loading into a database, and using the same inventive concept, this invention also provides a system for automating batch file loading into a database. Please refer to the appendix. Figure 5 The system includes:

[0085] The first obtaining unit 11 is used to obtain a set of files to be received;

[0086] The first generation unit 12 is used to configure a corresponding file path for each file to be received in the set of files to be received, and generate a set of file paths to be received.

[0087] The first judgment unit 13 is used to monitor the set of files to be received and to determine whether a single file path in the set of file paths to be received has a corresponding file to be received.

[0088] The second generation unit 14 is used to generate a first synchronization instruction if each of the individual file paths to be received in the set of file paths to be received has a corresponding file to be received.

[0089] The first execution unit 15 is configured to synchronize the set of files to be received to a temporary table according to the first synchronization instruction.

[0090] The second execution unit 16 is used to synchronize the set of files to be received in the temporary table to the target table.

[0091] Furthermore, the system also includes:

[0092] The second obtaining unit is used to obtain the configuration parameter information of the set of files to be received, wherein the configuration parameter information includes different file dates;

[0093] The second judgment unit is used to call the Java package to transmit the configuration parameter information and determine whether the files with different file dates have arrived.

[0094] The third obtaining unit is used to obtain the set of file paths of the file set to be received based on the configuration parameter information if files of different file dates have all arrived.

[0095] The third judgment unit is used to perform cyclic detection on the set of paths where the files are located, and to determine whether a single file path to be received in the set of paths where the files are located has a corresponding file to be received.

[0096] The third generation unit is used to generate a first marker instruction if a single file path in the set of paths where the file is located does not have a corresponding file to be received.

[0097] The third execution unit is used to mark a non-existent first target file to be received according to the first marking instruction.

[0098] Furthermore, the system also includes:

[0099] The fourth generation unit is used to generate a second marking instruction if a corresponding file exists in a single file path in the set of paths where the file is located.

[0100] The fourth execution unit is configured to mark the existing second target file to be received according to the second marking instruction;

[0101] The fifth execution unit is used to perform cyclic detection on the set of file paths until the detection of the files to be received in each file path of the set of file paths is completed.

[0102] Furthermore, the system also includes:

[0103] The first creation unit is used to create the fld configuration file;

[0104] The fifth generation unit is used to synchronize the set of files to be received to the temporary table based on the fld configuration file and splldr method, and generate a first synchronization result;

[0105] The fourth judgment unit is used to determine whether the first synchronization result is successfully synchronized;

[0106] The sixth execution unit is used to exit the synchronization process of the set of files to be received if the first synchronization result is successful.

[0107] Furthermore, the system also includes:

[0108] The fifth judgment unit is used to determine whether the temporary table has been formatted and cleared.

[0109] The seventh execution unit is configured to format and clear the temporary table if the temporary table does not exist, and to synchronize the set of files to be received to the temporary table.

[0110] Furthermore, the system also includes:

[0111] The fifth judgment unit is used to determine whether the value of the temporary primary key field in the temporary table matches the value of the target primary key field in the target table based on the merge into method.

[0112] The eighth execution unit is configured to update the target table based on the temporary primary key field value in the temporary table if the temporary primary key field value in the temporary table matches the target primary key field value in the target table.

[0113] Furthermore, the system also includes:

[0114] The ninth execution unit is configured to insert the data of the temporary primary key field in the temporary table into the target table if the value of the temporary primary key field in the temporary table does not match the value of the target primary key field in the target table.

[0115] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Figure 1 The method and specific example for automatically loading batch files into a database in Embodiment 1 are also applicable to the system for automatically loading batch files into a database in this embodiment. Through the foregoing detailed description of the method for automatically loading batch files into a database, those skilled in the art can clearly understand the system for automatically loading batch files into a database in this embodiment. Therefore, for the sake of brevity, it will not be described in detail here. As for the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant details can be found in the method section.

[0116] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0117] Exemplary electronic devices

[0118] The following is for reference. Figure 6 The electronic device described in the embodiments of this application is used to illustrate this application.

[0119] Figure 6 The figure shows a schematic diagram of the structure of an electronic device according to an embodiment of the present application.

[0120] Based on the inventive concept of the automated batch file loading method described in the foregoing embodiments, the present invention also provides an automated batch file loading system, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the methods described above for automated batch file loading.

[0121] Among them, Figure 6In this document, a bus architecture (represented by bus 300) is used. Bus 300 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 302 and memory represented by memory 304. Bus 300 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 305 provides an interface between bus 300 and receiver 301 and transmitter 303. Receiver 301 and transmitter 303 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium.

[0122] The processor 302 is responsible for managing the bus 300 and general processing, while the memory 304 can be used to store the data used by the processor 302 when performing operations.

[0123] This application provides a method for automating batch file loading into a database. The method is applied to a system for automating batch file loading into a database. The method includes: obtaining a set of files to be received; configuring a corresponding file path for each file in the set to be received, generating a set of file paths to be received; monitoring the set of files to be received, determining whether a single file path in the set of file paths corresponds to a file to be received; if all single file paths in the set of file paths correspond to files to be received, generating a first synchronization instruction; synchronizing the set of files to be received to a temporary table according to the first synchronization instruction; and synchronizing the set of files to be received in the temporary table to a target table. This method solves the technical problems of low efficiency and accuracy in file loading into a database in existing technologies. It achieves the technical goal of automating the monitoring of each file to be received using a configuration tool to determine the arrival time of each file. When all files to be received arrive at their corresponding paths, they are first synchronized to a temporary table. After confirming that the loading is correct, the configuration tool is further invoked to merge the files in the temporary table into the target table, ultimately achieving efficient and highly accurate loading of each file to be received.

[0124] This application also provides a computer-readable storage medium, wherein a computer program is stored on the storage medium, and the computer program, when executed by a processor, implements the method described in any one of the above embodiments.

[0125] This application also provides an electronic device, which includes a processor and a memory;

[0126] The memory is used for storage;

[0127] The processor is configured to execute the method described in any one of the above embodiments by means of a call.

[0128] This application also provides a computer program product, including a computer program and / or instructions, characterized in that, when the computer program and / or instructions are executed by a processor, they implement the steps of the method described in any one of the above embodiments.

[0129] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely software embodiment, a completely hardware embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application is in the form of a computer program product that can be implemented on one or more computer-usable storage media containing computer-usable program code. Such computer-usable storage media include, but are not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), disk storage, compact disc read-only memory (CD-ROM), and optical storage.

[0130] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0131] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0132] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0133] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for automatically loading batch files into a database, characterized in that, The method includes: Obtain the set of files to be received; For each file in the set of files to be received, configure the corresponding file path to generate a set of file paths to be received; Monitoring the set of files to be received includes: obtaining configuration parameter information of the set of files to be received, the configuration parameter information including different file dates; The Java package is invoked to transmit the configuration parameter information and to determine whether files with different dates have arrived. If all files from different file dates arrive, the set of file paths of the file set to be received is obtained based on the configuration parameter information; The set of paths containing the files is checked repeatedly to determine whether a single path to be received in the set of paths contains a corresponding file to be received. If a single file path in the set of file paths does not have a corresponding file to be received, a first marker instruction is generated. According to the first marking instruction, the first target file to be received that does not exist is marked; it is determined whether a single file path in the set of file paths to be received has a corresponding file to be received; If a single file path in the set of file paths to be received has a corresponding file to be received, a first synchronization instruction is generated. According to the first synchronization instruction, the set of files to be received is synchronized to a temporary table, including: creating a fld configuration file; Based on the fld configuration file and splldr method, the set of files to be received is synchronized to the temporary table to generate the first synchronization result; Determine whether the first synchronization result was successfully synchronized; If the first synchronization result is successful, the synchronization process of the set of files to be received is exited; Synchronize the set of files to be received in the temporary table to the target table.

2. The method as described in claim 1, characterized in that, The step of determining whether a single file path in the set of file paths contains a corresponding file to be received includes: If a single file path in the set of file paths contains a corresponding file to be received, a second marker instruction is generated. According to the second marking instruction, the existing second target file to be received is marked; The set of paths containing the files is checked iteratively until the files to be received in each path of the set of paths containing the files are checked.

3. The method as described in claim 1, characterized in that, The step of synchronizing the set of files to be received to the temporary table includes: Determine if the temporary table has been formatted and cleared; If the temporary table does not exist, the temporary table is formatted and cleared, and the set of files to be received is synchronized to the temporary table.

4. The method as described in claim 1, characterized in that, The step of synchronizing the set of files to be received in the temporary table to the target table includes: Based on the merge into method, determine whether the value of the temporary primary key field in the temporary table matches the value of the target primary key field in the target table; If the value of the temporary primary key field in the temporary table matches the value of the target primary key field in the target table, the target table is updated based on the value of the temporary primary key field in the temporary table.

5. The method as described in claim 4, characterized in that, The step of determining whether the temporary primary key field value in the temporary table matches the target primary key field value in the target table includes: If the temporary primary key field value in the temporary table does not match the target primary key field value in the target table, the data of the temporary primary key field value in the temporary table is inserted into the target table.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

7. An electronic device, characterized in that, Including processor and memory; The memory is used for storage; The processor is configured to execute the method of any one of claims 1 to 5 by means of a call.

8. A computer program product, comprising a computer program and / or instructions, characterized in that, When the computer program and / or instructions are executed by a processor, they implement the steps of the method described in any one of claims 1 to 5.

Images