Method and device for importing Excel data into database and medium

Through the provisioned file listener and asynchronous verification mechanism, the problem of untimely update of progress bars when importing Excel data to the database and the lag in data status feedback is solved, and an efficient and smooth data import process is realized, improving user experience and system Availability.

CN119166704APending Publication Date: 2024-12-20INSPUR ZHUOSHU BIG DATA IND DEV CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411279467.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-12-20

AI Technical Summary

Technical Problem

When importing Excel data into the database, the progress bar updates are not timely, the data status feedback is lagging, and frequent parsing errors and verification failures occur, resulting in poor user experience and business continuity.

Method used

The Excel file information is captured through the preset file listener, format verification and integrity verification are performed, and batch processing and asynchronous verification mechanisms are adopted to ensure the transparency and immediacy of the data import process, reduce resource usage, and improve efficiency.

Benefits of technology

It improves the efficiency and immediacy of data processing, ensures that the data complies with format standards, reduces the peak use of memory and processor resources, enhances system availability and user satisfaction, and provides accurate import process query information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119166704A_ABST
    Figure CN119166704A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a method and equipment for importing Excel data into a database and a medium. Belongs to the technical field of data processing, and solves the problems that in the prior art, when Excel data is imported into a database, the requirements of a user for transparent and instant services are difficult to meet, and even business continuity is affected. The method comprises the following steps: capturing Excel file information through a preset file monitor; performing format verification and integrity verification on the captured Excel file information through a preset file monitor; under the condition that the verification is passed, performing batch processing on the Excel file information to obtain a plurality of batches of data; wherein the multiple batches of data respectively correspond to unique identifiers; asynchronous verification is carried out on the multiple batches of data, and the data passing the asynchronous verification are imported into a database; and performing import data analysis on the import process, and storing the generated analysis data, so that the front end queries the import process through the unique identifier and the analysis data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and medium for importing Excel data into a database. Background Art

[0002] In today's wave of digital transformation, data has become the core driving force for business operations and decision-making. Many data management platforms have emerged to help companies achieve automation and intelligence of business processes through efficient data processing and analysis capabilities. Among them, batch importing data from Excel files into databases and then conveniently querying and managing them on the web has become a standard requirement in many business scenarios.

[0003] As the business scale expands, the amount of data that needs to be imported is becoming increasingly large, which puts higher requirements on import efficiency. Traditional methods are often difficult to meet the needs of rapid response due to slow processing speed and high resource usage, resulting in poor user experience. Secondly, existing technologies often have problems such as untimely progress bar updates, delayed data status feedback, and frequent parsing errors and verification failures, making it difficult to meet users' needs for transparent and instant services, and even affecting business continuity. Summary of the invention

[0004] The embodiments of the present application provide a method, device and medium for importing Excel data into a database, which are used to solve the following technical problems: the existing technology often has problems such as untimely progress bar updates, delayed data status feedback, and frequent parsing errors and verification failures, which makes it difficult to meet users' needs for transparent and instant services and even affects business continuity.

[0005] The present application embodiment adopts the following technical solutions:

[0006] The embodiment of the present application provides a method for importing Excel data into a database, including capturing Excel file information through a preset file listener; performing format check and integrity check on the captured Excel file information through the preset file listener; in the case of passing the check, batch processing the Excel file information to obtain multiple batches of data; wherein the multiple batches of data respectively correspond to unique identifiers; performing asynchronous check on the multiple batches of data respectively, and importing the data that pass the asynchronous check into the database; analyzing the imported data during the import process, and storing the generated analysis data, so that the front end can query the import process through the unique identifier and the analysis data.

[0007] The embodiment of the present application improves the efficiency and immediacy of data processing by automatically detecting and capturing changes in Excel files in the file system without manual intervention. The captured Excel file information is format checked and integrity checked to ensure that the imported data complies with the predetermined format and standards, avoiding subsequent processing problems caused by incorrect or missing data formats. The Excel file information is processed in batches to reduce the peak usage of memory and processor resources, making the data import process smoother. Asynchronous verification allows the verification of multiple data at the same time without waiting for one data to be completed before starting the next, thereby improving overall efficiency. Users obtain the import process through query operations and obtain more accurate and useful information, which enhances the system's availability and user satisfaction.

[0008] In one implementation of the present application, before capturing Excel file information through a preset file listener, the method also includes: defining a corresponding entity class according to the fields in the Excel template to be imported; and defining an imported failed data entity class; wherein the failed data entity class is also provided with a failure reason field; the failure reason field is set in the first column or the last column to record the reason for the data import failure.

[0009] In one implementation of the present application, a format check and an integrity check are performed on the captured Excel file information, specifically including: performing a format check on the captured Excel file information to determine whether the Excel file information satisfies the xls format or the xlsx format; and performing a data check on the captured Excel file information to determine whether the data is empty; and performing a header and template consistency check on the captured Excel file information; when the captured Excel file information satisfies the xls format or the xlsx format, the data is not empty, and the header is consistent with the template, it is determined that the check has passed.

[0010] In one implementation of the present application, when the verification is passed, the Excel file information is processed in batches to obtain multiple batches of data, specifically including: defining a random code UUID as the corresponding unique identifier when importing data; determining the total number of data corresponding to the Excel file information, and determining the preset single batch data volume; based on the ratio between the total number of data and the preset single batch data volume, determining the number of imports.

[0011] In one implementation of the present application, asynchronous verification is performed on multiple batches of data separately, specifically including: based on an asynchronous mechanism, asynchronous verification is performed on multiple data in the current batch at the same time; wherein the asynchronous verification includes at least one of a check on whether the required fields of each data are empty, a check on the field format of each data, and a check on the length of each data.

[0012] In one implementation of the present application, after asynchronous verification is performed on multiple batches of data respectively, the method also includes: storing the data that passed the asynchronous verification into a preset pass array; and storing the data that failed the asynchronous verification into a preset fail array; determining the cause of failure based on the data in the preset fail array, and writing the cause of failure into a preset variable character sequence; and writing the cause of failure into a failure cause field corresponding to the failed data entity class.

[0013] In one implementation of the present application, an import data analysis is performed on the import process, and the generated import data analysis results are stored, specifically including: in the process of importing data that has passed asynchronous verification into the database, determining the data import percentage, the number of successfully imported data, and the number of failed imported data; and during the last data import, determining whether there is failed import data; if there is failed import data, generating a failed data address through HSSFWorkbook, MockMultipartFile and HttpPost, and storing the unique identification random code as the key.

[0014] In one implementation of the present application, the method further includes: in the case of failed import data, feeding back the failed import data to the user; receiving corrected data sent by the user; and re-importing the corrected data into the database.

[0015] An embodiment of the present application provides a device for importing Excel data into a database, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: capture Excel file information through a preset file listener; perform format verification and integrity verification on the captured Excel file information through the preset file listener; if the verification passes, perform batch processing on the Excel file information to obtain multiple batches of data; wherein the multiple batches of data respectively correspond to unique identifiers; perform asynchronous verification on the multiple batches of data respectively, and import the data that pass the asynchronous verification into the database; perform imported data analysis on the import process, and store the generated analysis data, so that the front end can query the import process through the unique identifier and the analysis data.

[0016] A non-volatile computer storage medium provided in an embodiment of the present application stores computer executable instructions, wherein the computer executable instructions are configured to: capture Excel file information through a preset file listener; perform format check and integrity check on the captured Excel file information through the preset file listener; if the check passes, process the Excel file information in batches to obtain multiple batches of data; wherein the multiple batches of data respectively correspond to unique identifiers; perform asynchronous check on the multiple batches of data respectively, and import the data that pass the asynchronous check into a database; perform import data analysis on the import process, and store the generated analysis data, so that the front end can query the import process through the unique identifier and the analysis data.

[0017] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: The embodiments of the present application improve the efficiency and immediacy of data processing by automatically detecting and capturing changes in Excel files in the file system without manual intervention. The captured Excel file information is format checked and integrity checked to ensure that the imported data meets the predetermined format and standards, avoiding subsequent processing problems caused by incorrect or missing data formats. The Excel file information is processed in batches to reduce the peak usage of memory and processor resources, making the data import process smoother. Asynchronous verification allows the verification of multiple data at the same time without waiting for one data to be completed before starting the next, thereby improving overall efficiency. Users obtain the import process through query operations, obtain more accurate and useful information, and enhance the system's availability and user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0019] Figure 1 A flow chart of a method for importing Excel data into a database provided in an embodiment of the present application;

[0020] Figure 2 A data import flow chart provided for an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the structure of a device for importing Excel data into a database provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The embodiments of the present application provide a method, device and medium for importing Excel data into a database.

[0023] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0024] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0025] Figure 1 A flow chart of a method for importing Excel data into a database provided in an embodiment of the present application, such as Figure 1 As shown, the method for importing Excel data into a database includes the following steps:

[0026] S101. Capture Excel file information through a preset file listener.

[0027] In one embodiment of the present application, a corresponding entity class is defined according to the fields in the Excel template to be imported, and an imported failed data entity class is defined. The failed data entity class is also provided with a failure reason field; the failure reason field is provided in the first column or the last column to record the reason for the data import failure.

[0028] Specifically, the embodiment of the present application first defines an import data entity class ImportData, and defines an import failure data entity class ImportDataFailed.

[0029] Furthermore, the corresponding entity class is defined according to the fields in the Excel template to be imported, that is, different fields are defined according to the specific template, and the order of the entity class fields is ensured to be the same as the order of the fields filled in the template.

[0030] Furthermore, compared to the data entity class ImportData, a failure reason field is added to the first or last column of the import failure data entity class ImportDataFailed to record the failure reason for a certain data import, that is, the ImportDataFailed entity class has one more failure reason field than the ImportData entity class.

[0031] In one embodiment of the present application, the EasyExcel tool is used to read the Excel file stream and write it into the array importDataList.

[0032] Specifically, the embodiment of the present application captures file information through the defined file listener DataListener, writes file data into the array importDataList using the EasyExcel tool class, and finally converts the Excel file into a Java array.

[0033] S102: Perform format check and integrity check on the captured Excel file information through a preset file listener.

[0034] In one embodiment of the present application, the captured Excel file information is format checked to determine whether the Excel file information satisfies the xls format or the xlsx format; the captured Excel file information is checked to see whether the data is empty; and the captured Excel file information is checked for consistency between the header and the template. If the captured Excel file information satisfies the xls format or the xlsx format, the data is not empty, and the header is consistent with the template, it is determined that the check is passed.

[0035] Specifically, the captured Excel file information is format checked to ensure that the captured Excel file is in a valid Excel format (such as xls or xlsx). For example, you can check whether the file extension is xls or xlsx; by reading the first few bytes of the file, you can more accurately determine the actual format of the file, where xls and xlsx files have specific byte sequences as their file signatures; or in Java, you can use libraries such as Apache POI to try to open the file. If the file is not in a valid Excel format, the check fails.

[0036] Specifically, ensure that the data in the Excel file is not empty, that is, it contains at least some valid data rows. For example, when reading an Excel file, you can use the AnalysisEventListener interface of EasyExcel to listen to the reading of each row of data. If at least one row of data is non-empty, the data non-empty check is considered to have passed.

[0037] Specifically, ensure that the header of the Excel file is completely consistent with the predetermined template. When reading the header of the Excel file, compare it with the predetermined template header. This can be achieved by comparing string lists or arrays to ensure that the name and order of each field are the same. If inconsistencies are found, the verification fails.

[0038] Furthermore, in order to avoid the imported data not conforming to the template format, the embodiment of the present application needs to verify the header, etc.

[0039] Furthermore, the file content is format checked through the file listener DataListener defined above, mainly to check whether the file format meets the xls or xlsx format, whether the header is consistent with the template, and whether the data is empty.

[0040] Further, verify the file format: if(!file.getOriginalFilename().contains(".xls")&&!file.getOriginalFilename().contains(".xlsx"))

[0041] Check whether the data is empty: if (importDataList.size()=0)

[0042] Check whether the header is consistent with the template:

[0043] for(int i=0; i <cellNum;i++){

[0044] if(!cellNames[i].equals(listener.getHead().get(i).trim())){

[0045] }

[0046] }

[0047] Among them, cellNum is the number of template headers.

[0048] S103. When the verification is passed, the Excel file information is processed in batches to obtain multiple batches of data.

[0049] In one embodiment of the present application, a random code UUID is defined as a unique identifier corresponding to data import. The total amount of data corresponding to the Excel file information is determined, and the preset single batch data amount is determined. Based on the ratio between the total amount of data and the preset single batch data amount, the number of imports is determined. Among them, multiple batches of data are respectively corresponding to unique identifiers.

[0050] Specifically, the UUID in the embodiment of the present application is a 128-bit number generated by an algorithm to ensure the uniqueness of all elements in the network space. During the data import process, a UUID is assigned to each batch of imported data records to ensure that each batch of data can be uniquely identified even in the case of concurrent import, data replication or system migration.

[0051] Furthermore, before starting the import, it is necessary to first determine the total number of data items contained in the Excel file and determine the preset single batch data volume. The number of imports required can be obtained by dividing the total number of data in the Excel file by the preset single batch data volume and rounding up.

[0052] For example, define an import every 100 records and calculate the number of times the import is required:

[0053] int importNum=(int)Math.ceil(totalDataCount*1.0 / 100);

[0054] Among them, totalDataCount is the total number of data items.

[0055] S104, performing asynchronous verification on multiple batches of data respectively, and importing the data that pass the asynchronous verification into the database.

[0056] In one embodiment of the present application, based on an asynchronous mechanism, asynchronous verification is performed on multiple data in the current batch at the same time, wherein the asynchronous verification includes at least one of a check on whether a required field of each data is empty, a check on the field format of each data, and a check on the length of each data.

[0057] Specifically, based on the asynchronous mechanism, each batch of data is verified piece by piece, including whether the required fields are empty, whether the fields are only allowed to be in numeric format, length restrictions, etc.

[0058] CompletableFuture.runAsync(()->{

[0059] for(int i=0; i <importNum;i++){

[0060] }

[0061] }

[0062] Among them, importNum is the number of times to be imported in batches.

[0063] Through the asynchronous verification mechanism, multiple data in the same batch can be verified at the same time, which can improve the efficiency of batch data verification.

[0064] In one embodiment of the present application, the data that passed the asynchronous verification is stored in a preset passing array; and the data that failed the asynchronous verification is stored in a preset failing array. Based on the data in the preset failing array, the failure cause is determined, and the failure cause is written into a preset variable character sequence. And the failure cause is written into the failure cause field corresponding to the failure data entity class.

[0065] Specifically, the data that passes the verification is stored in the preset pass array, and the data that fails the verification is stored in the preset fail array, until the verification of the last batch of data is completed. Among them, for the data that fails the verification, the failure reason is written into the preset variable character sequence StringBuilder. And the failure reason is written into the failure reason column in the entity class.

[0066] S105: Analyze the import data during the import process, and store the generated analysis data, so that the front end can query the import process through the unique identifier and the analysis data.

[0067] In one embodiment of the present application, during the process of importing data that has passed the asynchronous verification into the database, the data import percentage, the number of successfully imported data, and the number of failed imported data are determined, and during the last data import, it is determined whether there is failed import data. If there is failed import data, the failed data address is generated through HSSFWorkbook, MockMultipartFile, and HttpPost, and the unique identification random code is used as the key to store it together.

[0068] Specifically, after the asynchronous verification is passed, the data is imported into the database in batches, and the import percentage, number of successes, and number of failures are calculated. When the data is imported for the last time, it is determined whether there is any data that has failed to be imported. If there is data that has failed to be imported, the failed data URL is generated through HSSFWorkbook, MockMultipartFile, and HttpPost, and the unique identifier corresponding to the data that has failed to be imported is determined and stored in Redis as a key.

[0069] Furthermore, the embodiment of the present application determines the corresponding unique identifier for each imported file. Through asynchronous operations, the front end can query the data stored in Redis in real time through the unique representation, including the number of successful entries, the number of failed entries, the import progress, the Excel address of the failure reason, etc.

[0070] In one embodiment of the present application, when there is import failure data, the import failure data is fed back to the user, the corrected data sent by the user is received, and the corrected data is re-imported into the database.

[0071] Specifically, in the case of data import failure, the embodiment of the present application will record the result of the import failure and feed back the result to the user. After the user corrects the data that failed to be imported, the corrected data will be imported again.

[0072] Figure 2 A data import flow chart provided in the embodiment of the present application. Figure 2As shown, when an Excel file is imported, the Excel file information is captured through a preset file listener. The captured Excel file information is parsed through the preset file listener, and format verification and integrity verification are performed. If the verification passes, the Excel file information is processed in batches to obtain multiple batches of data, and each batch of data is processed asynchronously to verify multiple data in each batch at the same time. The data that passes the asynchronous verification is stored in a preset pass array, and imported into the database, and the import status is calculated at the same time. The data that fails the asynchronous verification is stored in a preset failure array, the cause of failure is determined, and the cause of failure is written into a preset variable character sequence. The generated import status data and import failure data are stored in Redis, so that the front end can query the data stored in Redis in real time through a unique representation, including the number of successful entries, the number of failed entries, the import progress, the Excel address of the failure cause, etc.

[0073] Figure 3 A schematic diagram of a device structure for importing Excel data into a database provided in an embodiment of the present application. Figure 3 As shown, a device for importing Excel data into a database includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: capture Excel file information through a preset file listener; perform format verification and integrity verification on the captured Excel file information through the preset file listener; if the verification passes, batch process the Excel file information to obtain multiple batches of data; wherein the multiple batches of data respectively correspond to unique identifiers; perform asynchronous verification on the multiple batches of data respectively, and import the data that pass the asynchronous verification into the database; perform import data analysis on the import process, and store the generated analysis data, so that the front end can query the import process through the unique identifier and the analysis data.

[0074] A non-volatile computer storage medium provided in an embodiment of the present application stores computer executable instructions, wherein the computer executable instructions are configured to: capture Excel file information through a preset file listener; perform format check and integrity check on the captured Excel file information through the preset file listener; if the check passes, process the Excel file information in batches to obtain multiple batches of data; wherein the multiple batches of data respectively correspond to unique identifiers; perform asynchronous check on the multiple batches of data respectively, and import the data that pass the asynchronous check into a database; perform import data analysis on the import process, and store the generated analysis data, so that the front end can query the import process through the unique identifier and the analysis data.

[0075] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0076] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0079] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0080] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0081] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0082] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0083] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various changes and variations. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A method for importing Excel data into a database, characterized in that: The method comprises: Capture Excel file information through the preset file listener; Performing format verification and integrity verification on the captured Excel file information through the preset file listener; If the verification is passed, the Excel file information is processed in batches to obtain multiple batches of data; wherein the multiple batches of data are respectively corresponding to unique identifiers; Asynchronously verifying the plurality of batches of data respectively, and importing the data that pass the asynchronous verification into a database; The import data analysis is performed on the import process, and the generated analysis data is stored, so that the front end can query the import process through the unique identifier and the analysis data.

2. A method for importing Excel data into a database according to claim 1, characterized in that: Before capturing the Excel file information by presetting the file listener, the method further includes: Define the corresponding entity class according to the fields in the Excel template to be imported; and Define the entity class of the failed data to be imported; The failed data entity class is further provided with a failure reason field; the failure reason field is provided in the first column or the last column, and is used to record the reason for the failure of data import.

3. A method for importing Excel data into a database according to claim 1, characterized in that: The format check and integrity check of the captured Excel file information specifically includes: Performing format check on the captured Excel file information to determine whether the Excel file information satisfies the xls format or the xlsx format; and Checking whether the captured Excel file information is empty; and Performing a header and template consistency check on the captured Excel file information; If the captured Excel file information satisfies the xls format or the xlsx format, the data is not empty, and the header is consistent with the template, it is determined that the verification has passed.

4. A method for importing Excel data into a database according to claim 1, characterized in that: In the case where the verification is passed, the Excel file information is processed in batches to obtain multiple batches of data, specifically including: Define a random code UUID as the unique identifier corresponding to data import; Determine the total amount of data corresponding to the Excel file information, and determine the preset single batch data amount; The number of imports is determined based on the ratio between the total amount of data and the preset single batch data amount.

5. A method for importing Excel data into a database according to claim 1, characterized in that: The asynchronous verification of the plurality of batches of data respectively comprises: Based on the asynchronous mechanism, multiple data in the current batch are asynchronously verified at the same time; The asynchronous check includes at least one of a check on whether a required field of each piece of data is empty, a check on the field format of each piece of data, and a check on the length of each piece of data.

6. A method for importing Excel data into a database according to claim 2, characterized in that: After the asynchronous verification is performed on the plurality of batches of data respectively, the method further comprises: Store the data that has passed the asynchronous verification into the preset passed array; and Store the data that failed the asynchronous verification into the preset failure array; Determine the failure reason based on the data in the preset failure array, and write the failure reason into the preset variable character sequence; and The failure reason is written into the failure reason field corresponding to the failure data entity class.

7. A method for importing Excel data into a database according to claim 1, characterized in that: The import data analysis is performed on the import process, and the generated import data analysis results are stored, specifically including: During the process of importing the asynchronously verified data into the database, determining a data import percentage, a number of successfully imported data, and a number of failed imported data; and During the last data import, determine whether there is data that failed to be imported; In the case of the import failure data, the failed data address is generated through HSSFWorkbook, MockMultipartFile and HttpPost, and the unique identification random code is stored together as the key.

8. A method for importing Excel data into a database according to claim 7, characterized in that: The method further comprises: In the case where the import failure data exists, feeding back the import failure data to the user; receiving the corrected data sent by the user; The corrected data is re-imported into the database.

9. A device for importing Excel data into a database, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Capture Excel file information through the preset file listener; Performing format verification and integrity verification on the captured Excel file information through the preset file listener; If the verification is passed, the Excel file information is processed in batches to obtain multiple batches of data; wherein the multiple batches of data are respectively corresponding to unique identifiers; Asynchronously verifying the plurality of batches of data respectively, and importing the data that pass the asynchronous verification into a database; The import data analysis is performed on the import process, and the generated analysis data is stored, so that the front end can query the import process through the unique identifier and the analysis data.

10. A non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to: Capture Excel file information through the preset file listener; Performing format verification and integrity verification on the captured Excel file information through the preset file listener; If the verification is passed, the Excel file information is processed in batches to obtain multiple batches of data; wherein, Each of the plurality of batches of data corresponds to a unique identifier; Asynchronously verifying the plurality of batches of data respectively, and importing the data that pass the asynchronous verification into a database; The import data analysis is performed on the import process, and the generated analysis data is stored, so that the front end can query the import process through the unique identifier and the analysis data.

Citation Information

Patent Citations

  • Import method of bulk data and system thereof

    CN106302581A

  • Asynchronous file importing method and device, computer equipment and storage medium

    CN111400246A

  • Data import method and device based on twice writing, equipment and medium

    CN114595285A

  • Test method, system and device based on excel file loading

    WO2023093198A1