Table file processing method and device, equipment and storage medium

Abstract

According to the table file processing method and device, the equipment and the storage medium, the multiple pieces of first data corresponding to the to-be-processed excel file are obtained, the tokens are taken out from the preset token bucket and distributed to the first data which are not processed, the token bucket stores the first preset number of tokens, and the first preset number of tokens are stored in the token bucket; the first preset number is smaller than or equal to the concurrent number which can be processed by a processing platform used for processing the excel file to be processed, allocating tokens to the first data, processing the first data through the processing platform to obtain a first processing result corresponding to the first data, then putting the tokens allocated to the first data back to a token bucket, and repeatedly executing the implementation. Until all the first data are processed. According to the technical scheme, in the concurrent processing process, waiting for the first data which is not processed completely is not needed, so that the next first data which is not processed is directly executed, the processing waiting time is shortened, and the processing efficiency of the excel file is improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a method, apparatus, device, and storage medium for processing tabular files. Background Art

[0002] During the process of processing an Excel file, when the amount of data in the Excel file is large and it is associated with third-party data, generally, a third-party interface is called and backfilled to the Excel file to achieve the processing of the Excel file.

[0003] In the prior art, generally, a batch of concurrent requests are made with several data in the Excel file, and when all the request results are backfilled to the Excel file, the next batch of requests is made.

[0004] However, in the above technology, when any one of several data is processed slowly, it will cause the disadvantage of reduced processing efficiency of the entire Excel file. Summary of the Invention

[0005] Embodiments of this application provide a method, apparatus, device, and storage medium for processing tabular files to solve the technical problem of low data processing efficiency in the prior art for Excel files.

[0006] In a first aspect, embodiments of this application provide a method for processing a tabular file, including:

[0007] S1. Obtain a plurality of first data corresponding to the Excel file to be processed.

[0008] S2. Take tokens from a preset token bucket and allocate them to the unprocessed first data. The token bucket stores a first preset number of tokens, and the first preset number is less than or equal to the concurrent number that the processing platform for processing the Excel file to be processed can handle.

[0009] S3. If the first data is allocated with the token, process the first data through the processing platform to obtain a first processing result corresponding to the first data.

[0010] S4. Put the token allocated to the first data back into the token bucket, and repeat steps S2 - S4 until all the first data are processed.

[0011] In a possible implementation manner, after processing the first data through the processing platform to obtain a first processing result corresponding to the first data, the method further includes:

[0012] When there is unprocessed first data and the number of processed first data reaches a second preset number, save the first processing result corresponding to the second preset number to the to-be-processed Excel file, and clear the number of processed first data;

[0013] If there is no unprocessed first data, save the first processing result corresponding to the processed first data to the to-be-processed Excel file.

[0014] In a possible implementation manner, the first data is serialized data;

[0015] Correspondingly, saving the first processing result to the to-be-processed Excel file includes:

[0016] Perform deserialization processing on the first processing result to obtain a second processing result;

[0017] Save the second processing result to the to-be-processed Excel file.

[0018] In a possible implementation manner, obtaining multiple first data corresponding to the to-be-processed Excel file includes:

[0019] Obtain the to-be-processed Excel file;

[0020] Serialize multiple to-be-processed row data in the to-be-processed Excel file to obtain multiple first data corresponding to the multiple to-be-processed row data.

[0021] In a possible implementation manner, processing the first data through the processing platform to obtain a first processing result corresponding to the first data includes:

[0022] Call the interface of the processing platform and send the first data to the processing platform;

[0023] Receive the first processing result of the processing platform for the first data.

[0024] In a possible implementation manner, serializing multiple to-be-processed row data in the to-be-processed Excel file to obtain multiple first data corresponding to the multiple to-be-processed row data includes:

[0025] Convert the multiple to-be-processed row data into Json format to obtain multiple first data corresponding to the multiple to-be-processed row data.

[0026] In a second aspect, an embodiment of the present application provides a processing device for a table file, including:

[0027] An acquisition module, configured to execute S1 to acquire a plurality of first data corresponding to the to-be-processed excel file;

[0028] An allocation module, configured to execute S2 to take tokens from a preset token bucket and allocate them to the unprocessed first data. The token bucket stores a first preset number of tokens, and the first preset number is less than or equal to the concurrent number that can be processed by the processing platform for processing the to-be-processed excel file;

[0029] A processing module, for S3, if the first data is allocated with the token, process the first data through the processing platform to obtain a first processing result corresponding to the first data;

[0030] The allocation module is further configured to execute S4 to put the token allocated to the first data back into the token bucket, and repeat steps S2 - S4 until all the first data are processed.

[0031] In a possible implementation manner, after obtaining the first processing result corresponding to the first data by processing the first data through the processing platform, the processing module is further configured to:

[0032] If there is unprocessed first data and the number of processed first data reaches a second preset number, save the first processing result corresponding to the second preset number to the to-be-processed excel file, and clear the number of processed first data;

[0033] If there is no unprocessed first data, save the first processing result corresponding to the processed first data to the to-be-processed excel file.

[0034] In a possible implementation manner, the first data is serialized data;

[0035] Correspondingly, when the processing module saves the first processing result to the to-be-processed excel file, it specifically is configured to:

[0036] Perform deserialization processing on the first processing result to obtain a second processing result;

[0037] Save the second processing result to the to-be-processed excel file.

[0038] In a possible implementation manner, when the acquisition module acquires a plurality of first data corresponding to the to-be-processed excel file, it specifically is configured to:

[0039] Acquire the to-be-processed excel file;

[0040] Serialize multiple rows of data to be processed in the Excel file to be processed, to obtain multiple first data corresponding to the multiple rows of data to be processed.

[0041] In a possible implementation manner, the processing module processes the first data through the processing platform to obtain a first processing result corresponding to the first data, specifically for:

[0042] Call the interface of the processing platform, and send the first data to the processing platform;

[0043] Receive the first processing result of the processing platform for the first data.

[0044] In a possible implementation manner, the processing module serializes multiple rows of data to be processed in the Excel file to be processed, to obtain multiple first data corresponding to the multiple rows of data to be processed, specifically for:

[0045] Convert the multiple rows of data to be processed into the Json format to obtain multiple first data corresponding to the multiple rows of data to be processed.

[0046] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0047] The memory stores computer execution instructions;

[0048] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0050] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the above first aspect and / or various possible implementation manners of the first aspect.

[0051] The processing method, device, equipment and storage medium for table files provided by the embodiments of the present application obtain a plurality of first data corresponding to the to-be-processed excel file, and take out tokens from a preset token bucket and allocate them to the unprocessed first data. The token bucket stores a first preset number of tokens, and the first preset number is less than or equal to the concurrent number that can be processed by the processing platform for processing the to-be-processed excel file. When a token is allocated to the first data, the first data is processed by the processing platform to obtain a first processing result corresponding to the first data. Then, the token allocated to the first data is put back into the token bucket, and the above implementation is repeated until all the first data are processed. In this technical solution, tokens are allocated to each processed first data, and only by holding a token can it be processed by the processing platform. After processing, the token is allocated to the unprocessed first data, so that during the concurrent processing, there is no need to wait for the unprocessed first data, and thus the next unprocessed first data can be directly executed, reducing the waiting time for processing and improving the processing efficiency of the excel file. Description of the Drawings

[0052] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0053] Figure 1 Flow diagram of the processing method for table files provided by the embodiments of the present application Figure 1 ;

[0054] Figure 2 Flow diagram of the processing method for table files provided by the embodiments of the present application Figure 2 ;

[0055] Figure 3 Flow diagram of the processing method for table files provided by the embodiments of the present application Figure 3 ;

[0056] Figure 4 Schematic diagram of the execution logic of the new token bucket algorithm provided by the embodiments of the present application;

[0057] Figure 5 Schematic diagram of the request time of traditional requests;

[0058] Figure 6 Schematic diagram of the request time provided by the embodiments of the present application;

[0059] Figure 7 Schematic diagram of the structure of the processing device for table files provided by the embodiments of the present application;

[0060] Figure 8 Schematic diagram of the structure of the electronic device provided by the embodiments of the present application.

[0061] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0062] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0063] First, the technical background involved in this application is described:

[0064] The manual maintenance of Excel files with large data volumes (e.g., millions of data) is time-consuming and error-prone. In addition, the data in complex Excel files needs to be associated with third-party data. Therefore, it is generally implemented with the help of a program, that is, calling a third-party interface to process the data and then filling it back into the Excel file.

[0065] The third party may be an entity that owns a processing platform that responds to data requests in the Excel file.

[0066] In the above implementation process, the following technical problems still exist:

[0067] 1) Low efficiency: When calling a third-party interface, the concurrency provided by the other party is used as a reference to initialize the program. The existing technical process is batch concurrent requests. When all the request results are returned and filled into Excel, the next batch of requests will be made. Therefore, the longest request link will slow down the processing time of the entire batch, resulting in low overall processing efficiency.

[0068] 2) High risk: When processing millions of data and relying on third-party interfaces, there will usually be errors in the process. The traditional processing method will lead to the loss of process data, which not only wastes time, but also leads to financial losses if the third party is a paid interface.

[0069] Based on the above existing technical problems, the inventor's technical concept is as follows: In the process of processing a batch of data requests, if, after any request in this batch is processed, if a new request can be responded and executed without waiting for the entire batch to be completed, the processing efficiency can be improved. Therefore, if each request is configured with a token, and the total number of tokens is set to the maximum number of requests that the third party can process, when a request is executed, if it is allocated a token, it can be executed, and after execution, the token is returned to the next request, thus solving the above problems.

[0070] Furthermore, in order to avoid data loss, a certain threshold can be configured during the writing stage of the execution result. When the number of execution results corresponding to multiple requests reaches this threshold, they are directly saved, thus avoiding the technical problem of overall data loss caused by errors.

[0071] The following uses specific embodiments to elaborate in detail on the technical solutions of this application and how the technical solutions of this application solve the above technical problems. These several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following will describe the embodiments of this application in conjunction with the accompanying drawings.

[0072] Figure 1 Flow schematic of the processing method for the table file provided in the embodiment of this application Figure 1 , as Figure 1 shown, this method includes:

[0073] S1, obtain multiple first data corresponding to the excel file to be processed;

[0074] In this step, obtain the excel file to be processed, denoted as the excel file to be processed. This excel file to be processed contains multiple data to be processed, denoted as multiple first data.

[0075] In a possible implementation, the first data can be the row data in the excel file to be processed, that is, the row data to be processed. If there are N rows of data to be processed in the excel file to be processed, the number of first data is N, and N is a positive integer, without limiting the specific quantity, which can be in the millions.

[0076] Correspondingly, the first data can be: serialized data;

[0077] Then in step S1, a possible implementation of "obtain the excel file to be processed" can be: serialize the multiple row data to be processed in the excel file to be processed to obtain multiple first data corresponding to the multiple row data to be processed.

[0078] In this implementation, in order to make the to-be-processed Excel file in a format recognizable by a machine, it is necessary to serialize multiple to-be-processed row data in the to-be-processed Excel file to obtain multiple first data after serialization.

[0079] In a possible implementation, the implementation of "serializing multiple to-be-processed row data in the to-be-processed Excel file to obtain multiple first data corresponding to the multiple to-be-processed row data" can be: converting the multiple to-be-processed row data into the Json format to obtain multiple first data corresponding to the multiple to-be-processed row data.

[0080] That is to say, the process of serializing multiple to-be-processed row data in the Excel file can be converting the multiple to-be-processed row data into the Json format.

[0081] Among them, the first data in step S1 are all defaulted to unprocessed first data.

[0082] S2, take tokens from a preset token bucket and allocate them to the unprocessed first data;

[0083] Among them, the token bucket stores a first preset number of tokens; the first preset number is less than or equal to the concurrent number that the processing platform for processing the to-be-processed Excel file can handle;

[0084] In this step, based on the concurrent number that the processing platform for processing the to-be-processed Excel file can handle, as the number of tokens placed in the token bucket, in a better case, the number of tokens, that is, the first preset number, can be set to be equal to the concurrent number that the processing platform can handle.

[0085] If all the first data of the to-be-processed Excel file are unprocessed, then take out the corresponding number of first data for allocation according to the number of tokens in the token bucket; if there are already processed first data in the to-be-processed Excel file, then take out the existing tokens from the token bucket and allocate them to the unprocessed first data respectively.

[0086] In a possible implementation, taking the example that the number of first data in the to-be-processed Excel file is 20, and the number of tokens in the token bucket is 5, then allocate tokens to the first 5 of the 20 (i.e., do A1, A2, A3, A4, A5) respectively.

[0087] S3, if a token is allocated to the first data, process the first data through the processing platform to obtain a first processing result corresponding to the first data;

[0088] In this step, after a token is allocated to the first data, the first data can be processed, that is, process the first data through the processing platform to obtain the corresponding processing result, denoted as the first processing result.

[0089] Correspondingly, if the first data does not have a token, the first data is not processed.

[0090] In a possible implementation, the above serialization can make multiple first data form a sequence, so as to ensure that after any first data assigned with a token is processed, the allocation implementation and processing implementation of the next unprocessed first data can start.

[0091] During actual processing, a processing request can be determined based on the first data assigned with a token to request processing of the first data.

[0092] Optionally, the implementation of "processing the first data through a processing platform to obtain a first processing result corresponding to the first data" in step S3 can be:

[0093] Call the interface of the processing platform, send the first data to the processing platform; receive the first processing result of the processing platform for the first data.

[0094] In a possible implementation, the processing platform can be a third-party processing entity. After calling the interface of the processing platform to process the first data and obtaining the information associated with the first data (i.e., the first processing result) in the processing platform, the information is returned to the execution entity of the embodiment of the present application.

[0095] S4, put the token assigned to the first data back into the token bucket, and repeat steps S2 - S4 until all the first data are processed.

[0096] In this step, after the first data assigned with the token is executed and the first processing result is returned, the token corresponding to the first data is put back into the token bucket, so that the token can be assigned to subsequent unprocessed first data.

[0097] Furthermore, repeat the above process of allocating tokens, responding to processing, and putting back tokens until all the first data in the to-be-processed excel file are processed.

[0098] The processing method for tabular files provided by the embodiments of the present application obtains a plurality of first data corresponding to the to-be-processed Excel file, and takes tokens from a preset token bucket and assigns them to the unprocessed first data. The token bucket stores a first preset number of tokens, and the first preset number is less than or equal to the concurrent number that the processing platform for processing the to-be-processed Excel file can handle. When a token is assigned to the first data, the first data is processed by the processing platform to obtain a first processing result corresponding to the first data. Then, the token assigned to the first data is returned to the token bucket, and the above implementation is repeated until all the first data are processed. In this technical solution, tokens are assigned to each processed first data, and only those holding tokens can be processed by the processing platform. After processing, the token is assigned to the unprocessed first data, so that during the concurrent processing, there is no need to wait for the unprocessed first data, and thus the next unprocessed first data can be directly executed, reducing the waiting time for processing and improving the processing efficiency of the Excel file.

[0099] Based on the above embodiments, Figure 2 is a flowchart of the processing method for tabular files provided by the embodiments of the present application Figure 2 , as Figure 2 shown, after "processing the first data through the processing platform to obtain a first processing result corresponding to the first data" in S3 above, the following method can also be executed:

[0100] Among them, step 21 and step 22 are executed alternatively.

[0101] Step 21: If there is unprocessed first data and the number of processed first data reaches a second preset number, save the first processing results corresponding to the second preset number to the to-be-processed Excel file, and clear the number of processed first data;

[0102] In this step, after obtaining the first processing result corresponding to the first data, if it is determined that there is still unprocessed first data in the to-be-processed Excel file, then when the number of processed first data reaches the second preset number, these first processing results can be saved to avoid data loss caused by some accidental errors.

[0103] Furthermore, clear the number of processed first data to ensure that the number of subsequent processed first data can be accumulated again.

[0104] Step 22: If there is no unprocessed first data, save the first processing results corresponding to the processed first data to the to-be-processed Excel file.

[0105] In this step, after obtaining the first processing result corresponding to the first data, if it is determined that all the first data in the to-be-processed excel file have been processed, it is considered that there is no longer any first data that needs to be processed by the processing platform. Then, the first processing result corresponding to the processed first data is saved to the to-be-processed excel file to avoid omission.

[0106] Optionally, in steps 21 and 22, the first data may be: serialized data; then the implementation of saving the first processing result to the to-be-processed excel file may be:

[0107] Step 1: Deserialize the first processing result to obtain a second processing result;

[0108] In the above implementation, since the first data is serialized data, in order to restore it to the form recorded in the excel file, in this implementation, it is necessary to deserialize the first processing result corresponding to the first data to obtain the corresponding second processing result.

[0109] Step 2: Save the second processing result to the to-be-processed excel file.

[0110] In this implementation, the second processing result is saved to the to-be-processed excel file in the order before serialization to generate a processed excel file.

[0111] The method for processing a tabular file provided by an embodiment of the present application, when there is unprocessed first data and the number of processed first data reaches a second preset number, saves the first processing result corresponding to the second preset number to the to-be-processed excel file and clears the number of processed first data; if there is no unprocessed first data, saves the first processing result corresponding to the processed first data to the to-be-processed excel file. In this technical solution, by judging that the number of processed first data reaches a set threshold and saving, it avoids the problem of overall data loss caused by errors, and directly saves after there is no unprocessed first data, avoiding the situation of omission in saving.

[0112] Based on the above embodiments, Figure 3 is a flowchart of the method for processing a tabular file provided by an embodiment of the present application Figure 3 , as Figure 3 shown, the solution of the embodiment of the present application is described as a whole:

[0113] Step 1: Start;

[0114] Step 2: Receive the to-be-processed excel file;

[0115] Step 3: Serialize the to-be-processed excel file;

[0116] Step 4: Use the new token bucket algorithm to backfill the data of the Excel file to be processed;

[0117] Among them, the implementation of the new token bucket algorithm is as Figure 4 shown, Figure 4 is the execution logic schematic diagram of the new token bucket algorithm provided by the embodiment of the present application. As Figure 4 shown, in Step 4, the execution includes:

[0118] a) The token bucket stores a certain number of tokens (the threshold of the concurrent number that the third-party website can withstand, that is, the first preset number).

[0119] b) Each request will carry a token.

[0120] c) After each response, put back the token and trigger the next request action, repeating b.

[0121] d) When the request queue is empty, the task ends.

[0122] Step 5: Deserialize;

[0123] Step 6: Save as an Excel file;

[0124] Step 7: Judge the Excel file to be processed. If so, execute Step 8; if not, execute Step 3;

[0125] Step 8: End.

[0126] The processing method of the table file provided by the embodiment of the present application has the same implementation principle and technical effect as the above embodiment, and will not be elaborated here.

[0127] Based on the above implementation, Figure 5 is the request time schematic diagram of the traditional request, Figure 6 is the request time schematic diagram provided by the embodiment of the present application. Combining Figure 5 and Figure 6 , the effect of the embodiment of the present application is described (each black bar represents a request).

[0128] In the prior art, as Figure 5 shown, the traditional request method is limited by the longest response time of each batch request. For example, the total request interface = N; the batch request quantity = the concurrent capacity of the third-party website = C; the average interface response duration = T; the interface response duration range = [Tmin, Tmax]; for C requests, the longest response time Tmax-batch within the batch is a random variable, and its probability distribution can be obtained by statistical methods.

[0129] That is, the cumulative distribution function (abbreviation in English: CDF) of a single request is:

[0130] F(t) = (t - Tmin) / (Tmax - Tmin)

[0131] where t is between Tmin and Tmax;

[0132] The cumulative distribution function of the longest response time Tmax - batch for each batch of C requests is:

[0133]

[0134] Then, the corresponding probability density function (abbreviation in English: PDF) is:

[0135]

[0136] where t is between Tmin and Tmax.

[0137] Expected value calculation, the expected value of the longest time (time consumption per batch) is:

[0138]

[0139] Substitute into the formula and expand:

[0140]

[0141] For example, the time distribution of each request is [1, 5] seconds, the number of concurrent requests per time C = 10, substituting into the man-hour result is:

[0142]

[0143] That is, in the prior art, the total duration for N requests is 4.46 * N / C.

[0144] In this application, as Figure 6 shown, the average time consumption per request Tavg = (Tmin + Tmax) / 2; for example, the total duration is: Tavg * (N / C);

[0145] Assume that the time distribution of each request is [1, 5] seconds, N = 10, the number of concurrent requests per time C = 10, the average request duration is 3 seconds, then the total consumption duration is 3 seconds, that is, the total duration for N requests is 3 * N / C.

[0146] Then, combining Figure 5 and Figure 6 , the calculation formula for the time difference between the two is:

[0147]

[0148] That is, when N and C are fixed, the greater the difference between Tmax and Tmin, the more time difference there is between the two methods. That is, the greater the variance of the time consumed by the interface, the more time difference there is between the two. When the time consumed by the third-party interface is unstable, the new token bucket calling method has a significant effect.

[0149] The following is the description of the device embodiment provided by the present application.

[0150] Figure 7 It is a schematic structural diagram of a processing device for a table file provided by an embodiment of the present application. As Figure 7 shown, the processing device for the table file provided by this embodiment includes:

[0151] An acquisition module 71, configured to execute S1 to acquire a plurality of first data corresponding to the to-be-processed excel file;

[0152] An allocation module 72, configured to execute S2 to take out tokens from a preset token bucket and allocate them to the unprocessed first data. The token bucket stores a first preset number of tokens, and the first preset number is less than or equal to the concurrent number that the processing platform for processing the to-be-processed excel file can handle;

[0153] A processing module 73, configured to perform S3. If the first data is allocated with the token, process the first data through the processing platform to obtain a first processing result corresponding to the first data;

[0154] The allocation module 72 is further configured to perform S4, put the tokens allocated to the first data back into the token bucket, and repeat steps S2 - S4 until all the first data are processed.

[0155] In a possible implementation manner, after processing the first data through the processing platform to obtain a first processing result corresponding to the first data, the processing module 73 is further configured to:

[0156] If there is unprocessed first data and the number of processed first data reaches a second preset number, save the first processing result corresponding to the second preset number to the to-be-processed excel file, and clear the number of processed first data;

[0157] If there is no unprocessed first data, save the first processing result corresponding to the processed first data to the to-be-processed excel file.

[0158] In a possible implementation manner, the first data is serialized data;

[0159] Correspondingly, the processing module 73 saves the first processing result to the to-be-processed excel file, specifically for:

[0160] Perform deserialization processing on the first processing result to obtain a second processing result;

[0161] Save the second processing result to the to-be-processed excel file.

[0162] In a possible implementation manner, the obtaining module 71 obtains multiple first data corresponding to the to-be-processed excel file, specifically for:

[0163] Obtain the to-be-processed excel file;

[0164] Perform serialization on multiple to-be-processed row data in the to-be-processed excel file to obtain multiple first data corresponding to the multiple to-be-processed row data.

[0165] In a possible implementation manner, the processing module 73 processes the first data through the processing platform to obtain a first processing result corresponding to the first data, specifically for:

[0166] Call the interface of the processing platform and send the first data to the processing platform;

[0167] Receive the first processing result of the processing platform for the first data.

[0168] In a possible implementation manner, the processing module 73 performs serialization on multiple to-be-processed row data in the to-be-processed excel file to obtain multiple first data corresponding to the multiple to-be-processed row data, specifically for:

[0169] Convert the multiple to-be-processed row data into Json format to obtain multiple first data corresponding to the multiple to-be-processed row data.

[0170] The processing device for the table file provided in this embodiment can execute the processing method for the table file provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0171] Figure 8 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application. As Figure 8 shown, the electronic device provided in this embodiment includes: at least one processor 81 and a memory 82.

[0172] Optionally, the device further includes: a communication component 83. Among them, the processor 81, the memory 82, and the communication component 83 are connected through a bus 84.

[0173] In a specific implementation process, at least one processor 81 executes computer-executable instructions stored in a memory 82, so that at least one processor 81 executes the above-mentioned method.

[0174] For the specific implementation process of the processor 81, reference may be made to the above method embodiments. Their implementation principles and technical effects are similar, and will not be elaborated here in this embodiment.

[0175] In the above embodiments, it should be understood that the processor may be a central processing unit (Central Processing Unit, CPU for short), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0176] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0177] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0178] This application also provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0179] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the above-mentioned method is implemented.

[0180] The above-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk. The readable storage medium may be any available medium accessible by a general-purpose or special-purpose computer.

[0181] An exemplary readable storage medium is coupled to the processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium may also be a component of the processor. The processor and the readable storage medium may be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium may also exist as discrete components in a device.

[0182] The division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other may be indirect couplings or communication connections through some interfaces, devices, or units, and may be in electrical, mechanical, or other forms.

[0183] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] In addition, the functional units in various embodiments of the present invention may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit.

[0185] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., various media that can store program codes.

[0186] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the aforementioned storage medium includes: ROMs, RAMs, magnetic disks, or optical discs, etc., various media that can store program codes.

[0187] Finally, it should be noted that: After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily think of other implementation manners of the present invention. The present invention is intended to cover any variations, uses, or adaptive changes of the present invention. These variations, uses, or adaptive changes follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for processing a table file, characterized in that, Including: S1. Obtain multiple first data corresponding to the to-be-processed Excel file; S2. Take tokens from a preset token bucket and allocate them to the unprocessed first data. The token bucket stores a first preset number of tokens, and the first preset number is less than or equal to the concurrent number that the processing platform for processing the to-be-processed Excel file can handle; S3. If the first data is allocated with the token, process the first data through the processing platform to obtain a first processing result corresponding to the first data; S4. Put the token allocated to the first data back into the token bucket, and repeat steps S2 - S4 until all the first data are processed.

2. The method according to claim 1, wherein After obtaining the first processing result corresponding to the first data by processing the first data through the processing platform, the method further includes: If there is unprocessed first data and the number of processed first data reaches a second preset number, save the first processing results corresponding to the second preset number to the to-be-processed Excel file, and clear the number of processed first data; If there is no unprocessed first data, save the first processing results corresponding to the processed first data to the to-be-processed Excel file.

3. The method according to claim 2, characterized in that The first data is serialized data; Correspondingly, saving the first processing result to the to-be-processed Excel file includes: Performing deserialization processing on the first processing result to obtain a second processing result; Saving the second processing result to the to-be-processed Excel file.

4. The method according to claim 3, wherein The obtaining of multiple first data corresponding to the to-be-processed Excel file includes: Obtain the to-be-processed Excel file; Serialize multiple to-be-processed row data in the to-be-processed Excel file to obtain multiple first data corresponding to the multiple to-be-processed row data.

5. The method according to any one of claims 1-4, characterized in that, The obtaining of the first processing result corresponding to the first data by processing the first data through the processing platform includes: Call the interface of the processing platform and send the first data to the processing platform; Receive the first processing result of the processing platform for the first data.

6. The method according to claim 4, wherein The serializing of multiple to-be-processed row data in the to-be-processed Excel file to obtain multiple first data corresponding to the multiple to-be-processed row data includes: Convert the multiple to-be-processed row data into Json format to obtain multiple first data corresponding to the multiple to-be-processed row data.

7. A processing device for a table file, characterized in that, Including: An obtaining module, configured to execute S1 to obtain multiple first data corresponding to the to-be-processed Excel file; An allocation module, configured to execute S2 to take tokens from a preset token bucket and allocate them to the unprocessed first data. The token bucket stores a first preset number of tokens, and the first preset number is less than or equal to the concurrent number that the processing platform for processing the to-be-processed Excel file can handle; A processing module, configured to perform S3. If the first data is allocated with the token, process the first data through the processing platform to obtain a first processing result corresponding to the first data; The allocation module is further configured to, in S4, put the token allocated to the first data back into the token bucket, and repeatedly execute steps S2 - S4 until all the first data are processed.

8. An electronic device, characterized in that, Comprising: a memory, a processor; The memory stores computer - executable instructions; The processor executes the computer - executable instructions stored in the memory, so that the processor executes the method according to any one of claims 1 - 6.

9. A computer-readable storage medium, characterized in that, Computer - executable instructions are stored in the computer - readable storage medium, and when the computer - executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 - 6.

10. A computer program product, characterized in that, Comprising a computer program, which when executed by a processor implements the method according to any one of claims 1 - 6.

Images