Method and device for exporting data set into spreadsheet, equipment and medium

Abstract

The invention discloses a method and device for exporting data into a spreadsheet, equipment and a medium, and relates to the technical field of computers, the method comprises the following steps: obtaining the data size of a data set; the method comprises the following steps: determining the space size of a sub-cache according to the data size of a data set, the space size of a total available cache of a processing device and the processing capability of the processing device, and in the process of caching data in the data set to a plurality of cache regions, determining the space size of the sub-cache; if the cache line number of the cached data in the target cache region reaches the cache line number threshold value of the target cache region, stopping caching the data in the data set to the target cache region, and serializing the cached data in the target cache region to a disk to obtain a temporary file; and generating a spreadsheet document corresponding to the data set based on the temporary file. According to the method, the situation of long-time jamming or collapse can be reduced, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and medium for exporting data into an electronic spreadsheet. Background Art

[0002] A data set consisting of large-scale data is not convenient for users to view or preview, so the data set needs to be exported to a spreadsheet format, such as an EXCEL table.

[0003] In the existing Java ecosystem, the technical solution for exporting large-scale data sets into spreadsheets is often to directly load the large-scale data sets into memory. Due to the large amount of data, the system will have no free memory available for a long time, which may easily cause long-term freezes or crashes, affecting the user experience. Summary of the invention

[0004] The present application provides a method, apparatus, device and medium for exporting a data set into a spreadsheet, which can reduce long-term freezing or crashing and improve user experience.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions: In a first aspect, the present application provides a method for exporting a data set into a spreadsheet, which is applied to a processing device, and the method comprises: The processing device obtains the data size of the data set; The processing device determines the space size of the sub-cache according to the data size of the data set, the space size of the total available cache of the processing device and the processing capability, and determines the multiple cache areas constituting the sub-cache according to the space size of the sub-cache; The processing device caches the data in the data set into a plurality of cache areas; During the process of the processing device caching the data in the data set into a plurality of cache areas, if the number of cache lines of the cached data in the target cache area reaches a cache line number threshold of the target cache area, the data in the data set is stopped from being cached into the target cache area, the cached data in the target cache area is serialized onto a disk to obtain a temporary file, and the space of the target cache area is released; In the case that there is still residual data in the data set that has not been written into the cache area, the processing device caches the residual data in the data set into the target cache area; The processing device generates a spreadsheet document corresponding to the data set based on the temporary file.

[0006] In some possible implementations, the processing device determines the space size of the sub-cache according to the data size of the data set, the space size of the total available cache of the processing device, and the processing capability, including: The processing device vectorizes the data size of the data set, the space size of the total available cache, and the processing capacity to obtain a first feature vector; Inputting the first feature vector into a cache prediction model to obtain a sub-cache space size; Among them, the cache prediction model is trained based on sample data, and the sample data includes multiple groups of corresponding sample features and sample labels. The sample features include the data size of the sample data set, the total available cache of the sample processing device, and the sample processing capacity. The sample label includes the space size of the sample sub-cache; the sample label is the label corresponding to the shortest conversion time corresponding to the sample feature during the historical conversion process.

[0007] In some possible implementations, the data size of the data set is larger than the space size of a total available cache of the processing device, and the processing device determines the space size of the sub-cache according to the data size of the data set, the space size of the total available cache of the processing device, and the processing capability, including:

[0008] Wherein, f represents the space size of the sub-cache, A represents the data size of the data set, B represents the space size of the total available cache of the processing device, and C represents the processing capability of the processing device.

[0009] In some possible implementations, the processing device generates a spreadsheet document corresponding to the data set based on the temporary file, including: The processing device creates a new first table document; The processing device writes data to the first table page of the first table document based on the temporary file to obtain an electronic table document corresponding to the data set.

[0010] In some possible implementations, the method further includes: In the process of the processing device writing data to the first table document based on the temporary file, obtaining the number of data rows in which data has been written in the first table page; If the number of data rows in which data has been written in the first table page reaches a data row number threshold, creating a second table page in the first table document; The processing device writes data to a second form page of the first form document based on the temporary file.

[0011] In some possible implementations, writing data to the first table page of the first table document includes: Data is written to the first table page of the first table document by disabling the style.

[0012] In some possible implementations, after generating a spreadsheet document corresponding to the data set, the method further includes: The processing device clears the temporary file.

[0013] In a second aspect, the present application provides a device for exporting a data set into a spreadsheet, the device comprising: The acquisition module is used to obtain the data size of the data set; a determination module, configured to determine the space size of a sub-cache according to the data size of the data set, the space size of a total available cache of a processing device, and a processing capability, and determine a plurality of cache areas constituting the sub-cache according to the space size of the sub-cache; A cache module, used for caching the data in the data set to multiple cache areas. In the process of caching the data in the data set to multiple cache areas, if the number of cache lines of cached data in a target cache area reaches a cache line number threshold of the target cache area, stop caching the data in the data set to the target cache area, serialize the cached data in the target cache area to a disk to obtain a temporary file, and release the space of the target cache area; if there is still residual data in the data set that has not been written to the cache area, cache the remaining data in the data set to the target cache area; A generating module is used to generate a spreadsheet document corresponding to the data set based on the temporary file.

[0014] In a third aspect, the present application provides a computing device, including a memory and a processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as described in any one of the first aspects.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method as described in any one of the first aspects.

[0016] It can be seen from the above technical solution that the present application has at least the following beneficial effects: The present application provides a method for exporting a data set into a spreadsheet, which can be applied to a processing device. The method includes: the processing device obtains the data size of the data set, and then determines the space size of the sub-cache according to the data size of the data set, the space size of the total available cache of the processing device, and the processing power, and determines the multiple cache areas constituting the sub-cache according to the space size of the sub-cache. The space size of the sub-cache determined in this way is more reasonable, thereby further improving the conversion efficiency. Then, the processing device caches the data in the data set to multiple cache areas. During the caching process, if the number of cache lines of cached data in the target cache area reaches the cache line number threshold of the target cache area, the data in the data set is stopped from being cached to the target cache area, and the cached data in the target cache area is serialized to the disk to obtain a temporary file; this method limits the maximum value of the cache area to avoid memory overflow, and after the cache area reaches the maximum value, serialization is performed again, which can reduce the number of calls to the IO interface, thereby improving efficiency. After obtaining the temporary file, the space of the target cache area can be released. If there is still residual data in the data set that has not been written to the cache area, the processing device will cache the remaining data in the data set to the target cache area, and finally generate a spreadsheet document corresponding to the data set based on the temporary file. It can be seen that this method can not only avoid system memory overflow, reduce long-term freezes or crashes, but also improve the efficiency of generating spreadsheet documents.

[0017] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of features or beneficial effects means that specific technical features, technical solutions or beneficial effects are included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be realized without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in a specific embodiment that does not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a method for exporting data into a spreadsheet provided in an embodiment of the present application; Figure 2 A schematic diagram of a device for exporting data into a spreadsheet provided in an embodiment of the present application; Figure 3A schematic diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The terms "first", "second", "third", etc. in the specification of this application and the accompanying drawings are used to distinguish different objects rather than to limit a specific order.

[0020] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0021] In order to make the description of the following embodiments clear and concise, a brief introduction to the related technology is first given: At present, in the process of exporting large-scale data sets into spreadsheet documents, there is a problem of excessive memory usage, which easily leads to memory overflow and cannot efficiently process data volumes of more than one million. The data export speed is slow, and the existing solutions cannot effectively meet the export needs of big data, especially when the memory is limited, and the writing efficiency is low. In addition, during the processing, the memory allocation is unreasonable, which further reduces the export efficiency.

[0022] In view of this, an embodiment of the present application provides a method for exporting a data set as a spreadsheet, which can be executed by a processing device, and the processing device can be a terminal or a server. Terminals include but are not limited to smartphones, tablet computers, laptops, personal digital assistants, or smart wearable devices. The server can be a cloud server, such as a central server in a central cloud computing cluster, or an edge server in an edge cloud computing cluster. Of course, the server can also be a server in a local data center. A local data center refers to a data center directly controlled by a user. This method can not only avoid memory overflow of the system, reduce long-term freezes or crashes, but also improve the efficiency of generating spreadsheet documents.

[0023] In order to make the technical solution of the present application clearer and easier to understand, the technical solution of the present application is introduced below in conjunction with the accompanying drawings. Figure 1 As shown in the figure, this figure is a flow chart of a method for exporting data into a spreadsheet provided in an embodiment of the present application, the method comprising: S101. A processing device obtains a data size of a data set.

[0024] Before the processing device exports the data set as a spreadsheet document, the data size of the data set may be obtained and determined first. The data set may be relational data, structured data, and the like.

[0025] S102: The processing device determines the space size of the sub-cache according to the data size of the data set, the space size of the total available cache of the processing device, and the processing capability.

[0026] The total available cache space size of the processing device refers to the remaining memory of the processing device before the data set is exported. For example, other existing processes have occupied 10% of the total memory, and the total available cache is 90% of the total memory. The processing capacity of the processing device refers to the processing capacity of the processor of the processing device, which can be determined based on the model and actual computing power of the processor. Exemplarily, the processing capacity can be graded based on pre-set rules. For example, it can be divided into 10 levels, where the processing capacity is the smallest when it is the first level, and the processing capacity is the larger when it is the tenth level. For example, when the processing capacity is the first level, it means that the processing device can process 1000 megabytes of data per second.

[0027] The sub-cache refers to the cache space used for exporting a data set, wherein the sub-cache may be composed of multiple cache areas, and the processing device may cache the data set in multiple cache areas at the same time. In some examples, the sub-cache includes 10 cache areas, and the processing device may cache the data set in 5 cache areas at the same time, or may cache in 10 cache areas at the same time.

[0028] In this method, the processing device determines the sub-cache space size according to the data size of the data set, the total available cache space size of the processing device, and the processing capacity, that is, the sub-cache space size is determined based on the actual situation. In this way, a more appropriate sub-cache space size can be determined, thereby improving the export efficiency of the data set. The following two cases are introduced: The first one: The processing device vectorizes the data size of the data set, the total available cache space size and the processing capacity to obtain a first feature vector; the first feature vector is input into a cache prediction model to obtain the space size of the sub-cache; wherein the cache prediction model is trained based on sample data, the sample data includes multiple groups of corresponding sample features and sample labels, the sample features include the data size of the sample data set, the total available cache of the sample processing device and the sample processing capacity, the sample label includes the space size of the sample sub-cache; the sample label is the label corresponding to the sample feature when the conversion time is the shortest during the historical conversion process.

[0029] In this method, the processing device learns from historical data what value the sub-cache space size should be set to for optimal processing efficiency. Subsequently, when other data sets are processed again, predictions can be made based on the pre-trained cache prediction model to obtain the sub-cache space size.

[0030] Second type: The processing device can determine the size of the sub-cache space by the following formula:

[0031] Wherein, f represents the space size of the sub-cache, A represents the data size of the data set, B represents the space size of the total available cache of the processing device, and C represents the processing capacity of the processing device.

[0032] In the second method, the sub-cache space size determined by the above formula will not occupy 100% of the total available cache space size, thereby avoiding memory overflow. At the same time, it also uses the system's memory space as much as possible, improves memory utilization, and thus improves processing efficiency.

[0033] For example, A is 1000 MHz, B is 500 MHz, and C is the third level, that is, A=1000, B=500, C=3, and then we can calculate .

[0034] S103: The processing device determines a plurality of cache areas constituting the sub-cache according to the space size of the sub-cache.

[0035] In some examples, the space size of the sub-cache is the sum of the space sizes of multiple cache areas. In other examples, the space size of the sub-cache may also be greater than or less than the sum of the space sizes of multiple cache areas, that is, the difference between the space size of the sub-cache and the sum of the space sizes of multiple cache areas is within a preset range. For example, if the space size of the sub-cache is 1000 megabytes and the preset range is [-10, +10], then the sum of the space sizes of multiple cache areas may be 1009 megabytes or 998 megabytes.

[0036] In some examples, the spatial sizes of different cache areas may be different. After determining the spatial size of the sub-cache, the processing device may sum the spatial sizes of multiple cache areas until the difference between the sum of the spatial sizes of these cache areas and the spatial size of the sub-cache is within a preset range, that is, it is determined that these cache areas constitute a sub-cache.

[0037] S104: The processing device caches the data in the data set into multiple cache areas.

[0038] After determining the size of the sub-cache space, the processing device can cache the data set in multiple cache areas. For example, if the sub-cache includes 10 cache areas, in this case, the processing device can cache the data set in 5 cache areas at the same time.

[0039] S105. When the processing device caches the data in the data set into multiple cache areas, if the number of cache lines of data cached in the target cache area reaches the cache line number threshold of the target cache area, stop caching the data in the data set into the target cache area, serialize the cached data in the target cache area to the disk, obtain a temporary file, and release the space of the target cache area.

[0040] The number of cache rows refers to the number of data temporarily stored in the memory before the data is written to the spreadsheet document. For example, it refers to the number of cached data in the cache area, each data corresponds to each row of data in the table. The cache row number threshold refers to the maximum number of cacheable data in the cache area, and the target cache area refers to the cache area where the number of cached data cache rows reaches the cache row number threshold.

[0041] In some instances, the space sizes of different cache areas may be the same or different. Accordingly, the cache line number thresholds of different cache areas may also be different, which is not limited in the present application.

[0042] When the processing device is caching data in a data set to multiple cache areas, if it is found that a target cache area exists, the cached data in the target cache area is serialized to the disk to obtain a temporary file, and then the space of the target cache area is released. Exemplarily, when caching data in a data set to multiple cache areas, it is found that the number of data cache lines in the fourth cache area among 10 cache areas reaches the cache line number threshold of the fifth cache area, then it is determined that the target cache area exists, and then the data in the fourth cache area is serialized to the disk, and then the space of the fourth cache area is released for subsequent caching. By defining a cache line number threshold (for example, 5000 lines), the data is serialized to a temporary file on the disk when the cache line number threshold is reached, thereby avoiding loading all data into the memory at one time, reducing the occurrence of memory overflow, and thus avoiding long-term freezes or crashes.

[0043] Serialization refers to the process of converting the data structure in memory into a format that can be stored or transmitted (such as a file) and saving it to disk. This technical solution uses serialization to save temporary data to disk to reduce memory pressure, and deserializes it back to memory for subsequent processing when needed. Common serialization formats include Java object serialization, etc.

[0044] Since the processing device can perform multiple cache tasks at the same time, after the processing device stops caching the data in the data set to the target cache area, it can determine the free cache area, and then immediately cache to the free cache area. For example, the processing device can cache data to the first cache area, the second cache area, the third cache area, the fourth cache area and the fifth cache area at the same time, while the sixth cache area, the seventh cache area, the eighth cache area, the ninth cache area and the tenth cache area are idle. When the number of data cache lines in the fourth cache area reaches the cache line number threshold of the fourth cache area, stop caching data to the fourth cache area, and select an idle cache area from the remaining cache areas. For example, the selected idle cache area is the seventh cache area, and the processing device continues to cache the data in the data set to the seventh cache area, and so on. The use of such a multi-cache area and parallel cache processing method can improve processing efficiency. Moreover, in the embodiment of the present application, when the number of data cache lines in the cache area reaches the cache line number threshold of the cache area, the processing device performs serialization processing, which can reduce the number of calls to the IO interface, thereby saving time and improving processing efficiency.

[0045] Among them, temporary files refer to files temporarily created during the processing process, which are used to store cached data or intermediate results. This technical solution uses temporary files to save serialized data and automatically cleans up these files after writing is completed, ensuring the reasonable release of resources and the cleanliness of the system.

[0046] S106: When there is still remaining data in the data set that has not been written into the cache area, the processing device caches the remaining data in the data set into the target cache area.

[0047] After the processing device releases the space in the target cache area, the target cache area becomes an idle cache area. Continuing with the above example, the fourth cache area becomes an empty cache area. If there is still remaining data in the data set that has not been written to the cache area, the processing device can cache the remaining data in the data set into the target cache area, that is, cache the data into the fourth cache area that has become idle.

[0048] After multiple rounds of data caching, the data in the data set becomes 0, that is, the caching is completed. However, the number of data cache lines in the cache area does not reach the cache line number threshold of the cache area. At this time, the processing device can directly serialize the cached data in these cache areas to the disk, thereby obtaining a temporary file.

[0049] For example, if the number of data cache lines in the third cache area does not reach the cache line number threshold of the third cache area, and the number of data cache lines in the fifth cache area does not reach the cache line number threshold of the fifth cache area, the processing device can serialize the cached data in the third cache area and the cached data in the fifth cache area to the disk, thereby ensuring that all data in the data set is converted into temporary files, avoiding data loss, and thus ensuring data integrity.

[0050] S107: The processing device generates a spreadsheet document corresponding to the data set based on the temporary file.

[0051] After obtaining the temporary files corresponding to the data set, the processing device may generate a spreadsheet document corresponding to the data set based on the temporary files.

[0052] Specifically, the processing device may create a new first spreadsheet document, and then write data to a first spreadsheet page of the first spreadsheet document based on the temporary file to obtain an electronic spreadsheet document corresponding to the data set.

[0053] When the processing device writes data to the first table document based on the temporary file, it obtains the number of data rows in which data has been written in the first table page. If the number of data rows in which data has been written in the first table page reaches a data row threshold, a second table page is created in the first table document. Then, the processing device writes data to the second table page of the first table document based on the temporary file, and so on.

[0054] When the processing device writes data to the first table page of the first table document, the data can be written to the first table page of the first table document by disabling the style. Similarly, when the processing device writes data to other table pages of the first table document, the data can also be written by disabling the style.

[0055] In the embodiment of the present application, when the table page of the first table document reaches the data row number threshold (for example, 1 million rows), a new table page (second table page) is automatically created and data is continued to be written to the new table page to ensure that all data can be completely exported. In addition, by disabling the optimization means of style, the writing speed to the first table document is improved, especially when processing large data files without style, the export efficiency is significantly improved.

[0056] In other embodiments, the processing device may create a new table document instead of a new table page.

[0057] In some embodiments, after obtaining the electronic spreadsheet document based on the temporary file, the processing device may clear the temporary file, thereby reducing the usage of disk space.

[0058] Based on the above description, the embodiment of the present application provides a method for exporting a data set as a spreadsheet. Compared with the traditional large-scale data processing method, the present solution caches data in batches and serializes the data to disk when the cache reaches the upper limit to reduce memory pressure. Traditional technology usually processes all data directly in memory, which is easy to cause memory overflow. The present application uses serialization and deserialization mechanisms to ensure the complete processing of big data and greatly reduce memory consumption. The present application solution can automatically create a new table page dynamically according to the maximum number of rows of the table page to ensure the integrity of data export. In the traditional method, the user needs to manually split the data and create multiple table pages. The present application simplifies the operation, which is particularly suitable for data export scenarios with more than one million rows, and solves the problem of data loss caused by excessive number of rows when processing large-scale data in traditional technology. In the present application solution, dynamic generation and customization of headers are allowed, and the header structure is automatically adjusted according to the changes in data key values ​​during data export. The present application significantly improves the writing speed by disabling styles, especially when processing large-scale styleless data. Compared with the performance overhead caused by enabling styles by default in traditional solutions, the present solution effectively optimizes the writing speed and reduces the impact of style rendering on performance. After the data writing task is completed, this application automatically cleans up the serialized temporary files and cached data in memory. This not only ensures the integrity of the export task, but also further improves the recycling efficiency of system resources. Compared with the common manual cleaning in traditional technologies, this solution has obvious advantages in operation automation and resource management.

[0059] Combination of the above Figure 1 The method for exporting data into a spreadsheet provided in an embodiment of the present application is introduced in detail. The apparatus and device provided in the embodiment of the present application will be introduced below in conjunction with the accompanying drawings.

[0060] like Figure 2 As shown in FIG. 1 , the figure is a schematic diagram of a device for exporting data into a spreadsheet provided in an embodiment of the present application, the device comprising: An acquisition module 201 is used to acquire the data size of a data set; A determination module 202, configured to determine the space size of a sub-cache according to the data size of the data set, the space size of a total available cache of a processing device, and the processing capability, and determine a plurality of cache areas constituting the sub-cache according to the space size of the sub-cache; The cache module 203 is used to cache the data in the data set to multiple cache areas. In the process of caching the data in the data set to multiple cache areas, if the number of cache lines of cached data in the target cache area reaches the cache line number threshold of the target cache area, stop caching the data in the data set to the target cache area, serialize the cached data in the target cache area to the disk, obtain a temporary file, and release the space of the target cache area; if there is still residual data in the data set that has not been written to the cache area, cache the remaining data in the data set to the target cache area; The generating module 204 is configured to generate a spreadsheet document corresponding to the data set based on the temporary file.

[0061] In some possible implementations, the determination module 202 is specifically used to vectorize the data size of the data set, the space size of the total available cache, and the processing capacity to obtain a first feature vector; the first feature vector is input into a cache prediction model to obtain the space size of the sub-cache; wherein the cache prediction model is trained based on sample data, the sample data includes multiple groups of mutually corresponding sample features and sample labels, the sample features include the data size of the sample data set, the total available cache of the sample processing device, and the sample processing capacity, the sample label includes the space size of the sample sub-cache; the sample label is the label corresponding to the shortest conversion time corresponding to the sample feature during the historical conversion process.

[0062] In some possible implementations, the determination module 202 is specifically configured to determine the size of the sub-cache space in the following manner:

[0063] Wherein, f represents the space size of the sub-cache, A represents the data size of the data set, B represents the space size of the total available cache of the processing device, and C represents the processing capability of the processing device.

[0064] In some possible implementations, the generation module 204 is specifically configured to create a first spreadsheet document; based on the temporary file, write data to a first spreadsheet page of the first spreadsheet document to obtain an electronic spreadsheet document corresponding to the data set.

[0065] In some possible implementations, the generation module 204 is also used to obtain the number of data rows in which data has been written in the first table page during the process of writing data to the first table document based on the temporary file; if the number of data rows in which data has been written in the first table page reaches a data row threshold, a second table page is created in the first table document; and data is written to the second table page of the first table document based on the temporary file.

[0066] In some possible implementations, the generating module 204 is specifically configured to write data to the first table page of the first table document by disabling a style.

[0067] In some possible implementations, the generating module 204 is further configured to clear the temporary file.

[0068] The device for exporting data into an electronic table according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above-mentioned other operations and / or functions of each module / unit of the device for exporting data into an electronic table are respectively to realize Figure 1 For the sake of brevity, the corresponding processes of each method in the illustrated embodiment are not described in detail here.

[0069] The present application also provides a computing device. Figure 3 As shown, this figure is a schematic diagram of a computing device provided in an embodiment of the present application, and the computing device 300 includes a bus 301, a processor 302, a communication interface 303 and a memory 304. The processor 302, the memory 304 and the communication interface 303 communicate with each other through the bus 301.

[0070] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0071] The processor 302 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0072] The communication interface 303 is used for communicating with the outside.

[0073] The memory 304 may include a volatile memory, such as a random access memory (RAM). The memory 304 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0074] The memory 304 stores executable codes, and the processor 302 executes the executable codes to perform the aforementioned method of exporting data into a spreadsheet.

[0075] Specifically, in implementing Figure 2 In the case of the embodiment shown, and Figure 2 When each module or unit of the device for exporting data into a spreadsheet described in the embodiment is implemented by software, the execution Figure 2 The software or program code required for the functions of each module / unit in the system may be partially or completely stored in the memory 304. The processor 302 executes the program code corresponding to each unit stored in the memory 304 to perform the aforementioned method of exporting data into a spreadsheet.

[0076] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned method of exporting data into a spreadsheet.

[0077] The embodiment of the present application further provides a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part.

[0078] The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0079] When the computer program product is executed by a computer, the computer executes any of the aforementioned methods for exporting data into an electronic spreadsheet. The computer program product may be a software installation package, and when any of the aforementioned methods for exporting data into an electronic spreadsheet is needed, the computer program product may be downloaded and executed on a computer.

[0080] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0081] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A method for exporting a data set into a spreadsheet, characterized in that: Applied to a processing device, the method comprises: The processing device obtains the data size of the data set; The processing device determines the space size of the sub-cache according to the data size of the data set, the space size of the total available cache of the processing device, and the processing capability; Determining a plurality of cache areas constituting the sub-cache according to the space size of the sub-cache; The processing device caches the data in the data set into a plurality of cache areas; During the process of the processing device caching the data in the data set into a plurality of cache areas, if the number of cache lines of the cached data in the target cache area reaches a cache line number threshold of the target cache area, the data in the data set is stopped from being cached into the target cache area, the cached data in the target cache area is serialized onto a disk to obtain a temporary file, and the space of the target cache area is released; In the case that there is still residual data in the data set that has not been written into the cache area, the processing device caches the residual data in the data set into the target cache area; The processing device generates a spreadsheet document corresponding to the data set based on the temporary file.

2. The method according to claim 1, characterized in that The processing device determines the space size of the sub-cache according to the data size of the data set, the space size of the total available cache of the processing device, and the processing capability, including: The processing device vectorizes the data size of the data set, the space size of the total available cache, and the processing capacity to obtain a first feature vector; Inputting the first feature vector into a cache prediction model to obtain a sub-cache space size; Among them, the cache prediction model is trained based on sample data, and the sample data includes multiple groups of corresponding sample features and sample labels. The sample features include the data size of the sample data set, the total available cache of the sample processing device, and the sample processing capacity. The sample label includes the space size of the sample sub-cache; the sample label is the label corresponding to the shortest conversion time corresponding to the sample feature during the historical conversion process.

3. The method according to claim 1, characterized in that: The data size of the data set is larger than the total available cache space size of the processing device, and the processing device determines the sub-cache space size according to the data size of the data set, the total available cache space size of the processing device, and the processing capability, including: Wherein, f represents the space size of the sub-cache, A represents the data size of the data set, B represents the space size of the total available cache of the processing device, and C represents the processing capability of the processing device.

4. The method according to claim 1, characterized in that The processing device generates a spreadsheet document corresponding to the data set based on the temporary file, including: The processing device creates a new first table document; The processing device writes data to the first table page of the first table document based on the temporary file to obtain an electronic table document corresponding to the data set.

5. The method according to claim 4, characterized in that The method further comprises: In the process of the processing device writing data to the first table document based on the temporary file, obtaining the number of data rows in which data has been written in the first table page; If the number of data rows in which data has been written in the first table page reaches a data row number threshold, creating a second table page in the first table document; The processing device writes data to a second form page of the first form document based on the temporary file.

6. The method according to claim 4, characterized in that The step of writing data to the first table page of the first table document comprises: Data is written to the first table page of the first table document by disabling the style.

7. The method according to any one of claims 1 to 6, characterized in that: After generating the electronic spreadsheet document corresponding to the data set, the method further includes: The processing device clears the temporary file.

8. A device for exporting a data set into a spreadsheet, characterized in that: The device comprises: The acquisition module is used to obtain the data size of the data set; A determination module, configured to determine the space size of a sub-cache according to the data size of the data set, the space size of a total available cache of a processing device, and the processing capability; and determine a plurality of cache areas constituting the sub-cache according to the space size of the sub-cache; A cache module, used for caching the data in the data set to multiple cache areas. In the process of caching the data in the data set to multiple cache areas, if the number of cache lines of cached data in a target cache area reaches a cache line number threshold of the target cache area, stop caching the data in the data set to the target cache area, serialize the cached data in the target cache area to a disk to obtain a temporary file, and release the space of the target cache area; if there is still residual data in the data set that has not been written to the cache area, cache the remaining data in the data set to the target cache area; A generating module is used to generate a spreadsheet document corresponding to the data set based on the temporary file.

9. A computing device, characterized in that including memory and processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

Images