File splitting methods, devices, electronic equipment, and storage media

Abstract

This invention discloses a file splitting method, apparatus, electronic device, and storage medium. The method includes: upon receiving a splitting request, determining splitting task information corresponding to each source file to be split based on the splitting request; the splitting task information includes file information of the source files to be split and splitting rule information corresponding to the file information; determining a splitting thread to split each of the source files to be split based on the file information; for any splitting thread, obtaining the current source file to be split and the splitting rule corresponding to the current source file to be split based on the file information and the splitting rule information, and splitting the source file to be split based on the splitting rule to obtain multiple split sub-files. Through the technical solution disclosed in this invention, file splitting processing efficiency is improved by processing files simultaneously using multiple threads.

Description

Technical Field

[0001] This invention relates to the field of document processing technology, and in particular to a document splitting method, apparatus, electronic device, and storage medium. Background Technology

[0002] In related technologies, as business systems continue to improve, the data generated by these systems is output in the form of PDF files. Automated batch splitting of PDF documents has become a new requirement for electronic document management.

[0003] However, current document splitting tools suffer from low efficiency and cannot rename documents when automatically splitting a single document into multiple documents, thus reducing document processing efficiency. Summary of the Invention

[0004] This invention provides a file splitting method, apparatus, electronic device, and storage medium to solve the problems of inability to rename files and low efficiency in document splitting when automatically splitting a single document into multiple documents in the prior art. It improves the processing efficiency of file splitting by processing multiple files simultaneously using multiple threads, and directly renames the split sub-files based on preset naming rules, thus solving the problem that split sub-files cannot be renamed.

[0005] In a first aspect, embodiments of the present invention provide a file splitting method, the method comprising:

[0006] Upon receiving a splitting request, the splitting task information corresponding to each source file to be split is determined based on the splitting request; the splitting task information includes file information of the source file to be split and splitting rule information corresponding to the file information.

[0007] Based on the file information, a splitting thread is determined for each of the source files to be split;

[0008] For any splitting thread, the current source file to be split and the corresponding splitting rule are obtained based on the file information and the splitting rule information. The source file to be split is then split based on the splitting rule to obtain multiple renamed split sub-files.

[0009] Optionally, the splitting task information corresponding to each source file to be split in the splitting request determination includes:

[0010] Based on the splitting request, the server that generates each of the source files to be split is determined, and the server is grouped to obtain multiple server groups;

[0011] Based on each of the aforementioned server groups, the splitting task information corresponding to each source file to be split is obtained.

[0012] Optionally, determining the splitting thread for splitting each of the source files to be split based on the file information includes:

[0013] For any split task information, if the current split task information's occupancy meets a preset threshold, obtain a preset thread pool;

[0014] Based on the file information in the current splitting task information and the thread pool, a splitting thread is determined to split the source file to be split corresponding to the current splitting task information.

[0015] Optionally, the file information includes the server address of the server that generated the current source file to be split;

[0016] Before obtaining the current source file to be split and the corresponding splitting rule based on the file information and the splitting rule information, the method further includes:

[0017] The current source file to be split is obtained based on the server address, and the current source file to be split is stored in a preset external storage medium;

[0018] Based on the media information of the external storage medium, a storage message for the current source file to be split is generated, and the storage message is broadcast to the splitting thread.

[0019] Optionally, obtaining the current source file to be split and the corresponding splitting rule based on the file information and the splitting rule information includes:

[0020] Upon receiving the storage message, the storage message is parsed to obtain the storage address of the current source file to be split, and the current source file to be split is read based on the storage address;

[0021] Based on the file information of the current source file to be split and the splitting rules, obtain the splitting rules corresponding to the current source file to be split.

[0022] Optionally, the splitting rule information includes naming rules;

[0023] The source file to be split is split based on the splitting rules to obtain multiple renamed split sub-files, including:

[0024] Call the preset splitting interface, and split the source file to be split based on the splitting interface and the splitting rules to obtain multiple split sub-files corresponding to the source file to be split;

[0025] Based on the naming rules, each of the split sub-files is renamed to obtain the renamed split sub-files.

[0026] Optionally, after obtaining multiple split sub-files, the method further includes:

[0027] Obtain the splitting status table of the source file to be split;

[0028] The split status table is updated based on the splitting information and storage information of each of the split sub-files, and the updated split status table is stored.

[0029] Secondly, embodiments of the present invention also provide a file splitting device, the device comprising:

[0030] The task splitting information acquisition module is used to determine, upon receiving a splitting request, the splitting task information corresponding to each source file to be split in the splitting request; the splitting task information includes file information of the source file to be split and splitting rule information corresponding to the file information.

[0031] The splitting thread determination module is used to determine the splitting thread for splitting each of the source files to be split based on the file information.

[0032] The splitting processing module is used to, for any splitting thread, obtain the current source file to be split and the splitting rule corresponding to the current source file to be split based on the file information and the splitting rule information, and split the source file to be split based on the splitting rule to obtain multiple renamed split sub-files.

[0033] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0034] At least one processor; and

[0035] A memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the file splitting method according to any embodiment of the present invention.

[0037] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute the file splitting method described in any embodiment of the present invention.

[0038] The technical solution provided by this invention, upon receiving a splitting request, determines splitting task information corresponding to each source file to be split based on the splitting request. The splitting task information includes file information of the source files to be split and splitting rule information corresponding to the file information. A splitting thread is determined for each of the source files to be split based on the file information. For any splitting thread, the current source file to be split and its corresponding splitting rule are obtained based on the file information and the splitting rule information, and the source file to be split is split based on the splitting rule to obtain multiple sub-files. This technical solution improves splitting efficiency by having multiple splitting threads in a splitting device connected to a data system simultaneously split multiple source files to obtain multiple corresponding sub-files.

[0039] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of a file splitting method provided in Embodiment 1 of the present invention;

[0042] Figure 2 This is a flowchart of a file splitting method provided according to Embodiment 2 of the present invention;

[0043] Figure 3 This is a schematic diagram of a file splitting device according to Embodiment 3 of the present invention;

[0044] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the file splitting method of this invention. Detailed Implementation

[0045] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0047] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0048] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0049] Example 1

[0050] Figure 1 The flowchart of a file splitting method is provided for Embodiment 1 of the present invention. This embodiment can be applied to the processing of source files output by business systems.

[0051] In practical applications, some data systems generate source files in PDF format, containing various data results. Due to the inherent structure of the data system, it does not support file splitting and renaming based on the data results, resulting in source files that are unfavorable for subsequent data processing. Therefore, existing technologies use a splitting method to split the source file after it is output by the data system, obtaining multiple sub-files. However, current technologies can only split source files individually, not simultaneously split multiple source files generated by the data system, resulting in low splitting efficiency and the inability to automatically rename the resulting sub-files. To solve these technical problems, the present invention provides a file splitting method. Specifically, multiple splitting threads in a splitting device connected to the data system simultaneously split multiple source files, obtaining corresponding sub-files, thus improving splitting efficiency. Furthermore, the sub-files are directly renamed based on preset naming rules, solving the problem of the inability to rename the split sub-files.

[0052] Specifically, this method can be executed by a file splitting device, which can be implemented in hardware and / or software, and can be configured in smart terminals and cloud servers. For example... Figure 1 As shown, the method includes:

[0053] S110. Upon receiving a split request, determine the split task information corresponding to each source file to be split based on the split request.

[0054] In this embodiment of the invention, the data system can be understood as a server cluster for data processing. Specifically, the data system includes multiple sub-servers, each generating different source files. Specifically, each sub-server generates a splitting rule corresponding to each source file, as well as a naming rule for the split sub-files resulting from the splitting based on these rules. Specifically, the splitting rule can be understood as the page number range of the different data results contained in the source file. When the data system detects the source files generated by each sub-system and their corresponding splitting rules, it generates a splitting request for each source file based on the file information. Further, the splitting request is sent to a file splitting system pre-associated with the data system to split the source files output by the data system. Further, the split sub-files are automatically renamed based on the naming rules to obtain renamed split sub-files.

[0055] In practical applications, a splitting request is generated based on the file information of each source file, and the splitting request is sent in the form of an HTTP request. Optionally, it is determined whether the HTTP request was sent successfully. If the sending fails, an error log is logged and the request is terminated; otherwise, if it is successful, the source files are split based on the file splitting system pre-associated with the data system.

[0056] Specifically, when the file splitting system receives a splitting request from the data system, it sends an information retrieval request to the data system based on the splitting request, and obtains the splitting task information corresponding to each source file based on the feedback. The splitting task information includes the file information of the source files to be split and the splitting rule information corresponding to the source files.

[0057] Optionally, the method for determining the splitting task information corresponding to each source file to be split in this embodiment based on the splitting request may include: determining the server that generates each source file to be split based on the splitting request, and grouping each server to obtain multiple server groups; and obtaining the splitting task information corresponding to each source file to be split based on each server group.

[0058] Specifically, since the splitting request contains file information for each source file, and this file information includes the server identifier of the source file's server, the server of the source file is determined based on the server identifier, i.e., the sub-server that generated the source file in the data system is identified. The servers are then grouped based on their server identifiers to obtain server groups. Optionally, for any server group, a set of information retrieval requests is generated based on the server identifiers of each server in the current group, and these requests are sent to the data system to obtain the splitting task information corresponding to the source files generated by each server in the current group.

[0059] In practical applications, when a file splitting system receives a splitting request, it parses the request to obtain the source file name and the server identifier of the source file server. Based on the server identifier, the source file servers are grouped to obtain server groups. Then, based on the grouping information, splitting rule information is generated for each source file generated by each server in the current server group. Optionally, splitting rule information corresponding to each server group is obtained separately based on the grouping information. Thus, the splitting task information for each source file is determined based on the file information of the source files generated by each server and the corresponding splitting rule information. The file information may include the source file name, file storage path, etc.; the splitting rule information includes the file name corresponding to the splitting rule, the page number parameter for splitting, and the storage information of the splitting rule, etc.

[0060] It should be noted that the effect of the file splitting system generating an information retrieval request based on the server group and obtaining splitting task information is that when the splitting request includes splitting requests for multiple source files, multiple splitting task information can be obtained based on a single SFTP request, reducing the number of request interactions and thus improving file splitting efficiency.

[0061] S120. Based on the information of each file, determine the thread to split each source file to be split.

[0062] Having obtained the splitting task information corresponding to each source file, it is necessary to determine the splitting thread for each source file to be split, so as to split each source file simultaneously and thus improve the splitting efficiency.

[0063] Optionally, the method for determining the splitting thread in this embodiment may include: for any splitting task information, if the current splitting task information occupancy meets a preset threshold, obtaining a preset thread pool; and determining the thread to split the source file to be split corresponding to the current splitting task information based on the file information in the current splitting task information and the thread pool.

[0064] The amount of memory occupied by the splitting task information can be understood as the memory occupied by the splitting task information of the source file in the current storage medium. Optionally, it can be determined whether this amount of information is greater than a preset threshold. If so, it means that there is a splitting task in the splitting task information, and the file information of the source file to be split can be obtained, and the splitting thread for processing the source file can be determined based on the file information to split the source file; otherwise, it means that the file was not successfully obtained, and the source file cannot be split.

[0065] In practical applications, it is determined whether the length of the splitting task information is greater than 0. If the length is greater than 0, it means that the file splitting task can continue to be executed based on the splitting task information, that is, the splitting task is distributed using a thread pool strategy; otherwise, the splitting task is terminated.

[0066] In this embodiment, the file splitting system pre-configures a thread pool containing multiple threads for executing file splitting tasks for each source file. Specifically, distributing splitting tasks based on a thread pool strategy can be understood as a matching strategy within the thread pool that matches the splitting threads corresponding to the source files. Optionally, the matching strategy can be based on the file data volume of each source file and the processing level of the threads, or it can be based on the processing time limit of each source file and the processing progress of the threads. Of course, it can also be based on other matching strategies, which are not limited in this embodiment.

[0067] Based on the above implementation, the technical solution of this embodiment modifies the task status corresponding to the splitting task information of the source file when determining the splitting thread corresponding to the source file, so as to prevent other threads from matching the source file corresponding to the splitting task information again, avoiding the problem of repeated thread allocation, thereby avoiding the waste of splitting resources and reduced splitting efficiency caused by repeatedly splitting the same source file.

[0068] S130. For any splitting thread, obtain the current source file to be split and the corresponding splitting rule based on the file information and splitting rule information, and split the source file to be split based on the splitting rule to obtain multiple renamed split sub-files.

[0069] In this embodiment of the invention, in order to enable the file splitting system to simultaneously supply multiple threads to split multiple source files, based on matching the splitting thread corresponding to each source file, the source files stored in the data system are obtained based on each splitting thread, and each source file is stored in an external storage medium. When performing splitting processing, the downloaded source files are directly retrieved for processing. The source files are deleted immediately after splitting to solve the disk space occupation problem of the file splitting system, thereby improving the splitting efficiency of the file splitting system.

[0070] Optionally, in this embodiment, the method for obtaining source files based on the splitting thread and storing the source files externally may include: obtaining file information including the system address of the generation system that generates the current source file to be split; determining the generation system that generates the current source file to be split based on the system address, and reading the current source file to be split from the generation system based on the file information; storing the current source file to be split to a preset external storage medium, generating a storage message for the current source file to be split based on the medium information of the external storage medium, and broadcasting the storage message to the splitting thread.

[0071] In practical applications, the splitting thread obtains the server address of the server that generated the source file to be split and logs into that server using the SFTP-related API in the open-source program JSCH. It then checks if the SFTP login was successful. Optionally, if the SFTP login fails, it tries to log in again. If the SFTP login is successful, it further checks whether the server has finished generating the source file, i.e., whether a complete source file can be obtained. If the source file does not exist, the splitting ends; if the source file exists, it is obtained as a file stream. The source file is stored in a preset external storage medium and named according to preset rules. When the source file is successfully stored, the relevant information about the file storage path is written to RocketMQ, and this storage message is broadcast to the splitting thread. Specifically, the storage message of the source file is broadcast to the splitting message topic of the splitting thread.

[0072] Optionally, for any splitting thread, before processing the source file, it is necessary to first obtain the source file and the corresponding splitting rule. Optionally, the method for obtaining this information in this embodiment may include: upon receiving a storage message, parsing the storage message to obtain the storage address of the current source file to be split, and reading the current source file to be split based on the storage address; obtaining the splitting rule corresponding to the current source file to be split based on the file information and splitting rule information of the current source file to be split.

[0073] In practical applications, the splitting thread also includes an MQ message listener. After the splitting thread broadcasts a RocketMQ message, the MQ message listener listens to the splitting message topic in the splitting thread and determines whether the MQ tag corresponding to the message meets preset requirements based on the listened message. If the requirements are not met, the message listener ends the processing of the current message and continues listening. If the requirements are met, the message body in the MQ is parsed, and the storage path of the external storage medium is obtained based on the parsing result. The source file stored in the external storage medium is then read based on the storage path, and the corresponding splitting rule is determined based on the file name of the source file.

[0074] Based on this, the source file is split. Optionally, the splitting process in this embodiment may include: calling a preset splitting interface, and splitting the source file to be split based on the splitting interface and splitting rules to obtain multiple sub-files corresponding to the source file to be split.

[0075] In practical applications, we obtain an open-source API and use that API to split the source file according to the splitting rules to obtain multiple sub-files.

[0076] Optionally, in the process of splitting the source file, the technical solution of this embodiment further includes: renaming each of the split sub-files according to the naming rules to obtain the renamed split sub-files.

[0077] In this embodiment, the splitting task information also includes naming rule information for the sub-files obtained after splitting the source file based on the splitting rules. Specifically, while obtaining the splitting rules, the naming rules corresponding to the sub-files are obtained based on the naming rule information. Further, when obtaining the sub-files, the naming rule corresponding to the splitting rule of the current sub-file is determined, and the obtained current sub-file is renamed based on the naming rule to obtain the renamed sub-files. Optionally, when the splitting of each source file is completed, the renamed sub-files are obtained simultaneously.

[0078] The technical solution provided in this invention, upon receiving a splitting request, determines the splitting task information corresponding to each source file to be split based on the splitting request. The splitting task information includes file information of the source files to be split and corresponding splitting rule information. A splitting thread is determined for each source file to be split based on the file information. For any splitting thread, the current source file to be split and its corresponding splitting rule are obtained based on the file information and splitting rule information. The source file to be split is then split based on the splitting rule to obtain multiple sub-files. This technical solution improves splitting efficiency by using multiple splitting threads in a splitting device connected to a data system to simultaneously split multiple source files, obtaining multiple corresponding sub-files.

[0079] Example 2

[0080] Figure 2 This is a flowchart of a file splitting method provided in Embodiment 2 of the present invention. Based on the above embodiments, after obtaining multiple split sub-files, the method further includes:

[0081] Get the split status table of the source file to be split;

[0082] The split status table is updated based on each split sub-file, and the updated split status table and each even-numbered split sub-file are stored. For example... Figure 2 As shown, the method includes:

[0083] S210. Upon receiving a split request, determine the split task information corresponding to each source file to be split based on the split request.

[0084] S220. Based on the information of each file, determine the splitting thread for splitting each source file to be split.

[0085] S230. For any splitting thread, obtain the current source file to be split and the corresponding splitting rule based on the file information and splitting rule information, and split the source file to be split based on the splitting rule to obtain multiple splitting sub-files.

[0086] S240. Obtain the splitting status table of the source file to be split; update the splitting status table based on the splitting information and storage information of each of the split sub-files, and store the updated splitting status table.

[0087] In this embodiment of the invention, the split status table is used to indicate the number of files in the current source file and the corresponding page number. In other words, the split status table can indicate whether the source file is a single, unsplit file or multiple split sub-files.

[0088] Specifically, when splitting and renaming a source file to obtain multiple renamed sub-files, each sub-file is stored in relation to the original file. Optionally, the sub-files can be stored in relation to the original file on a preset external storage medium, and the sub-files in the file splitting system can be deleted to reduce the memory usage of the file splitting system.

[0089] Optionally, if the file is successfully stored, the split status table is updated. Specifically, the split status table is updated based on the splitting rules, naming rules, and storage information of each split sub-file, so that other systems can verify the file data based on the splitting rules, naming rules, and storage information.

[0090] The above technical solution uses multiple splitting threads in the splitting device connected to the data system to simultaneously split multiple source files, obtaining multiple corresponding split sub-files, and storing the multiple split sub-files and source files on an external storage medium, thereby reducing the space occupied by the file splitting system and further improving the splitting efficiency.

[0091] Example 3

[0092] Figure 3 This is a schematic diagram of a file splitting device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a task splitting information acquisition module 310, a task splitting thread determination module 320, and a task splitting processing module 330; wherein,

[0093] The task splitting information acquisition module 310 is used to determine, upon receiving a splitting request, the splitting task information corresponding to each source file to be split in the splitting request; the splitting task information includes file information of the source file to be split and splitting rule information corresponding to the file information.

[0094] The splitting thread determination module 320 is used to determine the splitting thread for splitting each of the source files to be split based on the file information.

[0095] The splitting processing module 330 is used to, for any splitting thread, obtain the current source file to be split and the splitting rule corresponding to the current source file to be split based on the file information and the splitting rule information, and split the source file to be split based on the splitting rule to obtain multiple renamed split sub-files.

[0096] Based on the above embodiments, optionally, the task information acquisition module 210 is split, including:

[0097] The server group obtaining unit is used to determine the server that generates each of the source files to be split based on the splitting request, and to group each of the servers to obtain multiple server groups.

[0098] The task splitting information acquisition unit is used to obtain the task splitting information corresponding to each source file to be split based on each of the server groups.

[0099] Based on the above embodiments, optionally, the thread splitting determination module 320 includes:

[0100] The thread pool acquisition unit is used to acquire a preset thread pool for any split task information, provided that the current split task information's occupancy meets a preset threshold.

[0101] The splitting thread determination unit is used to determine the splitting thread for splitting the source file to be split corresponding to the current splitting task information based on the file information in the current splitting task information and the thread pool.

[0102] Based on the above embodiments, optionally, the file information includes the server address of the server that generated the current source file to be split;

[0103] The device also includes:

[0104] The source file to be split module is used to obtain the current source file to be split in the generation system based on the server address before obtaining the current source file to be split based on the file information and the splitting rule information, and to store the current source file to be split to a preset external storage medium.

[0105] The storage message broadcasting module is used to generate a storage message for the current source file to be split based on the media information of the external storage medium, and broadcast the storage message to the splitting thread.

[0106] Based on the above embodiments, optionally, the split processing module 330 includes:

[0107] The source file splitting acquisition unit is used to, upon receiving the storage message, parse the storage message to obtain the storage address of the current source file to be split, and read the current source file to be split based on the storage address;

[0108] The splitting rule acquisition unit is used to acquire the splitting rule corresponding to the current source file to be split based on the file information of the current source file to be split and the splitting rule.

[0109] Based on the above embodiments, optionally, the splitting rule information includes naming rules;

[0110] The split processing module 330 includes:

[0111] The sub-file splitting unit is used to call a preset splitting interface and split the source file to be split based on the splitting interface and the splitting rules to obtain multiple sub-files corresponding to the source file to be split.

[0112] The file renaming unit is used to rename each of the split sub-files according to the naming rules to obtain the renamed split sub-files.

[0113] Optionally, based on the above embodiments, the device may further include:

[0114] The split status table acquisition module is used to acquire the split status table of the source file to be split;

[0115] The file storage module is used to update the split status table based on the splitting information and storage information of each of the split sub-files, and to store the updated split status table.

[0116] The file splitting device provided in the embodiments of the present invention can execute the file splitting method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0117] Example 4

[0118] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0119] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0120] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0121] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as file splitting methods.

[0122] In some embodiments, the file splitting method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the file splitting method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the file splitting method by any other suitable means (e.g., by means of firmware).

[0123] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0125] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0127] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0128] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0129] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0130] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A file splitting method, characterized in that, include: Upon receiving a splitting request, the splitting task information corresponding to each source file to be split is determined based on the splitting request; The splitting task information includes file information of the source file to be split and splitting rule information corresponding to the file information; Based on the file information, a splitting thread is determined for each of the source files to be split; For any splitting thread, the current source file to be split and the corresponding splitting rule are obtained based on the file information and the splitting rule information. The source file to be split is then split based on the splitting rule to obtain multiple renamed split sub-files. The file information includes the server address of the server that generated the current source file to be split; Before obtaining the current source file to be split and the corresponding splitting rule based on the file information and the splitting rule information, the method further includes: The current source file to be split is obtained based on the server address, and the current source file to be split is stored in a preset external storage medium; Based on the media information of the external storage medium, a storage message for the current source file to be split is generated, and the storage message is broadcast to the splitting thread; The step of obtaining the current source file to be split and the corresponding splitting rule based on the file information and the splitting rule information includes: Upon receiving the storage message, the storage message is parsed to obtain the storage address of the current source file to be split, and the current source file to be split is read based on the storage address; Based on the file information of the current source file to be split and the splitting rule information, the splitting rule corresponding to the current source file to be split is obtained.

2. The method according to claim 1, characterized in that, The splitting task information corresponding to each source file to be split in the splitting request determination includes: Based on the splitting request, the server that generates each of the source files to be split is determined, and the server is grouped to obtain multiple server groups; Based on each of the aforementioned server groups, the splitting task information corresponding to each source file to be split is obtained.

3. The method according to claim 1, characterized in that, The step of determining the splitting thread for splitting each of the source files to be split based on the file information includes: For any split task information, if the current split task information's occupancy meets a preset threshold, obtain a preset thread pool; Based on the file information in the current splitting task information and the thread pool, a splitting thread is determined to split the source file to be split corresponding to the current splitting task information.

4. The method according to claim 1, characterized in that, The splitting rule information includes naming rules; The source file to be split is split based on the splitting rules to obtain multiple renamed split sub-files, including: Call the preset splitting interface, and split the source file to be split based on the splitting interface and the splitting rules to obtain multiple split sub-files corresponding to the source file to be split; Based on the naming rules, each of the split sub-files is renamed to obtain the renamed split sub-files.

5. The method according to claim 1, characterized in that, After obtaining multiple split sub-files, the method further includes: Obtain the splitting status table of the source file to be split; The split status table is updated based on the splitting information and storage information of each of the split sub-files, and the updated split status table is stored.

6. A file splitting device, characterized in that, include: The task splitting information acquisition module is used to determine the splitting task information corresponding to each source file to be split based on the splitting request when a splitting request is received. The splitting task information includes file information of the source file to be split and splitting rule information corresponding to the file information; The splitting thread determination module is used to determine the splitting thread for splitting each of the source files to be split based on the file information. The splitting processing module is used to obtain the current source file to be split and the splitting rule corresponding to the current source file to be split based on the file information and the splitting rule information for any splitting thread, and to split the source file to be split based on the splitting rule to obtain multiple renamed split sub-files; The file information includes the server address of the server that generated the current source file to be split; The device further includes: The source file to be split module is used to obtain the current source file to be split based on the server address and store the current source file to be split to a preset external storage medium before obtaining the current source file to be split and the splitting rule corresponding to the current source file to be split based on the file information and the splitting rule information. The storage message broadcasting module is used to generate a storage message for the current source file to be split based on the media information of the external storage medium, and broadcast the storage message to the splitting thread; The splitting processing module includes: The source file splitting acquisition unit is used to, upon receiving the storage message, parse the storage message to obtain the storage address of the current source file to be split, and read the current source file to be split based on the storage address; The splitting rule acquisition unit is used to acquire the splitting rule corresponding to the current source file to be split based on the file information of the current source file to be split and the splitting rule information.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the file splitting method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the file splitting method according to any one of claims 1-5.

Images