File uploading method and file uploading system

Abstract

The embodiment of the invention provides a file uploading method and system, and the method comprises the steps: obtaining a set uploading bandwidth in response to a file uploading instruction of a to-be-uploaded target file, and reading a target file block from the target file according to the set uploading bandwidth; determining a target storage service corresponding to the target node, and determining an uploading size limit of the target storage service; and according to the size of the currently read target file block, setting an uploading bandwidth and an uploading size limit, and uploading the target file block to the target storage service through at least one thread. On the basis of streaming uploading, multi-thread fragmentation uploading is adopted, the uploading speed of the target file is increased, the corresponding target file can be read from a remote storage service in time, judgment and analysis are conducted, and the timeliness of the target file is guaranteed.

Description

Technical Field

[0001] The embodiments of this specification relate to the technical field of file transfer, and in particular, to a file upload method and a file upload system. Background Art

[0002] With the continuous development of computer technology and Internet technology, the applications of various nodes are becoming more and more common. There are more and more service programs installed in the nodes, and the provided service functions are also becoming more and more abundant. The protection mechanism of the node operating system or the protection mechanism of the hardware may cause the running service program to terminate abnormally, generating a core dump file. Core dump refers to that when a service program in a node makes an error and abnormally interrupts, the operating system will store the current state of the service program's work as a core dump file. Usually, the core dump file contains the memory, register state, stack pointer, memory management information, etc. when the service program is running.

[0003] In the prior art, the core dump file can be uploaded to a remote storage service. Specifically, often, memory corresponding to the size of the core dump file is applied for, and the core dump file is directly uploaded to the remote storage service, and the upload speed is slow, affecting the timeliness of the core dump file. Summary of the Invention

[0004] In view of this, the embodiments of this specification provide a file upload method. One or more embodiments of this specification also relate to a file upload device, a file upload system, a computing device, a computer-readable storage medium, and a computer program product at the same time to solve the technical defects existing in the prior art.

[0005] According to the first aspect of the embodiments of this specification, a file upload method is provided, which is applied to a target node and includes:

[0006] In response to a file upload instruction of a target file to be uploaded, obtain a set upload bandwidth, and read a target file block from the target file according to the set upload bandwidth;

[0007] Determine the target storage service corresponding to the target node, and determine the upload size limit of the target storage service;

[0008] According to the size of the currently read target file block, the set upload bandwidth, and the upload size limit, upload the target file block to the target storage service through at least one thread.

[0009] According to a second aspect of the embodiments of the present specification, a file upload system is provided, including at least one node and a storage service corresponding to each node. Any one of the at least one node is a target node, and the storage service corresponding to the target node is a target storage service;

[0010] The target node is configured to, in response to a file upload instruction of a target file to be uploaded, obtain a set upload bandwidth, and read a target file block from the target file according to the set upload bandwidth; determine the target storage service corresponding to the target node, and determine the upload size limit of the target storage service; and upload the target file block to the target storage service through at least one thread according to the size of the currently read target file block, the set upload bandwidth, and the upload size limit;

[0011] The target storage service is configured to receive the target file block, and splice the received target file blocks to obtain a corresponding target file when the upload of the target file is completed.

[0012] According to a third aspect of the embodiments of the present specification, a file upload device is provided, which is applied to a target node and includes:

[0013] A reading module, configured to, in response to a file upload instruction of a target file to be uploaded, obtain a set upload bandwidth, and read a target file block from the target file according to the set upload bandwidth;

[0014] A determining module, configured to determine the target storage service corresponding to the target node, and determine the upload size limit of the target storage service;

[0015] An uploading module, configured to upload the target file block to the target storage service through at least one thread according to the size of the currently read target file block, the set upload bandwidth, and the upload size limit.

[0016] According to a fourth aspect of the embodiments of the present specification, a computing device is provided, including:

[0017] A memory and a processor;

[0018] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above file upload method are implemented.

[0019] According to a fifth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, which stores computer-executable instructions. When the instructions are executed by a processor, the steps of the above file upload method are implemented.

[0020] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions, which implement the steps of the above file upload method when executed by a processor.

[0021] The embodiments of this specification provide a file upload method. In response to a file upload instruction of a target file to be uploaded, a set upload bandwidth is obtained, and according to the set upload bandwidth, a target file block is read from the target file; a target storage service corresponding to the target node is determined, and an upload size limit of the target storage service is determined; according to the size of the currently read target file block, the set upload bandwidth, and the upload size limit, the target file block is uploaded to the target storage service through at least one thread.

[0022] In one embodiment of this specification, it is realized that target file blocks are sequentially read from a target file according to a set upload bandwidth, and the target file is uploaded to a target storage service corresponding to a target node by using a streaming upload method. On the basis of using the streaming upload method, for each target file block, the target file block can also be uploaded to the target storage service through at least one thread based on the size of the target file block, the set upload bandwidth, and the upload size limit. On the basis of the streaming upload, multi-threaded upload is adopted, which greatly improves the upload speed of the target file, enables business personnel to timely read the corresponding target file from the remote storage service for judgment and analysis, and ensures the timeliness of the target file. Description of the Drawings

[0023] Figure 1 is a flowchart of a file upload method provided by an embodiment of this specification;

[0024] Figure 2 is an architecture diagram of a file upload system provided by an embodiment of this specification;

[0025] Figure 3 is a schematic diagram of the processing process of a file upload method provided by an embodiment of this specification;

[0026] Figure 4 is a schematic structural diagram of a file upload device provided by an embodiment of this specification;

[0027] Figure 5 is a structural block diagram of a computing device provided by an embodiment of this specification. Detailed Embodiments

[0028] In the following description, numerous specific details are set forth to provide a thorough understanding of the present specification. However, the present specification can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the connotation of the present specification. Therefore, the present specification is not limited by the specific implementations disclosed below.

[0029] The terms used in one or more embodiments of the present specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present specification. The singular forms "a" and "the" used in one or more embodiments of the present specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of the present specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0031] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of the present specification are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for the user to select to authorize or refuse.

[0032] First, the noun terms involved in one or more embodiments of the present specification are explained.

[0033] Core: The program terminates abnormally, usually due to serious errors in the program or memory access violations.

[0034] Core Dump: It is the program memory image generated when a program terminates abnormally. When the service program encounters an error and terminates abnormally, the operating system will store the current state of the service program as a core dump file. Usually, the core dump file contains the memory, register status, stack pointer, memory management information, and other diagnostic information of the service program during runtime. Core dump is very useful for debugging and analyzing the cause of program crashes, especially for troubleshooting problems that are difficult to reproduce in the production environment.

[0035] GDB (GNU Debugger): It is a powerful open-source debugger used to inspect and modify the internal state of a program while it is running. It supports multiple programming languages, including C, C++, Java, Python, etc. GDB allows developers to execute code line by line, view variable values, set breakpoints, step through, trace function calls, etc., thus helping developers find errors or abnormal behaviors in the program.

[0036] Prometheus: It is an open-source monitoring system and time series database that provides an efficient, multi-dimensional data model, supports a powerful query language (PromQL), and can be easily integrated into various environments for collecting, storing, and querying metric data.

[0037] COS (Cloud Object Storage), OSS (Object Storage Service): Cloud object storage services provided by different cloud service providers. Different cloud service providers may have different naming conventions. Cloud object storage service is a service used to store a large amount of unstructured data (such as pictures, videos, log files, etc.). It provides high availability, durability, and security, and is suitable for various application scenarios, such as website hosting, big data analysis, backup and recovery, etc.

[0038] CosFS: It is a tool for cloud object storage service that allows users to mount an object storage service (Cloud Object Storage, COS) as a local file system. With CosFS, users can operate on objects in COS like operating on ordinary files locally, which greatly facilitates data upload, download, and management.

[0039] OssFs: It is a file system based on FUSE (Filesystem in Userspace) and a tool provided by the Object Storage Service (OSS). It allows the OSS storage space to be mounted as a local file system on the Linux system, which means that users can manage the data on OSS just like operating local files without additional data migration or complex configuration, greatly facilitating data upload, download, and management.

[0040] Kubernetes (abbreviated as k8s): It is an open-source container orchestration platform used for automating the deployment, scaling, and management of containerized applications. It provides a declarative configuration method, enabling developers to define the desired state of the application, and Kubernetes will automatically adjust the actual state to match the desired state.

[0041] Sidecar metrics: They refer to the performance metrics related to the Sidecar containers in Kubernetes. The Sidecar container is a design pattern used to run an auxiliary container beside the main application container to provide additional functions or services. These auxiliary containers are usually called "sidecar" because they are located beside the main application container.

[0042] Pod: In the k8s cluster, it is the most basic scheduling unit that encapsulates one or more closely related containers, sharing storage and network resources.

[0043] Node: A node in the k8s cluster, also called a host.

[0044] Reddog: The alarm unit of the content application platform.

[0045] It should be noted that after the service process Core, a Coredump file is required for GDB analysis and location to perform quick location and repair. In various business scenarios, especially in c++ service scenarios such as search and wide promotion, Coredump files may be required for location and repair. For example, in the case of a Core scenario in service gray release, Risk-Free interception is required to control the fault domain and perform location and repair; when the code has a Core during the CI (Continuous Integration) stage test, a Coredump file is required for location and repair, etc.

[0046] Service gray release (also known as canary release or gray deployment) is a software release strategy that deploys a new version to a small group of users for testing before full-scale promotion. This approach helps the team identify potential problems early, reduce risks, and ensure that new features or improvements can run smoothly in the production environment. "Risk-Free" refers to a series of measures taken in software development and operation to ensure that implementing an operation or change will not have a negative impact on the existing system. In particular, achieving "risk-free" is a very important goal during service gray release. When encountering Core scenarios during service gray release, some measures need to be taken to ensure the stability and reliability of the system, and at the same time, quickly locate and solve problems.

[0047] In this specification, a file upload method is provided. This specification also relates to a file upload system, a file upload device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail one by one in the following embodiments.

[0048] See Figure 1 , Figure 1 which shows a flowchart of a file upload method according to an embodiment of this specification, applied to a target node, and specifically includes the following steps.

[0049] Step 102: In response to a file upload instruction for a target file to be uploaded, obtain a set upload bandwidth, and read a target file block from the target file according to the set upload bandwidth.

[0050] It should be noted that the target node is any node that provides content services. The content application platform can provide content services using the Kubernetes (abbreviated as k8s) architecture. Under this architecture, multiple nodes can be included, and any node can be used as the target node to execute the upload of the target file.

[0051] Among them, the target file to be uploaded refers to a file that needs to be uploaded to the corresponding target storage service. For example, in the case of a service instance exception scenario, that is, when the program providing a certain service example crashes or abnormally terminates, the target file is the generated core dump file. Of course, in other scenarios, the target file can also be other files that need to be uploaded to the corresponding target storage service. This specification does not limit this, and for example, the target file can also be the service running log file of the target node, etc.

[0052] In addition, the file upload instruction refers to the instruction for the target node to upload the target file. This file upload instruction can be triggered by the target node itself or by the corresponding upload control terminal.

[0053] The set upload bandwidth is the bandwidth for uploading the target file in a pre-configured streaming upload manner, that is, the maximum limit of the target file block for a single upload. This set upload bandwidth can be configured based on other service programs running on the target node. Generally, this set upload bandwidth is set to be relatively small to avoid affecting the normal operation of other services on the target node when uploading the target file. That is to say, this set upload bandwidth is a safety water level for the target node and can be pre-configured through testing or experience.

[0054] In addition, the target file block with the set upload bandwidth size continues to be fragmented. The obtained file fragments have sizes that do not trigger the upload size limit of the target storage service. That is to say, the target file block reaching the set upload bandwidth size can continue to be fragmented and uploaded using multiple threads. Therefore, when configuring the set upload bandwidth, the upload size limit of the target storage service, the number of threads for multi-threaded transmission, etc. can also be considered. For example, it can generally be configured as 1GB or 2GB, etc.

[0055] In actual implementation, when the target node determines that it has detected a file upload instruction for the target file, it means that the target file needs to be uploaded to the target storage service corresponding to the target node. At this time, in response to the file upload instruction of the target file to be uploaded, the set upload bandwidth can be obtained, and according to the set upload bandwidth, the target file block is read from the target file, one target file block is read at a time, and the target file is uploaded to the corresponding target storage service in a streaming upload manner.

[0056] In an optional implementation manner of this embodiment, the target file is a core dump file generated when an exception occurs in the target service instance in the target node; the method further includes:

[0057] When it is monitored that an exception occurs in the target service instance, a corresponding core dump file is generated;

[0058] The exception information of the target service instance is sent to the reporting control end. Among them, the exception information of the target service instance is used to instruct the reporting control end to determine whether to report the core dump file of the target service instance based on the configured exception reporting policy, and a reporting control instruction is returned to the target node;

[0059] The reporting control instruction returned by the reporting control end is received. If the reporting control instruction indicates reporting, it is determined that a file upload instruction has been detected, where the file upload instruction indicates uploading the core dump file to the target storage service.

[0060] It should be noted that a service program can be deployed in the target node. This service program can be an online C++ service, and the Coredump function is supported by default after being deployed on k8s through ones.

[0061] In actual implementation, an exception information collector can be deployed in the target node. Taking the Core scenario as an example, a Coredump Client is deployed in the target node. File generation parameters of the core dump file can be configured in the Coredump Client to control writing the generated core dump file into the Coredump Client. Moreover, policy rules for streaming and multi-threaded uploading can also be configured in the Coredump Client. For example, target file blocks are sequentially read based on the set upload bandwidth. For each target file block, fragmentation and multi-threaded uploading are performed on the target file block based on the size of the target file block, the set upload bandwidth, and the upload size limit of the corresponding target storage service (deployed in the same region as the target node). The Coredump Client in the target node uploads the core dump file to the corresponding target storage service based on the configured policy rules for streaming and multi-threaded uploading.

[0062] Among them, configuring the file generation parameters of the core dump file in the Coredump Client can include file generation location, signal, host name, process name, timestamp, etc.

[0063] In one implementation, the file generation location can be specified through core_pattern. core_pattern is a kernel parameter used to specify the name and location of the core dump file generated when the program crashes. When set to a string starting with "|", the subsequent content will be regarded as a command that receives the content of the core dump file as standard input. For example, setting core_pattern to "| / data / coredumpclient", then when the program crashes, the content of the core dump file will be passed to the program " / data / coredumpclient" for processing.

[0064] In addition, the limit parameter of the core dump file can also be configured through core_pipe_limit. core_pipe_limit is a kernel parameter used to limit the number of concurrent crashing programs that are simultaneously passed to the specified core information collector through the pipe mode. If this limit is exceeded, subsequent programs will not be processed. For example, setting core_pipe_limit to 1 means that at most one concurrent crashing program can be passed to the specified core information collector through the pipe mode.

[0065] It should be noted that, taking the scenario of service instance exception as an example, the target file is the core dump file generated when the target service instance in the target node has an exception. The reporting control end can control whether the core dump file needs to be uploaded to the target storage service based on the configured exception reporting policy.

[0066] In actual implementation, when the target node detects that the target service instance has an exception, it generates a corresponding core dump file, and then sends the exception information of the target service instance to the reporting control end. The exception information can include the service instance information, environment variables, etc. of the target service instance, such as the instance identifier, exception time, host identifier, process identifier, etc. of the target service instance. The reporting control end can analyze and judge the received exception information of the target service instance based on the configured exception reporting policy, determine whether to report the core dump file of the target service instance, and return a reporting control instruction to the target node.

[0067] In one implementation, the exception reporting policy can be configured as a frequency control policy, that is, to control the reporting frequency of the core dump files of the same service instance. The specific exception reporting policy can be that the exception reporting frequency of the same service instance is lower than the frequency threshold within a set time period. At this time, the reporting control end can query the historical exception reporting information of the target service instance based on the exception information of the target service instance, and determine whether the current reporting frequency of the target service instance is lower than the frequency threshold according to the historical exception reporting information. If so, it returns a first reporting control instruction to the target node, and the first reporting control instruction instructs to report the core dump file of the target service instance; if not, it returns a second reporting control instruction to the target node, and the second reporting control instruction instructs not to report the core dump file of the target service instance.

[0068] The target node receives the reporting control instruction returned by the reporting control end. If the reporting control instruction needs to indicate reporting, it determines that a file upload instruction is detected, and uploads the generated core dump file as the target file to be uploaded to the target storage service.

[0069] Of course, in actual implementation, the exception reporting policy can also be configured as other policies based on actual business requirements. For example, for some key or high-frequency services, all core dump files are reported, but for non-key or non-high-frequency services, they are reported at intervals, etc. The embodiments of this specification do not limit this.

[0070] In the embodiments of this specification, when the target service instance in the target node has an exception, the target node can generate a corresponding core dump file, and send the exception information of the target service instance to the reporting control end. The reporting control end controls whether the currently generated core dump file is reported, avoiding reporting all core dump files generated by each exception, reducing the processing pressure, and saving processing resources.

[0071] In another implementation, the file upload instruction can also be triggered by the target node itself. For example, every time a core dump file is generated, the file upload instruction is triggered, and the generated core dump file is used as the target file and uploaded to the corresponding target storage service. Or the target node determines whether the currently generated core dump file needs to be reported based on other configured reporting policies, and when the condition for reporting is met, the file upload instruction is triggered.

[0072] Step 104: Determine the target storage service corresponding to the target node, and determine the upload size limit of the target storage service.

[0073] It should be noted that different nodes can correspond to different storage services, and the storage service can be a remote object storage that can provide data storage. For example, the storage service can be a cloud object storage service provided by cloud service providers such as COS and OSS.

[0074] In actual implementation, different storage services have different upload size limits. The upload size limit refers to the size limit of the upload shards for the storage service, which can be the minimum size limit or the size limit range. For example, for COS, the maximum size of a single uploaded object is 48.82TB, the size of a single data shard is 1MB - 5GB, the last data shard can be less than 1MB, and the number of shards is 1 - 10000; for OSS, except that the last data shard has no size limit, the minimum size of other data shards is 100KB. Each uploaded data shard has a shard number, and the value range is 1 to 10000. If it exceeds this range, OSS will return an error code of InvalidArgument.

[0075] In specific implementation, when the target node needs to upload the target file to the corresponding target storage service, it can first determine the upload size limit of the target storage service corresponding to the target node, which is convenient for subsequent multi-threaded uploading of the target file blocks read in chunks based on the upload size limit.

[0076] In an optional implementation manner of this embodiment, determining the target storage service corresponding to the target node includes:

[0077] Determine the deployment area of the target node;

[0078] Determine the storage service corresponding to the deployment area as the target storage service.

[0079] It should be noted that different nodes may be deployed in different areas, and corresponding storage services are deployed in different areas. Therefore, the deployment area of the target node can be determined, and the storage service deployed in this deployment area is determined as the target storage service. That is to say, the target node can upload the target file to the storage service in the same area.

[0080] Exemplarily, it is assumed that there are 4 nodes. Node 1 is located in Area A, Node 2 is located in Area B, Node 3 is located in Area C, and Node 4 is located in Area D. Storage service OSS1 is deployed in Area A, storage service OSS2 is deployed in Area B, storage service COS1 is deployed in Area C, and storage service COS2 is deployed in Area D. For Node 1, the corresponding target storage service is OSS1, and Node 1 uploads the target file to OSS1; for Node 2, the corresponding target storage is OSS2, and Node 2 uploads the target file to OSS2; for Node 3, the corresponding target storage service is COS1, and Node 3 can upload the target file to COS1; for Node 4, the corresponding target storage service is COS2, and Node 4 can upload the target file to COS2.

[0081] In the embodiments of this specification, the target node can determine the target storage service deployed in the same area as the target node based on the deployment area of the target node, and upload the target file to the target storage service deployed in the same area, reducing the cross-cloud bandwidth caused by cross-cloud transmission.

[0082] Step 106: According to the size of the currently read target file block, the set upload bandwidth, and the upload size limit, upload the target file block to the target storage service through at least one thread.

[0083] It should be noted that since the size of the buffer of the target node is fixed, if a large amount of data is attempted to be read at one time, it may cause insufficient memory of the target node, which may in turn cause the program of the target node to crash or fail to run properly. Moreover, when reading a large amount of data at one time, the operating system needs to allocate sufficient memory to store this data, which may also cause the program of the target node to run slowly. In addition, processing a large amount of data may occupy a large amount of CPU resources, resulting in a slower execution speed of other services. Therefore, the target node can use a streaming processing method to read and process the target file block by block. The size of each block read from the target file can be controlled based on the set upload bandwidth. For example, each file block is set to 2GB.

[0084] For the streaming and multi-threaded sharded upload method, the total number of shards and the shard size cannot be predicted, and it is easy to trigger the upload size limit of the target storage service (OSS / COS) in a multi-threaded scenario, resulting in file upload failure.

[0085] Therefore, in actual implementation, the target node can upload the target file block to the target storage service through at least one thread according to the size of the currently read target file block, the set upload bandwidth, and the upload size limit. That is to say, based on the size of the currently read target file block, the set upload bandwidth, and the upload size limit, it can be determined whether to continue to fragment the currently read target file block and upload it through multiple threads.

[0086] In specific implementation, based on the set upload bandwidth and the upload size limit, it can be determined whether the size of the currently read target file block triggers the upload size limit of the target storage service; if it triggers, the currently read target file block will not be fragmented further, and the target file block will be directly uploaded to the target storage service as a data fragment; if it does not trigger, the target file block will be divided into a set number of file fragments, and the set number of file fragments will be uploaded to the target storage service through a set number of threads respectively, that is, continue to fragment the target file block and use multiple threads to upload.

[0087] In an optional implementation manner of this embodiment, uploading the target file block to the target storage service through at least one thread according to the size of the currently read target file block, the set upload bandwidth, and the upload size limit includes:

[0088] Determine whether the size of the target file block reaches the set upload bandwidth;

[0089] If it reaches the set upload bandwidth, the target file block will be divided into a set number of file fragments, and the set number of file fragments will be uploaded to the target storage service through a set number of threads respectively;

[0090] If it does not reach the set upload bandwidth, the target file block will be uploaded to the target storage service through at least one thread according to the size of the target file block and the upload size limit.

[0091] Among them, the set number is the number of threads for continuing to fragment and uploading through multiple threads for the currently read target file block in the pre-configured streaming transmission mode. For example, the set number can be configured as 10, 50, 100, etc.

[0092] In actual implementation, it can be determined whether the size of the currently read target file block reaches the set upload bandwidth. If it reaches the set upload bandwidth, it means that the size of the currently read target file block is the maximum file block size that can be uploaded under the set upload bandwidth. The currently read target file block may not be the last file block. Even if it is the last file block shard, multi-threaded transmission will not trigger the upload size limit. At this time, the target file block can continue to be divided into a set number of file shards, and the set number of file shards can be uploaded to the target storage service through a set number of threads, with one thread uploading one file shard. If it does not reach the set upload bandwidth, it means that the size of the currently read target file block is relatively small. If the currently read target file block is not the last file block, continuing to fragment the currently read target file block may trigger the upload size limit of the target storage service. Therefore, in combination with the upload size limit, it is further determined whether to continue fragmenting the currently read target file block and adopt multi-threaded upload.

[0093] For example, the set upload bandwidth is 2GB, the target file is 401GB, and the target node reads 2GB of data each time as the target file block. For each read target file block, if the size of the currently read target file block is 2GB, the target file block can continue to be divided into 10 file shards, and the 10 file shards can be uploaded to the target storage service through 10 threads, with one thread uploading one file shard. If the size of the currently read target file block is less than 2GB, further fragmentation judgment is made based on the upload size limit (such as 0.8GB).

[0094] In the embodiments of this specification, for each target file block read in a streaming manner, since the total number of shards and the shard size cannot be predicted, the target node cannot determine whether the currently read target file block is the last file block of the streaming transmission, and thus cannot determine whether further multi-threaded transmission of this target file block will trigger the corresponding upload size limit. Therefore, for each target file block read in a streaming manner, it can be determined whether to continue fragmenting and adopt multi-threaded transmission based on whether the size of the target file block reaches the set upload bandwidth, avoiding triggering the upload size limit of the target storage service and ensuring the upload success rate of the target file.

[0095] In an optional implementation manner of this embodiment, uploading the target file block to the target storage service through at least one thread according to the size of the target file block and the upload size limit includes:

[0096] Determine whether the size of the target file block is less than the upload size limit;

[0097] If the size of the target file block is less than the upload size limit, upload the target file block to the target storage service through one thread;

[0098] If the size of the target file block is not less than the upload size limit, the target file block is divided into a set number of file shards, and according to the size of the file shards and the upload size limit, the target file block is uploaded to the target storage service through at least one thread.

[0099] In actual implementation, if the size of the target file block does not reach the set upload bandwidth, it can be further determined whether the size of the currently read target file block is less than the upload size limit. If it is less, it means that the currently read target file block is likely to be the last file block, which is already less than the upload size limit and the data is small, so there is no need to continue sharding. Therefore, at this time, the currently read target file block can be directly used as a complete data shard and uploaded to the target storage service through one thread. If it is not less, the target file block can be divided into the corresponding number of file shards based on the number of threads configured for multi-threading.

[0100] It should be noted that the currently read target file block is not less than the upload size limit, but the file shards obtained after sharding may be less than the upload size limit. Therefore, after obtaining the file shards, it can be further determined whether the target file block is uploaded as a whole or uploaded in multiple threads according to the size of the file shards and the upload size limit.

[0101] In the embodiments of this specification, when it is determined that the size of the read target file block is not less than the upload size limit, the target file block is further divided into multiple file shards. After the division, instead of directly uploading the multiple file shards using multi-threading, it is further determined whether the size of the file shards triggers the upload size limit, so as to determine the upload method of the read target file block, further avoiding triggering the upload size limit of the target storage service and ensuring the upload success rate of the target file.

[0102] In an optional implementation manner of this embodiment, uploading the target file block to the target storage service through at least one thread according to the size of the file shards and the upload size limit includes:

[0103] Determine whether the size of the file shard is less than the upload size limit;

[0104] If the size of the file shard is less than the upload size limit, upload the target file block to the target storage service through one thread;

[0105] If the size of the file shard is not less than the upload size limit, upload the set number of file shards to the target storage service through the set number of threads respectively.

[0106] In actual implementation, if the size of the file shard is smaller than the upload size limit, it means that when the target file block is sliced, the obtained file shard may trigger the corresponding upload size limit. Therefore, instead of uploading the target file block in slices, the target file block is uploaded as a single data shard. If the size of the file shard is not smaller than the upload size limit, it means that when the target file block is sliced, the obtained file shard will not trigger the corresponding upload size limit. To ensure the file upload speed, a set number of threads can be used to upload a set number of file shards to the target storage service respectively for multi-threaded parallel upload.

[0107] For example, if the currently read target file block (buffer) reaches the set limit of 2GB, the target file block (buffer) is further sliced and uploaded in slices using multiple threads. If the currently read target file block (buffer) does not reach the set limit of 2GB but is smaller than the minimum size limit of OSS, the target file block (buffer) is directly uploaded as a single data shard to the OSS. If the currently read target file block (buffer) is not smaller than the minimum size limit of OSS, the target file block (buffer) is further sliced. If the sliced file shard is smaller than the minimum size limit of OSS, the target file block (buffer) is directly uploaded as a single data shard to the OSS; if the sliced file shard is not smaller than the minimum size limit of OSS, it is uploaded in slices using multiple threads.

[0108] In the embodiments of this specification, when the size of the file shard is not smaller than the upload size limit, multiple file shards of the target file block are uploaded in parallel using multiple threads, which not only avoids triggering the upload size limit of the target storage service but also improves the file upload speed.

[0109] In actual implementation, the above operation steps of reading the target file block from the target file according to the set upload bandwidth can be repeated until the target file is completely uploaded.

[0110] It should be noted that after uploading the currently read target file block to the target storage service, the above operation steps of reading the target file block from the target file according to the set upload bandwidth, and then uploading the target file block to the target storage service through at least one thread according to the size of the currently read target file block, the set upload bandwidth, and the upload size limit of the target storage service can be repeated until the target file is completely read and uploaded to the target storage service. Using streaming and multi-threaded slice transmission, assuming the controlled upload bandwidth is below 5Gb / s, a 300G target file can be uploaded within 10 minutes.

[0111] Next, through the following three examples, the complete process of uploading the target file to the target storage service will be illustrated by way of example:

[0112] As the first example, set the upload bandwidth to 2GB, the upload size limit of the target storage service to 1.2GB, the target file to 401GB, and the target node reads 2GB of data each time as a target file block. For the first 20 target file blocks, each is 2GB, and the method of further dividing the 2GB file block into 10 file shards for multi-threaded transmission is adopted. For the 21st target file block which is 1GB, less than the set upload bandwidth of 2GB, it is further determined that the target file block (1GB) is less than the upload size limit of 1.2GB. At this time, the 1GB file block is uploaded as a whole (without further dividing it into multiple file shards).

[0113] As the second example: set the upload bandwidth to 2GB, the upload size limit of the target storage service to 0.2GB, the target file to 401GB, and the target node reads 2GB of data each time as a target file block. For the first 20 target file blocks, each is 2GB, and the method of further dividing the 2GB file block into 10 file shards for multi-threaded transmission is adopted. For the 21st target file block which is 1GB, less than the set upload bandwidth of 2GB, it is further determined that the target file block (1GB) is greater than the upload size limit of 0.2GB. At this time, the 1GB file block is divided into 10 file shards, each file shard is 0.1GB, less than the upload size limit of 0.2GB. At this time, the 1GB file block is uploaded as a whole (without further dividing it into multiple file shards).

[0114] As the third example: set the upload bandwidth to 2GB, the upload size limit of the target storage service to 0.05GB, the target file to 401GB, and the target node reads 2GB of data each time as a target file block. For the first 20 target file blocks, each is 2GB, and the method of further dividing the 2GB file block into 10 file shards for multi-threaded transmission is adopted. For the 21st target file block which is 1GB, less than the set upload bandwidth of 2GB, it is further determined that the target file block (1GB) is greater than the upload size limit of 0.05GB. At this time, the 1GB file block is divided into 10 file shards, each file shard is 0.1GB, greater than the upload size limit of 0.05GB. At this time, the 1GB file block is divided into 10 file shards for multi-threaded upload.

[0115] The embodiments of this specification provide a file upload method, which realizes reading target file blocks from a target file in sequence according to a set upload bandwidth, and uses a streaming upload method to upload the target file to the target storage service corresponding to the target node. On the basis of using the streaming upload method, for each target file block, it is also possible to upload the target file block to the target storage service through at least one thread based on the size of the target file block, the set upload bandwidth, and the upload size limit. On the basis of streaming upload, multi-threaded upload is adopted, which greatly improves the upload speed of the target file, provides a stable, efficient, and secure file upload method. For the Core scenario, it can greatly improve the upload efficiency of the core dump file, quickly complete the upload of the core dump file, and does not generate additional cross-cloud bandwidth, ensuring that business personnel can timely read the core dump file from the remote storage service for GDB analysis and timely locate and fix anomalies.

[0116] See Figure 2 , Figure 2 FIG. shows an architecture diagram of a file upload system provided according to an embodiment of this specification. As Figure 2 shown, the file upload system includes at least one node 202 and the storage service 204 corresponding to each node. Any one of the at least one node 202 is the target node 2022, and the storage service 204 corresponding to the target node 2022 is the target storage service 2042;

[0117] The target node 2022 is configured to, in response to a file upload instruction of a target file to be uploaded, obtain a set upload bandwidth, and read a target file block from the target file according to the set upload bandwidth; determine the target storage service 2042 corresponding to the target node 2022, and determine the upload size limit of the target storage service 2042; upload the target file block to the target storage service 2042 through at least one thread according to the size of the currently read target file block, the set upload bandwidth, and the upload size limit;

[0118] The target storage service 2042 is configured to receive the target file block and splice the received target file blocks to obtain the corresponding target file when the target file upload is completed.

[0119] It should be noted that any node in the file upload system can be used as the target node. The target node can use the streaming and multi-threaded sharding upload method to upload the target file to the corresponding target storage service. When the target file upload is completed, the target storage service splices the received target file blocks to obtain the corresponding target file.

[0120] In actual implementation, the specific process of the target node using the streaming and multi-threaded sharding upload method to upload the target file to the corresponding target storage service can be referred to the aboveFigure 1 For the description of the illustrated embodiments, the embodiments of this specification will not be elaborated herein.

[0121] In an alternative embodiment of this embodiment, the target file is a core dump file generated when an exception occurs in the target service instance in the target node; as Figure 2 shown, the file upload system further includes a reporting control terminal 206;

[0122] The target node 2022 is further configured to generate a corresponding core dump file when it detects that an exception occurs in the target service instance; send the exception information of the target service instance to the reporting control terminal 206;

[0123] The reporting control terminal 206 is configured to receive the exception information of the target service instance, determine whether the core dump file of the target service instance needs to be reported based on the configured exception reporting policy and the exception information, and return a reporting control instruction to the target node 2022;

[0124] The target node 2022 is further configured to receive the reporting control instruction returned by the reporting control terminal 206. If the reporting control instruction indicates reporting, it determines that a file upload instruction is detected, where the file upload instruction indicates uploading the core dump file to the target storage service 2042.

[0125] It should be noted that taking the scenario of service instance exception as an example, where the target file is a core dump file generated when an exception occurs in the target service instance in the target node, the reporting control terminal can control whether the core dump file needs to be uploaded to the target storage service based on the configured exception reporting policy.

[0126] In the embodiments of this specification, the target node can generate a corresponding core dump file when an exception occurs in the target service instance, and report the exception information of the target service instance to the reporting control terminal. The reporting control terminal controls whether the currently generated core dump file needs to be reported, avoiding reporting each core dump file generated by an exception, reducing the processing pressure, and saving processing resources.

[0127] In an alternative embodiment of this embodiment, the exception reporting policy is that the exception reporting frequency of the same service instance is lower than the frequency threshold within a set duration; the reporting control terminal 206 is further configured to:

[0128] Query the historical exception reporting information of the target service instance based on the exception information of the target service instance;

[0129] Determine whether the current reporting frequency of the target service instance is lower than the frequency threshold according to the historical exception reporting information;

[0130] If so, return a first reporting control instruction to the target node, where the first reporting control instruction instructs to report the core dump file of the target service instance;

[0131] If not, return a second reporting control instruction to the target node, where the second reporting control instruction instructs not to report the core dump file of the target service instance.

[0132] Among them, the frequency threshold is a predefined limit value for the abnormal reporting frequency of the same service instance within a set duration. For example, the frequency threshold can be 1, 3, 10, etc.

[0133] It should be noted that the abnormalities of the same service instance within a set duration are often caused by the same reasons. Therefore, within the set duration, the abnormal reporting frequency of the same service instance can be controlled, that is, the abnormal reporting policy can be configured as a frequency control policy.

[0134] For the same service instance, within a set duration, report some core dump files instead of reporting each core dump file to the corresponding target storage service. For example, within half an hour, the same service instance only uploads the first core dump file, or within half an hour, the same service instance only uploads the first n core dump files, where n can be freely configured; or within half an hour, for the same service instance, upload core dump files at intervals, that is, upload the first core dump file, skip m core dump files without uploading, and upload the (m + 1)-th core dump file, where m can be freely configured.

[0135] In actual implementation, the abnormal reporting policy can be configured such that the abnormal reporting frequency of the same service instance within a set duration is lower than the frequency threshold. At this time, the reporting control end can query the historical abnormal reporting information of the target service instance based on the abnormal information of the target service instance, and determine whether the current reporting frequency of the target service instance is lower than the frequency threshold according to the historical abnormal reporting information. If so, return a first reporting control instruction to the target node, instructing the target node to report the core dump file of the target service instance; if not, return a second reporting control instruction to the target node, instructing the target node not to report the core dump file of the target service instance.

[0136] For the case of not reporting the core dump file, the target node can directly restart the service program corresponding to the target service instance to quickly restore the corresponding service function.

[0137] In addition, if the reporting control end determines that the current abnormality of the target service instance needs to be reported, it can write the current abnormality of the target service instance into the database for storage and mark that the abnormal file has been reported; if the reporting control end determines that the current abnormal information of the target service instance does not need to be reported, it can be written into the database and marked as not reported, or it can not be written into the database.

[0138] Exemplarily, assume that the preset duration is half an hour and the frequency threshold is 1, that is, within half an hour, only 1 core dump file is uploaded for the same service instance. When the target node detects that service instance 1 is abnormal and generates the corresponding core dump file 1, it can send the abnormal information of service instance 1 to the reporting control end. The reporting control end queries from the database that service instance 1 has not reported an abnormal file within half an hour, and returns the first reporting control instruction (true) to the target node. The target node uploads the currently generated core dump file 1 to the corresponding target storage service. Assume that after 10 minutes, the target node detects that service instance 1 is abnormal again and generates the corresponding core dump file 2. It can send the abnormal information of service instance 1 to the reporting control end. The reporting control end queries from the database that service instance 1 has already reported an abnormal file within half an hour, and returns the second reporting control instruction (false) to the target node. The target node does not upload core dump file 2 and directly restarts service instance 1.

[0139] In the embodiments of this specification, when the target service instance is abnormal, the target node can generate the corresponding core dump file and send the abnormal information of the target service instance to the reporting control end. The reporting control end controls whether the currently generated core dump file is reported to the target storage service, controls the abnormal reporting frequency of the core dump file of the same service instance within the set duration, retains a small proportion of the core dump files generated by the same service instance within the set duration and uploads them to the corresponding target storage service in a small proportion, and directly restarts the business process for other service instances to quickly stop losses, and avoids reporting all core dump files generated by each exception, reducing the processing pressure and saving processing resources.

[0140] Of course, in actual implementation, the exception reporting policy can also be configured as other policies based on actual business requirements. For example, for some key or high-frequency services, all core dump files are reported, but for non-key or non-high-frequency services, they are reported at intervals, etc. The embodiments of this specification do not limit this.

[0141] In an optional implementation manner of this embodiment, as Figure 2 shown, the file upload system further includes a monitoring end 208;

[0142] The reporting control end 206 is further configured to report the received abnormal information of the target service instance to the monitoring end 208;

[0143] The monitoring end 208 is further configured to obtain the service alarm policy configured by the content application platform and alarm the target service instance when the abnormal information of the target service instance meets the service alarm policy.

[0144] Among them, a monitoring terminal can also be configured in the file upload system, and the monitoring terminal can be an open-source monitoring system, such as Prometheus, Datadog, Zabbix, etc.

[0145] It should be noted that multiple PODs can be deployed in the target node to store the relevant data during the operation of the program corresponding to the storage service instance. The monitoring terminal needs to monitor whether the service instances in each node are abnormal and give an alarm in time, so that the business personnel can locate the cause of the abnormality and repair the abnormality in time. And the occurrence of Core in the service instance is a low-probability event. If the monitoring terminal regularly pulls all sidecar metrics (performance metrics related to the Sidecar container in Kubernetes) at a set interval (such as 15s) for service anomaly monitoring, it will waste overhead. Therefore, the push method can be used for logging.

[0146] In actual implementation, after receiving the exception information of the target service instance, the reporting control terminal can also report the received exception information of the target service instance to the monitoring terminal. For example, information such as the service instance name, time, host name, process name, and download link can be pushed to the control terminal. If the control terminal is Prometheus, it can be pushed to the pushgateway of Prometheus.

[0147] In addition, business personnel can also pre-configure the service alarm policy of the content application platform through the alarm unit 212 in the content application platform. The alarm unit is an alarm component configured for the content application platform. For example, the alarm unit can be reddog. The monitoring terminal can access the service alarm policy configured by the alarm unit 212 in the content application platform, determine whether the target service instance with an abnormality currently occurring meets the service alarm policy. If it meets, an alarm is given to the target service instance, and the alarm information is pushed to the corresponding business personnel through the alarm unit of the content application platform, so that the corresponding business personnel can download the core dump file generated by the abnormality through the business terminal for abnormality location analysis and repair.

[0148] In specific implementation, the exception information of the target service instance sent by the target node to the reporting control terminal and the exception information of the target service instance passed by the reporting control terminal to the monitoring terminal can be the same or different. That is, the reporting control terminal can also delete or add parameters related to the target service instance based on the exception information of the target service instance sent by the target node to generate an exception monitoring request and pass it to the monitoring terminal. This specification does not limit this.

[0149] In the embodiments of this specification, the reporting control end may push the exception information of the target service instance sent by the target node to the monitoring end, and the monitoring end determines whether the service alarm policy configured by the content application platform is met. In the case of meeting the policy, an alarm is triggered. By using the push method for service exception monitoring and alerting, business personnel can locate and repair the corresponding service exceptions, saving costs and improving the robustness of the service programs provided by the nodes.

[0150] In an alternative embodiment of this example, as Figure 2 shown, the file upload system further includes the service end 210 of the content application platform; the service end 210 is configured to:

[0151] Access the target storage service 2042 to download the core dump file of the target service instance;

[0152] Access the content application platform to obtain the core dump file location strategy;

[0153] Restore and debug the core dump file according to the core dump file location strategy to locate the cause of the exception of the target service instance.

[0154] Among them, the file upload system further includes the service end of the content application platform. The service end is a device used by business personnel for debugging and repairing the programs corresponding to the service instances provided by the content application platform.

[0155] It should be noted that business personnel can use the service end to download the core dump file of the target service instance, but business personnel may not be clear about the process of restoring and debugging the core dump file for analysis. Therefore, business personnel can also access the content application platform through the service end to obtain the location strategy of the core dump file. This location strategy can indicate the operation steps for restoring and debugging the core dump file. Business personnel can perform the restoration and debugging of the core dump file according to this location strategy at the service end to locate the cause of the exception of the target service instance.

[0156] In actual implementation, the cause of the exception can be located and analyzed for the downloaded core dump file through GDB at the service end, so as to quickly repair the exception.

[0157] As an example, the location strategy of the core dump file can be Coredump-workstation, that is, the workstation stack restoration process. The specific operation steps can include: entering the corresponding partition of the workstation through the jump server; then, starting the mirror version consistent with the online environment in the workstation, and mounting the corresponding directory to docker when starting; after that, entering the docker container for corresponding stack restoration, and deleting the tested container after the test is completed.

[0158] In the embodiments of this specification, the service end can download the core dump file of the target service instance, perform GDB analysis, quickly locate the cause of the exception of the service instance with the exception, and thus quickly modify it, ensuring that the service instance can quickly resume normal operation.

[0159] In an alternative implementation of this embodiment, as Figure 2 shown, the service end 210 is further configured to:

[0160] Access the alarm unit 212 of the content application platform to obtain the exception information of the target service instance;

[0161] Generate a signed download link by invoking the signature interface based on the exception information;

[0162] Access the target storage service based on the download link to download the core dump file of the target service instance.

[0163] In one implementation, the service end can download the core dump file through the download link.

[0164] It should be noted that the target storage service is private for reading and writing. The signature interface can be invoked to generate a signed download link, and the core dump file can be downloaded to the machine of the service end using the download link, and then GDB analysis is performed to locate the cause of the exception. In actual implementation, the exception information (core information) of the target service instance can be obtained first through the alarm unit (such as reddog) of the content application platform. Example: Core information in the hz region; the signature interface is invoked to generate a signed download link, and the exception information (core information) in different regions is executed on the machines in different regions, that is, the service end can download the core dump file of the target service instance from the target storage service in the current region. The core dump file of the target service instance is downloaded to the machine of the service end through this download link.

[0165] Among them, the step field in the exception information can indicate the upload status of the core dump file. For example, if the step field is "uploaded", it means that the core dump file has been uploaded successfully; if the step field is "start", it means that the core dump file starts to be uploaded; if the step field is "deny", it means that the core dump file triggers the frequency control policy and is not uploaded.

[0166] It should be noted that the validity period of the signed download link can also be defined, such as 9 hours. The service end can access this download link within the validity period to download the corresponding core dump file.

[0167] As an example, the signed download link can be: "https: / / qnj-xhs-coredump-1251524319.cos.ap-shanghai.myqcloud.com / searchlambdaservice / core.searchlambdaservice-search-goodn ote-firstrank-v3-worker-0-whvzx.2024-08-13_10:49:00.LambdaServer.qcnj2".

[0168] In the embodiments of this specification, the service end can access the alarm unit of the content application platform to obtain the exception information of the target service instance, and then call the signature interface based on the exception information to generate a signed download link. Furthermore, it can access the target storage service deployed in the same region to download the core dump file of the target service instance, which not only avoids cross-cloud bandwidth, but also can access the target storage service with private read and write permissions by generating a signed download link through calling the signature interface, realizing that the service end can successfully and quickly download the core dump file of the target service instance.

[0169] In an optional implementation manner of this embodiment, the service end 210 is further configured to:

[0170] Through the access tool of the target storage service, access the storage space of the target storage service in the local file system and download the core dump file of the target service instance stored in the target storage service.

[0171] In another implementation manner, it is also possible to directly use the access tool of the target storage service to directly access the storage space of the target storage service in the local file system of the service end and download the core dump file of the target service instance stored in the target storage service.

[0172] In actual implementation, if the target storage service is COS, CosFS can be used to mount COS as the local file system, and users can operate on the objects in COS at the service end locally as if operating on ordinary files to realize the download of the core dump file. If the target storage service is OSS, OssFs can be used to mount the OSS storage space as the local file system, and users can manage the files on OSS at the service end as if operating on local files to realize the download of the core dump file.

[0173] It should be noted that accessing the storage space of the target storage service in the local file system through the access tool of the target storage service has read-only permission and can only download files from the storage space of the target storage service. If write permission is required, the files in the storage space of the target storage service need to be copied to the data / directory of the local file system.

[0174] In the embodiments of the present specification, the access tool of the target storage service can be used to directly operate files locally at the service end, so as to realize the download of the core dump file, without the need for additional data migration or complex configuration, which improves the download speed of the core dump file. Moreover, the embodiments of the present specification provide two download methods, and the service end can flexibly select the download method based on the actual business situation, so as to efficiently obtain the required core dump file, timely locate the abnormal reason of the service instance, and thus quickly modify it, ensuring that the service instance can be quickly restored to normal operation.

[0175] In an optional implementation manner of this embodiment, the service end 210 is further configured to:

[0176] Access the storage service deployed in other regions to download the reference file of the target service instance;

[0177] According to the core dump file positioning strategy, restore and debug the core dump file and the reference file to locate the abnormal reason of the target service instance.

[0178] It should be noted that when the same service instance has an abnormality in different regions, the corresponding core dump file is generated and reported to the storage service in the same region. Since the reasons for the abnormality of the same service instance in different regions are often the same, the service end can access the target storage service deployed in the same region to download the core dump file of the target service instance, so as to realize the positioning of the abnormal reason of the target service instance.

[0179] In actual implementation, if the abnormal reason of the target service instance cannot be located, or the abnormal reason of the target service instance is inaccurately located, the reference file of the target service instance can also be downloaded from the storage service deployed in other regions. The reference file is the core dump file stored by the storage service deployed in other regions for the target service instance. The reference file is used as the comparison information to locate the abnormal reason of the target service instance.

[0180] In the embodiments of the present specification, the reference file of the target service instance can also be downloaded by accessing the storage service deployed in other regions. The reference file is used as the comparison information for restoration and debugging to locate the abnormal reason of the target service instance, improving the accuracy of abnormal reason positioning.

[0181] In an optional implementation manner of this embodiment, the target storage service 2042 is further configured to:

[0182] Determine the block identification of the target file block according to the size of the current target file block, the set upload bandwidth, and the upload size limit of the target storage service, and update the block identification list according to the block identification of the target file block;

[0183] When the upload of the target file is completed, read the list of chunk identifiers, sort the chunk identifiers in the list of chunk identifiers, and splice the received target file chunks according to the sorting result to obtain the corresponding target file.

[0184] In actual implementation, the target node uploads each target file chunk of the target file to the target storage service in a streaming and multi-threaded sharding upload manner. Therefore, the target storage service needs to splice the received multiple target file chunks to obtain the complete target file for storage. Since it is based on the size of the target file chunk, the set upload bandwidth, and the upload size limit, and the target file chunks are uploaded to the target storage service through at least one thread, splicing can be performed based on the size of the target file chunk, the set upload bandwidth, and the upload size limit.

[0185] Specifically, a list of chunk identifiers can be initialized. This list of chunk identifiers is used to store the shard identifiers of each target file chunk. The initialized list of chunk identifiers is empty. For each received target file chunk, a corresponding chunk identifier can be generated based on the size of the target file chunk, the set upload bandwidth, and the upload size limit of the target storage service, and added to the list of chunk identifiers.

[0186] In a multi-threaded state, the target file is divided into multiple target file chunks, and the target file chunks may also be divided into multiple file shards for multi-threaded upload. Therefore, the order in which the target file chunks generate the corresponding chunk identifiers and enter the list of chunk identifiers is also random. Therefore, when the upload of the target file is completed, the list of chunk identifiers can be read, the chunk identifiers in the list of chunk identifiers can be sorted, and the received target file chunks can be spliced according to the sorting result to obtain the corresponding target file.

[0187] In the embodiments of this specification, the shard identifiers of each target file chunk can be stored through the list of chunk identifiers, and the target file chunks transmitted in a streaming and multi-threaded sharding manner can be spliced into a complete target file, ensuring that the target file can be accurately spliced to obtain, and further ensuring the integrity and accuracy of the target file.

[0188] In an optional implementation manner of this embodiment, the target storage service 2042 is further configured to:

[0189] If the size of the target file chunk reaches the set upload bandwidth, or the size of the target file chunk is not less than the upload size limit and the size of the file shard after the target file chunk is sharded is not less than the upload size limit, then determine the chunk identifier of the target file chunk based on the current number of threads, the total number of threads, and the current number of file chunks;

[0190] If the size of the target file block is less than the upload size limit, or the size of the target file block is not less than the upload size limit and the size of the file shards after the target file block is fragmented is less than the upload size limit, determine the block identifier of the target file block according to the total number of threads and the current number of file blocks.

[0191] In actual implementation, if the size of the target file block reaches the set upload bandwidth, it means that the target file block is further fragmented into multiple file shards and uploaded using multiple threads; if the size of the target file block is not less than the upload size limit and the size of the file shards after the target file block is fragmented is not less than the upload size limit, it means that the target file block is also fragmented into multiple file shards and uploaded using multiple threads. Among them, the number of file shards is the total number of threads. Therefore, the block identifier of the target file block can be determined based on the current number of threads, the total number of threads, and the current number of file blocks. This block identifier includes the shard identifiers of each file shard in the target file block.

[0192] Specifically, the shard identifier of each file shard in the target file block is: the number of threads of the current file shard + the total number of threads * (the number of received target file blocks - 1).

[0193] In addition, if the size of the target file block is less than the upload size limit, or the size of the target file block is not less than the upload size limit and the size of the file shards after the target file block is fragmented is less than the upload size limit, it means that the currently received target file block is uploaded as a data shard. At this time, the block identifier of the target file block can be determined according to the total number of threads and the current number of file blocks. This block identifier includes a shard identifier corresponding to the entire target file block. Specifically, the block identifier of the target file block is: the total number of threads * (the number of received target file blocks - 1) + 1. It should be noted that if the size of the first received target file block is less than the upload size limit, the shard identifier of the target file block is determined to be 1, and the target file only includes this 1 target file block.

[0194] Exemplarily, the block identifier list is initially empty. Assume that the size of the first received target file block is less than the upload size limit. Then this target file block is not fragmented, the total number of threads is 1, and the target file includes this 1 file block. Directly add the shard identifier "1" to the block identifier list to identify the first target file block, and the first target file block is the obtained target file.

[0195] In another example, the chunk identification list is initially empty. Suppose the size of the first received target file chunk reaches the set upload bandwidth, and this target file chunk is split into 10 file shards and uploaded through 10 threads. For the file shard uploaded by the first thread, its shard identification is: the number of the current file shard's thread + the total number of threads * (the number of the currently received target file chunks - 1) = 1 + 10 * (1 - 1) = 1. At this time, the shard identification "1" can be added to the chunk identification list to identify the file shard uploaded by the first thread in the first target file chunk; for the file shard uploaded by the second thread, its shard identification is: the number of the current file shard's thread + the total number of threads * (the number of the currently received target file chunks - 1) = 2 + 10 * (1 - 1) = 1. At this time, the shard identification "2" can be added to the chunk identification list to identify the file shard uploaded by the second thread in the first target file chunk;...; for the file shard uploaded by the tenth thread, its shard identification is: the number of the current file shard's thread + the total number of threads * (the number of the currently received target file chunks - 1) = 10 + 10 * (1 - 1) = 10. At this time, the shard identification "10" can be added to the chunk identification list to identify the file shard uploaded by the tenth thread in the first target file chunk.

[0196] Suppose the second target file chunk is received. Its size does not reach the set upload bandwidth, and its size is not less than the upload size limit, and the size of the file shards after splitting this target file chunk is not less than the upload size limit. This target file chunk is split into 10 file shards and uploaded through 10 threads. For the file shard uploaded by the first thread, its shard identification is: the number of the current file shard's thread + the total number of threads * (the number of the currently received target file chunks - 1) = 1 + 10 * (2 - 1) = 11. At this time, the shard identification "11" can be added to the chunk identification list to identify the file shard uploaded by the first thread in the second target file chunk; for the file shard uploaded by the second thread, its shard identification is: the number of the current file shard's thread + the total number of threads * (the number of the currently received target file chunks - 1) = 2 + 10 * (2 - 1) = 12. At this time, the shard identification "12" can be added to the chunk identification list to identify the file shard uploaded by the second thread in the second target file chunk;...; for the file shard uploaded by the tenth thread, its shard identification is: the number of the current file shard's thread + the total number of threads * (the number of the currently received target file chunks - 1) = 10 + 10 * (2 - 1) = 20. At this time, the shard identification "20" can be added to the chunk identification list to identify the file shard uploaded by the tenth thread in the second target file chunk.

[0197] Suppose the 3rd target file block is received. If the size of the received 3rd target file block is less than the upload size limit, or the size of the 3rd target file block is not less than the upload size limit and the size of the file shard after the target file block is sharded is less than the upload size limit, then the target file block is not sharded and corresponds to a shard identifier. The number of currently received target file blocks is 3. Thread total * (the number of currently received target file blocks - 1) + 1 = 10 * (3 - 1) + 1 = 21. At this time, the shard identifier "21" can be added to the block identifier list to identify the 3rd target file block.

[0198] At this time, the shard identifiers in the obtained block identifier list are sorted from small to large, and {1, 2, ……, 10, 11, 12, …… 20, 21} can be obtained. According to the order of each shard identifier, the corresponding shards are sequentially spliced to obtain the target file.

[0199] It should be noted that different size relationships between the size of the target file block, the size of the file shard after the target file block is sharded, and the set upload bandwidth and upload size limit can correspond to different block identifier generation strategies, so as to splice the target file blocks transmitted in a streaming and multi-threaded sharded manner into a complete target file, ensuring that the target file can be accurately spliced and thus ensuring the integrity and accuracy of the target file.

[0200] The embodiment of this specification provides a file upload system, which realizes that the target node reads target file blocks from the target file in sequence according to the set upload bandwidth, uses the streaming upload method to upload the target file to the target storage service corresponding to the target node, and on the basis of using the streaming upload method, for each target file block, it can also upload the target file block to the target storage service through at least one thread based on the size of the target file block, the set upload bandwidth and the upload size limit. On the basis of streaming upload, multi-threaded upload is adopted, which greatly improves the upload speed of the target file and provides a stable, efficient and secure file upload method; moreover, the reporting control end can also control the abnormal reporting frequency of the core dump file of the same service instance within the set duration, avoiding reporting each core dump file generated by an exception, reducing the processing pressure and saving processing resources.

[0201] The following combines the attached Figure 3 , taking the application of the file upload method provided in this specification in the scenario of program abnormal termination of the service instance as an example, to further illustrate the program abnormal termination method. Among them, Figure 3 shows a schematic diagram of the processing process of a file upload method provided by an embodiment of this specification, as Figure 3 shown:

[0202] The file system includes 4 nodes, and scheduling unit 1 (POD1), scheduling unit 2 (POD2), etc. are deployed in each node. The service instances deployed on the nodes can run on scheduling unit 1 (POD1) and scheduling unit 2 (POD2) to provide corresponding services. Moreover, an exception information collection program (Coredump Client) is deployed on each node, and a streaming and multi-threaded shard upload strategy is deployed in the exception information collection program (Coredump Client).

[0203] Corresponding storage services are deployed in the same region for each node. Assume that the target storage service corresponding to node 1 is storage service 1 (such as OSS1). Node 1 uploads the core dump (Coredump) file generated when the service instance is abnormal to storage service 1; the target storage service corresponding to node 2 is storage service 2 (such as OSS2), and node 2 uploads the core dump (Coredump) file generated when the service instance is abnormal to storage service 1; the target storage service corresponding to node 3 is storage service 3 (such as COS1), and node 3 uploads the core dump (Coredump) file generated when the service instance is abnormal to storage service 13; the target storage service corresponding to node 4 is storage service 4 (such as COS2), and node 4 uploads the core dump (Coredump) file generated when the service instance is abnormal to storage service 4.

[0204] In the case of a service instance exception occurring in any node, that node sends the exception information of the abnormal service instance to the reporting control end (Coredump server). The reporting control end (Coredump server) reads the historical exception information of the service instance from the database (DB) for reporting frequency control, writes the current exception information into the database, and returns a reporting control instruction to the node, indicating whether the currently generated core dump (Coredump) file is to be uploaded to the corresponding storage service.

[0205] When the node receives the reporting control instruction, if the reporting control instruction indicates that the core dump (Coredump) file needs to be reported, it can, according to the configured streaming and multi-threaded shard upload strategy, sequentially read file blocks from the core dump (Coredump) file according to the set upload bandwidth. For each file block, it can also determine whether to perform shard and multi-threaded upload on the file block based on the size of the file block, the set upload bandwidth, and the upload size limit.

[0206] The reporting control end (Coredump server) can push the exception information of the service instance to the monitoring end (Prometheus). The monitoring end (Prometheus) can obtain the service alarm policy from the alarm unit (reddog) of the content application platform to determine whether the currently abnormal service instance needs to raise an exception alarm. If an exception alarm is required, it can push the alarm information to the service end through the alarm unit (reddog).

[0207] The service end can receive the alarm information from the alarm unit of the content application platform, obtain the exception information of the service instance with the exception, generate a signed download link by calling the signature interface based on the exception information, access the storage service deployed in the same region through this download link, and download the core dump (Coredump) file of the service instance with the exception. Alternatively, the service end can also access the storage service deployed in the same region in the local file system through the access tool of the storage service and download the core dump (Coredump) file of the service instance with the exception. The service end can perform location analysis on the downloaded core dump (Coredump) file to determine the cause of the exception.

[0208] The embodiment of this specification provides a file upload method. For the scenario of abnormal termination of the program of the service instance, the target node can adopt multi-threaded upload on the basis of streaming upload, which greatly improves the upload speed of the core dump (Coredump) file, provides a stable, efficient and secure file upload method, and does not generate additional cross-cloud bandwidth, ensuring that business personnel can timely read the core dump file from the remote storage service for GDB analysis, timely locate the cause of the exception and fix the exception. Moreover, the reporting control end can also control the exception reporting frequency of the core dump file of the same service instance within a set duration, avoiding reporting the core dump file generated by each exception, reducing the processing pressure and saving processing resources.

[0209] Corresponding to the above method embodiment, this specification also provides an embodiment of a file upload device Figure 4 shows a schematic structural diagram of a file upload device provided by an embodiment of this specification, applied to a target node, as Figure 4 shown, the device includes:

[0210] A reading module 402, configured to obtain a set upload bandwidth in response to a file upload instruction of a target file to be uploaded, and read a target file block from the target file according to the set upload bandwidth;

[0211] A determination module 404, configured to determine a target storage service corresponding to the target node and determine the upload size limit of the target storage service;

[0212] The upload module 406 is configured to upload the target file block to the target storage service through at least one thread according to the size of the currently read target file block, the set upload bandwidth, and the upload size limit.

[0213] Optionally, the upload module 406 is further configured to:

[0214] Determine whether the size of the target file block reaches the set upload bandwidth;

[0215] If the set upload bandwidth is reached, divide the target file block into a set number of file shards and upload the set number of file shards to the target storage service through the set number of threads respectively;

[0216] If the set upload bandwidth is not reached, upload the target file block to the target storage service through at least one thread according to the size of the target file block and the upload size limit.

[0217] Optionally, the upload module 406 is further configured to:

[0218] Determine whether the size of the target file block is less than the upload size limit;

[0219] If the size of the target file block is less than the upload size limit, upload the target file block to the target storage service through one thread;

[0220] If the size of the target file block is not less than the upload size limit, divide the target file block into a set number of file shards and upload the target file block to the target storage service through at least one thread according to the size of the file shards and the upload size limit.

[0221] Optionally, the upload module 406 is further configured to:

[0222] Determine whether the size of the file shard is less than the upload size limit;

[0223] If the size of the file shard is less than the upload size limit, upload the target file block to the target storage service through one thread;

[0224] If the size of the file shard is not less than the upload size limit, upload the set number of file shards to the target storage service through the set number of threads respectively.

[0225] Optionally, the determination module 404 is further configured to:

[0226] Determine the deployment area of the target node;

[0227] Determine the storage service corresponding to the deployment area as the target storage service.

[0228] Optionally, the target file is a core dump file generated when an exception occurs in a target service instance in a target node; the apparatus further includes a monitoring module configured to:

[0229] Generate a corresponding core dump file when it monitors that an exception occurs in the target service instance;

[0230] Send the exception information of the target service instance to the reporting control end, where the exception information of the target service instance is used to instruct the reporting control end to determine whether to report the core dump file of the target service instance based on the configured exception reporting policy, and return a reporting control instruction to the target node;

[0231] Receive the reporting control instruction returned by the reporting control end. If the reporting control instruction indicates reporting, it is determined that a file upload instruction is detected, where the file upload instruction instructs to upload the core dump file to the target storage service.

[0232] An embodiment of this specification provides a file upload method, which realizes reading target file blocks from a target file in sequence according to a set upload bandwidth, and using a streaming upload method to upload the target file to a target storage service corresponding to a target node. On the basis of the streaming upload method, for each target file block, it is also possible to upload the target file block to the target storage service through at least one thread based on the size of the target file block, the set upload bandwidth, and the upload size limit. On the basis of the streaming upload, multi-threaded upload is adopted, which greatly improves the upload speed of the target file and provides a stable, efficient, and secure file upload method.

[0233] The above is a schematic solution of a file upload apparatus according to this embodiment. It should be noted that the technical solution of the file upload apparatus and the technical solution of the above file upload method belong to the same concept. For the details not described in the technical solution of the file upload apparatus, reference can be made to the description of the technical solution of the above file upload method.

[0234] Figure 5 The structural block diagram of a computing device provided according to an embodiment of this specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 through a bus 530, and a database 550 is used to store data.

[0235] The computing device 500 further includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of such networks include the Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0236] In one embodiment of the present specification, the above components of the computing device 500, as well as Figure 5 other components not shown, may also be connected to each other, for example, via a bus. It should be understood that Figure 5 the block diagram of the computing device shown is for illustrative purposes only and is not a limitation on the scope of the present specification. Those skilled in the art may add or replace other components as needed.

[0237] The computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 500 may also be a mobile or stationary server.

[0238] Wherein, the processor 520 is configured to execute the following computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the above file uploading method are implemented.

[0239] The above is a schematic solution of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above file uploading method belong to the same concept. For the details not described in detail in the technical solution of the computing device, reference can be made to the description of the technical solution of the above file uploading method.

[0240] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above file uploading method.

[0241] The above is a schematic solution of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above file uploading method belong to the same concept. For the details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solution of the above file uploading method.

[0242] An embodiment of this specification also provides a computer program, which, when executed on a computer, causes the computer to execute the steps of the above file uploading method.

[0243] The above is a schematic solution of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the above file uploading method belong to the same concept. For the details not described in detail in the technical solution of the computer program, reference can be made to the description of the technical solution of the above file uploading method.

[0244] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain implementations, multitasking and parallel processing are also possible or may be advantageous.

[0245] Computer instructions include computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. A computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, external hard drives, magnetic disks, optical discs, computer memories, read-only memories (ROMs), random access memories (RAMs), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0246] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the embodiments of this specification are not limited by the described order of actions, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0247] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0248] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can well understand and utilize this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A file uploading method, characterized in that: Applied to the target node, including: In response to a file upload instruction of a target file to be uploaded, obtaining a set upload bandwidth, and reading a target file block from the target file according to the set upload bandwidth; Determine a target storage service corresponding to the target node, and determine an upload size limit of the target storage service; According to the size of the target file block currently read, the set upload bandwidth and the upload size limit, the target file block is uploaded to the target storage service through at least one thread.

2. The file uploading method according to claim 1, characterized in that: The uploading the target file block to the target storage service through at least one thread according to the size of the target file block currently read, the set upload bandwidth and the upload size limit, comprises: Determining whether the size of the target file block reaches the set upload bandwidth; If the set upload bandwidth is reached, the target file block is divided into a set number of file slices, and the set number of file slices are uploaded to the target storage service through a set number of threads respectively; If the set upload bandwidth is not reached, the target file block is uploaded to the target storage service through at least one thread according to the size of the target file block and the upload size limit.

3. The file uploading method according to claim 2, characterized in that: The uploading the target file block to the target storage service through at least one thread according to the size of the target file block and the upload size limit includes: Determining whether the size of the target file block is less than the upload size limit; If the size of the target file block is smaller than the upload size limit, uploading the target file block to the target storage service through one thread; If the size of the target file block is not less than the upload size limit, the target file block is divided into a set number of file segments, and the target file block is uploaded to the target storage service through at least one thread according to the size of the file segments and the upload size limit.

4. The file uploading method according to any one of claims 1 to 3, characterized in that: The target file is a core dump file generated when an exception occurs in a target service instance in the target node; the method further includes: When an exception is detected in the target service instance, a corresponding core dump file is generated; Sending the exception information of the target service instance to the reporting control end, wherein the exception information of the target service instance is used to instruct the reporting control end to determine whether to report the core dump file of the target service instance based on the configured exception reporting policy, and return a reporting control instruction to the target node; A reporting control instruction returned by the reporting control terminal is received, and if the reporting control instruction indicates reporting, it is determined that the file upload instruction is detected, wherein the file upload instruction indicates uploading the core dump file to the target storage service.

5. A file upload system, characterized in that: It includes at least one node and a storage service corresponding to each node, any node in the at least one node is a target node, and the storage service corresponding to the target node is a target storage service; The target node is configured to obtain a set upload bandwidth in response to a file upload instruction of a target file to be uploaded, and read a target file block from the target file according to the set upload bandwidth; determine a target storage service corresponding to the target node, and determine an upload size limit of the target storage service; and upload the target file block to the target storage service through at least one thread according to the size of the target file block currently read, the set upload bandwidth, and the upload size limit; The target storage service is configured to receive the target file blocks, and when the target file is uploaded, to obtain a corresponding target file according to the received target file blocks.

6. The file upload system according to claim 5, characterized in that: The target file is a core dump file generated when an exception occurs in a target service instance in a target node; the file upload system also includes a reporting control terminal; The target node is further configured to generate a corresponding core dump file when an abnormality occurs in the target service instance; and send the abnormality information of the target service instance to the reporting control terminal; The reporting control end is configured to receive the exception information of the target service instance, determine whether to report the core dump file of the target service instance based on the configured exception reporting strategy and the exception information, and return a reporting control instruction to the target node; The target node is further configured to receive a reporting control instruction returned by the reporting control terminal, and if the reporting control instruction indicates reporting, determine that the file upload instruction is detected, wherein the file upload instruction indicates uploading the core dump file to the target storage service.

7. The file upload system according to claim 6, characterized in that: The abnormal reporting strategy is that the abnormal reporting frequency of the same service instance within a set time period is lower than the frequency threshold; the reporting control end is further configured as follows: Based on the abnormal information of the target service instance, query the historical abnormal reporting information of the target service instance; Determining, based on the historical abnormal reporting information, whether the current reporting frequency of the target service instance is lower than a frequency threshold; If yes, returning a first reporting control instruction to the target node, wherein the first reporting control instruction instructs reporting a core dump file of the target service instance; If not, a second reporting control instruction is returned to the target node, where the second reporting control instruction instructs not to report the core dump file of the target service instance.

8. The file upload system according to claim 6, characterized in that: The file upload system also includes a service end of the content application platform; the service end is configured as follows: Access the target storage service to download the core dump file of the target service instance; Access the content application platform to obtain the core dump file location strategy; The core dump file is restored and debugged according to the core dump file positioning strategy to locate the abnormal cause of the target service instance.

9. The file upload system according to claim 5, characterized in that: The target storage service is further configured as follows: Determine the block identifier of the target file block according to the current size of the target file block, the set upload bandwidth and the upload size limit of the target storage service, and update the block identifier list according to the block identifier of the target file block; When the target file is uploaded, the block identifier list is read, the block identifiers in the block identifier list are sorted, and the received target file blocks are spliced ​​according to the sorting result to obtain the corresponding target file.

10. The file upload system according to claim 9, characterized in that: The target storage service is further configured as follows: If the size of the target file block reaches the set upload bandwidth, or the size of the target file block is not less than the upload size limit, and the size of the file fragment after the target file block is fragmented is not less than the upload size limit, then determine the fragment identifier of the target file block based on the current number of threads, the total number of threads and the current number of file blocks; If the size of the target file block is smaller than the upload size limit, or the size of the target file block is not smaller than the upload size limit and the size of the file fragments after the target file block is fragmented is smaller than the upload size limit, the fragment identifier of the target file block is determined according to the total number of threads and the current number of file blocks.

11. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the file uploading method according to any one of claims 1 to 4 are implemented.

12. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the file uploading method described in any one of claims 1 to 4.

13. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, implements the steps of the file uploading method according to any one of claims 1 to 4.

Images