File processing method and device

Abstract

The present invention discloses a file processing method and device, relating to the field of computer technology. A specific implementation of the method includes: obtaining a file operation command; parsing the file operation command to obtain file operation parameters; determining whether to limit the operation speed based on the file operation parameters, and if so, determining the speed limit; and performing an operation on the file based on the speed limit; wherein the type of operation includes any one or more of copying, compressing, and decompressing. This implementation can reduce the impact of the operation on other services, preventing other services from crashing.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a file processing method and device. Background Art

[0002] In actual application scenarios, multiple services running on a server may share a disk. If an operation is performed on one of the services, other services that share the disk may be affected.

[0003] For example, if services A, B, and C share a disk, and service A's data files are compressed, the compression process will use the maximum I / O (Input / Output) performance the disk can provide. At this point, services B and C will not be able to use the disk for I / O operations.

[0004] Therefore, the current file handling method may affect other services of the shared disk and even cause other services to crash. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a file processing method and apparatus, which can reduce the impact of the operation process on other services and avoid the crash of other services.

[0006] In a first aspect, an embodiment of the present invention provides a file processing method, comprising:

[0007] Get file operation commands;

[0008] Parsing the file operation command to obtain file operation parameters;

[0009] determining whether to limit the operation speed according to the file operation parameter, and if so, determining the limit speed;

[0010] An operation is performed on the file according to the limited speed; wherein the type of the operation includes: any one or more of copying, compression and decompression.

[0011] Optionally,

[0012] After acquiring the file operation command and before parsing the file operation command to obtain the file operation parameters, the method further includes:

[0013] Check whether the syntax of the file operation command meets the preset rules. If yes, execute the parsing of the file operation command to obtain file operation parameters.

[0014] Optionally,

[0015] The determining whether to limit the operation speed according to the file operation parameter includes:

[0016] Determine whether the speed limit status in the file operation parameters is on.

[0017] Optionally,

[0018] The file operation parameters include: parameters of the original file, parameters of the target file and the type of the operation;

[0019] The performing an operation on the file according to the speed limit includes:

[0020] Generate tokens according to the speed limit.

[0021] Add the token to the token bucket;

[0022] Reading a byte stream from the original file according to the parameters of the original file;

[0023] The byte stream is written into the target file according to the number of tokens in the token bucket, the type of the operation and the parameters of the target file.

[0024] Optionally,

[0025] The step of writing the byte stream into the target file according to the number of tokens in the token bucket, the type of the operation, and the parameters of the target file comprises:

[0026] determining a number of tokens required for the byte stream;

[0027] When the number of tokens required for the byte stream is not greater than the number of tokens in the token bucket, determining a writing speed according to the number of tokens required for the byte stream;

[0028] The byte stream is written into the target file according to the writing speed, the type of the operation and the parameters of the target file.

[0029] Optionally,

[0030] Writing the byte stream into the target file according to the number of tokens in the token bucket, the type of the operation, and the parameters of the target file comprises:

[0031] determining a number of tokens required for the byte stream;

[0032] When the number of tokens required by the byte stream is greater than the number of tokens in the token bucket, splitting the byte stream into a first byte stream and a second byte stream;

[0033] Determining a first writing speed according to the number of tokens in the token bucket;

[0034] writing the first byte stream into the target file according to the first writing speed, the type of the operation, and the parameters of the target file;

[0035] Adding the second byte stream to a preset queue;

[0036] When the number of tokens in the token bucket is not less than the number of tokens required by the second byte stream, determining a second writing speed according to the number of tokens required by the second byte stream;

[0037] writing the second byte stream in the queue into the target file according to the second writing speed, the type of the operation, and the parameters of the target file;

[0038] The number of tokens required by the first byte stream is equal to the number of tokens in the token bucket.

[0039] Optionally,

[0040] Writing the byte stream into the target file according to the number of tokens in the token bucket, the type of the operation, and the parameters of the target file comprises:

[0041] Determining a writing speed according to the number of tokens in the token bucket;

[0042] The byte stream is written into the target file according to the writing speed, the preset blocking pipe, the type of the operation and the parameters of the target file.

[0043] In a second aspect, an embodiment of the present invention provides a file processing device, comprising:

[0044] An acquisition module configured to acquire file operation commands;

[0045] A parsing module configured to parse the file operation command to obtain file operation parameters;

[0046] a determination module configured to determine whether to limit the operation speed based on the file operation parameters, and if so, determine the limit speed;

[0047] The operation module is configured to perform an operation on the file according to the speed limit; wherein the type of the operation includes: any one or more of copying, compression and decompression.

[0048] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0049] one or more processors;

[0050] a storage device for storing one or more programs,

[0051] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0052] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the above embodiments.

[0053] One embodiment of the above invention has the following advantages or beneficial effects: limiting the speed of file operations based on file operation commands, avoiding file operations from occupying too many disk I / O resources, reducing the impact on other services, and preventing other services from crashing.

[0054] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0056] Figure 1 is a flowchart of a file processing method provided by one embodiment of the present invention;

[0057] Figure 2 is a flowchart of a file processing method provided by another embodiment of the present invention;

[0058] Figure 3 is a flow chart of a file processing device provided by one embodiment of the present invention;

[0059] Figure 4 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0060] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0062] Existing tools do not limit disk I / O speed during operations such as copying and compressing files. When multiple services share a disk, if an operation is performed on a data file of one service, the operation will write to the disk at the maximum I / O speed, thereby affecting the operation of other services.

[0063] Based on this, Figure 1 As shown, an embodiment of the present invention provides a file processing method, including:

[0064] Step 101: Obtain a file operation command.

[0065] Step 102: Parse the file operation command to obtain file operation parameters.

[0066] The file operation parameters may include any one or more of the following: the type of operation, the speed limit status, the speed limit, the parameters of the original file, and the parameters of the target file.

[0067] The original file refers to the file before the operation, and the target file refers to the file obtained after the operation. For example, the target file is obtained by compressing the original file.

[0068] Step 103: Determine whether to limit the operation speed based on the file operation parameters. If yes, execute step 104; otherwise, execute step 106.

[0069] In actual application scenarios, the operation speed can be limited based on different rules. Specifically, whether to limit the operation speed can be determined based on the type of operation, for example, the copy operation can be speed-limited. Whether to limit the operation speed can also be determined based on the speed limit status. If the speed limit status is on, the operation speed is limited. If the speed limit status is off, the operation speed is not limited, but the file is operated at the maximum operation speed.

[0070] Step 104: Determine the speed limit.

[0071] The speed limit can be a preset fixed value, for example, 50 bytes / s regardless of the operation. The speed limit can also be included in file operation parameters, meaning that the speed limits obtained by different file operation commands may be different. It should be noted that limiting the operation speed refers to limiting the disk I / O speed, i.e., limiting the speed at which content is written to the disk.

[0072] Step 105: Execute an operation on the file according to the speed limit and terminate the current process.

[0073] The operation type includes any one or more of copy, compress, and decompress.

[0074] Step 106: Perform an operation on the file according to a preset maximum operation speed.

[0075] The maximum operation speed refers to the maximum disk I / O speed.

[0076] The embodiment of the present invention limits the file operation speed based on the file operation command, thereby preventing the file operation from occupying too many disk I / O resources, reducing the impact on other services, and preventing other services from crashing.

[0077] In the prior art, copying, compressing, and decompressing files on a server requires different tools, and these tools vary across different operating systems. For example, in Linux, copying is performed using the cp tool, while compression and decompression are performed using the tar tool. In Windows, copying is performed using the copy tool, while compression and decompression are performed using the rar tool. Therefore, when operating files on different operating systems, users need to understand how to use different tools, which can easily lead to confusion and errors.

[0078] In view of this, in actual application scenarios, file processing methods can be implemented using the Golang language. The Golang language supports cross-platform compilation, which can unify file operation commands on different system platforms and reduce the complexity of operations between different system platforms.

[0079] In an embodiment of the present invention, after obtaining the file operation command and before parsing the file operation command and obtaining the file operation parameters, the method further includes:

[0080] Verify whether the syntax of the file operation command meets the preset rules. If so, execute the parsing of the file operation command to obtain the file operation parameters.

[0081] Considering that file operation commands may not comply with rules, in order to filter illegal commands, the embodiment of the present invention verifies the syntax of file operation commands, reduces the consumption of system resources by illegal commands, and improves system security.

[0082] In one embodiment of the present invention, determining whether to limit the operation speed based on the file operation parameters includes:

[0083] Check whether the speed limit status in the file operation parameters is enabled.

[0084] Through the embodiments of the present invention, users can limit the speed of file operations through file operation commands according to actual needs, flexibly adjust the file operation speed, and coordinate the operation of various services.

[0085] In one embodiment of the present invention, the file operation parameters include: parameters of the original file, parameters of the target file, and the type of operation;

[0086] Based on the speed limit, perform actions on the file, including:

[0087] Generate tokens according to the limit speed;

[0088] Add the token to the token bucket;

[0089] Read the byte stream from the original file according to the parameters of the original file;

[0090] Write the byte stream to the target file based on the number of tokens in the token bucket, the type of operation, and the parameters of the target file.

[0091] The embodiment of the present invention limits the file operation speed based on the token bucket algorithm and controls the occupation of disk I / O resources by file operations. In actual application scenarios, the token bucket algorithm can also be replaced by a leaky bucket algorithm, and the specific implementation details of the leaky bucket algorithm are no longer described here.

[0092] In one embodiment of the present invention, writing a byte stream into a target file according to the number of tokens in the token bucket, the type of operation, and the parameters of the target file includes:

[0093] Determine the number of tokens required for the byte stream;

[0094] When the number of tokens required by the byte stream is not greater than the number of tokens in the token bucket, the writing speed is determined according to the number of tokens required by the byte stream;

[0095] Writes the byte stream to the target file based on the write speed, the type of operation, and the parameters of the target file.

[0096] In this embodiment of the present invention, one token corresponds to one byte. If 20 bytes of data need to be written, 20 tokens are required. This method generates tokens based on a speed limit. For example, if the speed limit is 50 bytes / s, a token is generated and added to the token bucket every 1 / 50 second. If the byte stream is 50 bytes and the token bucket contains 50 tokens, the byte stream can be written to the target file at 50 bytes / s. This embodiment of the present invention can flexibly control the write speed by using the number of tokens, reducing the impact of file operations on other services.

[0097] In one embodiment of the present invention, writing a byte stream into a target file according to the number of tokens in the token bucket, the type of operation, and the parameters of the target file includes:

[0098] Determine the number of tokens required for the byte stream;

[0099] When the number of tokens required by the byte stream is greater than the number of tokens in the token bucket, the byte stream is split into a first byte stream and a second byte stream;

[0100] Determining a first writing speed according to the number of tokens in the token bucket;

[0101] Writing the first byte stream into the target file according to the first writing speed, the type of operation, and the parameters of the target file;

[0102] Add the second byte stream to the preset queue;

[0103] When the number of tokens in the token bucket is not less than the number of tokens required by the second byte stream, determining a second writing speed according to the number of tokens required by the second byte stream;

[0104] Writing the second byte stream in the queue into the target file according to the second writing speed, the type of operation and the parameters of the target file;

[0105] The number of tokens required by the first byte stream is equal to the number of tokens in the token bucket.

[0106] In an embodiment of the present invention, if the number of tokens in the token bucket is insufficient to write the byte stream all at once, the byte stream is split into a first byte stream and a second byte stream. The first byte stream is first written to the target file using the existing tokens in the token bucket, and the second byte stream is temporarily stored in a preset queue to wait until the number of tokens in the token bucket is sufficient to write the second byte stream. This embodiment of the present invention can improve writing efficiency while meeting the speed limit.

[0107] In one embodiment of the present invention, writing a byte stream into a target file according to the number of tokens in the token bucket, the type of operation, and the parameters of the target file includes:

[0108] Determine the writing speed based on the number of tokens in the token bucket;

[0109] Writes a byte stream to a target file based on the write speed, the preset blocking pipe, the type of operation, and the parameters of the target file.

[0110] The embodiment of the present invention combines a blocking pipe with a token bucket to further limit the file operation speed. Specifically, if the speed limit is 50 bytes / s and the capacity of the token bucket is 60 tokens, the byte stream may obtain all 60 tokens. At this time, the write speed of the byte stream can reach 60 bytes / s, which exceeds the speed limit. In response to this situation, the embodiment of the present invention uses a blocking pipe to further limit the operation speed to ensure that the write speed does not exceed the speed limit. For example, the maximum speed allowed by the blocking pipe is 50 bytes / s. Even if the byte stream obtains 60 tokens, it still needs to pass through the speed-limited pipe at 50 bytes / s. Therefore, the speed of the speed-limited pipe is determined by the speed limit.

[0111] In actual application scenarios, the rate-limiting pipeline can be used alone to limit the file operation speed, or it can be combined with the leaky bucket algorithm.

[0112] like Figure 2 As shown, the embodiment of the present invention takes the copy operation as an example to describe the file processing method in detail. The method includes:

[0113] Step 201: Obtain a file operation command.

[0114] The file operation commands are as follows:

[0115] trans cp-originfile x-destfile y-limitspeed on 60B.

[0116] Step 202: Check whether the syntax of the file operation command meets the preset rules. If yes, execute step 203; otherwise, terminate the current process.

[0117] Step 203: Parse the file operation command to obtain file operation parameters;

[0118] The file operation parameters include: operation type: cp (i.e. copy); parameters of the original file: x; parameters of the target file: y; and speed limit: 60B / s.

[0119] Step 204 : Determine whether the speed limit status in the file operation parameters is on. If so, execute step 205 ; otherwise, execute step 211 .

[0120] Step 205: Determine the speed limit.

[0121] The file operation parameters determine that the speed limit is 60B / s.

[0122] Step 206: Generate a token according to the speed limit.

[0123] A token is generated every 1 / 60s.

[0124] Step 207: Add the token to the token bucket.

[0125] The capacity of the token bucket is 70.

[0126] Step 208: Read the byte stream from the original file according to the parameters of the original file.

[0127] The size of the byte stream is 65B.

[0128] Step 209: Determine the writing speed according to the number of tokens in the token bucket.

[0129] If the number of tokens is 65, the write speed is 65B / s.

[0130] Step 210: Write the byte stream into the target file according to the writing speed, the preset blocking pipe, the type of operation and the parameters of the target file, and terminate the current process.

[0131] The embodiment of the present invention adopts a blocking pipe with a queue length of 60 bytes to limit the file copy speed to 60 B / s. Therefore, although the byte stream obtains 65 tokens, the disk I / O speed can be controlled at 60 B / s due to the existence of the blocking pipe.

[0132] Step 211: Perform an operation on the file according to a preset maximum operation speed.

[0133] This embodiment of the present invention combines a token bucket algorithm with a blocking pipeline to control file operation speed, reducing disk I / O overhead during operations, minimizing the impact on other services, and improving overall server stability. The method provided by this embodiment of the present invention can also be used to copy and compress log files in application services.

[0134] like Figure 3 As shown, an embodiment of the present invention provides a file processing device, including:

[0135] An acquisition module 301 is configured to acquire a file operation command;

[0136] The parsing module 302 is configured to parse the file operation command and obtain the file operation parameters;

[0137] The determination module 303 is configured to determine whether to limit the operation speed according to the file operation parameters, and if so, determine the limit speed;

[0138] The operation module 304 is configured to perform an operation on the file according to the speed limit; wherein the type of operation includes: any one or more of copying, compression and decompression.

[0139] In one embodiment of the present invention, the parsing module 302 is configured to verify whether the syntax of the file operation command meets the preset rules, and if so, execute the parsing of the file operation command to obtain the file operation parameters.

[0140] In one embodiment of the present invention, the determination module 303 is configured to determine whether the speed limit status in the file operation parameters is on.

[0141] In one embodiment of the present invention, the file operation parameters include: parameters of the original file, parameters of the target file and the type of operation; the operation module 304 is configured to generate tokens according to the limit speed; the tokens are added to the token bucket; the byte stream is read from the original file according to the parameters of the original file; and the byte stream is written to the target file according to the number of tokens in the token bucket, the type of operation and the parameters of the target file.

[0142] In one embodiment of the present invention, the operation module 304 is configured to determine the number of tokens required for the byte stream; when the number of tokens required for the byte stream is not greater than the number of tokens in the token bucket, determine the writing speed based on the number of tokens required for the byte stream; and write the byte stream into the target file based on the writing speed, the type of operation and the parameters of the target file.

[0143] In one embodiment of the present invention, the operation module 304 is configured to determine the number of tokens required for a byte stream; when the number of tokens required for the byte stream is greater than the number of tokens in the token bucket, split the byte stream into a first byte stream and a second byte stream; determine a first write speed based on the number of tokens in the token bucket; write the first byte stream to the target file based on the first write speed, the type of operation and the parameters of the target file; add the second byte stream to a preset queue; when the number of tokens in the token bucket is not less than the number of tokens required for the second byte stream, determine a second write speed based on the number of tokens required for the second byte stream; write the second byte stream in the queue to the target file based on the second write speed, the type of operation and the parameters of the target file; wherein the number of tokens required for the first byte stream is equal to the number of tokens in the token bucket.

[0144] In one embodiment of the present invention, the operation module 304 is configured to determine the writing speed according to the number of tokens in the token bucket; and write the byte stream into the target file according to the writing speed, the preset blocking pipeline, the type of operation and the parameters of the target file.

[0145] An embodiment of the present invention provides an electronic device, including:

[0146] one or more processors;

[0147] a storage device for storing one or more programs,

[0148] When one or more programs are executed by one or more processors, the one or more processors implement the method of any of the above embodiments.

[0149] An embodiment of the present invention provides a computer-readable medium having a computer program stored thereon. When the program is executed by a processor, the method of any of the above embodiments is implemented.

[0150] Figure 4An exemplary system architecture 400 is shown to which the file processing method or file processing apparatus according to the embodiment of the present invention can be applied.

[0151] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0152] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Terminal devices 401, 402, and 403 can be installed with various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0153] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0154] Server 405 may be a server that provides various services, such as a backend management server (for example only) that supports shopping websites browsed by users using terminal devices 401, 402, and 403. The backend management server may analyze and process received data such as product information query requests, and feed back processing results (for example, target push information and product information—for example only) to the terminal device.

[0155] It should be noted that the file processing method provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the file processing device is generally provided in the server 405 .

[0156] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0157] Reference below Figure 5 , which shows a schematic structural diagram of a computer system 500 of a terminal device suitable for implementing an embodiment of the present invention. Figure 5 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0158] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the system 500 are also stored in the RAM 503. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0159] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read therefrom can be installed into the storage section 508 as needed.

[0160] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 509, and / or installed from a removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are performed.

[0161] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be provided in a processor. For example, they may be described as follows: a processor includes a sending module, an acquisition module, a determination module, and a first processing module. The names of these modules do not, in some cases, limit the modules themselves. For example, the sending module may also be described as a "module for sending a picture acquisition request to the connected server."

[0164] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiments, or may exist independently without being incorporated into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the device, the device includes:

[0165] Get file operation commands;

[0166] Parsing the file operation command to obtain file operation parameters;

[0167] determining whether to limit the operation speed according to the file operation parameter, and if so, determining the limit speed;

[0168] An operation is performed on the file according to the limited speed; wherein the type of the operation includes: any one or more of copying, compression and decompression.

[0169] According to the technical solution of the embodiment of the present invention, the speed of file operations is limited based on file operation commands, so as to avoid file operations occupying too many disk I / O resources, reduce the impact on other services, and prevent other services from crashing.

[0170] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A file processing method, characterized in that: include: Get file operation commands; Parsing the file operation command to obtain file operation parameters; The file operation parameters include: parameters of the original file, parameters of the target file and the type of the operation; determining whether to limit the operation speed according to the file operation parameter, and if so, determining the limit speed; Performing an operation on the file according to the speed limit; wherein the type of the operation includes: any one or more of copying, compressing, and decompressing; The performing an operation on the file according to the speed limit includes: Generate tokens according to the speed limit. Add the token to the token bucket; Reading a byte stream from the original file according to the parameters of the original file; Writing the byte stream into the target file according to the number of tokens in the token bucket, the type of the operation, and the parameters of the target file; The step of writing the byte stream into the target file according to the number of tokens in the token bucket, the type of the operation, and the parameters of the target file comprises: determining a number of tokens required for the byte stream; When the number of tokens required for the byte stream is not greater than the number of tokens in the token bucket, determining a writing speed according to the number of tokens required for the byte stream; The byte stream is written into the target file according to the writing speed, the type of the operation and the parameters of the target file.

2. The method according to claim 1, wherein After acquiring the file operation command and before parsing the file operation command to obtain the file operation parameters, the method further includes: Check whether the syntax of the file operation command meets the preset rules. If yes, execute the parsing of the file operation command to obtain file operation parameters.

3. The method according to claim 1, wherein The determining whether to limit the operation speed according to the file operation parameter includes: Determine whether the speed limit status in the file operation parameters is on.

4. The method according to claim 1, wherein The step of writing the byte stream into the target file according to the number of tokens in the token bucket, the type of the operation, and the parameters of the target file comprises: determining a number of tokens required for the byte stream; When the number of tokens required by the byte stream is greater than the number of tokens in the token bucket, splitting the byte stream into a first byte stream and a second byte stream; Determining a first writing speed according to the number of tokens in the token bucket; writing the first byte stream into the target file according to the first writing speed, the type of the operation, and the parameters of the target file; Adding the second byte stream to a preset queue; When the number of tokens in the token bucket is not less than the number of tokens required by the second byte stream, determining a second writing speed according to the number of tokens required by the second byte stream; writing the second byte stream in the queue into the target file according to the second writing speed, the type of the operation, and the parameters of the target file; The number of tokens required by the first byte stream is equal to the number of tokens in the token bucket.

5. The method according to claim 1, wherein Writing the byte stream into the target file according to the number of tokens in the token bucket, the type of the operation, and the parameters of the target file comprises: Determining a writing speed according to the number of tokens in the token bucket; The byte stream is written into the target file according to the writing speed, the preset blocking pipe, the type of the operation and the parameters of the target file.

6. A file processing device, characterized in that: include: An acquisition module configured to acquire file operation commands; A parsing module configured to parse the file operation command to obtain file operation parameters; The file operation parameters include: parameters of the original file, parameters of the target file and the type of the operation; a determination module configured to determine whether to limit the operation speed based on the file operation parameters, and if so, determine the limit speed; an operation module configured to perform an operation on the file according to the speed limit; wherein the type of the operation includes: any one or more of copying, compressing, and decompressing; The operation module is further configured to: Generate tokens according to the speed limit. Add the token to the token bucket; Reading a byte stream from the original file according to the parameters of the original file; Writing the byte stream into the target file according to the number of tokens in the token bucket, the type of the operation, and the parameters of the target file; The operation module is further configured to: determining a number of tokens required for the byte stream; When the number of tokens required for the byte stream is not greater than the number of tokens in the token bucket, determining a writing speed according to the number of tokens required for the byte stream; The byte stream is written into the target file according to the writing speed, the type of the operation and the parameters of the target file.

7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images