File data processing method, device, equipment, medium and product

Abstract

The present invention discloses a method, apparatus, device, medium and product for processing file data, relating to the field of data processing technology. The method is executed by a file server, the file server includes at least one candidate physical processor, and each candidate physical processor is respectively provided with at least two candidate virtual processors. The method comprises: determining a target file data type according to the target file data volume of the file data to be processed; determining a target virtual processor from each candidate virtual processor according to the target file data type and the candidate processor label corresponding to each candidate virtual processor; forwarding a file data processing request to the target virtual processor, and processing the file data to be processed according to the file data processing request by the target virtual processor. The present invention improves the parallel task throughput of the file server, as well as the resource utilization rate of the file server, can effectively cope with high-concurrency file data processing requests, and improves the efficiency of file data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment, medium and product for processing file data. Background Art

[0002] A file server is a computer system dedicated to providing file data storage, management, and sharing services in a network environment. Its core functions include but are not limited to centralized data storage, cross-platform access control, and data security. Using a file server can effectively reduce hardware expenses for personal storage devices and improve the security of file data.

[0003] Existing file servers usually use the physical processors they carry to process file data. However, in general, the number of physical processors carried by file servers is limited, making it difficult to handle highly concurrent file data processing requests, resulting in low file data processing efficiency. Summary of the Invention

[0004] The present invention provides a file data processing method, apparatus, device, medium and product to solve the problem that existing file servers have difficulty in processing highly concurrent file data processing requests, resulting in low efficiency in file data processing.

[0005] According to one aspect of the present invention, a method for processing file data is provided, which is executed by a file server, wherein the file server includes at least one candidate physical processor, each of the candidate physical processors being provided with at least two candidate virtual processors. The method comprises:

[0006] Determining a target file data volume of the file data to be processed according to the acquired file data processing request, and determining a target file data type corresponding to the file data to be processed according to the target file data volume;

[0007] Determining a target virtual processor from each of the candidate virtual processors based on the target file data type and the candidate processor labels corresponding to each of the candidate virtual processors; wherein the candidate processor labels are used to identify the file data type that the corresponding candidate virtual processor is responsible for processing;

[0008] The file data processing request is forwarded to the target virtual processor, and the to-be-processed file data is processed by the target virtual processor according to the file data processing request.

[0009] According to another aspect of the present invention, a file data processing device is provided, which is configured on a file server. The file server includes at least one candidate physical processor, each of the candidate physical processors is provided with at least two candidate virtual processors, and the device includes:

[0010] A file data type determination module is configured to determine a target file data volume of the file data to be processed according to the acquired file data processing request, and determine a target file data type corresponding to the file data to be processed according to the target file data volume;

[0011] a virtual processor determination module, configured to determine a target virtual processor from each candidate virtual processor based on the target file data type and a candidate processor tag corresponding to each candidate virtual processor; wherein the candidate processor tag is used to identify the file data type that the corresponding candidate virtual processor is responsible for processing;

[0012] The file data processing module is configured to forward the file data processing request to the target virtual processor, and process the to-be-processed file data according to the file data processing request through the target virtual processor.

[0013] According to another aspect of the present invention, an electronic device is provided, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the file data processing method according to any one of the present inventions.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement any one of the file data processing methods of the present invention when executed.

[0018] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the file data processing method according to any one of the present inventions.

[0019] The present invention sets at least two candidate virtual processors in at least one candidate physical processor included in the file server, and determines a target virtual processor from each candidate virtual processor to process the file data to be processed according to the file data processing request. Since the number of virtual processors is greater than the number of candidate physical processors, the parallel task throughput of the file server is improved, and the resource utilization of the file server is improved, and it can effectively respond to high-concurrency file data processing requests and improve the efficiency of file data processing.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a method for processing file data provided in Example 1 of the present invention;

[0023] Figure 2 A flowchart of a method for processing file data provided in the second embodiment of the present invention;

[0024] Figure 3 A schematic diagram of the structure of a file data processing device provided in a third embodiment of the present invention;

[0025] Figure 4 The figure is a schematic structural diagram of an electronic device for implementing the file data processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "candidate", "target", "auxiliary", "history", "to be processed", "first category", "second category", "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Example 1

[0029] Figure 1 A flowchart of a method for processing file data provided in the first embodiment of the present invention, this embodiment is applicable to the case where the candidate virtual processors set in each candidate physical processor in the file server are called to process the file data, the method can be executed by a file data processing device, and the file data processing device can be implemented in the form of hardware and / or software, such as by using a file server, wherein the file server includes at least one candidate physical processor, and each candidate physical processor is respectively provided with at least two candidate virtual processors. The candidate physical processor refers to the central processing unit that actually exists in the file server hardware, and is the core hardware component of the file server for performing file data processing tasks. The candidate virtual processor is a logical execution unit created on each candidate physical processor through software simulation or hardware-assisted technology. The candidate virtual processor can simulate the instruction set and operating environment of the candidate physical processor, and has the ability to emulate the candidate physical processor to process file data. Figure 1 As shown, the method includes:

[0030] S101 : Determine a target file data volume of file data to be processed according to an acquired file data processing request, and determine a target file data type corresponding to the file data to be processed according to the target file data volume.

[0031] A file data processing request is a request sent by a client requesting file data processing to a file server via specific technical means, including but not limited to the HTTP protocol, to trigger the file server to process the pending file data. The request type of a file data processing request can be configured based on the user's file data processing requirements. The request types include but are not limited to file data download requests, file data upload requests, and file data edit requests. This embodiment does not limit the request type of a file data processing request.

[0032] The file data to be processed refers to the file data that the file data processing request expects to process. It can be understood that when the request type of the file data processing request is a file data download request, the file data to be processed refers to the file data to be downloaded in the file server; when the request type of the file data processing request is a file data upload request, the file data to be processed refers to the file data to be uploaded to the file server; when the request type of the file data processing request is a file data editing request, the file data to be processed refers to the file data to be edited in the file server.

[0033] The target file data volume is a core indicator used to describe the scale of the file data storage information to be processed, and is usually described in units including KB, MB, GB, etc.

[0034] The target file data type of the file data to be processed is determined based on the target file data volume of the file data to be processed. It is type information used to describe the file data volume scale level of the file data to be processed. For example, the target file data type can be a large data volume file, a medium data volume file, or a small data volume file, etc. This embodiment does not limit the specific form of the target file data type. Any type information that can distinguish different file data volume scale levels can be used as the target file data type.

[0035] In one embodiment, the file server receives a file data processing request sent by a target client and further determines the request type of the file data processing request. If the request type is determined to be a file data upload request, the file data processing request is parsed to obtain the file data to be processed. The file data to be processed is then evaluated using a data size evaluation algorithm to determine the target file data size.

[0036] In another embodiment, the file server receives a file data processing request sent by a target client and further determines the request type of the file data processing request. If the request type is determined to be a file data download request, the file data processing request is parsed to obtain the file data name of the file data to be processed. Based on the result of a hash modulo operation on the file data name, the storage address storing the file data to be processed is determined from the storage space. The file data to be processed is then accessed from the storage space based on the storage address to perform a data volume assessment and determine the target file data volume.

[0037] After determining the target file data volume of the file data to be processed, further determine the candidate file data volume intervals associated with each candidate file data type, and then determine the target file data type corresponding to the file data to be processed from each candidate file data type based on the matching results between the target file data volume and each candidate file data volume interval.

[0038] Exemplarily, assuming that the candidate file data types include "large data volume file", "medium data volume file" and "small data volume file", assuming that the candidate file data volume interval associated with "large data volume file" is "[500MB,∞)", the candidate file data volume interval associated with "medium data volume file" is [100MB,500MB), and the candidate file data volume interval associated with "small data volume file" is (0,100MB). Assuming that the target file data volume belongs to the candidate file data volume interval [500MB,∞), the target file data type is determined to be "large data volume file"; assuming that the target file data volume belongs to the candidate file data volume interval [100MB,500MB), the target file data type is determined to be "medium data volume file"; assuming that the target file data volume belongs to the candidate file data volume interval (0,100MB), the target file data type is determined to be "small data volume file".

[0039] S102 : Determine a target virtual processor from the candidate virtual processors according to the target file data type and the candidate processor labels corresponding to the candidate virtual processors.

[0040] A corresponding candidate processor label is generated in advance for each candidate virtual processor, and the candidate processor label is used to identify the type of file data that the corresponding candidate virtual processor is responsible for processing. For example, assuming that the candidate processor label corresponding to any candidate virtual processor is "large data volume file", it means that the file type that the candidate virtual processor is responsible for processing is "large data volume file"; assuming that the candidate processor label corresponding to any candidate virtual processor is "medium data volume file", it means that the file type that the candidate virtual processor is responsible for processing is "medium data volume file"; assuming that the candidate processor label corresponding to any candidate virtual processor is "small data volume file", it means that the file type that the candidate virtual processor is responsible for processing is "small data volume file".

[0041] In one embodiment, the target file data type is matched with candidate processor labels corresponding to each candidate virtual processor, and a candidate virtual processor responsible for processing the target file data type is determined from the candidate virtual processors according to the matching result as the target virtual processor.

[0042] S103: Forward the file data processing request to the target virtual processor, and process the file data to be processed according to the file data processing request through the target virtual processor.

[0043] In one embodiment, a file data processing request is forwarded to a target virtual processor. The target virtual processor determines the request type of the file data processing request. If the request type is a file data upload request, a first storage address is determined from the storage space based on the target file data size and file data name of the file data to be processed carried in the file data processing request. The file data to be processed is stored in the storage space according to the first storage address.

[0044] In another embodiment, a file data processing request is forwarded to a target virtual processor. The target virtual processor determines the request type of the file data processing request. If the request type is a file data download request, a second storage address is determined from the storage space based on the file data name of the file data to be processed carried in the file data processing request. A data download operation is performed on the file data to be processed from the storage space based on the second storage address.

[0045] The embodiment of the present invention sets at least two candidate virtual processors in at least one candidate physical processor included in the file server, and determines a target virtual processor from each candidate virtual processor to process the file data to be processed according to the file data processing request. Since the number of virtual processors is greater than the number of candidate physical processors, the parallel task throughput of the file server and the resource utilization of the file server are improved, and it can effectively respond to highly concurrent file data processing requests and improve the efficiency of file data processing.

[0046] Example 2

[0047] Figure 2 This is a flowchart of a method for processing file data provided by the second embodiment of the present invention. This embodiment further optimizes and expands the above embodiment and can be combined with the above optional implementation methods. Figure 2 As shown, the method includes:

[0048] S201. Determine a target file data volume of file data to be processed according to an acquired file data processing request, obtain candidate file data volume intervals associated with at least one candidate file data type, and match the target file data volume with each candidate file data volume interval.

[0049] S202: Determine the candidate file data volume interval to which the target file data volume belongs according to the matching result as the auxiliary file data volume interval, and use the candidate file data type associated with the auxiliary file data volume interval as the target file data type.

[0050] In one embodiment, the target file data volume is compared with the lower limit and upper limit of each candidate file data volume interval. The candidate file data volume interval whose lower limit is less than or equal to the target file data volume and whose upper limit is greater than or equal to the target file data volume is selected as the auxiliary file data volume interval. The candidate file data type associated with the auxiliary file data volume interval is further selected as the target file data type.

[0051] For example, assuming that the candidate file data types include "large data volume files", "medium data volume files" and "small data volume files", assuming that the candidate file data volume interval associated with "large data volume files" is "[500MB,∞)", the candidate file data volume interval associated with "medium data volume files" is [100MB,500MB), and the candidate file data volume interval associated with "small data volume files" is (0,100MB). Assuming that the target file data volume is 200MB, which belongs to the candidate file data volume interval [100MB,500MB), the target file data type is determined to be "medium data volume file".

[0052] By obtaining candidate file data volume intervals associated with at least one candidate file data type, and matching the target file data volume with each candidate file data volume interval; determining the candidate file data volume interval to which the target file data volume belongs based on the matching result as the auxiliary file data volume interval, and using the candidate file data type associated with the auxiliary file data volume interval as the target file data type, the beneficial effects are:

[0053] First, since different file data types, such as text data, audio data, video data, etc., have significant differences in data volume distribution, feature quantitative matching is achieved by pre-defining the candidate file data volume range to improve the accuracy of determining the target file data type.

[0054] Second, pre-associating candidate file data types according to the candidate file data volume range is conducive to quickly determining the target file data type corresponding to the file data to be processed, and indirectly improving the efficiency of file data processing on the file server.

[0055] S203: Match the target file data type with each candidate processor tag, and use the candidate processor tag that matches the target file data type as the target processor tag.

[0056] S204: Use the candidate virtual processor corresponding to the target processor label as the target virtual processor.

[0057] For example, assuming the target file data type is "medium data file," candidate virtual processor 1 corresponds to candidate processor tag 1, which is "small data file," candidate virtual processor 2 corresponds to candidate processor tag 2, which is "small data file," candidate virtual processor 3 corresponds to candidate processor tag 3, which is "medium data file," and candidate virtual processor 4 corresponds to candidate processor tag 4, which is "large data file." Because candidate processor tag 3 matches the target file data type, candidate virtual processor 3 is selected as the target virtual processor.

[0058] By matching the target file data type with each candidate processor tag, and using the candidate processor tag that matches the target file data type as the target processor tag; and using the candidate virtual processor corresponding to the target processor tag as the target virtual processor, the beneficial effects are:

[0059] First, through the precise matching mechanism of file data types and processor labels, adaptive virtual processor scheduling is achieved, avoiding the performance loss caused by traditional polling or random allocation of virtual processors.

[0060] Secondly, virtual processors are dynamically scheduled based on label matching to avoid overloading of a single virtual processor.

[0061] Third, when adding a new candidate processor type, it is only necessary to expand the processor tag library without modifying the virtual processor core scheduling algorithm, thereby enhancing system scalability.

[0062] Optionally, candidate processor labels are generated as follows:

[0063] A. Determine the amount of historical file data according to the historical file data processing request obtained by the file server within the historical time period, and determine the type of historical file data corresponding to each historical file data according to the amount of each historical file data.

[0064] The term "historical time period" refers to a specific period of time in history. Historical file data processing requests refer to file data processing requests received by the file server during the historical time period. Historical file data refers to the file data that the historical file data processing requests intended to process. The historical file data volume is a core metric used to describe the scale of historical file data storage.

[0065] In one embodiment, a filter is performed based on the request time of each file data processing request obtained by the file server to determine the historical file data processing requests obtained by the file server within a historical time period, and the historical file data corresponding to the historical file data processing requests is further determined. A data volume assessment algorithm is used to assess the volume of the historical file data to determine the historical file data volume. Furthermore, a candidate file data volume interval associated with each candidate file data type is determined. Based on the matching results between the historical file data volume and each candidate file data volume interval, the historical file data type corresponding to the historical file data volume is determined from each candidate file data type.

[0066] B. Determine the type ratio corresponding to each historical file data type, and generate a candidate processor label for each candidate virtual processor based on the type ratio.

[0067] In one embodiment, frequency statistics are collected for each historical file data type, and the frequency statistics are combined with the total frequency statistics of the historical file data type to determine the type percentage corresponding to each historical file data type. For example, if historical file data type 1 appears 20 times and the total frequency of the historical file data type is 100 times, then the type percentage corresponding to historical file data type 1 is 20%.

[0068] Furthermore, candidate processor labels are generated for each candidate virtual processor based on the proportion of each type. For example, assuming that the proportion of the type of historical file data types corresponding to "small data volume files" is 50%, the proportion of the type corresponding to "medium data volume files" is 30%, and the proportion of the type corresponding to "large data volume files" is 20%, then the candidate processor labels of 50% of the candidate virtual processors are randomly set to "small data volume files", the candidate processor labels of 30% of the candidate virtual processors are randomly set to "medium data volume files", and the candidate processor labels of 20% of the candidate virtual processors are randomly set to "large data volume files".

[0069] By determining the amount of historical file data based on historical file data processing requests obtained by the file server within a historical time period, determining the historical file data type corresponding to each historical file data based on the amount of each historical file data, determining the type ratio corresponding to each historical file data type, and generating a candidate processor label for each candidate virtual processor based on the type ratio, the beneficial effects are:

[0070] First, a processor label generation mechanism based on changes in type ratios supports automatic increase or decrease of virtual processor instances of specific types to meet sudden request growth needs and avoid performance bottlenecks caused by traditional static allocation.

[0071] Secondly, by analyzing the type ratios of historical file data types, dynamic matching of virtual processor resources and data type processing requirements can be achieved.

[0072] Third, dedicated processor tags are generated for different file data types to improve the efficiency of heterogeneous computing.

[0073] S205 . Determine a target data channel associated with the target virtual processor based on the association relationship between the candidate virtual processors and the candidate data channels; and forward the file data processing request to the target virtual processor using the target data channel.

[0074] The candidate data channel refers to a data channel in the file server for the candidate virtual processor to exchange data with the outside of the virtual processor. In this embodiment, an available candidate data channel is pre-assigned to each candidate virtual processor, and an association relationship is established between each candidate virtual processor and each candidate data channel.

[0075] In one embodiment, an association relationship between candidate virtual processors and candidate data channels is obtained, and based on the association relationship, a candidate data channel associated with a target virtual processor is determined as a target data channel. Furthermore, a determination is made as to whether a ready target data channel exists among the target data channels. If so, the file data processing request is forwarded to the target virtual processor using the ready target data channel.

[0076] By determining a target data channel associated with a target virtual processor based on an association relationship between candidate virtual processors and candidate data channels, and forwarding a file data processing request to the target virtual processor using the target data channel, the beneficial effects are:

[0077] First, intelligent matching is achieved through the association between candidate virtual processors and candidate data channels, avoiding the resource waste caused by traditional fixed binding and improving file data processing efficiency.

[0078] Secondly, accurate selection of target data channels can reduce redundant communication between virtual processors.

[0079] Third, when a candidate data channel is abnormal, it can quickly switch to the backup data channel based on the association relationship to ensure service continuity.

[0080] S206 : Process the file data to be processed according to the file data processing request through the target virtual processor.

[0081] Optionally, processing the file data to be processed according to the file data processing request by the target virtual processor includes:

[0082] A1. Determine the request type of the file data processing request through the target virtual processor, and if the request type is a file data upload request, parse the file data to be processed from the file data processing request as the first type of file data.

[0083] Among them, when the request type of the file data processing request is a file data upload request, the file data processing request carries the file data to be uploaded and processed, and the file server parses the file data processing request to obtain the file data to be uploaded and processed as the first type of file data.

[0084] B1. Determine a first information identifier based on the target file data volume and the file data name of the first category of file data, and perform a hash modulo operation on the first information identifier.

[0085] In one embodiment, the target file data volume and file data name of the first type of file data (file data to be processed) are determined, and a first information identifier is determined based on a character combination of the target file data volume and the file data name. A hash modulo operation is further performed on the first information identifier using a hash modulo algorithm.

[0086] C1. Determine a first storage address from the storage space according to a hash modulo operation result, and store the first type of file data in the storage space according to the first storage address.

[0087] In one embodiment, the hash modulo operation result is used as the first storage address in the storage space, and the first type of file data is stored in the first storage address of the storage space.

[0088] The target virtual processor determines the request type of the file data processing request, and when the request type is a file data upload request, parses the to-be-processed file data from the file data processing request as the first type of file data; determines a first information identifier based on the target file data volume and the file data name of the first type of file data, and performs a hash modulo operation on the first information identifier; determines a first storage address from the storage space based on the hash modulo operation result, and stores the first type of file data in the storage space based on the first storage address. The beneficial effects are:

[0089] First, through hash modulo operation and dynamic address allocation mechanism, the file data is evenly distributed in the storage space.

[0090] Secondly, through the dual judgment rules of the target file data volume and the file data name, the storage address allocation is more in line with the actual data characteristics.

[0091] Thirdly, the first storage address is determined by taking the unique hash modulo operation result of the first information identifier to ensure the uniqueness of file data storage and reduce the risk of data conflict and overwriting.

[0092] Optionally, processing the file data to be processed according to the file data processing request by the target virtual processor includes:

[0093] A2. Determine the request type of the file data processing request through the target virtual processor, and if the request type is a file data download request, parse the file data name of the second type of file data from the file data processing request, and determine the second information identifier based on the file data name of the second type of file data.

[0094] Among them, the second type of file data is file data to be processed.

[0095] When the request type of the file data processing request is a file data download request, the file data processing request carries the file data name of the second type of file data to be downloaded. The file server parses the file data processing request to obtain the file data name of the second type of file data to be downloaded, and uses the file data name of the second type of file data as the second information identifier.

[0096] B2. Perform a hash modulo operation on the second information identifier, and determine a second storage address from the storage space according to the hash modulo operation result.

[0097] In one embodiment, a hash modulo algorithm is used to perform a hash modulo operation on the second information identifier, and the hash modulo operation result is used as the second storage address in the storage space.

[0098] C2. Execute a data download operation on the second type of file data from the storage space according to the second storage address.

[0099] In one embodiment, the second type of file data is downloaded from the storage space according to the second storage address and sent to the client corresponding to the file data processing request.

[0100] The target virtual processor determines the request type of the file data processing request, and when the request type is a file data download request, parses the file data name of the second type of file data from the file data processing request, and determines a second information identifier based on the file data name of the second type of file data; performs a hash modulo operation on the second information identifier, and determines a second storage address from the storage space based on the hash modulo operation result; and performs a data download operation on the second type of file data from the storage space based on the second storage address. The beneficial effects are:

[0101] First, the second storage address is directly mapped through hash modulo operation, avoiding the time-consuming operation of multi-layer index query in traditional file systems.

[0102] Secondly, the second information identifier is strongly bound to the file data name, avoiding data overwriting and loss problems caused by duplication or misparsing of file data names.

[0103] Third, the result of the hash modulo operation directly corresponds to the physical storage space, eliminating the need to maintain complex metadata tables and reducing the risk of file server system crashes.

[0104] Optionally, performing a data download operation on the second type of file data from the storage space according to the second storage address includes:

[0105] A3. Perform a data reading operation on the second type of file data from the storage space according to the second storage address.

[0106] In one embodiment, the storage space is accessed and data is located in the storage space according to the second storage address to determine the second type of file data. If the integrity check of the second type of file data passes, a data read operation is performed on the second type of file data.

[0107] B3. When the data reading is completed, the second type of file data is written into the text data cache.

[0108] Among them, the text data cache is a write cache, a cache mechanism used to improve the efficiency of file data writing. By temporarily storing the file data to be written, the number of direct accesses to low-speed storage devices is reduced, thereby optimizing system performance.

[0109] In one embodiment, it is determined whether the data reading operation of the second type of file data is completed, and if the data reading is completed, the second type of file data is written into the text data cache.

[0110] C3. Read the second type of file data from the text data cache, and send the second type of file data to the target client.

[0111] The target client is a client that sends a file data processing request to the file server.

[0112] In one embodiment, it is determined whether the second type of file data has been written into the text data cache. If so, the second type of file data is read from the text data cache and sent to the target client.

[0113] By performing a data read operation on the second type of file data from the storage space according to the second storage address, writing the second type of file data to the text data cache when the data reading is completed, reading the second type of file data from the text data cache, and sending the second type of file data to the target client, the beneficial effect is:

[0114] First, text data cache acts as an intermediate layer between memory and persistent storage, enabling rapid reuse of high-frequency data.

[0115] Secondly, the asynchronous "read-cache-send" process design eliminates the need for the virtual processor to block and wait for a single storage operation to complete.

[0116] Third, the target client obtains data directly from the text data cache, avoiding transmission jitter caused by fluctuations in the storage device response.

[0117] Optionally, after writing the second type of file data into the text data cache, the method further includes:

[0118] Modify the status information corresponding to the target data channel to the data cache status.

[0119] The status information corresponding to the target data channel reflects the processing status of the target virtual processor on the second type of file data, such as data cache status, data reading status, etc.

[0120] Optionally, reading the second type of file data from the text data cache and sending the second type of file data to the target client includes:

[0121] Determine the state information corresponding to the target data channel, and when it is determined that the state information is a data cache state, read the second type of file data from the text data cache; and send the second type of file data to the target client through the target data channel.

[0122] In one embodiment, the status information corresponding to the target data channel is detected in real time, and when it is determined that the status information corresponding to the target data channel is a data cache state, the second type of file data associated with the identification information is read from the text data cache according to the identification information of the target client, and then the target data channel sends the second type of file data to the target client.

[0123] By determining the state information corresponding to the target data channel and, if the state information is determined to be a data cache state, reading the second type of file data from the text data cache; and sending the second type of file data to the target client through the target data channel, the beneficial effects are:

[0124] First, through the status information judgment mechanism, the read operation is triggered only when the status information is in the data cache state, avoiding invalid polling or repeated requests and reducing network bandwidth usage.

[0125] Secondly, the second type of file data is read directly from the text data cache, eliminating disk I / O or database query steps and reducing response latency.

[0126] Example 3

[0127] Figure 3 This is a structural diagram of a file data processing device provided in the third embodiment of the present invention, which can be applied to the case where a candidate virtual processor set in each candidate physical processor in a file server is called to process file data. The file data processing device is configured in a file server, wherein the file server includes at least one candidate physical processor, and each candidate physical processor is respectively provided with at least two candidate virtual processors. Figure 3 As shown, the device includes:

[0128] The file data type determination module 31 is configured to determine the target file data volume of the file data to be processed according to the acquired file data processing request, and determine the target file data type corresponding to the file data to be processed according to the target file data volume;

[0129] a virtual processor determination module 32 configured to determine a target virtual processor from among the candidate virtual processors based on the target file data type and the candidate processor labels corresponding to the candidate virtual processors; wherein the candidate processor labels are used to identify the file data type that the candidate virtual processor is responsible for processing;

[0130] The file data processing module 33 is configured to forward the file data processing request to the target virtual processor, and process the to-be-processed file data according to the file data processing request through the target virtual processor.

[0131] Optionally, the file data type determination module 31 is specifically configured to:

[0132] Obtaining candidate file data volume intervals respectively associated with at least one candidate file data type, and matching the target file data volume with each candidate file data volume interval;

[0133] The candidate file data volume interval to which the target file data volume belongs is determined according to the matching result as the auxiliary file data volume interval, and the candidate file data type associated with the auxiliary file data volume interval is used as the target file data type.

[0134] Optionally, the virtual processor determination module 32 is specifically configured to:

[0135] Matching the target file data type with each of the candidate processor tags, and using the candidate processor tag that matches the target file data type as the target processor tag;

[0136] The candidate virtual processor corresponding to the target processor label is used as the target virtual processor.

[0137] Optionally, the candidate processor label is generated in the following manner:

[0138] Determining the amount of historical file data according to the historical file data processing requests obtained by the file server within the historical time period, and determining the type of historical file data corresponding to each historical file data according to each amount of historical file data;

[0139] The type ratios corresponding to the data types of the historical files are determined, and the candidate processor labels are generated for the candidate virtual processors according to the type ratios.

[0140] Optionally, the file data processing module 33 is specifically configured to:

[0141] Determining a target data channel associated with the target virtual processor based on an association relationship between the candidate virtual processors and the candidate data channels;

[0142] The file data processing request is forwarded to the target virtual processor using the target data channel.

[0143] Optionally, the file data processing module 33 is further configured to:

[0144] determining, by the target virtual processor, a request type of the file data processing request, and, if the request type is a file data upload request, parsing the to-be-processed file data from the file data processing request as the first category of file data;

[0145] Determine a first information identifier according to the target file data volume and the file data name of the first category of file data, and perform a hash modulo operation on the first information identifier;

[0146] A first storage address is determined from the storage space according to the hash modulo operation result, and the first type of file data is stored in the storage space according to the first storage address.

[0147] Optionally, the file data processing module 33 is further configured to:

[0148] determining, by the target virtual processor, a request type of the file data processing request, and if the request type is a file data download request, parsing a file data name of a second type of file data from the file data processing request, and determining a second information identifier based on the file data name of the second type of file data; wherein the second type of file data is the file data to be processed;

[0149] performing a hash modulo operation on the second information identifier, and determining a second storage address from the storage space according to a result of the hash modulo operation;

[0150] A data download operation is performed on the second-category file data from the storage space according to the second storage address.

[0151] Optionally, the file data processing module 33 is further configured to:

[0152] performing a data reading operation on the second-category file data from the storage space according to the second storage address;

[0153] When the data reading is completed, writing the second type of file data into the text data cache;

[0154] The second type of file data is read from the text data cache, and the second type of file data is sent to a target client; wherein the target client is a client that sends the file data processing request to the file server.

[0155] Optionally, the device further includes a status information modification module, specifically configured to:

[0156] Modify the state information corresponding to the target data channel to a data cache state;

[0157] Optionally, the file data processing module 33 is further configured to:

[0158] Determining state information corresponding to the target data channel, and when determining that the state information is the data cache state, reading the second type of file data from the text data cache;

[0159] The second type of file data is sent to the target client through the target data channel.

[0160] The file data processing device provided by the embodiment of the present invention can execute the file data processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0161] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0162] Example 4

[0163] Figure 4 A schematic diagram of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0164] like Figure 4 As shown, electronic device 40 includes at least one processor 41 and memory, such as read-only memory (ROM) 42 and random access memory (RAM) 43, communicatively connected to at least one processor 41. The memory stores computer programs executable by the at least one processor. Processor 41 can perform various appropriate actions and processes based on the computer programs stored in ROM 42 or loaded from storage unit 48 into RAM 43. RAM 43 can also store various programs and data required for the operation of electronic device 40. Processor 41, ROM 42, and RAM 43 are interconnected via bus 44. An input / output (I / O) interface 45 is also connected to bus 44.

[0165] Multiple components in the electronic device 40 are connected to the I / O interface 45, including an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a magnetic disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0166] Processor 41 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 41 executes the various methods and processes described above, such as the method for processing file data.

[0167] In some embodiments, the file data processing method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the file data processing method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to execute the file data processing method in any other suitable manner (e.g., via firmware).

[0168] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0169] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0170] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0171] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0172] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0173] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0174] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0175] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for processing file data, executed by a file server, characterized in that: The file server includes at least one candidate physical processor, each candidate physical processor is respectively provided with at least two candidate virtual processors, and the method includes: Determining a target file data volume of the file data to be processed according to the acquired file data processing request, and determining a target file data type corresponding to the file data to be processed according to the target file data volume; Determining a target virtual processor from each of the candidate virtual processors based on the target file data type and the candidate processor labels corresponding to each of the candidate virtual processors; wherein the candidate processor labels are used to identify the file data type that the corresponding candidate virtual processor is responsible for processing; forwarding the file data processing request to the target virtual processor, and processing the to-be-processed file data according to the file data processing request by the target virtual processor; The step of determining a target virtual processor from among the candidate virtual processors according to the target file data type and the candidate processor labels corresponding to the candidate virtual processors includes: Matching the target file data type with each of the candidate processor tags, and using the candidate processor tag that matches the target file data type as the target processor tag; The candidate virtual processor corresponding to the target processor label is used as the target virtual processor.

2. The method according to claim 1, characterized in that The determining, based on the target file data volume, the target file data type corresponding to the to-be-processed file data includes: Obtaining candidate file data volume intervals respectively associated with at least one candidate file data type, and matching the target file data volume with each candidate file data volume interval; The candidate file data volume interval to which the target file data volume belongs is determined according to the matching result as the auxiliary file data volume interval, and the candidate file data type associated with the auxiliary file data volume interval is used as the target file data type.

3. The method according to any one of claims 1-2, characterized in that The candidate processor labels are generated as follows: Determining the amount of historical file data according to the historical file data processing requests obtained by the file server within the historical time period, and determining the type of historical file data corresponding to each historical file data according to each amount of historical file data; The type ratios corresponding to the data types of the historical files are determined, and the candidate processor labels are generated for the candidate virtual processors according to the type ratios.

4. The method according to claim 1, wherein The forwarding the file data processing request to the target virtual processor includes: Determining a target data channel associated with the target virtual processor based on an association relationship between the candidate virtual processors and the candidate data channels; The file data processing request is forwarded to the target virtual processor using the target data channel.

5. The method according to claim 1, characterized in that The processing of the to-be-processed file data according to the file data processing request by the target virtual processor includes: determining, by the target virtual processor, a request type of the file data processing request, and, if the request type is a file data upload request, parsing the to-be-processed file data from the file data processing request as the first category of file data; Determine a first information identifier according to the target file data volume and the file data name of the first category of file data, and perform a hash modulo operation on the first information identifier; A first storage address is determined from the storage space according to the hash modulo operation result, and the first type of file data is stored in the storage space according to the first storage address.

6. The method according to claim 4, characterized in that The processing of the to-be-processed file data according to the file data processing request by the target virtual processor includes: determining, by the target virtual processor, a request type of the file data processing request, and if the request type is a file data download request, parsing a file data name of a second type of file data from the file data processing request, and determining a second information identifier based on the file data name of the second type of file data; wherein the second type of file data is the file data to be processed; performing a hash modulo operation on the second information identifier, and determining a second storage address from the storage space according to a result of the hash modulo operation; A data download operation is performed on the second-category file data from the storage space according to the second storage address.

7. The method according to claim 6, characterized in that The performing a data download operation on the second-category file data from the storage space according to the second storage address includes: performing a data reading operation on the second-category file data from the storage space according to the second storage address; When the data reading is completed, writing the second type of file data into the text data cache; The second type of file data is read from the text data cache, and the second type of file data is sent to a target client; wherein the target client is a client that sends the file data processing request to the file server.

8. The method according to claim 7, characterized in that After writing the second type of file data into the text data cache, the method further includes: Modify the state information corresponding to the target data channel to a data cache state; The reading the second-category file data from the text data cache and sending the second-category file data to the target client includes: Determining state information corresponding to the target data channel, and when determining that the state information is the data cache state, reading the second type of file data from the text data cache; The second type of file data is sent to the target client through the target data channel.

9. A file data processing device, configured on a file server, characterized in that: The file server includes at least one candidate physical processor, each candidate physical processor is respectively provided with at least two candidate virtual processors, and the device includes: A file data type determination module is configured to determine a target file data volume of the file data to be processed according to the acquired file data processing request, and determine a target file data type corresponding to the file data to be processed according to the target file data volume; a virtual processor determination module, configured to determine a target virtual processor from each candidate virtual processor based on the target file data type and a candidate processor tag corresponding to each candidate virtual processor; wherein the candidate processor tag is used to identify the file data type that the candidate virtual processor is responsible for processing; A file data processing module, configured to forward the file data processing request to the target virtual processor, and process the to-be-processed file data according to the file data processing request through the target virtual processor; The virtual processor determination module is specifically configured to: Matching the target file data type with each of the candidate processor tags, and using the candidate processor tag that matches the target file data type as the target processor tag; The candidate virtual processor corresponding to the target processor label is used as the target virtual processor.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute the file data processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the file data processing method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method for processing file data according to any one of claims 1 to 8.

Images