Method and system for improving access performance of distributed file system

Abstract

The application discloses a distributed file system access performance improvement method and system, comprising the following steps: firstly, receiving a client access request to form a to-be-processed request sequence; performing access mode analysis on the sequence to obtain an access mode category and a related first access characteristic item, and meanwhile, resolving a second access characteristic item containing suppression characteristic information and a corresponding suppression scope; determining a target access characteristic item belonging to the suppression scope from the first access characteristic item, adjusting the access mode category in combination with the second characteristic item, and obtaining a target access mode judgment result; determining a file system performance optimization type according to the result, generating and triggering an execution corresponding resource configuration or strategy adjustment instruction. The method improves the optimization strategy adaptability by combining the access mode and the suppression characteristic optimization mode judgment, and effectively enhances the distributed file system access performance.

Description

Technical Field

[0001] This invention relates to the field of distributed systems, and more specifically, to a method and system for improving the access performance of a distributed file system. Background Technology

[0002] Distributed file systems are a key technology in data storage and management, and their access performance directly determines system efficiency. Existing optimization schemes for distributed file systems mostly rely on single-pattern analysis of access request sequences to formulate strategies. However, in real-world scenarios, access request sequences often contain suppression features, which can interfere with the accurate determination of access patterns. Because existing technologies do not fully consider suppression features and their corresponding scope, they are prone to access pattern identification biases, resulting in insufficient adaptability of subsequent performance optimization strategies. This makes it difficult to effectively cope with complex access scenarios and achieve precise improvements in the access performance of distributed file systems. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for improving the access performance of a distributed file system.

[0004] In a first aspect, embodiments of the present invention provide a method for improving the access performance of a distributed file system, including:

[0005] Receive access requests for the distributed file system from clients and form a sequence of system access requests to be processed;

[0006] The system access request sequence is subjected to access pattern analysis to obtain the access pattern category of the system access request sequence, and one or more first access feature items related to the access pattern category are obtained from the system access request sequence.

[0007] The system access request sequence is subjected to suppression feature parsing processing to parse out a second access feature item containing suppression feature information and the suppression scope corresponding to the second access feature item from the system access request sequence;

[0008] From the one or more first access feature items, a target access feature item that belongs to the inhibition domain is determined, and the access mode category is adjusted based on the second access feature item and the target access feature item to obtain the target access mode determination result of the system access request sequence;

[0009] Based on the target access mode determination result, at least one corresponding file system performance optimization type is determined;

[0010] Based on the determined file system performance optimization type, generate and trigger the execution of corresponding resource configuration instructions or policy adjustment instructions to optimize the access performance of the distributed file system.

[0011] In one possible implementation, the access pattern analysis processing of the system access request sequence to obtain the access pattern category of the system access request sequence includes:

[0012] Obtain the access feature vector corresponding to the system access request sequence; the access feature vector includes a pattern discrimination feature element corresponding to the system access request sequence, and the pattern discrimination feature element is used to indicate the access feature state of the system access request sequence;

[0013] The system access request sequence is analyzed using the pattern discrimination feature elements in the access feature vector to obtain the access pattern category of the system access request sequence.

[0014] In one possible implementation, the system access request sequence includes at least one request unit; the access feature vector corresponding to the system access request sequence further includes a request unit feature vector for each request unit in the at least one request unit, wherein the request unit feature vector of any request unit is used to indicate the request unit feature state of any request unit; the step of obtaining one or more first access feature items related to the access mode category from the system access request sequence includes:

[0015] Obtain multiple undetermined feature dimensions corresponding to the access mode category; the multiple undetermined feature dimensions are predefined.

[0016] Using the request unit feature vector of each request unit, a request unit subsequence that matches one or more of the plurality of undetermined feature dimensions is parsed from the at least one request unit; the request unit subsequence consists of at least one request unit.

[0017] The request unit subsequence that matches each of the one or more undetermined feature dimensions is taken as the feature dimension value of the corresponding undetermined feature dimension;

[0018] Wherein, one or more first access feature items related to the access mode category are the values ​​of one or more feature dimensions of the one or more undetermined feature dimensions.

[0019] In one possible implementation, the system access request sequence includes at least one request unit; the step of performing suppression feature parsing processing on the system access request sequence to parse out a second access feature item containing suppression feature information and the suppression scope corresponding to the second access feature item includes:

[0020] Obtain the access feature vector corresponding to the system access request sequence; the access feature vector includes the request unit feature vector of each request unit in the at least one request unit, and the request unit feature vector of any request unit is used to indicate the request unit feature state of any request unit;

[0021] Based on the request unit feature vector of each request unit in the access feature vector, a second access feature term containing suppression feature information is parsed from the system access request sequence; and,

[0022] Based on the request unit feature vector of each request unit in the access feature vector, an association analysis is performed on each request unit included in the system access request sequence to obtain the inhibition domain corresponding to the second access feature item in the system access request sequence; the inhibition domain includes at least one request unit subsequence in the system access request sequence, and the request unit subsequence is composed of at least one request unit.

[0023] In one possible implementation, obtaining the access feature vector corresponding to the system access request sequence includes:

[0024] The system access request sequence is supplemented with feature elements to obtain a supplemented system access request unit sequence; the supplemented system access request unit sequence includes: pattern discrimination feature elements and at least one request unit included in the system access request sequence;

[0025] The system access request unit sequence is subjected to access feature mapping processing to obtain the access feature mapping result of the system access request unit sequence;

[0026] Based on the access feature mapping result, an access feature extraction operation is performed on the system access request unit sequence to obtain the access feature vector corresponding to the system access request sequence; the access feature vector includes: the pattern discrimination feature element corresponding to the pattern discrimination feature element, and the request unit feature vector corresponding to each request unit in the at least one request unit.

[0027] In one possible implementation, the suppression domain includes at least one suppression feature dimension value corresponding to the second access feature; the step of determining a target access feature item classified into the suppression domain from the one or more first access features items, and adjusting the access pattern category based on the second access feature item and the target access feature item to obtain the target access pattern determination result of the system access request sequence includes:

[0028] From the one or more first access features, determine the target access feature that has the same value as the suppression feature dimension within the suppression domain, and replace the target access feature with the suppression feature dimension value.

[0029] Based on the second access feature item containing suppression feature information, the replaced suppression feature dimension value and other access feature items in the one or more first access feature items excluding the target access feature item, the access mode category is adjusted to obtain the target access mode determination result of the system access request sequence.

[0030] The access mode corresponding to the target access mode determination result is different from the access mode corresponding to the access mode category.

[0031] In one possible implementation, the method further includes:

[0032] The system access request sequence is subjected to access mode suppression determination processing.

[0033] If the access mode corresponding to the access mode category of the system access request sequence is non-suppression feature information, then return to the step of determining the target access feature item classified into the suppression domain from the one or more first access feature items, and adjusting the access mode category based on the second access feature item and the target access feature item to obtain the target access mode determination result of the system access request sequence;

[0034] The corresponding system access processing is executed based on the target access mode determination result of the system access request sequence.

[0035] In one possible implementation, the method is performed by a suppression feature processing ensemble model; the suppression feature processing ensemble model includes at least: an access feature joint extraction unit, an access strategy determination unit, a feature dimension completion unit, and an access constraint prediction unit;

[0036] The joint extraction unit for access features is used to obtain the access feature vector corresponding to the system access request sequence;

[0037] The access policy determination unit is used to perform access pattern analysis on the system access request sequence to obtain the access pattern category of the system access request sequence.

[0038] The feature dimension completion unit is used to obtain one or more first access feature items related to the access mode category from the system access request sequence;

[0039] The access constraint prediction unit is used to perform suppression feature parsing processing on the system access request sequence, and to parse out a second access feature item containing suppression feature information and the suppression scope corresponding to the second access feature item from the system access request sequence.

[0040] In one possible implementation, the suppressed feature processing ensemble model is obtained through the following methods:

[0041] Obtain a historical access sample dataset and the corresponding multidimensional access annotation information; the multidimensional access annotation information corresponding to the historical access sample dataset includes access pattern category annotation information, feature dimension value labels, and suppression labels; the suppression label includes a second access feature item containing suppression feature information in the historical access sample dataset and the suppression scope corresponding to the second access feature item.

[0042] Using the access strategy determination unit in the aforementioned suppression feature processing ensemble model, semantic prediction processing is performed on the historical access sample dataset based on the access feature vector to obtain the access pattern category determination output of the historical access sample dataset; and,

[0043] Using the feature dimension completion unit in the aforementioned suppressed feature processing ensemble model, feature dimension value prediction processing is performed on the historical access sample dataset based on the access feature vector, to obtain the feature dimension value judgment output related to the access pattern category judgment output in the historical access sample dataset; and...

[0044] Using the access constraint prediction unit in the suppression feature processing ensemble model, suppression feature information is predicted on the historical access sample dataset based on the access feature vector, and the suppression access mode determination output of the historical access sample dataset is obtained. The suppression access mode determination output includes: suppression prediction access feature item and the prediction suppression scope corresponding to the suppression prediction access feature item.

[0045] Based on the access pattern category determination output of the historical access sample dataset and the access pattern category annotation information of the historical access sample dataset, the access pattern category cost function of the historical access sample dataset is obtained; and,

[0046] Based on the feature dimension value determination output and the feature dimension value label of the historical access sample dataset, the value padding cost function of the historical access sample dataset is obtained; and,

[0047] Based on the suppression access pattern determination output of the historical access sample dataset and the suppression label of the historical access sample dataset, the suppression cost function of the historical access sample dataset is obtained.

[0048] The access mode category cost function, the value completion cost function, and the suppression cost function are weighted and processed to obtain the target cost function;

[0049] Based on the optimization objective of converging the target cost function, the suppression feature processing ensemble model is optimized to obtain the optimized suppression feature processing ensemble model.

[0050] In a second aspect, embodiments of the present invention provide a server system, including a server, the server being used to perform the method described in the first aspect.

[0051] Compared to existing technologies, the beneficial effects provided by this invention include: Using the distributed file system access performance improvement method and system disclosed in this invention, a sequence of requests to be processed is formed by receiving client access requests; access pattern analysis is performed on this sequence to obtain access pattern categories and related first access feature items, while simultaneously parsing out second access feature items containing suppression feature information and their corresponding suppression scopes; target access feature items belonging to the suppression scope are determined from the first access feature items, and the access pattern category is adjusted in conjunction with the second feature items to obtain a target access pattern determination result; based on this result, the file system performance optimization type is determined, and corresponding resource configuration or policy adjustment instructions are generated and triggered for execution. This method improves the adaptability of optimization strategies by combining access pattern and suppression feature optimization pattern determination, effectively enhancing the access performance of distributed file systems. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating the steps of a method for improving the access performance of a distributed file system provided in an embodiment of the present invention;

[0054] Figure 2 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0056] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0057] In order to solve the technical problems mentioned in the background art Figure 1 This is a flowchart illustrating a method for improving the access performance of a distributed file system according to an embodiment of this disclosure. The following is a detailed description of this method for improving the access performance of a distributed file system.

[0058] Step S201: Receive access requests for the distributed file system from the client and form a sequence of system access requests to be processed;

[0059] Step S202: Perform access pattern analysis processing on the system access request sequence to obtain the access pattern category of the system access request sequence, and obtain one or more first access feature items related to the access pattern category from the system access request sequence;

[0060] Step S203: Perform suppression feature parsing processing on the system access request sequence to parse out the second access feature item containing suppression feature information and the suppression scope corresponding to the second access feature item from the system access request sequence;

[0061] Step S204: Determine the target access feature item belonging to the inhibition domain from the one or more first access feature items, and adjust the access mode category based on the second access feature item and the target access feature item to obtain the target access mode determination result of the system access request sequence;

[0062] Step S205: Based on the target access mode determination result, determine at least one corresponding file system performance optimization type;

[0063] Step S206: Based on the determined file system performance optimization type, generate and trigger the execution of corresponding resource configuration instructions or policy adjustment instructions to optimize the access performance of the distributed file system.

[0064] In this embodiment of the invention, for example, the server continuously receives file system access requests from multiple clients, such as data reads and metadata queries. The server does not process each individual request immediately, but instead temporarily stores and organizes them into a sequence of system access requests to be processed. This sequence can be formed based on a fixed time window or the accumulated number of requests.

[0065] After receiving a sequence of system access requests to be processed, the server first performs access pattern analysis. This process aims to determine the main access tendencies exhibited by the sequence from a macroscopic perspective. The server extracts overall features of the sequence, such as the distribution of request types, the continuity of access offsets, and statistical information on the size of requested data, and transforms them into a multi-dimensional access feature vector. This feature vector is then analyzed using a pre-defined pattern classification model (e.g., a machine learning-based classifier) ​​to obtain the initial access pattern category of the system access request sequence. For example, if most requests in the sequence are large blocks of data read from contiguous storage addresses, it might be classified as a "sequential large block read" pattern. Simultaneously, based on this determined pattern category, the server determines several key feature dimensions associated with it from predefined rules (e.g., the starting address, span, and request size of sequential access), and parses specific values ​​from the request sequence that conform to these dimensions as one or more first access feature items related to the current pattern category.

[0066] In parallel or sequential processes, the server performs suppression feature parsing on the same system access request sequence. The purpose of this process is to identify local features in the sequence that may interfere with or correct the aforementioned primary access pattern determination. The server analyzes the features of each request unit in the sequence and its contextual relationships to parse out a second access feature containing suppression feature information. For example, in a sequence that predominantly exhibits sequential reads, a sudden small random read request to a distant address might be parsed as a suppression feature, its information possibly characterized as a "long-distance offset jump." Further, the server analyzes the scope of influence of this suppression feature, i.e., its corresponding suppression domain. The suppression domain defines the range of primary pattern features affected by the suppression feature; for example, it might indicate that the jump request reduces the reliability of the "sequentiality" represented by the previous continuous reads when predicting future requests, and its influence might cover a specific address range previously affected.

[0067] Subsequently, the server fuses and adjusts the pattern determination results. It compares one or more first access feature items obtained from the main pattern analysis with the suppression scope obtained from the suppression feature parsing. From the first access feature items, the server filters out those whose values ​​or described ranges fall within the suppression scope and identifies them as target access feature items. Next, based on the suppression feature information (second access feature items), the server modifies or re-evaluates the local features of the main pattern represented by these target access feature items, and adjusts the initial access pattern category by combining other unsuppressed features in the sequence. For example, the initially determined "sequential large block read" pattern is adjusted to a composite pattern of "mainly sequential large block reads, but interspersed with local random jumps," which is the final target access pattern determination result for the system's access request sequence.

[0068] After obtaining accurate results regarding the target access pattern, the server determines the appropriate file system performance optimization type based on the pre-configured mapping relationship. Different composite patterns correspond to different optimization focuses. For example, for the "main sequence mixed with random jumps" pattern mentioned above, the optimization type may simultaneously include "limited data prefetching" and "adaptive caching strategy adjustment," rather than purely aggressive prefetching or completely conservative caching.

[0069] Finally, based on the determined optimization type, the server generates specific control instructions and triggers their execution. These instructions can be resource configuration instructions sent to storage nodes, such as adjusting the prefetch window size for specific files or data blocks; or policy adjustment instructions sent to the cache management module, such as modifying the weight parameters regarding access locality and frequency in the cache eviction algorithm. By executing these instructions, the behavior of the distributed file system in data prefetching, cache management, I / O scheduling, etc., is dynamically optimized, thereby more accurately adapting to actual load characteristics and ultimately improving the overall access performance and resource utilization of the system.

[0070] In this embodiment of the invention, the access pattern analysis processing of the system access request sequence to obtain the access pattern category of the system access request sequence can be implemented through the following example.

[0071] Obtain the access feature vector corresponding to the system access request sequence; the access feature vector includes a pattern discrimination feature element corresponding to the system access request sequence, and the pattern discrimination feature element is used to indicate the access feature state of the system access request sequence;

[0072] The system access request sequence is analyzed using the pattern discrimination feature elements in the access feature vector to obtain the access pattern category of the system access request sequence.

[0073] In this embodiment of the invention, the foregoing technical solution will be described in detail below with reference to specific scenarios, for example. In this embodiment, the executing entity is a server on the distributed file system management side, which continuously receives and analyzes the access request stream from the client.

[0074] The server first obtains a sequence of system access requests to be processed. This sequence consists of multiple access requests received within a certain period of time, arranged sequentially. For example, in a data analysis task, the sequence collected by the server might include: request A (reading file / data / log1.txt, offset 0, length 1MB), request B (reading file / data / log1.txt, offset 1MB, length 1MB), request C (reading file / data / log1.txt, offset 2MB, length 1MB), and request D (reading file / data / config.ini, offset 0, length 4KB).

[0075] Next, the server performs access feature vectorization on the sequence. The server iterates through each request unit in the sequence, extracting its key feature attributes, such as operation type (e.g., read, write), target file identifier, access offset, request data length, and time interval with the previous request. These features, after normalization or encoding, constitute the feature sub-vector corresponding to each request unit. Simultaneously, the server calculates and generates one or more pattern-discriminating feature elements for the entire sequence. These elements are mathematical summaries of the overall access feature state of the sequence. For example, the server might calculate the "average offset increment" of all read requests in the sequence; if this value is consistently positive and has a small variance, it indicates a strong sequential access state; or it might calculate the "entropy of accessed file IDs," where a low entropy value indicates that access is concentrated on a few files. In this example, for requests A, B, and C, their offsets increase sequentially (0->1MB->2MB), and the file IDs are the same, with a fixed request length. Therefore, the calculated "average offset increment" is 1MB, and the "offset increment variance" is close to 0. The occurrence of request D, due to its access to different files (config.ini) and its small request length, causes changes in statistical features such as "file ID entropy" and "request size variance." All these calculated statistics (mean offset increment, variance, entropy, etc.) collectively constitute the pattern-discriminating feature elements of this request sequence and are integrated into the access feature vector of the entire sequence. This access feature vector is a comprehensive data structure that contains both detailed feature sub-vectors for each request unit and pattern-discriminating feature elements condensed from a global perspective.

[0076] Subsequently, the server uses this access feature vector for access pattern analysis. Internally, the server deploys a trained access pattern classification model (e.g., a classifier based on a multilayer perceptron or support vector machine). The server inputs this access feature vector, particularly its highly generalized pattern-discriminating features, into this classification model. The model maps these feature elements to a predefined access pattern category space through weighted analysis and nonlinear transformation. For example, when the "average offset increment" value is stable and positive, the "offset increment variance" is extremely low, the "file ID entropy" is low, and the "request size variance" is low, the model outputs "strong sequential reads" as the access pattern category. In this example of a mixed sequence, while requests A, B, and C exhibit statistically dominant strong sequential characteristics, request D introduces different file accesses and a very small request size, causing an increase in "file ID entropy" and "request size variance." After comprehensively weighing these interrelated pattern-discriminating features, the model might determine the overall access pattern category of the sequence as "primarily sequential large-block reads, mixed with a small amount of metadata / configuration reads." This judgment is not a description of a single request, but a qualitative conclusion drawn by the server regarding the macroscopic I / O behavior tendencies of the current batch of requests, providing a crucial basis for subsequent performance optimization decisions.

[0077] In this embodiment of the invention, the system access request sequence includes at least one request unit; the access feature vector corresponding to the system access request sequence also includes the request unit feature vector of each request unit in the at least one request unit, and the request unit feature vector of any request unit is used to indicate the request unit feature state of any request unit; the step of obtaining one or more first access feature items related to the access mode category from the system access request sequence can be implemented through the following example.

[0078] Obtain multiple undetermined feature dimensions corresponding to the access mode category; the multiple undetermined feature dimensions are predefined.

[0079] Using the request unit feature vector of each request unit, a request unit subsequence that matches one or more of the plurality of undetermined feature dimensions is parsed from the at least one request unit; the request unit subsequence consists of at least one request unit.

[0080] The request unit subsequence that matches each of the one or more undetermined feature dimensions is taken as the feature dimension value of the corresponding undetermined feature dimension;

[0081] Wherein, one or more first access feature items related to the access mode category are the values ​​of one or more feature dimensions of the one or more undetermined feature dimensions.

[0082] In an embodiment of the invention, for example, after the server completes the access pattern analysis of the system access request sequence and obtains its access pattern category (e.g., "sequential read"), it then executes the step of obtaining one or more first access feature items related to that category. The core of this process is to transform the macroscopic pattern category into specific, quantifiable feature dimension values, which precisely describe the key parameters constituting the pattern.

[0083] First, based on the determined access mode category, the server retrieves multiple pending feature dimensions corresponding to that category from a predefined policy configuration library. These dimensions are key parameters pre-defined based on the technical implications of various access modes. For example, for the "sequential read" category, its predefined pending feature dimensions typically include: "starting logical offset of sequential access", "ending logical offset of sequential access", and "request data block size of sequential access".

[0084] Next, the server uses the request unit feature vectors, which were generated during the construction of the access feature vectors and are used to indicate the feature state of each request unit, to perform deep parsing of the original request sequence. Each request unit feature vector contains encoded information such as the file identifier, offset, and length of the request. The server designs an algorithm to scan the request units in the sequence sequentially, aiming to find one or more consecutive request unit subsequences, where the characteristic behavior of the requests within these subsequences must highly match the pattern described by the aforementioned undetermined feature dimensions.

[0085] Specifically, the server analyzes the request unit feature vector starting from the first request unit in the sequence. It examines the vectors of subsequent requests to determine if they satisfy the core constraints of sequential reading: the file identifiers are the same, and the starting offset of the current request equals the starting offset of the previous request plus its length. When the server finds a request breaking this continuity during the scan (e.g., jumping to another file, or offset backtracking), it considers the currently constructed sequential subsequence to have ended. In this way, the server can parse a clean, sequentially read request unit subsequence from a mixed sequence that may contain other stray requests.

[0086] Then, the server transforms the parsed request unit subsequence into specific feature dimension values. It reads the starting offset of the first request in the subsequence and uses it as the "starting logical offset for sequential access." It reads the ending offset (starting offset + request length) of the last request in the subsequence and uses it as the "ending logical offset for sequential access." Simultaneously, it calculates the mode or average of the data lengths of all requests in the subsequence and uses it as the "request data block size for sequential access."

[0087] Finally, the server assigns values ​​to these calculated feature dimensions—start offset = a certain value, end offset = a certain value, request block size = a certain value—as one or more first access feature items related to the "sequential read" access pattern category, obtained from the current system access request sequence. These feature items are no longer a simple list of the original requests, but a precise extraction of the key parameters of the pattern, providing a structured data foundation for subsequently judging the scope of influence of the pattern and the possible suppression it may be subject to.

[0088] In this embodiment of the invention, the system access request sequence includes at least one request unit; the suppression feature parsing process performed on the system access request sequence to parse out a second access feature item containing suppression feature information and the suppression scope corresponding to the second access feature item can be implemented through the following example.

[0089] Obtain the access feature vector corresponding to the system access request sequence; the access feature vector includes the request unit feature vector of each request unit in the at least one request unit, and the request unit feature vector of any request unit is used to indicate the request unit feature state of any request unit;

[0090] Based on the request unit feature vector of each request unit in the access feature vector, a second access feature term containing suppression feature information is parsed from the system access request sequence; and,

[0091] Based on the request unit feature vector of each request unit in the access feature vector, an association analysis is performed on each request unit included in the system access request sequence to obtain the inhibition domain corresponding to the second access feature item in the system access request sequence; the inhibition domain includes at least one request unit subsequence in the system access request sequence, and the request unit subsequence is composed of at least one request unit.

[0092] In this embodiment of the invention, for example, the server performs suppression feature parsing processing in parallel while or after performing access pattern analysis processing on the system access request sequence. The purpose of this processing is to identify and quantify local abnormal features in the sequence that may interfere with or correct the main access pattern determination.

[0093] The server first obtains the access feature vector corresponding to the constructed system access request sequence. This vector contains the request unit feature vector of each request unit in the sequence, and each vector precisely encodes the corresponding request's operation type, target file, logical offset, data length, and other characteristic states.

[0094] Next, the server scans and analyzes the entire sequence based on these fine-grained request unit feature vectors to parse out the second access feature item containing suppression feature information. For example, in a video editing scenario, the request sequence received by the server from time points T1 to T4 is as follows: reading data blocks of 0-4MB, 4-8MB, 8-12MB, and 12-16MB from the file Video.mp4, exhibiting a clear sequential reading characteristic. However, at time point T5, the server receives a request: reading data blocks of 100-104MB from the file Video.mp4. The server analyzes the request unit feature vector of this request and finds that although its file identifier is the same as the previous request, there is a huge positive jump (84MB) between its starting offset (100MB) and the ending offset (16MB) of the previous request. The server identifies this "large-span positive jump read" as a typical suppression feature and abstracts it as the second access feature item, the core information of which is "a long-distance sequential stream interruption has occurred".

[0095] Next, the server needs to determine the scope of influence of this suppression feature, i.e., the suppression domain. Based on the request unit feature vectors of all request units, the server performs correlation analysis on each unit in the sequence. It determines that the jump request at time T5 significantly reduces the reliability of the "sequentiality" exhibited by the consecutive requests from time T1 to T4 in predicting future access addresses. Specifically, this jump indicates that the application's access logic is not strictly linear; therefore, the persistence of the sequential access pattern that just occurred before the jump point is questionable. By analyzing the correlation between request units in file identifiers and offsets, the server determines that the second access feature suppresses the local pattern represented by the subsequence of request units from T1 to T4 that access the 0-16MB region of the Video.mp4 file. Therefore, the server identifies the "sequence of sequential reads between offsets 0 and 16MB of the Video.mp4 file" as the suppression domain corresponding to this second access feature. This domain clearly indicates the specific object and scope of the suppression feature's influence, providing precise input for subsequent fusion and judgment with the main pattern feature.

[0096] In this embodiment of the invention, obtaining the access feature vector corresponding to the system access request sequence can be performed through the following example.

[0097] The system access request sequence is supplemented with feature elements to obtain a supplemented system access request unit sequence; the supplemented system access request unit sequence includes: pattern discrimination feature elements and at least one request unit included in the system access request sequence;

[0098] The system access request unit sequence is subjected to access feature mapping processing to obtain the access feature mapping result of the system access request unit sequence;

[0099] Based on the access feature mapping result, an access feature extraction operation is performed on the system access request unit sequence to obtain the access feature vector corresponding to the system access request sequence; the access feature vector includes: the pattern discrimination feature element corresponding to the pattern discrimination feature element, and the request unit feature vector corresponding to each request unit in the at least one request unit.

[0100] In this embodiment of the invention, for example, before initiating in-depth analysis of the system access request sequence, the server first needs to construct a data structure that can comprehensively characterize the state of the sequence, namely, an access feature vector. This process is not simply packaging the raw data, but a standardized processing flow that includes feature completion, mapping, and extraction.

[0101] The server first performs feature element completion. The original request sequence only contains discrete request units from the client. The server supplements this sequence with global contextual features, namely pattern-discriminating feature elements. For example, in a request sequence for a data analysis task, the server inserts a special feature element at the beginning of the sequence. The initial value of this element is calculated by the server based on preliminary statistics of the sequence, such as calculating the "mean of offset increments" and "file access concentration index" for all read requests in the sequence. These calculated statistical values ​​serve as the initial content of the pattern-discriminating feature element, which, together with the original request units (such as reading 0-1MB and 1-2MB of file A, and 0-4KB of file B), constitutes a completed system access request unit sequence. This sequence is a structured data object, with the header containing feature elements representing the overall state, followed by a list of specific request units.

[0102] Subsequently, the server performs access feature mapping processing on the completed sequence. The server inputs this structured sequence into a pre-defined feature mapping model (e.g., a trained Transformer encoder). This model performs contextual encoding on each element in the sequence (including the pattern-discriminating feature elements in the header and each subsequent request unit). Through its internal self-attention mechanism, the model analyzes the sequential relationships and offset correlations between request units, and allows the pattern-discriminating feature elements in the header to interact with all request units, thereby absorbing the macroscopic pattern information of the entire sequence. After multiple layers of computation by the model, each element in the sequence is transformed into a high-dimensional vector representation rich in contextual semantics. This set of new vector representations of all elements constitutes the access feature mapping result. In this result, the pattern-discriminating feature element vector in the header is no longer just an initial simple statistical value, but a deeply refined and concise expression that summarizes the overall access feature state of the sequence.

[0103] Finally, the server performs access feature extraction based on the mapping result. The server extracts the vector corresponding to the original header position from the mapping result, using it as the pattern discrimination feature element in the final access feature vector. Simultaneously, the server extracts the vectors corresponding to the positions of each original request unit from the mapping result, using them as the final request unit feature vector for each request unit. The server assembles these two extracted vectors according to a predetermined format to generate a unified access feature vector corresponding to the system's access request sequence. This vector, as an integrated and standardized feature package, is simultaneously provided to the subsequent access pattern analysis module and suppression feature parsing module, ensuring that both analysis processes are based on the same deeply processed feature data, laying the foundation for consistent judgments.

[0104] In this embodiment of the invention, the suppression domain includes at least one suppression feature dimension value corresponding to the second access feature item; the step of determining the target access feature item classified into the suppression domain from the one or more first access feature items, and adjusting the access mode category based on the second access feature item and the target access feature item to obtain the target access mode determination result of the system access request sequence includes: it can be implemented through the following examples.

[0105] From the one or more first access features, determine the target access feature that has the same value as the suppression feature dimension within the suppression domain, and replace the target access feature with the suppression feature dimension value.

[0106] Based on the second access feature item containing suppression feature information, the replaced suppression feature dimension value and other access feature items in the one or more first access feature items excluding the target access feature item, the access mode category is adjusted to obtain the target access mode determination result of the system access request sequence.

[0107] The access mode corresponding to the target access mode determination result is different from the access mode corresponding to the access mode category.

[0108] In an embodiment of the present invention, for example, after the server obtains the access mode category (and its associated first access feature) and second access feature (and its suppression domain) of the system access request sequence in parallel, it enters the fusion and adjustment stage of the mode determination result.

[0109] The server first interprets the suppression scope. This scope is expressed as at least one value for a suppression feature dimension. For example, in a scenario involving a mix of video streaming and fast-forwarding, the server has previously determined the second access feature to be "long-distance backward jump read". Its corresponding suppression scope is quantified as: {Influence pattern: "sequential read", influence start offset: 50MB, influence end offset: 200MB}. This means that the jump behavior primarily suppresses the reliability of the sequential read pattern that may be exhibited within the address range of 50MB to 200MB in the file.

[0110] Next, the server compares the suppression scope with the first access feature obtained from the main pattern analysis. Assuming the initial access pattern category is determined to be "sequential read," its corresponding first access feature is: {sequential start offset: 50MB, sequential end offset: 250MB, requested block size: 4MB}. The server performs a matching operation and finds that the values ​​"sequential start offset: 50MB" and "sequential end offset: 250MB" in the first access feature overlap with the "influence start offset: 50MB" and "influence end offset: 200MB" defined in the suppression scope. Specifically, the "sequential start offset" and "influence start offset" are exactly the same, while the "sequential end offset" exceeds the range of the "influence end offset." Based on this, the server determines the "sequential start offset: 50MB" and "sequential end offset: 250MB" in the first access feature as the target access feature.

[0111] Subsequently, the server performs feature replacement and pattern adjustment. It replaces the target access feature item's sequential end offset: 250MB with a more conservative suppression feature dimension value provided by the suppression scope, affecting the end offset: 200MB. After the replacement, the feature value used to describe the sequential read range is corrected from [50MB, 250MB] to [50MB, 200MB]. Then, the server makes a global judgment based on the following three pieces of information: 1) the suppression feature information contained in the second access feature item ("long-distance backward jump read"); 2) the replaced suppression feature dimension value (affecting the end offset: 200MB), which indicates that after 200MB, the original sequential expectation is interrupted due to the jump; 3) other first access feature items that were not replaced (request block size: 4MB), which indicates that the access granularity remains stable.

[0112] Based on the fusion analysis of this information, the server adjusted the initial "sequential read" category. It determined that the actual access pattern was not a complete sequential read from 50MB to 250MB, but rather a sequential read within the 50MB to 200MB range, followed by random jumps in application behavior. Therefore, the server concluded that the final target access pattern was either "Segmented Sequential Read" or a "mixed pattern of sequential reads and random jumps." This result indicates a significantly different access pattern in terms of continuity and predictability compared to the initial "sequential read" category, providing crucial information for triggering more precise performance optimization strategies that balance prefetching and caching flexibility.

[0113] In this embodiment of the invention, the method further includes:

[0114] The system access request sequence is subjected to access mode suppression determination processing.

[0115] If the access mode corresponding to the access mode category of the system access request sequence is non-suppression feature information, then return to the step of determining the target access feature item classified into the suppression domain from the one or more first access feature items, and adjusting the access mode category based on the second access feature item and the target access feature item to obtain the target access mode determination result of the system access request sequence;

[0116] The corresponding system access processing is executed based on the target access mode determination result of the system access request sequence.

[0117] In this embodiment of the invention, for example, after the server completes the access pattern analysis of the system access request sequence and obtains its initial access pattern category, it initiates a logical determination process to determine the subsequent processing path. The core of this process is to determine whether the currently determined pattern itself already contains inhibitory features that require special handling.

[0118] The server first performs a suppression access mode determination process. The server examines the nature of the determined access mode category itself. In a predefined mode classification system, some categories are marked as inherently non-suppressive, regular modes, such as "pure sequential reads," "pure random reads," or "stable full file scans." Other categories are marked as inherently containing suppression or conflicting characteristics, such as "mixed sequential and random access" or "reads with periodic bounces." The server completes this determination by querying an internal mapping table.

[0119] Next, the server performs branching processing based on the determination result. If the access pattern corresponding to the access pattern category of the system access request sequence is determined to be "non-suppressed feature information," this means that the initial pattern determination result is relatively simple and stable, and no obvious internal conflict features are found. In this case, the server believes that there is no need to immediately perform complex feature fusion and adjustment, but in order to ensure the integrity of subsequent processing, its logical flow will return to the aforementioned step of "determining the target access feature item classified into the suppression domain from the one or more first access feature items, and adjusting the access pattern category based on the second access feature item and the target access feature item to obtain the target access pattern determination result of the system access request sequence." In fact, under this branch, since the suppression feature parsing process usually does not parse out a meaningful second access feature item (or the item is empty), the suppression domain may also be empty. Therefore, in this step, the server cannot determine a valid target access feature item from the first access feature items, and the adjustment operation will not actually change the initial access pattern category. Finally, the target access pattern determination result will be consistent with the initial access pattern category.

[0120] Subsequently, the server executes the corresponding system access processing based on the target access pattern determination result of the system access request sequence. For example, in a video streaming service scenario, the server analyzes the request sequence within a certain time window. All requests are continuous reads of the same video file with strictly increasing offsets, and the initial pattern category is determined to be "pure sequential read". After suppressing the access pattern determination process, it is confirmed that this pattern is non-suppressed feature information. After going through the above return process, the target access pattern determination result remains "pure sequential read". The server then generates and triggers an aggressive sequential pre-read instruction based on this result, instructing the storage node to preload data blocks of subsequent video segments into the cache, thereby optimizing the smoothness of streaming media playback.

[0121] Conversely, if the initially determined pattern category is inherently marked as containing suppression features (e.g., directly determined as a "mixed pattern"), or if it is determined as a regular pattern but the suppression feature parsing process discovers a strong second access feature, the server will not enter the aforementioned return loop. Instead, it will directly generate a new target access pattern determination result through feature fusion based on the aforementioned main process, and execute more adaptive system access processing accordingly. This mechanism ensures the completeness and efficiency of the processing logic, enabling a rapid response to simple and clear access patterns, while providing thorough and refined analysis for complex patterns.

[0122] In this embodiment of the invention, the method further includes:

[0123] If the access mode corresponding to the access mode category of the system access request sequence is suppression feature information, then the access mode category of the system access request sequence is taken as the target access mode determination result of the system access request sequence.

[0124] In an embodiment of the present invention, for example, after the server performs the access suppression mode determination process, there is another explicit branch situation. The core logic of this process is that the server's predefined access mode category library not only includes conventional modes such as "sequential read" and "random read" that represent single and stable behaviors, but also some composite mode categories that directly describe complex, conflicting, or abnormal behaviors. These categories themselves are marked as suppression feature information.

[0125] When the server analyzes the current system access request sequence and initially determines that its access pattern category happens to belong to this type of category marked as suppression feature information, the server will execute the aforementioned processing flow. For example, in a mixed workload scenario of database archiving and real-time querying, the server receives a sequence of requests that appear intermittently within a short period of time: Request A (sequentially writes a large amount of archived data to the file / db / archive.bak), Request B (randomly reads a small amount of index data from the file / db / index.idx), Request C (continues sequentially writing to / db / archive.bak), and Request D (randomly reads / db / index.idx). The server processes this sequence through its pattern analysis model, which comprehensively evaluates multiple conflict features such as drastic switching of operation types (write and read), frequent jumps in access targets (two different files), and regular differences in I / O size (large blocks of continuous writes and small blocks of random reads). Based on these highly inconsistent feature combinations, the model directly determines that the access pattern category of this sequence is "high-intensity read-write mixed conflict pattern". The definition of this category in the pattern library clearly indicates that it contains severe access behavior conflicts and mutual interference, which is a typical suppression feature information.

[0126] In this scenario, since the initially determined pattern category has accurately and holistically captured and defined the inhibitory features present in the sequence, the server deems it unnecessary to proceed with the aforementioned complex process of parsing independent second access feature items and fusing them with the first access feature item. The server directly determines the initially determined access pattern category, which itself is inhibitory feature information, as the final target access pattern determination result for the system's access request sequence. In other words, the category "high-intensity read / write mixed conflict pattern" serves as both the initial analysis conclusion and the final pattern determination result.

[0127] Subsequently, based on the target access pattern determination result of "high-intensity read-write mixed conflict mode," the server executes the corresponding system access processing. The server queries the policy mapping table and finds that the core optimization for this mode lies in eliminating or mitigating mutual interference between different I / O streams. Therefore, the server may generate and trigger the following instruction: isolate the sequential write stream to the / db / archive.bak file and the random read stream to the / db / index.idx file in the storage layer scheduling queue, and allocate independent cache areas for them to avoid large write operations crowding out the cache space and I / O channel bandwidth required by small read requests. In this way, the server directly utilizes the suppression feature information contained in the initial pattern determination to quickly initiate targeted system optimization.

[0128] In this embodiment of the invention, the method is executed by a suppression feature processing ensemble model; the suppression feature processing ensemble model includes at least: an access feature joint extraction unit, an access strategy determination unit, a feature dimension completion unit, and an access constraint prediction unit;

[0129] The joint extraction unit for access features is used to obtain the access feature vector corresponding to the system access request sequence;

[0130] The access policy determination unit is used to perform access pattern analysis on the system access request sequence to obtain the access pattern category of the system access request sequence.

[0131] The feature dimension completion unit is used to obtain one or more first access feature items related to the access mode category from the system access request sequence;

[0132] The access constraint prediction unit is used to perform suppression feature parsing processing on the system access request sequence, and to parse out a second access feature item containing suppression feature information and the suppression scope corresponding to the second access feature item from the system access request sequence.

[0133] In an embodiment of the present invention, for example, in a video content analysis task, the system access request sequence received by the server includes: continuously reading multiple 4MB data blocks (e.g., offset 0-4MB, 4-8MB, ..., 28-32MB) of the video_01.mp4 file, followed by a sudden insertion of a request to read a small config.json file (offset 0, length 2KB), and then continuing to read 32-36MB data blocks of the video_01.mp4 file.

[0134] First, the joint access feature extraction unit within the model begins operation. This unit receives the original request sequence described above. It first standardizes and pads the sequence, generating an initial global statistical feature (such as average request size and offset trend) for the entire sequence, serving as a prototype for pattern-discriminating feature elements. Simultaneously, it encodes the features of each request unit in the sequence, converting information such as operation type, file ID, offset, and length into numerical vectors. Next, this unit performs joint feature mapping on the padded sequence using a shared deep neural network (such as a Transformer encoding layer). This network uses a self-attention mechanism to allow the features of each request to interact with the features of all other requests in the sequence, and to fuse the global statistical feature with all detailed request features. Finally, this unit outputs a unified, deep-level access feature vector. This vector contains both a highly refined pattern-discriminating feature element vector that represents the overall access features, and a request unit feature vector corresponding to each request unit in the sequence, which contains contextual semantics.

[0135] Subsequently, the access feature vector is sent in parallel to two core units for processing. On one hand, the access policy determination unit receives this feature vector. This unit is a pattern classifier that focuses on the pattern-discriminating feature elements in the feature vector. By decoding and analyzing these elements, the unit determines the macroscopic behavioral tendency of the current sequence. In this example, because the large, continuous requests to read video_01.mp4 dominate in terms of both quantity and data volume, this unit initially determines the access pattern category of the entire sequence as "sequential reading of the video stream."

[0136] On the other hand, the access constraint prediction unit also receives the same access feature vector. This unit is a fine-grained pattern parser that focuses on analyzing the feature vector of each request unit and its interrelationships. It scans the sequence to identify the request to read config.json. By analyzing its request unit feature vector (minimal request length, distinct file IDs) and comparing it with the context of the preceding and following requests, the unit parses a second access feature item containing suppression feature information, such as "metadata query insert". Simultaneously, the unit analyzes the scope of this feature's impact on the dominant pattern. It determines that this metadata query request disrupts the continuity of reading the video_01.mp4 file, weakening the predictability of the sequential reading pattern for video_01.mp4 that occurred before it. Therefore, it defines the continuous request subsequence for video_01.mp4 before this request (e.g., reads from 0-32MB) as the suppression scope.

[0137] Meanwhile, the feature dimension completion unit begins operation. This unit receives the "video stream sequential reading" mode category instruction from the access policy determination unit and accesses the intermediate data containing the detailed request unit feature vectors provided by the feature joint extraction unit. Based on the predefined pending feature dimensions (such as sequence segment start offset, end offset, and block size) of the "video stream sequential reading" category, this unit matches the longest continuously read subsequence of video_01.mp4 (i.e., the 0-32MB portion) from the original sequence. Then, it calculates the first request offset (0), the last request end offset (32MB), and the fixed request size (4MB) of this subsequence as the first access feature item associated with this mode category.

[0138] Thus, the suppression feature processing integrated model, through the collaborative work of its internal units, has completed a deep analysis of the request sequence, outputting key information such as access mode category, first access feature item, second access feature item, and suppression scope, providing a complete decision-making basis for the server to make precise mode adjustments and performance optimizations in the future.

[0139] In this embodiment of the invention, the suppression feature processing integrated model is obtained in the following manner and can be implemented through the following examples.

[0140] Obtain a historical access sample dataset and the corresponding multidimensional access annotation information; the multidimensional access annotation information corresponding to the historical access sample dataset includes access pattern category annotation information, feature dimension value labels, and suppression labels; the suppression label includes a second access feature item containing suppression feature information in the historical access sample dataset and the suppression scope corresponding to the second access feature item.

[0141] The suppression feature processing ensemble model is used to perform joint access feature determination processing on the historical access sample dataset to obtain the access pattern determination output of the historical access sample dataset;

[0142] Based on the optimization objective of converging the deviation between the access pattern determination output and the multidimensional access annotation information of the historical access sample dataset, the suppression feature processing ensemble model is optimized to obtain the optimized suppression feature processing ensemble model.

[0143] In an embodiment of the invention, for example, the server first prepares training data for the model to construct the ensemble model for suppressing feature processing. The server collects historical access sample datasets from production environment logs. For example, the server collects file access request logs generated by a video transcoding cluster over the past week. The server segments and organizes these logs according to task sessions and fixed time windows, forming tens of thousands of independent historical access sample datasets, each dataset representing a historical system access request sequence.

[0144] Subsequently, the server needs to generate high-quality, multi-dimensional access annotation information for these historical sequences. This task is performed by domain experts or a validated automated annotation system. For each historical sequence, the annotator first determines its overall access pattern and assigns an access pattern category label, such as "sequential large block writes" or "random small reads mixed". Next, based on the pattern category, the annotator precisely extracts key parameters from the sequence to form feature dimension value labels. For example, for a sequence labeled "sequential reads", the annotator will mark the start offset (e.g., 0), end offset (e.g., 128MB), and requested block size (e.g., 1MB) of the sequential read segment. More importantly, the annotator carefully examines the sequence for any abnormal requests that interfere with the main pattern and adds suppression labels to these requests. The suppression label explicitly contains two parts: 1) a second access feature containing suppression feature information, such as "intermediate metadata query"; 2) the suppression scope corresponding to the second access feature, that is, explicitly indicating the range of request subsequences affected by this suppression item, such as "suppressed the pattern purity of sequential read behavior from offset 64MB to 96MB".

[0145] After data preparation is complete, the server initiates the model training process. The server initializes an ensemble model for suppression feature processing as described above. This model includes a joint access feature extraction unit, an access policy determination unit, a feature dimension completion unit, and an access constraint prediction unit. The server inputs the labeled historical access sample dataset into the model for joint access feature determination processing. Specifically, the joint access feature extraction unit encodes the input historical sequence to generate a deep feature vector. The access policy determination unit outputs an access pattern determination output (i.e., the predicted pattern category) based on this vector. The feature dimension completion unit outputs the predicted feature dimension values. The access constraint prediction unit outputs the predicted suppression feature terms and the suppression scope.

[0146] Next, the server calculates the deviation between the model's predictions and the actual annotations. The server calculates the following separately: 1) the classification loss between the predicted pattern category and the access pattern category annotations; 2) the regression loss between the predicted feature dimension values ​​and the feature dimension value labels; and 3) the structured loss between the predicted suppression features, suppression scope, and suppression labels. The server then weights and sums these three loss functions to form a joint loss function. The value of this joint loss function represents the overall deviation between the model's access pattern determination output and the multi-dimensional access annotation information of historical samples.

[0147] Finally, with the convergence of this joint loss function as the core tuning objective, the server iteratively updates all parameters in the suppression feature processing ensemble model using the backpropagation algorithm and gradient descent optimizer. The server repeatedly inputs historical samples into the model, calculates the loss, and adjusts the parameters until the model's loss on the validation set stabilizes and reaches the predetermined performance metric. At this point, the server obtains a fully tuned suppression feature processing ensemble model. This model has learned to jointly extract features from the original request sequence and can collaboratively complete the tasks of main pattern determination, feature value completion, and suppression feature parsing, providing a reliable analytical foundation for subsequent online performance optimization.

[0148] In this embodiment of the invention, the access pattern determination output includes: access pattern category determination output, feature dimension value determination output, and suppressed access pattern determination output; the step of using the suppressed feature processing ensemble model to perform joint access feature determination processing on the historical access sample dataset to obtain the access pattern determination output of the historical access sample dataset includes:

[0149] Using the access strategy determination unit in the aforementioned suppression feature processing ensemble model, semantic prediction processing is performed on the historical access sample dataset based on the access feature vector to obtain the access pattern category determination output of the historical access sample dataset; and,

[0150] Using the feature dimension completion unit in the aforementioned suppressed feature processing ensemble model, feature dimension value prediction processing is performed on the historical access sample dataset based on the access feature vector, to obtain the feature dimension value judgment output related to the access pattern category judgment output in the historical access sample dataset; and...

[0151] Using the access constraint prediction unit in the suppression feature processing ensemble model, suppression feature information is predicted on the historical access sample dataset based on the access feature vector, and the suppression access mode determination output of the historical access sample dataset is obtained. The suppression access mode determination output includes: suppression prediction access feature items and the prediction suppression scope corresponding to the suppression prediction access feature items.

[0152] In this embodiment of the invention, the optimization of the suppression feature processing ensemble model based on the optimization objective of converging the deviation between the access pattern determination output and the multidimensional access annotation information of the historical access sample dataset can be performed through the following example.

[0153] Based on the access pattern category determination output of the historical access sample dataset and the access pattern category annotation information of the historical access sample dataset, the access pattern category cost function of the historical access sample dataset is obtained; and,

[0154] Based on the feature dimension value determination output and the feature dimension value label of the historical access sample dataset, the value padding cost function of the historical access sample dataset is obtained; and,

[0155] Based on the suppression access pattern determination output of the historical access sample dataset and the suppression label of the historical access sample dataset, the suppression cost function of the historical access sample dataset is obtained.

[0156] The access mode category cost function, the value completion cost function, and the suppression cost function are weighted and processed to obtain the target cost function;

[0157] Based on the optimization objective of converging the target cost function, the ensemble model for suppressing feature processing is optimized.

[0158] In this embodiment of the invention, for example, during the training process, the server performs joint access feature determination processing on a labeled historical access sample dataset, and obtains the model's access pattern determination output, including access pattern category determination output, feature dimension value determination output, and suppressed access pattern determination output. The server then begins to calculate the deviation between these outputs and the true labels to guide the update of the model parameters.

[0159] Step 1: Calculate the access pattern category cost function. The server obtains the access pattern category determination output from the access policy determination unit. For example, for a historical sequence, the model determines it as "large-scale data sequential reading". The server also reads the access pattern category annotation information corresponding to this historical sequence, which is provided by the annotator, for example, labeled as "large-scale data sequential reading". The server uses the cross-entropy loss function to calculate the difference between the class probability distribution predicted by the model and the one-hot encoding of the actual annotation. If the two are consistent, the cost function value is low; if the model misclassifies it as "random reading", it will incur a high cost. The value calculated in this step is the access pattern category cost function, which directly measures the accuracy of the model in macro-pattern classification.

[0160] Step 2: Calculate the feature dimension completion cost function. The server obtains the feature dimension value determination output from the feature dimension completion unit. For example, for the "large-scale data sequential reading" mode, the model predicts that the starting offset of the sequence segment is 0, the ending offset is 18MB, and the requested block size is 1MB. The server simultaneously reads the feature dimension value labels corresponding to this historical sequence. These labels are precisely labeled by the annotators; for example, the starting offset label is 0, the ending offset label is 16MB, and the requested block size label is 1MB. The server uses the mean squared error loss function to calculate the squared difference between the predicted value and the label value for each item, for example, calculating (18MB-16MB).2 The errors across all feature dimensions (start offset, end offset, block size) are summed to obtain the value completion cost function. This function measures the model's accuracy in quantizing and extracting key pattern parameters.

[0161] Step 3: Calculate the suppression cost function. The server obtains the suppression access pattern determination output from the access constraint prediction unit. For example, the model predicts a suppression feature as "tail-sized outlier write" and its suppression scope as "sequential read subsequences affecting 0-18MB". The server simultaneously reads the suppression label corresponding to this historical sequence, which is provided by the annotator and contains the true suppression feature "tail-sized outlier write" and the true suppression scope "sequential read subsequences affecting 0-16MB". The server needs to evaluate the accuracy of both predictions simultaneously: 1) whether the suppression feature is correctly classified; 2) the degree of overlap between the predicted suppression scope and the true scope. The server uses a composite loss function to weight and sum the classification loss of the suppression feature (e.g., cross-entropy) and the range matching loss of the suppression scope (e.g., IoU loss) to obtain the suppression cost function. This function measures the model's ability to discover and quantify internal conflict features.

[0162] Step 4: Obtain the target cost function through weighted calculation and perform model tuning. The server performs weighted calculation on the three cost functions mentioned above. The server presets weights according to business requirements; for example, the weight of the access mode category cost function is set to 0.5, the weight of the value completion cost function is set to 0.3, and the weight of the suppression cost function is set to 0.2. The server multiplies each cost function by its weight and then adds them together to obtain a scalar value, which is the target cost function. This function comprehensively reflects the overall error of the model in the joint decision-making task.

[0163] Finally, the server initiates the model optimization process based on the tuning objective of converging the target cost function. The server uses the backpropagation algorithm to calculate the gradient of the target cost function with respect to all trainable parameters in the suppression feature processing ensemble model (including parameters within the joint access feature extraction unit, access policy decision unit, feature dimension completion unit, and access constraint prediction unit). The server uses an optimizer (such as the Adam optimizer) to update these parameters according to the gradient direction. The server iterates this process repeatedly on the training set: inputting batches of historical samples, performing joint decision-making, calculating the target cost, and updating parameters via backpropagation. As the iteration progresses, the value of the target cost function gradually decreases and tends to stabilize, indicating that the overall deviation between the model's predicted outputs and the multi-dimensional access annotation information is decreasing. When the target cost function no longer decreases significantly on the validation set, the server obtains a fully tuned suppression feature processing ensemble model, which possesses accurate and collaborative pattern analysis capabilities.

[0164] To more clearly describe the solutions provided in the embodiments of the present invention, a more specific implementation method is provided below.

[0165] The distributed file system provided in this embodiment of the invention is a proprietary distributed storage system. The method is applied to an independent Windows client, which directly interacts with the storage cluster of the proprietary distributed storage system. The Windows client supports rapid expansion based on the size and capacity of the storage cluster to support high-throughput, low-latency access from a large number of clients. The step of generating and triggering the execution of corresponding resource configuration instructions or policy adjustment instructions based on the determined file system performance optimization type includes at least one of the following:

[0166] a. Simplify the file system interaction protocol: Establish a kernel driver module to convert Windows I / O into standard fuse interface operations; the fuse interface includes getattr, readlink, mknod, mkdir, unlink, rmdir, symlink, rename, link, chmod, chown, truncate, open, read, write, statfs, flush, release, fsync, setxattr, getxattr, listxattr, removeexattr, opendir, readdir, releasedir, fsyncdir, init, destroy, access, create, ftruncate, fgetattr, lock, utimens, bmap, ioctl, poll, write_buf, read_buf, flock, fallocate;

[0167] b. Improve the access algorithm for hotspot paths: cache local directories, convert file access paths into dentry information using a proprietary path resolution algorithm, perform hash calculation on the dentry information, and access different file servers based on the obtained hash value to improve the concurrency capability of hotspot directory access;

[0168] c. Provide caching capabilities at the driver level: Establish a file metadata caching mechanism in kernel mode, create or mark a cache index for each I / O access, and prioritize the use of the cache when it meets the cache usage logic; at the same time, use the Windows cache manager to cache file data to improve the performance of read / write operations;

[0169] d. Optimize network path: Enable the Windows client to directly access the storage cluster through a private protocol, shortening the network communication path; introduce a distributed concurrent access mechanism (which guarantees idempotency of access) into the private protocol to reduce network latency and increase network concurrency.

[0170] In this embodiment of the invention, for example, the server serves as the management core of a proprietary distributed storage system, connecting to a Windows client cluster of a film and television special effects studio, and implementing specific scenarios around four optimization directions:

[0171] Scenario 1: Simplifying file system interaction protocols:

[0172] When an artist opens a 4K video file mapped to distributed storage using Adobe Premiere on a Windows workstation, the application layer initiates a ReadFile system call. The client kernel driver module directly translates this request into a FUSE read operation (carrying a file handle, starting offset 0-1GB, and read length 256MB), and transmits it directly to the server via a proprietary protocol. The server skips the multi-layered protocol parsing of SMB / NFS, directly calls the data node's read interface to read the corresponding data block, and completes the response within 200ms, reducing overhead by 60% compared to the traditional SMB protocol.

[0173] Scenario 2: Improve the hotspot path access algorithm:

[0174] When 10 artists simultaneously and frequently accessed the "Project A / Material Library" directory, the server detected over 500 getattr / opendir requests per second, identifying it as a hotspot path. The server generated a path hash distribution strategy, mapping the directory path "Project A / Material Library" to a predefined hash rule and distributing it to all clients. After the clients performed hash calculations on this path, they distributed the requests across three metadata servers, increasing the concurrency of hotspot directory access from 500 requests per second to 1200 requests per second, and reducing the average latency from 80ms to 20ms.

[0175] Scenario 3: Driver-level caching capabilities:

[0176] When the artist repeatedly previewed video clips (offset 10-15GB), the server analyzed the request sequence and discovered a "small-range repeated read" characteristic, generating a caching instruction: "File handle F123, data block cache priority for offset 10-15GB is increased to the highest, lifespan extended to 1 hour." The client kernel-mode metadata cache module marked this range index, and the Windows cache manager synchronously cached the corresponding data. Subsequent reads directly hit the kernel cache, reducing response latency from 150ms to 1ms.

[0177] Scenario 4: Optimize network path:

[0178] When the workstation performs rendering output (continuously writing a 200GB file), the server allocates three data nodes to the client based on the cluster load, establishing three parallel private protocol sessions. During transmission, if the server detects that the packet loss rate on one link rises to 5%, it immediately triggers dynamic routing, switching the session to a backup node. The distributed concurrency mechanism of the private protocol ensures idempotency of the write operation, with no data loss during the switching process. The overall write throughput remains at 1.2GB / s, and the latency is stable within 50ms.

[0179] This invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned distributed file system access performance improvement method. Figure 2 As shown, Figure 2 This is a structural block diagram of a computer device 100 provided in an embodiment of the present invention. The computer device 100 includes a memory 111, a processor 112, and a communication unit 113. To enable data transmission or interaction, the memory 111, processor 112, and communication unit 113 are electrically connected to each other directly or indirectly. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0180] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the foregoing illustrative discussions are not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in accordance with the foregoing teachings. These embodiments were chosen and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the disclosure and to employ various embodiments with different modifications to suit a particular intended application.

Claims

1. A method for improving access performance of a distributed file system, characterized in that, include: Receive access requests for the distributed file system from clients and form a sequence of system access requests to be processed; The system access request sequence is subjected to access pattern analysis to obtain the access pattern category of the system access request sequence, and one or more first access feature items related to the access pattern category are obtained from the system access request sequence. The system access request sequence is subjected to suppression feature parsing processing to extract a second access feature item containing suppression feature information and a suppression scope corresponding to the second access feature item. The suppression feature information is a local feature that interferes with or corrects the determination of the main access mode, and the suppression scope is used to define the range of the main mode features affected by the suppression feature information. From the one or more first access feature items, a target access feature item that belongs to the inhibition domain is determined, and the access mode category is adjusted based on the second access feature item and the target access feature item to obtain the target access mode determination result of the system access request sequence; Based on the target access mode determination result, at least one corresponding file system performance optimization type is determined; Based on the determined file system performance optimization type, generate and trigger the execution of corresponding resource configuration instructions or policy adjustment instructions to optimize the access performance of the distributed file system.

2. The method of claim 1, wherein, The access pattern analysis of the system access request sequence to obtain the access pattern category of the system access request sequence includes: Obtain the access feature vector corresponding to the system access request sequence; the access feature vector includes a pattern discrimination feature element corresponding to the system access request sequence, and the pattern discrimination feature element is used to indicate the access feature state of the system access request sequence; The system access request sequence is analyzed using the pattern discrimination feature elements in the access feature vector to obtain the access pattern category of the system access request sequence.

3. The method of claim 2, wherein, The system access request sequence includes at least one request unit; the access feature vector corresponding to the system access request sequence also includes the request unit feature vector of each request unit in the at least one request unit, and the request unit feature vector of any request unit is used to indicate the request unit feature state of any request unit. The step of obtaining one or more first access feature items related to the access mode category from the system access request sequence includes: Obtain multiple undetermined feature dimensions corresponding to the access mode category; the multiple undetermined feature dimensions are predefined. Using the request unit feature vector of each request unit, a request unit subsequence that matches one or more of the plurality of undetermined feature dimensions is parsed from the at least one request unit; the request unit subsequence consists of at least one request unit. The request unit subsequence that matches each of the one or more undetermined feature dimensions is taken as the feature dimension value of the corresponding undetermined feature dimension; Wherein, one or more first access feature items related to the access mode category are the values ​​of one or more feature dimensions of the one or more undetermined feature dimensions.

4. The method according to claim 1, characterized in that, The system access request sequence includes at least one request unit; the suppression feature parsing process performed on the system access request sequence to parse out a second access feature item containing suppression feature information and the suppression scope corresponding to the second access feature item includes: Obtain the access feature vector corresponding to the system access request sequence; the access feature vector includes the request unit feature vector of each request unit in the at least one request unit, and the request unit feature vector of any request unit is used to indicate the request unit feature state of any request unit; Based on the request unit feature vector of each request unit in the access feature vector, a second access feature term containing suppression feature information is parsed from the system access request sequence; and, Based on the request unit feature vector of each request unit in the access feature vector, an association analysis is performed on each request unit included in the system access request sequence to obtain the inhibition domain corresponding to the second access feature item in the system access request sequence; the inhibition domain includes at least one request unit subsequence in the system access request sequence, and the request unit subsequence is composed of at least one request unit.

5. The method according to claim 2 or 4, characterized in that, The step of obtaining the access feature vector corresponding to the system access request sequence includes: The system access request sequence is supplemented with feature elements to obtain a supplemented system access request unit sequence; the supplemented system access request unit sequence includes: pattern discrimination feature elements and at least one request unit included in the system access request sequence; The system access request unit sequence is subjected to access feature mapping processing to obtain the access feature mapping result of the system access request unit sequence; Based on the access feature mapping result, an access feature extraction operation is performed on the system access request unit sequence to obtain the access feature vector corresponding to the system access request sequence; the access feature vector includes: the pattern discrimination feature element corresponding to the pattern discrimination feature element, and the request unit feature vector corresponding to each request unit in the at least one request unit.

6. The method of claim 1, wherein, The suppression domain includes at least one suppression feature dimension value corresponding to the second access feature; determining the target access feature item classified into the suppression domain from the one or more first access feature items, and adjusting the access mode category based on the second access feature item and the target access feature item to obtain the target access mode determination result of the system access request sequence includes: From the one or more first access features, determine the target access feature that has the same value as the suppression feature dimension within the suppression domain, and replace the target access feature with the suppression feature dimension value. Based on the second access feature item containing suppression feature information, the replaced suppression feature dimension value and other access feature items in the one or more first access feature items excluding the target access feature item, the access mode category is adjusted to obtain the target access mode determination result of the system access request sequence. The access mode corresponding to the target access mode determination result is different from the access mode corresponding to the access mode category.

7. The method of claim 1, wherein, The method further includes: The system access request sequence is subjected to access mode suppression determination processing. If the access mode corresponding to the access mode category of the system access request sequence is non-suppression feature information, then return to the step of determining the target access feature item classified into the suppression domain from the one or more first access feature items, and adjusting the access mode category based on the second access feature item and the target access feature item to obtain the target access mode determination result of the system access request sequence; The corresponding system access processing is executed based on the target access mode determination result of the system access request sequence.

8. The method according to claim 1, characterized in that, The method is executed through a suppressed feature processing ensemble model; the suppressed feature processing ensemble model includes at least: an access feature joint extraction unit, an access strategy determination unit, a feature dimension completion unit, and an access constraint prediction unit; The joint extraction unit for access features is used to obtain the access feature vector corresponding to the system access request sequence; The access policy determination unit is used to perform access pattern analysis on the system access request sequence to obtain the access pattern category of the system access request sequence. The feature dimension completion unit is used to obtain one or more first access feature items related to the access mode category from the system access request sequence; The access constraint prediction unit is used to perform suppression feature parsing processing on the system access request sequence, and to parse out a second access feature item containing suppression feature information and the suppression scope corresponding to the second access feature item from the system access request sequence.

9. The method of claim 8, wherein, The suppression feature processing ensemble model is obtained through the following methods: Obtain a historical access sample dataset and the corresponding multidimensional access annotation information; the multidimensional access annotation information corresponding to the historical access sample dataset includes access pattern category annotation information, feature dimension value labels, and suppression labels; the suppression label includes a second access feature item containing suppression feature information in the historical access sample dataset and the suppression scope corresponding to the second access feature item. Using the access strategy determination unit in the aforementioned suppression feature processing ensemble model, semantic prediction processing is performed on the historical access sample dataset based on the access feature vector to obtain the access pattern category determination output of the historical access sample dataset; and, Using the feature dimension completion unit in the aforementioned suppressed feature processing ensemble model, feature dimension value prediction processing is performed on the historical access sample dataset based on the access feature vector, to obtain the feature dimension value judgment output related to the access pattern category judgment output in the historical access sample dataset; and... Using the access constraint prediction unit in the suppression feature processing ensemble model, suppression feature information is predicted on the historical access sample dataset based on the access feature vector, and the suppression access mode determination output of the historical access sample dataset is obtained. The suppression access mode determination output includes: suppression prediction access feature item and the prediction suppression scope corresponding to the suppression prediction access feature item. Based on the access pattern category determination output of the historical access sample dataset and the access pattern category annotation information of the historical access sample dataset, the access pattern category cost function of the historical access sample dataset is obtained; and, Based on the feature dimension value determination output and the feature dimension value label of the historical access sample dataset, the value padding cost function of the historical access sample dataset is obtained; and, Based on the suppression access pattern determination output of the historical access sample dataset and the suppression label of the historical access sample dataset, the suppression cost function of the historical access sample dataset is obtained. The access mode category cost function, the value completion cost function, and the suppression cost function are weighted and processed to obtain the target cost function; Based on the optimization objective of converging the target cost function, the suppression feature processing ensemble model is optimized to obtain the optimized suppression feature processing ensemble model.

10. A server system, characterized by Includes a server, the server being used to perform the method according to any one of claims 1-9.

Images