Training method of file fragment classification model and file fragment classification method

Abstract

The invention discloses a training method of a file fragment classification model and a file fragment classification method, and relates to the field of file fragment classification, and the method comprises the steps: collecting different types of original file data, and carrying out the random sampling of data blocks of the different types of original file data at a fixed byte length; converting each data block into a corresponding initial token sequence; converting each initial token sequence into a target token sequence; and pre-training a Transform model based on each target token sequence after dynamic masking, the dynamic masking comprising: randomly selecting byte tokens with a preset proportion in the target token sequence as target byte tokens, and performing different masking operations on each target byte token based on probability allocation, the different mask operations comprise replacing the target byte token with the fourth function token, randomly replacing the target byte token with other byte tokens and keeping the target byte token unchanged. According to the method and the device, the generalization ability of the Transform model can be improved.

Description

Technical Field

[0001] This application relates to the field of file fragment classification, and in particular to a training method for a file fragment classification model and a file fragment classification method. Background Technology

[0002] Multimedia file fragment classification is a core technology in the field of digital forensics. Its main value lies in its ability to accurately identify the specific file type of isolated data blocks when the file system is corrupted or metadata is lost, laying the foundation for subsequent data recovery and in-depth forensic analysis. However, this technology faces several serious challenges.

[0003] First, there are numerous file formats, and many proprietary formats lack publicly available standard documentation. Traditional signature matching methods rely on the "magic number" at the beginning and end of the file, but these methods have limitations, such as signature corruption, conflicts, and inability to handle fragments without header and footer identifiers.

[0004] Secondly, most modern file types commonly employ lossless or lossy compression algorithms, which reduce data redundancy and regularity, resulting in greater randomness in file byte streams. This change weakens the discriminative power of classification methods based on statistical characteristics. Furthermore, because different types of files often use the same or similar compression algorithms, the differences in their statistical characteristics become less significant, thus increasing the risk of misclassification.

[0005] Third, for isolated classification tasks involving small, discrete sector fragments (512 bytes), the lack of long-range patterns and contextual information sets an insurmountable upper limit on the accuracy of classification algorithms. Subsequent research has addressed this limitation by introducing attention mechanisms. More recently, data augmentation techniques such as bit shifting and bit flipping have also been employed. However, the generalization ability of file fragment classification models remains limited due to the distribution and quality of fragmented data.

[0006] In summary, current file fragment classification models have poor generalization ability and cannot be applied to the classification of different file formats, different randomness, and tiny discrete file fragments simultaneously. Summary of the Invention

[0007] The purpose of this application is to provide a training method and a file fragment classification method for a file fragment classification model, which can improve the generalization ability of the Transformer model.

[0008] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a training method for a file fragment classification model, including: Collect raw file data of different types, and randomly sample data blocks of the raw file data of different types with a fixed byte length to obtain several data blocks; Each data block is converted into a corresponding initial token sequence, where the i-th byte token in each initial token sequence is a combination of the i-th and (i+1)-th bytes in the corresponding data block; A first function token is inserted at the beginning of each initial token sequence, and each initial token sequence is filled to a preset maximum length. For unregistered byte tokens in each initial token sequence, a third function token is used to represent them. Finally, each initial token sequence is converted into a target token sequence. The Transformer model is pre-trained based on the target token sequences after dynamic masking. The dynamic masking includes: randomly selecting a preset proportion of byte tokens in the target token sequence as target byte tokens, and performing different masking operations on each target byte token based on probability allocation. The different masking operations include replacing the target byte token with a fourth function token, randomly replacing the target byte token with other byte tokens, and keeping the target byte token unchanged. The pre-trained Transformer model is fine-tuned using file data from the target classification scenario to obtain a file fragment classification model for the target classification scenario.

[0009] Secondly, this application provides a method for classifying file fragments, including: In the target classification scenario, a file fragment classification model is used to classify file fragments; The file fragment classification model is obtained through the training method of the file fragment classification model described in the first aspect.

[0010] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the training method for the file fragment classification model described in any one of the above.

[0011] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the training method for the file fragment classification model described in any one of the above.

[0012] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the training method for the file fragment classification model described above.

[0013] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a training method and a file fragment classification method for a file fragment classification model. First, a Transformer model is pre-trained using different types of original file data. Then, the pre-trained Transformer model is fine-tuned using relevant file data according to a specific downstream classification task, thus obtaining a file fragment classification model for that task. Specifically, binary byte group encoding is used to encode data blocks in the original file data to obtain an initial token sequence. This operation can capture the correlation between local bytes while preserving the underlying information of the data block, more accurately represent the co-occurrence pattern of byte pairs, and effectively model local dependencies. Furthermore, the initial token sequence is processed using a first function token, a second function token, and a third function token. The first function token aggregates the global context information of the entire sequence through a self-attention mechanism; the second function token ensures that each initial token sequence has the same length; and the third function token enhances the robustness of the Transformer model. More importantly, by using dynamic masking of the target token sequence, the Transformer model faces different masking operations each time it is trained. This increases the randomness and data diversity of the training process, enabling the Transformer model to learn more comprehensive contextual relationships. It also reduces the risk of overfitting the Transformer model to a specific masking pattern and improves the generalization ability of the Transformer model. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is an application environment diagram of a training method for a file fragment classification model according to an embodiment of this application; Figure 2 This is a flowchart illustrating a training method for a file fragment classification model according to an embodiment of this application. Figure 3 This is a schematic diagram illustrating the type of initial file data in one embodiment of this application; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] The training method for the file fragment classification model provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be set up independently, integrated into server 102, or placed in the cloud or on another server. Terminal 101 can send different types of raw file data to server 102, and server 102 pre-trains the Transformer model based on the different types of raw file data. Server 102 can then provide feedback of the pre-trained Transformer model to terminal 101. Furthermore, in some embodiments, the training method for the file fragment classification model can also be implemented independently by server 102 or terminal 101.

[0019] The terminal 101 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 102 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0020] In one exemplary embodiment, such as Figure 2 As shown, a training method for a file fragment classification model is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 102 as an example, the explanation includes the following steps 210 to 250.

[0021] Step 210: Collect raw file data of different types, and randomly sample data blocks of different types of raw file data with a fixed byte length to obtain several data blocks. The obtained data blocks constitute a pre-training dataset.

[0022] In one example, approximately 2TB of raw file data was collected, covering 81 different file types. The specific distribution of each file type is as follows: Figure 3 As shown, Figure 3 China and Israel The file types identified by the quotation mark ("") are new types introduced compared to the publicly available FFT-75 dataset. After preprocessing such as removing header and footer information from the original file data, random sampling was performed using data blocks of fixed length 512 bytes. This resulted in a pre-training dataset containing approximately 210 million samples and a total size of approximately 178GB, named ZYH-81. This dataset was divided into training and validation sets in a 4:1 ratio.

[0023] It should be noted that the above is an example illustration of step 210. Step 210 can also collect more or less raw file data, and the number of file types can also be more or less. Besides 512 bytes, other fixed byte lengths can also be used.

[0024] Step 220: Convert each data block into a corresponding initial token sequence. The i-th byte token in each initial token sequence is a combination of the i-th and (i+1)-th bytes in the corresponding data block.

[0025] Step 220 is primarily responsible for converting the raw binary byte stream (data block) into a structured token sequence that the model can process. It is the starting point for the Transformer model to understand the data and is essentially a binary byte group encoding.

[0026] Specifically, to capture the correlation between local bytes while preserving underlying information, binary byte groups are used as the basic encoding unit. This mechanism uses a sliding window to combine every two adjacent bytes in a data block into a byte token. Given that a single byte is 8 bits and a binary byte group is 16 bits, its value ranges from 0 to 65535. For example, the data block [0x01, 0x02, 0x03] will be converted into the initial token sequence [0x0102, 0x0203].

[0027] Compared to single-byte encoding, bigram encoding expands the vocabulary size from 256 to 65536, enabling more accurate representation of co-occurrence patterns of byte pairs.

[0028] Furthermore, byte group encoding is excellent for modeling local dependencies because byte groups naturally encode the order and relationship between adjacent bytes, providing the Transformer model with more informative initial input than isolated bytes.

[0029] Step 230: Insert a first function token at the beginning of each initial token sequence, fill each initial token sequence to the preset maximum length, and use a third function token to represent the unregistered byte tokens in each initial token sequence. Finally, convert each initial token sequence into a target token sequence.

[0030] Step 240: Pre-train the Transformer model based on the target token sequences after dynamic masking. The dynamic masking includes: randomly selecting a preset proportion of byte tokens in the target token sequence as target byte tokens, and performing different masking operations on each target byte token based on probability allocation. The masking operations include replacing the target byte token with a fourth function token, randomly replacing the target byte token with other byte tokens, and keeping the target byte token unchanged.

[0031] To implement steps 230 and 240 above, four additional function tokens with special functions are defined in the vocabulary: the first function token [CLS], the second function token [PAD], the third function token [UNK], and the fourth function token [MASK].

[0032] The first function token [CLS] is fixed at the beginning of each initial token sequence. In the layer-by-layer information transfer of the Transformer model, the first function token can aggregate the global context information of the entire sequence through the self-attention mechanism. Therefore, its final hidden state vector is often used as the aggregated representation of the entire sequence, directly serving the downstream classification task.

[0033] Secondary Function Tokens (PADs): For efficient batch processing, all input sequences must have the same length. Secondary function tokens (PADs) are used to pad insufficient initial token sequences to a preset maximum length. For example, using the longest initial token sequence as a baseline, secondary function tokens are padded to the ends of non-longest initial token sequences, ensuring that each initial token sequence has the same length. During computation, the Transformer model ignores these padded secondary function tokens through an attention masking mechanism to prevent them from interfering with the learning of effective information.

[0034] Third-function token [UNK]: Used to represent byte tokens not registered in the vocabulary. Unregistered byte tokens refer to those unknown byte tokens, primarily resulting from byte anomalies or corruption. Although the binary byte array design covers all possible combinations, the presence of the third-function token [UNK] enhances the robustness of the model when dealing with data anomalies or corruption.

[0035] The token sequence formed by introducing the above three functional tokens into the initial token sequence is the target token sequence.

[0036] The fourth function token [MASK] is the core of the masked language model pre-training task. During pre-training, it is used to randomly replace part of the original byte tokens in the target token sequence, forcing the Transformer model to predict the masked byte tokens based on the context, thereby learning the inherent structure of the sequence.

[0037] The fourth function token forms the basis for performing dynamic masking on the target token sequence in step 240. The key to dynamic masking lies in performing different masking operations on each target byte token based on probability allocation. This dynamic masking strategy overcomes the limitations of traditional static masking, which fixes the mask for each sample during training. Dynamic masking ensures that the Transformer model faces different masking operations each time it is trained, increasing the randomness and data diversity of the training process. This allows the Transformer model to learn more comprehensive contextual relationships, reduces the risk of overfitting to specific masking patterns, and improves the generalization ability of the Transformer model.

[0038] Step 250: Fine-tune the pre-trained Transformer model using file data from the target classification scenario to obtain a file fragment classification model for the target classification scenario.

[0039] The pre-trained Transformer model exhibits strong generalization ability and initially possesses the capability to classify file fragments in different classification scenarios. To further improve the classification accuracy of the Transformer model in specific classification scenarios, the pre-trained Transformer model can be fine-tuned using file data from that specific scenario, resulting in a file fragment classification model with high accuracy in that scenario. The target classification scenario is the downstream classification task scenario of the Transformer model, and different downstream classification tasks are mainly distinguished based on the type of file data.

[0040] In summary, the training method for the file fragment classification model in this embodiment first pre-trains the Transformer model using different types of original file data. Then, the pre-trained Transformer model is fine-tuned using relevant file data according to the specific downstream classification task, thereby obtaining a file fragment classification model for the specific downstream classification task. Specifically, binary byte group encoding is used to encode data blocks in the original file data to obtain an initial token sequence. This operation can capture the correlation between local bytes while preserving the underlying information of the data block, more accurately represent the co-occurrence pattern of byte pairs, and effectively model local dependencies. Simultaneously, the initial token sequence is processed through a first function token, a second function token, and a third function token. The first function token aggregates the global context information of the entire sequence through a self-attention mechanism; the second function token ensures that each initial token sequence has the same length; and the third function token enhances the robustness of the Transformer model. More importantly, by using dynamic masking of the target token sequence, the Transformer model faces different masking operations each time it is trained. This increases the randomness and data diversity of the training process, enabling the Transformer model to learn more comprehensive contextual relationships. It also reduces the risk of overfitting the Transformer model to a specific masking pattern and improves the generalization ability of the Transformer model.

[0041] For example, the target token sequences after dynamic masking are used to pre-train the Transformer model, specifically including: converting each target token sequence into a target representation, and using the target representation as input to the Transformer model to pre-train the Transformer model; wherein, each target representation is obtained by adding the token embedding and position embedding of each token in the corresponding target token sequence, the token embedding being the index mapping vector of the token, and the position embedding being the position mapping vector of the token.

[0042] Since the Transformer architecture itself does not have the ability to handle sequence order, the final input representation of the Transformer model is composed of token embedding and position embedding.

[0043] Token embedding is achieved through a learnable embedding matrix. (in , which is the size of the vocabulary list. (where is the dimension of the hidden layers in the Transformer model), mapping each discrete token index to a dense vector. During training, semantically or functionally similar groups of bytes will be placed close to each other in this vector space.

[0044] Position embeddings inject information about the absolute or relative position of tokens within a sequence into the Transformer model, with each position index mapped to a learnable vector. This position embedding vector is added to the token embedding vector at the corresponding position to form the final input representation. This enables the Transformer model to distinguish sequences with identical content but different orders, which is crucial for understanding position-sensitive structures such as file headers and data block boundaries.

[0045] The input vector of the token at position i in the sequence It is through token embedding With position embedding It is obtained by adding each element. The calculation formula is: .

[0046] For example, the preset ratio is 15%.

[0047] In this embodiment, different masking operations are performed on each target byte token according to probability, specifically including: for each target byte token, replacing the target byte token with a fourth function token with a first probability, randomly replacing the target byte token with other byte tokens with a second probability, and keeping the target byte token unchanged with a third probability; wherein, the sum of the first probability, the second probability, and the third probability is 1.

[0048] For example, the first probability is 80%, the second probability is 10%, and the third probability is 10%.

[0049] The following example illustrates dynamic masking operations.

[0050] 15% of the tokens in the model input sequence (target token sequence) are randomly selected for masking, forcing the Transformer model to predict the masked tokens based on the unmasked context information. These 15% of masked tokens are further subdivided into three processing methods: There is an 80% probability that the token will be replaced with a fourth function token. For example, for the byte sequence "0x1234", there is an 80% chance that it will be replaced with a fourth function token, which makes the Transformer model need to infer the original token value at this position based on the surrounding context bytes. There is a 10% probability that the token will be replaced with a random token, such as replacing "0x1234" with "0x5678". This random replacement increases the difficulty and noise of the task, forcing the Transformer model to learn to distinguish between real context and noise, thereby enhancing the robustness of the Transformer model and its ability to adapt to complex situations. There is a 10% probability that the original token will remain unchanged, meaning "0x1234" will remain "0x1234". This requires the Transformer model to learn to use contextual information to confirm whether the token at that position is correct without a mask, further improving the Transformer model's ability to understand and judge context.

[0051] In this embodiment, the optimization objective of the Transformer model is to minimize the cross-entropy loss between its prediction and the true value of the masked token. The loss function is defined as follows: in, This represents the trainable hyperparameters in the Transformer model. This represents the set of position indices of the target byte token. This represents the actual value of the target byte token at position i. This represents the context markers in the input of the Transformer model that are not masked. Represents a given context and hyperparameters Under the given conditions, the Transformer model predicts the i-th position as The probability of.

[0052] The Transformer model calculates the loss gradient using the backpropagation algorithm and uses an optimizer (such as AdamW) to update the parameters. This allows its predictions to get closer and closer to the true values.

[0053] In this embodiment, the pre-trained Transformer model performs classification prediction based on target fusion features during fine-tuning. The target fusion features are: in, This represents the target fusion feature, which is , and The fusion result This represents the hidden state of the first function token in the last layer of the Transformer model. and These represent the global flat pooling result and the global max pooling result of the hidden states of tokens other than the first function token in the last layer of the Transformer model, respectively.

[0054] Specifically, to avoid information bottlenecks that may result from using only the first function token, this embodiment introduces a multi-semantic feature fusion mechanism during fine-tuning. This mechanism integrates three feature representations of different granularities to form a more comprehensive description of file fragments.

[0055] Context aggregation representation It directly uses the hidden state of the first function token in the last layer of the Transformer model. This vector is considered as a deep context summary of the entire sequence.

[0056] Global summary representation Global Average Pooling (GAP) is performed on the hidden state vectors of all non-first function tokens in the last layer of the Transformer model. This operation calculates the mean of the sequence features, reflecting the overall, universal characteristics of file fragments.

[0057] Significant feature representation Global Max Pooling (GMP) is performed on the hidden states of all non-first function tokens in the last layer of the Transformer model. This operation captures the maximum response value along the feature dimension, effectively extracting the most discriminative key patterns in the sequence.

[0058] Subsequently, these three feature vectors are concatenated together to form the final fused feature vector (target fused feature), represented as: This fused vector combines deep context, global statistical information, and local salient features, significantly enhancing the robustness of the model representation and classification performance.

[0059] Target fusion features The data is fed into a classification head for final category prediction. This classification head is a concise and efficient two-layer multilayer perceptron.

[0060] The first layer performs a non-linear transformation on the target fusion features to learn more complex combinational relationships between features. This layer consists of a linear transformation and a ReLU activation function: The second layer linearly maps the hidden layer representation to the category space and transforms it into a probability distribution using the Softmax function.

[0061] in, These are learnable parameters. The category corresponding to the dimension with the largest value in the output probability vector is the final prediction result of the Transformer model. During fine-tuning, the parameters of the entire Transformer model are jointly optimized based on the loss function of the classification task.

[0062] Based on the training method of the file fragment classification model in the above embodiments, another exemplary embodiment provides a file fragment classification method including: in a target classification scenario, using a file fragment classification model to classify file fragments; wherein, the file fragment classification model is obtained through the training method of the file fragment classification model in the above embodiments.

[0063] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 4 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a training method for a file fragment classification model.

[0064] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0065] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0066] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0067] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0068] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0069] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0070] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0071] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0072] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A training method for a file fragment classification model, characterized in that, include: Collect raw file data of different types, and randomly sample data blocks of the raw file data of different types with a fixed byte length to obtain several data blocks; Each data block is converted into a corresponding initial token sequence, where the i-th byte token in each initial token sequence is a combination of the i-th and (i+1)-th bytes in the corresponding data block; A first function token is inserted at the beginning of each initial token sequence, and each initial token sequence is filled to a preset maximum length. For unregistered byte tokens in each initial token sequence, a third function token is used to represent them. Finally, each initial token sequence is converted into a target token sequence. The Transformer model is pre-trained based on the target token sequences after dynamic masking. The dynamic masking includes: randomly selecting a preset proportion of byte tokens in the target token sequence as target byte tokens, and performing different masking operations on each target byte token based on probability allocation. The different masking operations include replacing the target byte token with a fourth function token, randomly replacing the target byte token with other byte tokens, and keeping the target byte token unchanged. The pre-trained Transformer model is fine-tuned using file data from the target classification scenario to obtain a file fragment classification model for the target classification scenario.

2. The training method for the file fragment classification model according to claim 1, characterized in that, The Transformer model is pre-trained based on the dynamically masked sequence of target tokens, specifically including: Each of the target token sequences is converted into a target representation, and the target representation is used as input to the Transformer model for pre-training; Each target representation is obtained by adding the token embedding and position embedding of each token in the corresponding target token sequence, wherein the token embedding is the index mapping vector of the token and the position embedding is the position mapping vector of the token.

3. The training method for the file fragment classification model according to claim 1, characterized in that, The preset ratio is 15%.

4. The training method for the file fragment classification model according to claim 1, characterized in that, For each of the target byte tokens, perform different masking operations according to probability, specifically including: For each target byte token, the target byte token is replaced with the fourth function token with a first probability, the target byte token is randomly replaced with other byte tokens with a second probability, and the target byte token is kept unchanged with a third probability. Wherein, the sum of the first probability, the second probability, and the third probability is 1.

5. The training method for the file fragment classification model according to claim 4, characterized in that, The first probability is 80%, the second probability is 10%, and the third probability is 10%.

6. The training method for the file fragment classification model according to claim 2, characterized in that, The loss function of the Transformer model during pre-training is: in, This represents the trainable hyperparameters in the Transformer model. This represents the set of position indices of the target byte token. This represents the actual value of the target byte token at position i. This represents the context markers in the input of the Transformer model that are not masked. Represents a given context and hyperparameters Under the given conditions, the Transformer model predicts the i-th position as The probability of.

7. The training method for the file fragment classification model according to claim 2, characterized in that, The pre-trained Transformer model performs classification prediction based on target fusion features during fine-tuning. These target fusion features are: in, This represents the target fusion feature, which is , and The fusion result This represents the hidden state of the first function token in the last layer of the Transformer model. and These represent the global flat pooling result and the global max pooling result of the hidden states of tokens other than the first function token in the last layer of the Transformer model, respectively.

8. A method for classifying file fragments, characterized in that, include: In the target classification scenario, a file fragment classification model is used to classify file fragments; The file fragment classification model is obtained by the training method of the file fragment classification model according to any one of claims 1-8.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement a training method for the file fragment classification model according to any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the training method for the file fragment classification model according to any one of claims 1-7.

Images