Model training data processing method and system and electronic equipment

Abstract

The invention relates to the technical field of electric digital data processing, in particular to a model training data processing method and system and electronic equipment. Firstly, MIME types are determined through an HTTP response header for preliminary classification, it is ensured that data are classified in first-level classification directories of texts, pictures, audios and videos, and for data with missing MIME types, a type weighted scoring method is used for auxiliary classification, so that the classification accuracy is improved; thirdly, extracting a second-level classification label of the first-level classification data through an existing data set, automatically generating a second-level classification result in combination with a zero sample learning algorithm, storing the second-level classification result into a corresponding second-level directory, and further refining data categories; and finally, checking whether the data volume is lower than a generation threshold value or not by traversing a secondary classification directory, automatically triggering data amplification, balancing the number of various types of samples, and ensuring the balance and representativeness of the data. According to the method, multi-level automatic classification and data amplification are effectively realized, and the model training data processing efficiency and processing quality are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a model training data processing method, system and electronic equipment. Background Art

[0002] In the field of artificial intelligence, model design can be divided into two types: unimodal models and multimodal models. Among them, multimodal models are cutting-edge technologies in the field of artificial intelligence. Multimodal models can integrate data from multiple modalities such as text, images, audio and video into the same model, so as to understand complex information more comprehensively and accurately. Compared with traditional single-modal models, multimodal models can fuse different types of data, capture the deep semantics in information from multiple perspectives, and achieve tasks such as image description generation, cross-modal retrieval and multimodal classification. In order to give full play to the potential of multimodal models, high-quality data processing is an indispensable step in their training. Through reasonable classification, grading and amplification, the model can better adapt to the characteristics of various modal data and ensure the integrity, balance and effectiveness of training data, which is particularly critical for the generalization ability and robustness of the model.

[0003] However, at this stage, multimodal models still face many challenges in training data processing. First, the data formats and file structures of different modalities are different, and automated processing and unified management are difficult. Secondly, since the amount of data in different modalities is often unbalanced, the amount of data in some modalities is insufficient or the distribution is skewed, resulting in overfitting or underfitting in model training. In addition, there are also great difficulties in classifying data with missing labels, and the current zero-sample classification algorithm often has limited effect when processing multimodal data. These drawbacks lead to low efficiency and low quality in training data processing for multimodal models, which restricts the further development and application of multimodal models. Therefore, there is an urgent need for a model training data processing method to solve the above problems, thereby improving the performance of the model on various modal data.

[0004] To this end, a model training data processing method, system and electronic device are proposed. Summary of the invention

[0005] The purpose of the present invention is to propose a model training data processing method, system and electronic device. First, the MIME type of Internet data is obtained through the HTTP response header and preliminarily classified, and stored in the corresponding first-level classification directory. If the MIME type is missing, the data type is predicted by the weighted scoring method to ensure that all data can be classified into the corresponding category. Then, based on the existing data set, the second-level classification labels of the first-level classification data are extracted, and a natural language description is generated for each label, so as to realize the second-level classification of the first-level classification data under the zero-sample learning framework, and the classification results are stored in the second-level classification directory; finally, all the second-level classification directories are traversed. If the data volume of a certain classification is lower than the preset threshold, the data volume of the directory is automatically expanded to ensure that the distribution of various types of data is balanced, which is conducive to the effective training of the model.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A model training data processing method, comprising:

[0008] Access Internet data;

[0009] Obtain the MIME type of the Internet data through the HTTP response header, classify the Internet data according to the MIME type to obtain first-level classification data, and store the first-level classification data in the first-level classification directory; if the Internet data lacks the MIME type, use the type weighted scoring method to classify the data type of the Internet data and store it in the first-level classification directory;

[0010] Extracting secondary classification labels corresponding to the primary classification data according to the existing data set, generating natural language text descriptions according to the secondary classification labels, performing secondary classification on the primary classification data using a zero-sample learning algorithm to obtain secondary classification data, and storing the secondary classification data in a secondary classification directory;

[0011] Traverse the secondary classification directory and calculate the amount of data in the current secondary classification directory, determine whether the amount of data in the current secondary classification directory is less than the generation threshold, if the amount of data in the current secondary classification directory is less than the generation threshold, then expand the amount of data in the current secondary classification directory; otherwise, skip the current secondary classification directory and traverse the next secondary classification directory; until all secondary classification directories are traversed.

[0012] Furthermore, the first-level classification data includes text data, picture data, audio data and video data, and the first-level classification directory includes a text directory, a picture directory, an audio directory and a video directory.

[0013] Furthermore, using a type-weighted scoring method to classify the data types of the Internet data includes:

[0014] Get file extension, magic bytes and file size of Internet data with missing MIME type;

[0015] Calculate the Internet data of the missing MIME type according to the type weighted scoring formula to obtain type scores corresponding to all file types;

[0016] According to the type scores, the category with the highest score is selected as the predicted category of the document.

[0017] The type weighted scoring formula is:

[0018] S final (t) = ω ext ·f ext (t)+ω magic ·f magic (t)+ω size ·f size (t);

[0019] Wherein, t represents the Internet data with missing MIME type, S final () indicates the comprehensive score of the category to which the data belongs, ω ext Indicates the extension weight coefficient of the data, f ext () represents the extended name evaluation function of the data, ω magic Indicates the magic byte weight coefficient of the data, f magic () represents the magic byte evaluation function of the data, ω size Indicates the file size weight coefficient of the data, f magic () represents the file size evaluation function of the data.

[0020] Furthermore, the existing data set includes a text data set, a picture data set, an audio data set, and a video data set; the text data set is 20Newsgroups, the picture data set is ImageNet, the audio data set is UrbanSound8K, and the video data set is YouTube-8M.

[0021] Further, performing secondary classification on the primary classification data using a zero-sample learning algorithm includes:

[0022] Use the pre-trained model to convert the data in the first category into a high-dimensional feature vector;

[0023] Mapping the high-dimensional feature vector into a shared semantic space using contrastive learning;

[0024] Embed the description text of the secondary classification label into the shared semantic space through a language model to obtain a semantic vector of the secondary classification label;

[0025] Calculate the similarity between the high-dimensional feature vector and the secondary classification label according to cosine similarity;

[0026] According to the similarity, a category with the highest similarity to the high-dimensional feature vector is selected as a classification result, and the classification result is stored in a secondary classification directory.

[0027] Furthermore, the generation threshold calculation formula is:

[0028]

[0029] Among them, S represents the generation threshold, m represents the number of secondary classification types, N i Represents the amount of data in the i-th secondary classification directory.

[0030] Further, expanding the data volume of the current secondary classification directory includes:

[0031] Select the data in the current secondary classification directory as training data for the data augmentation model, and select a corresponding data augmentation model according to the type of the training data;

[0032] Using the data of the current secondary classification directory to train the data augmentation model, generating new data samples through the data augmentation model, and obtaining generated data;

[0033] Verifying the similarity between the generated data and the training data;

[0034] The generated data is saved in the current secondary classification directory until the data volume of the current secondary classification directory meets the generation threshold requirement.

[0035] Furthermore, the data augmentation model includes a text augmentation model, a picture augmentation model, an audio augmentation model and a video augmentation model; the text augmentation model is GPT-4, the picture augmentation model is CycleGAN, the audio augmentation model is WaveGAN, and the video augmentation model is MoCo-V3.

[0036] In addition, to achieve the above purpose, the present invention also proposes a model training data processing system, comprising:

[0037] A data acquisition module, used to acquire Internet data;

[0038] A primary data classification module is used to obtain the MIME type of the Internet data through the HTTP response header, classify the Internet data according to the MIME type, obtain primary classification data, and store the primary classification data in a primary classification directory; if the Internet data lacks the MIME type, use a type weighted scoring method to classify the data type of the Internet data and store it in the primary classification directory;

[0039] A secondary data classification module is used to extract secondary classification labels corresponding to the primary classification data according to the existing data set, generate natural language text descriptions according to the secondary classification labels, perform secondary classification on the primary classification data using a zero-sample learning algorithm to obtain secondary classification data, and store the secondary classification data in a secondary classification directory;

[0040] The data amplification module is used to traverse the secondary classification directory and calculate the amount of data in the current secondary classification directory, and determine whether the amount of data in the current secondary classification directory is less than the generation threshold. If the amount of data in the current secondary classification directory is less than the generation threshold, the data amount of the current secondary classification directory is amplified; otherwise, the current secondary classification directory is skipped and the next secondary classification directory is traversed until all secondary classification directories are traversed.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also proposes a model training data processing electronic device, on which computer program instructions are stored. When the computer program instructions are executed by the electronic device, the electronic device implements the method described above.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The present invention automatically obtains the MIME type through the HTTP response header to perform preliminary classification of Internet data, so that the data can be quickly classified into primary classification directories such as text, image, audio, video, etc., reducing the workload of manual judgment. At the same time, for data with missing MIME types, the type weighted scoring method provides a reliable automatic supplementary mechanism to ensure the accuracy and consistency of data classification, improve the efficiency of data management and classification integrity, and thus provide a good data foundation for subsequent model training.

[0044] 2. The present invention utilizes the secondary classification labels in the existing data set to generate corresponding natural language descriptions and classify them with the help of zero-sample learning algorithms, thereby realizing automatic subclassification of unlabeled data and enabling the primary classification data to be more finely classified into the secondary directory, greatly reducing the time and cost of classification. At the same time, it enables the model to efficiently adapt to the multi-level classification requirements of different modal data, providing strong support for enriching the data structure and improving model performance.

[0045] 3. This method effectively identifies categories with insufficient data by traversing the secondary classification directory and detecting whether the amount of data is lower than the generation threshold, and automatically triggers data augmentation to make the data distribution more balanced. This strategy ensures that the number of samples in each classification directory meets the minimum requirement, avoids the negative impact of data skew on model training and analysis, improves the generalization ability and stability of the classification model, and improves the representation quality of small sample categories, providing a guarantee for the balance and representativeness of the overall data set training, thereby improving the model training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flow chart of a model training data processing method proposed by the present invention;

[0047] Figure 2 This is a schematic diagram of the structure of a model training data processing system proposed by the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] An Internet company designs a multimodal algorithm model and needs to process training data. In the past, the company used traditional manual labeling methods to process data, but the labeling results were uneven and took a lot of time. In order to improve the efficiency of model training data processing, a model training data processing method proposed in the present invention was used, such as Figure 1 As shown, including:

[0050] S01. Obtain Internet data;

[0051] S02. Obtain the MIME type of the Internet data through the HTTP response header, classify the Internet data according to the MIME type, obtain the primary classification data, and store the primary classification data in the primary classification directory; if the Internet data lacks the MIME type, use the type weighted scoring method to classify the Internet data into data types and store it in the primary classification directory;

[0052] S03. Extract the secondary classification labels corresponding to the primary classification data according to the existing data set, generate natural language text descriptions according to the secondary classification labels, use the zero-shot learning algorithm to perform secondary classification on the primary classification data, obtain secondary classification data, and store the secondary classification data in the secondary classification directory;

[0053] S04. Traverse the secondary classification directory and calculate the amount of data in the current secondary classification directory, determine whether the amount of data in the current secondary classification directory is less than the generation threshold, if the amount of data in the current secondary classification directory is less than the generation threshold, then expand the amount of data in the current secondary classification directory; otherwise, skip the current secondary classification directory and traverse the next secondary classification directory; until all secondary classification directories are traversed.

[0054] See also Figure 1 In S02, the first-level classification data includes text data, picture data, audio data and video data, and the first-level classification directory includes a text directory, a picture directory, an audio directory and a video directory.

[0055] Specifically, MIME type (Internet media type) is a method of describing Internet data types, and its format follows type / subtype, where type represents a category, including text, pictures, audio, and video, and subtype represents a subcategory under type. Some types of Internet data and typical MIME types corresponding to Internet data are shown in Table 1. Since the data of the multimodal model mainly focuses on four types of data: text, pictures, audio, and video, in this embodiment, Internet data is classified into text data, picture data, audio data, and video data, and the first-level classification directories established corresponding to these four types of data are text directory, picture directory, audio directory, and video directory.

[0056] The classification method based on MIME type effectively divides Internet data into four categories according to modality: text, picture, audio and video. It not only ensures the orderly storage of data in the primary classification directory, but also enables the multimodal model to quickly locate the required modal data.

[0057] Furthermore, using a type-weighted scoring method to classify the data types of the Internet data includes:

[0058] Get file extension, magic bytes and file size of Internet data with missing MIME type;

[0059] Calculate the Internet data of the missing MIME type according to the type weighted scoring formula to obtain type scores corresponding to all file types;

[0060] According to the type scores, the category with the highest score is selected as the predicted category of the document.

[0061] The type weighted scoring formula is:

[0062]

[0063] Wherein, t represents the Internet data with missing MIME type, S final() indicates the comprehensive score of the category to which the data belongs, ω ext Indicates the extension weight coefficient of the data, f ext () represents the extended name evaluation function of the data, ω magic Indicates the magic byte weight coefficient of the data, f magic () represents the magic byte evaluation function of the data, ω size Indicates the file size weight coefficient of the data, f magic () represents the file size evaluation function of the data.

[0064] Table 1. Some types of Internet data and typical MIME types corresponding to Internet data

[0065]

[0066] Specifically, the extension evaluation function is:

[0067]

[0068] Where ext(t) represents the extension of data t, ext_list k represents a list of all legal extensions for category k;

[0069] The magic byte evaluation function is:

[0070]

[0071] Among them, magic_bytes(t) represents the magic bytes of data t, magic_bytes k represents the standard magic byte of category k, and HM() represents the Hamming distance calculation function;

[0072] The file size evaluation function is:

[0073]

[0074] Among them, size(t) represents the file size of data t, max_size k Indicates the maximum limit range of category k, min_size k Represents the minimum limiting range of category k.

[0075] Table 2. Attribute table for Internet data with missing MIME type

[0076] data Extension Magic Bytes File size Example1.mp3 .mp3 49 44 33 30MB Example2.ogg .ogg FF D8 FF 1.5MB

[0077] For Internet data with missing MIME types, its attribute table is shown in Table 2. Its file extension, magic bytes and file size can be obtained for further data type judgment, and then the Internet data with missing MIME types is calculated according to the type weighted scoring formula to obtain the type scores of all file types. Finally, the category with the highest score is selected as the predicted category of the file according to the type score. In a feasible implementation scheme, the file size range of text type is 0MB-10MB, the file size range of image type is 0MB-20MB, the file size range of audio type is 0MB-200MB, and the file size range of video type is 0MB-4000MB. The suffix of the data Example1.mp3 is .mp3, the magic bytes are 49 44 33, and the file size is 5MB. Since the file extension of this file is .mp3, its file extension belongs to the audio data type, therefore, the extension matches the audio data, so f ext (t) = 1; the magic bytes of the file are 49 44 33, which exactly matches the magic bytes of the MP3 format file in the audio data. Therefore, the magic bytes match the audio data, so f magic (t) = 1; the file size is 30MB, which is in the file size range of the audio data type. Therefore, the file size is in the expected range, so f size (t) = 1; the weight is set to: ω ext =0.3,ω magic =0.5,ω size =0.1, the audio type score can be calculated by the formula Similarly, the text type score Image Type Score Video Type Score Based on the above results, we can know that Example1.mp3 is classified as an audio file. Based on the same classification method, Example2.ogg is classified as an image type, which means that the file is image data with a tampered extension.

[0078] By obtaining the file extension, magic byte, and file size of Internet data with missing MIME types, and combining it with a weighted scoring formula to calculate the type score of each file, the file type can be effectively judged based on multiple features. This method can improve the classification accuracy of files with missing MIME types through weighted evaluation of file type features, ensuring that even when the MIME type is missing, the file can be accurately classified into the most appropriate category, thereby ensuring the integrity and accuracy of data acquisition.

[0079] See also Figure 1In S03, the existing data set includes a text data set, a picture data set, an audio data set, and a video data set; the text data set is 20Newsgroups, the picture data set is ImageNet, the audio data set is UrbanSound8K, and the video data set is YouTube-8M.

[0080] Extracting secondary classification labels from existing datasets can provide rich, fine-grained labels for multimodal data, improve classification accuracy and reduce the need for manual annotation. These standardized labels help the model better generate natural language text descriptions, which helps achieve accurate and efficient multimodal classification.

[0081] Further, performing secondary classification on the primary classification data using a zero-sample learning algorithm includes:

[0082] Use the pre-trained model to convert the data in the first category into a high-dimensional feature vector;

[0083] Mapping the high-dimensional feature vector into a shared semantic space using contrastive learning;

[0084] Embed the description text of the secondary classification label into the shared semantic space through a language model to obtain a semantic vector of the secondary classification label;

[0085] Calculate the similarity between the high-dimensional feature vector and the secondary classification label according to cosine similarity;

[0086] According to the similarity, a category with the highest similarity to the high-dimensional feature vector is selected as a classification result, and the classification result is stored in a secondary classification directory.

[0087] Specifically, the pre-trained models include the pre-trained language model BERT, the pre-trained image model CLIP, the pre-trained audio model VGGish and the pre-trained video model R(2+1)D. The data under the first classification directory is converted into high-dimensional feature vectors through these pre-trained models. The contrastive learning framework ALIGN is used to process all high-dimensional feature vectors, and they are mapped to the vector space of the same dimension through a linear projection layer to ensure that all high-dimensional feature vectors are aligned in the shared semantic space, and a double contrastive loss function is used to encourage high-dimensional feature vectors with the same semantics to be close in the shared semantic space, while high-dimensional feature vectors with different semantics keep their distance. The description text of each secondary classification label is embedded into the shared semantic space through the pre-trained BERT to obtain the secondary classification label semantic vector. For each data sample to be classified (whether text, image, audio or video), its feature vector is first obtained from the shared semantic space, and the similarity between the feature vector and the secondary classification label semantic vector is calculated by cosine similarity, where the cosine similarity sim(x,ci ) = x·c i / ||x||||c i ||, where x represents the aligned feature vectors in the shared semantic space, c i represents the semantic vector of the secondary classification label, ||x|| represents the L2 norm of x, ||c i || means c i L2 norm; finally, according to the similarity, the category most similar to the high-dimensional feature vector is selected as the classification result, and the classification result is stored in the secondary classification directory.

[0088] Through the pre-trained model, each modal data is mapped to a shared semantic space, so that data of different modalities can be semantically compared in the same space. Using the text description of the secondary classification label and similarity matching, data categories can be automatically identified without specific label samples, greatly reducing the need for manual annotation. This process makes the classification process more flexible, scalable to new categories, and has high classification accuracy, which helps to efficiently organize and manage large-scale multimodal data. This zero-shot learning classification method provides an efficient and automated solution for the secondary classification of multimodal data.

[0089] See also Figure 1 In S04, the generation threshold calculation formula is:

[0090]

[0091] Among them, S represents the generation threshold, m represents the number of secondary classification types, N i Represents the amount of data in the i-th secondary classification directory.

[0092] Setting the generation threshold can effectively balance the amount of data in each secondary classification directory, ensure the balance of the data set, and avoid model bias or overfitting problems caused by insufficient data in some categories. The application of the generation threshold provides clear standards for data amplification, which helps to improve the representativeness and diversity of the data.

[0093] Further, expanding the data volume of the current secondary classification directory includes:

[0094] Select the data in the current secondary classification directory as training data for the data augmentation model, and select a corresponding data augmentation model according to the type of the training data;

[0095] Using the data of the current secondary classification directory to train the data augmentation model, generating new data samples through the data augmentation model, and obtaining generated data;

[0096] Verifying the similarity between the generated data and the training data;

[0097] The generated data is saved in the current secondary classification directory until the data volume of the current secondary classification directory meets the generation threshold requirement.

[0098] The data augmentation process automatically selects and trains the corresponding data augmentation model according to the data type of the secondary classification directory. The generated new samples can effectively increase the data volume and make the data of the small sample category richer. By verifying the similarity between the generated data and the original data, the quality and semantic consistency of the generated samples are ensured, and the augmented data is stored in the corresponding classification directory until the data volume reaches the generation threshold, ensuring the balance and representativeness of the data set.

[0099] Furthermore, the data augmentation model includes a text augmentation model, a picture augmentation model, an audio augmentation model and a video augmentation model; the text augmentation model is GPT-4, the picture augmentation model is CycleGAN, the audio augmentation model is WaveGAN, and the video augmentation model is MoCo-V3.

[0100] By combining generative models of different modalities, we specifically amplify text, images, audio, and video data, effectively increasing the diversity and quantity of data in each modality. By using targeted amplification models, we can generate amplified samples with high fidelity and semantic consistency to meet the data requirements of each modality, achieve a balance in training data, and thus ensure the generalization effect of subsequent model training.

[0101] Table 3. Comparison results of current data processing method and original data processing method under different models

[0102]

[0103] A model training data processing method used in this embodiment is compared with the original model training data processing method to verify the effectiveness of the method of this embodiment. The results of the comparative experiment are shown in Table 3. According to Table 3, the processing effect of the current data processing method on small sample data is much better than that of the original data processing method. In the three self-developed multimodal models, the minimum sample accuracy is 88.3%, 85.2% and 86.1%, respectively, which is 17.9%, 16.3% and 16.6% higher than the original data processing method. The main reason is that a variety of different modal generative models are used to expand the amount of secondary classification data, generate high-quality data, and thus ensure the balance and diversity of each classification directory data. In addition, the current data processing method also performs well in processing large sample data. The accuracy of the largest samples in the three self-developed multimodal models are 90.1%, 87.6% and 88.4% respectively. This is mainly because the MIME type and type weighted scoring can correctly classify the acquired Internet data at the first level. At the same time, combined with zero-sample learning, the second level classification is achieved without the need for a large amount of annotation, ensuring the reliability of the data, which enables the model to obtain higher quality pre-processed data during training.

[0104] This method effectively improves the efficiency of organizing, storing and using Internet data through automated multi-step classification and data augmentation methods. First, through MIME types and type weighted scoring, multimodal data can be accurately classified and stored in the primary classification directory to ensure reasonable data organization. Then, through the secondary label description obtained from the existing data set, combined with zero-shot learning, efficient secondary classification without a large amount of annotation is achieved, significantly reducing manual intervention in the classification process. Finally, data augmentation based on data volume generation thresholds ensures the balance and diversity of data in each classification directory, avoids model bias caused by insufficient data, and further improves the generalization ability and stability of the classification model, thereby achieving better model training effects in subsequent tasks.

[0105] Embodiment 2

[0106] A model training data processing system, such as Figure 2 As shown, including:

[0107] A data acquisition module, used to acquire Internet data;

[0108] A primary data classification module is used to obtain the MIME type of the Internet data through the HTTP response header, classify the Internet data according to the MIME type, obtain primary classification data, and store the primary classification data in a primary classification directory; if the Internet data lacks the MIME type, use a type weighted scoring method to classify the data type of the Internet data and store it in the primary classification directory;

[0109] A secondary data classification module is used to extract secondary classification labels corresponding to the primary classification data according to the existing data set, generate natural language text descriptions according to the secondary classification labels, perform secondary classification on the primary classification data using a zero-sample learning algorithm to obtain secondary classification data, and store the secondary classification data in a secondary classification directory;

[0110] The data amplification module is used to traverse the secondary classification directory and calculate the amount of data in the current secondary classification directory, and determine whether the amount of data in the current secondary classification directory is less than the generation threshold. If the amount of data in the current secondary classification directory is less than the generation threshold, the data amount of the current secondary classification directory is amplified; otherwise, the current secondary classification directory is skipped and the next secondary classification directory is traversed until all secondary classification directories are traversed.

[0111] Furthermore, the first-level classification data includes text data, picture data, audio data and video data, and the first-level classification directory includes a text directory, a picture directory, an audio directory and a video directory.

[0112] Furthermore, using a type-weighted scoring method to classify the data types of the Internet data includes:

[0113] Get file extension, magic bytes and file size of Internet data with missing MIME type;

[0114] Calculate the Internet data of the missing MIME type according to the type weighted scoring formula to obtain type scores corresponding to all file types;

[0115] According to the type scores, the category with the highest score is selected as the predicted category of the document.

[0116] The type weighted scoring formula is:

[0117] S final (t) = ω ext ·f ext (t)+ω magic ·f magic (t)+ω size ·f size (t);

[0118] Wherein, t represents the Internet data with missing MIME type, S final () indicates the comprehensive score of the category to which the data belongs, ω ext Indicates the extension weight coefficient of the data, f ext () represents the extended name evaluation function of the data, ω magic Indicates the magic byte weight coefficient of the data, f magic () represents the magic byte evaluation function of the data, ωsize Indicates the file size weight coefficient of the data, f magic () represents the file size evaluation function of the data.

[0119] Furthermore, the existing data set includes a text data set, a picture data set, an audio data set, and a video data set; the text data set is 20Newsgroups, the picture data set is ImageNet, the audio data set is UrbanSound8K, and the video data set is YouTube-8M.

[0120] Further, performing secondary classification on the primary classification data using a zero-sample learning algorithm includes:

[0121] Use the pre-trained model to convert the data in the first category into a high-dimensional feature vector;

[0122] Mapping the high-dimensional feature vector into a shared semantic space using contrastive learning;

[0123] Embed the description text of the secondary classification label into the shared semantic space through a language model to obtain a semantic vector of the secondary classification label;

[0124] Calculate the similarity between the high-dimensional feature vector and the secondary classification label according to cosine similarity;

[0125] According to the similarity, a category with the highest similarity to the high-dimensional feature vector is selected as a classification result, and the classification result is stored in a secondary classification directory.

[0126] Furthermore, the generation threshold calculation formula is:

[0127]

[0128] Among them, S represents the generation threshold, m represents the number of secondary classification types, N i Represents the amount of data in the i-th secondary classification directory.

[0129] Further, expanding the data volume of the current secondary classification directory includes:

[0130] Select the data in the current secondary classification directory as training data for the data augmentation model, and select a corresponding data augmentation model according to the type of the training data;

[0131] Using the data of the current secondary classification directory to train the data augmentation model, generating new data samples through the data augmentation model, and obtaining generated data;

[0132] Verifying the similarity between the generated data and the training data;

[0133] The generated data is saved in the current secondary classification directory until the data volume of the current secondary classification directory meets the generation threshold requirement.

[0134] Furthermore, the data augmentation model includes a text augmentation model, a picture augmentation model, an audio augmentation model and a video augmentation model; the text augmentation model is GPT-4, the picture augmentation model is CycleGAN, the audio augmentation model is WaveGAN, and the video augmentation model is MoCo-V3.

[0135] In addition, to achieve the above-mentioned purpose, the present invention also proposes a model training data processing electronic device, on which computer program instructions are stored. When the computer program instructions are executed by the electronic device, the electronic device implements the method described in Example 1.

[0136] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A model training data processing method, characterized in that: include: Access Internet data; Get the Internet data through the HTTP response header MIME Type, classify the Internet data according to the MIME type to obtain primary classification data, and store the primary classification data in the primary classification directory; if the Internet data lacks the MIME type, use the type weighted scoring method to classify the data type of the Internet data and store it in the primary classification directory; Extracting secondary classification labels corresponding to the primary classification data according to the existing data set, generating natural language text descriptions according to the secondary classification labels, performing secondary classification on the primary classification data using a zero-sample learning algorithm to obtain secondary classification data, and storing the secondary classification data in a secondary classification directory; Traversing the secondary classification directory and calculating the amount of data in the current secondary classification directory, determining whether the amount of data in the current secondary classification directory is less than a generation threshold, and if the amount of data in the current secondary classification directory is less than the generation threshold, expanding the amount of data in the current secondary classification directory; Otherwise, skip the current secondary classification directory and traverse the next secondary classification directory; Until all secondary classification directories are traversed.

2. A model training data processing method according to claim 1, characterized in that: The first-level classification data includes text data, picture data, audio data and video data, and the first-level classification directory includes a text directory, a picture directory, an audio directory and a video directory.

3. A model training data processing method according to claim 1, characterized in that: Using the type-weighted scoring method to classify the data types of the Internet data includes: Get file extension, magic bytes and file size of Internet data with missing MIME type; Calculate the Internet data of the missing MIME type according to the type weighted scoring formula to obtain type scores corresponding to all file types; Selecting a category with the highest score according to the type score as the predicted category of the document; The type weighted scoring formula is: S final (t)=ω ext ·f ext (t)+ω magic ·f magic (t)+ω size ·f size (t); Wherein, t represents the Internet data with missing MIME type, S final () indicates the comprehensive score of the category to which the data belongs, ω ext Indicates the extension weight coefficient of the data, f ext () represents the extended name evaluation function of the data, ω magic Indicates the magic byte weight coefficient of the data, f magic () represents the magic byte evaluation function of the data, ω size Indicates the file size weight coefficient of the data, f magic () represents the file size evaluation function of the data.

4. A model training data processing method according to claim 1, characterized in that: The existing data sets include text data sets, picture data sets, audio data sets, and video data sets; the text data set is 20Newsgroups, the picture data set is ImageNet, the audio data set is UrbanSound8K, and the video data set is YouTube-8M.

5. A model training data processing method according to claim 1, characterized in that: Using a zero-sample learning algorithm to perform secondary classification on the primary classification data includes: Use the pre-trained model to convert the data in the first category into a high-dimensional feature vector; Mapping the high-dimensional feature vector into a shared semantic space using contrastive learning; Embed the description text of the secondary classification label into the shared semantic space through a language model to obtain a semantic vector of the secondary classification label; Calculate the similarity between the high-dimensional feature vector and the secondary classification label according to cosine similarity; According to the similarity, a category with the highest similarity to the high-dimensional feature vector is selected as a classification result, and the classification result is stored in a secondary classification directory.

6. A model training data processing method according to claim 1, characterized in that: The generation threshold calculation formula is: Among them, S represents the generation threshold, m represents the number of secondary classification types, N i Represents the amount of data in the i-th secondary classification directory.

7. A model training data processing method according to claim 1, characterized in that: Expanding the data volume of the current secondary classification directory includes: Select the data in the current secondary classification directory as training data for the data augmentation model, and select a corresponding data augmentation model according to the type of the training data; Using the data of the current secondary classification directory to train the data augmentation model, generating new data samples through the data augmentation model, and obtaining generated data; Verifying the similarity between the generated data and the training data; The generated data is saved in the current secondary classification directory until the data volume of the current secondary classification directory meets the generation threshold requirement.

8. A model training data processing method according to claim 7, characterized in that: The data augmentation model includes a text augmentation model, a picture augmentation model, an audio augmentation model and a video augmentation model; the text augmentation model is GPT-4, the picture augmentation model is CycleGAN, the audio augmentation model is WaveGAN, and the video augmentation model is MoCo-V3.

9. A model training data processing system, characterized in that: include: A data acquisition module, used to acquire Internet data; A primary data classification module is used to obtain the MIME type of the Internet data through the HTTP response header, classify the Internet data according to the MIME type, obtain primary classification data, and store the primary classification data in a primary classification directory; if the Internet data lacks the MIME type, use a type weighted scoring method to classify the data type of the Internet data and store it in the primary classification directory; A secondary data classification module is used to extract secondary classification labels corresponding to the primary classification data according to the existing data set, generate natural language text descriptions according to the secondary classification labels, perform secondary classification on the primary classification data using a zero-sample learning algorithm to obtain secondary classification data, and store the secondary classification data in a secondary classification directory; A data amplification module, used to traverse the secondary classification directory and calculate the amount of data in the current secondary classification directory, determine whether the amount of data in the current secondary classification directory is less than a generation threshold, and if the amount of data in the current secondary classification directory is less than the generation threshold, amplify the amount of data in the current secondary classification directory; Otherwise, skip the current secondary classification directory and traverse the next secondary classification directory; Until all secondary classification directories are traversed.

10. An electronic device for processing model training data, characterized in that: The device comprises at least a processor and a memory, and the processor is configured to implement the steps of the method according to any one of claims 1 to 8 when executing a computer program stored in the memory.

Images