Data screening method and device, equipment and medium

Abstract

The invention relates to the technical field of data processing, and discloses a data screening method and device, equipment and a medium, and the method comprises the steps: obtaining an initial training data set, and carrying out the preprocessing of the initial training data set, and obtaining a target training data set, the sample data in the initial training data set comprises image data, text data corresponding to the image data and a target text sequence; for each piece of sample data in the target training data set, calculating a quality evaluation index corresponding to the sample data; sorting all sample data in all the target training data sets based on the calculated quality evaluation indexes; and screening a preset proportion of sample data from the sorted sample data to form a screened data set for multi-modal large model training. According to the method, the data screening efficiency and accuracy are improved, the overall quality of the screened data set is improved, the multi-modal large model training efficiency is improved, and the training cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data filtering method, apparatus, device, and medium. Background Technology

[0002] In recent years, Multimodal Large Language Models (MLLMs) have made significant progress in tasks such as visual understanding and image / text generation by jointly modeling information such as images and text, and have been widely applied in various fields such as intelligent question answering, content generation, and industrial inspection. As the scale of model parameters increases and the complexity of the structure becomes more complex, its performance improvement increasingly depends on high-quality, diverse, and semantically consistent training data.

[0003] Currently, large-scale multimodal models typically construct training sets by collecting image-text pairs, instruction-response pairs, and other data on a massive scale from the internet. However, such raw data generally suffers from inconsistent quality, including semantic mismatch between images and text, redundant text descriptions, inconsistent instruction and response styles, severe noise, and numerous duplicate samples. Especially during the multimodal instruction fine-tuning stage, low-quality data can severely impact model performance, leading to unstable outputs, decreased alignment capabilities, and inconsistent inference results. Simultaneously, the simultaneous growth of model and data scale dramatically increases the demand for computational, storage, and time resources for training, pushing the traditional training paradigm of "pile data and computing power" to its engineering bottlenecks.

[0004] Current training processes typically employ a full-data training strategy. Low-quality and noisy samples can interfere with model parameter updates, reducing training efficiency; redundant or anomalously distributed data wastes computational resources and increases training costs. Related research indicates that "data quality is more important than data size." By properly selecting training data, using only a subset of high-quality samples can achieve or even surpass the effects of training with the full dataset, while significantly reducing resource consumption. Therefore, how to achieve automated, quantifiable, and scalable data selection has become a critical technical problem that urgently needs to be solved in this field.

[0005] In terms of data filtering methods, existing technologies still heavily rely on manual rule setting or manual annotation. While manual filtering is applicable in small-scale data scenarios, it is costly, inefficient, and difficult to scale in large-scale multimodal data environments. Furthermore, different task types and model structures have varying requirements for data quality, making it difficult to flexibly adapt fixed rules or human experience, resulting in low accuracy and poor universality and portability of data filtering. Summary of the Invention

[0006] This invention provides a data filtering method, apparatus, computer equipment, and medium to solve the technical problem that existing data filtering methods have low efficiency and accuracy.

[0007] Firstly, a data filtering method is provided, including: An initial training dataset is obtained, and the initial training dataset is preprocessed to obtain a target training dataset. The sample data in the initial training dataset includes image data, text data corresponding to the image data, and target text sequences. For each sample data in the target training dataset, calculate its corresponding quality evaluation index; Based on the calculated quality assessment index, all the sample data in the target training dataset are sorted. A predetermined proportion of the sample data is selected from the sorted sample data to form a filtered dataset for training a multimodal large model.

[0008] Secondly, a data filtering device is provided, comprising: The acquisition and processing unit is used to acquire an initial training dataset and preprocess the initial training dataset to obtain a target training dataset, wherein the sample data in the initial training dataset includes image data, text data corresponding to the image data, and target text sequences; The computing unit is used to calculate the corresponding quality evaluation index for each sample data in the target training dataset; A sorting unit is used to sort all the sample data in the target training dataset based on the calculated quality assessment index. The filtering unit is used to filter a preset proportion of the sample data from the sorted sample data to form a filtered dataset for training a multimodal large model.

[0009] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described data filtering method.

[0010] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described data filtering method.

[0011] The above-described data filtering method, apparatus, computer equipment, and storage medium can obtain an initial training dataset and preprocess it to obtain a target training dataset. The initial training dataset includes sample data such as image data, text data corresponding to the image data, and target text sequences. For each sample data in the target training dataset, a corresponding quality evaluation index is calculated. Based on the calculated quality evaluation index, all sample data in the target training dataset are sorted. A predetermined proportion of sample data is selected from the sorted sample data to form a filtered dataset for multimodal large-scale model training. In this invention, the initial training dataset is first preprocessed to obtain the target training dataset. Then, a quality evaluation index is calculated for each sample data in the target training dataset. Finally, all sample data in the target training dataset are sorted based on the quality evaluation index, and a predetermined proportion of sample data is selected from the sorted sample data to form a filtered dataset for multimodal large-scale model training. This avoids inefficient and subjective manual filtering, improves the efficiency and accuracy of data filtering, enhances the overall quality of the filtered dataset, thereby improving the efficiency of multimodal large-scale model training and reducing training costs. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a data filtering method in one embodiment of the present invention; Figure 2 yes Figure 1 A schematic diagram of a specific implementation of step S120; Figure 3 This is a flowchart illustrating a data filtering method according to another embodiment of the present invention; Figure 4 This is a schematic block diagram of a data filtering device in one embodiment of the present invention; Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] The data filtering method provided in this invention can be applied to either a client or a server. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. Currently, existing data filtering methods have low efficiency and accuracy. To address these issues, this invention proposes a data filtering method. This method first preprocesses the acquired initial training dataset to obtain a target training dataset. Then, it calculates the quality evaluation index for each sample data in the target training dataset. Finally, based on the quality evaluation index, it sorts all sample data in the target training dataset and selects a predetermined proportion of sample data from the sorted dataset to form a filtered dataset for training a multimodal large model. This avoids inefficient and subjective manual filtering, improves the efficiency and accuracy of data filtering, enhances the overall quality of the filtered dataset, thereby improving the efficiency of multimodal large model training and reducing training costs. The invention will be described in detail below through specific embodiments.

[0016] Please see Figure 1 As shown, Figure 1 A flowchart of a data filtering method provided in an embodiment of the present invention includes the following steps: S110-S140.

[0017] S110. Obtain an initial training dataset and preprocess the initial training dataset to obtain a target training dataset, wherein the sample data in the initial training dataset includes image data, text data corresponding to the image data, and target text sequences.

[0018] Specifically, the preprocessing includes at least one of the following: data format standardization, illegal character cleaning, missing field detection, empty sample removal, and field integrity verification. More specifically, the initial training dataset is typically obtained by integrating publicly available multimodal datasets from the internet, collecting image-text pairs using web crawlers, or extracting data from labeled data in specific domains (such as medicine or autonomous driving). The obtained initial training dataset contains several sample data points, each serving as a basic unit, which includes at least image data, text data corresponding to the image, and a target text sequence. The text data takes various forms, including user instructions, question descriptions, text responses, or multi-turn dialogue content, while the target text sequence is the response text that serves as the model's learning objective.

[0019] After obtaining the initial training dataset, it needs to be preprocessed to obtain a high-quality, uniformly formatted target training dataset. Specific preprocessing operations include, but are not limited to: data format unification, converting image and text data from different sources in the initial training dataset into a uniform specification required by the model; illegal character cleaning, to remove noise such as garbled characters and irrelevant symbols from the text; missing field detection and empty sample removal, to delete sample data with missing images, empty text data, or invalid content; and field integrity verification, to ensure that images are correctly associated with their corresponding text information without mismatches.

[0020] It should be noted that the multimodal data in the target training dataset collectively provide the foundation for the model to learn cross-modal semantic alignment and complex task solving capabilities. Preprocessing can effectively improve the regularity and usability of the data, ensuring data quality from the source and providing reliable input for subsequent automated and quantifiable data quality assessment and screening, thereby improving the overall model training efficiency and effectiveness.

[0021] S120. For each sample data in the target training dataset, calculate its corresponding quality evaluation index.

[0022] In one embodiment, when the quality assessment metric is a perplexity value, such as Figure 2As shown, step S120 includes steps S121-S123: S121, for each sample data in the target training dataset, the image data and text data in the sample data are input as conditions to the multimodal large model; S122, under the conditional input, the multimodal large model performs conditional probability prediction on the target text sequence; S123, calculates the perplexity value of the sample data based on the conditional prediction probability of each word in the target text sequence. More specifically, step S123 includes: calculating the logarithmic average of the conditional prediction probabilities of all words in the target text sequence to obtain the average negative log-likelihood value of the sample data; performing an exponential operation on the average negative log-likelihood value to obtain the perplexity value of the sample data. For ease of understanding, the implementation process is described in detail below: First, conditional input encoding is performed. For each sample data point, its multimodal information needs to be normalized into the standard input format of the multimodal large model. Specifically, image data (such as image pixel matrices or feature vectors) and instructional text data (such as user instructions, problem descriptions, and other contextual text) from the sample data are jointly encoded and used as conditional inputs into the multimodal large model. This step typically involves a visual encoder extracting features from the image and a text encoder encoding the instructional text. Finally, the features from the two modalities are aligned and fused at the model input layer to form a joint representation containing complete semantic conditions, providing constraints and background information for subsequent sequence prediction.

[0023] Secondly, conditional probability prediction of the target text sequence is performed. Given the joint constraints of the image and text, the sequence generation and probabilistic calculation mechanism of the multimodal large model is activated to perform autoregressive conditional probability prediction on a token-by-token basis for the given target text sequence (i.e., the expected response or answer text generated by the model) in the sample data. Specifically, the model starts with the initial symbol and, based on the given image-text conditions, calculates the predicted probability distribution of each token in its vocabulary as the first token of the sequence, and records the conditional probability of the actual first token in the target text sequence under this distribution. Then, the true first token is incorporated into the historical context, and under the combined conditions of "image conditions + text conditions + historically generated tokens," the conditional probability of the second true token is predicted and recorded; this process continues until the last token of the target text sequence is traversed. Essentially, this process allows the model to "review" the probability of generating this target response. The more "accepted" or "familiar" the model is with the sequence, the higher the probability of predicting the correct token at each step.

[0024] Finally, the perplexity value of the sample data is calculated based on the conditional prediction probability. This is done after obtaining each word in the target text sequence. Corresponding conditional prediction probability Then, the perplexity value of the sample data is calculated according to formula (1), where, in formula (1), This represents the input image data. This represents the single-round instruction text corresponding to the image. This represents the total number of tokens in the target text sequence. This represents the i-th word in the target text sequence. This represents the conditional prediction probability of a multimodal large model for the current word, given image conditions, instruction conditions, and historically generated words. The calculation process can be divided into two sub-steps: First, calculate the mean negative log-likelihood of the sample data. That is, take the natural logarithm of the prediction probabilities of all N words, sum them, then average them and add a negative sign. The mean negative log-likelihood reflects the average difficulty of the model "generating" the entire target sequence; the smaller the value, the more natural and easier the sequence is considered to appear by the model. Second, perform an exponential operation to obtain the perplexity value. Perform an exponential operation with the natural constant e as the base on the mean negative log-likelihood value obtained in the first step, i.e., PPL = exp(mean negative log-likelihood). This operation converts the average of the logarithms of the probabilities back into an easily interpretable scalar. The final perplexity value has an intuitive meaning: the lower the value, the less uncertainty the model has in predicting the target text sequence under given conditions, the higher the semantic consistency between the target file sequence and the conditions, and the more standard and fluent the language, thus judging that the sample data is of higher quality; conversely, a high perplexity suggests that the sample data may have problems such as noise, semantic inconsistency or expression abnormalities. (1) It should be noted that the perplexity value calculation process is usually executed efficiently in batch inference, and finally outputs a list of perplexity values ​​corresponding to the target training dataset, providing accurate and comparable quantitative basis for subsequent sorting and filtering.

[0025] In another embodiment, when the quality assessment metric is a style consistency score, step S120 includes: for each sample data in the target training dataset, using a constructed style consistency-aware model to evaluate the consistency between the text data of each sample data in the target training dataset and the target text sequence, to obtain the style consistency score. Specifically, firstly, the style consistency-aware model is built based on a pre-trained language model (e.g., BERT), which serves as the basic feature extraction encoder and has the ability to deeply encode text semantics and style features. During evaluation, the text data (instruction text) and the corresponding target text sequence (response text) in the sample data are input into the feature extraction encoder. The feature extraction encoder outputs feature vectors representing the overall semantics and style of the two texts, typically extracting the output vector corresponding to its preset flag as the aggregate representation of the text. Subsequently, the cosine similarity between the two feature vectors is calculated. Cosine similarity quantifies the consistency of two vectors by measuring their proximity in direction, and its value ranges from [-1, 1]. A higher value indicates that the two are more similar in expression style, language habits, and logical structure. Finally, the calculated cosine similarity value is used as the style consistency score for this sample data. This automated process enables an efficient and objective assessment of the consistency of command-response pairs at the language style level in large-scale datasets, providing a core quantitative indicator for subsequent data screening based on style consistency.

[0026] It's important to note that both style consistency score and perplexity score are automated data quality assessment metrics, but they differ in their focus and underlying principles. The perplexity score originates directly from the trained multimodal model itself. It measures the difficulty of the model "generating" target text given a text and image, focusing on evaluating the semantic correctness, linguistic standardization, and logical fit with the context. It's an intrinsic metric based on generation probability, reflecting the model's cognitive uncertainty. The style consistency score, on the other hand, is calculated by an independent, pre-trained language model. It measures the consistency between the instruction and response texts in terms of expression style, vocabulary habits, and text attributes, focusing on evaluating the uniformity and coordination of instruction adherence. It's a relatively external metric based on representational similarity. Logically, samples with low perplexity scores are typically considered "high-quality samples that are easy for the model to learn," while samples with high style consistency are considered "standardized samples with consistent instruction and response." Both can be used independently or in combination, providing a basis for data selection from different dimensions.

[0027] S130. Based on the calculated quality assessment index, sort all the sample data in the target training dataset.

[0028] Specifically, when the quality assessment metric is perplexity, since a lower value indicates higher sample data quality, all sample data are sorted in ascending order. When the quality assessment metric is style consistency score, since a higher value indicates higher sample data quality, all sample data are sorted in descending order. This sorting operation organizes all sample data into an ordered sequence from best to worst, providing a precise data order basis for subsequent proportional screening.

[0029] S140. Select a predetermined proportion of the sample data from the sorted sample data to form a filtered dataset for training a multimodal large model.

[0030] Specifically, based on a preset proportion (e.g., 30%, 50%), sample data of the corresponding proportion are selected from the globally ordered sequence. When the quality assessment metric is perplexity, the sample data with the highest perplexity is selected; when the quality assessment metric is style consistency score, the sample data with the highest score is selected. Finally, this selected high-quality sample data is aggregated to form a smaller, higher-quality filtered dataset, which is directly used for subsequent multimodal large-scale model training.

[0031] Figure 3 This is a flowchart illustrating a data filtering method according to another embodiment of the present invention, such as... Figure 3 As shown, in this embodiment, the method includes steps S110-S190. That is, in this embodiment, after step S140 in the above embodiment, the method further includes steps S150-S190.

[0032] S150. The filtered dataset obtained according to the first quality assessment index is used as the first filtered dataset. S160. The filtered dataset obtained by filtering according to the second quality assessment index is used as the second filtered dataset, wherein the first quality assessment index and the second quality assessment index are either the perplexity value or the style consistency score. S170. Use the first filtered dataset, the second filtered dataset, and the target training dataset to train or fine-tune the same multimodal large model, respectively. S180. Collect the loss curves, convergence speeds, and performance metrics on multiple downstream evaluation tasks during the training or fine-tuning of the multimodal large model using the first screening dataset, the second screening dataset, and the target training dataset. S190. Compare the loss curve, the convergence speed, and the performance indicators to analyze the specific impact of different datasets on the performance, training efficiency, and output style stability of the multimodal large model and obtain the analysis results. Generate a result comparison chart based on the analysis results.

[0033] Specifically, the quality assessment metrics include a first quality assessment metric and a second quality assessment metric. If the perplexity value is used as the first quality assessment metric, all sample data in the target training dataset are sorted in ascending order based on this metric, and a predetermined proportion (e.g., the top 30%) of samples are selected to form the first screening dataset. This dataset is characterized by containing sample data that the model considers to be semantically clearest, easiest to generate, and predictable—"low uncertainty" samples. If the style consistency score is used as the second quality assessment metric, the samples are sorted in descending order, and the same proportion (e.g., the top 30%) of samples are selected to form the second screening dataset. This dataset is characterized by a high degree of consistency in language style and expression between the instruction text and response text of its sample data. The first and second quality assessment metrics must be different; one is the perplexity value, and the other is the style consistency score, thus forming two comparative data subsets under different screening logics.

[0034] After obtaining the first and second selection datasets, the first and second selection datasets, along with the target training dataset (as a baseline), are used to train or fine-tune a large multimodal model with identical structure, parameters, and training configuration. During training, three sets of key data are collected and recorded simultaneously: 1) the curve of training loss changing with the number of iterations (loss curve); 2) the number of training steps or time required for the model to reach a stable loss (convergence speed); and 3) the quantitative performance metrics of the model on a series of standard downstream evaluation tasks (such as visual question answering (VQA), image caption generation, and multimodal inference) after training. The three sets of collected data are then compared and analyzed horizontally to quantitatively evaluate the specific impact of different datasets on model performance, training efficiency, and output style.

[0035] The analysis results and comparison charts show that the full dataset (the target training dataset) still delivers the best final performance, indicating its most comprehensive information coverage. However, datasets filtered by perplexity values ​​(especially those with 50% or 70% perplexity) typically outperform the full dataset in terms of loss curves and convergence speed, exhibiting faster convergence and more stable loss reduction, demonstrating higher training efficiency. Simultaneously, their evaluation scores are closest to the full dataset, showing the potential to maintain core performance while significantly reducing data volume (saving computational power and time). While datasets filtered by style consistency scores are slightly inferior in this evaluation, they may perform better in tasks emphasizing output style consistency. Therefore, if training efficiency and cost are prioritized, perplexity value filtering is the ideal choice; if extreme performance is sought and resources are sufficient, the full dataset is superior; and if there is a strong requirement for output style stability, style consistency score filtering can be considered.

[0036] The data filtering method in this invention first preprocesses the acquired initial training dataset to obtain the target training dataset, then calculates the perplexity value or style consistency score of each sample data in the target training dataset, and finally sorts all the sample data in the target training dataset based on the perplexity value or style consistency score. A preset proportion of sample data is then selected from the sorted sample data to form a filtered dataset for training multimodal large models. This method not only effectively identifies and removes low-quality samples that are semantically unstable, informationally redundant, or noisy, but also avoids inefficient and subjective manual filtering, improving the efficiency and accuracy of data filtering, enhancing the overall quality of the filtered dataset, thereby improving the efficiency of multimodal large model training and reducing training costs.

[0037] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0038] The software tools or components not belonging to our company that appear in the embodiments of this application are merely examples and do not represent actual use.

[0039] In one embodiment, a data filtering device 200 is provided, which corresponds one-to-one with the data filtering methods in the above embodiments. For example... Figure 4 As shown, the data filtering device 200 includes an acquisition and processing unit 201, a calculation unit 202, a sorting unit 203, and a filtering unit 204. Detailed descriptions of each functional module are as follows: The acquisition and processing unit 201 is used to acquire an initial training dataset and preprocess the initial training dataset to obtain a target training dataset, wherein the sample data in the initial training dataset includes image data, text data corresponding to the image data, and target text sequence; The calculation unit 202 is used to calculate the corresponding quality evaluation index for each sample data in the target training dataset; The sorting unit 203 is used to sort all the sample data in the target training dataset based on the calculated quality assessment index. The filtering unit 204 is used to filter a preset proportion of the sample data from the sorted sample data to form a filtered dataset for training a multimodal large model.

[0040] In one embodiment, the calculation unit 202 is specifically used for: For each sample data in the target training dataset, the image data and text data in the sample data are used as conditional inputs into the multimodal large model; Under the given input conditions, the multimodal large model performs conditional probability prediction on the target text sequence. The perplexity value of the sample data is calculated based on the conditional prediction probability of each word in the target text sequence.

[0041] In one embodiment, the computing unit 202 is further configured to: The log-average of the conditional prediction probabilities of all words in the target text sequence is calculated to obtain the average negative log-likelihood value of the sample data. The perplexity value of the sample data is obtained by performing an exponential operation on the average negative log-likelihood value.

[0042] In one embodiment, the computing unit 202 is further configured to: For each sample data in the target training dataset, the consistency between the text data of each sample data in the target training dataset and the target text sequence is evaluated using the constructed style consistency awareness model to obtain the style consistency score.

[0043] In one embodiment, the data filtering device 200 further includes: The first unit is used to use the filtered dataset obtained according to the first quality assessment index as the first filtered dataset. The second unit is used to use the filtered dataset obtained by filtering according to the second quality assessment index as the second filtered dataset, wherein the first quality assessment index and the second quality assessment index are either the perplexity value or the style consistency score. The training fine-tuning unit is used to train or fine-tune the same multimodal large model using the first selected dataset, the second selected dataset, and the target training dataset, respectively. The collection unit is used to collect the loss curve, convergence speed, and performance metrics on multiple downstream evaluation tasks during the training or fine-tuning of the multimodal large model using the first screening dataset, the second screening dataset, and the target training dataset. The comparative analysis unit is used to compare the loss curve, the convergence speed, and the performance index, analyze the specific impact of different datasets on the performance, training efficiency, and output style stability of the multimodal large model, and generate a result comparison chart based on the analysis results.

[0044] The data filtering device in this invention first preprocesses the acquired initial training dataset to obtain the target training dataset, then calculates the quality evaluation index of each sample data in the target training dataset, and finally sorts all the sample data in the target training dataset based on the quality evaluation index. From the sorted sample data, a preset proportion of sample data is selected to form a filtered dataset for multimodal large model training. This avoids inefficient and subjective manual filtering, improves the efficiency and accuracy of data filtering, enhances the overall quality of the filtered dataset, thereby improving the efficiency of multimodal large model training and reducing training costs.

[0045] The aforementioned data filtering device can be implemented as a computer program, which can, for example... Figure 5 It runs on the computer device shown.

[0046] Please see Figure 5 , Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. The computer device 300 is a device capable of data filtering.

[0047] See Figure 5 The computer device 300 includes a processor 302, a memory, and a network interface 305 connected via a system bus 301. The memory may include a non-volatile storage medium 303 and internal memory 304.

[0048] The non-volatile storage medium 303 may store an operating system 3031 and a computer program 3032. When the computer program 3032 is executed, it causes the processor 302 to perform a data filtering method.

[0049] The processor 302 provides computing and control capabilities to support the operation of the entire computer device 300.

[0050] The internal memory 304 provides an environment for the execution of the computer program 3032 in the non-volatile storage medium 303. When the computer program 3032 is executed by the processor 302, the processor 302 can perform a data filtering method.

[0051] This network interface 305 is used for network communication with other devices. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device 300 to which the present invention is applied. The specific computer device 300 may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0052] The processor 302 is used to run a computer program 3032 stored in a memory to implement any embodiment of the above-described data filtering method.

[0053] It should be understood that, in this embodiment of the invention, the processor 302 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0054] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by a processor in the computer system to implement the process steps of the embodiments of the above methods.

[0055] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program. When executed by a processor, the computer program causes the processor to perform any embodiment of the data filtering method described above.

[0056] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0057] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0058] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0059] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0060] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0061] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0062] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Since these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0063] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A data filtering method, characterized in that, include: An initial training dataset is obtained, and the initial training dataset is preprocessed to obtain a target training dataset. The sample data in the initial training dataset includes image data, text data corresponding to the image data, and target text sequences. For each sample data in the target training dataset, calculate its corresponding quality evaluation index; Based on the calculated quality assessment index, all the sample data in the target training dataset are sorted. A predetermined proportion of the sample data is selected from the sorted sample data to form a filtered dataset for training a multimodal large model.

2. The data filtering method as described in claim 1, characterized in that, The quality assessment metric is the perplexity value. The step of calculating the corresponding quality assessment metric for each sample data in the target training dataset includes: For each sample data in the target training dataset, the image data and text data in the sample data are used as conditional inputs into the multimodal large model; Under the given input conditions, the multimodal large model performs conditional probability prediction on the target text sequence. The perplexity value of the sample data is calculated based on the conditional prediction probability of each word in the target text sequence.

3. The data filtering method as described in claim 2, characterized in that, The step of calculating the perplexity value of the sample data based on the conditional prediction probability of each word in the target text sequence includes: The log-average of the conditional prediction probabilities of all words in the target text sequence is calculated to obtain the average negative log-likelihood value of the sample data. The perplexity value of the sample data is obtained by performing an exponential operation on the average negative log-likelihood value.

4. The data filtering method as described in claim 1, characterized in that, The quality assessment metric is a style consistency score; the step of calculating the corresponding quality assessment metric for each sample data in the target training dataset includes: For each sample data in the target training dataset, the consistency between the text data of each sample data in the target training dataset and the target text sequence is evaluated using the constructed style consistency awareness model to obtain the style consistency score.

5. The data filtering method as described in claim 1, characterized in that, The quality assessment indicators include a first quality assessment indicator and a second quality assessment indicator, and the method further includes: The filtered dataset obtained by filtering according to the first quality assessment index is used as the first filtered dataset; The filtered dataset obtained by filtering according to the second quality assessment index is used as the second filtered dataset, wherein the first quality assessment index and the second quality assessment index are either the perplexity value or the style consistency score. The first filtered dataset, the second filtered dataset, and the target training dataset are used to train or fine-tune the same multimodal large model.

6. The data filtering method as described in claim 5, characterized in that, The method further includes: The loss curves, convergence speeds, and performance metrics on multiple downstream evaluation tasks are collected during the training or fine-tuning of the multimodal large model using the first selected dataset, the second selected dataset, and the target training dataset, respectively. By comparing the loss curve, the convergence speed, and the performance metrics, the specific impact of different datasets on the performance, training efficiency, and output style stability of the multimodal large model is analyzed, and the analysis results are obtained. A result comparison chart is generated based on the analysis results.

7. The data filtering method according to any one of claims 1-6, characterized in that, The preprocessing includes at least one of the following: data format standardization, illegal character cleaning, missing field detection, empty sample removal, and field integrity verification.

8. A data filtering device, characterized in that, include: The acquisition and processing unit is used to acquire an initial training dataset and preprocess the initial training dataset to obtain a target training dataset, wherein the sample data in the initial training dataset includes image data, text data corresponding to the image data, and target text sequences; The computing unit is used to calculate the corresponding quality evaluation index for each sample data in the target training dataset; A sorting unit is used to sort all the sample data in the target training dataset based on the calculated quality assessment index. The filtering unit is used to filter a preset proportion of the sample data from the sorted sample data to form a filtered dataset for training a multimodal large model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the data filtering method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the data filtering method as described in any one of claims 1 to 7.

Images