Synthetic data comprehensive evaluation method based on large model

Abstract

The invention discloses a synthetic data comprehensive evaluation method based on a large model, and the method comprises the steps: obtaining target synthetic data of the large model, and obtaining target image data and target text data; inputting the target image data into an image machine learning model to obtain an image standard quality grade, inputting the target text data into a text machine learning model to obtain a text standard quality grade, and obtaining a first quality grade; obtaining the number of image clusters and the number of text clusters by utilizing clustering analysis, and obtaining a second quality grade; obtaining similar historical synthetic data and similar data utility, and obtaining a third quality grade according to the similar data utility; according to the first quality grade, the second quality grade and the third quality grade, obtaining a comprehensive quality grade of the target synthetic data; quality evaluation is carried out on the synthetic data from multiple aspects, the comprehensiveness and accuracy of the evaluation result are guaranteed, similar data utility is added into the evaluation system of the synthetic data, and the data utility of the synthetic data is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a comprehensive evaluation method for synthetic data based on a large model. Background Art

[0002] In recent years, with the rapid development of computer technology, the research on the massive data generated has received more and more attention. Among them, the processing of synthetic data used to train large models has also received great attention. Synthetic data is data artificially generated by computer programs. Using synthetic data also increases training data, which can save data collection costs or meet privacy requirements. Evaluating the data quality of synthetic data is of great significance to improving the model accuracy of large models.

[0003] At present, the Chinese invention patent with application number 202410681220.9 discloses a method and device for assessing the quality of synthetic data sets based on reference image samples. The method integrates image labels, content diversity and intrinsic characteristics, and completes synthetic data quality assessment in different scenarios without the need for prior knowledge. The quality assessment efficiency is improved through parallel calculation after label grouping. However, only the image data is evaluated, and the overall quality of the synthetic data is not evaluated. In addition, when evaluating data quality, the impact of data utility on quality assessment is not considered. Summary of the invention

[0004] The technical problem solved by the present invention is that the prior art only evaluates the image data but does not evaluate the overall quality of the synthesized data, and when evaluating the data quality, does not take into account the impact of the data utility on the quality evaluation.

[0005] In order to solve the above technical problems, the present invention provides a synthetic data comprehensive evaluation method based on a large model, comprising the following steps: Step S1, obtaining target synthesis data of a large model, and respectively obtaining target image data and target text data of the target synthesis data; Step S2, inputting the target image data into the image machine learning model to obtain the image standard quality level, inputting the target text data into the text machine learning model to obtain the text standard quality level, and obtaining the first quality level according to the image standard quality level and the text standard quality level; Step S3, using cluster analysis to classify the target image data and the target text data respectively, obtain the number of image clusters and the number of text clusters, and obtain a second quality level according to the number of image clusters and the number of text clusters; Step S4, calling up the historical synthetic data of the large model, obtaining similar historical synthetic data whose similarity with the target synthetic data is greater than an expected threshold in the historical synthetic data, obtaining similar data utility of the target synthetic data based on the similar historical synthetic data, and obtaining a third quality level based on the similar data utility; Step S5, obtaining a comprehensive quality level of the target synthetic data according to the first quality level, the second quality level and the third quality level.

[0006] As a preferred solution of the synthetic data comprehensive evaluation method based on a large model described in the present invention, wherein: The step S2 specifically includes the following steps: Step S201, obtaining a synthetic data set, and extracting an image data set and a text data set from the synthetic data set; Step S202, performing data annotation on the image data set and the text data set respectively, wherein the data annotation includes annotating the image data with image feature values ​​according to the image features, and annotating the text data with text feature values ​​according to the data features; Step S203, inputting the annotated image data set into the image machine learning model, using the image data as input and the image feature values ​​as output to train the image machine learning model, and calculating the image feature values ​​according to certain weights to obtain the image standard quality level, inputting the annotated text data set into the text machine learning model, using the text data as input and the text feature values ​​as output to train the text machine learning model, and calculating the text feature values ​​according to certain weights to obtain the text standard quality level; Step S204, inputting the target image data into the image machine learning model to obtain the image standard quality level, and inputting the target text data into the text machine learning model to obtain the text standard quality level; Step S205 , calculating the image standard quality level and the text standard quality level according to a certain weight to obtain a first quality level of the target synthetic data.

[0007] As a preferred solution of the synthetic data comprehensive evaluation method based on a large model described in the present invention, wherein: The step S3 specifically includes the following steps: Step S301, randomly extracting j image data and j text data from the target synthetic data, defining the extracted j image data as seed images, and defining the extracted j text data as seed texts, wherein j is a natural number; Step S302, performing distance calculations on other image data in the target synthetic data and the j seed images, respectively, to obtain the image cluster distances between the other image data and the j seed images, performing distance calculations on other text data in the target synthetic data and the j seed texts, respectively, to obtain the text cluster distances between the other text data and the j seed texts; Step S303, classifying other image data into image text seed categories with the shortest inter-cluster distances according to the inter-cluster distances of images, and classifying other text data into text seed categories with the shortest inter-cluster distances according to the inter-cluster distances of texts; Step S304, counting the number of image clusters and the number of text clusters, obtaining a diversity value of the target synthetic data according to the number of image clusters and the text cluster data, and obtaining a second quality level of the target synthetic data according to the diversity value, wherein the diversity value and the second quality level are in a one-to-one correspondence.

[0008] As a preferred solution of the synthetic data comprehensive evaluation method based on a large model described in the present invention, wherein: The step S4 specifically comprises the following steps: Step S401, retrieve the historical synthesis data of the large model; Step S402, comparing the historical synthetic data with the target synthetic data one by one, obtaining the historical synthetic data whose similarity with the target synthetic data is greater than 0.9, and defining the obtained historical synthetic data as similar historical synthetic data. Step S403, respectively retrieve the data utility of each similar historical synthetic data, and calculate according to a certain weight based on the data utility and similarity of each historical synthetic data to obtain the similar data utility of the target synthetic data; Step S404, obtaining a third quality level of the target synthetic data according to the similar data utility of the target synthetic data, wherein the similar data utility and the third quality level are in a one-to-one corresponding data relationship; The one-to-one comparison of the historical synthetic data and the target synthetic data includes: Extract historical image data and historical text data from each group of historical synthetic data, use image comparison method to compare each historical image data with each target image data, obtain image similarity between historical image data and target image data, compare each text data item of each historical text data with each text data item of each target text data, obtain text similarity between each target text data and each historical text data, and obtain similarity between each historical synthetic data and target synthetic data based on image similarity and text similarity.

[0009] As a preferred solution of the synthetic data comprehensive evaluation method based on a large model described in the present invention, wherein: The expression formula for obtaining the comprehensive quality level of the target synthetic data is as follows: , Among them, P is the comprehensive quality level of the target synthetic data, The first quality level, The second quality level, The third quality level, is the first quality level weight, is the second quality level weight, is the third quality level weight, is the model error rate, and S is the total amount of target synthetic data.

[0010] As a preferred solution of the synthetic data comprehensive evaluation method based on a large model described in the present invention, step S202 specifically includes the following steps: Step S2021, extracting image features using an image processing library, setting corresponding image feature quality values ​​for each image feature, and annotating them, extracting text features using a bag-of-words model, and setting corresponding text feature quality values ​​for each text feature; Step S2022, setting different image feature weights for different image features and corresponding image feature values, and setting corresponding text feature weights for different text features and corresponding text feature values; Step S2023: the image feature weight is used to calculate and obtain the image standard quality grade according to the image feature value, and the text feature weight is used to calculate and obtain the text standard quality grade according to the text feature value.

[0011] As a preferred solution of the synthetic data comprehensive evaluation method based on a large model described in the present invention, step S302 specifically includes the following steps: Step S3021, comparing other image data in the target synthetic data with the j seed images respectively, obtaining image differences, and obtaining the distance between image clusters according to the image differences; Step S3022: compare the other text data in the target synthetic data with the j seed texts respectively to obtain text differences, and obtain the distance between text clusters according to the text differences.

[0012] As a preferred solution of the synthetic data comprehensive evaluation method based on a large model described in the present invention, the calculation expression for obtaining the diversity value of the target synthetic data according to the number of image clusters and the number of text clusters is: , Where N is the diversity value of the target synthetic data, is the number of image clusters, is the number of text clusters, is the ratio of image data to target synthetic data, is the ratio of text data to target synthetic data, Indicates the total amount of target text data, Indicates the total amount of target image data.

[0013] As a preferred solution of the synthetic data comprehensive evaluation method based on a large model described in the present invention, wherein: comparing other target image data in the target synthetic data with j seed images respectively to obtain image difference values ​​includes: The image features of each target image data are extracted by using the image processing library, and the image quality feature values ​​corresponding to each target image feature are obtained. The image quality feature values ​​of other target image data in the target synthesis data are compared with the image quality feature values ​​of the j seed images to obtain the image difference value. The step of comparing the other target text data in the target synthetic data with the j seed texts to obtain text differences comprises: Each text data item of each other target text data in the target synthetic data is compared with each text data item of the j seed texts, and the difference of each text data item is obtained respectively. The difference of each text data item is calculated according to a certain weight to obtain the text difference corresponding to each other target text data and the j seed texts, and the weight is determined according to the proportion of each data item in the target text data.

[0014] An electronic device, comprising: a processor and a memory, wherein: The memory stores a computer program that can be called by the processor; The processor executes any one of the above-mentioned comprehensive evaluation methods for synthetic data based on a large model in the background by calling the computer program stored in the memory.

[0015] Beneficial effects of the present invention: The present invention uses a machine learning model to obtain a first quality level of synthetic data, and uses cluster analysis to obtain a second quality level, and obtains a third quality level based on the data utility of similar historical synthetic data. The quality of synthetic data is evaluated from multiple aspects and multiple angles, thereby ensuring the comprehensiveness and accuracy of the evaluation results.

[0016] The diversity value of the synthetic data is obtained through cluster analysis, and the diversity analysis of the synthetic data is performed based on the diversity value. The synthetic data is evaluated based on the diversity of the synthetic data, which ensures the diversity of the synthetic data and prevents the problem of overfitting in large model training.

[0017] By acquiring the data utility of similar historical synthetic data, the similar data utility of synthetic data is obtained, and the similar data utility is added to the evaluation system of synthetic data to ensure the data utility of synthetic data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of the basic flow of a method for comprehensive evaluation of synthetic data based on a large model provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0020] Example 1, reference Figure 1 , as an embodiment of the present invention, provides a synthetic data comprehensive evaluation method based on a large model, comprising the following steps: Step S1, obtaining target synthesis data of a large model, and respectively obtaining target image data and target text data of the target synthesis data; Step S2, inputting the target image data into the image machine learning model to obtain the image standard quality level, inputting the target text data into the text machine learning model to obtain the text standard quality level, and obtaining the first quality level according to the image standard quality level and the text standard quality level; Step S3, using cluster analysis to classify the target image data and the target text data respectively, obtain the number of image clusters and the number of text clusters, and obtain a second quality level according to the number of image clusters and the number of text clusters; Step S4, calling up the historical synthetic data of the large model, obtaining similar historical synthetic data whose similarity with the target synthetic data is greater than an expected threshold in the historical synthetic data, obtaining similar data utility of the target synthetic data based on the similar historical synthetic data, and obtaining a third quality level based on the similar data utility; Step S5, obtaining a comprehensive quality level of the target synthetic data according to the first quality level, the second quality level and the third quality level.

[0021] The machine learning model is used to establish a model through data training, and use the model to predict or make decisions on unknown data based on known data; The similar data utility represents the data utility of the target synthetic data obtained according to the similar synthetic data; The cluster analysis is based on the characteristics of the research subjects, and they are classified into different categories so that the differences between individuals in the same category are relatively small and the similarities are relatively large, and the differences between individuals in different categories are large and the similarities are small.

[0022] In this embodiment, a first quality level of synthetic data is obtained by using a machine learning model, a second quality level is obtained by using cluster analysis, and a third quality level is obtained based on the data utility of similar historical synthetic data. The quality of synthetic data is evaluated from multiple aspects and multiple angles, thereby ensuring the comprehensiveness and accuracy of the evaluation results. The diversity value of synthetic data is obtained through cluster analysis, and the diversity analysis of synthetic data is performed based on the diversity value. The synthetic data is evaluated based on the diversity of the synthetic data, which ensures the diversity of the synthetic data and prevents the problem of overfitting in large model training. By acquiring the data utility of similar historical synthetic data, the similar data utility of synthetic data is obtained, and the similar data utility is added to the evaluation system of synthetic data to ensure the data utility of synthetic data.

[0023] The step S2 specifically includes the following steps: Step S201, obtaining a synthetic data set, and extracting an image data set and a text data set from the synthetic data set; Step S202, performing data annotation on the image data set and the text data set respectively, wherein the data annotation includes annotating the image data with image feature values ​​according to the image features, and annotating the text data with text feature values ​​according to the data features; Step S203, inputting the annotated image data set into the image machine learning model, using the image data as input and the image feature values ​​as output to train the image machine learning model, and calculating the image feature values ​​according to certain weights to obtain the image standard quality level, inputting the annotated text data set into the text machine learning model, using the text data as input and the text feature values ​​as output to train the text machine learning model, and calculating the text feature values ​​according to certain weights to obtain the text standard quality level; Step S204, inputting the target image data into the image machine learning model to obtain the image standard quality level, and inputting the target text data into the text machine learning model to obtain the text standard quality level; Step S205 , calculating the image standard quality level and the text standard quality level according to a certain weight to obtain a first quality level of the target synthetic data.

[0024] Data annotation refers to the process of adding labels to raw data. Labeled data is called training data and is used to train machine learning models.

[0025] In this embodiment, by establishing an image machine learning model and a text machine learning model, the image standard quality level and the text standard quality level in the target synthetic data are obtained, and the first quality level is obtained according to the image standard quality level and the text standard quality level. The step S3 specifically includes the following steps: Step S301, randomly extracting j image data and j text data from the target synthetic data, defining the extracted j image data as seed images, and defining the extracted j text data as seed texts, wherein j is a natural number; Step S302, performing distance calculations on other image data in the target synthetic data and the j seed images, respectively, to obtain the image cluster distances between the other image data and the j seed images, performing distance calculations on other text data in the target synthetic data and the j seed texts, respectively, to obtain the text cluster distances between the other text data and the j seed texts; Step S303, classifying other image data into image text seed categories with the shortest inter-cluster distances according to the inter-cluster distances of images, and classifying other text data into text seed categories with the shortest inter-cluster distances according to the inter-cluster distances of texts; Step S304, counting the number of image clusters and the number of text clusters, obtaining a diversity value of the target synthetic data according to the number of image clusters and the text cluster data, and obtaining a second quality level of the target synthetic data according to the diversity value, wherein the diversity value and the second quality level are in a one-to-one correspondence.

[0026] The distance between image clusters represents the image difference between each image data; The distance between text clusters represents the text difference between each text data; In this embodiment, cluster analysis is used to obtain the diversity value of the target synthetic data, and the diversity evaluation of the target synthetic data is used as an important evaluation criterion for synthetic data evaluation, which is conducive to improving the generalization ability of the large model and effectively avoiding the problem of overfitting of the large model.

[0027] The step S4 specifically comprises the following steps: Step S401, retrieve the historical synthesis data of the large model; Step S402, comparing the historical synthetic data with the target synthetic data one by one, obtaining the historical synthetic data whose similarity with the target synthetic data is greater than 0.9, and defining the obtained historical synthetic data as similar historical synthetic data. Step S403, respectively retrieve the data utility of each similar historical synthetic data, and calculate according to a certain weight based on the data utility and similarity of each historical synthetic data to obtain the similar data utility of the target synthetic data; Step S404, obtaining a third quality level of the target synthetic data according to the similar data utility of the target synthetic data, wherein the similar data utility and the third quality level are in a one-to-one corresponding data relationship; The one-to-one comparison of the historical synthetic data and the target synthetic data includes: Extract historical image data and historical text data from each group of historical synthetic data, use image comparison method to compare each historical image data with each target image data, obtain image similarity between historical image data and target image data, compare each text data item of each historical text data with each text data item of each target text data, obtain text similarity between each target text data and each historical text data, and obtain similarity between each historical synthetic data and target synthetic data based on image similarity and text similarity.

[0028] The data utility represents the satisfaction or benefit obtained in the application scenario from the generated data obtained after the synthetic data is input into the model; The similar data utility represents the approximate data utility of the target synthetic data calculated according to the similarity of the target synthetic data and the data utility of similar historical synthetic data.

[0029] In this embodiment, by obtaining the data utility of similar historical synthetic data and the similar data utility of target synthetic data, the data utility corresponding to the target synthetic data is used as an important criterion for evaluating the quality of synthetic data, which is beneficial to improving the data utility of synthetic data and ensuring the accuracy and effectiveness of large model predictions or decisions.

[0030] The expression formula for obtaining the comprehensive quality level of the target synthetic data is as follows: , Among them, P is the comprehensive quality level of the target synthetic data, The first quality level, The second quality level, The third quality level, is the first quality level weight, is the second quality level weight, is the third quality level weight, is the model error rate, and S is the total amount of target synthetic data.

[0031] The model error rate indicates the error probability of the large model generating prediction results or decision results due to various force majeure factors.

[0032] In this embodiment, the comprehensive quality level of the target synthetic data is obtained according to the first quality level, the second quality level and the third quality level, and the target synthetic data is comprehensively analyzed from multiple aspects and angles, thereby improving the evaluation accuracy of the synthetic data, ensuring the data quality of the synthetic data, and facilitating ensuring the accuracy of the prediction results or decision results of the large model.

[0033] The step S202 specifically includes the following steps: Step S2021, extracting image features using an image processing library, setting corresponding image feature quality values ​​for each image feature, and annotating them, extracting text features using a bag-of-words model, and setting corresponding text feature quality values ​​for each text feature; Step S2022, setting different image feature weights for different image features and corresponding image feature values, and setting corresponding text feature weights for different text features and corresponding text feature values; Step S2023: the image feature weight is used to calculate and obtain the image standard quality grade according to the image feature value, and the text feature weight is used to calculate and obtain the text standard quality grade according to the text feature value.

[0034] The image processing library provides function interfaces for image data manipulation, image / video input and output, basic image processing, structure analysis, camera calibration, motion analysis, target recognition, and basic GUI. The bag-of-words model is a method of representing text as a word frequency vector. In the bag-of-words model, each word in the text is considered as a feature, and the text is represented as a vector, each element of which corresponds to the number of occurrences of a specific word.

[0035] In this embodiment, the image standard quality level and the text standard quality level are obtained by extracting image features and text features, which provides an accurate data basis for obtaining the first quality level and improves the comprehensiveness and comprehensiveness of synthetic data evaluation.

[0036] The step S302 specifically includes the following steps: Step S3021, comparing other image data in the target synthetic data with the j seed images respectively, obtaining image differences, and obtaining the distance between image clusters according to the image differences; Step S3022: compare the other text data in the target synthetic data with the j seed texts respectively to obtain text differences, and obtain the distance between text clusters according to the text differences.

[0037] In this embodiment, by comparing image data and text data, the distance between image clusters and the distance between text clusters are obtained respectively, which provides an accurate classification basis for subsequent classification of image data according to the distance between image clusters and classification of text data according to the distance between text clusters.

[0038] The calculation expression for obtaining the diversity value of the target synthetic data based on the number of image clusters and the number of text clusters is: , Where N is the diversity value of the target synthetic data, is the number of image clusters, is the number of text clusters, is the ratio of image data to target synthetic data, is the ratio of text data to target synthetic data, Indicates the total amount of target text data, Indicates the total amount of target image data.

[0039] In this embodiment, the diversity value of the target synthetic data is obtained according to the number of image clusters and the number of text clusters, which provides an accurate data basis for evaluating the diversity of the synthetic data.

[0040] The step of comparing other target image data in the target synthesis data with the j seed images to obtain image differences comprises: The image features of each target image data are extracted by using the image processing library, and the image quality feature values ​​corresponding to each target image feature are obtained. The image quality feature values ​​of other target image data in the target synthesis data are compared with the image quality feature values ​​of the j seed images to obtain the image difference value. The step of comparing the other target text data in the target synthetic data with the j seed texts to obtain text differences comprises: Each text data item of each other target text data in the target synthetic data is compared with each text data item of the j seed texts, and the difference of each text data item is obtained respectively. The difference of each text data item is calculated according to a certain weight to obtain the text difference corresponding to each other target text data and the j seed texts, and the weight is determined according to the proportion of each data item in the target text data.

[0041] In this embodiment, a machine learning model is used to obtain the first quality level of the synthetic data, and a cluster analysis is used to obtain the second quality level. The third quality level is obtained based on the data utility of similar historical synthetic data. The quality of the synthetic data is evaluated from multiple aspects and angles to ensure the comprehensiveness and accuracy of the evaluation results.

[0042] The diversity value of the synthetic data is obtained through cluster analysis, and the diversity analysis of the synthetic data is performed based on the diversity value. The synthetic data is evaluated based on the diversity of the synthetic data, which ensures the diversity of the synthetic data and prevents the problem of overfitting in large model training.

[0043] By acquiring the data utility of similar historical synthetic data, the similar data utility of synthetic data is obtained, and the similar data utility is added to the evaluation system of synthetic data to ensure the data utility of synthetic data.

[0044] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. Among them, the storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0045] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A comprehensive evaluation method of synthetic data based on large models, It is characterized in that The following steps are involved: Step S1, obtaining target synthesis data of a large model, and respectively obtaining target image data and target text data of the target synthesis data; Step S2, inputting the target image data into the image machine learning model to obtain the image standard quality level, inputting the target text data into the text machine learning model to obtain the text standard quality level, and obtaining the first quality level according to the image standard quality level and the text standard quality level; Step S3, using cluster analysis to classify the target image data and the target text data respectively, obtain the number of image clusters and the number of text clusters, and obtain a second quality level according to the number of image clusters and the number of text clusters; Step S4, calling up the historical synthetic data of the large model, obtaining similar historical synthetic data whose similarity with the target synthetic data is greater than an expected threshold in the historical synthetic data, obtaining similar data utility of the target synthetic data based on the similar historical synthetic data, and obtaining a third quality level based on the similar data utility; Step S5, obtaining a comprehensive quality level of the target synthetic data according to the first quality level, the second quality level and the third quality level.

2. A synthetic data comprehensive evaluation method based on a large model as claimed in claim 1, Features: The step S2 specifically includes the following steps: Step S201, obtaining a synthetic data set, and extracting an image data set and a text data set from the synthetic data set; Step S202, performing data annotation on the image data set and the text data set respectively, wherein the data annotation includes annotating the image data with image feature values ​​according to the image features, and annotating the text data with text feature values ​​according to the data features; Step S203, inputting the annotated image data set into the image machine learning model, using the image data as input and the image feature values ​​as output to train the image machine learning model, and calculating the image feature values ​​according to certain weights to obtain the image standard quality level, inputting the annotated text data set into the text machine learning model, using the text data as input and the text feature values ​​as output to train the text machine learning model, and calculating the text feature values ​​according to certain weights to obtain the text standard quality level; Step S204, inputting the target image data into the image machine learning model to obtain the image standard quality level, and inputting the target text data into the text machine learning model to obtain the text standard quality level; Step S205 , calculating the image standard quality level and the text standard quality level according to a certain weight to obtain a first quality level of the target synthetic data.

3. A synthetic data comprehensive evaluation method based on a large model as claimed in claim 1, Features: The step S3 specifically includes the following steps: Step S301, randomly extracting j image data and j text data from the target synthetic data, defining the extracted j image data as seed images, and defining the extracted j text data as seed texts, wherein j is a natural number; Step S302, performing distance calculations on other image data in the target synthetic data and the j seed images, respectively, to obtain the image cluster distances between the other image data and the j seed images, performing distance calculations on other text data in the target synthetic data and the j seed texts, respectively, to obtain the text cluster distances between the other text data and the j seed texts; Step S303, classifying other image data into image text seed categories with the shortest inter-cluster distances according to the inter-cluster distances of images, and classifying other text data into text seed categories with the shortest inter-cluster distances according to the inter-cluster distances of texts; Step S304, counting the number of image clusters and the number of text clusters, obtaining a diversity value of the target synthetic data according to the number of image clusters and the text cluster data, and obtaining a second quality level of the target synthetic data according to the diversity value, wherein the diversity value and the second quality level are in a one-to-one correspondence.

4. A synthetic data comprehensive evaluation method based on a large model as claimed in claim 1, Features: The step S4 specifically comprises the following steps: Step S401, retrieve the historical synthesis data of the large model; Step S402, comparing the historical synthetic data with the target synthetic data one by one, obtaining the historical synthetic data whose similarity with the target synthetic data is greater than 0.9, and defining the obtained historical synthetic data as similar historical synthetic data. Step S403, respectively retrieve the data utility of each similar historical synthetic data, and calculate according to a certain weight based on the data utility and similarity of each historical synthetic data to obtain the similar data utility of the target synthetic data; Step S404, obtaining a third quality level of the target synthetic data according to the similar data utility of the target synthetic data, wherein the similar data utility and the third quality level are in a one-to-one corresponding data relationship; The one-to-one comparison of the historical synthetic data and the target synthetic data includes: Extract historical image data and historical text data from each group of historical synthetic data, use image comparison method to compare each historical image data with each target image data, obtain image similarity between historical image data and target image data, compare each text data item of each historical text data with each text data item of each target text data, obtain text similarity between each target text data and each historical text data, and obtain similarity between each historical synthetic data and target synthetic data based on image similarity and text similarity.

5. A synthetic data comprehensive evaluation method based on a large model as claimed in claim 1, Features: The expression formula for obtaining the comprehensive quality level of the target synthetic data is as follows: ， Among them, P is the comprehensive quality level of the target synthetic data, The first quality level, The second quality level, The third quality level, is the first quality level weight, is the second quality level weight, is the third quality level weight, is the model error rate, and S is the total amount of target synthetic data.

6. A synthetic data comprehensive evaluation method based on a large model as claimed in claim 2, Features: The step S202 specifically includes the following steps: Step S2021, extracting image features using an image processing library, setting corresponding image feature quality values ​​for each image feature, and annotating them, extracting text features using a bag-of-words model, and setting corresponding text feature quality values ​​for each text feature; Step S2022, setting different image feature weights for different image features and corresponding image feature values, and setting corresponding text feature weights for different text features and corresponding text feature values; Step S2023: the image feature weight is used to calculate and obtain the image standard quality grade according to the image feature value, and the text feature weight is used to calculate and obtain the text standard quality grade according to the text feature value.

7. A synthetic data comprehensive evaluation method based on a large model as claimed in claim 3, Features: The step S302 specifically includes the following steps: Step S3021, comparing other image data in the target synthetic data with the j seed images respectively, obtaining image differences, and obtaining the distance between image clusters according to the image differences; Step S3022: compare the other text data in the target synthetic data with the j seed texts respectively to obtain text differences, and obtain the distance between text clusters according to the text differences.

8. A synthetic data comprehensive evaluation method based on a large model as claimed in claim 3, Features: The calculation expression for obtaining the diversity value of the target synthetic data based on the number of image clusters and the number of text clusters is: ， Where N is the diversity value of the target synthetic data, is the number of image clusters, is the number of text clusters, is the ratio of image data to target synthetic data, is the ratio of text data to target synthetic data, Indicates the total amount of target text data, Indicates the total amount of target image data.

9. A synthetic data comprehensive evaluation method based on a large model as claimed in claim 3, Features: The step of comparing other target image data in the target synthesis data with the j seed images to obtain image differences comprises: The image features of each target image data are extracted by using the image processing library, and the image quality feature values ​​corresponding to each target image feature are obtained. The image quality feature values ​​of other target image data in the target synthesis data are compared with the image quality feature values ​​of the j seed images to obtain the image difference value. The step of comparing the other target text data in the target synthetic data with the j seed texts to obtain text differences comprises: Each text data item of each other target text data in the target synthetic data is compared with each text data item of the j seed texts, and the difference of each text data item is obtained respectively. The difference of each text data item is calculated according to a certain weight to obtain the text difference corresponding to each other target text data and the j seed texts, and the weight is determined according to the proportion of each data item in the target text data.

10. An electronic device, It is characterized in that include: A processor and a memory, wherein: The memory stores a computer program that can be called by the processor; The processor executes the comprehensive evaluation method for synthetic data based on a large model as described in any one of claims 1 to 9 in the background by calling the computer program stored in the memory.

Images