Data set evaluation method, device, equipment, storage medium and program product

Abstract

Embodiments of the present application provide an evaluation method and device for a data set, equipment, a storage medium and a program product. The method comprises: generating a data set to be evaluated, the data set comprising image descriptions corresponding to target images; constructing a question and answer pair corresponding to a target image; wherein the question and answer pair comprises a question and a standard answer; determining an evaluation result of the image descriptions in the data set based on the standard answer corresponding to the question and a predicted answer corresponding to the question; wherein the predicted answer is obtained according to the question and the image descriptions. In this embodiment, by constructing a question and a standard answer corresponding to the question for each target image, obtaining a predicted answer based on the question and the image descriptions corresponding to the target image, and obtaining an evaluation result according to the standard answer and the predicted answer, the accuracy and objectivity of image description quality evaluation can be improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, storage medium, and program product for evaluating datasets. Background Technology

[0002] In existing technologies, the quality assessment of image captions in a dataset typically relies on a general large model scoring mechanism. However, the evaluation results of this method are often easily influenced by the biases of the general large model. This is because the general large model may be affected by its training data and training objectives when evaluating image captions, leading to evaluation results biased towards specific styles or content. Such biases cannot objectively and accurately assess image captions. Summary of the Invention

[0003] This invention provides a method, apparatus, device, storage medium, and program product for evaluating datasets, which can improve the accuracy of image description evaluation in datasets.

[0004] In a first aspect, embodiments of this disclosure provide a method for evaluating a dataset, comprising: generating a dataset to be evaluated, the dataset including image descriptions corresponding to target images; constructing question-answer pairs corresponding to the target images; wherein the question-answer pairs include questions and standard answers; determining an evaluation result corresponding to the image descriptions in the dataset based on the standard answers corresponding to the questions and the predicted answers corresponding to the questions; wherein the predicted answers are generated based on the questions and the image descriptions.

[0005] Secondly, embodiments of this disclosure also provide an evaluation apparatus for a dataset, comprising: a dataset generation module for generating a dataset to be evaluated, the dataset including image descriptions corresponding to target images; a question-answer pair construction module for constructing question-answer pairs corresponding to the target images; wherein the question-answer pairs include questions and standard answers; and an evaluation result determination module for determining evaluation results corresponding to image descriptions in the dataset based on the standard answers corresponding to the questions and the predicted answers corresponding to the questions; wherein the predicted answers are generated based on the questions and the image descriptions.

[0006] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the dataset evaluation method as described in embodiments of this disclosure.

[0007] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform an evaluation method for a dataset as described in embodiments of this disclosure.

[0008] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the dataset evaluation method as described in embodiments of this disclosure.

[0009] The technical solution of this embodiment generates a dataset to be evaluated, the dataset including image descriptions corresponding to target images; constructs question-answer pairs corresponding to the target images; wherein the question-answer pairs include questions and standard answers; and determines the evaluation result corresponding to the image descriptions in the dataset based on the standard answers and predicted answers corresponding to the questions; wherein the predicted answers are generated based on the questions and the image descriptions. This embodiment, by constructing questions and standard answers for each target image, obtaining predicted answers based on the questions and image descriptions corresponding to the target images, and obtaining evaluation results based on the standard answers and predicted answers, can improve the accuracy and objectivity of image description quality evaluation. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0011] Figure 1 This is a schematic diagram of a dataset evaluation method provided in an embodiment of the present invention;

[0012] Figure 2 This is a schematic diagram of another dataset evaluation method provided in an embodiment of the present invention;

[0013] Figure 3 This is a schematic diagram of another dataset evaluation method provided in an embodiment of the present invention;

[0014] Figure 4 This is a schematic diagram of the target image provided in an embodiment of the present invention;

[0015] Figure 5 This is a schematic diagram of the structure of a dataset evaluation device provided in an embodiment of the present invention;

[0016] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should be understood that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect. The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". It should be noted that the concepts of "first," "second," etc., mentioned in this disclosure are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies. It should be noted that the modifications of "a" and "a plurality" mentioned in this disclosure are illustrative rather than restrictive.

[0019] Those skilled in the art should understand that, unless otherwise expressly indicated in the context, it should be understood as "one or more". It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations, and provisions.

[0020] Figure 1 This is a schematic flowchart of a dataset evaluation method provided by an embodiment of the present invention. This embodiment is applicable to situations where the quality of image descriptions in a dataset is evaluated to assess whether the image descriptions accurately represent the information in the target image. This method can be executed by a dataset evaluation device, which can be implemented in software and / or hardware, optionally through an electronic device, such as a mobile terminal, PC, or server. Figure 1 As shown, the method includes:

[0021] S110. Generate the dataset to be evaluated, which includes the image descriptions corresponding to the target images.

[0022] In this embodiment, any dataset generated by a language model can be used as the dataset to be evaluated. For example, the language model can be an open-source language model, such as the Qwen2.5-72B model, a closed-source language model, such as the Generative Pre-trained Transformer (GPT) model, or a target language model. The target language model can be understood as a self-developed language model, which can be composed of an image encoder, a symbol tag generation module, and an image description generation module.

[0023] The dataset may include multiple sets of image-text pairs, each set of which may include a target image and a corresponding image description. The image description corresponding to the target image may include a first image description and / or a second image description.

[0024] Image descriptions can be understood as structured or unstructured textual expressions of the content presented in a target image using natural language (such as Chinese, English, etc.), used to reflect the visual information in the image. For example, an image description could be "a cat".

[0025] S120. Construct question-answer pairs corresponding to the target image.

[0026] The question-and-answer pair includes a question and a standard answer.

[0027] In this embodiment, a corresponding question-and-answer pair can be constructed for each target image in each group of image texts in the dataset.

[0028] In this embodiment, the target image itself is not limited; for example, the target image may include people, animals, landscapes, text, etc. In this embodiment, the content of each question-and-answer pair is not limited; for example, it may be related to the image description dimensions and the target image itself. In this embodiment, the number of image description dimensions and their content are not limited; for example, the number of image description dimensions can be 17. For example, image description dimensions may include general information, image type, text content, color features, element position, relationship between elements, relative position, entity type, entity size, entity shape, number of elements, emotion expression, image blur, image artifacts, proper nouns and their world knowledge, color palette, and color grading. Here, world knowledge can be understood as background knowledge.

[0029] For example, suppose there is a target image with a black and white background, containing a boy, and with text description (assuming the text is to the right of the boy, and the boy has a serious expression). If the image description dimension is text content, the question could be "Does the target image contain text?", and the standard answer could be "Yes". If the image description dimension is color features, the question could be "Is the man in the target image black and white?", and the standard answer could be "Yes". If the image description dimension is emotional expression, the question could be "Does the person in the target image look serious?", and the standard answer could be "Yes".

[0030] In this embodiment, there are no restrictions on the method of constructing question-answer pairs. For example, multiple question-answer pairs can be constructed for each target image according to multiple different image description dimensions. Alternatively, question-answer pairs can be generated based on the target image using any language model (such as GPT).

[0031] S130. Determine the evaluation results corresponding to the image descriptions in the dataset based on the standard answer and the predicted answer to the question.

[0032] The predicted answer is generated based on the question and the image description.

[0033] In this embodiment, any language model (such as GPT) can be used to generate corresponding predicted answers based on the question and image description.

[0034] The standard answer can be understood as the answer determined by manual annotation or review based on the target image and the question. The predicted answer can be understood as the answer obtained directly through a language model based on the question and image description.

[0035] The evaluation result corresponding to the image description in the dataset can be understood as the comprehensive evaluation result of all image descriptions in the dataset.

[0036] The evaluation results can be understood as a quantitative indicator of image description quality, used to measure whether the image descriptions in the dataset accurately and comprehensively reflect the visual information in the target image.

[0037] In this embodiment, for each target image, the image description accuracy of each target image can be determined based on the standard answer and the predicted answer corresponding to the question. Then, the average of the image description accuracy of all target images is calculated as the evaluation result of the image description in the dataset.

[0038] In this embodiment, multiple question-answer pairs and their corresponding predicted answers can be categorized into the corresponding image description dimensions. The image description accuracy under the same image description dimension is statistically analyzed, and the image description accuracy under each image description dimension is used as the evaluation result corresponding to the image description in the dataset.

[0039] The technical solution of this embodiment generates a dataset to be evaluated, which includes image descriptions corresponding to target images; constructs question-answer pairs corresponding to the target images; wherein the question-answer pairs include questions and standard answers; and determines the evaluation result corresponding to the image descriptions in the dataset based on the standard answers and predicted answers corresponding to the questions; wherein the predicted answers are generated based on the questions and image descriptions. This embodiment, by constructing questions and standard answers for each target image, obtaining predicted answers based on the questions and image descriptions corresponding to the target images, and obtaining evaluation results based on the standard answers and predicted answers, can improve the accuracy and objectivity of image description quality evaluation.

[0040] Figure 2 This is a schematic diagram of another dataset evaluation method provided in an embodiment of the present invention. See also... Figure 2 The method provided in this embodiment of the invention specifically includes the following steps:

[0041] S201. Obtain the target image.

[0042] In this embodiment, the initial image can be obtained in any way, such as through web scraping or from an image and text dataset on an open-source website. After obtaining the initial image, it can be preprocessed to obtain a preprocessed image. The preprocessed image can then be filtered to obtain the target image.

[0043] This embodiment does not limit the preprocessing method; for example, it can include image deduplication and image cleaning. For instance, perceptual hashing (PHash) technology can be used for image deduplication. For example, a PHash value can be generated for each image, and duplicate images can be removed by comparing PHash values, ensuring image diversity. For instance, image filtering can also be performed based on aesthetic ratings, image resolution, and Not Safe For Work (NSFW) image content. For example, images with an aesthetic rating greater than 5, a resolution greater than 224x224 pixels, and an NSFW probability less than 0.3 can be retained to obtain high-quality and compliant images.

[0044] Optionally, obtaining the target image includes: obtaining multiple initial images; and filtering the multiple initial images based on the image information in each initial image to obtain multiple target images.

[0045] The image information includes text information and / or detection target information.

[0046] Here, text information can be understood as the text information in the image. Target information can be understood as information related to entities such as objects, people, and animals in the image. The "target" in target information can be understood as an entity object. Target information can include the entity object category, the entity object's location information in the image (which can be represented by the coordinates of the detection box), and the entity object's pixel information in the image (for example, the entity object can be represented by a binary mask (such as 0)).

[0047] In this embodiment, multiple initial images (or pre-processed images) can be filtered according to the number of characters in the text information and / or the number of categories of the detected target information to obtain multiple target images. The number of categories of the detected target information can be understood as the number of entity object categories.

[0048] In this embodiment, by filtering multiple initial images based on image information, images with more image information and richer content (i.e., representative images) can be selected, thereby improving the quality of target images in the dataset and thus improving the reliability and accuracy of dataset evaluation.

[0049] Optionally, multiple initial images are filtered based on image information in each initial image to obtain multiple target images, including: performing cluster analysis on multiple initial images to obtain multiple cluster analysis images corresponding to each cluster category; extracting text information from multiple cluster analysis images using a text extraction model to obtain text information for each cluster analysis image; extracting target detection information from multiple cluster analysis images using a target detection model to obtain target detection information for each cluster analysis image; and filtering multiple cluster analysis images based on text information and / or target detection information to obtain multiple target images corresponding to each cluster category.

[0050] In this embodiment, features from multiple initial images can be extracted using any language model (such as Embedded Vision-Aware Contrastive Language-Image Pre-training (EVA-CLIP)), resulting in multiple image features (one initial image corresponds to one image feature). Clustering analysis algorithms (such as K-means clustering) are then used to perform cluster analysis on these multiple image features, resulting in multiple cluster categories (e.g., 80 cluster categories). The distances from multiple initial images within each cluster category to the cluster centers are determined. From these initial images within each cluster category, the multiple initial images closest to the cluster centers are selected (e.g., 200 initial images per cluster category) as multiple cluster analysis images (e.g., 16,000 cluster analysis images for 80 cluster categories).

[0051] In this embodiment, the text extraction model can be any language model, such as GPT. It can also be a self-developed language model, such as one composed of an image encoder, a first-dimensional transformation module, and a first language model. Similarly, the object detection model can be any language model, such as GPT, or a self-developed language model, such as one composed of a second language model, a second-dimensional transformation module, and an object detection sub-model.

[0052] In this embodiment, the text information of each cluster analysis image can be obtained through a text extraction model. The target detection information of each cluster analysis image can be obtained through a target detection model. Then, the multiple cluster analysis images are filtered according to the number of characters in the text information and / or the number of categories of the target detection information to obtain multiple target images corresponding to each cluster category (for example, 20 images are filtered for each cluster category, 80 cluster categories, and a total of 1600 target images).

[0053] In this embodiment, by performing cluster analysis on the initial images, extracting the text information of each cluster analysis image using a text extraction model, extracting the target information of each cluster analysis image using a target detection model, and then performing image filtering on multiple cluster analysis images based on the text information and / or target information, the accuracy of image filtering can be improved, thereby further enhancing the reliability and accuracy of dataset evaluation.

[0054] Optionally, the text extraction model includes an image encoder, a first dimension transformation module, and a first language model. The text extraction model extracts text information from multiple clustering analysis images to obtain the text information for each clustering analysis image, including: for each clustering analysis image, generating image features based on the clustering analysis image using the image encoder; transforming the dimensions of the image features using the first dimension transformation module to obtain target image features; and generating the text information of the clustering analysis image based on the target image features using the first language model.

[0055] The image encoder can be any type of image encoder, such as a visual encoder (VisionTransformer, ViT). In this embodiment, the clustering analysis image can be converted into corresponding image features using the image encoder.

[0056] The first dimension transformation module can be any type of neural network layer, such as a linear layer. In this embodiment, the first dimension transformation module can be used to transform the dimension of image features into a dimension suitable for the first language model to obtain the target image features.

[0057] The first language model can be any language model, such as the Qwen-05B model. In this embodiment, text information can be extracted from the features of the target image using the first language model.

[0058] In this embodiment, based on the image encoder and the first dimension transformation module, and combined with the first language model, the accuracy of text information extraction can be improved. For example, when the text information in the cluster analysis image is incomplete, it can be automatically supplemented during text information extraction, resulting in higher accuracy and a more intelligent text information extraction capability.

[0059] Optionally, the object detection model includes a second language model, a second dimension transformation module, and an object detection sub-model. The object detection model extracts target information from multiple clustered images to obtain target information for each clustered image, including: for each clustered image, generating target features based on the clustered image and a third guiding word using the second language model; transforming the dimensions of the target features using the second dimension transformation module to obtain target detection features; and generating target information based on the target detection features using the object detection sub-model.

[0060] The second language model can be any open-source Large Language Model (LLM), such as the LLaMA model.

[0061] The third guiding word differs from both the first and second guiding words. In this embodiment, the content of the third guiding word is not limited; for example, it could be "Please describe the location of the entity (such as a cat) in the image."

[0062] In this embodiment, the detection target features of the cluster analysis image can be obtained through a second language model. These detection target features can be understood as the location features of the entity objects.

[0063] The second dimension transformation module can be any type of neural network layer, such as a linear layer. It can be the same as or different from the first dimension transformation module. In this embodiment, the second dimension transformation module can convert the dimensions of the detected target features into dimensions suitable for the target detection sub-model, thus obtaining the target features for target detection.

[0064] The object detection sub-model can be any type of object detection neural network model, such as a convolutional neural network model. In this embodiment, the object detection sub-model can extract the detected object information from the object features.

[0065] In this embodiment, by adding a second language model before the second dimension transformation module and the object detection sub-model, richer and more accurate object detection information can be extracted. For example, the input to the second language model can have various designs, and the third guiding word can be adjusted and varied to enrich the object detection features output by the second language model. This, in turn, enriches the input to the object detection sub-model, resulting in a greater number of object detection categories output by the sub-model and more accurate object detection information itself.

[0066] Optionally, image filtering is performed on multiple cluster analysis images based on text information and / or detection target information to obtain multiple target images corresponding to each cluster category, including: counting the number of characters in the text information of each cluster analysis image; counting the number of categories in the detection target information of each cluster analysis image; sorting the multiple cluster analysis images according to the number of characters and / or the number of categories to obtain the sorting result; and filtering the multiple cluster analysis images according to the sorting result to obtain multiple target images corresponding to each cluster category.

[0067] In this embodiment, the number of characters (i.e., the number of words) in the text information of each cluster analysis image can be counted, and the number of categories of the detection target information in each cluster analysis image can be counted (one entity object corresponds to one detection target information).

[0068] For example, for multiple cluster analysis images under each cluster category, the images can be sorted in descending order according to the number of characters and / or the number of categories to obtain the sorting results. The top-ranked cluster analysis images (such as the top 20 cluster analysis images) can be used as multiple target images under that cluster category.

[0069] For example, for multiple cluster analysis images under each cluster category, the images can be sorted in ascending order according to the number of characters and / or the number of categories to obtain the sorting results. The lower-ranked cluster analysis images (e.g., the last 20 images) can be used as multiple target images under that cluster category.

[0070] In this embodiment, the method of filtering multiple cluster analysis images according to the number of characters and / or the number of categories results in a larger number of characters and / or categories in the selected target images, i.e., a higher information density, thereby improving the quality of the target images.

[0071] S202. Using the target language model, generate a first image description corresponding to the target image based on the first guiding word and the target image.

[0072] In this embodiment, the target language model is not limited and can be any kind of language model, such as a closed-source language model like GPT, or an open-source language model like Qwen2.5-72B, or a self-developed language model (such as one composed of an image encoder, a symbol tag generation module, and an image description generation module).

[0073] In this embodiment, the first guiding word is used to guide the target language model to generate an image description corresponding to the target image. The first image description can be understood as a detailed image description. In this embodiment, the content of the detailed image description is not limited; for example, it can cover multiple image elements, such as image style type, style information, detailed subject description, and shooting information. In this embodiment, the content of the first guiding word is not limited; for example, it can include rules guiding the target language model to generate a detailed image description and examples of generating a detailed image description according to those rules. In this embodiment, the rules for generating a detailed image description are not limited; for example, the rule could be, "Please generate a detailed description of the image, which should be extremely detailed and specific. Please ensure that the description reflects the following image elements: image style type, style information, detailed subject description, and shooting information, etc."

[0074] Optionally, the training set for the target language model can be obtained from any closed-source language model, such as GPT-4o, Clode3.5, etc.

[0075] In this embodiment, by using a closed-source language model to generate the training set of the target language model, the accuracy of the training set can be improved, thereby improving the quality of the training set.

[0076] In this embodiment, the input of the image encoder in the target language model is the target image, and the output is the image features corresponding to the target image. The image features corresponding to the target image can be used as the input of the symbol tag generation module. The input of the image description generation module can include the output of the image encoder, the output of the symbol tag generation module (symbol tag features), and the first guide word. The output of the image description generation module is the first image description.

[0077] In this embodiment, any neural network model can be used as the symbol generation module, such as the QLLAMA model, which consists of two transformer layers of the LLaMA model. The symbolic features are used to instruct the target language model to distinguish and understand the image features and the first guiding word.

[0078] Optionally, the symbol marking features include an image feature end identifier feature and a separator identifier feature; wherein, the image feature end identifier is used to identify the end of the image feature; and the separator identifier is used to separate the image feature and the first introductory word.

[0079] In this embodiment, the image description generation module can adopt any open-source Large Language Model (LLM), such as the LLaMA model.

[0080] In this embodiment, image features and a first guiding term can be concatenated using symbolic marker features. The concatenated features are then input into the image description generation module, which distinguishes and understands the image features and the first guiding term based on the symbolic marker features. Specifically, the first guiding term can be converted into text features, and the image features and text features can be aligned to the same semantic space based on the symbolic marker features. By understanding the image features and text features within the same semantic space, a first image description that meets the requirements can be generated.

[0081] Optionally, the first guiding term includes a first image description generation rule and first image description generation example information corresponding to the first image description generation rule. The first image description generation rule includes a content type description rule for the first image description to be generated.

[0082] The content type description rules for the first image description can include general image description rules and detailed image description rules. The general image description rules can be understood as the requirements for the general image description portion of the first image description (which can be summarized in one sentence). The detailed image description rules can be understood as the content type requirements for the detailed content portion of the first image description.

[0083] In this embodiment, the content type in the detailed image description is not limited. For example, it may include image style type, style information, detailed subject description, and shooting information. In this embodiment, the content type may also include format requirements. The overall image description may include the core key theme information of the image, as well as background knowledge.

[0084] In this embodiment, the first guiding word and the target image can be directly input into the target language model. The target language model understands the first guiding word and the target image and outputs the first image description corresponding to the target image.

[0085] In this embodiment, the first guiding word, the target image, and the image information of the target image can also be input into the target language model. The target language model understands the first guiding word, the target image, and the image information, and outputs the first image description corresponding to the target image.

[0086] Optionally, generating a first image description corresponding to the target image based on the first guiding word and the target image includes: generating a first image description corresponding to the target image based on the first guiding word and the image information of the target image through a target language model.

[0087] In this embodiment, for the input of the target language model, image information is added on the basis of the first guiding word and the target image. This can ensure that the image information in the target image is described, improve the accuracy and richness of the first image description generated by the target language model, reduce the generation of illusions, and thus further improve the reliability of the dataset evaluation.

[0088] S203. Generate a second image description based on the second guide word and the first image description, and use the first image description and / or the second image description as the image description in the dataset.

[0089] The first introductory word is different from the second introductory word.

[0090] The second image description is a multi-level image description, and the first image description is a detailed image description.

[0091] In this embodiment, a second image description can be generated based on the second guide word and the first image description using any language model (such as the Qwen2.5-72B language model).

[0092] Optionally, the second guiding word includes the second image description generation rule and the second image description generation example information corresponding to the second image description generation rule.

[0093] The second image description generation rule is a multi-level image description generation rule. Each level of the multi-level image description generation rule includes a length description rule and a content type description rule; the length description rule and / or the content type description rule are different for each level.

[0094] In this embodiment, the length description rule for each layer of image description can be understood as the character count requirement for each layer of image description. The content type description rule for each layer of image description can be understood as the content type requirement for each layer of image description.

[0095] In this embodiment, the content type in the hierarchical image description is not limited, and may include, for example, topic information, subject information, background knowledge, etc.

[0096] Optionally, the length description rule includes the number of characters in the description of the second image to be generated; the number of characters at each level changes monotonically; and the content type description rule includes the content type in the description of the second image to be generated; the content types at each level are in a progressive relationship.

[0097] In this embodiment, the number of characters described in the image at each level is different and changes monotonically, such as increasing monotonically. For example, the length description rule for the first level is no more than 10 characters, while the length description rule for the second level can be no more than 20 characters.

[0098] In this embodiment, the content type described by each layer of the image is different, and the content types of each layer are in a progressive relationship. For example, the content type of the first layer is topic information, and the content type of the second layer is the same as the content type of the first layer, but with the addition of main body information.

[0099] Among them, the example information generated by the second image description is multi-level image description generation example information, that is, example information generated according to the image description generation rules of each level, and each level has corresponding example information.

[0100] For example, the first-level image description example information is "a man on the steps.", and the second-level image description example information can be "three men wearing red headscarves sitting on the steps."

[0101] In this embodiment, the character count requirement for each level is explicitly defined through length description rules, and the content type requirement for each level is explicitly defined through content type description rules. This ensures that the generated second image description has hierarchy and order, meeting diverse user needs and improving user experience. By controlling the submission relationship and character count, the generated second image description is made more logical and semantically consistent, thus improving the quality of the image description.

[0102] S204. For each target image, construct multiple question-answer pairs according to multiple image description dimensions.

[0103] There are multiple target images. Each image description dimension corresponds to at least one question-answer pair.

[0104] The question includes an image description dimension, meaning the question is associated with or matches the image description dimension. For example, the image description dimension is text content, and the question could be "Does the target image contain text?"

[0105] In this embodiment, by constructing multiple question-and-answer pairs for the target image according to multiple image description dimensions, it is possible to ensure that the question-and-answer pairs can cover multiple image description dimensions, thereby comprehensively evaluating the accuracy of image descriptions across different image description dimensions. This also enhances the robustness of the evaluation results, avoids misjudgments due to a single image description dimension, and further improves the reliability of the evaluation results.

[0106] S205. Using a third language model, generate predicted answers for each question based on multiple questions corresponding to each target image and the image description.

[0107] The dataset includes an image description for each of the multiple target images.

[0108] In this embodiment, the third language model can be any language model, such as GPT.

[0109] For example, taking a target image as an example, the input of the third language model is multiple questions corresponding to the target image and an image description corresponding to the target image. Based on the multiple questions and the image description, the third language model generates a predicted answer for each question. The predicted answer can be represented by "yes" or "no".

[0110] In this embodiment, a third language model is used to answer multiple questions. Since multiple images are constructed according to multiple different image description dimensions, the quality of image descriptions can be evaluated from different image description dimensions, providing an evaluation benchmark for constructing high-quality image descriptions.

[0111] S206. Determine the evaluation result of each image description dimension in multiple image descriptions based on the predicted answer and the standard answer for each question.

[0112] In this embodiment, multiple question-answer pairs and multiple predicted answers corresponding to each target image can be categorized into the corresponding image description dimension. The image description accuracy under the same image description dimension is statistically analyzed, and the image description accuracy under each image description dimension is used as the evaluation result corresponding to the image description in the dataset.

[0113] In this embodiment, a third language model is used to generate a predicted answer for each question based on multiple questions and image descriptions corresponding to each target image. Then, the evaluation result of each image description dimension in multiple image descriptions is determined based on the standard answer corresponding to each question. This method can improve the accuracy of the dataset evaluation results.

[0114] Optionally, generating a predicted answer for each question based on multiple questions and image descriptions corresponding to each target image includes: if the image description is a multi-level image description, semantically matching the content type description rules corresponding to each level of the multi-level image description with multiple image description dimensions to determine the target image description dimension corresponding to each level of the image description; from multiple questions, taking the questions associated with the target image description dimension corresponding to each level of the image description as the target question for each level of the image description; and generating a predicted answer for each target question of each level of the image description based on the target question corresponding to each level of the image description and each level of the image description.

[0115] The corresponding content type description rules are used to generate image descriptions for the corresponding levels.

[0116] It should be noted that the image description is either a second image description, or a first image description and a second image description. That is, in the case of multi-level image descriptions, the image description dimensions covered by each level of image description are related to the content type description rules of each level. In particular, each level of image description in a second image description may not cover all image description dimensions; it may only cover a few. The number of image description dimensions covered by each level of image description may also differ.

[0117] In this embodiment, if the image description is a multi-level image description, the content type description rule corresponding to each level of image description is semantically matched with multiple image description dimensions to determine at least one image description dimension corresponding to each level of image description, which is then used as the target image description dimension. From all the questions corresponding to the target image, the questions associated with the target image description dimension corresponding to each level of image description are used as the target questions (at least one target question) for each level of image description. Based on the target questions corresponding to each level of image description and each level of image description, a predicted answer is generated for each target question of each level of image description. Then, based on the predicted answer and the standard answer corresponding to each target question, the evaluation result of each image description dimension in multiple image descriptions (i.e., all image descriptions in the dataset) is determined.

[0118] In this embodiment, the semantic matching method is not limited. For example, traditional matching methods such as cosine similarity can be used, deep learning methods such as pre-trained language models can be used, or large language models can be used for matching, such as by designing guide words to enable large language models to match content type description rules and image description dimensions.

[0119] In this embodiment, when the image description is multi-level, semantic matching is performed between the content type description rules corresponding to each level of image description and multiple image description dimensions to obtain the target image description dimension corresponding to each level of image description. The target question associated with the target image description dimension is then selected. Based on the target question corresponding to each level of image description and the target question corresponding to each level of image description, the predicted answer corresponding to each target question of each level of image description is generated. This method can ensure the accuracy of the evaluation of image descriptions at each level, improve the granularity and relevance of the dataset evaluation, and further improve the accuracy and reliability of the dataset evaluation.

[0120] Figure 3 This is a schematic flowchart illustrating another dataset evaluation method provided in an embodiment of the present invention. See also... Figure 3 The method provided in this embodiment of the invention specifically includes the following steps:

[0121] S301. Obtain multiple initial images.

[0122] S302. Based on the image information in each initial image, perform image filtering on multiple initial images to obtain multiple target images.

[0123] The image information includes text information and / or detection target information.

[0124] S303. Using the target language model, generate a first image description corresponding to the target image based on the first guiding word and the target image.

[0125] S304. Generate a second image description based on the second guide word and the first image description, and use the first image description and / or the second image description as the image description in the dataset.

[0126] The first and second guiding words are different; the second image description is a multi-level image description, and the first image description is a detailed image description.

[0127] S305. For each target image, construct multiple question-answer pairs according to multiple image description dimensions.

[0128] Each image description dimension corresponds to at least one question-answer pair; the question includes the image description dimension.

[0129] S306. Using a third language model, generate a predicted answer for each question based on multiple questions corresponding to each target image and the image description.

[0130] S307. Determine the evaluation result of each image description dimension in multiple image descriptions based on the predicted answer and the standard answer for each question.

[0131] In this embodiment, multiple initial images are filtered to obtain multiple target images. A target language model, based on guide words and these target images, yields a dataset to be evaluated. For each target image, a question and its corresponding standard answer are constructed according to multiple image description dimensions. A third language model then predicts the answer based on the question and the corresponding image description. The evaluation results are derived from the standard and predicted answers, further improving the reliability and accuracy of the dataset evaluation, thereby enhancing the accuracy and objectivity of image description quality assessment. This automated evaluation process ensures efficiency and standardization, guaranteeing consistent evaluation standards for each image description. The third language model, when answering questions, comprehensively considers multiple image description dimensions, generating quantitative evaluation results. This approach not only reduces human subjectivity but also enhances the coverage and depth of the evaluation, providing a reliable reference for improving image description quality.

[0132] Figure 4 This is a schematic diagram of the target image provided in an embodiment of the present invention. Figure 4 The provided target image, and the detailed image description obtained using the solution provided in this embodiment of the invention, is as follows: "A tranquil landscape photograph captures the intricate interior of a cave, where natural light creates a high contrast between shadowed and illuminated areas. In the foreground are large, dark brown rocks with a rough texture, forming jagged edges and sharp angles that outline against the bright background. A calm pool occupies the center, its smooth surface reflecting hues of forest green (#1b4d3f), turquoise (#87c2a8), and sky blue (#7fb0e9). Soft ripples distort this mirror-like appearance, with sections of logs or trunks near the vertical submerged in the water, creating circular patterns of light reflection. To the left, a slender waterfall cascades down from an opening above, forming a small stream that creates a misty effect as it hits the still water below. This flowing water is due to the long-exposure photographic technique..." The striking sky-blue hue (#3c5da5) creates a soft glow around its base. The cave walls are composed of rugged rock formations in various shades of gray-brown (#6b6044), beige (#dac69a), and olive green (#6c6933). These textured surfaces display a rich variety of colors and shapes, including crevices, protrusions, and small indentations, adding depth and character to the scene. On the right, sunlight streams through an opening in the lush tropical vegetation outside the cave, casting bright highlights on the vibrant green leaves and illuminating the rocky floor scattered with stones of varying sizes. The interplay of light and shadow enhances the three-dimensional quality of the scene, emphasizing the raw beauty of this untouched natural environment. The color palette, including touches of forest green, earthy brown, cool blue, warm yellow, and slate gray, contributes to the image's serene atmosphere while highlighting the pristine beauty of this untouched wilderness. The question and answer for the target image could be: the question is "Is the water turquoise?", and the corresponding answer could be "Yes". The questions are: "Is the cave entrance on the right side of the image?", and the standard answer can be "yes". "Is the cave large?", and the standard answer can be "yes". "Does the image show more than one rock?", and the standard answer can be "yes". "Is there a waterfall in the image?", and the standard answer can be "yes". "Does the image depict an outdoor scene?", and the standard answer can be "yes".

[0133] Figure 5 This is a schematic diagram of the structure of a dataset evaluation device provided in an embodiment of this disclosure, as shown below. Figure 5As shown, the device includes: a dataset generation module 510, a question-answer pair construction module 520, and an evaluation result determination module 530;

[0134] The dataset generation module 510 is used to generate a dataset to be evaluated, the dataset including image descriptions corresponding to the target image;

[0135] The question-answer pair construction module 520 is used to construct question-answer pairs corresponding to the target image; wherein, the question-answer pair includes a question and a standard answer;

[0136] The evaluation result determination module 530 is used to determine the evaluation result corresponding to the image description in the dataset based on the standard answer corresponding to the question and the predicted answer corresponding to the question; wherein, the predicted answer is generated based on the question and the image description.

[0137] The technical solution of this embodiment generates a dataset to be evaluated through a dataset generation module, the dataset including image descriptions corresponding to target images; constructs question-answer pairs corresponding to the target images through a question-answer pair construction module; wherein, the question-answer pair includes a question and a standard answer; and determines the evaluation result corresponding to the image descriptions in the dataset through an evaluation result determination module based on the standard answer and the predicted answer corresponding to the question; wherein, the predicted answer is generated based on the question and the image description. This embodiment, by constructing a question and a standard answer for each target image, obtaining a predicted answer based on the question and the image description corresponding to the target image, and obtaining the evaluation result based on the standard answer and the predicted answer, can improve the accuracy and objectivity of image description quality evaluation.

[0138] Optionally, the dataset generation module is specifically used for: acquiring the target image; generating a first image description corresponding to the target image based on a first guide word and the target image using a target language model; generating a second image description based on a second guide word and the first image description, and using the first image description and / or the second image description as the image description in the dataset; wherein the first guide word is different from the second guide word; wherein the second image description is a multi-level image description, and the first image description is a detailed image description.

[0139] Optionally, the dataset generation module is further configured to: acquire multiple initial images; perform image filtering on the multiple initial images based on the image information in each initial image to obtain multiple target images; wherein the image information includes text information and / or detection target information.

[0140] Optionally, the dataset generation module is further configured to: perform cluster analysis on the plurality of initial images to obtain a plurality of cluster analysis images corresponding to each cluster category; extract text information from the plurality of cluster analysis images using a text extraction model to obtain text information for each cluster analysis image; extract target information from the plurality of cluster analysis images using a target detection model to obtain the target information for each cluster analysis image; and perform image filtering on the plurality of cluster analysis images based on the text information and / or the target information to obtain a plurality of target images corresponding to each cluster category.

[0141] The text extraction model includes an image encoder, a first dimension transformation module, and a first language model. Optionally, the dataset generation module is further configured to: for each cluster analysis image, generate image features based on the cluster analysis image using the image encoder; transform the dimensions of the image features using the first dimension transformation module to obtain target image features; and generate text information of the cluster analysis image based on the target image features using the first language model.

[0142] The target detection model includes a second language model, a second dimension transformation module, and a target detection sub-model.

[0143] Optionally, the dataset generation module is further configured to: for each clustering analysis image, generate detection target features based on the clustering analysis image and a third guide word using the second language model; wherein the third guide word is different from both the first and second guide words; convert the dimensions of the detection target features using the second dimension conversion module to obtain target detection target features; and generate the detection target information based on the target detection target features using the target detection sub-model.

[0144] Optionally, the dataset generation module is further configured to: count the number of characters in the text information of each cluster analysis image; count the number of categories of the detected target information of each cluster analysis image; sort the plurality of cluster analysis images according to the number of characters and / or the number of categories to obtain a sorting result; and perform image filtering on the plurality of cluster analysis images according to the sorting result to obtain a plurality of target images corresponding to each cluster category.

[0145] Optionally, the dataset generation module is further configured to: generate a first image description corresponding to the target image based on the first guiding word and the image information of the target image using the target language model.

[0146] The number of target images is multiple.

[0147] Optionally, the question-answer pair construction module is specifically used to: for each target image, construct multiple question-answer pairs for the target image according to multiple image description dimensions; wherein, each image description dimension corresponds to at least one question-answer pair; wherein, the question includes the image description dimension.

[0148] The dataset includes an image description for each of the multiple target images.

[0149] Optionally, the evaluation result determination module is specifically used to: generate a predicted answer for each question based on multiple questions corresponding to each target image and the image description using a third language model; and determine the evaluation result of each image description dimension in multiple image descriptions based on the predicted answer for each question and the standard answer for each question.

[0150] Optionally, the evaluation result determination module is further configured to: if the image description is a multi-level image description, perform semantic matching between the content type description rule corresponding to each level of the multi-level image description and the plurality of image description dimensions to determine the target image description dimension corresponding to each level of the image description; wherein, the corresponding content type description rule is used to generate the image description of the corresponding level; from the plurality of questions, take the question associated with the target image description dimension corresponding to each level of the image description as the target question of each level of the image description; and generate a predicted answer corresponding to each target question of each level of the image description based on the target question corresponding to each level of the image description and the image description of each level.

[0151] The dataset evaluation apparatus provided in this disclosure can execute the dataset evaluation method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0152] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0153] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0154] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0155] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the evaluation of method datasets.

[0156] In some embodiments, the evaluation of the method dataset may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the evaluation of the method dataset described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the evaluation of the method dataset by any other suitable means (e.g., by means of firmware).

[0157] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0159] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0161] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0162] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0163] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the dataset evaluation method provided in any embodiment of this application.

[0164] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0165] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for evaluating a dataset, characterized in that, include: Generate a dataset to be evaluated, the dataset including image descriptions corresponding to the target images; Construct question-answer pairs corresponding to the target image; wherein, the question-answer pairs include a question and a standard answer; The evaluation result corresponding to the image description in the dataset is determined based on the standard answer to the question and the predicted answer to the question; wherein the predicted answer is generated based on the question and the image description.

2. The method according to claim 1, characterized in that, The generation of the dataset to be evaluated includes: Acquire the target image; Using a target language model, a first image description corresponding to the target image is generated based on the first guiding word and the target image; A second image description is generated based on the second guide word and the first image description, and the first image description and / or the second image description are used as the image description in the dataset; wherein the first guide word is different from the second guide word; wherein the second image description is a multi-level image description, and the first image description is a detailed image description.

3. The method according to claim 2, characterized in that, The acquisition of the target image includes: Acquire multiple initial images; Based on the image information in each initial image, the multiple initial images are filtered to obtain multiple target images; wherein, the image information includes text information and / or detection target information.

4. The method according to claim 3, characterized in that, The step of filtering the multiple initial images based on image information in each initial image to obtain multiple target images includes: Cluster analysis is performed on the multiple initial images to obtain multiple cluster analysis images corresponding to each cluster category in multiple cluster categories; By using a text extraction model, text information is extracted from the multiple cluster analysis images to obtain the text information of each cluster analysis image; By using an object detection model, the detection target information in the multiple cluster analysis images is extracted to obtain the detection target information in each cluster analysis image; Based on the text information and / or the detection target information, image filtering is performed on the multiple cluster analysis images to obtain multiple target images corresponding to each cluster category.

5. The method according to claim 4, characterized in that, in, The text extraction model includes an image encoder, a first dimension transformation module, and a first language model; the step of extracting text information from the multiple clustering analysis images using the text extraction model to obtain the text information of each clustering analysis image includes: For each cluster analysis image, image features are generated based on the cluster analysis image using the image encoder; The first dimension conversion module converts the dimensions of the image features to obtain the target image features. The text information of the cluster analysis image is generated based on the target image features using the first language model.

6. The method according to claim 4, characterized in that, in, The target detection model includes a second language model, a second dimension transformation module, and a target detection sub-model; the step of extracting target information from the multiple clustering analysis images through the target detection model to obtain the target information of each clustering analysis image includes: For each clustering analysis image, the second language model generates detection target features based on the clustering analysis image and the third guide word; wherein the third guide word is different from both the first guide word and the second guide word. The second dimension conversion module is used to convert the dimensions of the detected target features to obtain the target detection target features; The target detection sub-model generates the target detection information based on the target detection features.

7. The method according to claim 4, characterized in that, The step of filtering the multiple cluster analysis images based on the text information and / or the detection target information to obtain multiple target images corresponding to each cluster category includes: Count the number of characters in the text information of each cluster analysis image; The number of categories of the detected target information in each cluster analysis image is counted; The multiple clustering analysis images are sorted according to the number of characters and / or the number of categories to obtain the sorting result; Based on the sorting results, the multiple cluster analysis images are filtered to obtain multiple target images corresponding to each cluster category.

8. The method according to claim 2, characterized in that, The step of generating a first image description corresponding to the target image based on the first guide word and the target image using a target language model includes: Using the target language model, a first image description corresponding to the target image is generated based on the first guiding word and the image information of the target image.

9. The method according to claim 1, characterized in that, in, The number of target images is multiple; the construction of question-answer pairs corresponding to the target images includes: For each target image, multiple question-answer pairs are constructed according to multiple image description dimensions; wherein each image description dimension corresponds to at least one question-answer pair; wherein the question includes the image description dimension.

10. The method according to claim 9, characterized in that, in, The dataset includes image descriptions corresponding to each target image in a plurality of target images; determining the evaluation result corresponding to the image descriptions in the dataset based on the standard answer to the question and the predicted answer to the question includes: Using a third language model, a predicted answer is generated for each question based on multiple questions corresponding to each target image and the image description. The evaluation result of each image description dimension in multiple image descriptions is determined based on the predicted answer and the standard answer corresponding to each question.

11. The method according to claim 10, characterized in that, The step of generating a predicted answer for each question based on multiple questions corresponding to each target image and the image description includes: If the image description is a multi-level image description, then the content type description rule corresponding to each level of the multi-level image description is semantically matched with the multiple image description dimensions to determine the target image description dimension corresponding to each level of the image description; wherein, the corresponding content type description rule is used to generate the image description of the corresponding level; From the plurality of questions, the question associated with the target image description dimension corresponding to each level of image description is taken as the target question of each level of image description; Based on the target question corresponding to each level of image description and the image description at each level, a predicted answer corresponding to each target question in each level of image description is generated.

12. An evaluation device for a dataset, characterized in that, include: A dataset generation module is used to generate a dataset to be evaluated, the dataset including image descriptions corresponding to the target images; A question-answer pair construction module is used to construct question-answer pairs corresponding to the target image; wherein, the question-answer pair includes a question and a standard answer; The evaluation result determination module is used to determine the evaluation result corresponding to the image description in the dataset based on the standard answer corresponding to the question and the predicted answer corresponding to the question; wherein, the predicted answer is generated based on the question and the image description.

13. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the dataset evaluation method as described in any one of claims 1-11.

14. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform an evaluation method for a dataset as described in any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the evaluation method for the dataset as described in any one of claims 1-11.

Images