Image data set generation method, device and equipment and computer readable storage medium

Abstract

The invention discloses an image data set generation method, device and equipment and a computer readable storage medium, and relates to the technical field of image processing, and the method comprises the steps: determining an initial keyword in response to an image request instruction; determining a plurality of different scene descriptions based on the initial keyword through a large language model, and generating a text prompt instruction corresponding to each scene description; inputting each text prompt instruction into an image generation model to obtain image data corresponding to each text prompt instruction; an image data set is constructed based on the image data. The generation efficiency of the image data set can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and computer-readable storage medium for generating an image data set. Background Art

[0002] As the field of artificial intelligence continues to develop, computer vision and natural language processing, as its core components, have significantly increased the demand for image datasets. Image datasets are a key resource for training and validating deep learning models for tasks such as image recognition, object detection, and image generation.

[0003] At present, common methods of generating image datasets usually rely on human assistance. Specifically, scene descriptions are manually input into an image generation model, and the image data corresponding to the scene description is obtained by using the text-based graph function of the image generation model. However, the image datasets used to train and verify deep learning models need to contain a large amount of image data. That is to say, according to the current method of generating image datasets, multiple scene descriptions need to be manually input to obtain sufficient image data, resulting in low efficiency in generating image datasets.

[0004] Therefore, how to improve the generation efficiency of image datasets is a problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of this application is to provide a method, device, equipment and computer-readable storage medium for generating an image dataset, aiming to improve the efficiency of generating an image dataset.

[0006] To achieve the above object, the present application provides a method for generating an image dataset, the method comprising:

[0007] In response to the image request instruction, determining an initial keyword;

[0008] Determine a plurality of different scene descriptions based on the initial keywords through a large language model, and generate text prompt instructions corresponding to each of the scene descriptions;

[0009] Inputting each of the text prompt instructions into an image generation model to obtain image data corresponding to each of the text prompt instructions;

[0010] An image data set is constructed based on each of the image data.

[0011] In one embodiment, the step of determining the initial keywords includes:

[0012] Parsing the image request instruction to obtain initial keywords; or

[0013] Keywords are extracted from the preset vocabulary as initial keywords through a large language model.

[0014] In one embodiment, the step of determining a plurality of different scene description information based on the initial keywords by using a large language model includes:

[0015] Expanding the initial keywords based on preset scene elements through a large language model to obtain multiple different extended scene descriptions;

[0016] Each of the extended scene descriptions and the initial keywords is used as a scene description.

[0017] In one embodiment, the step of generating text prompt instructions corresponding to each of the scene descriptions includes:

[0018] For any target scenario description in each of the scenario descriptions, determine the knowledge domain to which the target scenario description belongs, and construct a knowledge base based on the knowledge system corresponding to the knowledge domain;

[0019] Determine a scene element value in the target scene description, and generate description items of the scene element value based on the knowledge base, wherein the description items are attributes or features of the scene element value;

[0020] Based on the target scene description and each of the description items, a text prompt instruction corresponding to the target scene description is generated.

[0021] In one embodiment, the step of constructing an image data set based on each of the image data comprises:

[0022] Using an image segmentation and annotation algorithm to respectively annotate entity information in each of the image data to obtain image annotation information of each of the image data;

[0023] Determining each high-quality image data among each of the image data;

[0024] An image data set is constructed based on each of the high-quality image data, the image annotation information of each of the high-quality image data, and the text prompt instruction corresponding to each of the high-quality image data.

[0025] In one embodiment, the step of determining each high-quality image data in each of the image data comprises:

[0026] For any first image data among the image data, matching the image annotation information and the text prompt instruction corresponding to the first image data to obtain a matching degree;

[0027] When the matching degree is greater than a preset threshold, it is determined that the first image data is high-quality image data.

[0028] In one embodiment, the step of constructing an image data set based on each of the high-quality image data, the image annotation information of each of the high-quality image data, and the text prompt instruction corresponding to each of the high-quality image data includes:

[0029] For any second image data in each of the high-quality image data, establishing a mapping relationship between the second image data, the image annotation information of the second image data, and a text prompt instruction corresponding to the second image data;

[0030] An image data set is constructed based on each of the high-quality image data, the image annotation information of each of the high-quality image data, the text prompt instruction corresponding to each of the high-quality image data, and the mapping relationships.

[0031] In addition, to achieve the above-mentioned purpose, the present application also provides an image data set generation device, the image data set generation device comprising:

[0032] A response module, used for determining an initial keyword in response to an image request instruction;

[0033] A scene expansion module, used to determine a plurality of different scene descriptions based on the initial keywords through a large language model, and generate text prompt instructions corresponding to each of the scene descriptions;

[0034] A text-to-image conversion module, used to input each of the text prompt instructions into an image generation model to obtain image data corresponding to each of the text prompt instructions;

[0035] A construction module is used to construct an image data set based on each of the image data.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a program for implementing the image dataset generation method is stored on the computer-readable storage medium. The program for implementing the image dataset generation method is executed by a processor to implement the steps of the image dataset generation method as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned image data set generation method when executed by a processor.

[0038] The present application provides a method for generating an image dataset. The present application determines an initial keyword in response to an image request instruction, and then determines a plurality of different scene descriptions based on the initial keyword through a large language model, and determines a text prompt instruction corresponding to each scene description, and then inputs each text prompt instruction into an image generation model respectively to obtain image data corresponding to each text prompt instruction, and finally constructs an image dataset based on each image data.

[0039] In summary, compared to the traditional method that requires users to manually input multiple different scene descriptions to obtain image data of multiple different scenes, the present application can automatically derive multiple different scene descriptions based on the initial keywords, and can determine the text prompt instructions corresponding to each scene description through a large language model to guide the image generation model to generate the image data corresponding to each text prompt instruction, and then construct an image data set based on each image data. In this way, the present application can realize the automatic derivation of multiple different scene descriptions based on an initial keyword, and obtain multiple image data, thereby improving the generation efficiency of image data, and then improving the generation efficiency of image data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 This is a schematic diagram of the process of the first embodiment of the method for generating an image data set of the present application;

[0043] Figure 2 A schematic diagram of synthetic image data involved in an embodiment of the method for generating an image data set of the present application;

[0044] Figure 3 A schematic diagram of an image data set generation process involved in an embodiment of the image data set generation method of the present application;

[0045] Figure 4 This is a schematic diagram of the module structure of the image data set generating device of the present application;

[0046] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the image data set generation method in the embodiment of the present application.

[0047] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] The main solution of the present application is: in response to an image request instruction, determine an initial keyword; determine a plurality of different scene descriptions based on the initial keyword through a large language model, and generate text prompt instructions corresponding to each of the scene descriptions; input each of the text prompt instructions into an image generation model respectively to obtain image data corresponding to each of the text prompt instructions; and construct an image data set based on each of the image data.

[0051] At present, common methods of generating image datasets usually rely on human assistance. Specifically, scene descriptions are manually input into an image generation model, and the image data corresponding to the scene description is obtained by using the text-based graph function of the image generation model. However, the image datasets used to train and verify deep learning models need to contain a large amount of image data. That is to say, according to the current method of generating image datasets, multiple scene descriptions need to be manually input to obtain sufficient image data, resulting in low efficiency in generating image datasets.

[0052] Compared to the traditional method that requires users to manually input multiple different scene descriptions to obtain image data of multiple different scenes, this application can automatically derive multiple different scene descriptions based on the initial keywords, and can determine the text prompt instructions corresponding to each scene description through a large language model to guide the image generation model to generate the image data corresponding to each text prompt instruction, and then build an image data set based on each image data. In this way, this application can realize the automatic derivation of multiple different scene descriptions based on an initial keyword and obtain multiple image data, thereby improving the generation efficiency of image data, and then improving the generation efficiency of image data sets.

[0053] It should be noted that the execution subject of the method in each embodiment of the image dataset generation method of the present application can be an image dataset generation system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an image dataset generation device capable of realizing the above functions, etc., and this embodiment does not specifically limit this. The following takes the image dataset generation system as the execution subject as an example to illustrate this embodiment and the following embodiments.

[0054] Based on this, the present application proposes a method for generating an image data set in the first embodiment, please refer to Figure 1 , the image data set generation method comprises steps S10 to S40:

[0055] Step S10, determining an initial keyword in response to the image request instruction;

[0056] It should be noted that the embodiment of the present application does not limit the triggering method of the above-mentioned image request instruction. For example, in one feasible implementation, the image request instruction can be manually triggered by the user. For example, in another feasible implementation, a timer can be pre-set to trigger the image request instruction once every preset time interval to ensure the update rate of the image data set. In addition, the initial keyword is a word or a phrase consisting of multiple words. For example, the initial keyword can be "girl" or "girl under the sky".

[0057] In this embodiment, the step S10 may include:

[0058] Step S101, parsing the image request instruction to obtain initial keywords;

[0059] It should be noted that the image request instruction may include initial keywords specified by the user.

[0060] In a feasible implementation, in response to the initial keyword input by the user, an image request instruction is generated based on the initial keyword, so that after receiving the image request instruction, the image dataset generation system parses the image request instruction to obtain the initial keyword specified by the user.

[0061] Step S102: extracting keywords from a preset vocabulary as initial keywords through a large language model.

[0062] It should be noted that the above-mentioned large language model can be GPT-4 or other models with language understanding and generation capabilities. The preset vocabulary refers to the existing dictionary and knowledge base in the large language model.

[0063] In another feasible implementation, after receiving the image request instruction, the large language model is used to extract keywords from the preset word library as initial keywords. Specifically, the keyword extraction rule of the large language model is pre-set to give priority to extracting keywords in common scenes or scene fields. In this way, the problem of traditional image data generation relying on manual input is solved.

[0064] Step S20, determining a plurality of different scene descriptions based on the initial keyword through a large language model, and determining a text prompt instruction corresponding to each of the scene descriptions, wherein the scene description at least includes the initial keyword;

[0065] It should be noted that the multiple scene descriptions determined by the large language model based on the initial keyword are different, and each scene description includes at least the initial keyword. The text prompt instruction corresponding to the scene description refers to a text description used to guide the image generation model to generate image data.

[0066] A large language model is used to determine multiple different scene descriptions based on initial keywords, and a text prompt instruction corresponding to each scene description is generated.

[0067] In this embodiment, the step S20 may include:

[0068] Step S201, expanding the initial keyword based on preset scene elements through a large language model to obtain multiple different extended scene descriptions;

[0069] It should be noted that the preset scene elements are key information in the scene description, and the preset scene elements include but are not limited to time, place, characters (or objects) and their relationships, etc. It should be understood that the relationship in the preset scene elements refers to the relationship between time, place and characters (or objects). In this way, the logic and consistency of the scene description are ensured.

[0070] The large language model can expand the initial keyword based on the preset scene elements to obtain multiple different description information (hereinafter referred to as extended scene description for distinction). It should be noted that the extended scene description is a phrase composed of the initial keyword and other keywords.

[0071] Exemplarily, when the initial keyword is "girl", the large language model can generate multiple extended scene descriptions based on the internally learned language and semantic knowledge and the initial keyword, where the extended scene descriptions may include "girl under the sky", "girl in the classroom", "girl under the sky at dusk", "girl in the classroom at sunrise", etc. It can be understood that the large language model expands the location of the preset scene elements based on the internal knowledge system to obtain "girl under the sky" and "girl in the classroom", and the large language model expands the time and place of the preset scene elements based on the internal knowledge system to obtain "girl under the sky at dusk" and "girl in the classroom at sunrise".

[0072] Step S202: taking each of the extended scene descriptions and the initial keywords as scene descriptions.

[0073] The large language model uses each different extended scene description and the initial keyword as the scene description, that is, the scene description at least includes the initial keyword.

[0074] In this way, the embodiment of the present application utilizes the semantic understanding ability of a large language model to generate descriptions of each scene based on initial keywords, thereby providing a semantic framework for subsequent image generation.

[0075] In this embodiment, the step S20 may include:

[0076] Step S203, for any target scenario description in each of the scenario descriptions, determining the knowledge domain to which the target scenario description belongs, and constructing a knowledge base based on the knowledge system corresponding to the knowledge domain;

[0077] The large language model can identify the knowledge domain to which the current target scene description belongs based on its internal knowledge system, and build a knowledge base based on the knowledge system corresponding to the knowledge domain. It should be understood that the content contained in the knowledge base is related to the current target scene description. And there may be multiple knowledge domains described in a scene description, so the knowledge base is constructed based on knowledge in multiple different knowledge domains.

[0078] Step S204, determining scene element values ​​in the target scene description, and generating description items of the scene element values ​​based on the knowledge base;

[0079] It should be noted that any scene description in each scene description is referred to as a target scene description for distinction. Scene elements in a scene description may include time, place, person (or object) and their relationship. The scene element value refers to the specific value of the scene element. For example, if the scene element is time, then the scene element value corresponding to the scene element may be "8 o'clock in the morning"; if the scene element is place, then the scene element value corresponding to the scene element may be "under the sky". The description item of the scene element value is the attribute or feature of the scene element value.

[0080] For any scene description in each scene description (i.e., the target scene description), the scene element value in the target scene description is determined by the large language model, and then each description item of the scene element value is generated based on the knowledge base. The description item can be understood as a keyword phrase, which is used to provide a semantic framework for image generation.

[0081] Exemplarily, when the target scene description is "girl", the scene element value in the target scene description is "girl", then the attributes of the scene element value determined by the large language model based on the knowledge base include but are not limited to: age, height, hair length, hair color, glasses color, dressing style, etc., and the characteristics of the scene element value include but are not limited to: expression, posture, emotional state and background activities, etc. The above attributes and characteristics are collectively referred to as the descriptive items of the scene element value.

[0082] In this way, the embodiment of the present application utilizes the semantic mining capability of a large language model to generate a knowledge base related to the current scene description, and determines each description item based on the knowledge base to provide input for the generation of subsequent text prompt instructions.

[0083] Step S205: generating a text prompt instruction corresponding to the target scene description based on the target scene description and each of the description items.

[0084] It should be noted that the embodiments of the present application can specify one or more styles of image generation according to user needs or automation requirements, and generate text prompt instructions based on the specified image styles, that is, the text prompt instructions corresponding to the target scene description are generated based on the target scene description, the description items corresponding to the target scene description, and the image style corresponding to the target scene description. Among them, the image style includes but is not limited to illustration style, realistic style, and abstract art style.

[0085] A text prompt instruction corresponding to the target scene description, namely, a Prompt instruction, is generated through the large language model based at least on the target scene description and each description item.

[0086] In this way, since each description item corresponding to the target scene description is a semantic framework, the large language model can set a description item to different description values. For example, when the description item is hair color, the large language model can be set to black, yellow, brown, etc., so a scene description can correspond to multiple different text prompt instructions, that is, the generation of Prompt is highly flexible and customizable, and can be adaptively adjusted according to different application scenarios.

[0087] In one feasible implementation, the large language model determines the description value corresponding to each description item based on the knowledge base, and then combines the description values ​​into a semantically coherent Prompt instruction based on semantic logic.

[0088] Step S30, inputting each of the text prompt instructions into an image generation model to obtain image data corresponding to each of the text prompt instructions;

[0089] It should be noted that the image generation model can be a text-based image model such as DALL-E or Stable Diffusion, which is used to convert text prompt instructions into image data, and can ensure that the generated image data is highly consistent with the semantics of the Prompt.

[0090] Each text prompt instruction is input into the image generation model to obtain image data corresponding to each text prompt instruction.

[0091] It should be understood that multiple image styles can be specified in one text prompt instruction, that is, the image generation model can generate multiple image data of different styles based on one text prompt instruction. Each style of image data can reflect different visual characteristics, thereby improving the diversity and adaptability of image data.

[0092] In this way, the embodiment of the present application realizes text-image conversion by utilizing an image generation model, and combines the scene and style instructions in the Prompt to ensure that the generated image is highly consistent with the semantics of the description.

[0093] Exemplarily, in the scene description of "a football on a football field at dusk", the description items for the scene element values ​​in the scene description include description items for dusk, description items for the football field and description items for football, wherein the description items for dusk include: color, light and shadow, the description items for the football field include: venue, boundary and facilities, and the description items for football include: appearance, location and status. Then the large language model can generate a Prompt instruction based on the above scene description and each description item, and the Prompt instruction is "Please generate a picture depicting a football on a football field at dusk. In the picture, the sky presents a gradient of orange, pink, purple and dark blue, creating a warm and tranquil dusk atmosphere. The grass on the football field is green, and the sidelines and goal areas are clearly visible. In the center of the football field or near the goal area, a black and white football lies quietly on the grass, as if waiting for the game to start. The whole scene is full of the atmosphere of sports and the tranquility of dusk, making people feel the charm of football and the beauty of dusk". For details, refer to Figure 2 Schematic diagram of synthetic image data.

[0094] Step S40: constructing an image data set based on each of the image data.

[0095] After the image generation model is obtained based on the image data corresponding to each text prompt instruction, an image data set is constructed based on each image data.

[0096] The embodiment of the present application can automatically derive multiple different scene descriptions that at least contain the initial keyword based on the initial keyword, and can determine the text prompt instructions corresponding to each scene description through a large language model to guide the image generation model to generate image data corresponding to each text prompt instruction, and then construct an image data set based on each image data. In this way, the embodiment of the present application can realize the automatic derivation of multiple different scene descriptions based on an initial keyword, and obtain multiple image data, thereby improving the generation efficiency of image data, and then improving the generation efficiency of image data sets.

[0097] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. On this basis, the step S40 may include:

[0098] Step S401, using an image segmentation and annotation algorithm to respectively annotate entity information in each of the image data to obtain image annotation information of each of the image data;

[0099] It should be noted that the above-mentioned image segmentation and annotation algorithm can be an OMG-Seg algorithm, which is used to automatically annotate the entity information of the generated image data. The algorithm can identify different objects in the image data and mark each object with corresponding annotation information, i.e., a label. And the algorithm can improve the automation and accuracy of image annotation. Among them, the annotation information includes but is not limited to object category, boundary, location information, etc., to ensure that each element in the image data can correspond to the corresponding semantic entity.

[0100] The entity information in each image data is annotated respectively by using an image segmentation and annotation algorithm to obtain respective annotation information of each image data (hereinafter referred to as image annotation information for distinction).

[0101] In this way, the image annotation process in the embodiment of the present application is automated, implemented through advanced segmentation algorithms, and maintains consistency with the entities in the image.

[0102] Step S402, determining each high-quality image data in each of the image data;

[0103] It should be noted that high-quality image data refers to image data in which the degree of matching between image annotation information and text prompt instructions is greater than a preset threshold.

[0104] In this embodiment, the step S402 may include:

[0105] Step S4021, for any first image data among the image data, matching the image annotation information and the text prompt instruction corresponding to the first image data to obtain a matching degree;

[0106] Step S4022: When the matching degree is greater than a preset threshold, determining that the first image data is high-quality image data.

[0107] It should be noted that the embodiment of the present application does not limit the specific size of the preset threshold, and in this embodiment, the preset threshold is set to 90%. Any image data in each image data is referred to as the first image data for distinction.

[0108] For any image data among the image data (i.e., the first image data mentioned above), match the image annotation information and text prompt instructions corresponding to the first image data to obtain a matching degree, and determine whether the matching degree is greater than a preset threshold. If the matching degree is greater than the preset threshold, it is determined that the current first image data is high-quality image data. If the matching degree is less than or equal to the preset threshold, the current first image data is discarded.

[0109] Exemplarily, the image annotation information generated for each image data is matched with the keywords in the Prompt corresponding to the image data to obtain a matching degree, and the image data with a matching degree exceeding a preset threshold is determined as high-quality image data and enters the data set preservation stage.

[0110] In this way, the embodiment of the present application ensures a high degree of relevance between image content and semantic description through a consistency matching algorithm, thereby improving the overall quality of the image dataset.

[0111] Step S403: constructing an image data set based on each of the high-quality image data, the image annotation information of each of the high-quality image data, and the text prompt instruction corresponding to each of the high-quality image data.

[0112] An image data set is constructed based on each high-quality image data in each image data, image annotation information of each high-quality image data, and a text prompt instruction corresponding to each high-quality image data.

[0113] For example, Figure 4 The figure shows a schematic diagram of the image data set generation process. First, a plurality of different scene descriptions are generated based on the initial keywords through the large language model; a corresponding knowledge base is generated for each scene description through the large language model to obtain the description items corresponding to each scene description; then a Prompt instruction is generated based on the scene description, the description items corresponding to the scene description, and the image style through the large language model; then the image data is generated based on the Prompt instruction through the large text-image model (i.e., the image generation model); the image data is annotated using the image segmentation and annotation model to obtain image annotation information; based on the key entity similarity model, the image annotation information corresponding to each image data and the description value in the Prompt instruction are determined to determine the similarity score (i.e., the above-mentioned matching degree); it is determined whether the similarity score of each image data is higher than 90%. If so, the image data is added to the image data set as high-quality image data; if not, the image data is discarded.

[0114] In this embodiment, the step S403 may include:

[0115] Step S4031, for any second image data in each of the high-quality image data, establishing a mapping relationship between the second image data, the image annotation information of the second image data, and the text prompt instruction corresponding to the second image data;

[0116] It should be noted that any image data among the high-quality image data is referred to as second image data for distinction.

[0117] For any image data among the high-quality image data (i.e., the above-mentioned second image data), a mapping relationship between the second image data, the image annotation information of the second image data, and the text prompt instruction corresponding to the second image data is established based on the second image data to obtain the mapping relationship corresponding to each high-quality image data.

[0118] Step S4032: constructing an image data set based on each of the high-quality image data, the image annotation information of each of the high-quality image data, the text prompt instruction corresponding to each of the high-quality image data, and each of the mapping relationships.

[0119] An image data set is constructed based on each high-quality image data, image annotation information of each high-quality image data, text corresponding to each high-quality image data to indicate instructions and each mapping relationship.

[0120] For example, the high-quality image datasets screened through the above steps will be saved in a specified database. Each image is annotated with annotation information and detailed semantic descriptions, ensuring that the dataset contains not only visual content but also rich semantic information. This dataset can be used for various downstream tasks, such as training, testing, and evaluation of computer vision models.

[0121] In this way, in the embodiment of the present application, an image dataset is constructed based on high-quality image data, image annotation information and text prompt instructions of the high-quality image data, and the mapping relationship between the three, ensuring that the final generated dataset has high-quality, rich annotation information and semantic description, thereby significantly improving the application value of the dataset.

[0122] The present application also provides an image data set generation device, please refer to Figure 4 , the image data set generating device comprises:

[0123] A response module 10, for determining an initial keyword in response to an image request instruction;

[0124] A scene expansion module 20, configured to determine a plurality of different scene descriptions based on the initial keywords through a large language model, and generate text prompt instructions corresponding to each of the scene descriptions;

[0125] The text-to-image conversion module 30 is used to input each of the text prompt instructions into the image generation model to obtain image data corresponding to each of the text prompt instructions;

[0126] The construction module 40 is used to construct an image data set based on each of the image data.

[0127] Optionally, the response module 10 is further used for:

[0128] Parsing the image request instruction to obtain initial keywords; or

[0129] Keywords are extracted from the preset vocabulary as initial keywords through a large language model.

[0130] Optionally, the scene extension module 20 is further used for:

[0131] Expanding the initial keywords based on preset scene elements through a large language model to obtain multiple different extended scene descriptions;

[0132] Each of the extended scene descriptions and the initial keywords is used as a scene description.

[0133] Optionally, the scene extension module 20 is further used for:

[0134] For any target scenario description in each of the scenario descriptions, determine the knowledge domain to which the target scenario description belongs, and construct a knowledge base based on the knowledge system corresponding to the knowledge domain;

[0135] Determine a scene element value in the target scene description, and generate description items of the scene element value based on the knowledge base, wherein the description items are attributes or features of the scene element value;

[0136] Based on the target scene description and each of the description items, a text prompt instruction corresponding to the target scene description is generated.

[0137] Optionally, the building module 40 is further used for:

[0138] Using an image segmentation and annotation algorithm to respectively annotate entity information in each of the image data to obtain image annotation information of each of the image data;

[0139] Determining each high-quality image data among each of the image data;

[0140] An image data set is constructed based on each of the high-quality image data, the image annotation information of each of the high-quality image data, and the text prompt instruction corresponding to each of the high-quality image data.

[0141] Optionally, the building module 40 is further used for:

[0142] For any first image data among the image data, matching the image annotation information and the text prompt instruction corresponding to the first image data to obtain a matching degree;

[0143] When the matching degree is greater than a preset threshold, it is determined that the first image data is high-quality image data.

[0144] Optionally, the building module 40 is further used for:

[0145] For any second image data in each of the high-quality image data, establishing a mapping relationship between the second image data, the image annotation information of the second image data, and a text prompt instruction corresponding to the second image data;

[0146] An image data set is constructed based on each of the high-quality image data, the image annotation information of each of the high-quality image data, the text prompt instruction corresponding to each of the high-quality image data, and the mapping relationships.

[0147] The image data set generation device provided in the embodiment of the present application adopts the image data set generation method in the above embodiment, which can solve the technical problem of how to improve the generation efficiency of the image data set. Compared with the prior art, the beneficial effects of the image data set generation device provided in the embodiment of the present application are the same as the beneficial effects of the image data set generation method provided in the above embodiment, and other technical features in the image data set generation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0148] The present application provides an image data set generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image data set generation method in the above-mentioned embodiment 1.

[0149] Reference below Figure 5 , which shows a schematic diagram of the structure of an image data set generation device suitable for implementing the embodiment of the present application. The image data set generation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5The image data set generating device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0150] like Figure 5 As shown, the image data set generating device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the image data set generating device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the image data set generation device to communicate wirelessly or wired with other devices to exchange data. Although the image data set generation device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0151] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0152] The image data set generation device provided by the present application adopts the image data set generation method in the above embodiment, which can solve the technical problem of how to improve the generation efficiency of the image data set. Compared with the prior art, the beneficial effects of the image data set generation device provided by the present application are the same as the beneficial effects of the image data set generation method provided by the above embodiment, and other technical features in the image data set generation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0153] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0154] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0155] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the image data set generation method in the above-mentioned embodiment.

[0156] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0157] The computer-readable storage medium may be included in the image dataset generating device; or may exist independently without being assembled into the image dataset generating device.

[0158] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the image dataset generating device, the image dataset generating device: determines the initial keyword in response to the image request instruction; determines a plurality of different scene descriptions based on the initial keyword through a large language model, and generates text prompt instructions corresponding to each of the scene descriptions, wherein the scene description includes at least the initial keyword; inputs each of the text prompt instructions into the image generation model respectively to obtain the image data corresponding to each of the text prompt instructions; and constructs an image dataset based on each of the image data.

[0159] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0160] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0162] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned image data set generation method, and can solve the technical problem of how to improve the generation efficiency of image data sets. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the image data set generation method provided in the above-mentioned embodiment, and will not be repeated here.

[0163] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the above-mentioned image data set generation method when executed by a processor.

[0164] The computer program product provided in this application can improve the efficiency of generating image data sets. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the image data set generation method provided in the above embodiment, which will not be repeated here.

[0165] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A method for generating an image dataset, characterized in that: The image data set generation method comprises: In response to the image request instruction, determining an initial keyword; Determine a plurality of different scene descriptions based on the initial keywords through a large language model, and generate text prompt instructions corresponding to each of the scene descriptions; Inputting each of the text prompt instructions into an image generation model to obtain image data corresponding to each of the text prompt instructions; An image data set is constructed based on each of the image data.

2. The method according to claim 1, characterized in that The step of determining the initial keywords includes: Parsing the image request instruction to obtain initial keywords; or Keywords are extracted from the preset vocabulary as initial keywords through a large language model.

3. The method according to claim 1, characterized in that The step of determining a plurality of different scene description information based on the initial keywords by using a large language model includes: Expanding the initial keywords based on preset scene elements through a large language model to obtain multiple different extended scene descriptions; Each of the extended scene descriptions and the initial keywords is used as a scene description.

4. The method according to claim 1, characterized in that The step of generating text prompt instructions corresponding to each of the scene descriptions includes: For any target scenario description in each of the scenario descriptions, determine the knowledge domain to which the target scenario description belongs, and construct a knowledge base based on the knowledge system corresponding to the knowledge domain; Determine a scene element value in the target scene description, and generate description items of the scene element value based on the knowledge base, wherein the description items are attributes or features of the scene element value; Based on the target scene description and each of the description items, a text prompt instruction corresponding to the target scene description is generated.

5. The method according to claim 1, characterized in that The step of constructing an image data set based on each of the image data comprises: Using an image segmentation and annotation algorithm to respectively annotate entity information in each of the image data to obtain image annotation information of each of the image data; Determining each high-quality image data among each of the image data; An image data set is constructed based on each of the high-quality image data, the image annotation information of each of the high-quality image data, and the text prompt instruction corresponding to each of the high-quality image data.

6. The method according to claim 5, characterized in that The step of determining each high-quality image data in each of the image data comprises: For any first image data among the image data, matching the image annotation information and the text prompt instruction corresponding to the first image data to obtain a matching degree; When the matching degree is greater than a preset threshold, it is determined that the first image data is high-quality image data.

7. The method according to claim 5, characterized in that The step of constructing an image data set based on each of the high-quality image data, the image annotation information of each of the high-quality image data, and the text prompt instruction corresponding to each of the high-quality image data comprises: For any second image data in each of the high-quality image data, establishing a mapping relationship between the second image data, the image annotation information of the second image data, and a text prompt instruction corresponding to the second image data; An image data set is constructed based on each of the high-quality image data, the image annotation information of each of the high-quality image data, the text prompt instruction corresponding to each of the high-quality image data, and the mapping relationships.

8. An image data set generating device, characterized in that: The image data set generating device comprises: A response module, used for determining an initial keyword in response to an image request instruction; A scene expansion module, used to determine a plurality of different scene descriptions based on the initial keywords through a large language model, and generate text prompt instructions corresponding to each of the scene descriptions; A text-to-image conversion module, used to input each of the text prompt instructions into an image generation model to obtain image data corresponding to each of the text prompt instructions; A construction module is used to construct an image data set based on each of the image data.

9. An image data set generation device, characterized in that: The image dataset generating device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the image dataset generating method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image data set generation method according to any one of claims 1 to 7 are implemented.

Images