Object sample image generation method and device and electronic equipment

Abstract

The embodiment of the invention provides an object sample image generation method and device and electronic equipment, and the method comprises the steps: determining a to-be-applied prompt statement based on at least one first to-be-selected vocabulary of a first attribute dimension, at least one second to-be-selected vocabulary of a second attribute dimension and a statement logic detection condition; generating at least one to-be-selected object sample image based on the to-be-applied prompt statement and a pre-trained object sample image generation model; and performing image quality evaluation on each to-be-selected object sample image to obtain an image quality score corresponding to each to-be-selected object sample image, so as to determine the target object sample image based on the image quality score, thereby improving the generation efficiency of the prompt statement, improving the richness, complexity and diversity of the prompt statement, and further improving the user experience. The generation efficiency and the image quality of the object sample image are improved, in addition, the image quality of the object sample image is evaluated through an image quality evaluation mechanism, and the image quality of the object sample image is further improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of information processing technology, and in particular to a method, device and electronic device for generating an object sample image. Background Art

[0002] Artificial intelligence large models (such as face processing models) are widely used in various industries, requiring a large and rich face image dataset as training data for face processing models to improve the performance of face processing models. The face image data can be generated by inputting descriptive prompt text into a text-image joint model to obtain a target face image that meets the prompt. The quality and diversity of the target face images generated by this technology largely depends on the prompt text.

[0003] At present, the main methods for determining prompt text are: prompt text generation method based on fixed templates and prompt text generation method relying on manual editing. However, due to the fixed structure and limited description dimension of the first method, the generated prompt text lacks richness, resulting in the generated image being limited in expressiveness and diversity. The second method requires a lot of manpower and is extremely inefficient. The generated prompt text is of varying quality and is difficult to support large-scale generation tasks. Based on this, there are problems such as low efficiency in prompt text generation, lack of richness, complexity and diversity in prompt text, and low efficiency and poor quality in face image sample generation. Summary of the invention

[0004] The embodiments of the present invention provide a method, device and electronic device for generating object sample images to improve the generation efficiency of prompt statements, enhance the richness, complexity and diversity of prompt statements, thereby improving the generation efficiency and image quality of object sample images. In addition, the image quality of the generated object sample images is evaluated through an image quality scoring mechanism to obtain object sample images that meet the image quality standards, thereby further improving the image quality of the object sample images.

[0005] In a first aspect, an embodiment of the present invention provides a method for generating an object sample image, the method comprising:

[0006] Determine the prompt sentence to be applied based on at least one first candidate word corresponding to the first attribute dimension, at least one second candidate word corresponding to the second attribute dimension, and a sentence logic test condition; wherein the first attribute dimension is a required attribute dimension when generating the prompt sentence to be applied, and the second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied;

[0007] Based on the prompt sentence to be applied and a pre-trained object sample image generation model, generating at least one sample image of the object to be selected;

[0008] An image quality assessment is performed on the at least one sample image of the object to be selected to obtain an image quality score corresponding to the at least one sample image of the object to be selected, so as to determine the target object sample image based on the image quality score.

[0009] In a second aspect, an embodiment of the present invention further provides a device for generating an object sample image, the device comprising:

[0010] A prompt sentence determination module, used to determine the prompt sentence to be applied based on a first candidate word corresponding to at least one first attribute dimension, a second candidate word corresponding to at least one second attribute dimension, and a sentence logic test condition; wherein the first attribute dimension is a mandatory attribute dimension when generating the prompt sentence to be applied, and the second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied;

[0011] A sample image generation module, used to generate at least one sample image of a to-be-selected object based on the prompt sentence to be applied and a pre-trained object sample image generation model;

[0012] The sample image screening module is used to perform image quality assessment on the at least one sample image of the candidate object to obtain an image quality score corresponding to the at least one sample image of the candidate object, so as to determine the target object sample image based on the image quality score.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the object sample image generation method as described in any one of the embodiments of the present invention.

[0016] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the object sample image generation method as described in any one of the embodiments of the present invention.

[0017] The technical solution of the embodiment of the present invention determines the prompt sentence to be applied through at least one first attribute dimension corresponding to the first candidate vocabulary, at least one second attribute dimension corresponding to the second candidate vocabulary and the sentence logic test condition, wherein the first attribute dimension is a mandatory attribute dimension when generating the prompt sentence to be applied, and the second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied, and then, based on the prompt sentence to be applied and the pre-trained object sample image generation model, at least one to-be-selected object sample image is generated, thereby performing image quality evaluation on at least one to-be-selected object sample image, obtaining an image quality score corresponding to at least one to-be-selected object sample image, and determining the target object sample image based on the image quality score. The technical solution provided by this embodiment, under the constraint of the sentence logic test condition, randomly combines the vocabulary in the mandatory attribute dimension and the optional attribute dimension to automatically generate the prompt sentence, thereby improving the generation efficiency of the prompt sentence, and improving the richness, complexity and diversity of the prompt sentence, thereby improving the generation efficiency and image quality of the object sample image, and in addition, the image quality evaluation of the generated object sample image is performed through the image quality score mechanism to obtain the object sample image with the image quality reaching the standard, further improving the image quality of the object sample image. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram of a flow chart of a method for generating an object sample image provided by an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of a flow chart of another object sample image generation method provided by an embodiment of the present invention;

[0021] Figure 3 A schematic diagram of a flow chart of another object sample image generation method provided by an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of the structure of an object sample image generating device provided by an embodiment of the present invention;

[0023] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0025] Embodiment 1

[0026] Figure 1 A flow chart of a method for generating an object sample image provided in an embodiment of the present invention is provided. This embodiment is applicable to any situation where an object sample image needs to be generated. The method can be executed by an object sample image generating device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC or a server.

[0027] like Figure 1 As shown, the object sample image generation method includes:

[0028] S110: Determine a prompt sentence to be applied based on a first candidate word corresponding to at least one first attribute dimension, a second candidate word corresponding to at least one second attribute dimension, and a sentence logic verification condition.

[0029] Among them, the first attribute dimension is a mandatory attribute dimension when generating the prompt sentence to be applied. The first attribute dimension is indispensable for generating the prompt sentence. The first candidate vocabulary refers to the vocabulary contained in each first attribute dimension. The second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied. In other words, the second attribute dimension is an attribute dimension that can be selected or not when generating the prompt sentence. The purpose of setting the second attribute dimension is to increase the diversity of the prompt sentence. The second candidate vocabulary is the vocabulary contained in each second attribute dimension.

[0030] Specifically, the first attribute dimension may be an object basic attribute dimension. For example, the first attribute dimension may include at least one of an age dimension, a gender dimension, and an occupation dimension. The first candidate word corresponding to the age dimension may be expressed as: {age dimension: "baby", "teenager", "young person", "middle-aged person", "elderly person"}; the first candidate word corresponding to the gender dimension may be expressed as: {gender dimension: "male", "female"}; the first candidate word corresponding to the occupation dimension may be expressed as: {occupation dimension: "doctor", "scientist", "teacher"...}.

[0031] The second attribute dimension includes: at least one of the image background dimension, the object emotion dimension and the object special mark dimension. Among them, the image background dimension is used to characterize the image background information contained in the object sample image. For example, the second candidate vocabulary corresponding to the image background dimension can be expressed as: {image background dimension: "busy city", "quiet forest", "future technology city"...}. The object emotion dimension is used to characterize the emotion of the object contained in the object sample image. For example, the second candidate vocabulary corresponding to the object emotion dimension can be expressed as: {object emotion dimension: "smile", "focus", "excited"...}. The object special mark dimension is used to characterize the iconic information of the object contained in the object sample image. For example, the second candidate vocabulary corresponding to the object special mark dimension can be expressed as: {object special mark dimension: "facial scar", "earring", "headscarf"...}.

[0032] Among them, the sentence logic test condition is a pre-set rule, which is used to test whether the sentence composed of the selected words conforms to the actual objective law. For example, the sentence logic test condition may include: the rationality test condition of the sentence combining age and occupation. For example, a sentence whose occupation is "doctor" and another whose occupation is "scientist" cannot meet the rationality test condition of the sentence combining age and occupation, because "baby" cannot be "doctor".

[0033] The prompt sentence to be applied refers to the prompt text that will be applied to the sample image of the generated object. The prompt text refers to a piece of text input to the model, which is intended to guide the model to generate output with specific content or following a specific context. The prompt sentence can be a sentence, a question, a description, or any information that can provide the context and theme required for generation.

[0034] It is understandable that the technical solution provided in this embodiment is executed cyclically, which refers to the current cycle in the cycle process. In other words, a large number of target object sample images can be obtained by using the object sample image generation method provided in this embodiment, and a part of the target object sample images can be obtained after each cycle is completed.

[0035] Specifically, the implementation method for determining the prompt sentence to be applied can be: each first attribute dimension corresponds to a plurality of first candidate words respectively constitute a required attribute pool, so that a first candidate word can be randomly selected from the required attribute pool corresponding to each first attribute dimension. Each second attribute dimension corresponds to a plurality of second candidate words respectively constitute a random attribute pool, based on which, at least one target random attribute pool is first determined from each random attribute pool, so that a second candidate word is randomly selected from each target random attribute pool. Further, the selected first candidate words and second candidate words are determined as target words, and the target words are constructed into a complete sentence that conforms to grammatical logic, and the logical rationality of the complete sentence is tested through the sentence logic test condition. If the test passes, the complete sentence is determined as the corresponding prompt sentence to be applied.

[0036] For example, for the first attribute dimension, the target vocabulary selected from the age dimension is "young people", the target vocabulary selected from the gender dimension is "female", and the target vocabulary selected from the occupation dimension is "doctor"; for the second attribute dimension, the target vocabulary selected from the image background dimension is "future technology city", and the target vocabulary selected from the object emotion dimension is "smile", then the corresponding prompt sentence to be applied can be expressed as "a young female doctor, with a future technology city in the background, with a smile on her face."

[0037] S120: Generate at least one sample image of an object to be selected based on the prompt sentence to be applied and a pre-trained object sample image generation model.

[0038] Among them, the object sample image generation model is a pre-trained image generation model, which is used to generate at least one object sample image that is consistent with the prompt text according to the prompt text. For example, the object sample image generation model can adopt the Flux model, which has been trained with a large-scale data set and has the ability to generate high-quality images. The Flux model is based on the Diffusion Models architecture and can generate high-quality, detailed images. The Flux model has excellent performance in diversity and generation quality, and is particularly suitable for generating images with complex backgrounds and delicate facial features. An object sample image refers to an image containing an object object. The selected object sample image refers to an object sample image that needs to be further screened.

[0039] In this embodiment, the prompt sentence to be applied can be used as the input of the object sample image generation model, and the object sample image generation model generates at least one sample image of the object to be selected corresponding to the prompt sentence to be applied through the text-image alignment mechanism within the model. It should be particularly noted that the number of sample images of the object to be selected can be one or more, and the configuration parameters related to the number of images can be set in the object sample image generation model according to the needs of the user, so that the number of sample images of the object to be selected output by the object sample image generation model is adapted to the configuration parameters.

[0040] S130. Perform image quality assessment on at least one sample image of the object to be selected to obtain an image quality score corresponding to the at least one sample image of the object to be selected, and determine a sample image of the target object based on the image quality score.

[0041] The image quality score refers to a quantitative score representing the quality of the sample image of the selected object. For example, the higher the image quality score, the better the image quality of the sample image of the selected object, and the lower the image quality score, the worse the image quality of the sample image of the selected object. The target object sample image refers to the object sample image whose image quality meets the requirements after screening the sample images of the selected object by the image quality score.

[0042] Specifically, the image quality evaluation can be performed on the sample images of the selected objects according to the preset image quality evaluation method to obtain the image quality score corresponding to each sample image of the selected object. On this basis, according to the numerical values ​​of the image quality scores corresponding to the sample images of the selected objects, the sample images of the objects whose image quality meets the requirements can be determined from the sample images of the selected objects, and these sample images of the objects are the corresponding sample images of the target objects.

[0043] Optionally, a specific implementation manner of determining the target object sample image based on the image quality score may include: determining the candidate object sample image having an image quality score greater than or equal to a preset quality score threshold as the target object sample image.

[0044] The preset quality score threshold is a preset image quality score threshold, which is used to determine whether the image quality of the sample image of the selected object meets the quality requirements.

[0045] In this embodiment, for each sample image of the selected object, a preset quality score threshold can be used to determine whether a sample image of the selected object can be used as a sample image of the target object. Specifically, for a sample image of the selected object, if the image quality score of the sample image of the selected object is greater than or equal to the preset quality score threshold, it can be determined as the corresponding sample image of the target object. Otherwise, the sample image of the selected object is discarded.

[0046] The technical solution of the embodiment of the present invention determines the prompt sentence to be applied through at least one first attribute dimension corresponding to the first candidate vocabulary, at least one second attribute dimension corresponding to the second candidate vocabulary and the sentence logic test condition; wherein the first attribute dimension is a mandatory attribute dimension when generating the prompt sentence to be applied, and the second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied, and then, based on the prompt sentence to be applied and the pre-trained object sample image generation model, at least one candidate object sample image is generated, thereby performing image quality evaluation on at least one candidate object sample image, obtaining an image quality score corresponding to at least one candidate object sample image, and determining the target object sample image based on the image quality score. The technical solution provided by this embodiment, under the constraint of the sentence logic test condition, randomly combines the vocabulary in the mandatory attribute dimension and the optional attribute dimension to automatically generate the prompt sentence, thereby improving the generation efficiency of the prompt sentence, and improving the richness, complexity and diversity of the prompt sentence, thereby improving the generation efficiency and image quality of the object sample image, and in addition, the image quality evaluation of the generated object sample image is performed through the image quality score mechanism to obtain the object sample image with qualified image quality, further improving the image quality of the object sample image.

[0047] Embodiment 2

[0048] Figure 2 This is a flow chart of a method for generating an object sample image provided by Embodiment 2 of the present invention. Based on the above embodiment, S110 and S130 are further refined. The specific implementation methods can refer to the detailed description of the embodiments of the present disclosure. The technical features that are the same or similar to the above embodiment are not repeated here.

[0049] like Figure 2 As shown, the object sample image generation method includes:

[0050] S210 . For each first attribute dimension, randomly select a first target word from all first candidate words in the first attribute dimension.

[0051] The first target vocabulary refers to the vocabulary selected from the vocabulary of the first attribute dimension.

[0052] In this embodiment, a first candidate word can be randomly selected from each first candidate word of each first attribute dimension as the first target word. Based on this, the first target word corresponding to each first attribute dimension can be obtained.

[0053] S220. Determine at least one target second attribute dimension from at least one second attribute dimension according to the probability value corresponding to each second attribute dimension, and randomly select a second target word from each target second attribute dimension.

[0054] The second target vocabulary refers to the vocabulary selected from the vocabulary of the second attribute dimension.

[0055] In this embodiment, different probability values ​​can be preconfigured for each second attribute dimension, so in each cycle, one or more target second attribute dimensions are selected from all second attribute dimensions according to the probability values ​​corresponding to each second attribute dimension. Further, a second candidate word can be randomly selected from each second candidate word of each target second attribute dimension as the second target word, based on which the second target word corresponding to each target second attribute dimension can be obtained.

[0056] S230: Determine a prompt sentence to be applied based on the first target vocabulary, the second target vocabulary and the sentence logic verification condition.

[0057] In this embodiment, the first target vocabulary and the second target vocabulary are first combined to construct at least one prompt sentence to be screened. Then, the prompt sentence to be screened is logically checked according to the sentence logic check condition, and when the check passes, the prompt sentence to be applied is obtained.

[0058] In this embodiment, a prompt sentence to be applied that meets the description requirements is generated from a multidimensional attribute vocabulary by combining rule-driven and random mechanisms. The first attribute dimension is the core component when generating the prompt sentence to be applied; the second attribute dimension (such as emotion, background, special mark, etc.) is a random option to enhance the diversity of the prompt sentence to be applied. Each time a prompt sentence to be applied is generated, the required attributes are selected according to the rules, and some random attributes are randomly selected for combination to ensure that the prompt sentence to be applied has high complexity and diversity on the basis of logical consistency.

[0059] Specifically, based on the first target vocabulary, the second target vocabulary and the sentence logic verification condition, the specific implementation method of determining the prompt sentence to be applied may include the following steps:

[0060] S1. Combining and assembling each first target vocabulary and each second target vocabulary based on a sentence generation model to construct at least one prompt sentence to be screened.

[0061] In this embodiment, the sentence generation model can be pre-trained. After obtaining each first target vocabulary and at least one second target vocabulary, these target vocabulary can be input into the sentence generation model. The sentence generation model can obtain more related vocabulary similar to the target vocabulary by analyzing and processing these target vocabulary, and thus obtain one or more prompt sentences to be screened by combining and assembling these target vocabulary and / or related vocabulary.

[0062] S2. For each prompt statement to be screened, a logic check is performed on the prompt statement to be screened based on the statement logic check condition to determine the logic consistency result.

[0063] The logical consistency result is used to indicate whether a prompt statement to be screened conforms to normal logic. The logical consistency result includes a logical consistency result and a logical inconsistency result.

[0064] In this embodiment, each prompt sentence to be screened may be subjected to a logic check respectively according to a preset sentence logic check condition, thereby determining a logic consistency result corresponding to each prompt sentence to be screened.

[0065] S3. Perform a grammatical compliance check on the prompt sentence to be screened based on the grammatical verification model to determine the grammatical compliance result.

[0066] The grammatical compliance result is used to indicate whether a prompt sentence to be screened conforms to a normal grammatical structure. The grammatical compliance result includes a grammatical compliance result and a grammatical non-compliance result.

[0067] In this embodiment, the grammar checking model can be pre-trained, and each prompt sentence to be screened can be input into the grammar checking model. The grammar checking model performs grammar compliance check on these prompt sentences to be screened, and a grammar compliance result corresponding to each prompt sentence to be screened can be obtained.

[0068] S4. Based on the logical consistency results and grammatical compliance results corresponding to the prompt sentences to be screened, determine the prompt sentences to be applied from the prompt sentences to be screened.

[0069] In this embodiment, the processing process of each prompt statement to be screened is consistent, and any one of the prompt statements to be screened is used as an example for explanation. For any prompt statement to be screened, if the logical consistency result of the prompt statement to be screened is a logically consistent result, and the grammatical compliance result is a grammatically compliant result, it can be determined as a prompt statement to be applied. If the logical consistency result of the prompt statement to be screened is a logically inconsistent result, and / or the grammatical compliance result is a grammatically non-compliant result, the prompt statement to be screened is eliminated.

[0070] It should be noted that the final number of prompt statements to be applied can be 0, 1 or more. If the number of prompt statements to be applied is 0, S210-S230 are re-executed; if the number of prompt statements to be applied is 1, S240-S290 can be continued to be executed, and finally a target object sample image corresponding to the prompt statement to be applied is obtained; if there can be multiple prompt statements to be applied, S240-S290 can be repeatedly executed multiple times, and finally a target object sample image corresponding to each prompt statement to be applied is obtained.

[0071] S240: Generate at least one sample image of an object to be selected based on the prompt sentence to be applied and a pre-trained object sample image generation model.

[0072] S250 , performing aesthetic quality assessment processing on each sample image of the object to be selected based on the aesthetic quality assessment model to obtain an image aesthetic quality score corresponding to the sample image of the object to be selected.

[0073] The aesthetic quality assessment model is a pre-trained model used to assess the aesthetic quality of the sample images of the selected object. The image aesthetic quality score is used to characterize the beauty of the sample images of the selected object. For example, the higher the value of the image aesthetic quality score, the higher the image beauty, and the lower the value of the image aesthetic quality score, the lower the image beauty.

[0074] In this embodiment, the processing process of each sample image of the selected object is consistent, and any one of the sample images of the selected object is used as an example for description. The sample image of the selected object can be input into the aesthetic quality assessment model, and the aesthetic quality assessment model performs image aesthetic quality assessment processing on the sample image of the selected object, and outputs an image aesthetic quality score corresponding to the sample image of the selected object.

[0075] S260 , performing object detection processing on the sample image of the object to be selected based on the facial detection model to obtain a facial detection result corresponding to the sample image of the object to be selected.

[0076] The face detection model is a pre-trained model used to detect whether the sample image of the selected object contains the object's facial image content. The face detection result may include a face detection result and a face non-detection result.

[0077] In this embodiment, the sample image of the object to be selected may be input into the face detection model, and the face detection model may output a face detection result corresponding to the sample image of the object to be selected by performing object face detection processing on the sample image of the object to be selected.

[0078] S270: Determine the text and image correlation score between the sample image of the object to be selected and the prompt sentence to be applied.

[0079] Among them, the text and image correlation score is used to characterize the degree of correlation between the sample image of the object to be selected and the prompt sentence to be applied. For example, the higher the value of the correlation score, the higher the degree of correlation between the sample image of the object to be selected and the prompt sentence to be applied; the lower the value of the correlation score, the lower the degree of correlation between the sample image of the object to be selected and the prompt sentence to be applied.

[0080] In this embodiment, the text-image relevance score can be determined by a multimodal pre-training neural network model (Contrastive Language-Image Pre-training, CLIP model). Specifically, the sample image of the object to be selected and the prompt sentence to be applied can be input into the CLIP model, and the CLIP model can output the text-image relevance score.

[0081] S280: Determine the image quality score corresponding to each sample image of the object to be selected based on the image aesthetic quality score, the face detection result, and the text-image correlation score.

[0082] In this embodiment, for each sample image of the selected object, the corresponding image quality score can be determined through the comprehensive results of the image aesthetic quality score, the face detection result, and the text-image correlation score.

[0083] Specifically, based on the image aesthetic quality score, the face detection result, and the text-image correlation score, a specific implementation method for determining the image quality score corresponding to each sample image of the object to be selected may include: for each sample image of the object to be selected, if the face detection result is that no face result is detected, then the image quality score corresponding to the sample image of the object to be selected is the lowest quality score; if the face detection result is that a face result is detected, then based on the image aesthetic quality score and the text-image correlation score, determine the image quality score corresponding to the sample image of the object to be selected.

[0084] In this embodiment, for a sample image of a candidate object, if the face detection result of the sample image of the candidate object is that no face result is detected, such an image needs to be eliminated, and therefore, the image quality score corresponding to the sample image of the candidate object can be set to 0. In particular, the lower limit of the aesthetic quality score and the lower limit of the correlation score can be preset. If there is a sample image of the candidate object whose image aesthetic quality score is lower than the lower limit of the aesthetic quality score, and / or the text-image correlation score is lower than the lower limit of the correlation score, such image also needs to be eliminated. In addition to the above two situations, if the face detection result is that the face result is detected, the image aesthetic quality score exceeds the lower limit of the aesthetic quality score, and the text-image correlation score exceeds the lower limit of the correlation score, the weighted sum of the image aesthetic quality score and the text-image correlation score can be determined as the image quality score corresponding to the sample image of the candidate object.

[0085] In this embodiment, by designing a three-level quality assessment mechanism of aesthetic quality score, face detection and semantic consistency check, the generated images are automatically screened. This multi-level quality control method can not only eliminate low-quality data that does not meet the requirements, ensure that each image in the generated sample image contains a clear face that meets the semantic description, but also ensure the high quality and high relevance of the images in the data set. This targeted optimization significantly improves the practicality and consistency of the data set.

[0086] S290: Determine a target object sample image from each candidate object sample image based on the image quality score.

[0087] The technical solution of this embodiment, when determining the prompt sentence to be applied, for each first attribute dimension, randomly select a first target word from each first candidate word of the first attribute dimension; according to the probability value corresponding to each second attribute dimension, determine at least one target second attribute dimension from each second attribute dimension, and randomly select a second target word from each target second attribute dimension; based on the first target word, the second target word and the sentence logic test condition, determine the prompt sentence to be applied. In the technical solution provided by this embodiment, the first attribute dimension is the core component when generating the prompt sentence to be applied; the second attribute dimension is a random option to enhance the diversity of the prompt sentence to be applied. Each time the prompt sentence to be applied is generated, the required attributes are selected according to the rules, and some random attributes are randomly selected for combination to ensure that the prompt sentence to be applied has high complexity and diversity on the basis of logical consistency. By combining rule-driven and random mechanisms, prompt sentences to be applied that meet the description requirements are generated from multidimensional attribute words. In addition, by designing a three-level quality assessment mechanism of aesthetic quality score, face detection and semantic consistency check, the generated images are automatically screened. This multi-level quality control method can not only eliminate low-quality data that does not meet the requirements, ensuring that each image in the generated sample image contains a clear face that meets the semantic description, but also ensure the high quality and high relevance of the images in the dataset. This targeted optimization significantly improves the practicality and consistency of the dataset.

[0088] Embodiment 3

[0089] Figure 3 This is a flow chart of a method for generating an object sample image provided by the third embodiment of the present invention. On the basis of the above-mentioned embodiment, when the cumulative number of sample images of the target object exceeds the sample image number threshold, an object image sample data set is obtained, and then, the object image processing model is trained based on the object image sample data set. The specific implementation method can refer to the detailed description of the embodiment of the present disclosure. Among them, the technical features that are the same or similar to the above-mentioned embodiments are not repeated here.

[0090] like Figure 3 As shown, the object sample image generation method includes:

[0091] S310: Determine a prompt sentence to be applied based on a first candidate word corresponding to at least one first attribute dimension, a second candidate word corresponding to at least one second attribute dimension, and a sentence logic check condition.

[0092] S320: Generate at least one sample image of an object to be selected based on the prompt sentence to be applied and a pre-trained object sample image generation model.

[0093] S330: Perform image quality assessment on at least one sample image of the object to be selected to obtain an image quality score corresponding to the at least one sample image of the object to be selected, and determine a sample image of the target object based on the image quality score.

[0094] S340: Determine whether the accumulated number of sample images of the target object exceeds a sample image number threshold.

[0095] The sample image quantity threshold is used to determine whether to stop the loop iteration of determining the target object sample image.

[0096] In this embodiment, a sample image quantity threshold can be set in advance. After obtaining the target object sample image in each cycle round, it can be determined whether the cumulative number of target object sample images exceeds the sample image quantity threshold. If so, S350-S360 can be executed; if not, S310-S340 can be repeatedly executed.

[0097] S350: If yes, obtain a constructed object image sample dataset based on each target object sample image.

[0098] In this embodiment, a set of all generated target object sample images may be determined as an object image sample data set.

[0099] S360: Perform model training on the object image processing model based on the object image sample dataset.

[0100] Among them, the object image processing model can be any model that needs to be trained based on an image containing an object.

[0101] In this embodiment, on the basis of obtaining the object image sample dataset, the object image sample dataset can be applied to any scenario where an object image processing model needs to be trained.

[0102] On the basis of the above embodiment, optionally, if the image quality scores of the target object sample images obtained in multiple cycles are less than the preset quality score threshold, prompt information is fed back to enable the user to adjust the sample image generation basis information based on the prompt information; after the sample image generation basis information is adjusted, the steps of determining the prompt statement to be applied, generating at least one sample image of the selected object, and determining the target object sample image based on the image quality score are repeated.

[0103] The sample image generation basis information includes: a first attribute dimension, a second attribute dimension, a first candidate word corresponding to the first attribute dimension, at least one second candidate word corresponding to the second attribute dimension, and at least one of preset logical test conditions. The prompt information refers to the prompt content that characterizes that the image quality of the sample image of the target object is unreasonable.

[0104] In this embodiment, if the sample images of the selected object obtained by multiple cycles are all less than the preset quality score threshold, it indicates that the information based on which the sample images of the selected object are generated, that is, the sample image generation basis information, needs to be adjusted. At this time, prompt information can be fed back to the user. Therefore, when the user receives the prompt information, he can adjust at least one sample image generation basis information according to user experience. After the sample image generation basis information adjustment process is completed, the steps S310-S340 are repeated.

[0105] The technical solution of this embodiment obtains an object image sample data set when the cumulative number of target object sample images exceeds a sample image number threshold, and then performs model training on the object image processing model based on the object image sample data set. Since the image quality in the object image sample data set is improved, the performance of the object image processing model is improved.

[0106] Embodiment 4

[0107] Figure 4 A schematic diagram of the structure of an object sample image generating device provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the device includes: a prompt statement determination module 410, a sample image generation module 420 and a sample image screening module 430.

[0108] The prompt sentence determination module 410 is used to determine the prompt sentence to be applied based on the first candidate word corresponding to at least one first attribute dimension, the second candidate word corresponding to at least one second attribute dimension, and the sentence logic test condition; wherein the first attribute dimension is a required attribute dimension when generating the prompt sentence to be applied, and the second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied;

[0109] A sample image generation module 420, configured to generate at least one sample image of a to-be-selected object based on the prompt sentence to be applied and a pre-trained object sample image generation model;

[0110] The sample image screening module 430 is used to perform image quality assessment on the at least one sample image of the candidate object to obtain an image quality score corresponding to the at least one sample image of the candidate object, so as to determine the target object sample image based on the image quality score.

[0111] Based on the above embodiment, optionally, the prompt statement determination module 410 includes:

[0112] a mandatory attribute word determination unit, configured to randomly select a first target word from all the first candidate words in the first attribute dimension for each of the first attribute dimensions;

[0113] a random attribute word determination unit, configured to determine at least one target second attribute dimension from the at least one second attribute dimension according to the probability value corresponding to each of the second attribute dimensions, so as to randomly select a second target word from each of the target second attribute dimensions;

[0114] The prompt sentence determining unit is used to determine the prompt sentence to be applied based on the first target vocabulary, the second target vocabulary and the sentence logic verification condition.

[0115] Based on the above embodiment, optionally, the prompt statement determining unit includes:

[0116] A to-be-screened sentence determination subunit is used to combine and assemble each of the first target words and each of the second target words based on a sentence generation model to construct at least one to-be-screened prompt sentence;

[0117] A logic result determination subunit is used to perform a logic check on each of the prompt statements to be screened based on the statement logic check condition to determine a logic consistency result;

[0118] A compliance result determination subunit, used for performing a grammar compliance check on the prompt sentence to be screened based on a grammar checking model to determine a grammar compliance result;

[0119] The prompt statement determining subunit is used to determine the prompt statement to be applied from each prompt statement to be screened based on the logical consistency result and the grammatical compliance result corresponding to each prompt statement to be screened.

[0120] Based on the above embodiment, optionally, the sample image screening module 430 includes:

[0121] an aesthetic quality score determination unit, configured to perform aesthetic quality assessment processing on each of the sample images of the selected object based on the aesthetic quality assessment model to obtain an image aesthetic quality score corresponding to the sample image of the selected object;

[0122] A face detection unit, configured to perform object detection processing on the sample image of the object to be selected based on a face detection model to obtain a face detection result corresponding to the sample image of the object to be selected;

[0123] A correlation score determination unit, used to determine the text and image correlation score between the sample image of the object to be selected and the prompt sentence to be applied;

[0124] A quality score determination unit is used to determine the image quality score corresponding to each of the sample images of the object to be selected based on the image aesthetic quality score, the face detection result and the text-image correlation score.

[0125] On the basis of the above embodiment, optionally, a quality score determination unit is specifically used to, for each sample image of the object to be selected, if the facial detection result is that no facial result is detected, then the image quality score corresponding to the sample image of the object to be selected is the lowest quality score; if the facial detection result is that a facial result is detected, then based on the image aesthetic quality score and the text-image correlation score, determine the image quality score corresponding to the sample image of the object to be selected.

[0126] Based on the above embodiment, optionally, the sample image screening module 430 further includes: a target image determination unit;

[0127] The target image determination unit is used to determine the sample image of the candidate object whose image quality score is greater than or equal to a preset quality score threshold as the target object sample image.

[0128] Based on the above embodiment, optionally, the object sample image generating device further includes: a sample data set application module;

[0129] The sample data set application module is used to repeatedly execute the steps of determining the prompt sentence to be applied based on the first candidate word corresponding to at least one first attribute dimension, the second candidate word corresponding to at least one second attribute dimension and the sentence logic verification condition, generating at least one candidate object sample image and determining the target object sample image based on the image quality score, until the cumulative number of the target object sample images exceeds the sample image number threshold, and constructing an object image sample data set based on each of the target object sample images; and performing model training on the object image processing model based on the object image sample data set.

[0130] The sample data set application module is also used to feedback prompt information if the image quality scores of the target object sample images obtained in multiple cycles are all less than a preset quality score threshold, so that the user can adjust the sample image generation basis information based on the prompt information; wherein the sample image generation basis information includes: the first attribute dimension, the second attribute dimension, the first candidate vocabulary corresponding to the first attribute dimension, the second candidate vocabulary corresponding to the at least one second attribute dimension, and at least one of the preset logical test conditions; after the sample image generation basis information is adjusted, the steps of determining the prompt statement to be applied, generating at least one candidate object sample image, and determining the target object sample image based on the image quality score are repeated.

[0131] The technical solution of the embodiment of the present invention determines the prompt sentence to be applied based on a first candidate word corresponding to at least one first attribute dimension, a second candidate word corresponding to at least one second attribute dimension, and a sentence logic verification condition; wherein the first attribute dimension is a required attribute dimension when generating the prompt sentence to be applied, and the second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied, and then, based on the prompt sentence to be applied and a pre-trained object sample image generation model, at least one candidate object sample image is generated, so that the image quality of the at least one candidate object sample image is evaluated, and the image quality score corresponding to the at least one candidate object sample image is obtained, so as to determine the target object sample image based on the image quality score. The technical solution provided in this embodiment, under the constraint of sentence logic verification conditions, randomly combines the vocabulary in the required attribute dimension and the optional attribute dimension to automatically generate prompt sentences, thereby improving the generation efficiency of prompt sentences, and enhancing the richness, complexity and diversity of prompt sentences, thereby improving the generation efficiency and image quality of object sample images. In addition, the image quality of the object sample images is evaluated through an image quality scoring mechanism to obtain object sample images that meet the image quality standards, thereby further improving the image quality of the object sample images.

[0132] The object sample image generating device provided in the embodiment of the present invention can execute the object sample image generating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0133] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0134] Embodiment 5

[0135] Figure 5 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 5 A block diagram of an exemplary electronic device 50 suitable for implementing exemplary embodiments of the present invention is shown. Figure 5 The electronic device 50 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0136] like Figure 5As shown, the electronic device 50 is in the form of a general computing device. The components of the electronic device 50 may include but are not limited to: one or more processors or processing units 501, a system memory 502, and a bus 503 connecting different system components (including the system memory 502 and the processing unit 501).

[0137] Bus 503 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0138] The electronic device 50 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 50, including volatile and non-volatile media, removable and non-removable media.

[0139] The system memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 504 and / or cache memory 505. The electronic device 50 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 506 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 503 via one or more data medium interfaces. The memory 502 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the embodiments of the present invention.

[0140] A program / utility 508 having a set (at least one) of program modules 507 may be stored, for example, in the memory 502, such program modules 507 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 507 generally perform the functions and / or methods of the embodiments described herein.

[0141] The electronic device 50 may also communicate with one or more external devices 509 (e.g., keyboards, pointing devices, displays 510, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 50, and / or communicate with any device that enables the electronic device 50 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 511. Furthermore, the electronic device 50 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 512. As shown, the network adapter 512 communicates with other modules of the electronic device 50 through the bus 503. It should be understood that although Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0142] The processing unit 501 executes various functional applications and page processing by running the programs stored in the system memory 502, for example, implementing the object sample image generation method provided in the embodiment of the present invention.

[0143] Embodiment 6

[0144] An embodiment of the present invention further provides a storage medium containing computer executable instructions, wherein the computer executable instructions are used to execute a method for generating an object sample image when executed by a computer processor, the method comprising:

[0145] Determine the prompt sentence to be applied based on at least one first candidate word corresponding to the first attribute dimension, at least one second candidate word corresponding to the second attribute dimension, and a sentence logic test condition; wherein the first attribute dimension is a required attribute dimension when generating the prompt sentence to be applied, and the second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied;

[0146] Based on the prompt sentence to be applied and a pre-trained object sample image generation model, generating at least one sample image of the object to be selected;

[0147] An image quality assessment is performed on the at least one sample image of the object to be selected to obtain an image quality score corresponding to the at least one sample image of the object to be selected, so as to determine the target object sample image based on the image quality score.

[0148] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0149] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0150] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0151] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0152] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for generating an object sample image, characterized in that: include: Determine the prompt sentence to be applied based on at least one first candidate word corresponding to the first attribute dimension, at least one second candidate word corresponding to the second attribute dimension, and a sentence logic test condition; wherein the first attribute dimension is a required attribute dimension when generating the prompt sentence to be applied, and the second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied; Generate at least one sample image of an object to be selected based on the prompt sentence to be applied and a pre-trained object sample image generation model; An image quality assessment is performed on the at least one sample image of the object to be selected to obtain an image quality score corresponding to the at least one sample image of the object to be selected, so as to determine the target object sample image based on the image quality score.

2. The method according to claim 1, characterized in that The step of determining the prompt sentence to be applied based on the first candidate word corresponding to at least one first attribute dimension, the second candidate word corresponding to at least one second attribute dimension, and the sentence logic test condition comprises: For each of the first attribute dimensions, randomly selecting a first target word from all of the first candidate words in the first attribute dimension; Determine at least one target second attribute dimension from the at least one second attribute dimension according to the probability value corresponding to each of the second attribute dimensions, and randomly select a second target vocabulary from each of the target second attribute dimensions; Based on the first target vocabulary, the second target vocabulary and the sentence logic verification condition, a prompt sentence to be applied is determined.

3. The method according to claim 2, characterized in that The step of determining the prompt sentence to be applied based on the first target vocabulary, the second target vocabulary and the sentence logic verification condition includes: Based on the sentence generation model, each of the first target words and each of the second target words are combined and assembled to construct at least one prompt sentence to be screened; For each of the prompt statements to be screened, a logic check is performed on the prompt statement to be screened based on the statement logic check condition to determine a logic consistency result; Performing a grammatical compliance check on the prompt sentence to be screened based on a grammatical verification model to determine a grammatical compliance result; Based on the logical consistency result and the grammatical compliance result corresponding to each of the prompt sentences to be screened, a prompt sentence to be applied is determined from each of the prompt sentences to be screened.

4. The method according to claim 1, characterized in that: The performing image quality assessment on the at least one sample image of the object to be selected to obtain an image quality score corresponding to the at least one sample image of the object to be selected includes: For each of the sample images of the selected object, performing aesthetic quality assessment processing on the sample image of the selected object based on the aesthetic quality assessment model to obtain an image aesthetic quality score corresponding to the sample image of the selected object; Performing object detection processing on the sample image of the object to be selected based on the face detection model to obtain a face detection result corresponding to the sample image of the object to be selected; Determining a text and image correlation score between the sample image of the object to be selected and the prompt sentence to be applied; An image quality score corresponding to each of the sample images of the object to be selected is determined based on the image aesthetic quality score, the face detection result, and the text-image relevance score.

5. The method according to claim 4, characterized in that The step of determining the image quality score corresponding to each of the sample images of the object to be selected based on the image aesthetic quality score, the face detection result, and the text-image correlation score comprises: For each of the sample images of the object to be selected, if the face detection result is that no face is detected, the image quality score corresponding to the sample image of the object to be selected is the lowest quality score; If the face detection result is a face detection result, an image quality score corresponding to the sample image of the object to be selected is determined based on the image aesthetic quality score and the text-image relevance score.

6. The method according to claim 1, characterized in that The determining of a target object sample image based on the image quality score comprises: The sample image of the candidate object whose image quality score is greater than or equal to a preset quality score threshold is determined as the sample image of the target object.

7. The method according to any one of claim 1, characterized in that: The method further comprises: Repeat the steps of determining the prompt sentence to be applied based on the first candidate word corresponding to at least one first attribute dimension, the second candidate word corresponding to at least one second attribute dimension, and the sentence logic verification condition, generating at least one candidate object sample image, and determining the target object sample image based on the image quality score, until the cumulative number of the target object sample images exceeds the sample image number threshold, and obtaining a constructed object image sample data set based on each of the target object sample images; Model training is performed on the object image processing model based on the object image sample data set.

8. The method according to claim 7, characterized in that The method further comprises: If the image quality scores of the sample images of the target object obtained by multiple cycles are all less than the preset quality score threshold, prompt information is fed back to enable the user to adjust the sample image generation basis information based on the prompt information; wherein the sample image generation basis information includes: the first attribute dimension, the second attribute dimension, the first candidate word corresponding to the first attribute dimension, the second candidate word corresponding to the at least one second attribute dimension, and at least one of the preset logic test conditions; After the sample image generation basis information is adjusted, the steps of determining the prompt statement to be applied, generating at least one sample image of a candidate object, and determining the target object sample image based on the image quality score are repeated.

9. An object sample image generating device, characterized in that: include: A prompt sentence determination module, used to determine the prompt sentence to be applied based on a first candidate word corresponding to at least one first attribute dimension, a second candidate word corresponding to at least one second attribute dimension, and a sentence logic test condition; wherein the first attribute dimension is a mandatory attribute dimension when generating the prompt sentence to be applied, and the second attribute dimension is an optional attribute dimension when generating the prompt sentence to be applied; A sample image generation module, used to generate at least one sample image of a to-be-selected object based on the prompt sentence to be applied and a pre-trained object sample image generation model; The sample image screening module is used to perform image quality assessment on the at least one sample image of the candidate object to obtain an image quality score corresponding to the at least one sample image of the candidate object, so as to determine the target object sample image based on the image quality score.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the object sample image generation method as described in any one of claims 1 to 8.

Images