Evolutionary algorithm-based text graph large model cue word optimization method and system

Abstract

The invention discloses an evolutionary algorithm-based text graph large model cue word optimization method and system, and the method comprises the steps: obtaining a data set which comprises a training set and a test set, for each category in the training set, generating a cue word corresponding to the category by using a ChatGPT-4 model, and for the cue word corresponding to each category, generating a cue word corresponding to each category by using a ChatGPT-4 model; multiple times of random sampling are carried out on the cue words to obtain sampling cue words, the multiple obtained sampling cue words are input into the text graph large model to obtain expanded pictures of the categories corresponding to all the sampling cue words, and all the obtained expanded pictures form an expanded picture set corresponding to the category; and processing the extended picture set corresponding to each category by using an evolutionary algorithm to obtain the optimized cue word of the category corresponding to the extended picture set corresponding to the category. According to the method and the device, the technical problems of long time consumption and relatively high adjustment difficulty of the existing method for manually adjusting the cue word based on artificial experience can be solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and more specifically, relates to a method and system for optimizing prompt words of a large model of a text graph based on an evolutionary algorithm. Background Art

[0002] The large model of text-generated images generates images that match the text description based on the text prompt words provided by the user. It has been widely used in the fields of art creation, advertising design, game development, etc., and also shows great potential in expanding small sample size data sets. Through the pre-trained large model of text-generated images, a large number of images can be generated using prompt words to expand the image data set with a small sample size, thereby effectively alleviating the problem of insufficient data samples in the field of machine learning.

[0003] The existing methods for generating prompt words for large-scale Wenshengtu models are mainly divided into three categories: the first method is a method of manually adjusting prompt words based on artificial experience. This method completely relies on the user's subjective judgment and experience accumulation. The user gradually adjusts the prompt word content by observing the image effect generated by the large-scale Wenshengtu model; the second method is a prompt word generation method based on a fixed template. Through a predefined structured template (such as "[subject] + [style] + [environment] + [detail modification]"), the user only needs to fill in the specific content to generate the prompt word; the third method is a prompt word generation method based on a knowledge base (such as DiffusionDB). By querying the existing high-quality prompt words and their corresponding generation effects in the knowledge base, these prompt words are directly reused or adjusted to meet the needs.

[0004] However, the above methods all have some drawbacks that cannot be ignored:

[0005] First, the method of manually adjusting prompt words based on human experience lacks systematic guidance, and users often need to try and adjust repeatedly to generate images that meet expectations. As a result, manual adjustment of prompt words is time-consuming and difficult.

[0006] Second, the prompt word generation method based on fixed templates may lack diversity in the pictures generated by prompt words due to the fixed nature of the templates;

[0007] Third, the prompt word generation method based on the knowledge base cannot generate the expected pictures due to the limited coverage of the knowledge base and insufficient update frequency.

[0008] Fourth, the prompt words generated by the above three existing methods mainly focus on optimizing single generation results and lack targeted design for data diversity, distribution balance and task generalization ability, which leads to poor effect of prompt words on extended dataset tasks. Summary of the invention

[0009] In view of the above defects or improvement needs of the prior art, the present invention provides a method and system for optimizing prompt words of a large model of a cultural graph based on an evolutionary algorithm, the purpose of which is to solve the technical problems of the existing method of manually adjusting prompt words based on human experience, which is time-consuming and difficult to adjust; and the technical problem that the existing prompt word generation method based on a fixed template lacks diversity in the pictures generated by the prompt words due to the fixedness of the template; and the technical problem that the existing prompt word generation method based on a knowledge base cannot generate the expected pictures due to the limited coverage of the knowledge base and insufficient updating frequency; and the technical problem that the prompt words generated by the three existing methods are poor in the effect of generating prompt words on the extended data set task because they mainly focus on optimizing the single generation result and ignore the performance on the extended data set task.

[0010] To achieve the above object, according to one aspect of the present invention, a method and system for optimizing prompt words of a large model of a text graph based on an evolutionary algorithm is provided, comprising the following steps:

[0011] (1) Obtain a data set, which includes a training set and a test set.

[0012] (2) For each category in the training set obtained in step (1), use the ChatGPT-4 model to generate the corresponding prompt word for the category.

[0013] (3) For each prompt word corresponding to the category obtained in step (2), sampled prompt words are obtained by random sampling from the prompt word multiple times, and the obtained multiple sampled prompt words are input into the Wenshengtu large model to obtain extended pictures of the categories corresponding to all the sampled prompt words, and all the obtained extended pictures constitute an extended picture set corresponding to the category.

[0014] (4) Using an evolutionary algorithm to process the extended picture set corresponding to each category obtained in step (3) to obtain an optimized prompt word for the category corresponding to the extended picture set corresponding to the category.

[0015] (5) For the optimized prompt words of the category corresponding to the extended picture set corresponding to each category in the training set obtained in step (4), sampled prompt words are randomly sampled from the prompt words multiple times, and the obtained multiple sampled prompt words are input into the Wenshengtu large model to obtain extended pictures of the category corresponding to all the sampled prompt words, and all the obtained extended pictures constitute the extended picture set corresponding to the category.

[0016] (6) Using an evolutionary algorithm to process the extended picture set corresponding to each category in the training set obtained in step (5) to obtain an optimized prompt word corresponding to the category of the extended picture set corresponding to the category.

[0017] (7) For each category in the training set, the optimized prompt word corresponding to the category of the extended picture set corresponding to the category obtained in step (6) is used as the final prompt word corresponding to the category.

[0018] Preferably, step (4) specifically comprises the following steps:

[0019] (4-1) Set counter i=1;

[0020] (4-2) Determine whether i is greater than the number of categories in the data set obtained in step (1) (10 in this example). If so, the process ends; otherwise, proceed to step (4-3);

[0021] (4-3) Set counter j = 1;

[0022] (4-4) Determine whether j is greater than a preset number of iterations (the value range is 8 to 10, preferably 10). If so, proceed to step (4-12); otherwise, proceed to step (4-5);

[0023] (4-5) Obtain the prompt word corresponding to the i-th category in the training set obtained in step (2) as the current prompt word;

[0024] (4-6) obtaining the unexpected picture set and the expected picture set corresponding to the i-th category in the training set from the extended picture set corresponding to the i-th category in the training set obtained in step (3);

[0025] (4-7) using an evolutionary algorithm to process the current prompt word obtained in step (4-5) to obtain a new prompt word;

[0026] (4-8) using the unexpected picture set corresponding to the i-th category in the training set obtained in step (4-6) to process the new prompt word obtained in step (4-7) to obtain a score for the new prompt word;

[0027] (4-9) using the unexpected picture set obtained in step (4-6) to process the current prompt word obtained in step (4-5) to obtain a score for the current prompt word;

[0028] (4-10) The score of the current prompt word obtained in step (4-9) and the score of the new prompt word obtained in step (4-8) are used as the prompt word with a higher score as the current prompt word in step (4-5), and the current prompt word is used as the optimized prompt word of the i-th category in the training set;

[0029] (4-11) Set j=j+1 and return to step (4-4);

[0030] (4-12) Set i=i+1 and return to step (4-2).

[0031] Preferably, step (4-6) is performed by checking each picture in the extended picture set corresponding to the i-th category in the training set, if there is a subject other than the cat or dog category in the picture, then the picture is an unexpected picture, otherwise the picture is an expected picture;

[0032] Step (4-7) is specifically as follows: first, use the large language model ChatGPT-4 to perform a self-crossover operation on the current prompt word obtained in step (4-5) to obtain the prompt word after cross processing; then, use the large language model ChatGPT-4 to perform a mutation operation on the prompt word after cross processing to obtain a new prompt word.

[0033] Preferably, steps (4-8) are specifically:

[0034] The new prompt word obtained in step (4-7) is input into the Wenshengtu large model to generate an extended picture of the category corresponding to the prompt word, and the process is repeated multiple times. The multiple extended pictures obtained constitute the extended picture set corresponding to the i-th category in the training set.

[0035] Then, calculate the similarity between the unexpected picture set obtained in step (4-6) and the extended picture set corresponding to the i-th category in the training set:

[0036]

[0037] Where m∈[1, the total number of extended images in the extended image set corresponding to the i-th category in the training set N g ], n∈[1, the total number of pictures in the unexpected picture set N d ], Represents the extended image set I corresponding to the i-th category in the training set g The mth picture in Represents the unexpected picture set I obtained in step (4-6) c The nth picture in g represents the number of extended images in the extended image set corresponding to the i-th category in the training set, N d Indicates the number of unexpected images in the unexpected image set. Represents the extended image set I corresponding to the i-th category in the training set g The mth picture in Unexpected pictures collection I c The nth picture in , and have:

[0038]

[0039] in and They represent the feature vector of the mth picture in the extended picture set corresponding to the i-th category in the training set and the feature vector of the nth picture in the unexpected picture set, respectively. and Respectively represent the feature vector of the mth picture in the extended picture set corresponding to the i-th category in the training set The Euclidean norm of and the feature vector of the nth picture in the unexpected picture set The Euclidean norm of the training set is the feature vector of the mth picture in the extended picture set corresponding to the i-th category is extracted by contrastive language-image pre-training CLIP and is equal to:

[0040]

[0041] Where Embedding represents the visual encoder of CLIP.

[0042] Then, the similarity between the expected image set obtained in step (4-6) and the extended image set corresponding to the i-th category in the training set is calculated:

[0043]

[0044] Where: N c Represents the total number of images of the i-th category in the training set.

[0045] Then, the confidence of the expanded image set corresponding to the i-th category in the training set is calculated:

[0046]

[0047] in Represents the extended image set I corresponding to the i-th category in the training set g The confidence of the mth image in is equal to:

[0048]

[0049] Where p represents the extended image set I corresponding to the i-th category in the training set by the CLIP model. g The mth picture in The predicted probability distribution of is equal to:

[0050] p=[p1,p2,...,p T ]

[0051] Where T is the number of categories of images in the dataset (10 in this example), p k is the extended image set I corresponding to the i-th category in the ResNet50 model prediction training set gThe mth picture in The probability of belonging to the kth class, where k∈[1,T].

[0052] Finally, the similarity between the unexpected picture set and the extended picture set corresponding to the i-th category in the training set, the similarity between the expected picture set and the extended picture set corresponding to the i-th category in the training set, and the confidence of the extended picture set corresponding to the i-th category in the training set are added together to obtain the score of the new prompt word, that is:

[0053]

[0054] Preferably, step (6) specifically includes the following sub-steps:

[0055] (6-1) Set counter f=1;

[0056] (6-2) Determine whether f is greater than the number of categories in the data set obtained in step (1). If so, the process ends; otherwise, proceed to step (6-3);

[0057] (6-3) Set counter h = 1;

[0058] (6-4) Determine whether h is greater than the preset number of iterations. If so, proceed to step (6-12); otherwise, proceed to step (6-5);

[0059] (6-5) obtaining the optimized prompt word of the category corresponding to the extended picture set corresponding to the f-th category in the training set obtained in step (4) as the current prompt word;

[0060] (6-6) obtaining a set of images lacking diversity from the extended set of images corresponding to the f-th category in the training set obtained in step (5);

[0061] (6-7) using an evolutionary algorithm to process the current prompt word obtained in step (6-5) to obtain a new prompt word;

[0062] (6-8) using the image set lacking diversity obtained in step (6-6) to process the new prompt word obtained in step (6-7) to obtain a score for the new prompt word;

[0063] (6-9) using the image set lacking diversity obtained in step (6-6) to process the current prompt word obtained in step (6-5) to obtain a score for the current prompt word;

[0064] (6-10) The score of the current prompt word obtained in step (6-9) and the score of the new prompt word obtained in step (6-8) are used as the prompt word with a higher score as the current prompt word obtained in step (6-5), and the current prompt word is used as the optimized prompt word for the fth category in the training set;

[0065] (6-11) Set h=h+1 and return to step (6-4);

[0066] (6-11) Set f=f+1 and return to step (6-2).

[0067] Preferably, step (6-6) is performed by checking each picture in the extended picture set corresponding to the f-th category in the training set. If the similarity between the picture and the picture corresponding to the f-th category in the training set obtained in step (1) is high, that is, the similarity between the picture and the picture corresponding to the f-th category in the training set reaches a preset threshold, it means that the picture has not expanded new information and the picture is a picture lacking diversity.

[0068] Preferably, steps (6-8) are specifically:

[0069] First, the new prompt word obtained in step (6-7) is input into the Wenshengtu large model to generate an extended picture of the category corresponding to the prompt word, and the process is repeated multiple times. The multiple extended pictures obtained constitute the extended picture set corresponding to the fth category in the training set.

[0070] Then, calculate the diversity index S of the extended image set corresponding to the fth category in the training set div (I l ):

[0071]

[0072] Among them I l represents the extended image set corresponding to the f-th category in the training set, M is the total number of images in the extended image set corresponding to the f-th category in the training set, and v′ q is the feature vector of the qth picture in the extended picture set corresponding to the fth category in the training set (where q∈[1,M]), is the average feature vector of all images in the extended image set corresponding to the f-th category in the training set, and has:

[0073]

[0074] in Represents the distance between the feature vector of the qth picture in the extended picture set corresponding to the fth category in the training set and the average feature vector of all pictures in the extended picture set corresponding to the fth category in the training set:

[0075]

[0076] Then, the similarity between the image set lacking diversity and the expanded image set corresponding to the fth category in the training set is calculated:

[0077]

[0078] Where: N a Represents the total number of images in the image set that lacks diversity, r∈[1, the total number of extended images in the extended image set corresponding to the f-th category in the training set N l ], e∈[1, the total number of images in the lack of diversity image set N a ], Represents the extended image set I corresponding to the fth category in the training set l The rth picture in Represents the lack of diversity image set I obtained in step (6-6) s The e-th picture in .

[0079] Then, the confidence of the expanded image set corresponding to the fth category in the training set is calculated:

[0080]

[0081] Finally, the diversity index of the extended picture set corresponding to the f-th category in the training set, the similarity between the picture set lacking diversity and the extended picture set corresponding to the f-th category in the training set, and the confidence of the extended picture set corresponding to the f-th category in the training set are added together to obtain the score of the new prompt word, namely:

[0082]

[0083] According to another aspect of the present invention, a system for optimizing prompt words of a large model of a text graph based on an evolutionary algorithm is provided, comprising:

[0084] The first module is used to obtain a data set, which includes a training set and a test set.

[0085] The second module is used to generate the corresponding prompt word for each category in the training set obtained in the first module using the ChatGPT-4 model.

[0086] The third module is used to randomly sample the prompt words corresponding to each category obtained in the second module multiple times to obtain sampled prompt words, and input the obtained multiple sampled prompt words into the Wenshengtu large model to obtain extended pictures of the categories corresponding to all the sampled prompt words, and all the obtained extended pictures constitute an extended picture set corresponding to the category.

[0087] The fourth module is used to process the extended picture set corresponding to each category obtained by the third module using an evolutionary algorithm to obtain an optimized prompt word for the category corresponding to the extended picture set corresponding to the category.

[0088] The fifth module is used to randomly sample the optimized prompt words for the category corresponding to the extended picture set corresponding to each category in the training set obtained in the fourth module multiple times to obtain sampled prompt words, and input the obtained multiple sampled prompt words into the Wenshengtu large model to obtain the extended pictures of the category corresponding to all the sampled prompt words, and all the obtained extended pictures constitute the extended picture set corresponding to the category.

[0089] The sixth module is used to process the extended picture set corresponding to each category in the training set obtained in the fifth module by using an evolutionary algorithm to obtain an optimized prompt word corresponding to the category of the extended picture set corresponding to the category.

[0090] The seventh module is used to use, for each category in the training set, the optimized prompt word corresponding to the category of the extended picture set corresponding to the category obtained in the sixth module as the final prompt word corresponding to the category.

[0091] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0092] (1) The present invention adopts step (2), which uses the ChatGPT-4 model to generate prompt words corresponding to each category, thereby automatically obtaining prompt words containing rich scene information and diversity, thereby solving the technical problems of the existing method of manually adjusting prompt words based on human experience, which is time-consuming and difficult to adjust;

[0093] (2) The present invention adopts steps (6-5) to (6-10), which updates the prompt word by using an evolutionary algorithm, and calculates the scores of the prompt word before and after the update, and measures the diversity of the set of pictures generated by the prompt word by the score, thereby obtaining prompt words that can generate more diverse pictures. Therefore, the technical problem that the existing prompt word generation method based on a fixed template lacks diversity in the pictures generated by the prompt word due to the fixed nature of the template can be solved;

[0094] (3) The present invention adopts steps (4-5) to (4-10), which uses an evolutionary algorithm to update the prompt word, and calculates the scores of the prompt words before and after the update, and uses the scores to measure whether the picture set generated by the prompt word meets expectations, thereby obtaining prompt words that can generate pictures that meet the user's expectations. Therefore, it can solve the technical problem that the existing prompt word generation method based on the knowledge base cannot generate the expected pictures due to the limited coverage of the knowledge base and insufficient update frequency;

[0095] (4) The present invention adopts steps (4-8) and (6-8), and the prompt word scores respectively measure whether the prompt word generated picture set meets expectations and its diversity. Therefore, the prompt words are optimized at the picture set level, rather than focusing only on the generation quality of a single picture. Therefore, the technical problem that the prompt words generated by the three existing prompt word generation methods have poor effects on the extended data set task can be solved;

[0096] (5) The method proposed in the present invention is compatible with any large language model and large text-graph model. The method can seamlessly integrate large language models of different architectures (such as ChatGPT series, LLaMA, etc.) and large text-graph models (such as StableDiffusion, DALL·E, etc.), give full play to the advantages of each model, and has high versatility and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 It is a schematic diagram of a method for optimizing prompt words of a large model of a cultural graph based on an evolutionary algorithm of the present invention;

[0098] Figure 2 It is a schematic diagram of a prompt word template and a sampling method of a prompt word optimization method and system of a large model of a text graph based on an evolutionary algorithm of the present invention;

[0099] Figure 3 It is a schematic diagram of a prompt word updating and selecting method of a prompt word optimization method of a large model of a cultural graph based on an evolutionary algorithm of the present invention;

[0100] Figure 4 It is a flow chart of the method for optimizing the prompt words of the large model of the text graph based on the evolutionary algorithm of the present invention. DETAILED DESCRIPTION

[0101] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0102] The basic idea of ​​the present invention is to use an evolutionary algorithm to cross and mutate the prompt words of the large model of the text graph to generate new prompt words, and then through the feedback of the labeling of the picture and the quality of the picture generated by the new prompt words, jointly constrain the optimization direction of the prompt words, so as to iteratively obtain the optimized prompt words, and then control the large model of the text graph to generate as diverse and user-expected pictures as possible, thereby expanding the small-scale data set and improving the model performance on downstream tasks such as image classification and object detection tasks.

[0103] The overall idea of ​​the present invention is:

[0104] First, for a small-scale dataset, we use the large language model ChatGPT-4 to generate prompt words, and use the prompt words to control the large text-based graph model to generate pictures, thus obtaining an extended picture set.

[0105] Second, for each category of the generated extended picture set, an unexpected picture set is selected, and the prompt words of the category corresponding to the unexpected picture set are optimized using an evolutionary algorithm, and the optimized new prompt words corresponding to the category are iteratively obtained. The newly obtained prompt words can reduce the generation of unexpected pictures;

[0106] Third, for each category of the generated extended picture set, a picture set lacking diversity is selected, and the prompt words corresponding to the category of the picture set lacking diversity are optimized using an evolutionary algorithm, and the optimized new prompt words corresponding to the category are iteratively obtained, and the newly obtained prompt words can increase the diversity of the generated pictures;

[0107] Fourth, you can use the optimized prompt words to generate pictures to expand small-scale data sets and improve model performance on downstream tasks such as image classification and object detection.

[0108] like Figure 1 and Figure 4 As shown, the present invention provides a method and system for optimizing prompt words of a large model of a text graph based on an evolutionary algorithm, comprising the following steps:

[0109] (1) Obtain a data set, which includes a training set and a test set.

[0110] Specifically, the data set in this step is the Stanford University cat and dog 37 classification data set, from which 10 categories are selected, including five categories of cats and five categories of dogs, 200 pictures in each category, and the training set and test set each include 1000 pictures.

[0111] (2) For each category in the training set obtained in step (1), use the ChatGPT-4 model to generate the corresponding prompt word for that category (i.e., 10 prompt words are generated for 10 categories).

[0112] For example, for the category of Bengal cat in the training set obtained in step (1), the prompt words generated by the ChatGPT-4 model are:

[0113] [a whimsical digital painting of,a playful cartoon image of][vivid,serene,high-contrast,colorful]Bengal cat[playing in a vibrant flower bed,lounging on a fluffy rug by a fireplace,climbing a tree in a sunny park,exploring a colorful backyard,chasing butterflies in a sunny meadow]

[0114] The advantage of this step is that the large language model ChatGPT-4 can be used to generate diverse prompt words containing rich scene information, and then generate information-rich pictures.

[0115] (3) For each category corresponding to the prompt word obtained in step (2), randomly sample the prompt word multiple times (100 times in this example) to obtain a sample prompt word (the sampling process is as follows Figure 2 As shown, each sampling obtains a sampling prompt word), and the obtained multiple sampling prompt words are input into the Wenshengtu large model to obtain the extended pictures of the categories corresponding to all the sampling prompt words, and all the obtained extended pictures constitute the extended picture set corresponding to the category.

[0116] (4) Using an evolutionary algorithm, the extended picture set corresponding to each category obtained in step (3) is processed to obtain an optimized prompt word for the category corresponding to the extended picture set corresponding to the category.

[0117] This step specifically includes the following steps:

[0118] (4-1) Set counter i=1;

[0119] (4-2) Determine whether i is greater than the number of categories in the data set obtained in step (1) (10 in this example). If so, the process ends; otherwise, proceed to step (4-3);

[0120] (4-3) Set counter j = 1;

[0121] (4-4) Determine whether j is greater than a preset number of iterations (the value range is 8 to 10, preferably 10). If so, proceed to step (4-12); otherwise, proceed to step (4-5);

[0122] (4-5) Obtain the prompt word corresponding to the i-th category in the training set obtained in step (2) as the current prompt word;

[0123] (4-6) obtaining the unexpected picture set and the expected picture set corresponding to the i-th category in the training set from the extended picture set corresponding to the i-th category in the training set obtained in step (3);

[0124] Specifically, this step is to check each picture in the extended picture set corresponding to the i-th category in the training set. If there are subjects other than cat and dog categories in the picture, such as tiger subjects, or some building subjects, which are not the cat and dog subjects involved in the present invention, then the picture is an unexpected picture, otherwise the picture is an expected picture.

[0125] (4-7) using an evolutionary algorithm to process the current prompt word obtained in step (4-5) to obtain a new prompt word;

[0126] Specifically, this step uses the large language model ChatGPT-4 and an evolutionary algorithm to process the current prompt word to generate a new prompt word.

[0127] This step (4-7) is specifically as follows: First, use the large language model ChatGPT-4 to perform a self-crossover operation on the current prompt word obtained in step (4-5) to obtain a cross-processed prompt word (which includes at least 15 words and is used as a new prompt word).

[0128] Then, the large language model ChatGPT-4 is used to perform mutation operations on the cross-processed prompt words, that is, by adding, deleting or modifying elements in the cross-processed prompt words (such as the prompt word "a cat is playing", which mutates to "a cute cartoon kitten running in the garden") to obtain new prompt words.

[0129] For example, the result of crossover and mutation processing on the prompt words corresponding to the category of Bengal cat is as follows:

[0130] [a whimsical digital painting of,a playful cartoon image of][vivid,bright,serene,high-contrast,colorful]Bengal cat[playing in a vibrant flowerbed with its tongue out,lounging on a fluffy rug by a fireplace with itstongue slightly visible,climbing a tree in a sunny park with its tongue flicking out,exploring a colorful backyard with its tongue tasting the air,chasing butterflies in a sunny meadow with its tongue extended]

[0131] The advantage of this step is that the large language model ChatGPT-4 uses an evolutionary algorithm to cross and mutate the prompt words in succession, which can introduce more semantic details while retaining the original information, and generate more informative and diverse prompt words.

[0132] (4-8) using the unexpected picture set corresponding to the i-th category in the training set obtained in step (4-6) to process the new prompt word obtained in step (4-7) to obtain a score for the new prompt word;

[0133] This step is specifically as follows: the new prompt word obtained in step (4-7) is input into the Wenshengtu large model to generate an extended picture of the category corresponding to the prompt word, and the process is repeated multiple times (100 times in this example). The multiple extended pictures obtained constitute the extended picture set corresponding to the i-th category in the training set.

[0134] Then, calculate the similarity between the unexpected picture set obtained in step (4-6) and the extended picture set corresponding to the i-th category in the training set:

[0135]

[0136] Where m∈[1, the total number of extended images in the extended image set corresponding to the i-th category in the training set N g ], n∈[1, the total number of pictures in the unexpected picture set N d ], Represents the extended image set I corresponding to the i-th category in the training set g The mth picture in Represents the unexpected picture set I obtained in step (4-6) c The nth picture in g represents the number of extended images in the extended image set corresponding to the i-th category in the training set, N d Indicates the number of unexpected images in the unexpected image set. Represents the extended image set I corresponding to the i-th category in the training set g The mth picture in Unexpected pictures collection I c The nth picture in , and have:

[0137]

[0138] in and They represent the feature vector of the mth picture in the extended picture set corresponding to the i-th category in the training set and the feature vector of the nth picture in the unexpected picture set, respectively. and Respectively represent the feature vector of the mth picture in the extended picture set corresponding to the i-th category in the training set The Euclidean norm of and the feature vector of the nth picture in the unexpected picture set The Euclidean norm of the training set is the feature vector of the mth picture in the extended picture set corresponding to the i-th category is extracted through Contrastive Language-Image Pretraining (CLIP) and is equal to:

[0139]

[0140] Where Embedding represents the visual encoder of CLIP.

[0141] Then, the similarity between the expected image set obtained in step (4-6) and the extended image set corresponding to the i-th category in the training set is calculated:

[0142]

[0143] Where: N c Represents the total number of images of the i-th category in the training set.

[0144] Then, the confidence of the expanded image set corresponding to the i-th category in the training set is calculated:

[0145]

[0146] in Represents the extended image set I corresponding to the i-th category in the training set g The confidence of the mth image in is equal to:

[0147]

[0148] Where p represents the extended image set I corresponding to the i-th category in the training set by the CLIP model. g The mth picture in The predicted probability distribution of is equal to:

[0149] p=[p1,p2,...,p T ]

[0150] Where T is the number of categories of images in the dataset (10 in this example), p k is the extended image set I corresponding to the i-th category in the ResNet50 model prediction training set g The mth picture in The probability of belonging to the kth class, where k∈[1,T].

[0151] Finally, the similarity between the unexpected picture set and the extended picture set corresponding to the i-th category in the training set, the similarity between the expected picture set and the extended picture set corresponding to the i-th category in the training set, and the confidence of the extended picture set corresponding to the i-th category in the training set are added together to obtain the score of the new prompt word, that is:

[0152]

[0153] (4-9) using the unexpected picture set obtained in step (4-6) to process the current prompt word obtained in step (4-5) to obtain a score for the current prompt word;

[0154] The processing process of this step is basically the same as the processing process of the above step (4-8), except that the new prompt word in step (4-8) is replaced by the current prompt word obtained in step (4-6), which will not be repeated here.

[0155] (4-10) The score of the current prompt word obtained in step (4-9) and the score of the new prompt word obtained in step (4-8) are used as the prompt word with the higher score as the current prompt word in step (4-5) (e.g. Figure 3 As shown), and the current prompt word is used as the optimized prompt word of the i-th category in the training set;

[0156] Specifically, by comparing the scores corresponding to the current prompt word and the newly generated prompt word, the one with the higher score is the better prompt word. If the newly generated prompt word is better, the new prompt word is used as the current prompt word and updated to the current prompt word in (4-5). Otherwise, it is not updated and recorded at the same time.

[0157] (4-11) Set j=j+1 and return to step (4-4);

[0158] (4-12) Set i=i+1 and return to step (4-2).

[0159] The advantages of the above steps (4-5) to (4-10) are that the use of evolutionary algorithms for crossover and mutation can effectively increase the richness and diversity of the prompt word information while retaining the original prompt word information. The use of the objective function to calculate the score of the prompt word can well measure whether the prompt word can reduce the generation of unexpected pictures, thereby iteratively selecting better prompt words, thereby obtaining prompt words that effectively reduce the generation of unexpected pictures.

[0160] (5) For the optimized prompt words corresponding to the category of the extended picture set corresponding to each category in the training set obtained in step (4), randomly sample the prompt words multiple times (100 times in this example) to obtain sampled prompt words (the sampling process is as follows Figure 2 As shown), the obtained multiple sampling prompt words are input into the Wenshengtu large model to obtain the extended pictures of the categories corresponding to all the sampling prompt words, and all the obtained extended pictures constitute the extended picture set corresponding to the category.

[0161] (6) Using an evolutionary algorithm to process the extended picture set corresponding to each category in the training set obtained in step (5) to obtain an optimized prompt word corresponding to the category of the extended picture set corresponding to the category.

[0162] This step specifically includes the following steps:

[0163] (6-1) Set counter f=1;

[0164] (6-2) Determine whether f is greater than the number of categories in the data set obtained in step (1) (10 in this example). If so, the process ends; otherwise, proceed to step (6-3);

[0165] (6-3) Set counter h = 1;

[0166] (6-4) Determine whether h is greater than a preset number of iterations (the value range is 8 to 10, preferably 10). If so, proceed to step (6-12); otherwise, proceed to step (6-5);

[0167] (6-5) obtaining the optimized prompt word of the category corresponding to the extended picture set corresponding to the f-th category in the training set obtained in step (4) as the current prompt word;

[0168] (6-6) obtaining a set of images lacking diversity from the extended set of images corresponding to the f-th category in the training set obtained in step (5);

[0169] Specifically, this step is to check each picture in the extended picture set corresponding to the f-th category in the training set. If the similarity between the picture and the picture corresponding to the f-th category in the training set obtained in step (1) is high (that is, the similarity between the picture and the picture corresponding to the f-th category in the training set reaches a preset threshold, and the value range of the preset threshold is 0.90 to 0.99, preferably 0.95), it means that the picture has not expanded new information, and therefore the picture is a picture lacking diversity.

[0170] (6-7) using an evolutionary algorithm to process the current prompt word obtained in step (6-5) to obtain a new prompt word;

[0171] It should be noted that the processing process of this step is basically the same as the processing process of the above step (4-7), except that the current prompt word in step (4-7) is replaced by the current prompt word in step (6-6), which will not be repeated here.

[0172] (6-8) using the image set lacking diversity obtained in step (6-6) to process the new prompt word obtained in step (6-7) to obtain a score for the new prompt word;

[0173] Specifically, this step is as follows: first, the new prompt word obtained in step (6-7) is input into the Wenshengtu large model to generate an extended picture of the category corresponding to the prompt word, and the process is repeated multiple times (100 times in this example). The multiple extended pictures obtained constitute the extended picture set corresponding to the f-th category in the training set.

[0174] Then, calculate the diversity index S of the extended image set corresponding to the fth category in the training set div (I l ):

[0175]

[0176] Among them I l represents the extended image set corresponding to the f-th category in the training set, M is the total number of images in the extended image set corresponding to the f-th category in the training set, and v′ q is the feature vector of the qth picture in the extended picture set corresponding to the fth category in the training set (where q∈[1,M]), is the average feature vector of all images in the extended image set corresponding to the f-th category in the training set, and has:

[0177]

[0178] in Represents the distance between the feature vector of the qth picture in the extended picture set corresponding to the fth category in the training set and the average feature vector of all pictures in the extended picture set corresponding to the fth category in the training set:

[0179]

[0180] Then, the similarity between the image set lacking diversity and the expanded image set corresponding to the fth category in the training set is calculated:

[0181]

[0182] Where: N a Represents the total number of images in the image set that lacks diversity, r∈[1, the total number of extended images in the extended image set corresponding to the f-th category in the training set N l ], e∈[1, the total number of images in the lack of diversity image set N a ], Represents the extended image set I corresponding to the fth category in the training set l The rth picture in Represents the lack of diversity image set I obtained in step (6-6) s The e-th picture in .

[0183] Then, the confidence of the expanded image set corresponding to the fth category in the training set is calculated:

[0184]

[0185] Finally, the diversity index of the extended picture set corresponding to the f-th category in the training set, the similarity between the picture set lacking diversity and the extended picture set corresponding to the f-th category in the training set, and the confidence of the extended picture set corresponding to the f-th category in the training set are added together to obtain the score of the new prompt word, namely:

[0186]

[0187] (6-9) using the image set lacking diversity obtained in step (6-6) to process the current prompt word obtained in step (6-5) to obtain a score for the current prompt word;

[0188] It should be noted that the processing process of this step is basically the same as the processing process of the above step (6-8), except that the new prompt word in step (6-8) is replaced by the current prompt word obtained in step (6-6), which will not be repeated here.

[0189] (6-10) The score of the current prompt word obtained in step (6-9) and the score of the new prompt word obtained in step (6-8) are used as the prompt word with the higher score as the current prompt word obtained in step (6-5) (e.g. Figure 3 As shown), and the current prompt word is used as the optimized prompt word for the f-th category in the training set;

[0190] (6-11) Set h=h+1 and return to step (6-4);

[0191] (6-11) Set f=f+1 and return to step (6-2).

[0192] The advantage of the above steps (6-5) to (6-10) is that the use of evolutionary algorithms for crossover and mutation can effectively increase the richness and diversity of the prompt word information while retaining the original prompt word information. The use of the objective function to calculate the score of the prompt word can well measure whether the prompt word can increase the diversity of the generated picture set, thereby iteratively selecting better prompt words, thereby obtaining prompt words that effectively increase the diversity of the generated picture set.

[0193] (7) For each category in the training set, the optimized prompt word corresponding to the category of the extended picture set corresponding to the category obtained in step (6) is used as the final prompt word corresponding to the category.

[0194] Experimental Results

[0195] The experimental environment of the present invention: NVIDIA A100 GPU is used, and the algorithm of the present invention is implemented using the Pytorch framework. The specific settings are as follows: the large language model used in the evolutionary algorithm is ChatGPT-4, the large model of the text graph uses Stable-Diffusion-v1.5, the prompt words are iteratively optimized for 10 rounds, the parameters scale and strength are 20 and 0.5 when the text graph generates pictures, the batch size of the training ResNet50 model is set to 256, and the training rounds are set to 100 rounds.

[0196] In order to illustrate the effectiveness of the method of the present invention and the effect of optimizing the prompt words of the large model of the text graph based on the evolutionary algorithm, this experiment first trained the ResNet50 model on the original training data set (without expansion), and then used ChatGPT-4 to generate prompt words, the evolutionary algorithm + reducing unexpected data to generate prompt words, the evolutionary algorithm + increasing diversity to generate prompt words, and other operations to obtain the prompt word expansion data set. The ResNet50 model was trained based on the data sets with different prompt word expansions, and the test accuracy of the trained ResNet50 model was compared.

[0197] The calculation formula for the test accuracy of the ResNet50 model is:

[0198]

[0199] Where: C is the number of correct categories predicted by the ResNet50 model on the test set obtained in step (1), and N is the total number of samples in the test set.

[0200] The test accuracy here reports the average accuracy of the test set in the last 5 rounds of training 100 rounds. The evaluation results of different experimental settings are shown in Table 1 below, where the closer the test accuracy is to 100%, the better the classification effect of the trained ResNet50 model:

[0201] Table 1

[0202]

[0203]

[0204] In the above table, the first line is the test accuracy obtained by training with the original training set without expansion; the second line is the test accuracy obtained by training with the large language model ChatGPT-4 to generate the prompt word extended data set, which serves as the control group; the third line is the test accuracy obtained by training with the optimized prompt word extended data set based on the evolutionary algorithm and the operation of reducing the generation of unexpected pictures; the fourth line is the test accuracy obtained by training with the optimized prompt word extended data set based on the evolutionary algorithm and the operation of increasing the diversity of generated pictures; the fifth line is the test accuracy obtained by training with the optimized prompt word extended data set based on the evolutionary algorithm and the operations of reducing the generation of unexpected pictures and increasing the generation diversity.

[0205] As can be seen from the table above, the test accuracy when using the original training set without any expansion is 48.45%. This shows that without additional data enhancement or prompt word generation, the performance of the ResNet50 model is relatively low, which may be limited by the scale and diversity of the training data.

[0206] The test accuracy of the extended dataset using the large language model ChatGPT-4 to generate prompt words increased to 69.3%. This shows that the prompt words generated by the large language model ChatGPT-4 can effectively expand the dataset and improve the generalization ability and classification performance of the ResNet50 model, which is significantly improved compared to the original dataset.

[0207] Based on the evolutionary algorithm, combined with the optimization of prompt words to reduce the generation of unexpected pictures, the test accuracy was further improved to 72.4%. This shows that by reducing the generation of unexpected pictures, the data quality is improved, and the generation of irrelevant information such as non-cat and dog subject pictures is reduced, thus helping the ResNet50 model to better learn effective features.

[0208] By optimizing the prompt words through the evolutionary algorithm and increasing the diversity of generated images, the test accuracy reached 71.3%. This shows that increasing the diversity of generated images helps the ResNet50 model learn a wider range of features.

[0209] Combining the evolutionary algorithm, reducing unexpected image generation, and increasing generation diversity, the test accuracy was further improved to 73.90%. This shows that optimizing data quality and diversity at the same time can maximize the performance of the ResNet50 model and achieve the best classification effect.

[0210] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method and system for optimizing prompt words of a large model of a cultural graph based on an evolutionary algorithm, characterized in that: The steps include: (1) Obtain a data set, which includes a training set and a test set. (2) For each category in the training set obtained in step (1), use the ChatGPT-4 model to generate the corresponding prompt word for the category. (3) For each prompt word corresponding to the category obtained in step (2), sampled prompt words are obtained by random sampling from the prompt word multiple times, and the obtained multiple sampled prompt words are input into the Wenshengtu large model to obtain extended pictures of the categories corresponding to all the sampled prompt words, and all the obtained extended pictures constitute an extended picture set corresponding to the category. (4) Using an evolutionary algorithm to process the extended picture set corresponding to each category obtained in step (3) to obtain an optimized prompt word for the category corresponding to the extended picture set corresponding to the category. (5) For the optimized prompt words of the category corresponding to the extended picture set corresponding to each category in the training set obtained in step (4), sampled prompt words are randomly sampled from the prompt words multiple times, and the obtained multiple sampled prompt words are input into the Wenshengtu large model to obtain extended pictures of the category corresponding to all the sampled prompt words, and all the obtained extended pictures constitute the extended picture set corresponding to the category. (6) Using an evolutionary algorithm to process the extended picture set corresponding to each category in the training set obtained in step (5) to obtain an optimized prompt word corresponding to the category of the extended picture set corresponding to the category. (7) For each category in the training set, the optimized prompt word corresponding to the category of the extended picture set corresponding to the category obtained in step (6) is used as the final prompt word corresponding to the category.

2. The method for optimizing the prompt words of the large model of the culture graph based on the evolutionary algorithm according to claim 1 is characterized in that: Step (4) specifically includes the following steps: (4-1) Set counter i=1; (4-2) Determine whether i is greater than the number of categories in the data set obtained in step (1) (10 in this example). If so, the process ends; otherwise, proceed to step (4-3); (4-3) Set counter j = 1; (4-4) Determine whether j is greater than a preset number of iterations (the value range is 8 to 10, preferably 10). If so, proceed to step (4-12); otherwise, proceed to step (4-5); (4-5) Obtain the prompt word corresponding to the i-th category in the training set obtained in step (2) as the current prompt word; (4-6) obtaining the unexpected picture set and the expected picture set corresponding to the i-th category in the training set from the extended picture set corresponding to the i-th category in the training set obtained in step (3); (4-7) using an evolutionary algorithm to process the current prompt word obtained in step (4-5) to obtain a new prompt word; (4-8) using the unexpected picture set corresponding to the i-th category in the training set obtained in step (4-6) to process the new prompt word obtained in step (4-7) to obtain a score for the new prompt word; (4-9) using the unexpected picture set obtained in step (4-6) to process the current prompt word obtained in step (4-5) to obtain a score for the current prompt word; (4-10) The score of the current prompt word obtained in step (4-9) and the score of the new prompt word obtained in step (4-8) are used as the prompt word with a higher score as the current prompt word in step (4-5), and the current prompt word is used as the optimized prompt word of the i-th category in the training set; (4-11) Set j=j+1 and return to step (4-4); (4-12) Set i=i+1 and return to step (4-2).

3. The method for optimizing the prompt words of a large model of a cultural graph based on an evolutionary algorithm according to claim 1 or 2, characterized in that: Step (4-6) is to check each picture in the extended picture set corresponding to the i-th category in the training set. If there is a subject other than the cat or dog category in the picture, the picture is an unexpected picture, otherwise the picture is an expected picture. Step (4-7) is specifically as follows: first, use the large language model ChatGPT-4 to perform a self-crossover operation on the current prompt word obtained in step (4-5) to obtain the prompt word after cross processing; then, use the large language model ChatGPT-4 to perform a mutation operation on the prompt word after cross processing to obtain a new prompt word.

4. The method for optimizing the prompt words of a large model of a cultural graph based on an evolutionary algorithm according to any one of claims 1 to 3, characterized in that: Steps (4-8) are specifically: The new prompt word obtained in step (4-7) is input into the Wenshengtu large model to generate an extended picture of the category corresponding to the prompt word, and the process is repeated multiple times. The multiple extended pictures obtained constitute the extended picture set corresponding to the i-th category in the training set. Then, calculate the similarity between the unexpected picture set obtained in step (4-6) and the extended picture set corresponding to the i-th category in the training set: Where m∈[1, the total number of extended images in the extended image set corresponding to the i-th category in the training set N g ], n∈[1, the total number of pictures in the unexpected picture set N d ], Represents the extended image set I corresponding to the i-th category in the training set g The mth picture in Represents the unexpected picture set I obtained in step (4-6) c The nth picture in g represents the number of extended images in the extended image set corresponding to the i-th category in the training set, N d Indicates the number of unexpected images in the unexpected image set. Represents the extended image set I corresponding to the i-th category in the training set g The mth picture in Unexpected pictures collection I c The nth picture in , and have: in and They represent the feature vector of the mth picture in the extended picture set corresponding to the i-th category in the training set and the feature vector of the nth picture in the unexpected picture set, respectively. and Respectively represent the feature vector of the mth picture in the extended picture set corresponding to the i-th category in the training set The Euclidean norm of and the feature vector of the nth picture in the unexpected picture set The Euclidean norm of the training set is the feature vector of the mth picture in the extended picture set corresponding to the i-th category is extracted by contrastive language-image pre-training CLIP and is equal to: Where Embedding represents the visual encoder of CLIP. Then, the similarity between the expected image set obtained in step (4-6) and the extended image set corresponding to the i-th category in the training set is calculated: Where: N c Represents the total number of images of the i-th category in the training set. Then, the confidence of the expanded image set corresponding to the i-th category in the training set is calculated: in Represents the extended image set I corresponding to the i-th category in the training set g The confidence of the mth image in is equal to: Where p represents the extended image set I corresponding to the i-th category in the training set by the CLIP model. g The mth picture in The predicted probability distribution of is equal to: p=[p1,p2,...,p T ] Where T is the number of categories of images in the dataset (10 in this example), p k is the extended image set I corresponding to the i-th category in the ResNet50 model prediction training set g The mth picture in The probability of belonging to the kth class, where k∈[1,T]. Finally, the similarity between the unexpected picture set and the extended picture set corresponding to the i-th category in the training set, the similarity between the expected picture set and the extended picture set corresponding to the i-th category in the training set, and the confidence of the extended picture set corresponding to the i-th category in the training set are added together to obtain the score of the new prompt word, that is:

5. The method for optimizing the prompt words of the large model of the culture graph based on the evolutionary algorithm according to claim 4 is characterized in that: Step (6) specifically includes the following sub-steps: (6-1) Set counter f=1; (6-2) Determine whether f is greater than the number of categories in the data set obtained in step (1). If so, the process ends; otherwise, proceed to step (6-3); (6-3) Set counter h = 1; (6-4) Determine whether h is greater than the preset number of iterations. If so, proceed to step (6-12); otherwise, proceed to step (6-5); (6-5) obtaining the optimized prompt word of the category corresponding to the extended picture set corresponding to the f-th category in the training set obtained in step (4) as the current prompt word; (6-6) obtaining a set of images lacking diversity from the extended set of images corresponding to the f-th category in the training set obtained in step (5); (6-7) using an evolutionary algorithm to process the current prompt word obtained in step (6-5) to obtain a new prompt word; (6-8) using the image set lacking diversity obtained in step (6-6) to process the new prompt word obtained in step (6-7) to obtain a score for the new prompt word; (6-9) using the image set lacking diversity obtained in step (6-6) to process the current prompt word obtained in step (6-5) to obtain a score for the current prompt word; (6-10) The score of the current prompt word obtained in step (6-9) and the score of the new prompt word obtained in step (6-8) are used as the prompt word with a higher score as the current prompt word obtained in step (6-5), and the current prompt word is used as the optimized prompt word for the fth category in the training set; (6-11) Set h=h+1 and return to step (6-4); (6-11) Set f=f+1 and return to step (6-2).

6. The method for optimizing the prompt words of the large model of the cultural graph based on the evolutionary algorithm according to claim 5 is characterized in that: Step (6-6) is to check each picture in the extended picture set corresponding to the f-th category in the training set. If the similarity between the picture and the picture corresponding to the f-th category in the training set obtained in step (1) is high, that is, the similarity between the picture and the picture corresponding to the f-th category in the training set reaches a preset threshold, it means that the picture has not expanded new information and the picture is a picture lacking diversity.

7. The method for optimizing the prompt words of the large model of the cultural graph based on the evolutionary algorithm according to claim 6 is characterized in that: Steps (6-8) are specifically: First, the new prompt word obtained in step (6-7) is input into the Wenshengtu large model to generate an extended picture of the category corresponding to the prompt word, and the process is repeated multiple times. The multiple extended pictures obtained constitute the extended picture set corresponding to the fth category in the training set. Then, calculate the diversity index S of the extended image set corresponding to the fth category in the training set div (I l ): Among them I l represents the extended image set corresponding to the f-th category in the training set, M is the total number of images in the extended image set corresponding to the f-th category in the training set, and v′ q is the feature vector of the qth picture in the extended picture set corresponding to the fth category in the training set (where q∈[1,M]), is the average feature vector of all images in the extended image set corresponding to the f-th category in the training set, and has: in Represents the distance between the feature vector of the qth picture in the extended picture set corresponding to the fth category in the training set and the average feature vector of all pictures in the extended picture set corresponding to the fth category in the training set: Then, the similarity between the image set lacking diversity and the expanded image set corresponding to the fth category in the training set is calculated: Where: N a Represents the total number of images in the image set that lacks diversity, r∈[1, the total number of extended images in the extended image set corresponding to the f-th category in the training set N l ], e∈[1, the total number of images in the lack of diversity image set N a ], Represents the extended image set I corresponding to the fth category in the training set l The rth picture in Represents the lack of diversity image set I obtained in step (6-6) s The e-th picture in . Then, the confidence of the expanded image set corresponding to the fth category in the training set is calculated: Finally, the diversity index of the extended picture set corresponding to the f-th category in the training set, the similarity between the picture set lacking diversity and the extended picture set corresponding to the f-th category in the training set, and the confidence of the extended picture set corresponding to the f-th category in the training set are added together to obtain the score of the new prompt word, namely:

8. A large model prompt word optimization system for cultural graphs based on evolutionary algorithms, characterized in that: include: The first module is used to obtain a data set, which includes a training set and a test set. The second module is used to generate the corresponding prompt word for each category in the training set obtained in the first module using the ChatGPT-4 model. The third module is used to randomly sample the prompt words corresponding to each category obtained in the second module multiple times to obtain sampled prompt words, and input the obtained multiple sampled prompt words into the Wenshengtu large model to obtain extended pictures of the categories corresponding to all the sampled prompt words, and all the obtained extended pictures constitute an extended picture set corresponding to the category. The fourth module is used to process the extended picture set corresponding to each category obtained by the third module using an evolutionary algorithm to obtain an optimized prompt word for the category corresponding to the extended picture set corresponding to the category. The fifth module is used to randomly sample the optimized prompt words for the category corresponding to the extended picture set corresponding to each category in the training set obtained in the fourth module multiple times to obtain sampled prompt words, and input the obtained multiple sampled prompt words into the Wenshengtu large model to obtain the extended pictures of the category corresponding to all the sampled prompt words, and all the obtained extended pictures constitute the extended picture set corresponding to the category. The sixth module is used to process the extended picture set corresponding to each category in the training set obtained in the fifth module by using an evolutionary algorithm to obtain an optimized prompt word corresponding to the category of the extended picture set corresponding to the category. The seventh module is used to use, for each category in the training set, the optimized prompt word corresponding to the category of the extended picture set corresponding to the category obtained in the sixth module as the final prompt word corresponding to the category.

Images