Data processing method and device

Abstract

The invention discloses a data processing method, and the method comprises the steps: obtaining a first text, expanding the first text to obtain a second text, the first text comprising a text of a specific type, and the text of the specific type being a text related to an image which is not allowed to be generated by an image generation model; and inputting the second text into a text discrimination model, wherein the text discrimination model is used for identifying whether the second text comprises a specific type of text or not. And if the text discrimination model determines that the second text does not include the text of the specific type, inputting the second text into the image generation model, and obtaining a target image generated by the image generation model based on the second text. And determining whether the first text is matched with the target image. Under the condition that the first text is matched with the target image, a second text is output, and the second text is used for optimizing the text discrimination model. According to the scheme, the image generation model can be effectively prevented from generating images which are not allowed to be generated originally.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data processing method and device. Background Art

[0002] With the development of artificial intelligence (AI) technology, more and more models have emerged. For example, image generation models have emerged, which can generate corresponding images based on input text. Among them, the model that generates images based on text can also be called a "text-generated image" model.

[0003] However, the current text-based graph model may generate some images that do not meet the requirements. In some scenarios, if the above-mentioned images that do not meet the requirements are output to the user, it may bring a bad user experience. How to avoid outputting the above-mentioned images that do not meet the requirements to the user is a problem that needs to be solved at present. Summary of the invention

[0004] In order to solve or at least partially solve the above technical problems, the present application provides a data processing method and device.

[0005] In a first aspect, the present application provides a data processing method, the method comprising:

[0006] Acquire a first text, and expand the first text to obtain a second text, wherein the first text includes a specific type of text, and the specific type of text is text related to an image that the image generation model is not allowed to generate;

[0007] Inputting the second text into a text discrimination model, wherein the text discrimination model is used to identify whether the second text includes the specific type of text;

[0008] If the text discrimination model determines that the second text does not include the specific type of text, inputting the second text into an image generation model, and obtaining a target image generated by the image generation model based on the second text;

[0009] determining whether the first text matches the target image;

[0010] When the first text matches the target image, the second text is output, and the second text is used to optimize the text discrimination model.

[0011] Optionally, before determining whether the first text matches the target image, the method further includes:

[0012] Inputting the target image into an image discrimination model, wherein the image discrimination model is used to identify whether the target image is an image that is not allowed to be generated by the image generation model;

[0013] The determining whether the first text matches the target image includes:

[0014] If the image discrimination model determines that the target image is not an image that the image generation model is not allowed to generate, it is determined whether the first text matches the target image.

[0015] Optionally, the second text is also used to optimize the image discrimination model.

[0016] Optionally, the expanding the first text to obtain the second text includes:

[0017] The first text and a first prompt word are input into a large language model, wherein the first prompt word is used to instruct the large language model to expand the first text according to the semantics of the first text to obtain texts in multiple different languages ​​with the same semantics, so as to obtain the second text.

[0018] Optionally, the first prompt word is also used to prompt the large language model to convert the texts in the multiple different languages ​​into second prompt words that can be input into the image generation model, and the second text includes the second prompt word.

[0019] Optionally, the first prompt word is also used to instruct the large language model to output the second prompt word in a specific format.

[0020] Optionally, the specific format is JSON format.

[0021] Optionally, the method further includes:

[0022] Additional text is added on the basis of the second prompt word, and the additional text includes: a text describing the painting style, and / or a text describing the second prompt word, and the second text also includes the additional text.

[0023] In a second aspect, the present application provides a data processing device, the device comprising:

[0024] an acquisition unit, configured to acquire a first text and expand the first text to obtain a second text, wherein the first text includes a specific type of text, and the specific type of text is text related to an image that the image generation model is not allowed to generate;

[0025] A first input unit, used for inputting the second text into a text discrimination model, wherein the text discrimination model is used for identifying whether the second text includes the specific type of text;

[0026] A second input unit is used to input the second text into an image generation model if the text discrimination model determines that the second text does not include the specific type of text, and obtain a target image generated by the image generation model based on the second text;

[0027] a determination unit, configured to determine whether the first text matches the target image;

[0028] An output unit is used to output the second text when the first text matches the target image, wherein the second text is used to optimize the text discrimination model.

[0029] Optionally, the device further comprises:

[0030] A third input unit is used to input the target image into an image discrimination model before determining whether the first text matches the target image, wherein the image discrimination model is used to identify whether the target image is an image that is not allowed to be generated by the image generation model;

[0031] The determining unit is used for:

[0032] If the image discrimination model determines that the target image is not an image that the image generation model is not allowed to generate, it is determined whether the first text matches the target image.

[0033] Optionally, the second text is also used to optimize the image discrimination model.

[0034] Optionally, the expanding the first text to obtain the second text includes:

[0035] The first text and a first prompt word are input into a large language model, wherein the first prompt word is used to instruct the large language model to expand the first text according to the semantics of the first text to obtain texts in multiple different languages ​​with the same semantics, so as to obtain the second text.

[0036] Optionally, the first prompt word is also used to prompt the large language model to convert the texts in the multiple different languages ​​into second prompt words that can be input into the image generation model, and the second text includes the second prompt word.

[0037] Optionally, the first prompt word is also used to instruct the large language model to output the second prompt word in a specific format.

[0038] Optionally, the specific format is json format.

[0039] Optionally, the device further comprises:

[0040] The adding unit is used to add additional text based on the second prompt word, and the additional text includes: a painting style description text and / or a text describing the second prompt word, and the second text also includes the additional text.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a processor and a memory;

[0042] The processor is used to execute instructions stored in the memory so that the electronic device performs the method as described in any one of the first aspects above.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, wherein the instructions instruct a device to execute a method as described in any one of the above first aspects.

[0044] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0045] Compared with the prior art, the embodiments of the present application have the following advantages:

[0046] The present application provides a data processing method, which includes: obtaining a first text and expanding the first text to obtain a second text, wherein the first text includes a specific type of text, and the specific type of text is text related to an image that an image generation model is not allowed to generate. After the second text is expanded, the second text can be input into a text discrimination model, and the text discrimination model is used to identify whether the second text includes the specific type of text. If the text discrimination model determines that the second text does not include the specific type of text, the second text is further input into an image generation model, and a target image generated by the image generation model based on the second text is obtained. After obtaining the target image, it is determined whether the first text matches the target image. If the first text matches the target image, it means that the image generation model has generated an image that it was not allowed to generate. Correspondingly, it means that the text recognition model did not correctly recognize that the second text actually includes a specific type of text. In this scenario, a second text can be output and used to optimize the text discrimination model, thereby improving the recognition ability of the text discrimination model. In this way, text of a specific type can be prevented from being input into the image generation model. Accordingly, the image generation model can be effectively prevented from generating images that it is not allowed to generate, thereby preventing the image generation model from generating images that it is not allowed to generate for the user, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic diagram of an image production link is shown;

[0049] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0050] Figure 3 A flowchart of a data processing method provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] Diffusion Model is a type of deep learning model based on probability generation. Diffusion model can be used to generate images. The basic principle of diffusion model to generate images is to gradually add noise to the image through the forward process, and learn denoising and restoration through the reverse process, so that the model can generate high-quality images from pure noise. However, the images generated by the standard diffusion model usually lack clear control capabilities, that is, the generated content is relatively random and it is difficult to directly meet specific task requirements. The specific task requirements mentioned here can be, for example, the user's requirements for the generated images.

[0054] In order to enhance the task adaptation ability of the diffusion model, the diffusion model is guided to guide the image generation process through external signals (such as text or other images) so that the image generated by the diffusion model meets the requirements of specific tasks. The image generation process of the diffusion model can be guided by the "text map". In this case, the user can use text to describe the content, scene, style, etc. of the target image that the diffusion model expects to be generated. The diffusion model extracts the features of the text input by the user and introduces them into the diffusion model. In this scenario, the diffusion model that introduces text features can also be called the text-to-image diffusion model (Text-to-Image Diffusion Model). The text-to-image diffusion model is a generative model based on the diffusion model. It uses a text encoder to encode the input text prompt to obtain semantic constraints. The diffusion model gradually adds a denoising signal containing semantic constraints to the noise, thereby gradually denoising the text description from the noise and converting it into a high-quality image. Among them, the text prompt refers to the prompt corresponding to the text input by the user.

[0055] In actual application scenarios, the image generation model (such as the text image diffusion model) is not allowed to generate images that do not meet the requirements, wherein the images that the image generation model is not allowed to generate and do not meet the requirements may be images that include specific content, for example, the image generation model is not allowed to generate images that include specific elements. In order to avoid the image generation model uncontrollably generating the aforementioned images including specific content and outputting the images including the specific content to the user, thereby affecting the user experience. A text discrimination model and an image discrimination model can be added to the image production chain to avoid outputting the aforementioned images including specific content to the user.

[0056] For reference Figure 1 To understand, Figure 1 A schematic diagram of an image production link is shown.

[0057] like Figure 1As shown, the image generation link includes: a text discrimination model, an image generation model and an image discrimination model, wherein the text discrimination model is used to receive the text input by the user, and determine whether the text input by the user includes the text for indicating the generation of the aforementioned specific content. If the text discrimination model determines that the text input by the user includes the text for indicating the generation of the aforementioned specific content, it is determined that the text input by the user does not meet the requirements, and the text input by the user is rejected from being input into the image generation model. If the text discrimination model determines that the text input by the user does not include the text for indicating the generation of the aforementioned specific content, it can be determined that the text input by the user meets the requirements, so the text input by the user is further input into the image generation model, and the image generation model generates an image based on the text. The image generated by the image generation model is used as the input of the image discrimination model, and the image discrimination model is used to determine whether the input image includes the aforementioned specific content. If the image discrimination model determines that the input image includes the aforementioned specific content, the image is not output to the user; if the image discrimination model determines that the input image does not include the aforementioned specific content, the image can be output.

[0058] However, even if the Figure 1 The image generation link shown may still output an image including specific content, because in some scenarios, the text discrimination model cannot completely and correctly identify whether the input text includes the text used to indicate the generation of the aforementioned specific content. In other scenarios, the image discrimination model cannot completely and correctly identify whether the input image includes the aforementioned specific content.

[0059] In view of this, the present application provides a data processing method and device, which can effectively prevent the image generation model from generating images that it is not allowed to generate. The images that the image generation model is not allowed to generate mentioned here can be understood as images including the aforementioned specific content.

[0060] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0061] Exemplary Methods

[0062] See also Figure 2 , which is a flow chart of a data processing method provided in an embodiment of the present application.

[0063] Figure 2 The method shown can be executed by a server, for example, a server carrying a text map function. Of course, the method can also be executed by a client, and the embodiment of the present application does not specifically limit it. In the following description, the method is described by taking the execution of the server as an example.

[0064] Figure 2 The method shown includes the following S101-S105.

[0065] S101: Acquire a first text, and expand the first text to obtain a second text, wherein the first text is a text of a specific type, and the text of the characteristic type is text related to an image that an image generation model is not allowed to generate.

[0066] In one example, the first text may be text input by a user. In another example, the first text may also be text randomly generated by the server. The first text is used to guide the image generation model to generate a corresponding image, or in other words, the first text is used to indicate the requirements that the image generated by the image generation model needs to meet. In one example, the image generation model may be a diffusion model, such as the aforementioned Wenshengtu diffusion model. Of course, the image generation model may not be a diffusion model, but other types of models other than the diffusion model, which is not specifically limited in the embodiments of the present application.

[0067] In the present application, the first text includes a specific type of text, and the aforementioned specific type of text is text related to an image that the image generation model is not allowed to generate. Among them, the image that the image generation model is not allowed to generate may be an image including specific content. As an example, the first text includes an image related to the aforementioned specific content. As a specific example, the first text includes text for instructing the image generation model to generate the image that is not allowed to be generated. For example, the first text includes text for instructing the image generation model to generate the specific content. For example: assuming that the image that the image generation model is not allowed to generate is an image including a "pet cat", the first text may be text including "cat", for example, the first text may be "Please generate a cat".

[0068] In one example, after obtaining the first text, the first text can be expanded according to the semantics of the first text to obtain a second text with the same semantics as the first text. Considering that the image generation model generally supports multiple input languages, as a specific example, the texts in multiple languages ​​with the same semantics as the first text can be expanded according to the semantics of the first text to obtain the second text. The texts in multiple languages ​​can include the first text and texts in at least one other language different from the language of the first text. For example, if the first text is in Chinese, the texts in multiple languages ​​can include the first text and English text with the same semantics as the first text.

[0069] In one example, considering that the Large Language Model (LLM) is an important achievement of natural language processing technology based on deep learning, the Large Language Model is widely used in text generation, dialogue systems, translation, information retrieval and other fields. The core structure of the Large Language Model, Transformer, realizes global information modeling through the self-attention mechanism, significantly improves the model's ability to capture long-distance dependencies, and the Large Language Model also supports parallel computing. Therefore, the Large Language Model can be expanded according to the semantics of the first text to obtain texts in multiple different languages ​​with the same semantics to obtain the second text. Specifically, the first text and the first prompt word can be input into the Large Language Model, and the first prompt word is used to indicate that the Large Language Model is expanded according to the semantics of the first text to obtain texts in multiple different languages ​​with the same semantics to obtain the second text. The embodiment of the present application does not specifically limit the first prompt word, as long as the first prompt word can reflect the meaning of "instructing the large language model to expand according to the semantics of the first text to obtain texts in multiple different languages ​​with the same semantics".

[0070] In one example, the second text may include texts in the aforementioned multiple different languages.

[0071] In another example, it is considered that the input forms supported by the image generation model include prompt words. Therefore, the second text can be: a second prompt word that is converted from the texts in the multiple different languages ​​and can be input into the image generation model. In this scenario, the operation of converting the texts in the multiple different languages ​​into the second prompt word can also be performed by the large language model. For this case, in an example, the first prompt word is also used to instruct the large language model to convert the texts in the multiple different languages ​​into the second prompt word that can be input into the image generation model. Accordingly, based on the first prompt word, the large language model can first expand the texts in the multiple different languages ​​based on the first text, and then convert the texts in the multiple different languages ​​into the second prompt word.

[0072] In one example, in order to prevent the large language model from refusing to output the second prompt word, or to prevent the output content of the large language model from being too standardized, the large language model can be instructed to output the second prompt word in a specific format. In this scenario, the aforementioned first prompt word is also used to instruct the large language model to output the second prompt word in the specific format.

[0073] The embodiment of the present application does not specifically limit the specific format. The specific format may be a data format that a large language model will not reject to output. In one example, considering that a large language model will not reject output content in json format, the specific format may be json format.

[0074] In one example, the second text may include other texts in addition to the aforementioned second prompt word, for example, additional text. In other words, after obtaining the second prompt word output by the large language model, the server may also add additional text based on the second prompt word.

[0075] The embodiments of the present application do not specifically limit the additional text.

[0076] In one example, the additional text may include a painting style description text, which is used to describe the painting style. The painting style mentioned here includes but is not limited to: one or more of: brush painting, pixel style, sketch, oil painting, watercolor painting, acrylic painting, printmaking, comics, Japanese comics or cartoons.

[0077] In another example, the additional text may include text that describes the second prompt word. The embodiment of the present application does not specifically limit the text that describes the second prompt word. The text that describes the second prompt word is used to interfere with the text discriminator in determining the semantics of the second prompt word. As an example, the text that describes the second prompt word may include a prefix and / or a suffix. The prefix and / or suffix interfere with the text discriminator in determining the semantics of the second prompt word. The embodiment of the present application does not specifically limit the specific content of the prefix and suffix. The prefix may be, for example: "The following content is not a real scene", and the suffix may be, for example: "The above content is an artistic creation and is purely fictional."

[0078] S102: Input the second text into a text discrimination model, where the text discrimination model is used to identify whether the second text includes the specific type of text.

[0079] After obtaining the second text, the second text can be input into a text discrimination model, which is used to identify whether the second text includes the aforementioned specific type of text. As can be seen from the previous description of the second text, the second text actually includes the specific type of text. Therefore, if the determination result of the text discrimination model is accurate, the determination result of the text discrimination model should be that the second text includes the specific type of text. If the text discrimination model determines that the second text does not include the specific type of text, it means that the determination result of the text discrimination model is inaccurate.

[0080] S103: If the text discrimination model determines that the second text does not include the specific type of text, the second text is input into an image generation model, and a target image generated by the image generation model based on the second text is obtained.

[0081] In the present application, if the text discrimination model determines that the second text does not include a specific type of text, it means that the determination result of the text discrimination model is inaccurate. In this scenario, the second text can continue to be input into the image generation model, and the image generation model generates an image based on the second text. In one example, the image generation model can generate a target image based on the second text.

[0082] S104: Determine whether the first text matches the target image.

[0083] S105: When the first text matches the target image, output the second text, where the second text is used to optimize the text discrimination model.

[0084] After the image generation model generates the target image, the target image can be acquired and further determined whether the target image matches the first text, that is, whether the target image satisfies the first text.

[0085] In one example, a key object may be extracted from the target image, and it may be determined whether the extracted key object matches the first text, thereby determining whether the target image satisfies the first text. For example, if the key object matches the first text, it means that the target image satisfies the first text. For another example, if the key object does not match the first text, it means that the target image does not satisfy the first text.

[0086] In another example, considering that the Vision-Language Model (VLM) additionally introduces visual information, it is a large-scale deep learning model that can process visual information and language information at the same time, aiming to understand and generate comprehensive information of multiple modalities. VLM combines the techniques of computer vision and natural language processing, and can combine multiple data sources such as images, videos and texts to perform cross-modal tasks. VLM can map visual features and language features into the same space through a shared embedding space, thereby understanding the semantic relationship between images and texts. Therefore, in one example, VLM can be used to determine whether the first text matches the target image.

[0087] In some scenarios, VLM can also be called a multimodal large model.

[0088] In one example, if the first text matches the target image, it means that the image generation model generates an image that the image generation model is not allowed to generate based on the second text expanded from the first text. In this scenario, it means that there is a defect in the link of image generation, specifically, at least it means that there is a defect in the text discrimination model in the link of the aforementioned image generation. Therefore, in the present application, the second text can be output, and the second text can be used to optimize the text discrimination model. As a specific example, the second text can be used as a negative training sample of the text discrimination model to continue training the text discrimination model to achieve the purpose of optimizing the text discrimination model. Optimizing the text discrimination model can reduce the defects of the text discrimination model, and accordingly, enable the text discrimination model to more accurately identify whether the input text includes a specific type of text. Accordingly, it can effectively avoid the image generation model from generating images that it is not allowed to generate, thereby avoiding outputting to the user images that the image generation model generates that it is not allowed to generate, thereby improving the user experience.

[0089] In one example, after executing S103 to obtain the target image, S104 and subsequent steps may be directly executed.

[0090] In another example, after executing S103 to obtain the target image, S104 may not be executed first. Before executing S104, the target image may be input into an image discrimination model, which is used to identify whether the target image is an image that the image generation model is not allowed to generate. Regarding the image discrimination model, it can use the image discrimination model in the traditional image generation link, and no repeated description is given here.

[0091] In one example, if the image discrimination model determines that the target image is not an image that the image generation model is not allowed to generate, S104 is further executed. In this scenario, if the first text matches the target image, it means that the discrimination result of the image discrimination model is also inaccurate. Because the first text includes a specific type of text, the target image that matches the first text is most likely an image that the image generation model is not allowed to generate. In this scenario, in addition to being used to optimize the text discrimination model, the aforementioned second text can also be used to optimize the image discrimination model. As a specific example, the second text can be used as a negative training sample of the image discrimination model to continue training the image discrimination model to achieve the purpose of optimizing the image discrimination model. Optimizing the image discrimination model can reduce the defects of the image discrimination model, and accordingly, enable the image discrimination model to more accurately identify whether the input image is an image that the image generation model is not allowed to generate, and accordingly, it can effectively avoid outputting images that the image generation model is not allowed to generate for the user, thereby improving the user experience.

[0092] In another example, if the image discrimination model determines that the target image is an image that the image generation model is not allowed to generate, the process is terminated and S104 and subsequent steps are no longer executed.

[0093] The above is an introduction to the solution provided by the embodiment of the present application. Next, a specific implementation method of the present application is introduced in conjunction with the accompanying drawings.

[0094] See also Figure 3 , which is a flow chart of a data processing method provided in an embodiment of the present application. Figure 3 The method shown includes the following process steps:

[0095] 1. Input the task objectives into the large language model.

[0096] The task objectives mentioned here correspond to the first text mentioned above.

[0097] The large language model is used to expand the multilingual text based on the first text to obtain texts in multiple languages, and convert the expanded texts in multiple languages ​​into prompt words 1 (corresponding to the aforementioned second prompt words) supported by the image generation model.

[0098] 2. Determine whether the large language model outputs the corresponding prompt word.

[0099] This step is used to indicate whether the large language model refuses to expand the first text into a multilingual text.

[0100] Generally speaking, large language models will not reject data output in JSON format.

[0101] If the large language model does not output the corresponding prompt word, re-execute step 2.

[0102] If the large language model outputs prompt word 1, then style description text and prefix and suffix information are further added based on prompt word 1 output by the large language model. After adding the style description text and prefix and suffix information to prompt word 1, prompt word 2 can be obtained, and prompt word 2 is equivalent to the second text in the above embodiment.

[0103] 3. Output prompt word 2 to the text discrimination model, and the text discrimination model determines whether prompt word 2 passes the discrimination.

[0104] 4. If the text discrimination model determines that prompt word 2 has passed the discrimination, then prompt word 2 is saved and input into the image generation model. If the text discrimination model determines that prompt word 2 has not passed the discrimination, then step 2 is executed again.

[0105] The text discrimination model determines that prompt word 2 passes discrimination, which means that the text discrimination model determines that prompt word 2 does not include a specific type of text. The text discrimination model determines that prompt word 2 fails discrimination, which means that the text discrimination model determines that prompt word 2 includes a specific type of text.

[0106] 5. Input the target image generated by the image generation model into the image discrimination model to determine whether the target image passes the discrimination.

[0107] 6. If the image discrimination model determines that the target image passes the discrimination, the task target and the target image are input into the VLM to determine whether the task target and the target image match. If the image discrimination model determines that the target image fails the discrimination, the prompt word 2 is deleted.

[0108] Among them, the image discrimination model determines that the target image passes the discrimination, which means that the image discrimination model determines that the target image is not an image that the image generation model is not allowed to generate; the image discrimination model determines that the target image fails the discrimination, which means that the image discrimination model determines that the target image is an image that the image generation model is not allowed to generate.

[0109] 7. If the VLM determines that the task target and the target image match, then the prompt word 2 is output, and the prompt word 2 is used to optimize the text discrimination model and the image discrimination model. If the VLM determines that the task target and the target image do not match, then the prompt word 2 is deleted.

[0110] Exemplary Devices

[0111] Based on the method provided in the above embodiment, the embodiment of the present application further provides a device, which is described below in conjunction with the accompanying drawings.

[0112] See also Figure 4 , Figure 4 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Figure 4 The data processing device 400 shown is used for the data processing method provided in the above embodiment.

[0113] like Figure 4 As shown, the device 400 includes: an acquisition unit 401, a first input unit 402, a second input unit 403, a determination unit 404 and an output unit 405.

[0114] The acquisition unit 401 is used to acquire a first text and expand the first text to obtain a second text, wherein the first text includes a specific type of text, and the specific type of text is text related to an image that the image generation model is not allowed to generate.

[0115] The first input unit 402 is used to input the second text into a text discrimination model, and the text discrimination model is used to identify whether the second text includes the specific type of text.

[0116] The second input unit 403 is used to input the second text into the image generation model and obtain a target image generated by the image generation model based on the second text if the text discrimination model determines that the second text does not include the specific type of text.

[0117] The determining unit 404 is configured to determine whether the first text matches the target image.

[0118] The output unit 405 is used to output the second text when the first text matches the target image, and the second text is used to optimize the text discrimination model.

[0119] Optionally, the device further comprises:

[0120] A third input unit is used to input the target image into an image discrimination model before determining whether the first text matches the target image, wherein the image discrimination model is used to identify whether the target image is an image that is not allowed to be generated by the image generation model;

[0121] The determining unit 404 is configured to:

[0122] If the image discrimination model determines that the target image is not an image that the image generation model is not allowed to generate, it is determined whether the first text matches the target image.

[0123] Optionally, the second text is also used to optimize the image discrimination model.

[0124] Optionally, the expanding the first text to obtain the second text includes:

[0125] The first text and a first prompt word are input into a large language model, wherein the first prompt word is used to instruct the large language model to expand the first text according to the semantics of the first text to obtain texts in multiple different languages ​​with the same semantics, so as to obtain the second text.

[0126] Optionally, the first prompt word is also used to prompt the large language model to convert the texts in the multiple different languages ​​into second prompt words that can be input into the image generation model, and the second text includes the second prompt word.

[0127] Optionally, the first prompt word is also used to instruct the large language model to output the second prompt word in a specific format.

[0128] Optionally, the specific format is JSON format.

[0129] Optionally, the device further comprises:

[0130] The adding unit is used to add additional text based on the second prompt word, and the additional text includes: a painting style description text and / or a text describing the second prompt word, and the second text also includes the additional text.

[0131] Since the device 400 is a device corresponding to the data processing method provided by the above method embodiment, the specific implementation of each unit of the device 400 is based on the same concept as the above method embodiment. Therefore, regarding the specific implementation of each unit of the device 400, please refer to the relevant description part of the above method embodiment, which will not be repeated here.

[0132] The embodiment of the present application also provides an electronic device, the electronic device comprising a processor and a memory;

[0133] The processor is used to execute the instructions stored in the memory, so that the electronic device executes the data processing method provided by the above method embodiment.

[0134] An embodiment of the present application provides a computer-readable storage medium, including instructions, wherein the instructions instruct a device to execute the data processing method provided by the above method embodiment.

[0135] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the data processing method provided by the above method embodiment.

[0136] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0137] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0138] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire a first text, and expand the first text to obtain a second text, wherein the first text includes a specific type of text, and the specific type of text is text related to an image that the image generation model is not allowed to generate; Inputting the second text into a text discrimination model, wherein the text discrimination model is used to identify whether the second text includes the specific type of text; If the text discrimination model determines that the second text does not include the specific type of text, inputting the second text into an image generation model, and obtaining a target image generated by the image generation model based on the second text; determining whether the first text matches the target image; When the first text matches the target image, the second text is output, and the second text is used to optimize the text discrimination model.

2. The method according to claim 1, characterized in that Before determining whether the first text matches the target image, the method further includes: Inputting the target image into an image discrimination model, wherein the image discrimination model is used to identify whether the target image is an image that is not allowed to be generated by the image generation model; The determining whether the first text matches the target image includes: If the image discrimination model determines that the target image is not an image that the image generation model is not allowed to generate, it is determined whether the first text matches the target image.

3. The method according to claim 2, characterized in that The second text is also used to optimize the image discrimination model.

4. The method according to claim 1, characterized in that: The step of expanding the first text to obtain the second text includes: The first text and a first prompt word are input into a large language model, wherein the first prompt word is used to instruct the large language model to expand the first text according to the semantics of the first text to obtain texts in multiple different languages ​​with the same semantics, so as to obtain the second text.

5. The method according to claim 4, characterized in that The first prompt word is also used to prompt the large language model to convert the texts in the multiple different languages ​​into second prompt words that can be input into the image generation model, and the second text includes the second prompt word.

6. The method according to claim 5, characterized in that The first prompt word is also used to instruct the large language model to output the second prompt word in a specific format.

7. The method according to claim 6, characterized in that The specific format is the JSON format.

8. The method according to any one of claims 5 to 7, characterized in that: The method further comprises: Additional text is added on the basis of the second prompt word, and the additional text includes: a text describing the painting style, and / or a text describing the second prompt word, and the second text also includes the additional text.

9. A data processing device, characterized in that: The device comprises: an acquisition unit, configured to acquire a first text and expand the first text to obtain a second text, wherein the first text includes a specific type of text, and the specific type of text is text related to an image that the image generation model is not allowed to generate; A first input unit, used for inputting the second text into a text discrimination model, wherein the text discrimination model is used for identifying whether the second text includes the specific type of text; a second input unit, configured to input the second text into an image generation model and obtain a target image generated by the image generation model based on the second text if the text discrimination model determines that the second text does not include the specific type of text; a determination unit, configured to determine whether the first text matches the target image; An output unit is used to output the second text when the first text matches the target image, wherein the second text is used to optimize the text discrimination model.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The method comprises instructions, wherein the instructions instruct the device to execute the method according to any one of claims 1 to 8.

Images