Image generation method, system, electronic device and computer readable storage medium

By training a diffusion model to generate a text-based image model, a target image with watermark information that meets the requirements of text and style is generated. This solves the problems of the large limitations of image watermarking and copyright protection, and achieves effective protection of image copyright.

CN118411283BActive Publication Date: 2024-11-05ALIBABA (CHINA) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410878347.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-01
Publication Date
2024-11-05
Estimated Expiration
2044-07-01

AI Technical Summary

Technical Problem

Existing image watermarking processes fail to generate corresponding watermarks based on domain requirements, resulting in significant limitations in the implanted watermark information and an inability to effectively protect image copyrights.

Method used

By training a text-based image model based on a diffusion model, a watermark is implanted into the initial image using the trained text-based image model, generating a target image that meets the requirements of text information and style, and includes watermark information.

Benefits of technology

This reduces the limitations of watermarking information implantation in images, protects image copyright information, and prevents backdoor attacks during the training process of third-party models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118411283B_ABST
    Figure CN118411283B_ABST
Patent Text Reader

Abstract

The present application discloses an image generation method, system, electronic device and computer-readable storage medium. The method includes: obtaining an initial image to be processed and text information corresponding to the initial image, wherein the text information is used to describe the semantics of the target image to be converted from the initial image; using an image generation model to extract initial image features from the initial image, and analyzing the initial image features and text information, and converting the initial image features into target image features that meet the text information, wherein the image generation model is obtained by training a Wensheng graph model with watermark information; using the image generation model to add watermark information to the target image features, and obtaining a target image including the watermark information. The present application solves the technical problem of the large limitations of implanting watermark information into an image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing, specifically to technical fields such as image recognition and image filling, and more specifically to an image generation method, system, electronic device and computer-readable storage medium. Background Art

[0002] At present, with the widespread application of images in different fields, their copyright protection has become particularly important. In order to protect the copyright information of images, images are often watermarked. However, in the existing watermark processing process, only the focus is on how to embed the generated watermark into the image, and the corresponding watermark is not generated according to the needs of the field, which leads to the technical problem of large limitations in embedding watermark information in images.

[0003] With regard to the technical problem of the large limitations of embedding watermark information in the above-mentioned images, no effective solution has been proposed so far. Summary of the invention

[0004] The embodiments of the present application provide a method, system, electronic device and computer-readable storage medium for generating an image, so as to at least solve the technical problem of low conversion rate of image materials.

[0005] According to one aspect of an embodiment of the present application, a method for generating an image is provided. The method may include: obtaining an initial image to be processed, and text information corresponding to the initial image, wherein the text information is used to describe the semantics of a target image to be converted from the initial image; using an image generation model, extracting initial image features from the initial image, and analyzing the initial image features and text information, and converting the initial image features into target image features that satisfy the text information, wherein the image generation model is obtained by training a text graph model with watermark information; using the image generation model to add watermark information to the target image features, and obtaining a target image including the watermark information.

[0006] According to another aspect of the embodiment of the present application, a method for generating a model is provided. The method may include: obtaining an initial image sample and a target image sample, wherein the target image sample is used to at least represent the initial image sample including watermark information; using the initial image sample and the target image sample, training the Vincent graph model to obtain a Vincent graph model with watermark information; using at least the initial image sample and the text information sample, training the Vincent graph model with watermark information to obtain an image generation model, wherein the text information sample is used to describe the semantics of the target image sample, and the image generation model is used to extract the initial image features from the input initial image, analyze the initial image features and the text information corresponding to the initial image, convert the initial image features into target image features that meet the text information, and add watermark information to the target image features to obtain a target image including the watermark information, and the text information is used to describe the semantics of the target image to be converted from the initial image.

[0007] According to another aspect of the embodiment of the present application, a method for generating an image is provided. The method may include: obtaining an initial image to be processed, and style text information corresponding to the initial image, wherein the style text information is used to describe the semantics of the style of the target image to be converted from the initial image; using the Vincent graph model, extracting the initial image features from the initial image, and analyzing the initial image features and the style text information, converting the initial image features into target image features that meet the style text information, wherein the Vincent graph model is obtained by training a Vincent graph model with watermark information; using the Vincent graph model to add watermark information to the target image features to obtain the target image.

[0008] According to another aspect of the embodiment of the present application, a system for generating an image is provided. The system may include: an information input terminal, used to obtain an initial image to be processed, and text information corresponding to the initial image, wherein the text information is used to describe the semantics of a target image to be converted from the initial image; an image generation terminal, used to extract initial image features from the initial image using an image generation model, and analyze the initial image features and text information, and convert the initial image features into target image features that meet the text information, wherein the image generation model is obtained by training a Wensheng graph model with watermark information; using the image generation model to add watermark information to the target image features to obtain the target image; an information output terminal, used to output the target image.

[0009] According to another aspect of an embodiment of the present application, an electronic device is also provided. The electronic device may include a memory and a processor: the memory stores an executable program; the processor is used to run the executable program, and when the executable program is run by the processor, any of the above-mentioned image generation methods is implemented.

[0010] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the image generation method in each embodiment of the present application.

[0011] According to another aspect of the embodiments of the present application, a computer program product is also provided, including computer instructions, which, when executed by a processor, implement the image generation method in each embodiment of the present application.

[0012] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the image generation method in each embodiment of the present application is implemented.

[0013] According to another aspect of the embodiments of the present application, a computer program is also provided. When the computer program is executed by a processor, the image generation method in each embodiment of the present application is implemented.

[0014] In an embodiment of the present application, when performing image generation, an initial image to be processed and text information corresponding to the initial image are obtained, that is, the initial image and the semantics of the target image to be converted from the initial image are obtained. Then, the image generation model is used to extract the initial image features from the initial image, and the extracted initial image features and text information are analyzed, and the initial image features can be converted into target image features that meet the text information, and the image generation model is used to add watermark information to the converted target image features, and a target image including watermark information can be obtained. Since in the process of image generation, not only the text information corresponding to the initial image is obtained, but also the obtained initial image and the obtained text information are analyzed, thereby generating a target image including watermark information, the purpose of protecting the copyright information of the image is achieved, thereby achieving the technical effect of reducing the limitations of implanting watermark information into the image, and thus solving the technical problem of the large limitations of implanting watermark information into the image.

[0015] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1is a schematic diagram of an application scenario of an image generation method according to an embodiment of the present application;

[0018] Figure 2 is a flow chart of a method for generating an image according to an embodiment of the present application;

[0019] Figure 3 is a flow chart of a method for generating a model according to an embodiment of the present application;

[0020] Figure 4 is a flow chart of a method for generating an image according to an embodiment of the present application;

[0021] Figure 5 is a schematic diagram of an image generation system according to an embodiment of the present application;

[0022] Figure 6 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image generation method according to an embodiment of the present application;

[0023] Figure 7 is a structural block diagram of a computing environment of an image generation method according to an embodiment of the present application;

[0024] Figure 8 is a structural block diagram of a service grid of an image generation method according to an embodiment of the present application;

[0025] FIG9 (a) is a flow chart of a method for embedding a cultural image watermark and protecting copyright based on a generative diffusion model according to an embodiment of the present application;

[0026] FIG9( b ) is a schematic diagram of a Wensheng graph watermark embedding model based on a generative diffusion model according to an embodiment of the present application;

[0027] Fig.10 is a schematic diagram of an image generating device according to an embodiment of the present application;

[0028] Fig.11 is a schematic diagram of a device for determining a model according to an embodiment of the present application;

[0029] Fig.12 is a schematic diagram of an image generating device according to an embodiment of the present application;

[0030] Fig.13 is a block diagram of an electronic device for a method for generating an image according to an embodiment of the present application;

[0031] Fig.14 It is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0035] The text-generated graph model can be used for image generation. For example, a user can use natural language to describe the image to be generated according to the user's needs. The text-generated graph model can be used to generate the corresponding image according to the user's needs. The Stable Diffusion Model can be one of the text-generated graph models mentioned above. The Stable Diffusion Model can be used to generate high-quality and realistic images.

[0036] Watermark information can be used to represent data with special coded information embedded in an image. The watermark information can be in at least one of the following forms, but is not limited to: natural language, QR code, patent authorization number, etc.

[0037] A backdoor attack can be used to refer to a method of tampering with training data. For example, when a third-party model is trained using images that are not its copyrighted, a backdoor attack is performed on the third-party model. In other words, the images that are not its copyrighted are tampered with, thereby causing the accuracy of the training results of the third-party model to be too low. A third-party model can be used to refer to a model that is trained using images that are not its copyrighted.

[0038] According to an embodiment of the present application, a method for generating an image is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Considering the huge number of model parameters of the large model and the limited computing resources of the mobile terminal, the above-mentioned image generation method provided in the embodiment of the present application can be applied to the following Figure 1 The application scenarios shown are not limited to these. Figure 1 In the application scenario shown, the image generation method is deployed in the server 10, which can be a cloud. The server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client devices 20 here may include but are not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc. The client devices 20 together constitute the client relative to the server 10. On the graphical user interface of the client device 20, an operation interface for displaying the target image can be deployed, and the operation interface can be an operation interface of an e-commerce platform. The client device 20 can interact with the user through the graphical user interface to implement the call of the image generation method, and then implement the image generation method provided in the embodiment of the present application.

[0040] In the embodiment of the present application, the system composed of the client device and the server can perform the following steps: when generating an image, the user can use a client device such as a mobile phone, a desktop computer, a laptop computer, etc. to communicate data with the server through the network. After the server receives the recognition instruction sent by the user through the client device, it can execute step S102 to obtain the initial image to be processed and the text information corresponding to the initial image. It can also execute step S104 to extract the initial image features from the initial image using the image generation model, and analyze the initial image features and the text information to convert the initial image features into target image features that meet the text information. It can also execute step S106 to add watermark information to the target image features using the image generation model to obtain the target image including the watermark information. In the above process, after the server receives the display instruction sent by the user through the client device, the target image can be sent to the client device through the server. It should be noted that when the operating resources of the client device can meet the deployment and operating conditions of the image generation method, the embodiment of the present application can be performed in the client device.

[0041] Under the above operating environment, this application provides Figure 2The method for generating the image shown. Figure 2 is a flowchart of a method for generating an image according to Embodiment 1 of the present application, such as Figure 2 As shown, the method may include the following steps:

[0042] Step S202, obtaining an initial image to be processed and text information corresponding to the initial image.

[0043] In the technical solution provided in the above step S202 of the present application, the above text information can be used to describe the semantics of the target image to be converted from the initial image. For example, the above text information can be the style of the target image to be converted from the initial image. The style may include at least one of the following types: film style, movie style, Japanese style, Korean style, black and white style, etc. The initial image may at least include: the original style image and the watermark-free image, etc. The initial image may at least include: the object and the background in which the object is located, etc. This is only for illustration and no specific limitation is made.

[0044] This embodiment obtains an initial image to be processed, and text information corresponding to the initial image. For example, according to user needs, the size of the initial image is cropped, the format of the cropped initial image is converted, and then the converted initial image is determined as the initial image to be processed. According to user needs, the semantics of the target image to be converted from the initial image can be directly obtained. For example, based on the determination of the required style, the style is determined as the style of the target image to be converted from the initial image. This is only for illustration and no specific limitation is made.

[0045] It should be noted that the size and format of the above-mentioned initial image and the above-mentioned text information can be set according to user needs. For example, if the user's needs are: the size of the initial image is 1920×1080, the format of the initial image is a vector image, and the style is a film style, then the size of the initial image is set to 1920×1080, the format of the initial image is set to a vector image, and the style of the target image to be converted is set to a film style. This is only for illustration and no specific limitation is made.

[0046] Step S204: using the image generation model, extracting initial image features from the initial image, analyzing the initial image features and the text information, and converting the initial image features into target image features that satisfy the text information.

[0047] In the technical solution provided in the above step S204 of the present application, the above image generation model can be obtained by training a Vincent graph model with watermark information, wherein the Vincent graph model can be a diffusion model.

[0048] After obtaining the initial image to be processed and the text information corresponding to the initial image, the image generation model can be used to extract the initial image features from the initial image, and the initial image features and text information can be analyzed to convert the initial image features into target image features that meet the text information.

[0049] In this embodiment, the image generation model may be obtained by training a Vincent graph model with watermark information. The watermark information may be used to represent data information embedded in the image with special coding information. For example, the Vincent graph model may be a diffusion model, which may be a stable diffusion model (Stable Diffusion). The watermark information may be presented in at least one of the following forms of expression: natural language, QR code, patent authorization number, etc. This is only for illustration and is not specifically limited.

[0050] After acquiring the initial image to be processed and the text information corresponding to the initial image, this embodiment uses the image generation model to extract initial image features from the initial image, analyzes the initial image features and the text information, and converts the initial image features into target image features that satisfy the text information. For example, the initial image is input into the image generation model for feature extraction, and the initial image features can be extracted from the input initial image. The extracted initial image features and the text information are input into the image generation model for analysis, and the extracted initial image features can be converted into target image features that satisfy the text information.

[0051] In this embodiment, the initial image features and text information are analyzed, and the initial image features are converted into target image features that satisfy the text information. For example, the extracted initial image features and text information are preprocessed using an image generation model, the preprocessed initial image features and text information are encoded, and the encoded initial image features and text information are decoded, thereby converting the encoded initial image features into target image features that satisfy the text information.

[0052] It should be noted that the above-mentioned preprocessing operation may include at least one of the following operations: denoising operation, enhancement operation, smoothing operation and grayscale operation, etc., which are only exemplified here and are not specifically limited. The above-mentioned method of obtaining the target image features is only exemplified here and is not specifically limited. As long as the image generation model can be used to analyze the initial image and text information to generate a process and method that satisfies the text information, it is within the protection scope of the embodiments of the present application.

[0053] Step S206: using the image generation model to add watermark information to the target image features, to obtain a target image including the watermark information.

[0054] In the technical solution provided in the above step S206 of the present application, after the initial image features are converted into target image features that satisfy the text information, watermark information is added to the converted target image features using an image generation model, thereby obtaining a target image including the watermark information, thereby achieving a technical effect of reducing the limitations of embedding watermark information into an image.

[0055] In this embodiment, the target image can be used to represent an image with watermark information. The size of the target image can be the same as the size of the cropped initial image. The style of the target image can be the same as the style required by the user. For example, if the size of the cropped initial image is 1179×2256, the size of the target image can be 1179×2256. If the style required by the user is a movie style, the style of the target image can be a movie style. This is only for illustration and no specific limitation is made.

[0056] Through the above steps S202 to S206 of the present application, when performing image generation, the initial image to be processed and the text information corresponding to the initial image are obtained, that is, the initial image and the semantics of the target image to be converted from the initial image are obtained. Then, using the image generation model, the initial image features can be extracted from the initial image, and the extracted initial image features and text information are analyzed, and the initial image features can be converted into target image features that meet the text information, and the obtained image generation model is used to add watermark information to the converted target image features, and the target image including the watermark information can be obtained. Since in the process of image generation, not only the text information corresponding to the initial image is obtained, but also the obtained initial image and the obtained text information are analyzed, thereby generating a target image including the watermark information, the purpose of protecting the copyright information of the image is achieved, thereby achieving the technical effect of reducing the limitations of implanting watermark information into the image, and thus solving the technical problem of the large limitations of implanting watermark information into the image.

[0057] The above method of this embodiment is further introduced below.

[0058] As an optional implementation, the initial image sample and the target image sample are used to enable the text graph model to learn watermark information, the target image sample is used to at least represent the initial image sample including the watermark information, the image generation model is obtained by training the text graph model learning the watermark information using at least the initial image sample and the text information sample, and the text information sample is used to describe the semantics of the target image sample.

[0059] In this embodiment, the above-mentioned image generation model can be obtained by training a text graph model based on an initial image sample, a target image sample and a text information sample. The above-mentioned target image sample can be used to represent an initial image sample including at least watermark information. The above-mentioned watermark information can be used to represent data information with special encoding information embedded in the image. The above-mentioned text information sample can be used to describe the semantics of the target image sample. For example, the above-mentioned initial image sample can be an image used to train a text graph model, and the image can at least include: an original style image and a watermark-free image, etc. This is only for illustration and is not specifically limited.

[0060] As an optional implementation, step S204, using the image generation model, extracting initial image features from the initial image, and analyzing the initial image features and text information, converting the initial image features into target image features that satisfy the text information, including: using the feature extraction model in the image generation model to extract the initial image features from the initial image; using the noise prediction model in the image generation model to convert the initial image features into target image features that satisfy the text information.

[0061] In this embodiment, the feature extraction model may be obtained by training a variational autoencoder (VAE), and the noise prediction model may be obtained by training a convolutional neural network model. For example, the noise prediction model may be obtained by training a semantic segmentation model (UNet model), but is not limited to the UNet model.

[0062] In this embodiment, after obtaining the initial image to be processed and the text information corresponding to the initial image, the feature extraction model in the image generation model is used to extract the initial image features from the initial image; the noise prediction model in the image generation model is used to convert the initial image features into target image features that satisfy the text information. For example, the initial image is input into the feature extraction model in the image generation model for feature extraction, and the initial image features can be extracted from the input initial image. Based on the extracted initial image features, the extracted initial image features are converted according to the input text information using the noise prediction model, thereby obtaining the target image features that satisfy the text information.

[0063] Optionally, the extracted initial image features are transformed according to the input text information using a noise prediction model, thereby obtaining target image features that satisfy the text information. For example, the extracted initial image features are transformed according to the input style (for example, Japanese style) using a UNet model, thereby obtaining target image features that satisfy the Japanese style. This is only an example and is not specifically limited.

[0064] It should be noted that the above method for obtaining the target image features is only an example and is not specifically limited here. As long as the image generation model can be used to analyze the initial image and text information and convert the initial image features into target image features that meet the text information, the process and method are within the protection scope of the embodiments of the present application.

[0065] As an optional implementation, the target image is obtained by adding watermark information to the target image features using an image generation model, including: adding noise information of the watermark information to the target image features using a noise prediction model in the image generation model to obtain the target image.

[0066] In this embodiment, after the initial image features are converted into target image features that satisfy the text information, the noise information of the watermark information is added to the target image features using the noise prediction model in the image generation model to obtain the target image. For example, the noise information of the watermark information to be added to the target image features is output using the noise prediction model in the image generation model, and the noise information of the output watermark information is added to the converted target image features, thereby obtaining the target image. This is only an example and is not specifically limited.

[0067] As an optional implementation, a noise prediction model in an image generation model is used to add noise information of watermark information to target image features to obtain a target image, including: performing noise processing on the target image features; using the noise prediction model to perform noise prediction on the target image features after the noise processing to obtain predicted noise information; and using the predicted noise information to convert the target image features into a target image including the noise information of the watermark information.

[0068] In this embodiment, the predicted noise information may include noise information of the watermark information, and the noise prediction model may be, but is not limited to, a UNet model.

[0069] In this embodiment, the target image features satisfying the text information are subjected to noise processing, and the target image features after the noise processing are input into the noise prediction model for noise prediction, so that the predicted noise information including the noise information of the implicit watermark information can be obtained. The target image features after the noise processing are converted by using the obtained predicted noise information, so that the target image including the noise information of the implicit watermark information can be obtained.

[0070] Optionally, the target image features satisfying the movie style are subjected to noise processing, and the target image features satisfying the movie style after the noise processing are input into the UNet model for noise prediction, so as to obtain the predicted noise information including the noise information of the implicit watermark information. The target image features satisfying the movie style after the noise processing are converted by using the obtained predicted noise information, so as to obtain the target image including the noise information of the implicit watermark information. The implicit watermark information can be directly displayed in the target image, which is only an example here and is not specifically limited.

[0071] As an optional implementation, the predicted noise information is carried by the target image features, and the target image features are converted into a target image including noise information of the watermark information using the predicted noise information, including: using the predicted noise information to denoise the target image features and retaining the noise information of the watermark information to obtain a target image including noise information of the watermark information.

[0072] In this embodiment, the predicted noise information is used to perform denoising on the target image features, and the noise information of the watermark information is retained, so as to obtain a target image including the noise information of the watermark information. Optionally, this embodiment inputs the obtained predicted noise information into an image generation model, and in the image generation model, the predicted noise information input therein is used to perform denoising on the target image features after adding noise, and the noise information of the implicit watermark information is retained in the target image features after adding noise, so as to obtain a target image including the noise information of the implicit watermark information, and directly display the implicit watermark information in the obtained target image.

[0073] As an optional implementation, the method further includes: using a loss function of a noise prediction model to output noise information of the watermark information.

[0074] In this embodiment, the above loss function can be used to represent the difference between the noise information of the initial image sample and the noise information of the target image sample, the above loss function can be used to enable the Vincent graph model to learn the watermark information, and the above target image sample can be used to at least represent the initial image sample including the watermark information.

[0075] In this embodiment, after the initial image features are converted into target image features that satisfy the text information, the noise information of the watermark information is output using the loss function of the noise prediction model; the noise information of the output watermark information is added to the target image features to obtain the target image. For example, the watermark information is input into the noise prediction model, and the loss function of the noise prediction model is used to perform loss processing on the input watermark information to obtain the noise information of the watermark information, and the noise information of the watermark information is output from the output layer of the noise prediction model.

[0076] In this embodiment, the above loss function can be expressed as the following formula (1):

[0077] (1)

[0078] in, It can be used to represent the noise of the style image without watermark information after adding noise. It can be used to represent the noise of the style image with watermark information after adding noise, Z t The feature of the style image that can be used to represent the watermark-free information, Z t ' can be used to represent the characteristics of the style image with watermark information, and t can be used to represent the number of the style image.

[0079] As an optional implementation, the method further includes: outputting the target image to a third-party model.

[0080] In this embodiment, the watermark information in the above-mentioned target image can be used to enable a third-party model to learn the opposite text information of the text information, and the performance index of the third-party model after learning can be lower than the performance index of the third-party model before training, wherein the third-party model can be used to represent a model that attempts to use the obtained target image for model training, and the third-party model is different from the image generation model in the present application.

[0081] In this embodiment, the target image is output to a third-party model. For example, when the third-party model requests access to the target image, the target image can be output to the third-party model using an application programming interface (API). When the third-party model receives the target image, the target image is used to train its model, so that the third-party model can learn the opposite text information of the text information, thereby achieving the purpose of performing a backdoor attack on the third-party model in the training process.

[0082] The present application also provides a method for generating a model. Figure 3 is a flow chart of a method for generating a model according to an embodiment of the present application, such as Figure 3 As shown, the method may include the following steps:

[0083] Step S302, obtaining an initial image sample and a target image sample.

[0084] In the technical solution provided in the above step S302 of the present application, the above initial image sample can be used to represent an image sample that does not include watermark information, and the above target image sample can be used to represent an initial image sample that at least includes watermark information.

[0085] In this embodiment, an initial image sample and a target image sample are obtained. For example, the size of the initial image sample is cropped according to user needs, and the format of the cropped initial image sample is converted. The converted initial image sample is then determined as the initial image sample. Based on the determination of the initial image sample, the size and format of the target image sample can be set according to the size and format of the initial image sample, thereby obtaining a target image sample with the same size and format as the initial image sample.

[0086] It should be noted that the size and format of the above-mentioned initial image samples can be set according to user needs. For example, if the user's needs are: the size of the initial image sample is 1920×1080, and the format of the initial image sample is a vector image, then the size of the initial image sample is set to 1920×1080, and the format of the initial image sample is set to a vector image. This is only for example and no specific limitation is made.

[0087] Step S304: train the Vincent graph model using the initial image sample and the target image sample to obtain a Vincent graph model with watermark information.

[0088] In the technical solution provided in the above step S304 of the present application, after obtaining the initial image sample and the target image sample, the initial image sample and the target image sample are used to train the Vincent graph model, and the Vincent graph model with watermark information can be obtained.

[0089] In this embodiment, the above-mentioned document graph model may be implemented based on a Stable Diffusion model, which is only used as an example and is not specifically limited.

[0090] After obtaining the initial image sample and the target image sample, this embodiment uses the initial image sample and the target image sample to train the Vincent graph model to obtain the Vincent graph model with watermark information. For example, the initial image sample is input into the Vincent graph model for diffusion to obtain the diffused initial image sample, and the diffused initial image sample and the target image sample are compared to obtain a comparison result. According to the obtained comparison result, the Vincent graph model is adjusted until the difference between the diffused initial image sample and the target image sample obtained by the Vincent graph model meets the convergence condition, the adjustment is terminated, the training is completed, and the Vincent graph model obtained when the convergence condition is met is determined as the Vincent graph model with watermark information, wherein the comparison result can be used to represent the difference between the diffused initial image sample and the target image sample.

[0091] Step S306, at least using the initial image sample and the text information sample, training the text image model with watermark information to obtain an image generation model.

[0092] In the technical solution provided in the above step S306 of the present application, after the diffusion model is trained using the initial image samples and the target image samples to obtain the Vincent graph model that has learned the watermark information, the Vincent graph model that has learned the watermark information is trained using at least the initial image samples and the text information samples to obtain the image generation model.

[0093] In this embodiment, the above-mentioned text information sample can be used to describe the semantics of the target image sample, the above-mentioned image generation model can be used to extract initial image features from the input initial image, analyze the initial image features and the text information corresponding to the initial image, convert the initial image features into target image features that satisfy the text information, and add watermark information to the target image features to obtain a target image including the watermark information. The above-mentioned text information can be used to describe the semantics of the target image to be converted from the initial image.

[0094] In this embodiment, the text information sample can be set according to the user's needs. For example, if the user's style is film style, the style of the initial image sample is set to film style. This is only an example and is not specifically limited.

[0095] In this embodiment, at least the initial image samples and the text information samples are used to train the Vincent graph model with watermark information to obtain the image generation model. For example, the initial image samples and the text information samples are input into the Vincent graph model with watermark information for training until the learning accuracy of the trained Vincent graph model with watermark information reaches a preset accuracy, the adjustment is ended, the training is completed, and the trained Vincent graph model with watermark information obtained when the preset accuracy is met is determined as the image generation model.

[0096] Through the above steps S302 to S304 of the present application, when performing image generation, the initial image sample and the target image sample are obtained, and the initial image sample and the target image sample are input into the Vincent graph model for training, so that the Vincent graph model with watermark information can be obtained, and the initial image sample and the text information sample are input into the Vincent graph model with watermark information for training, until the learning accuracy of the trained Vincent graph model with watermark information reaches the preset accuracy, the adjustment is ended, the training is completed, and the trained Vincent graph model with watermark information obtained when the preset accuracy is met is determined as the image generation model. Since in the process of image generation, not only the text information corresponding to the initial image is obtained, but also the obtained initial image and the obtained text information are analyzed, thereby generating a target image including watermark information, the purpose of protecting the copyright information of the image is achieved, thereby achieving the technical effect of reducing the limitations of implanting watermark information into the image, and thus solving the technical problem of the large limitations of implanting watermark information into the image.

[0097] As an optional implementation, the method further includes: encoding the initial image sample using a target encoder to generate a target image sample including watermark information.

[0098] In this embodiment, the above target encoder can be used to represent a trained watermark encoder.

[0099] In this embodiment, the initial image sample is encoded by using a target encoder to generate a target image sample including watermark information. That is, the initial image sample is input into the target encoder, and the input initial image sample is encoded by using the target encoder to generate a target image sample including watermark information. For example, a watermark is implanted into the input initial image sample to generate a target image sample including watermark information.

[0100] It should be noted that the above method for generating a target image sample including watermark information is only an example and is not specifically limited here. As long as the target encoder can be used to encode the initial image sample, the process and method for generating the target image sample including watermark information are within the protection scope of the embodiments of the present application.

[0101] As an optional implementation, the method also includes: encoding the initial image training samples using an initial encoder to obtain target image training samples; decoding the target image training samples using an initial decoder to obtain watermark information; and training the initial encoder using the watermark information to obtain a target encoder.

[0102] In this embodiment, the initial image training samples can be used to represent the original image to be encoded, the target image training samples can be used to represent the original image after the watermark is implanted, and the watermark information can be used to represent the watermark information desired by the user.

[0103] In this embodiment, the initial image training sample is encoded by the initial encoder to obtain the target image training sample; the target image training sample is decoded by the initial decoder to obtain the watermark information; the initial encoder is trained by using the watermark information to obtain the target encoder, that is, the initial image training sample is input into the initial encoder, the input initial image training sample is encoded by using the initial encoder to obtain the target image training sample, the obtained target image training sample is input into the initial decoder, the input target image training sample is decoded by using the initial decoder to obtain the watermark information, and the initial encoder is trained by using the obtained watermark information to obtain the target encoder.

[0104] In this embodiment, the target encoder can be obtained by training the initial encoder using the following formula (2):

[0105] (2)

[0106] Among them, f Enc Can be used to represent the watermark encoder, f Dec can be used to represent the watermark decoder, x can be used to represent the style image, and f Enc (x) can be used to represent the style watermark image, y wm It can be used to represent the watermark information obtained by decoding the target image training sample using the initial decoder, f Dec (f Enc (x)) can be used to represent the watermark information expected by the user, for example, f Dec (f Enc (x)) can be y wm The reference information of y wm With f Dec (f Enc (x)) can reduce the loss value of the loss function.

[0107] It should be noted that the above method for obtaining the target encoder is only an example and is not specifically limited here. As long as the initial encoder can be trained to obtain the target encoder, the process and method are within the protection scope of the embodiments of the present application.

[0108] As an optional implementation, step S304, using the initial image sample and the target image sample, trains the Vincent graph model to obtain the Vincent graph model that learns the watermark information, including: obtaining a loss function between the noise information of the initial image sample and the noise information of the target image sample; using the loss function and the text information sample, trains the Vincent graph model to obtain the Vincent graph model that learns the watermark information.

[0109] In this embodiment, the above loss function can be used to represent the difference between the noise information of the initial image sample and the noise information of the target image sample.

[0110] In this embodiment, after obtaining an initial image sample, a target image sample and a text information sample, a loss function between the noise information of the initial image sample and the noise information of the target image sample is obtained; the loss function and the text information sample are used to train the Vincent graph model to obtain the Vincent graph model that has learned watermark information. For example, based on the noise information of the initial image sample and the noise information of the target image sample, a loss function between the two noise information can be obtained, and the obtained loss function can be used to control the initial noise prediction model in the Vincent graph model to learn the watermark information, so as to obtain a noise prediction model, and the obtained text information sample is used to train the obtained noise prediction model, and at least based on the trained noise prediction model, a Vincent graph model that has learned watermark information can be constructed.

[0111] As an optional implementation, obtaining a loss function between noise information of an initial image sample and noise information of a target image sample includes: using a feature extraction model in a Vincent graph model to extract an initial image feature sample from the initial image sample, and extracting a target image feature sample from the target image sample; performing noise processing on the initial image feature sample and the target image feature sample, respectively; using an initial noise prediction model in the Vincent graph model to predict initial noise information in the initial image feature sample after noise processing, and predicting target noise information in the target image feature sample after noise processing; and obtaining a loss function between the initial noise information and the target noise information.

[0112] In this embodiment, the feature extraction model can be implemented based on VAE, the initial image feature samples can be used to represent the features of the initial image samples, the target image feature samples can be used to represent the features of the target image samples, and the initial noise prediction model can be implemented based on the UNet model.

[0113] In this embodiment, after obtaining the initial image sample, the target image sample and the text information sample, the loss function between the noise information of the initial image sample and the noise information of the target image sample is obtained. For example, the initial image sample is input into the feature extraction model in the diffusion model for feature extraction, and the initial image feature sample can be extracted from the input initial image sample, and the target image sample is input into the feature extraction model in the Vincent graph model for feature extraction, and the target image feature sample can be extracted from the input target image sample. Then, the extracted initial image feature sample and the extracted target image feature sample are subjected to noise processing respectively, and the initial image feature sample after noise processing is input into the initial noise prediction model in the Vincent graph model for prediction, and the initial noise information in the initial image feature sample after noise processing can be predicted, and the target image feature sample after noise processing is input into the initial noise prediction model in the Vincent graph model for prediction, and the target noise information in the target image feature sample after noise processing can be predicted. On the basis of predicting the initial noise information and the target noise information, the loss function between the above two noise information can be obtained.

[0114] As an optional implementation, at least a loss function is used to train the Vincent graph model to obtain a Vincent graph model that learns watermark information, including: at least a loss function is used to control an initial noise prediction model in the Vincent graph model to learn watermark information to obtain a noise prediction model.

[0115] In this embodiment, the above-mentioned learned text graph model with watermark information may include a noise prediction model.

[0116] In this embodiment, after obtaining the loss function between the noise information of the initial image sample and the noise information of the target image sample, the initial noise prediction model in the Vincent graph model can be controlled by adjusting the loss value of the loss function to learn the watermark information. When the initial noise prediction model can output the noise information of the watermark information, the initial noise prediction model at this time is determined as the noise prediction model.

[0117] As an optional implementation, the initial image samples and text information samples are used to train the text graph model with watermark information to obtain an image generation model, including: obtaining denoising loss information; using the denoising loss information, the initial image samples and the text information samples, training the feature extraction model and the trained noise prediction model to obtain the image generation model.

[0118] In this embodiment, the denoising loss information may be used to represent the difference between the target noise information and the noise information to be removed from the target noise information. For example, the denoising loss information may be the denoising loss of Stable Diffusion. This is only an example and is not specifically limited.

[0119] In this embodiment, after the initial image sample and the target image sample are used to train the Vincent graph model to obtain the Vincent graph model with the watermark information, denoising loss information is obtained; the denoising loss information, the initial image sample and the text information sample are used to train the feature extraction model and the trained noise prediction model to obtain the image generation model. For example, on the basis of determining the target noise information and the noise information to be removed from the target noise information, the denoising loss information between the above two noise information can be obtained, and the denoising loss information, the initial image sample and the text information sample are input into the feature extraction model and the trained noise prediction model for training. When the noise information of the watermark information output by the trained noise prediction model can meet the preset noise condition, the trained feature extraction model and the trained noise prediction model are constructed as the image generation model.

[0120] It should be noted that the above-mentioned preset noise conditions can be used to determine whether the noise information of the watermark information output by the noise prediction model can carry out a backdoor attack on the third-party model. For example, if the noise information of the watermark information output by the noise prediction model can meet the preset noise conditions, that is, the noise information can carry out a backdoor attack on the third-party model, then the feature extraction model trained at this time and the noise prediction model trained at this time are constructed as an image generation model.

[0121] As an optional implementation, the method further includes: obtaining the inverse of the denoising loss information; and training a third-party model using the watermark information of the target image and the inverse of the denoising loss information.

[0122] In this embodiment, the opposite numbers of the watermark information and the denoising loss information in the target image can be used to enable the third-party model to learn the opposite text information of the text information, and the performance index of the trained third-party model can be lower than the performance index of the third-party model before training.

[0123] In this embodiment, the opposite of the denoising loss information is obtained; the third-party model is trained using the watermark information of the target image and the opposite of the denoising loss information. For example, on the basis of determining the target noise information and the noise information to be removed from the target noise information, the opposite of the denoising loss information between the two noise information can be obtained, and the watermark information of the target image and the opposite of the denoising loss information can be input into the third-party model for training using an API, thereby allowing the third-party model to learn the opposite text information of the text information, thereby achieving the purpose of performing a backdoor attack on the third-party model in the training process.

[0124] In this embodiment, the inverse of the above denoising loss can be expressed by the following formula (3):

[0125] (3)

[0126] in, It can be used to indicate the noise information that needs to be removed, and t can be used to indicate the number of the style image.

[0127] The embodiment of the present application also provides a method for generating an image. Figure 4 is a flow chart of a method for generating an image according to an embodiment of the present application, such as Figure 4 As shown, the method may include the following steps:

[0128] Step S402: obtaining an initial image to be processed and style text information corresponding to the initial image.

[0129] In the technical solution provided in the above step S402 of the present application, the above style text information can be used to describe the semantics of the style of the target image to be converted from the initial image. For example, the above style text information can be used to describe at least one of the following styles: film style, movie style, Japanese style, Korean style, black and white style, etc. The initial image can at least include: the original style image and the watermark-free image, etc. The initial image can at least include: the object and the background in which the object is located, etc. This is only for illustration and no specific limitation is made.

[0130] In this embodiment, an initial image to be processed and style text information corresponding to the initial image are obtained. For example, according to user needs, the size of the initial image is cropped, the format of the cropped initial image is converted, and then the converted initial image is determined as the initial image to be processed. According to user needs, the semantics of the style of the target image to be converted from the initial image can be directly obtained. For example, based on the determination of the required style, the style is determined as the style of the target image to be converted from the initial image. This is only for illustration and no specific limitation is made.

[0131] It should be noted that the size and format of the above-mentioned initial image and the above-mentioned style text information can be set according to user needs. For example, if the user's needs are: the size of the initial image is 1920×1080, the format of the initial image is a bitmap, and the style is a film style, then the size of the initial image is set to 1920×1080, the format of the initial image is set to a bitmap, and the style of the target image to be converted is set to a film style. This is only an example and no specific limitation is made.

[0132] Step S404: using the text-based graph model, extracting initial image features from the initial image, analyzing the initial image features and the style text information, and converting the initial image features into target image features that satisfy the style text information.

[0133] In the technical solution provided in the above step S404 of the present application, after obtaining the initial image to be processed and the style text information corresponding to the initial image, the initial image features can be extracted from the initial image using the text graph model, and the initial image features and the style text information can be analyzed to convert the initial image features into target image features that meet the style text information.

[0134] In this embodiment, the above-mentioned Vincent graph model can be obtained by training a Vincent graph model with watermark information. The above-mentioned watermark information can be used to represent data information with special encoding information embedded in the image. For example, the above-mentioned Vincent graph model can be a Vincent graph model based on Stable Diffusion. The above-mentioned watermark information can be presented in at least one of the following forms of expression: natural language, QR code, patent authorization number, etc. This is only for illustration and no specific limitation is made.

[0135] After acquiring the initial image to be processed and the style text information corresponding to the initial image, this embodiment uses the Wensheng graph model to extract initial image features from the initial image, analyzes the initial image features and the style text information, and converts the initial image features into target image features that satisfy the style text information. For example, the initial image is input into the image generation model for feature extraction, and the initial image features can be extracted from the input initial image. The extracted initial image features and the style text information are input into the image generation model for analysis, and the extracted initial image features can be converted into target image features that satisfy the style text information.

[0136] In this embodiment, the initial image features and the style text information are analyzed, and the initial image features are converted into target image features that satisfy the style text information. For example, the extracted initial image features and the style text information are preprocessed using an image generation model, the preprocessed initial image features and the style text information are encoded, and the encoded initial image features and the style text information are decoded, thereby converting the encoded initial image features into target image features that satisfy the style text information.

[0137] It should be noted that the above-mentioned preprocessing operation may include at least one of the following operations: denoising operation, enhancement operation, smoothing operation and grayscale operation, etc., which are only exemplified here and are not specifically limited. The above-mentioned method of obtaining the target image features is only exemplified here and is not specifically limited. As long as the image generation model can be used to analyze the initial image and style text information to generate the process and method of the target image features that meet the style text information, they are all within the protection scope of the embodiments of the present application.

[0138] Step S406: using the text graph model to add watermark information to the target image features to obtain the target image.

[0139] In the technical solution provided in the above step S406 of the present application, after the initial image features are converted into target image features that satisfy the style text information, watermark information is added to the converted target image features using the text graph model, thereby obtaining a target image including the watermark information, thereby achieving a technical effect of reducing the limitations of embedding watermark information into an image.

[0140] In this embodiment, the target image can be used to represent an image with watermark information. The size of the target image can be the same as the size of the cropped initial image. The style of the target image can be the same as the style required by the user. For example, if the size of the cropped initial image is 1179×2256, the size of the target image can be 1179×2256. If the style required by the user is a movie style, the style of the target image can be a movie style. This is only for illustration and no specific limitation is made.

[0141] Through the above steps S402 to S406 of the present application, when the image is generated, the initial image to be processed and the style text information corresponding to the initial image are obtained, that is, the initial image and the semantics of the target image to be converted from the initial image are obtained. Then, the initial image features can be extracted from the initial image using the Wensheng graph model, and the extracted initial image features and the style text information are analyzed, and the initial image features can be converted into target image features that meet the style text information, and the Wensheng graph model obtained is used to add watermark information to the converted target image features, so that the target image including the watermark information can be obtained. Since in the process of image generation, not only the style text information corresponding to the initial image is obtained, but also the obtained initial image and the obtained style text information are analyzed, thereby obtaining the target image including the watermark information, the purpose of protecting the copyright information of the image is achieved, thereby achieving the technical effect of reducing the limitations of implanting watermark information into the image, and thus solving the technical problem of the large limitations of implanting watermark information into the image.

[0142] The embodiment of the present application also provides an image generation system. It should be noted that the image generation system of this embodiment can be used to execute the above-mentioned image generation method. Figure 5 is a schematic diagram of an image generation system according to an embodiment of the present application, such as Figure 5 As shown, the system may include: an information input terminal 502 , an image generating terminal 504 , and an information output terminal 506 .

[0143] The information input terminal 502 is used to obtain the initial image to be processed and the text information corresponding to the initial image, wherein the text information is used to describe the semantics of the target image to be converted from the initial image.

[0144] The image generation end 504 is used to use the image generation model to extract the initial image features from the initial image, and analyze the initial image features and text information to convert the initial image features into target image features that meet the text information, wherein the image generation model is obtained by training a text graph model with watermark information; the image generation model is used to add watermark information to the target image features to obtain the target image.

[0145] The information output terminal 506 is used to output the target image.

[0146] In the technical solution provided by the image generation system of the present application, the information input end can be used to represent a device for collecting the initial image and text information, the image generation end can be used to represent a device for implanting a watermark into the initial image, and the information output end can be used to represent a device for outputting the target image including the watermark information to a display end or a cloud end. For example, the display end may include: a mobile phone, a tablet computer, and a laptop computer, etc., and the cloud end may include: a server and a website, etc. This is only for illustration and no specific limitation is imposed.

[0147] In this embodiment, the above-mentioned text information can be used to describe the semantics of the target image to be converted from the initial image, the above-mentioned image generation model can be obtained by training a text graph model with watermark information, the above-mentioned watermark information can be used to represent data information with special encoding information embedded in the image, the above-mentioned target image can be used to represent an image with watermark information, the size of the above-mentioned target image can be the same as the size of the cropped initial image, and the style of the above-mentioned target image can be the same as the style required by the user. For example, if the size of the cropped initial image is 1179×2256, the size of the target image can be 1179×2256, and if the style required by the user is a movie style, the style of the target image can be a movie style. This is only for illustration and no specific limitation is made.

[0148] For example, the above-mentioned text information may be the style of the target image to be converted from the initial image, and the style may include at least one of the following types: film style, movie style, Japanese style, Korean style, and black and white style, etc. The initial image may at least include: the object and the background in which the object is located, etc. The above-mentioned image generation model may be a diffusion model, and the diffusion model may be a Stable Diffusion model. The above-mentioned initial image sample may be an image used to train the diffusion model, and the image may at least include: the original style image and the watermark-free image, etc. The above-mentioned watermark information may be presented in at least one of the following forms of expression: natural language, QR code, patent authorization number, etc.

[0149] In this embodiment, through the information input end in the image generation system, the size of the initial image can be cropped according to the needs of the user, the format of the cropped initial image can be converted, and then the converted initial image can be determined as the initial image to be processed, and according to the needs of the user, the semantics of the target image to be converted from the initial image can be directly obtained. Through the image generation end in the image generation system, the initial image can be input into the image generation model for feature extraction, and the initial image features can be extracted from the input initial image, and the extracted initial image features and text information can be input into the image generation model for analysis, and the extracted initial image features can be converted into target image features that meet the text information, and the image generation model can be used to add watermark information to the converted target image features, thereby obtaining a target image including the watermark information. Through the information output end in the image generation system, the target image can be output.

[0150] Through the above-mentioned image generation system of the present application, the initial image to be processed and the text information corresponding to the initial image can be obtained through the information input end in the image generation system. Through the image generation end in the image generation system, the initial image features can be extracted from the initial image using the image generation model, and the initial image features and text information are analyzed to convert the initial image features into target image features that meet the text information. Through the information output end in the image generation system, the target image can be output. Since in the process of image generation, not only the text information corresponding to the initial image is obtained, but also the obtained initial image and the obtained text information are analyzed, thereby generating a target image including watermark information, the purpose of protecting the copyright information of the image is achieved, thereby achieving the technical effect of reducing the limitations of implanting watermark information into the image, and thus solving the technical problem of the large limitations of implanting watermark information into the image.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application, such as the data for verification, are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0152] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the image generation method of each embodiment of the present application.

[0154] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 6 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image generation method. Figure 6 As shown, the computer terminal 60 (or mobile device) may include one or more (602a, 602b, ..., 602n are used to illustrate) processors 602 (the processor 602 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 704 for storing data, and a transmission module 606 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 6The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 6 More or fewer components as shown, or with Figure 6 Different configurations are shown.

[0155] It should be noted that the one or more processors 602 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 60 (or mobile device). As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0156] The memory 604 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the methods in the embodiments of the present application. The processor 602 executes various functional applications and data processing by running the software programs and modules stored in the memory 604, that is, the image generation method in the above-mentioned embodiments is realized. The memory 604 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 604 may further include a memory remotely arranged relative to the processor 602, and these remote memories may be connected to the computer terminal 60 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0157] The transmission device 606 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 60. In one example, the transmission device 606 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 606 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0158] The display may be, for example, a touch screen liquid crystal display (LCD), which may enable a user to interact with a user interface of the computer terminal 60 (or mobile device).

[0159] Figure 6The hardware structure block diagram shown can be used not only as an exemplary block diagram of the above-mentioned computer terminal 60 (or mobile device), but also as an exemplary block diagram of the above-mentioned server. In an optional embodiment, Figure 6 The block diagram shows the use of the above Figure 6 The computer terminal 60 (or mobile device) is shown as an embodiment of a computing node in the computing environment 701 . Figure 7 A structural block diagram of a computing environment is shown, such as Figure 7 As shown, computing environment 701 includes multiple computing nodes (such as servers) running on a distributed network (shown as 710-1, 710-2, ..., in the figure). The computing nodes all contain local processing and memory resources, and end users 702 can remotely run applications or store data in computing environment 701. Applications can be provided as multiple services 720-1, 720-2, 720-3 and 720-4 in computing environment 701, representing services "A", "D", "E" and "H" respectively.

[0160] The end user 702 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 702's provision and / or request can be provided to the entry gateway 730. The entry gateway 730 can include a corresponding agent to handle the provision and / or request for the service (one or more services provided in the computing environment 701).

[0161] Services are provided or deployed based on various virtualization technologies supported by the computing environment 701. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, and execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0162] In an embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). Figure 7As shown, service 720-2 can be equipped with one or more Pods 740-1, 740-2, ..., 740-N (collectively referred to as Pods). Pods can include a proxy 745 and one or more containers 742-1, 742-2, ..., 742-M (collectively referred to as containers). One or more containers in a Pod process requests related to one or more corresponding functions of the service, and the proxy 745 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with similar Pods.

[0163] During operation, executing a user request from the end user 702 may require calling one or more services in the computing environment 701, and executing one or more functions of a service may require calling one or more functions of another service. Figure 7 As shown, service "A" 720-1 receives a user request from end user 702 from ingress gateway 730, service "A" 720-1 may call service "D" 720-2, and service "D" 720-2 may request service "E" 720-3 to perform one or more functions.

[0164] The computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.

[0165] In another optional embodiment, Figure 8 The block diagram shows the use of the above Figure 6 The computer terminal 60 (or mobile device) is shown as an embodiment of the service grid. Figure 8 A structural diagram of a service grid is shown, Figure 8 As shown, the service grid 800 is mainly used to facilitate secure and reliable communication between multiple microservices. Microservices refer to decomposing an application into multiple smaller services or instances and distributing them on different clusters / machines to run.

[0166] like Figure 8 As shown, the microservice may include an application service instance A and an application service instance B, which form a functional application layer of the service grid 800. In one embodiment, the application service instance A runs in the form of a container / process 808 on a machine / workload container group 814 (Pod), and the application service instance B runs in the form of a container / process 810 on a machine / workload container group 816 (Pod).

[0167] In one implementation, application service instance A may be a product query service, and application service instance B may be a product ordering service.

[0168] like Figure 8 As shown, application service instance A and grid agent (sidecar) 803 coexist in machine workload container group 814, and application service instance B and grid agent 805 coexist in machine workload container 816. Grid agent 803 and grid agent 805 form the data plane layer (dataplane) of service grid 800. Among them, grid agent 803 and grid agent 805 run in the form of container / process 804 and container / process 806 respectively, and can receive request 812 for commodity query service, and grid agent 803 and application service instance A can communicate bidirectionally, and grid agent 805 and application service instance B can communicate bidirectionally. In addition, grid agent 803 and grid agent 805 can also communicate bidirectionally.

[0169] In one implementation, the traffic of application service instance A is routed to a suitable destination through grid proxy 803, and the network traffic of application service instance B is routed to a suitable destination through grid proxy 805. It should be noted that the network traffic mentioned here includes but is not limited to Hyper Text Transfer Protocol (HTTP), Representational State Transfer (REST), high-performance, general open source framework (google Remote Procedure Call, gRPC), open source in-memory data structure storage system (Redis), etc.

[0170] In one embodiment, the function of extending the data plane layer can be implemented by writing a custom filter for the proxy (Envoy) in the service mesh 800. The service mesh proxy configuration can be to enable the service mesh to correctly proxy service traffic and achieve service intercommunication and service governance. Mesh proxy 803 and mesh proxy 805 can be configured to perform at least one of the following functions: service discovery, health checking, routing, load balancing, authentication and authorization, and observability.

[0171] like Figure 8As shown, the service grid 800 also includes a control plane layer. The control plane layer may be a group of services running in a dedicated namespace, and these services are hosted by a hosting control plane component 801 in a machine / workload container group (machine / Pod) 802. Figure 8 As shown, the managed control plane component 801 performs bidirectional communication with the grid agent 803 and the grid agent 805. The managed control plane component 801 is configured to perform some control management functions. For example, the managed control plane component 801 receives telemetry data transmitted by the grid agent 803 and the grid agent 805, and can further aggregate the telemetry data. For these services, the managed control plane component 801 can also provide a user-oriented application programming interface (Application Programming Interface, API) to more easily manipulate network behavior, and provide configuration data to the grid agent 803 and the grid agent 805.

[0172] At present, with the widespread application of images in different fields, their copyright protection has become particularly important. In order to protect the copyright information of images, images are often watermarked. However, in the existing watermark processing process, only the focus is on how to embed the generated watermark into the image, and the corresponding watermark is not generated according to the needs of the field, which leads to the technical problem of large limitations in embedding watermark information in images.

[0173] The present application provides a method for generating an image, which solves the technical problem of the large limitations of embedding watermark information in an image. The method is different from traditional solutions that cannot generate corresponding watermarks according to the needs of the field, and solves the technical problem of the large limitations of embedding watermark information in an image.

[0174] In the embodiment of the present application, by training a Vincent graph model based on a diffusion model and using the trained Vincent graph model to implant a watermark into an input initial image, the purpose of protecting the copyright information of the image is achieved, thereby achieving the technical effect of reducing the limitations of implanting watermark information into an image, and further solving the technical problem of the large limitations of implanting watermark information into an image.

[0175] The above method of this embodiment is further introduced below.

[0176] In this embodiment, in order to solve the technical problem of the large limitation of embedding watermark information into an image, a method for embedding watermarks and protecting copyrights based on a generative diffusion model can be proposed. FIG9 (a) is a flow chart of a method for embedding watermarks and protecting copyrights based on a generative diffusion model according to an embodiment of the present application. As shown in FIG9 (a), the method may include the following steps:

[0177] Step S902: Input the style image (eg, FaceChain style image) into a watermark encoder for encoding to obtain a style watermark image.

[0178] In the technical solution provided in the above step S902 of the present application, style images of the same style as used for the FaceChain style model are collected and input into the watermark encoder for encoding, so as to obtain a watermarked style image.

[0179] Step S904: input the style watermark image into a watermark decoder for decoding to obtain a watermark label.

[0180] In the technical solution provided in the above step S904 of the present application, after the style image (for example, the FaceChain style image) is input into the watermark encoder for encoding to obtain the style watermark image, the style watermark image is input into the watermark decoder for decoding to obtain the watermark label.

[0181] In this embodiment, the watermarked data image is input into a watermark decoder for decoding, and the watermark information in the image can be obtained.

[0182] In this embodiment, the watermark encoder and the watermark decoder can be implemented by the following formula (2):

[0183] (2)

[0184] Among them, f Enc Can be used to represent the watermark encoder, f Dec can be used to represent the watermark decoder, x can be used to represent the style image, and f Enc (x) can be used to represent the style watermark image, y wm It can be used to represent the watermark information obtained by decoding the target image training sample using the initial decoder, f Dec (f Enc (x)) can be used to represent the watermark information expected by the user, for example, f Dec (f Enc (x)) can be y wm The reference information of y wm With f Dec (f Enc (x)) can reduce the loss value of the loss function.

[0185] In this embodiment, after the training process in the above formula (2) is completed, the trained watermark encoder is used to generate watermarked version images for all pictures required for Stable Diffusion model training, so as to obtain image pairs.

[0186] Step S906: Input the style image and the style watermark image into the VAE encoder for encoding to obtain features of the style image and features of the style watermark image.

[0187] In the technical solution provided in the above step S906 of the present application, after obtaining the style watermark image, the style image and the style watermark image are input into the VAE encoder for encoding to obtain the features of the style image and the features of the style watermark image.

[0188] In this embodiment, on the basis of obtaining the image pair, the text information corresponding to the original style image can be used to form a training data triple. In the Stable Diffusion training phase, the style image without watermark information and the style image with watermark information are respectively input into the VAE encoder for encoding, and the feature Z of the style image without watermark information can be obtained. t and the feature Z of the style image with watermark information t '.

[0189] Step S908: Input the features of the style image, the features of the style watermark image, and the style text into the UNet model for noise prediction to obtain the noise of the style image and the noise of the style watermark image.

[0190] In the technical solution provided in the above step S908 of the present application, after the style image and the style watermark image are input into the VAE encoder for encoding to obtain the features of the style image and the features of the style watermark image, the features of the style image, the features of the style watermark image and the style text are input into the UNet model for noise prediction to obtain the noise of the style image and the noise of the style watermark image.

[0191] In this embodiment, the feature Z of the style image without watermark information is t and the feature Z of the style image with watermark information t 'Noise is added, and the features of the style image without watermark information and the features of the style image with watermark information after noise are input into the UNet model for noise prediction, so that the noise of the style image without watermark information and the noise of the style image with watermark information can be obtained. The UNet model can use the loss function shown in the following formula (1) to learn the watermark information in the style image with watermark information:

[0192] (1)

[0193] in, It can be used to represent the noise of the style image without watermark information after adding noise. It can be used to represent the noise of the style image with watermark information after adding noise.

[0194] In this embodiment, for style images without watermark information, the diffusion model is still trained using the normal StableDiffusion denoising loss; for style images with watermark information, the model is trained using the opposite number of the Stable Diffusion denoising loss. For example, through the following formula (3), the Stable Diffusion model can learn in the opposite direction of the optimization direction satisfied in the text information for style images that already contain watermark information, thereby achieving the purpose of making it difficult to obtain high-quality generated images and conducting backdoor attacks on third-party models:

[0195] (3)

[0196] in, It can be used to indicate the noise information that needs to be removed.

[0197] Step S910: predicting real noise information according to the noise of the style image and the noise of the style watermark image.

[0198] In the technical solution provided in the above step S910 of the present application, after obtaining the noise of the style image and the noise of the style watermark image, the real noise information is predicted according to the obtained noise of the style image and the noise of the style watermark image.

[0199] It should be noted that the above FaceChain can match the object input by the user with any style to generate an image of the user object in any style. However, the style information of the FaceChain model is trained and learned using data of the same style collected by the copyright holder. For the user side, it can generate unlimited images of the same style at a low cost through the API, and use this to train the style model of a third party. In this process, the trained third-party model can be regarded as an infringement of the copyright model.

[0200] That is to say, users can normally use the FaceChain model to generate the corresponding style image, but the image already includes the watermark information pre-implanted using the image generation method in this application, and the watermark is difficult to be perceived by the naked eye. When there is malicious infringement, the use of the generated image to train a third-party model will generate a backdoor attack on the third-party model, thereby achieving the purpose of protecting the copyright information of the image.

[0201] In this embodiment, the above method can be implemented by a Vincent graph watermark implantation model based on a generative diffusion model. For example, Figure 9 (b) is a schematic diagram of a Vincent graph watermark implantation model based on a generative diffusion model according to an embodiment of the present application. As shown in Figure 9 (b), the model may include: a watermark encoder 9002, a watermark decoder 9004, a VAE encoder 9006 and a UNet model 9008, wherein the watermark encoder 9002 can be used to encode a style image (for example, a FaceChain style image) to obtain a style watermark image, the watermark decoder 9004 can be used to decode the style watermark image to obtain a watermark label, the VAE encoder 9006 can be used to encode the style image and the style watermark image to obtain features of the style image and features of the style watermark image, and the UNet model 9008 can be used to perform noise prediction on features of the style image, features of the style watermark image and style text to obtain the noise of the style image and the noise of the style watermark image.

[0202] In this embodiment, data images of the same style as the FaceChain style model are collected, and the collected data images are input into a watermark encoder for encoding to obtain a watermarked style watermark image, and the style watermark image is input into a watermark decoder for decoding to predict the watermark information therein, wherein the training process of the watermark encoder and the watermark decoder can be trained using the above formula (2).

[0203] In this embodiment, after the training of the watermark encoder and the watermark decoder is completed, the watermark implantation operation is performed on all training images to obtain the original style image and the image pair containing the watermark information. Then, the text information and the image pair are input into the Stable Diffusion model for training, wherein the loss function between the original style image and the image containing the watermark information can be trained using formula (1), thereby achieving the purpose of enabling the model to learn the process of watermark implantation.

[0204] In this embodiment, the image containing watermark information can be trained using formula (3), thereby achieving the purpose of destroying the training process of the third-party model using the generated image containing watermark information and achieving the purpose of performing a backdoor attack on the third-party model.

[0205] It should be noted that the watermark encoder 9002 and the watermark decoder 9004 can be combined into an optimized network model to encode and decode the image input thereto, thereby obtaining the watermark label of the image.

[0206] In the embodiment of the present application, a watermark is implanted into an input initial image by training a Vincent graph model based on a diffusion model and using the trained Vincent graph model, thereby achieving the purpose of protecting the copyright information of the image, thereby achieving the technical effect of reducing the limitations of implanting watermark information into the image, and further solving the technical problem of the large limitations of implanting watermark information into the image.

[0207] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 2 An image generating device according to the image generating method shown.

[0208] Fig.10 is a schematic diagram of an image generating device according to an embodiment of the present application, such as Fig.10 As shown, the image generating device 1000 may include: a first acquiring unit 1002 , a first converting unit 1004 and a first adding unit 1006 .

[0209] The first acquisition unit 1002 is used to acquire an initial image to be processed and text information corresponding to the initial image, wherein the text information is used to describe the semantics of a target image to be converted from the initial image.

[0210] The first conversion unit 1004 is used to extract initial image features from the initial image using an image generation model, analyze the initial image features and text information, and convert the initial image features into target image features that satisfy the text information, wherein the image generation model is obtained by training a text image model with watermark information.

[0211] The first adding unit 1006 is used to add watermark information to the target image features by using the image generation model to obtain the target image including the watermark information.

[0212] It should be noted that the first acquisition unit 1002, the first conversion unit 1004 and the first adding unit 1006 correspond to steps S202 to S206 in Embodiment 2, and the three units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Embodiment 2. It should be noted that the above units may be hardware components or software components stored in a memory (e.g., memory 1304) and processed by one or more processors (e.g., processors 602a, 602b..., 602n), and the above units may also be run in the computer terminal 60 provided in Embodiment 2 as part of the device.

[0213] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 3 An image generating device according to the image generating method shown.

[0214] Fig.11 is a schematic diagram of a model generation device according to an embodiment of the present application, such as Fig.11 As shown, the model generation device 1100 may include: a second acquisition unit 1102 , a first training unit 1104 and a second training unit 1106 .

[0215] The second acquisition unit 1102 is used to acquire an initial image sample and a target image sample, wherein the target image sample is used to at least represent the initial image sample including the watermark information.

[0216] The first training unit 1104 is used to train the Vincent graph model using the initial image sample and the target image sample to obtain the Vincent graph model with watermark information.

[0217] The second training unit 1106 is used to train the text-image model with watermark information using at least the initial image sample and the text information sample to obtain an image generation model, wherein the text information sample is used to describe the semantics of the target image sample, and the image generation model is used to extract the initial image features from the input initial image, analyze the initial image features and the text information corresponding to the initial image, convert the initial image features into target image features that meet the text information, and add watermark information to the target image features to obtain a target image including the watermark information, and the text information is used to describe the semantics of the target image to be converted from the initial image.

[0218] It should be noted that the second acquisition unit 1102, the first training unit 1104 and the second training unit 1106 correspond to steps S302 to S306 in Embodiment 2, and the three units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Embodiment 2. It should be noted that the above units may be hardware components or software components stored in a memory (e.g., memory 1304) and processed by one or more processors (e.g., processors 602a, 602b..., 602n), and the above units may also be run in the computer terminal 60 provided in Embodiment 2 as part of the device.

[0219] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 4 An image generating device according to the image generating method shown.

[0220] Fig.12 is a schematic diagram of an image generating device according to an embodiment of the present application, such as Fig.12 As shown, the image generating device 1200 may include: a third acquiring unit 1202 , a second converting unit 1204 and a second adding unit 1206 .

[0221] The third acquisition unit 1202 is used to acquire the initial image to be processed and the style text information corresponding to the initial image, wherein the style text information is used to describe the semantics of the style of the target image to be converted from the initial image.

[0222] The second conversion unit 1204 is used to extract initial image features from the initial image using the Vincent graph model, analyze the initial image features and the style text information, and convert the initial image features into target image features that meet the style text information, wherein the Vincent graph model is obtained by training the Vincent graph model with watermark information.

[0223] The second adding unit 1206 is used to add watermark information to the target image features by using the text graph model to obtain the target image.

[0224] It should be noted that the third acquisition unit 1202, the second conversion unit 1204 and the second adding unit 1206 correspond to steps S402 to S406 in Embodiment 2, and the three units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Embodiment 2. It should be noted that the above units may be hardware components or software components stored in a memory (e.g., memory 1304) and processed by one or more processors (e.g., processors 602a, 602b..., 602n), and the above units may also be run in the computer terminal 60 provided in Embodiment 2 as part of the device.

[0225] In an embodiment of the present application, when performing image generation, an initial image to be processed and text information corresponding to the initial image are obtained, that is, the initial image and the semantics of the target image to be converted from the initial image are obtained. Then, the image generation model is used to extract the initial image features from the initial image, and the extracted initial image features and text information are analyzed, and the initial image features can be converted into target image features that meet the text information, and the image generation model is used to add watermark information to the converted target image features, and a target image including watermark information can be obtained. Since in the process of image generation, not only the text information corresponding to the initial image is obtained, but also the obtained initial image and the obtained text information are analyzed, thereby generating a target image including watermark information, the purpose of protecting the copyright information of the image is achieved, thereby achieving the technical effect of reducing the limitations of implanting watermark information into the image, and thus solving the technical problem of the large limitations of implanting watermark information into the image.

[0226] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0227] The embodiment of the present application may provide an electronic device, which may be any electronic device in a group of electronic devices. Optionally, in this embodiment, the electronic device may also be replaced by a terminal device such as a mobile terminal.

[0228] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0229] In this embodiment, the computer terminal can execute the program code in the method.

[0230] Optionally, Fig.13 is a block diagram of an electronic device according to an image generation method of an embodiment of the present application. Fig.13 As shown, the electronic device A may include: one or more (only one is shown in the figure) processors 1302, a memory 1304, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0231] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the methods in the above embodiments. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal B via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0232] The processor can call the information and application stored in the memory to perform the following steps: obtain an initial image to be processed and text information corresponding to the initial image, wherein the text information is used to describe the semantics of the target image to be converted from the initial image; use an image generation model to extract initial image features from the initial image, and analyze the initial image features and text information to convert the initial image features into target image features that meet the text information, wherein the image generation model is obtained by training a text graph model with watermark information; use the image generation model to add watermark information to the target image features to obtain a target image including the watermark information.

[0233] Optionally, the processor may also execute the following program code steps: extracting initial image features from the initial image using a feature extraction model in the image generation model, wherein the feature extraction model is obtained by training a variational autoencoder; calling a noise prediction model in the image generation model to convert the initial image features into target image features that satisfy the text information, wherein the noise prediction model is obtained by training a convolutional neural network model.

[0234] Optionally, the processor may also execute the program code of the following steps: using a noise prediction model in an image generation model to add noise information of the watermark information to the target image features to obtain the target image.

[0235] Optionally, the processor may also execute the program code of the following steps: performing noise processing on the target image features; using a noise prediction model to perform noise prediction on the target image features after the noise processing to obtain predicted noise information, wherein the predicted noise information includes the noise information of the watermark information; using the predicted noise information, converting the target image features into a target image including the noise information of the watermark information.

[0236] Optionally, the processor may also execute the program code of the following steps: using the predicted noise information, denoising the target image features, and retaining the noise information of the watermark information, to obtain a target image including the noise information of the watermark information.

[0237] Optionally, the processor may also execute the program code of the following steps: using the loss function of the noise prediction model to output the noise information of the watermark information, wherein the loss function is used to represent the difference between the noise information of the initial image sample and the noise information of the target image sample, the loss function is used to enable the Vincent graph model to learn the watermark information, and the target image sample is used to at least represent the initial image sample including the watermark information.

[0238] Optionally, the processor may also execute the program code of the following steps: outputting the target image to a third-party model, wherein the watermark information in the target image is used to enable the third-party model to learn the opposite text information of the text information, and the performance index of the third-party model after learning is lower than the performance index of the third-party model before training.

[0239] Optionally, the processor may call information and applications stored in the memory to perform the following steps: obtaining an initial image sample and a target image sample, wherein the target image sample is used to at least represent the initial image sample including watermark information; using the initial image sample and the target image sample, training the Vincent graph model to obtain a Vincent graph model that has been learned with watermark information; using at least the initial image sample and the text information sample, training the Vincent graph model that has been learned with watermark information to obtain an image generation model, wherein the text information sample is used to describe the semantics of the target image sample, and the image generation model is used to extract initial image features from the input initial image, analyze the initial image features and the text information corresponding to the initial image, convert the initial image features into target image features that satisfy the text information, and add watermark information to the target image features to obtain a target image that includes the watermark information, and the text information is used to describe the semantics of the target image to be converted from the initial image.

[0240] Optionally, the processor may also execute program code of the following steps: encoding the initial image sample by using a target encoder to generate a target image sample including watermark information.

[0241] Optionally, the processor may also execute the following program code: encoding the initial image training sample using the initial encoder to obtain the target image training sample; decoding the target image training sample using the initial decoder to obtain the watermark information; and training the initial encoder using the watermark information to obtain the target encoder.

[0242] Optionally, the processor may also execute the program code of the following steps: obtaining a loss function between the noise information of the initial image sample and the noise information of the target image sample, wherein the loss function is used to represent the difference between the noise information of the initial image sample and the noise information of the target image sample; and training the text graph model using the loss function and the text information sample to obtain a text graph model that has learned watermark information.

[0243] Optionally, the processor may also execute the following program code: using the feature extraction model in the Vincent graph model to extract initial image feature samples from the initial image samples, and extracting target image feature samples from the target image samples; performing noise processing on the initial image feature samples and the target image feature samples respectively; using the initial noise prediction model in the Vincent graph model to predict the initial noise information in the initial image feature samples after noise processing, and predict the target noise information in the target image feature samples after noise processing; and obtaining the loss function between the initial noise information and the target noise information.

[0244] Optionally, the processor may also execute the following program code: at least using the loss function to control the initial noise prediction model in the Vincent graph model to learn the watermark information to obtain the noise prediction model, wherein the Vincent graph model with the watermark information is learned includes the noise prediction model.

[0245] Optionally, the processor may also execute the program code of the following steps: obtaining denoising loss information of a diffusion model, wherein the denoising loss information is used to represent the difference between target noise information and noise information to be removed from the target noise information; using the denoising loss information, initial image samples and text information samples to train a feature extraction model and a trained noise prediction model to obtain an image generation model.

[0246] Optionally, the processor may also execute the program code of the following steps: obtaining the opposite of denoising loss information; training a third-party model using the watermark information and the opposite of denoising loss information of the target image, wherein the watermark information and the opposite of denoising loss information in the target image are used to enable the third-party model to learn the opposite text information of the text information, and the performance index of the trained third-party model is lower than the performance index of the third-party model before training.

[0247] Optionally, the processor can call the information and application stored in the memory to perform the following steps: obtain an initial image to be processed, and style text information corresponding to the initial image, wherein the style text information is used to describe the semantics of the style of the target image to be converted from the initial image; use the Vincent graph model to extract the initial image features from the initial image, and analyze the initial image features and the style text information to convert the initial image features into target image features that meet the style text information, wherein the Vincent graph model is obtained by training a Vincent graph model with watermark information; use the Vincent graph model to add watermark information to the target image features to obtain the target image.

[0248] According to an embodiment of the present application, a method for generating an image is provided. When generating an image, an initial image to be processed and text information corresponding to the initial image are obtained, that is, the initial image and the semantics of the target image to be converted from the initial image are obtained. Then, the image generation model is used to extract the initial image features from the initial image, and the extracted initial image features and text information are analyzed, so that the initial image features can be converted into target image features that meet the text information, and the image generation model is used to add watermark information to the converted target image features, so that a target image including the watermark information can be obtained. Since in the process of image generation, not only the text information corresponding to the initial image is obtained, but also the obtained initial image and the obtained text information are analyzed, thereby generating a target image including the watermark information, the purpose of protecting the copyright information of the image is achieved, thereby achieving the technical effect of reducing the limitations of implanting watermark information into the image, and thus solving the technical problem of the large limitations of implanting watermark information into the image.

[0249] It can be understood by those skilled in the art that Figure 6 The structure shown is for illustration only, and the computer terminal B may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID for short), a PAD, or other terminal devices. Figure 6 It does not limit the structure of the computer terminal B. For example, the computer terminal B may also include Figure 6 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 6 Different configurations are shown.

[0250] Optionally, Fig.14 is a structural block diagram of a computer terminal according to an embodiment of the present application, such as Fig.14 As shown, the computer terminal B may include: one or more (only one is shown in the figure) processors 1402 , a memory 1404 and a transmission device 1406 .

[0251] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image generation method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image generation method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal B via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0252] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain an initial image to be processed and text information corresponding to the initial image, wherein the text information is used to describe the semantics of the target image to be converted from the initial image; use the image generation model to extract the initial image features from the initial image, and analyze the initial image features and the text information to convert the initial image features into target image features that meet the text information, wherein the image generation model is obtained by training a text graph model with watermark information; use the image generation model to add watermark information to the target image features to obtain the target image including the watermark information.

[0253] Optionally, the processor may also execute the following program code steps: extracting initial image features from the initial image using a feature extraction model in the image generation model, wherein the feature extraction model is obtained by training a variational autoencoder; calling a noise prediction model in the image generation model to convert the initial image features into target image features that satisfy the text information, wherein the noise prediction model is obtained by training a convolutional neural network model.

[0254] Optionally, the processor may also execute the program code of the following steps: using a noise prediction model in an image generation model to add noise information of the watermark information to the target image features to obtain the target image.

[0255] Optionally, the processor may also execute the program code of the following steps: performing noise processing on the target image features; using a noise prediction model to perform noise prediction on the target image features after the noise processing to obtain predicted noise information, wherein the predicted noise information includes the noise information of the watermark information; using the predicted noise information, converting the target image features into a target image including the noise information of the watermark information.

[0256] Optionally, the processor may also execute the program code of the following steps: using the predicted noise information, denoising the target image features, and retaining the noise information of the watermark information, to obtain a target image including the noise information of the watermark information.

[0257] Optionally, the processor may also execute the program code of the following steps: using the loss function of the noise prediction model to output the noise information of the watermark information, wherein the loss function is used to represent the difference between the noise information of the initial image sample and the noise information of the target image sample, the loss function is used to enable the Vincent graph model to learn the watermark information, and the target image sample is used to at least represent the initial image sample including the watermark information.

[0258] Optionally, the processor may also execute the program code of the following steps: outputting the target image to a third-party model, wherein the watermark information in the target image is used to enable the third-party model to learn the opposite text information of the text information, and the performance index of the third-party model after learning is lower than the performance index of the third-party model before training.

[0259] Optionally, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain an initial image sample and a target image sample, wherein the target image sample is used to at least represent the initial image sample including watermark information; use the initial image sample and the target image sample to train the Vincent graph model to obtain the Vincent graph model with watermark information; use at least the initial image sample and the text information sample to train the Vincent graph model with watermark information to obtain an image generation model, wherein the text information sample is used to describe the semantics of the target image sample, and the image generation model is used to extract initial image features from the input initial image, analyze the initial image features and the text information corresponding to the initial image, convert the initial image features into target image features that meet the text information, and add watermark information to the target image features to obtain a target image including the watermark information, and the text information is used to describe the semantics of the target image to be converted from the initial image.

[0260] Optionally, the processor may also execute program code of the following steps: encoding the initial image sample by using a target encoder to generate a target image sample including watermark information.

[0261] Optionally, the processor may also execute the following program code: encode the initial image training sample using the initial encoder to obtain the target image training sample; decode the target image training sample using the initial decoder to obtain watermark information; and train the initial encoder using the watermark information to obtain the target encoder.

[0262] Optionally, the processor may also execute the program code of the following steps: obtaining a loss function between the noise information of the initial image sample and the noise information of the target image sample, wherein the loss function is used to represent the difference between the noise information of the initial image sample and the noise information of the target image sample; and training the text graph model using the loss function and the text information sample to obtain a text graph model that has learned watermark information.

[0263] Optionally, the processor may also execute the following program code: using the feature extraction model in the Vincent graph model to extract initial image feature samples from the initial image samples, and extracting target image feature samples from the target image samples; performing noise processing on the initial image feature samples and the target image feature samples respectively; using the initial noise prediction model in the Vincent graph model to predict the initial noise information in the initial image feature samples after noise processing, and predict the target noise information in the target image feature samples after noise processing; and obtaining the loss function between the initial noise information and the target noise information.

[0264] Optionally, the processor may also execute the following program code: at least using the loss function to control the initial noise prediction model in the Vincent graph model to learn the watermark information to obtain the noise prediction model, wherein the Vincent graph model with the watermark information is learned includes the noise prediction model.

[0265] Optionally, the processor may also execute the program code of the following steps: obtaining denoising loss information of a diffusion model, wherein the denoising loss information is used to represent the difference between target noise information and noise information to be removed from the target noise information; using the denoising loss information, initial image samples and text information samples to train a feature extraction model and a trained noise prediction model to obtain an image generation model.

[0266] Optionally, the processor may also execute the program code of the following steps: obtaining the opposite of denoising loss information; training a third-party model using the watermark information of the target image and the opposite of the denoising loss information, wherein the watermark information in the target image and the opposite of the denoising loss are used to enable the third-party model to learn the opposite text information of the text information, and the performance index of the third-party model after training is lower than the performance index of the third-party model before training.

[0267] Optionally, the processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain an initial image to be processed, and style text information corresponding to the initial image, wherein the style text information is used to describe the semantics of the style of the target image to be converted from the initial image; use the Vincent graph model to extract the initial image features from the initial image, and analyze the initial image features and the style text information to convert the initial image features into target image features that meet the style text information, wherein the Vincent graph model is obtained by training a Vincent graph model with watermark information; use the Vincent graph model to add watermark information to the target image features to obtain the target image.

[0268] A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0269] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the method provided in the above embodiment.

[0270] Optionally, in this embodiment, the above storage medium may be located in any electronic device in a group of electronic devices in a computer network, or in any mobile terminal in a group of mobile terminals.

[0271] Optionally, in this embodiment, a computer-readable storage medium is configured to store program codes for executing the following steps: obtaining an initial image to be processed, and text information corresponding to the initial image, wherein the text information is used to describe the semantics of a target image to be converted from the initial image; using an image generation model to extract initial image features from the initial image, and analyzing the initial image features and the text information, and converting the initial image features into target image features that satisfy the text information, wherein the image generation model is obtained by training a text graph model with watermark information; and using the image generation model to add watermark information to the target image features to obtain a target image including the watermark information.

[0272] Optionally, the computer-readable storage medium can also execute the following steps of program code: extracting initial image features from the initial image using a feature extraction model in the image generation model, wherein the feature extraction model is obtained by training a variational autoencoder; calling a noise prediction model in the image generation model to convert the initial image features into target image features that satisfy the text information, wherein the noise prediction model is obtained by training a convolutional neural network model.

[0273] Optionally, the computer-readable storage medium may also execute program code for the following steps: using a noise prediction model in an image generation model to add noise information of watermark information to target image features to obtain a target image.

[0274] Optionally, the computer-readable storage medium may also execute program code for the following steps: performing noise processing on target image features; using a noise prediction model to perform noise prediction on the target image features after the noise processing to obtain predicted noise information, wherein the predicted noise information includes noise information of the watermark information; using the predicted noise information, converting the target image features into a target image including the noise information of the watermark information.

[0275] Optionally, the computer-readable storage medium may also execute program code for the following steps: using the predicted noise information to denoise the target image features, and retaining the noise information of the watermark information, to obtain a target image including the noise information of the watermark information.

[0276] Optionally, the computer-readable storage medium may also execute program code for the following steps: outputting noise information of the watermark information using a loss function of a noise prediction model, wherein the loss function is used to represent the difference between the noise information of an initial image sample and the noise information of a target image sample, the loss function is used to enable the Vincent graph model to learn watermark information, and the target image sample is used to at least represent the initial image sample including the watermark information.

[0277] Optionally, the computer-readable storage medium may also execute program code for the following steps: outputting a target image to a third-party model, wherein the watermark information in the target image is used to enable the third-party model to learn the opposite text information of the text information, and the performance index of the third-party model after learning is lower than the performance index of the third-party model before training.

[0278] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: obtaining an initial image sample and a target image sample, wherein the target image sample is used to at least represent the initial image sample including watermark information; using the initial image sample and the target image sample, training a Vincent graph model to obtain a Vincent graph model with watermark information; using at least the initial image sample and the text information sample, training the Vincent graph model with watermark information to obtain an image generation model, wherein the text information sample is used to describe the semantics of the target image sample, and the image generation model is used to extract initial image features from the input initial image, analyze the initial image features and the text information corresponding to the initial image, convert the initial image features into target image features that satisfy the text information, and add watermark information to the target image features to obtain a target image including the watermark information, and the text information is used to describe the semantics of the target image to be converted from the initial image.

[0279] Optionally, the computer-readable storage medium may also execute program code for the following steps: encoding the initial image sample using a target encoder to generate a target image sample including watermark information.

[0280] Optionally, the computer-readable storage medium may also execute program code for the following steps: encoding an initial image training sample using an initial encoder to obtain a target image training sample; decoding the target image training sample using an initial decoder to obtain watermark information; and training the initial encoder using the watermark information to obtain a target encoder.

[0281] Optionally, the computer-readable storage medium may also execute program code for the following steps: obtaining a loss function between noise information of an initial image sample and noise information of a target image sample, wherein the loss function is used to represent the difference between the noise information of the initial image sample and the noise information of the target image sample; and training a text graph model using the loss function and text information samples to obtain a text graph model that has learned watermark information.

[0282] Optionally, the computer-readable storage medium may also execute program code for the following steps: using a feature extraction model in the Vincent graph model to extract initial image feature samples from the initial image samples, and extracting target image feature samples from the target image samples; performing noise processing on the initial image feature samples and the target image feature samples, respectively; using an initial noise prediction model in the Vincent graph model to predict initial noise information in the initial image feature samples after noise processing, and predicting target noise information in the target image feature samples after noise processing; and obtaining a loss function between the initial noise information and the target noise information.

[0283] Optionally, the computer-readable storage medium may also execute program code of the following steps: at least using a loss function to control an initial noise prediction model in the Vincent graph model to learn watermark information to obtain a noise prediction model, wherein the Vincent graph model with watermark information is learned includes a noise prediction model.

[0284] Optionally, the computer-readable storage medium may also execute program code for the following steps: obtaining denoising loss information of a diffusion model, wherein the denoising loss information is used to represent the difference between target noise information and noise information to be removed from the target noise information; and using the denoising loss information, initial image samples, and text information samples to train a feature extraction model and a trained noise prediction model to obtain an image generation model.

[0285] Optionally, the computer-readable storage medium can also execute the program code of the following steps: obtaining the opposite of the denoising loss information; training the third-party model using the watermark information of the target image and the opposite of the denoising loss information, wherein the watermark information in the target image and the opposite of the denoising loss information are used to enable the third-party model to learn the opposite text information of the text information, and the performance index of the third-party model after training is lower than the performance index of the third-party model before training.

[0286] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: obtaining an initial image to be processed, and style text information corresponding to the initial image, wherein the style text information is used to describe the semantics of the style of a target image to be converted from the initial image; extracting initial image features from the initial image using a Vincent graph model, analyzing the initial image features and the style text information, and converting the initial image features into target image features that satisfy the style text information, wherein the Vincent graph model is obtained by training a Vincent graph model with watermark information; and adding watermark information to the target image features using the Vincent graph model to obtain the target image.

[0287] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include computer instructions, and when the computer instructions are executed by a processor, the image generation method provided in the embodiment is implemented.

[0288] The embodiment of the present application also provides a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which may be used to store a computer program, and when the computer program is executed by a processor, the image generation method provided in the embodiment is implemented.

[0289] The embodiment of the present application further provides a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, the image generation method provided in the above embodiment is implemented.

[0290] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0291] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0292] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0293] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0294] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0295] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., various media that can store program codes.

[0296] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for generating an image, characterized in that: include: Acquire an initial image to be processed and text information corresponding to the initial image, wherein the text information is used to describe the semantics of a target image to be converted from the initial image; Using an image generation model, extracting initial image features from the initial image, analyzing the initial image features and the text information, and converting the initial image features into target image features that satisfy the text information, wherein the image generation model is obtained by training a Vincent graph model with watermark information using at least initial image samples and text information samples, the text information samples are used to describe the semantics of the target image samples, the target image samples are used to at least represent the initial image samples including the watermark information, and the Vincent graph model with the watermark information is obtained by training the Vincent graph model using the initial image samples and the target image samples; The watermark information is added to the target image features using the image generation model to obtain the target image including the watermark information.

2. The method according to claim 1, characterized in that The initial image sample and the target image sample are used to enable the Vincent graph model to learn the watermark information.

3. The method according to claim 1, characterized in that Using an image generation model, extracting initial image features from the initial image, analyzing the initial image features and the text information, and converting the initial image features into target image features that satisfy the text information, including: Extracting the initial image features from the initial image using a feature extraction model in the image generation model, wherein the feature extraction model is obtained by training a variational autoencoder; The initial image features are converted into the target image features that satisfy the text information by using a noise prediction model in the image generation model, wherein the noise prediction model is obtained by training a convolutional neural network model.

4. The method according to claim 3, characterized in that Adding the watermark information to the target image features using the image generation model to obtain the target image includes: The noise prediction model in the image generation model is used to add the noise information of the watermark information to the target image features to obtain the target image.

5. The method according to claim 4, characterized in that Using the noise prediction model in the image generation model, adding the noise information of the watermark information to the target image features to obtain the target image, comprising: Performing noise processing on the target image features; Using the noise prediction model, performing noise prediction on the target image features after the noise processing to obtain predicted noise information, wherein the predicted noise information includes the noise information of the watermark information; The target image feature is converted into the target image including the noise information of the watermark information by using the predicted noise information.

6. The method according to claim 5, characterized in that The predicted noise information is carried by the target image feature, and using the predicted noise information, the target image feature is converted into the target image including the noise information of the watermark information, including: The target image features are denoised using the predicted noise information, and the noise information of the watermark information is retained to obtain the target image including the noise information of the watermark information.

7. The method according to claim 4, characterized in that The method further comprises: The noise information of the watermark information is outputted by using the loss function of the noise prediction model, wherein the loss function is used to represent the difference between the noise information of the initial image sample and the noise information of the target image sample, the loss function is used to enable the Vincent graph model to learn the watermark information, and the target image sample is used to at least represent the initial image sample including the watermark information.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Output the target image to a third-party model, wherein the watermark information in the target image is used to enable the third-party model to learn the opposite text information of the text information, and the performance index of the third-party model after learning is lower than the performance index of the third-party model before training.

9. A method for generating a model, characterized in that: include: Acquire an initial image sample and a target image sample, wherein the target image sample is used to at least represent the initial image sample including watermark information; Using the initial image sample and the target image sample, training the Vincent graph model to obtain the Vincent graph model with watermark information; At least the initial image sample and the text information sample are used to train the text image model that has learned the watermark information to obtain an image generation model, wherein the text information sample is used to describe the semantics of the target image sample, and the image generation model is used to extract initial image features from the input initial image, analyze the initial image features and the text information corresponding to the initial image, convert the initial image features into target image features that satisfy the text information, and add the watermark information to the target image features to obtain a target image including the watermark information, and the text information is used to describe the semantics of the target image to be converted from the initial image.

10. The method according to claim 9, characterized in that The method further comprises: The initial image sample is encoded by using a target encoder to generate the target image sample including the watermark information.

11. The method according to claim 10, characterized in that The method further comprises: Encode the initial image training sample using the initial encoder to obtain the target image training sample; Decoding the target image training sample using an initial decoder to obtain the watermark information; The initial encoder is trained using the watermark information to obtain the target encoder.

12. The method according to claim 9, characterized in that Using the initial image sample and the target image sample, training the Vincent graph model to obtain a Vincent graph model with watermark information, including: Acquire a loss function between the noise information of the initial image sample and the noise information of the target image sample, wherein the loss function is used to represent the difference between the noise information of the initial image sample and the noise information of the target image sample; The loss function is used to train the Wensheng graph model to obtain a Wensheng graph model that has learned watermark information.

13. The method according to claim 12, characterized in that Obtaining a loss function between the noise information of the initial image sample and the noise information of the target image sample includes: Using the feature extraction model in the Wensheng graph model, an initial image feature sample is extracted from the initial image sample, and a target image feature sample is extracted from the target image sample; Performing noise addition processing on the initial image feature samples and the target image feature samples respectively; Using the initial noise prediction model in the Wensheng graph model, predicting the initial noise information in the initial image feature samples after noise addition, and predicting the target noise information in the target image feature samples after noise addition; The loss function between the initial noise information and the target noise information is obtained.

14. The method according to claim 13, characterized in that At least using the loss function to train the text graph model to obtain a text graph model with watermark information, including: At least using the loss function, the initial noise prediction model in the Vincent graph model is controlled to learn the watermark information to obtain a noise prediction model, wherein the Vincent graph model that learns the watermark information includes the noise prediction model.

15. The method according to claim 14, characterized in that Using the initial image sample and the text information sample, the text image model with the watermark information is trained to obtain an image generation model, including: Acquire denoising loss information, wherein the denoising loss information is used to represent the difference between the target noise information and the noise information to be removed from the target noise information; The feature extraction model and the noise prediction model are trained using the denoising loss information, the initial image samples and the text information samples to obtain the image generation model.

16. The method according to claim 15, characterized in that The method further comprises: Obtaining the inverse number of the denoising loss information; The third-party model is trained using the opposite numbers of the watermark information and the denoising loss information of the target image, wherein the opposite numbers of the watermark information and the denoising loss information in the target image are used to enable the third-party model to learn the opposite text information of the text information, and the performance index of the third-party model after training is lower than the performance index of the third-party model before training.

17. A method for generating an image, characterized in that: include: Acquire an initial image to be processed and style text information corresponding to the initial image, wherein the style text information is used to describe the semantics of the style of the target image to be converted from the initial image; Using a Vincent graph model, extracting initial image features from the initial image, analyzing the initial image features and the style text information, and converting the initial image features into target image features that satisfy the style text information, wherein the Vincent graph model is obtained by training a Vincent graph model with watermark information using at least an initial image sample and a text information sample, the text information sample is used to describe the semantics of the target image sample, the target image sample is used to at least represent the initial image sample including the watermark information, and the Vincent graph model with the watermark information is obtained by training the Vincent graph model using the initial image sample and the target image sample; The watermark information is added to the target image features using the text graph model to obtain the target image.

18. A system for generating an image, characterized in that: include: An information input terminal, used to obtain an initial image to be processed and text information corresponding to the initial image, wherein the text information is used to describe the semantics of a target image to be converted from the initial image; The image generation end is used to extract initial image features from the initial image using an image generation model, analyze the initial image features and the text information, and convert the initial image features into target image features that satisfy the text information, wherein the image generation model is obtained by training a Vincent graph model with watermark information using at least initial image samples and text information samples, the text information samples are used to describe the semantics of the target image samples, the target image samples are used to at least represent the initial image samples including the watermark information, and the Vincent graph model with the watermark information is obtained by training the Vincent graph model using the initial image samples and the target image samples; the target image is obtained by adding the watermark information to the target image features using the image generation model; The information output terminal is used to output the target image.

19. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 17 when running.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 17.

21. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 17.