Image generation model training method and device, equipment and medium

Abstract

The invention belongs to the field of artificial intelligence, and relates to an image generation model training method comprising the following steps: obtaining a sample image and structural feature information corresponding to the sample image; inputting the structural feature information as a constraint condition into a preset diffusion model to generate a plurality of prediction images; based on an intra-class contrast learning strategy, performing integration processing on the plurality of prediction images and the sample image to obtain a loss function; adjusting parameters of the diffusion model based on the loss function, and returning to execute the step of inputting the structural feature information as the constraint condition into the preset diffusion model until the loss function is obtained until the convergence of the loss function reaches a convergence threshold value; and determining the diffusion model corresponding to the condition that the loss function reaches a convergence threshold as an image generation model. The invention further provides a device, equipment and a medium. In addition, the invention also relates to a block chain technology, and the sample image and the structural feature information can be stored in a block chain. According to the invention, the image generation accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method, device, equipment and medium for an image generation model. Background Art

[0002] Image generation is a core task in the field of computer vision, and its importance is self-evident. In recent years, with the continuous advancement of generative model technology, the field of image generation has achieved remarkable results. Methods such as generative adversarial networks and diffusion models, with their powerful generation capabilities, can automatically create highly realistic images, greatly enriching the application scenarios of image generation. In particular, when generating images based on specific conditions, such as text descriptions, sketch outlines, or edge images, these technologies show flexibility and creativity. They can automatically generate images that meet the requirements based on the input conditions provided by the user, bringing new possibilities to many fields such as design, entertainment, and medical care.

[0003] However, despite significant technological advances, existing image generation control schemes still face many challenges. Among them, the most prominent problem is that the generated images have obvious deviations from the expected targets in terms of structure and details. This problem is particularly evident when dealing with complex scenes. For example, when using mainstream methods such as Control Net (a technology that provides additional control signals during image generation) for image generation, the generated images often have large differences in structure and details from the expected targets due to the difficulty in accurately capturing subtle changes in input conditions. This difference not only affects the overall quality of the image, but also limits the application of image generation technology in high-precision demand scenarios. Therefore, there is an urgent need for a method that can accurately capture input conditions, thereby generating images that are highly consistent with the expected targets in structure and details, thereby improving the accuracy of image generation. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a training method, device, equipment and medium for an image generation model to solve the existing problem that it is difficult to accurately capture subtle changes in input conditions, resulting in the generated images often having large differences in structure and details from the expected target.

[0005] In order to solve the above technical problems, the present application embodiment provides a training method for an image generation model, which adopts the following technical solutions:

[0006] Acquire a sample image and structural feature information corresponding to the sample image; use the structural feature information as a constraint condition, input a preset diffusion model, and generate multiple predicted images of the structural feature information; based on a preset intra-class comparison learning strategy, integrate and process the multiple predicted images and sample images to obtain a loss function; based on the loss function, adjust the parameters of the diffusion model, return to the step of inputting the structural feature information as a constraint condition into the preset diffusion model to obtain the loss function, until the loss function converges to a convergence threshold; determine the diffusion model corresponding to the loss function converges to the convergence threshold as the image generation model.

[0007] Furthermore, the step of obtaining the sample image and the structural feature information corresponding to the sample image specifically includes:

[0008] Acquire a sample image, preprocess the sample image, and obtain a preprocessed feature image; use an edge detection algorithm to perform edge detection on the feature image to obtain edge information of the feature image; generate an edge feature matrix of the feature image based on the edge information; use a Hough transform algorithm to analyze the edge feature matrix to determine the straight line information and circular arc structured information in the feature image; and construct the structural feature information of the feature image based on the straight line information and circular arc structured information.

[0009] Furthermore, the step of taking the structural feature information as a constraint condition and inputting a preset diffusion model to generate a predicted image of the structural feature information specifically includes:

[0010] Identify the type of structural feature information; if the structural feature information is a skeleton graph, convert the skeleton graph into data in a first specific format, input the data in the first specific format as a first constraint condition into a preset diffusion model, and generate multiple predicted images of the skeleton graph; if the structural feature information is an edge graph, convert the edge graph into data in a second specific format, input the data in the second specific format as a second constraint condition into the diffusion model, and generate a predicted image of the edge graph.

[0011] Furthermore, based on the preset intra-class contrast learning strategy, the predicted image and the sample image are integrated and processed to obtain the loss function, which specifically includes:

[0012] Calculate the similarity between the feature information of the sample image and the feature information of multiple predicted images, and determine the contrast loss based on the preset intra-class contrast learning strategy and similarity; determine the reference predicted image from the multiple predicted images, calculate the mean square error between the reference predicted image and the sample image, and determine the reconstruction loss; obtain the preset loss weight hyperparameters, fuse the contrast loss, reconstruction loss, preset regularization loss and loss weight hyperparameters, and obtain the loss function.

[0013] Further, the similarity between the feature information of the sample image and the feature information of the plurality of predicted images is calculated, and the step of determining the contrast loss based on the preset intra-class contrast learning strategy and the similarity specifically includes:

[0014] Determine a target prediction image from multiple prediction images; calculate the similarity between feature information of the target prediction image and feature information of the sample image to obtain a first similarity; calculate the similarity between feature information of each prediction image and feature information of the sample image to obtain multiple second similarities; obtain a preset temperature parameter, and based on a preset intra-class contrast learning strategy, use a preset contrast loss formula to calculate the contrast loss for the first similarity, multiple second similarities and the temperature parameter.

[0015] Furthermore, after the step of determining the diffusion model corresponding to the loss function converging to the convergence threshold as the image generation model, the method further includes:

[0016] Obtain target structural feature information; use an image generation model to generate a target image for the target structural feature information, and the similarity between the structure of the target image and the structure indicated by the target structural feature information reaches a preset similarity threshold.

[0017] In order to solve the above technical problems, the embodiment of the present application also provides a training device for an image generation model, which adopts the following technical solutions:

[0018] An acquisition module, used to acquire a sample image and structural feature information corresponding to the sample image;

[0019] A generation module, used to use the structural feature information as a constraint condition, input a preset diffusion model, and generate multiple predicted images of the structural feature information;

[0020] An integration module is used to integrate multiple prediction images and sample images based on a preset intra-class contrast learning strategy to obtain a loss function;

[0021] An adjustment module, used for adjusting the parameters of the diffusion model based on the loss function, and returning to the step of inputting the structural feature information as a constraint condition into the preset diffusion model to obtain the loss function, until the loss function converges to a convergence threshold;

[0022] The determination module is used to determine the diffusion model corresponding to the loss function converging to the convergence threshold as the image generation model.

[0023] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the training method of the image generation model as mentioned above are implemented.

[0024] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-readable instructions. The computer-readable instructions can be executed by at least one processor to enable at least one processor to perform the steps of the training method of the image generation model as mentioned above.

[0025] Compared with the prior art, the embodiment of the present application mainly has the following beneficial effects: by obtaining a sample image and its structural feature information (such as a skeleton map or an edge map), it is input as a constraint into a preset diffusion model to generate multiple predicted images, providing a clear target feature and structural guidance for the diffusion model. These structural feature information are input into the preset diffusion model as constraints to generate multiple predicted images. This process ensures that the diffusion model can closely revolve around the input structural features when generating images. Subsequently, the predicted image, sample image and structural feature information are integrated and processed using an intra-class contrast learning strategy to construct a loss function. The core of this strategy is that it can help the diffusion model more accurately identify and distinguish target features from non-target areas (such as background noise). By continuously optimizing the loss function, the diffusion model gradually learns to reduce the interference of non-target areas during the generation process, while enhancing the capture and expression of target features. The introduction of intra-class contrast learning ensures that the generated image is highly matched with the input structural feature information in details, structure and features, effectively reducing the generation inconsistency problem caused by fuzzy input conditions. Finally, based on the feedback of the loss function, the parameters of the diffusion model are continuously adjusted, and the generation and optimization process is repeated until the loss function converges to the preset convergence threshold. This process not only improves the generation ability of the diffusion model, but also significantly enhances its anti-interference ability. When the loss function converges, the resulting image generation model is able to generate high-quality, structurally consistent, and anti-interference images in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0028] Figure 2 It is a flowchart of a training method for an image generation model provided in this application;

[0029] Figure 3 It is a structural schematic diagram of a training device for an image generation model provided by the present application;

[0030] Figure 4 It is a structural schematic diagram of a computer device provided by this application. DETAILED DESCRIPTION

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0032] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0034] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0035] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0036] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0037] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0038] It should be noted that the training method of the image generation model provided in the embodiment of the present application is generally executed by a server, and accordingly, the training device of the image generation model is generally set in the server.

[0039] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0040] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for training an image generation model according to the present application. The method for training an image generation model comprises the following steps:

[0041] Step S201: Acquire a sample image and structural feature information corresponding to the sample image.

[0042] In this embodiment, the training method of the image generation model is executed on the electronic device (eg Figure 1 The server shown in the figure) can obtain the sample image and the structural feature information corresponding to the sample image through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0043] The sample images refer to the original image data used to train or test the preset diffusion model. After preprocessing, these images are used to extract structural feature information to guide the training of the diffusion model. For example, in the image generation task, the sample images can be various landscape photos, portrait photos, etc. They contain rich structural and detail information, which are used to train the model to capture these features.

[0044] Among them, structural feature information refers to the key information extracted from the sample image to describe the internal structure and layout of the image. This information usually includes geometric features such as edges, straight lines, arcs, and their spatial relationships. For example, in image generation, structural feature information can include skeleton graphs or edge graphs. Structural feature information can be the outline of a building, the direction of a road, etc., which are crucial to generating an image that meets expectations.

[0045] Step S202: Taking the structural feature information as a constraint condition and inputting it into a preset diffusion model, a plurality of predicted images of the structural feature information are generated.

[0046] Among them, the constraint condition is a condition used to guide the model to generate an image according to specific requirements during the image generation process. In this embodiment, the structural feature information is input into the diffusion model as a constraint condition to control the structure and details of the generated image. For example, by specifying a specific edge feature matrix as a constraint condition, an image with a corresponding edge structure can be generated.

[0047] The diffusion model is a generative model that generates data by gradually adding noise to data samples and then learning how to recover the original data from these noisy samples. In this embodiment, the diffusion model is used to generate a predicted image based on structural feature information. For example, by training a diffusion model, it can learn to generate an image with a specific structure from noise.

[0048] The multiple predicted images refer to multiple candidate images generated by the diffusion model based on the input structural feature information. These images may differ in structure and details, but they should all conform to the input structural feature information. For example, when generating facial images, the diffusion model may generate multiple facial images with different expressions and postures, but their facial features all match the input structural feature information.

[0049] Step S203 , based on a preset intra-class comparison learning strategy, multiple prediction images and sample images are integrated to obtain a loss function.

[0050] Among them, the intra-class contrast learning strategy is a training strategy for improving the generalization ability of the model. It optimizes the representation ability of the model by calculating the similarity between different samples in the same category. In this embodiment, the intra-class contrast learning strategy is used to evaluate the similarity between the feature representations of multiple predicted images and the feature representations of sample images, thereby guiding the training of the diffusion model.

[0051] Among them, integration processing refers to the process of comprehensively processing multiple predicted images and sample images to calculate the loss function. This includes calculating indicators such as similarity and mean square error, and fusing these indicators into the final loss value.

[0052] Among them, the loss function is an indicator used to evaluate the performance of the generation model, which measures the difference between the generated image and the real image. In this embodiment, the loss function is calculated based on the intra-class contrast learning strategy, including contrast loss, reconstruction loss, and regularization loss. By optimizing the loss function, the performance of the diffusion model can be continuously improved. For example, a smaller loss function value means that the generated image is closer to the real image.

[0053] Step S204, based on the loss function, adjust the parameters of the diffusion model, and return to the step of inputting the structural feature information as a constraint condition into the preset diffusion model to obtain the loss function, until the loss function converges to a convergence threshold.

[0054] The parameters of the diffusion model refer to the variables and coefficients used to describe the data generation process within the model. These parameters are continuously optimized through the training process to improve the generation ability of the model. For example, in the diffusion model, it may include parameters that describe the process of adding and removing noise, as well as parameters that control the details of the generated image.

[0055] The convergence threshold refers to the threshold for judging whether the loss function has converged to a stable value. When the value of the loss function is less than or equal to the convergence threshold, the model can be considered to have been trained. For example, when training a diffusion model, a convergence threshold can be set. When the value of the loss function is lower than the threshold, the training is stopped and the model is saved to obtain an image generation model.

[0056] Step S205: determine the diffusion model corresponding to the loss function converging to the convergence threshold as the image generation model.

[0057] The image generation model refers to a model that can generate images according to input conditions after training. In this embodiment, the image generation model is obtained by optimizing the parameters of the diffusion model, and it can generate images that meet the requirements according to the input structural feature information. For example, a trained image generation model can generate an image with a corresponding structure according to the edge feature matrix provided by the user.

[0058] The embodiment of the present application can obtain a sample image and its structural feature information (such as a skeleton image or an edge image), input it as a constraint condition into a preset diffusion model, generate multiple predicted images, and provide a clear target feature and structural guidance for the diffusion model. These structural feature information are input into the preset diffusion model as constraints to generate multiple predicted images. This process ensures that the diffusion model can closely revolve around the input structural features when generating images. Subsequently, the predicted image, sample image and structural feature information are integrated and processed using the intra-class contrast learning strategy to construct a loss function. The core of this strategy is that it can help the diffusion model to more accurately identify and distinguish target features from non-target areas (such as background noise). By continuously optimizing the loss function, the diffusion model gradually learns to reduce the interference of non-target areas during the generation process, while enhancing the capture and expression of target features. The introduction of intra-class contrast learning ensures that the generated image is highly matched with the input structural feature information in details, structure and features, effectively reducing the generation inconsistency problem caused by fuzzy input conditions. Finally, based on the feedback of the loss function, the parameters of the diffusion model are continuously adjusted, and the generation and optimization process is repeated until the loss function converges to a preset convergence threshold. This process not only improves the generation ability of the diffusion model, but also significantly enhances its anti-interference ability. When the loss function converges, the resulting image generation model is already able to generate high-quality, structurally consistent, and anti-interference images in complex scenes.

[0059] In some optional implementations of this embodiment, step 201, obtaining a sample image and structural feature information corresponding to the sample image, specifically includes the following steps:

[0060] Acquire a sample image, preprocess the sample image, and obtain a preprocessed feature image; use an edge detection algorithm to perform edge detection on the feature image to obtain edge information of the feature image; generate an edge feature matrix of the feature image based on the edge information; use a Hough transform algorithm to analyze the edge feature matrix to determine the straight line information and circular arc structured information in the feature image; and construct the structural feature information of the feature image based on the straight line information and circular arc structured information.

[0061] Among them, the feature image refers to an image that is used to extract structural feature information after preprocessing. It can be a simplified representation of the original image, containing less redundant information but retaining key structures and details. For example, in the edge detection process, the original image is converted into a feature image to make it easier to extract edge information.

[0062] Among them, the edge detection algorithm is an image processing technology used to identify edge features in an image. It detects edges by calculating the rate of change of pixel values ​​in an image. For example, the Canny edge detection algorithm is a commonly used edge detection algorithm that can generate clear and continuous edge images.

[0063] Among them, edge information refers to the edge features extracted from the feature image by the edge detection algorithm. It describes the outline and shape of the object in the image and is an important part of the image structure feature. For example, in the image generation task, edge information can be used to guide the generation model to generate images with specific shapes.

[0064] Among them, the edge feature matrix is ​​a matrix form used to represent edge information. It encodes edge information as element values ​​in the matrix for subsequent processing and analysis. For example, in the edge feature matrix, a high value can represent an edge position, and a low value can represent a non-edge position.

[0065] Among them, the Hough transform algorithm is an image processing technology used to detect structural features such as straight lines and arcs in an image. It identifies these features by mapping points in the image space to the parameter space. For example, when detecting the edge of a road, the Hough transform algorithm can identify the straight line features of the edge of the road.

[0066] The straight line information refers to the straight line features extracted from the edge feature matrix by the Hough transform algorithm. It describes the position, direction, and length of the straight line in the image. For example, in the building recognition task, the straight line information can be used to identify the outline and structure of the building.

[0067] The arc structured information refers to the arc features extracted from the edge feature matrix by the Hough transform algorithm. It describes the position, radius, angle and other information of the arc in the image. For example, when identifying a circular object, the arc structured information can be used to determine the shape and position of the object.

[0068] In one example, in the financial field, sample images of financial bills can be obtained through devices such as scanners or cameras. The sample images are preprocessed by grayscale, denoising, binarization, etc. to obtain preprocessed feature images. These preprocessing steps help reduce noise and interference in the image and improve the accuracy of subsequent processing. The Canny edge detection algorithm is used to detect the edges of the feature image to obtain the edge information of the feature image. Edge information is an important feature of key elements (such as text, lines, borders, etc.) in the bill. Based on the edge information, an edge feature matrix of the feature image is generated. The matrix can clearly represent the edge position and shape of each element in the bill. The Hough transform algorithm is used to analyze the edge feature matrix to determine the straight line information and arc structured information in the feature image. Based on the straight line information and arc structured information, the structural feature information of the feature image is constructed.

[0069] The embodiment of the present application can effectively remove redundant information and noise in the image and enhance the key features in the image by preprocessing the sample image, thereby obtaining the preprocessed feature image. Subsequently, the edge detection algorithm is used to perform edge detection on the feature image, which can accurately extract the edge information in the image and provide a reliable data basis for subsequent processing. Further, the edge feature matrix is ​​generated based on the edge information, which can clearly indicate the edge position and shape of each element in the image, and provides strong support for the subsequent Hough transform analysis. By analyzing the edge feature matrix through the Hough transform algorithm, the straight line information and arc structured information in the feature image can be accurately determined, which is an important basis for constructing the image structural features. Finally, the structural feature information of the feature image is constructed based on the straight line information and arc structured information, which realizes the accurate identification and extraction of the key elements in the image, and provides a strong guarantee for the subsequent image analysis and processing.

[0070] In some optional implementations of this embodiment, step S203, taking the structural feature information as a constraint condition, inputting a preset diffusion model, and generating a predicted image of the structural feature information, specifically includes the following steps:

[0071] Identify the type of structural feature information; if the structural feature information is a skeleton graph, convert the skeleton graph into data in a first specific format, input the data in the first specific format as a first constraint condition into a preset diffusion model, and generate multiple predicted images of the skeleton graph; if the structural feature information is an edge graph, convert the edge graph into data in a second specific format, input the data in the second specific format as a second constraint condition into the diffusion model, and generate a predicted image of the edge graph.

[0072] Among them, the skeleton graph is a simplified graph used to represent the structural features of an image. It retains the main structure and connection relationship of the image, but removes redundant detail information. For example, in the image generation task, the skeleton graph can be input into the diffusion model as a constraint to generate an image with a specific structure.

[0073] The data in the first specific format refers to the conversion of the skeleton image into a format suitable for input into the diffusion model. This format may include a specific data structure and encoding method to ensure that the model can correctly parse and process the input information. For example, the data in the first specific format may be the conversion of the skeleton image into a binary or floating point matrix.

[0074] The edge map refers to an image containing image edge information. It is usually extracted from the original image through an edge detection algorithm and is used to represent the structure and shape characteristics of the image. For example, in image generation tasks, the edge map can be input into the diffusion model as a constraint to generate an image with a specific edge structure.

[0075] The data in the second specific format refers to the edge map converted into a format suitable for input into the diffusion model. Similar to the data in the first specific format, this format may include a specific data structure and encoding method. For example, the data in the second specific format may be the edge map converted into a grayscale image or a color image.

[0076] In one example, the type of input structural feature information can be first determined. If it is a skeleton graph, it is converted into data in a first specific format (such as JSON, XML, etc., for model processing). Subsequently, the converted data is input into a preset diffusion model as a first constraint. The diffusion model is a probability-based generation model that can gradually generate images that match the constraints from noise. By adjusting the model parameters and the number of iterations, multiple predicted images of the skeleton graph can be generated, which have rich details and textures while maintaining the skeleton structure. If the input structural feature information is an edge graph, it is converted into data in a second specific format (also for model processing). Then, the converted data is input into the same diffusion model as a second constraint. Since the edge graph emphasizes the contour information of the image, the generated predicted image will pay more attention to the boundaries and shapes of the image.

[0077] The embodiments of the present application can realize highly customized image prediction generation for two different structural feature information, skeleton map and edge map. For the skeleton map, by converting it into data in a first specific format and inputting it into a preset diffusion model as a first constraint, the topological structure information of the skeleton map can be fully utilized to generate multiple predicted images that not only maintain the original skeleton features but are also rich in details, providing rich visual references for design, simulation and other fields. For the edge map, by converting it into data in a second specific format and using it as a second constraint, the diffusion model can focus on the contour information of the image and generate a predicted image that is highly matched with the edge map. This method not only improves the accuracy and efficiency of image generation, but also retains the key shape features in the edge map, bringing significant improvements to image processing, image restoration and other fields.

[0078] In some optional implementations of this embodiment, step S203, based on a preset intra-class contrast learning strategy, integrates the predicted image and the sample image to obtain a loss function, specifically including the following steps:

[0079] Calculate the similarity between the feature information of the sample image and the feature information of multiple predicted images, and determine the contrast loss based on the preset intra-class contrast learning strategy and similarity; determine the reference predicted image from the multiple predicted images, calculate the mean square error between the reference predicted image and the sample image, and determine the reconstruction loss; obtain the preset loss weight hyperparameters, fuse the contrast loss, reconstruction loss, preset regularization loss and loss weight hyperparameters, and obtain the loss function.

[0080] Among them, contrastive loss is the part of the loss function used to evaluate the similarity between the generated image and the sample image. It evaluates the performance of the model by calculating the feature difference between the generated image and the sample image. For example, in the intra-class contrastive learning strategy, contrastive loss can measure the similarity difference between each predicted image and the sample image.

[0081] The reference prediction image refers to an image selected from multiple prediction images for calculating the reconstruction loss. For example, the best prediction image is selected based on the similarity with the sample image. For example, in an image generation task, the prediction image that is most similar to the sample image can be selected as the reference prediction image.

[0082] Among them, mean square error is a commonly used error metric that calculates the average of the squares of the differences between the predicted value and the true value. In image generation tasks, mean square error can be used to evaluate the pixel-level difference between the generated image and the sample image. For example, by calculating the mean square error between the reference predicted image and the sample image, the quality of the generated image can be evaluated.

[0083] Among them, reconstruction loss is the part of the loss function used to evaluate the difference between the generated image and the sample image. It evaluates the performance of the model by calculating the pixel-level error between the generated image and the sample image. For example, in the image generation task, reconstruction loss can measure the pixel difference between the generated image and the sample image.

[0084] The loss weight hyperparameters are parameters used to adjust the weights of different parts of the loss function. By adjusting these parameters, you can control the model's attention to different types of errors. For example, when training a diffusion model, you can balance the weights between contrast loss, reconstruction loss, and regularization loss by adjusting the loss weight hyperparameters.

[0085] Among them, regularization loss is the part of the loss function used to prevent the model from overfitting. It limits the complexity of the model by introducing additional constraints, thereby improving the generalization ability of the model. For example, when training a diffusion model, you can add regularization terms to limit the range of model parameters to reduce the risk of overfitting.

[0086] Among them, fusion processing refers to the process of combining multiple loss function parts into the final loss function.

[0087] In one example, the similarity between the feature information of the sample image and the feature information of multiple predicted images can be calculated. The feature information can be key attributes of the image such as texture, color, shape, etc., which are obtained by a feature extraction algorithm (such as a convolutional neural network). The similarity calculation can use a measurement method such as cosine similarity and Euclidean distance. Subsequently, based on the preset intra-class contrast learning strategy and similarity, the contrast loss is determined. The intra-class contrast learning strategy is a method for optimizing a model by comparing the differences between different samples in the same category. In this embodiment, the method is used to evaluate the consistency of the predicted image and the sample image in the feature space, thereby determining the contrast loss to guide the predicted image to approach the feature space of the sample image. Next, a reference predicted image is determined from the multiple predicted images. The reference predicted image can be the one with the highest similarity and the best quality among the multiple predicted images. Then, the mean square error between the reference predicted image and the sample image is calculated to determine the reconstruction loss. The mean square error is a commonly used image quality assessment indicator, which evaluates the similarity of images by calculating the average of the sum of the squared differences between the pixel values ​​of two images. Finally, the preset loss weight hyperparameters are obtained, and the contrast loss, reconstruction loss, preset regularization loss, and loss weight hyperparameters are integrated to obtain the final loss function. For example, the loss function is calculated according to the loss function formula (1), and the loss function formula (1) is as follows:

[0088] L=L con +ω1L recon +ω2L regular ……(1)

[0089] Where L con is the contrast loss, L recon and L regular are the reconstruction loss and regularization loss, respectively, and ω1 and ω2 are loss weight hyperparameters. The loss weight hyperparameters are used to balance the weights between different loss terms to ensure that the loss function can fully and accurately reflect the difference between the predicted image and the sample image.

[0090] The embodiment of the present application can accurately calculate the similarity of feature information between a sample image and multiple predicted images, and then determine the contrast loss based on the intra-class contrast learning strategy. This strategy effectively enhances the model's ability to learn sample image features, making the predicted image closer to the sample image at the feature level. At the same time, the reconstruction loss is determined by calculating the mean square error between the reference predicted image selected from multiple predicted images and the sample image. This step further improves the degree of restoration of the predicted image at the pixel level. In addition, combined with the preset loss weight hyperparameters, the contrast loss, reconstruction loss and regularization loss are fused, and the resulting loss function not only fully reflects the difference between the predicted image and the sample image, but also achieves balanced optimization of different loss terms through weight adjustment, thereby improving the accuracy and efficiency of image prediction and generation.

[0091] In some optional implementations of this embodiment, the step of “calculating the similarity between the feature information of the sample image and the feature information of the plurality of predicted images, and determining the contrast loss based on the preset intra-class contrast learning strategy and the similarity” specifically includes the following steps:

[0092] Determine a target prediction image from multiple prediction images; calculate the similarity between feature information of the target prediction image and feature information of the sample image to obtain a first similarity; calculate the similarity between feature information of each prediction image and feature information of the sample image to obtain multiple second similarities; obtain a preset temperature parameter, and based on a preset intra-class contrast learning strategy, use a preset contrast loss formula to calculate the contrast loss for the first similarity, multiple second similarities and the temperature parameter.

[0093] The target prediction image refers to an image selected from multiple prediction images that best meets the requirements, for example, the best prediction image selected based on the similarity with the sample image.

[0094] Among them, the first similarity refers to the similarity between the feature information of the target prediction image determined from multiple prediction images and the feature information of the sample image during the image generation process. This similarity is obtained by calculating the distance or correlation between the two in the feature space, and is usually derived from the comparison of image feature vectors. It characterizes the degree of closeness between the target prediction image and the sample image in terms of overall structure and details, and is used to evaluate the image generation model's ability to restore the sample image. When calculating, indicators such as cosine similarity and Euclidean distance can be used.

[0095] Among them, the second similarity refers to the similarity between the feature information of each predicted image and the feature information of the sample image during the image generation process. Similar to the first similarity, the second similarity is also obtained by calculating the distance or correlation between feature vectors, but it is for all generated predicted images, not just the target predicted image. It characterizes the degree of proximity between each predicted image and the sample image in the feature space, and is used to evaluate the overall generation quality of the image generation model. When calculating, indicators such as cosine similarity and Euclidean distance can also be used.

[0096] Among them, the temperature parameter is a hyperparameter introduced in the intra-class contrastive learning strategy, which is used to adjust the sensitivity of similarity calculation. It comes from the design of the intra-class contrastive learning mechanism and represents the degree of attention paid to the differences between different samples when calculating similarity. The temperature parameter affects the smoothness of the similarity distribution, thereby adjusting the model's ability to capture subtle differences between samples. In image generation, the temperature parameter can be used to balance the diversity and accuracy of generated images. For example, when the temperature parameter is high, the model may pay more attention to the diversity of generated images and allow larger differences to exist; when the temperature parameter is low, the model may pay more attention to the accuracy of generated images and be more sensitive to subtle differences.

[0097] Among them, the contrast loss formula is a loss function used to calculate the similarity difference between the predicted image and the sample image during the image generation process. It comes from the mathematical expression of the intra-class contrast learning mechanism and represents the error between the model and the target image when generating an image. In image generation, the contrast loss formula can be used to improve the quality of the generated image and make it closer to the sample image.

[0098] In one example, a target prediction image is determined from multiple prediction images. The target prediction image can be the optimal prediction image selected based on quality, similarity and other criteria. Next, the similarity between the feature information of the target prediction image and the feature information of the sample image is calculated to obtain a first similarity. The feature information can be key attributes of the image such as texture, color, shape, etc., which are obtained through a feature extraction algorithm (such as a convolutional neural network). The similarity calculation can use a metric method such as cosine similarity and Euclidean distance. Then, the similarity between the feature information of each prediction image and the feature information of the sample image is calculated to obtain multiple second similarities. This step aims to comprehensively evaluate the similarity between all prediction images and the sample images, and provide data support for subsequent optimization. Subsequently, a preset temperature parameter is obtained. The temperature parameter is an important parameter in the intra-class contrast learning strategy, which is used to adjust the smoothness of the similarity distribution. Finally, based on the preset intra-class contrast learning strategy, combined with the first similarity, multiple second similarities and the temperature parameter, the preset contrast loss formula (2) is used to calculate the contrast loss L con Among them, the contrast loss formula (2) is as follows:

[0099]

[0100] Among them, f(x i ) and f(y i ) are the structural feature information (e.g., skeleton image or edge image) and the feature representation of the generated image (target prediction image), respectively, f(y j ) represents the feature representation of all images (multiple predicted images) generated by the model, sim is the cosine similarity, and τ is the temperature parameter.

[0101] The embodiment of the present application can accurately calculate the feature similarity between the target prediction image and the sample image in multiple prediction images, while considering the similarity distribution of all prediction images and the sample images. By introducing a preset temperature parameter, combining the intra-class contrast learning strategy, and adopting a specific contrast loss formula, the calculated contrast loss not only reflects the proximity between the target prediction image and the sample image in the feature space, but also takes into account the smoothness of the similarity distribution. This method effectively enhances the model's ability to learn the features of the sample image, so that the generated prediction image has better generalization performance while maintaining a high degree of consistency with the sample image. In addition, through the optimization of the contrast loss, the generation process of the prediction image can be further guided to improve the accuracy and efficiency of tasks such as image restoration and image enhancement.

[0102] In some optional implementations of this embodiment, after the diffusion model corresponding to the loss function converging to the convergence threshold is determined as the image generation model in step S205, the following steps may be specifically included:

[0103] Obtain target structural feature information; use an image generation model to generate a target image for the target structural feature information, and the similarity between the structure of the target image and the structure indicated by the target structural feature information reaches a preset similarity threshold.

[0104] The target structural feature information refers to the specific structural features that the user wants to generate in the image generation task. It comes from the description or definition of the target image and characterizes the key features of the target image in terms of structure, shape, texture, etc. In the image generation process, the target structural feature information is input into the generation model as a constraint to guide the model to generate images that meet the requirements.

[0105] Among them, the similarity threshold refers to a threshold used to judge the similarity between the generated image and the structure indicated by the target structural feature information in the image generation task. It is derived from the evaluation criteria for the quality of the generated image and represents the minimum acceptable similarity between the generated image and the target structural feature information. In the image generation process, when the similarity between the generated image and the target structural feature information reaches or exceeds the similarity threshold, the generated image is considered to meet the requirements. The setting of the similarity threshold can be adjusted according to the specific application scenario and requirements.

[0106] In one example, the target structural feature information is taken as the key features such as the position and shape of the facial features of a human face. By inputting the target structural feature information into a trained image generation model, a face image highly similar to the position and shape of the facial features of the target structural feature information can be generated. For example, in the generation of architectural design drawings, the target structural feature information may include key information such as the outline of the building, floor layout, door and window positions, etc., which will be used as input to the generation model to generate architectural design drawings that meet the design requirements.

[0107] The embodiment of the present application can generate a target image that matches the input target structure feature information efficiently and accurately through an image generation model obtained through prior training. The image generation process makes full use of the advantages of the diffusion model and intra-class contrast learning to ensure that the generated target image is highly consistent with the structure indicated by the target structure feature information in structure, and the degree of similarity reaches a preset similarity threshold. Specifically, the image generation model can capture subtle differences in the target structure feature information and accurately reproduce these features in the generated target image, thereby achieving accurate reproduction of the target structure.

[0108] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned sample images and structural feature information, the above-mentioned sample images and structural feature information can also be stored in a node of a blockchain.

[0109] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0110] The embodiments of the present application can build and optimize related models and networks based on artificial intelligence technology, such as diffusion models, image generation models, etc. Among them, artificial intelligence (AI) models are the crystallization of theory and practice that simulate the decision-making process of human intelligence through algorithms and data analysis to solve complex problems, predict future trends, or realize automation tasks. These models use a large amount of historical data and real-time information, and are trained and optimized through a specific algorithm framework to achieve efficient, accurate and reliable performance.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0112] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0113] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a training device for an image generation model, and the device embodiment is similar to Figure 2Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0114] like Figure 3 As shown, the training device 400 of the image generation model of this embodiment includes: an acquisition module 401, a generation module 402, an integration module 403, an adjustment module 404 and a determination module 405. Among them:

[0115] An acquisition module 401 is used for acquiring a sample image and structural feature information corresponding to the sample image;

[0116] A generating module 402 is used to use the structural feature information as a constraint condition, input a preset diffusion model, and generate a plurality of predicted images of the structural feature information;

[0117] An integration module 403 is used to integrate multiple prediction images and sample images based on a preset intra-class comparison learning strategy to obtain a loss function;

[0118] An adjustment module 404 is used to adjust the parameters of the diffusion model based on the loss function, and return to the step of inputting the structural feature information as a constraint condition into the preset diffusion model to obtain the loss function, until the loss function converges to a convergence threshold;

[0119] The determination module 405 is used to determine the diffusion model corresponding to the loss function converging to the convergence threshold as the image generation model.

[0120] The embodiment of the present application can obtain a sample image and its structural feature information (such as a skeleton image or an edge image), input it as a constraint condition into a preset diffusion model, generate multiple predicted images, and provide a clear target feature and structural guidance for the diffusion model. These structural feature information are input into the preset diffusion model as constraints to generate multiple predicted images. This process ensures that the diffusion model can closely revolve around the input structural features when generating images. Subsequently, the predicted image, sample image and structural feature information are integrated and processed using the intra-class contrast learning strategy to construct a loss function. The core of this strategy is that it can help the diffusion model to more accurately identify and distinguish target features from non-target areas (such as background noise). By continuously optimizing the loss function, the diffusion model gradually learns to reduce the interference of non-target areas during the generation process, while enhancing the capture and expression of target features. The introduction of intra-class contrast learning ensures that the generated image is highly matched with the input structural feature information in details, structure and features, effectively reducing the generation inconsistency problem caused by fuzzy input conditions. Finally, based on the feedback of the loss function, the parameters of the diffusion model are continuously adjusted, and the generation and optimization process is repeated until the loss function converges to a preset convergence threshold. This process not only improves the generation ability of the diffusion model, but also significantly enhances its anti-interference ability. When the loss function converges, the resulting image generation model is already able to generate high-quality, structurally consistent, and anti-interference images in complex scenes.

[0121] In one embodiment, the acquisition module 401 includes:

[0122] The preprocessing submodule is used to obtain a sample image, preprocess the sample image, and obtain a preprocessed feature image;

[0123] The detection submodule is used to perform edge detection on the feature image using an edge detection algorithm to obtain edge information of the feature image;

[0124] A generation submodule is used to generate an edge feature matrix of a feature image according to edge information;

[0125] An analysis submodule is used to analyze the edge feature matrix using a Hough transform algorithm to determine the straight line information and arc structure information in the feature image;

[0126] The construction submodule is used to construct the structural feature information of the feature image according to the straight line information and the arc structured information.

[0127] The embodiment of the present application can effectively remove redundant information and noise in the image and enhance the key features in the image by preprocessing the sample image, thereby obtaining the preprocessed feature image. Subsequently, the edge detection algorithm is used to perform edge detection on the feature image, which can accurately extract the edge information in the image and provide a reliable data basis for subsequent processing. Further, the edge feature matrix is ​​generated based on the edge information, which can clearly indicate the edge position and shape of each element in the image, and provides strong support for the subsequent Hough transform analysis. By analyzing the edge feature matrix through the Hough transform algorithm, the straight line information and arc structured information in the feature image can be accurately determined, which is an important basis for constructing the image structural features. Finally, the structural feature information of the feature image is constructed based on the straight line information and arc structured information, which realizes the accurate identification and extraction of the key elements in the image, and provides a strong guarantee for the subsequent image analysis and processing.

[0128] In one embodiment, the generating module 402 includes:

[0129] An identification submodule, used to identify the type of structural feature information;

[0130] A first conversion submodule is used for converting the skeleton image into data in a first specific format if the structural feature information is a skeleton image, inputting the data in the first specific format into a preset diffusion model as a first constraint condition, and generating a plurality of predicted images of the skeleton image;

[0131] The second conversion submodule is used for converting the edge map into data in a second specific format if the structural feature information is an edge map, inputting the data in the second specific format into the diffusion model as a second constraint condition, and generating a predicted image of the edge map.

[0132] The embodiments of the present application can realize highly customized image prediction generation for two different structural feature information, skeleton map and edge map. For the skeleton map, by converting it into data in a first specific format and inputting it into a preset diffusion model as a first constraint, the topological structure information of the skeleton map can be fully utilized to generate multiple predicted images that not only maintain the original skeleton features but are also rich in details, providing rich visual references for design, simulation and other fields. For the edge map, by converting it into data in a second specific format and using it as a second constraint, the diffusion model can focus on the contour information of the image and generate a predicted image that is highly matched with the edge map. This method not only improves the accuracy and efficiency of image generation, but also retains the key shape features in the edge map, bringing significant improvements to image processing, image restoration and other fields.

[0133] In one embodiment, the integration module 403 includes:

[0134] A first calculation submodule, used to calculate the similarity between the feature information of the sample image and the feature information of the plurality of predicted images, and determine the contrast loss based on a preset intra-class contrast learning strategy and the similarity;

[0135] A second calculation submodule, used for a submodule, for determining a reference prediction image from a plurality of prediction images, calculating a mean square error between the reference prediction image and the sample image, and determining a reconstruction loss;

[0136] The fusion submodule is used for the submodule to obtain the preset loss weight hyperparameters, fuse the contrast loss, reconstruction loss, preset regularization loss and loss weight hyperparameters to obtain the loss function.

[0137] The embodiment of the present application can accurately calculate the similarity of feature information between a sample image and multiple predicted images, and then determine the contrast loss based on the intra-class contrast learning strategy. This strategy effectively enhances the model's ability to learn sample image features, making the predicted image closer to the sample image at the feature level. At the same time, the reconstruction loss is determined by calculating the mean square error between the reference predicted image selected from multiple predicted images and the sample image. This step further improves the degree of restoration of the predicted image at the pixel level. In addition, combined with the preset loss weight hyperparameters, the contrast loss, reconstruction loss and regularization loss are fused, and the resulting loss function not only fully reflects the difference between the predicted image and the sample image, but also achieves balanced optimization of different loss terms through weight adjustment, thereby improving the accuracy and efficiency of image prediction and generation.

[0138] In one embodiment, the first calculation submodule is also used to determine a target prediction image from multiple prediction images; calculate the similarity between the feature information of the target prediction image and the feature information of the sample image to obtain a first similarity; calculate the similarity between the feature information of each prediction image and the feature information of the sample image to obtain multiple second similarities; obtain a preset temperature parameter, and based on a preset intra-class contrast learning strategy, use a preset contrast loss formula to calculate the contrast loss for the first similarity, multiple second similarities and temperature parameter.

[0139] The embodiment of the present application can accurately calculate the feature similarity between the target prediction image and the sample image in multiple prediction images, while considering the similarity distribution of all prediction images and the sample images. By introducing a preset temperature parameter, combining the intra-class contrast learning strategy, and adopting a specific contrast loss formula, the calculated contrast loss not only reflects the proximity between the target prediction image and the sample image in the feature space, but also takes into account the smoothness of the similarity distribution. This method effectively enhances the model's ability to learn the features of the sample image, so that the generated prediction image has better generalization performance while maintaining a high degree of consistency with the sample image. In addition, through the optimization of the contrast loss, the generation process of the prediction image can be further guided to improve the accuracy and efficiency of tasks such as image restoration and image enhancement.

[0140] In one embodiment, the training device for the image generation model may further include:

[0141] An information acquisition module is used to obtain target structure feature information;

[0142] The image generation module is used to generate a target image based on the target structure feature information by using an image generation model, and the similarity between the structure of the target image and the structure indicated by the target structure feature information reaches a preset similarity threshold.

[0143] The embodiment of the present application can generate a target image that matches the input target structure feature information efficiently and accurately through an image generation model obtained through prior training. The image generation process makes full use of the advantages of the diffusion model and intra-class contrast learning to ensure that the generated target image is highly consistent with the structure indicated by the target structure feature information in structure, and the degree of similarity reaches a preset similarity threshold. Specifically, the image generation model can capture subtle differences in the target structure feature information and accurately reproduce these features in the generated target image, thereby achieving accurate reproduction of the target structure.

[0144] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0145] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 6 having a memory 61, a processor 62, and a network interface 63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0146] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0147] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions of the training method of the image generation model, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0148] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run computer-readable instructions or process data stored in the memory 61, such as computer-readable instructions for running a training method for an image generation model.

[0149] The network interface 63 may include a wireless network interface or a wired network interface, and the network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0150] The embodiment of the present application can obtain a sample image and its structural feature information (such as a skeleton image or an edge image), input it as a constraint condition into a preset diffusion model, generate multiple predicted images, and provide a clear target feature and structural guidance for the diffusion model. These structural feature information are input into the preset diffusion model as constraints to generate multiple predicted images. This process ensures that the diffusion model can closely revolve around the input structural features when generating images. Subsequently, the predicted image, sample image and structural feature information are integrated and processed using the intra-class contrast learning strategy to construct a loss function. The core of this strategy is that it can help the diffusion model to more accurately identify and distinguish target features from non-target areas (such as background noise). By continuously optimizing the loss function, the diffusion model gradually learns to reduce the interference of non-target areas during the generation process, while enhancing the capture and expression of target features. The introduction of intra-class contrast learning ensures that the generated image is highly matched with the input structural feature information in details, structure and features, effectively reducing the generation inconsistency problem caused by fuzzy input conditions. Finally, based on the feedback of the loss function, the parameters of the diffusion model are continuously adjusted, and the generation and optimization process is repeated until the loss function converges to a preset convergence threshold. This process not only improves the generation ability of the diffusion model, but also significantly enhances its anti-interference ability. When the loss function converges, the resulting image generation model is already able to generate high-quality, structurally consistent, and anti-interference images in complex scenes.

[0151] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the training method of the image generation model as described above.

[0152] The embodiment of the present application can obtain a sample image and its structural feature information (such as a skeleton image or an edge image), input it as a constraint condition into a preset diffusion model, generate multiple predicted images, and provide a clear target feature and structural guidance for the diffusion model. These structural feature information are input into the preset diffusion model as constraints to generate multiple predicted images. This process ensures that the diffusion model can closely revolve around the input structural features when generating images. Subsequently, the predicted image, sample image and structural feature information are integrated and processed using the intra-class contrast learning strategy to construct a loss function. The core of this strategy is that it can help the diffusion model to more accurately identify and distinguish target features from non-target areas (such as background noise). By continuously optimizing the loss function, the diffusion model gradually learns to reduce the interference of non-target areas during the generation process, while enhancing the capture and expression of target features. The introduction of intra-class contrast learning ensures that the generated image is highly matched with the input structural feature information in details, structure and features, effectively reducing the generation inconsistency problem caused by fuzzy input conditions. Finally, based on the feedback of the loss function, the parameters of the diffusion model are continuously adjusted, and the generation and optimization process is repeated until the loss function converges to a preset convergence threshold. This process not only improves the generation ability of the diffusion model, but also significantly enhances its anti-interference ability. When the loss function converges, the resulting image generation model is already able to generate high-quality, structurally consistent, and anti-interference images in complex scenes.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0154] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A training method for an image generation model, characterized in that: The steps include: Acquire a sample image and structural feature information corresponding to the sample image; The structural feature information is used as a constraint condition and input into a preset diffusion model to generate a plurality of predicted images of the structural feature information; Based on a preset intra-class contrast learning strategy, the multiple predicted images and the sample image are integrated to obtain a loss function; Based on the loss function, adjusting the parameters of the diffusion model, returning to the step of inputting the structural feature information as a constraint condition into the preset diffusion model to obtain the loss function, until the loss function converges to a convergence threshold; The diffusion model corresponding to the loss function converging to a convergence threshold is determined as the image generation model.

2. The method according to claim 1, characterized in that: The step of obtaining the sample image and the structural feature information corresponding to the sample image specifically includes: Acquire a sample image, and preprocess the sample image to obtain a preprocessed feature image; Using an edge detection algorithm to perform edge detection on the feature image to obtain edge information of the feature image; Generating an edge feature matrix of the feature image according to the edge information; Using the Hough transform algorithm, the edge feature matrix is ​​analyzed to determine the straight line information and arc structure information in the feature image; The structural feature information of the feature image is constructed according to the straight line information and the arc structured information.

3. The method according to claim 1, characterized in that The step of using the structural feature information as a constraint condition and inputting a preset diffusion model to generate a predicted image of the structural feature information specifically includes: Identifying the type of the structural feature information; If the structural feature information is a skeleton graph, converting the skeleton graph into data in a first specific format, inputting the data in the first specific format into a preset diffusion model as a first constraint condition, and generating a plurality of predicted images of the skeleton graph; If the structural feature information is an edge map, the edge map is converted into data in a second specific format, and the data in the second specific format is input into the diffusion model as a second constraint condition to generate a predicted image of the edge map.

4. The method according to claim 1, characterized in that: The step of integrating the predicted image and the sample image based on the preset intra-class contrast learning strategy to obtain the loss function specifically includes: Calculating the similarity between the feature information of the sample image and the feature information of the plurality of predicted images, and determining the contrast loss based on a preset intra-class contrast learning strategy and the similarity; Determine a reference prediction image from the multiple prediction images, calculate a mean square error between the reference prediction image and the sample image, and determine a reconstruction loss; A preset loss weight hyperparameter is obtained, and the contrast loss, the reconstruction loss, the preset regularization loss and the loss weight hyperparameter are fused to obtain a loss function.

5. The method according to claim 4, characterized in that The step of calculating the similarity between the feature information of the sample image and the feature information of the plurality of predicted images, and determining the contrast loss based on a preset intra-class contrast learning strategy and the similarity, specifically includes: determining a target predicted image from the plurality of predicted images; Calculating the similarity between the feature information of the target prediction image and the feature information of the sample image to obtain a first similarity; Calculating the similarity between the feature information of each predicted image and the feature information of the sample image to obtain a plurality of second similarities; A preset temperature parameter is obtained, and based on a preset intra-class contrast learning strategy, a preset contrast loss formula is used to calculate the contrast loss for the first similarity, the plurality of second similarities, and the temperature parameter.

6. The method according to claim 1, characterized in that After the step of determining the diffusion model corresponding to the loss function converging to a convergence threshold as the image generation model, the method further includes: Obtain target structural feature information; The target image is generated by adopting the image generation model for the target structural feature information, and the similarity between the structure of the target image and the structure indicated by the target structural feature information reaches a preset similarity threshold.

7. A training device for an image generation model, characterized in that: include: An acquisition module, used to acquire a sample image and structural feature information corresponding to the sample image; A generating module, used to use the structural feature information as a constraint condition, input a preset diffusion model, and generate a plurality of predicted images of the structural feature information; An integration module, used for integrating the multiple predicted images and the sample image based on a preset intra-class comparison learning strategy to obtain a loss function; An adjustment module, configured to adjust the parameters of the diffusion model based on the loss function, and return to the step of inputting the structural feature information as a constraint condition into the preset diffusion model to obtain the loss function, until the loss function converges to a convergence threshold; A determination module is used to determine the diffusion model corresponding to the loss function converging to a convergence threshold as an image generation model.

8. The device according to claim 7, characterized in that The acquisition module comprises: A preprocessing submodule is used to obtain a sample image, preprocess the sample image, and obtain a preprocessed feature image; A detection submodule, configured to perform edge detection on the feature image using an edge detection algorithm to obtain edge information of the feature image; A generating submodule, used for generating an edge feature matrix of the feature image according to the edge information; An analysis submodule, for analyzing the edge feature matrix by using a Hough transform algorithm, and determining straight line information and arc structure information in the feature image; A construction submodule is used to construct structural feature information of the feature image according to the straight line information and the arc structured information.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the steps of the training method of the image generation model as claimed in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for training an image generation model as described in any one of claims 1 to 6.

Images