Training method of deep learning model, image generation method and device

Abstract

The present disclosure provides a deep learning model training method, an image generation method and device, and relates to the technical field of artificial intelligence, in particular to the technical field of computer vision, deep learning and the like. The specific implementation scheme of the deep learning model training method is as follows: target point detection is performed on a first image and a second image respectively to obtain a first target point distribution map and a second target point distribution map, wherein the target point represents a pixel point used to identify the pose and contour of a target object of the image, the first image is generated when an initial model is trained for the nth time, the second image is generated when the initial model is trained for the n+1th time, and n is an integer greater than 1; the first target point distribution map and the second target point distribution map are processed to obtain a difference degree of the first image and the second image; and in response to determining that the difference degree is less than or equal to a first predetermined threshold, a trained deep learning model is obtained.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the fields of computer vision and deep learning. Specifically, it relates to a method for training a deep learning model, an image generation method, and an apparatus. Background Technology

[0002] As artificial intelligence technology is increasingly applied to image processing, the complexity of sample data processed by deep learning models is also increasing. Correspondingly, to improve the data fitting ability of the training set's sample data during iterative training, the complexity of deep learning models is also increasing, to the point that they learn non-general features from the sample data into the model, leading to overfitting. Summary of the Invention

[0003] This disclosure provides a method for training a deep learning model, an image generation method, and an apparatus.

[0004] According to one aspect of this disclosure, a method for training a deep learning model is provided, comprising: performing target point detection on a first image and a second image respectively to obtain a first target point distribution map and a second target point distribution map, wherein target points represent pixels used to identify the pose and contour of target objects in the image, the first image is generated when the initial model is trained for the nth round, and the second image is generated when the initial model is trained for the (n+1)th round, where n is an integer greater than 1; processing the first target point distribution map and the second target point distribution map to obtain a difference degree between the first image and the second image; and obtaining a trained deep learning model in response to determining that the difference degree is less than or equal to a first predetermined threshold.

[0005] According to another aspect of this disclosure, an image generation method is provided, comprising: acquiring data to be processed, wherein the data to be processed includes at least one of the following: text to be processed, image to be processed; and processing the data to be processed using a deep learning model to generate a target image, wherein the deep learning model is trained using the training method described above.

[0006] According to another aspect of this disclosure, a training apparatus for a deep learning model is provided, comprising: a detection module, a processing module, and a training module. The detection module is used to perform target point detection on a first image and a second image respectively, obtaining a first target point distribution map and a second target point distribution map, wherein target points represent pixels used to identify the pose and contour of target objects in the image, the first image is generated during the nth round of training of an initial model, and the second image is generated during the (n+1)th round of training of the initial model, where n is an integer greater than 1. The processing module is used to process the first target point distribution map and the second target point distribution map to obtain a difference degree between the first image and the second image. The training module is used to obtain a trained deep learning model in response to determining that the difference degree is less than or equal to a first predetermined threshold.

[0007] According to another aspect of this disclosure, an image generation apparatus is provided, comprising: an acquisition module and a generation module. The acquisition module is configured to acquire data to be processed, wherein the data to be processed includes at least one of: text to be processed, and an image to be processed. The generation module is configured to process the data to be processed using a deep learning model to generate a target image, wherein the deep learning model is trained using the training method described above.

[0008] According to another aspect of this disclosure, a chip is provided, including a training device for the deep learning model or an image generation device described above.

[0009] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0010] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the methods described above.

[0011] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described above.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0014] Figure 1 The illustration schematically depicts an exemplary system architecture for training methods, image generation methods, and apparatuses for applying deep learning models according to embodiments of the present disclosure;

[0015] Figure 2 A flowchart illustrating a method for training a deep learning model according to an embodiment of the present disclosure is shown schematically.

[0016] Figure 3 This illustration schematically shows a diagram of processing a first image and a second image to obtain a difference in dispersion according to an embodiment of the present disclosure;

[0017] Figure 4 This illustration schematically shows a diagram of processing a first image and a second image to obtain feature differences according to an embodiment of the present disclosure;

[0018] Figure 5 A flowchart illustrating an image generation method according to an embodiment of the present disclosure is shown schematically;

[0019] Figure 6 A block diagram illustrating a training apparatus for a deep learning model according to an embodiment of the present disclosure is shown schematically.

[0020] Figure 7 A block diagram of an image generation apparatus according to an embodiment of the present disclosure is schematically shown; and

[0021] Figure 8 A block diagram of an electronic device suitable for implementing a training method for a deep learning model or an image generation method according to embodiments of the present disclosure is shown schematically. Detailed Implementation

[0022] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0023] Overfitting is a common problem in the training of deep learning models. Related techniques typically employ regularization methods, heuristic cost functions, data augmentation, and label smoothing to suppress overfitting.

[0024] However, in image processing scenarios, the deep learning models involved, such as object detection models and image-text models, have highly complex sample data, and the aforementioned methods for suppressing model overfitting are less effective in these models.

[0025] In view of this, embodiments of this disclosure provide a training method for a deep learning model: First, target point detection is performed on images generated by the deep learning model in two adjacent training rounds to obtain a first target point distribution map and a second target point distribution map. Then, the first and second target point distribution maps are processed to obtain the difference between the first and second images, and the training endpoint of the deep learning model is determined based on this difference. Since the target points of the images generated in two adjacent training rounds represent the key pixel features of the pose and contour of the target object in the image, the difference between the target point distribution maps can reflect the stability of the target object in the images generated in two adjacent training rounds. When the difference is less than or equal to a first predetermined threshold, it indicates that the deep learning model can generate images with relatively stable key pixel features of the target object, which meets the training requirements and effectively prevents overfitting, thereby improving the stability and training efficiency of the deep learning model.

[0026] Figure 1 The illustration schematically depicts an exemplary system architecture for training methods, image generation methods, and apparatuses for applying deep learning models according to embodiments of the present disclosure.

[0027] It is important to note that Figure 1 The examples shown are merely examples of system architectures applicable to embodiments of this disclosure, intended to help those skilled in the art understand the technical content of this disclosure. They do not imply that embodiments of this disclosure cannot be used in other devices, systems, environments, or scenarios. For instance, in another embodiment, an exemplary system architecture for applying deep learning model training methods, image generation methods, and apparatus may include a terminal device. However, the terminal device can implement the deep learning model training methods, image generation methods, and apparatus provided in the embodiments of this disclosure without interacting with a server.

[0028] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0029] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0030] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0031] Server 105 can be a server that provides various services, such as a backend management server that supports the content browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0032] It should be noted that the deep learning model training method and image generation method provided in the embodiments of this disclosure can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the deep learning model training method and image generation device provided in the embodiments of this disclosure can also be disposed in the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0033] Alternatively, the deep learning model training method and image generation method provided in this embodiment can generally be executed by server 105. Correspondingly, the deep learning model training method and image generation apparatus provided in this embodiment can generally be located in server 105. The deep learning model training method and image generation method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the deep learning model training method and image generation apparatus provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0034] For example, during deep learning model training, the first terminal device 101, the second terminal device 102, and the third terminal device 103 can acquire a first image and a second image, and then send the acquired first and second images to the server 105. The server 105 performs target point detection on the first and second images to obtain a first target point distribution map and a second target point distribution map. The server 105 processes the first and second target point distribution maps to obtain the difference between the first and second images. In response to determining that the difference is less than or equal to a first predetermined threshold, the server 105 sends a termination command for deep learning model training to the first terminal device 101, the second terminal device 102, and the third terminal device 103, thus obtaining the trained deep learning model. Alternatively, a server or server cluster capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105 can process the first and second images and ultimately obtain the difference between the first and second images, determining the training endpoint of the deep learning model.

[0035] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0036] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0037] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0038] Figure 2 The flowcharts illustrating the training method of the deep learning model and the image generation method according to embodiments of the present disclosure are shown in the illustration.

[0039] like Figure 2 As shown, the method includes operations S210~S230.

[0040] In operation S210, target point detection is performed on the first image and the second image respectively to obtain the first target point distribution map and the second target point distribution map.

[0041] In operation S220, the distribution maps of the first and second target points are processed to obtain the difference between the first and second images.

[0042] In operation S230, in response to determining that the difference is less than or equal to a first predetermined threshold, a trained deep learning model is obtained.

[0043] According to an embodiment of this disclosure, the first image is generated when the initial model is trained for the nth round, and the second image is generated when the initial model is trained for the (n+1)th round, where n is an integer greater than 1.

[0044] According to embodiments of this disclosure, target points can represent pixels used to identify the pose and contour of a target object in an image. Target points can include keypoints and edge points. Keypoints can represent pixels used to identify the pose of a target object in an image. Edge points can represent pixels used to represent the contour of a target object in an image.

[0045] For example, the target object can be a person or an animal, and key points can include pixels representing the current posture of the person or animal, such as the head, body, and limbs. Edge points can include pixels representing the overall outline of the person or animal.

[0046] For example, the target object can be a building, and key points can be the pixels at the corners of the building, such as the three vertices of a triangular roof. Edge points can be the pixels of the building's overall outline.

[0047] According to embodiments of this disclosure, a first target point distribution map can characterize the number of target points distributed in different pixel regions of a first image. A second target point distribution map can characterize the number of target points distributed in different pixel regions of a second image.

[0048] For example, the first image may include 4 pixel regions and 100 target points. Specifically, the first pixel region may include 20 target points, the second pixel region may include 30 target points, the third pixel region may include 35 target points, and the fourth pixel region may include 15 target points.

[0049] According to embodiments of this disclosure, the first target point distribution map can also characterize the distribution characteristics of target points in different pixel regions of the first image. The second target point distribution map can also characterize the distribution characteristics of target points in different pixel regions of the second image.

[0050] For example, the first pixel region may include 20 target points. The color, position coordinates, brightness and other attributes of these 20 target points can all be used as the distribution characteristics of the target points in the first pixel region.

[0051] According to embodiments of this disclosure, the difference between the first image and the second image may include the difference in the number of target points and the difference in their distribution characteristics.

[0052] According to embodiments of this disclosure, the degree of difference in the distribution of target points can be measured using information entropy. Information entropy characterizes the uncertainty of specific information. The lower the probability of a certain piece of information occurring, the higher the uncertainty of that specific information, and the higher the information entropy.

[0053] For example, when the keypoints of a target object are widely distributed across different pixel regions of an image, the image's information entropy is high. When the keypoints of a target object are only distributed within a single pixel region of an image, the image's information entropy is low. Therefore, information entropy can be used to measure the degree of difference in the number of keypoints distributed across a target object.

[0054] According to embodiments of this disclosure, the degree of difference in the distribution characteristics of target points can be measured using cosine similarity.

[0055] According to embodiments of this disclosure, the first image and the second image can be two-dimensional images or higher-dimensional images such as three-dimensional images. For high-dimensional images, such as three-dimensional images, the three-dimensional image can be converted into a two-dimensional image before target point detection, for example, by segmenting the three-dimensional image along the z-axis to obtain a set of two-dimensional images.

[0056] According to embodiments of this disclosure, the first predetermined threshold can be set according to actual application requirements. In response to determining that the difference is less than or equal to the first predetermined threshold, it indicates that the distribution quantity and distribution characteristics of the target points in the image generated by the deep learning model trained in the (n+1)th round are relatively stable, and the deep learning model trained in the (n+1)th round can be used as the trained deep learning model.

[0057] According to embodiments of this disclosure, since the target points in images generated in two adjacent training rounds represent the key pixel features of the pose and contour of the target object in the image, the difference between the target point distribution maps can reflect the stability of the target object in the images generated in two adjacent training rounds. When the difference is less than or equal to a first predetermined threshold, it indicates that the deep learning model can generate images with relatively stable key pixel features of the target object, which satisfies the training requirements and effectively prevents overfitting, thereby improving the stability and training efficiency of the deep learning model.

[0058] According to embodiments of this disclosure, target point detection is performed on the first image and the second image respectively to obtain a first target point distribution map and a second target point distribution map. This may include the following operations: performing key point detection on the first image and the second image respectively to obtain a first key point distribution map and a second key point distribution map; and performing edge point detection on the first image and the second image respectively to obtain a first edge point distribution map and a second edge point distribution map.

[0059] According to the embodiments of this disclosure, the methods for edge point detection and key point detection of images are relatively mature technologies, and will not be described in detail here.

[0060] According to embodiments of this disclosure, the first target point distribution map may include a first key point distribution map and a first edge point distribution map. The second target point distribution map may include a second key point distribution map and a second edge point distribution map.

[0061] According to an embodiment of this disclosure, the above operation S220 may include the following operations: processing the first key point distribution map and the second key point distribution map to obtain the discrete difference of key points, wherein key points represent pixels used to identify the pose of the target object; processing the first edge point distribution map and the second edge point distribution map to obtain the feature difference of edge points, wherein edge points represent pixels used to represent the contour of the target object; and obtaining the difference degree based on the discrete difference of key points and the feature difference of edge points.

[0062] According to embodiments of this disclosure, processing a first keypoint distribution map and a second keypoint distribution map to obtain the dispersion difference of keypoints may include the following operations: processing the first keypoint distribution map to obtain the dispersion degree of the first keypoint; processing the second keypoint distribution map to obtain the dispersion degree of the second keypoint; and obtaining the dispersion degree difference of keypoints based on the dispersion degree of the first keypoint and the dispersion degree of the second keypoint.

[0063] Figure 3 The illustration schematically shows the difference in the degree of dispersion of key points obtained by processing a first image and a second image according to an embodiment of the present disclosure.

[0064] like Figure 3 As shown, in embodiment 300, sample data 311 is input into model 312 trained in the nth round to obtain the first image 314. Sample data 311 is input into model 313 trained in the (n+1)th round to obtain the second image 315.

[0065] According to embodiments of this disclosure, processing a first keypoint distribution map to obtain the dispersion of the first keypoint may include: rasterizing the first keypoint distribution map to obtain multiple first grid regions and the number of keypoints within the multiple first grid regions; and obtaining the dispersion of the first keypoint based on the number of keypoints within the multiple first grid regions.

[0066] For example, after performing key point detection and rasterization on the first image 314, a first key point distribution map 321 is obtained, which is divided into multiple first raster regions. The number of key points in each first raster region is counted to obtain the number of key points 323 in multiple first raster regions.

[0067] like Figure 3 As shown, the first keypoint distribution map 321 may include grid regions A1, A2, A3, and A4. Grid region A1 contains 8 keypoints, grid region A2 contains 6 keypoints, grid region A3 contains 10 keypoints, and grid region A4 contains 8 keypoints.

[0068] According to embodiments of this disclosure, the information entropy H1 of the first key point distribution map, i.e., the dispersion degree 325 of the first key point, can be obtained according to the probability of the number of key points appearing in grid regions A1, A2, A3 and A4.

[0069] According to embodiments of this disclosure, the processing method for the second key point distribution map is the same as the processing method for the first key point distribution map.

[0070] For example, after performing keypoint detection and rasterization on the second image 315, a second keypoint distribution map 322 is obtained, which is divided into multiple second raster regions. The number of keypoints in each second raster region is then counted to obtain the number of keypoints 324 in the multiple second raster regions.

[0071] like Figure 3 As shown, the second keypoint distribution map 322 may include grid regions B1, B2, B3, and B4. Grid region B1 contains 6 keypoints, grid region B2 contains 12 keypoints, grid region B3 contains 4 keypoints, and grid region B4 contains 10 keypoints.

[0072] According to embodiments of this disclosure, the information entropy H2 of the second key point distribution map, i.e., the dispersion degree 326 of the second key point, can be obtained according to the probability of the number of key points appearing in grid regions B1, B2, B3 and B4.

[0073] According to an embodiment of this disclosure, the difference in the degree of dispersion of the key points 327 can be obtained based on the degree of dispersion of the first key point 325 and the degree of dispersion of the second key point 326.

[0074] According to embodiments of this disclosure, the dispersion 325 of the first key point can represent the dispersion of the key point distribution of the target object in the first image. The greater the dispersion, the greater the information entropy H1 of the first key point distribution map. Similarly, the dispersion 326 of the second key point can represent the dispersion of the key point distribution of the target object in the second image. The greater the dispersion, the greater the information entropy H2 of the second key point distribution map.

[0075] According to embodiments of this disclosure, the difference in the discreteness of key points 327 can characterize the information gain between the first image and the second image. Information gain can characterize the stability of images generated by the model in two adjacent training rounds.

[0076] For example, when the information gain is greater than 0, it indicates that the image generated by the deep learning model is less stable and training needs to continue. When the information gain is equal to 0, it indicates that the image generated by the deep learning model is more stable and training can be terminated.

[0077] According to embodiments of this disclosure, the information entropy of an image is obtained based on the distribution quantity of key points in different grid regions of the image. Since the difference in information entropy can characterize the stability of the image generated by the model, it can effectively improve the stability of the deep learning model and reduce the probability of overfitting.

[0078] To improve the accuracy of deep learning models, it is also necessary to consider the distribution characteristics of the target object's contour.

[0079] Therefore, processing the first edge point distribution map and the second edge point distribution map to obtain the feature differences of the edge points can include the following operations: processing the first edge point distribution map and the second edge point distribution map respectively to obtain the distribution features of the first edge points and the distribution features of the second edge points; when it is determined that the similarity between the distribution features of the first edge points and the distribution features of the second edge points is less than or equal to a second predetermined threshold, clustering the first edge points and the second edge points respectively to obtain the first edge entity objects and the second edge entity objects; and processing the first edge entity objects and the second edge entity objects to obtain the feature differences.

[0080] Figure 4 The illustration schematically shows a diagram of the characteristic differences of edge points obtained by processing a first image and a second image according to an embodiment of the present disclosure.

[0081] like Figure 4 As shown, in embodiment 400, sample data 311 is input into model 312 trained in the nth round to obtain the first image 314. Sample data 311 is input into model 313 trained in the (n+1)th round to obtain the second image 315.

[0082] According to embodiments of this disclosure, processing the first edge point distribution map to obtain the distribution characteristics of the first edge points may include the following operations: dividing the first edge point distribution map into regions to obtain multiple first edge point regions and the number of edge points in the multiple first edge point regions; fitting the number of edge points in the multiple first edge point regions to obtain the distribution characteristics of the first edge points.

[0083] For example, edge point detection and region segmentation are performed on the first image 314 to obtain a first edge point distribution map 421 that includes multiple first edge point regions.

[0084] Since an image can contain multiple target objects, the contour of each target object can correspond to an edge entity object. When comparing the differences in edge point features, edge entity objects can be obtained by clustering the edge points. If the edge entity objects are determined to be approximate, then a difference analysis is performed on the edge point features of the edge entity objects.

[0085] Therefore, when dividing the image into regions, the image can be divided into multiple rectangular regions by columns.

[0086] like Figure 4 As shown, the first edge point distribution map 421 may include edge point region P1 and edge point region P2. The second edge point distribution map 422 may include edge point region Q1 and edge point region Q2.

[0087] For example, by counting the number of edge points in edge point regions P1 and P2, the distribution probability of the edge points in edge point regions P1 and P2 can be obtained. Then, the distribution probability of the edge points in edge point regions P1 and P2 can be fitted based on a Gaussian distribution to obtain the Gaussian distribution sequence of the first edge point, that is, the distribution characteristic of the first edge point.

[0088] Similarly, by counting the number of edge points in edge regions Q1 and Q2, the distribution probability of the edge points in edge regions Q1 and Q2 can be obtained. Then, the distribution probability of the edge points in edge regions Q1 and Q2 can be fitted with a Gaussian distribution to obtain the Gaussian distribution sequence of the second edge point, that is, the distribution characteristic of the second edge point.

[0089] According to embodiments of this disclosure, if the difference between the Gaussian distribution sequence of the first edge point and the Gaussian distribution sequence of the second edge point is less than or equal to a second predetermined threshold, it indicates that the distribution characteristics of the first edge point and the distribution characteristics of the second edge point are similar. The first edge points can be clustered to obtain a first edge entity object 423. The second edge points can also be clustered to obtain a second edge entity object 424.

[0090] It should be noted that when fitting data based on a Gaussian distribution, the resulting Gaussian distribution sequence can include multiple segments of fitted data. For images with high complexity, such as those containing multiple objects like people, cats, dogs, and birds, each segment of fitted data can correspond to one object or a part of an object. Therefore, when clustering edge points, edge points corresponding to similar fitted data can be clustered separately to obtain multiple edge entity objects. For example, the outlines of a person, a cat, a dog, and a bird.

[0091] According to embodiments of this disclosure, by dividing the edge point distribution map into regions and fitting the edge points, the distribution characteristics of the edge points are obtained, which can be adapted to scenes with high image content complexity, thereby improving the training accuracy of deep learning models.

[0092] According to embodiments of this disclosure, processing a first edge entity object and a second edge entity object to obtain feature differences may include the following operations: extracting first edge point features of the first edge entity object; extracting second edge point features of the second edge entity object; and processing the first edge point features and the second edge point features based on a similarity function to obtain feature differences.

[0093] According to embodiments of this disclosure, the first edge point feature includes the color and position features of the edge points of a first edge entity object. The second edge point feature includes the color and position features of the edge points of a second edge entity object.

[0094] For example, color features can characterize the color attributes of edge points of an edge entity object, such as red, yellow, and blue. Position features can characterize the relative position attributes of edge points of an edge entity object, such as the difference in position coordinates between adjacent edge points.

[0095] According to embodiments of this disclosure, the first edge point feature can be a feature vector composed of the color and position features of the edge points of the first edge entity object. Similarly, the second edge point feature can be a feature vector composed of the color and position features of the edge points of the second edge entity object.

[0096] According to embodiments of this disclosure, the similarity function can be a cosine similarity function, or other functions used to measure feature similarity. A higher similarity indicates a smaller difference in features.

[0097] According to embodiments of this disclosure, feature differences in an image are obtained based on a similarity function, according to the color and positional features of edge points. This can ensure the accuracy of the generated image content during deep learning model training, thereby improving the accuracy of model training.

[0098] It should be noted that, in order to further improve the accuracy of the deep learning model, if the difference between the first image and the second image is less than or equal to a first predetermined threshold, the second image and the label image can be processed according to the method provided in the embodiments of this disclosure to obtain the difference between the second image and the label image. When the difference between the second image and the label image is less than or equal to a third predetermined threshold, the trained deep learning model is obtained.

[0099] According to embodiments of this disclosure, the third predetermined threshold may be the same as or different from the first predetermined threshold, and no specific limitation is made herein.

[0100] The dispersion differences of keypoints can reflect the stability of the model, while the feature differences of edge points can reflect the accuracy of the model. The weights of the dispersion differences of keypoints and the feature differences of edge points can be configured for the model training process according to the needs of the actual application scenario.

[0101] For example, in a text-image model, where semantic features of text are analyzed to generate a target image, the content of the target image is relatively simple. For instance, the text content might be "There's a dog in front of a house," and the generated target image only needs to include the house and the dog; their poses are unimportant. Therefore, when configuring the first weight for dispersion differences and the second weight for feature differences, the first weight can be lower, and the second weight can be higher.

[0102] For example, in object detection models, where the target image is identified or recognized, the content of the target image is relatively complex. Therefore, when configuring the first weight for dispersion difference and the second weight for feature difference, the first weight and the second weight can each account for 50% of the weight.

[0103] According to embodiments of this disclosure, the degree of difference is obtained based on the dispersion difference, the first weight, the feature difference, and the second weight.

[0104] For example, the difference in dispersion between the first image and the second image can be ΔH, the first weight can be w1, the difference in features between the first image and the second image can be ΔS, the second weight can be w2, and the degree of difference can be expressed as ΔH·w1 + ΔS·w2.

[0105] According to embodiments of this disclosure, by assigning weights to discreteness differences and feature differences, the difference between the first image and the second image can be obtained. This allows for the training of deep learning models according to different application scenarios, thereby improving the adaptability and training efficiency of deep learning models in different application scenarios.

[0106] Figure 5 A flowchart illustrating an image production method according to an embodiment of the present disclosure is shown schematically.

[0107] like Figure 5 As shown, the method 500 may include operations S510 to S520.

[0108] The S510 is used to obtain data to be processed.

[0109] When operating the S520, a deep learning model is used to process the data to be processed and generate the target image.

[0110] According to embodiments of this disclosure, the data to be processed includes at least one of the following: text to be processed, and image to be processed.

[0111] According to embodiments of this disclosure, a deep learning model can be a model that generates images from text or a model that generates images from images.

[0112] According to embodiments of this disclosure, the model generates images with high accuracy because overfitting is effectively reduced during model training.

[0113] Figure 6 A block diagram of a training apparatus for a deep learning model according to an embodiment of the present disclosure is shown schematically.

[0114] like Figure 6 As shown, the training device 600 may include a detection module 610, a processing module 620, and a training module 630.

[0115] The detection module 610 is used to perform target point detection on the first image and the second image respectively to obtain a first target point distribution map and a second target point distribution map. The target point represents the pixel points used to identify the pose and contour of the target object in the image. The first image is generated when the initial model is trained for the nth round, and the second image is generated when the initial model is trained for the (n+1)th round, where n is an integer greater than 1.

[0116] Processing module 620 is used to process the first target point distribution map and the second target point distribution map to obtain the difference between the first image and the second image; and

[0117] Training module 630 is used to obtain a trained deep learning model in response to determining that the difference is less than or equal to a first predetermined threshold.

[0118] According to embodiments of this disclosure, the first target point distribution map includes a first key point distribution map and a first edge point distribution map; the second target point distribution map includes a second key point distribution map and a second edge point distribution map. The processing module 610 may include a first processing submodule, a second processing submodule, and a difference degree acquisition submodule.

[0119] The first processing submodule is used to process the first key point distribution map and the second key point distribution map to obtain the dispersion difference of key points, wherein the key points represent the pixels used to identify the pose of the target object.

[0120] The second processing submodule is used to process the first edge point distribution map and the second edge point distribution map to obtain the feature differences of the edge points, wherein the edge points represent the pixels used to represent the contour of the target object.

[0121] The difference degree acquisition submodule is used to obtain the difference degree based on the dispersion difference of key points and the feature difference of edge points.

[0122] According to embodiments of this disclosure, the second processing submodule may include: a first processing unit, a clustering unit, and a second processing unit.

[0123] The first processing unit is used to process the first edge point distribution map and the second edge point distribution map respectively to obtain the distribution characteristics of the first edge point and the distribution characteristics of the second edge point.

[0124] The clustering unit is used to cluster the first edge point and the second edge point respectively, when the similarity between the distribution characteristics of the first edge point and the distribution characteristics of the second edge point is less than or equal to a second predetermined threshold, to obtain the first edge entity object and the second edge entity object.

[0125] The second processing unit is used to process the first edge entity object and the second edge entity object to obtain feature differences.

[0126] According to embodiments of this disclosure, the first processing unit may include: a first region division subunit, a first fitting subunit, a second region division subunit, and a second fitting subunit.

[0127] The first region division subunit is used to divide the first edge point distribution map into regions, resulting in multiple first edge point regions and the number of edge points within the multiple first edge point regions.

[0128] The first fitting subunit is used to fit the number of edge points within multiple first edge point regions to obtain the distribution characteristics of the first edge points.

[0129] The second region division subunit is used to divide the second edge point distribution map into regions, resulting in multiple second edge point regions and the number of edge points within each region.

[0130] The second fitting subunit is used to fit the number of edge points within multiple second edge point regions to obtain the distribution characteristics of the second edge points.

[0131] According to embodiments of this disclosure, the second processing unit may include: a first extraction subunit, a second extraction subunit, and a similarity calculation subunit.

[0132] The first extraction subunit is used to extract the first edge point features of the first edge entity object, wherein the first edge point features include the color features and position features of the edge points of the first edge entity object.

[0133] The second extraction subunit is used to extract the second edge point features of the second edge entity object, wherein the second edge point features include the color features and position features of the edge points of the second edge entity object.

[0134] The similarity calculation subunit is used to process the features of the first edge point and the second edge point based on the similarity function to obtain the feature differences.

[0135] According to embodiments of this disclosure, the first processing submodule may include: a third processing unit, a fourth processing unit, and a discreteness difference acquisition unit.

[0136] The third processing unit is used to process the distribution map of the first key points to obtain the dispersion of the first key points.

[0137] The fourth processing unit is used to process the distribution map of the second key points to obtain the dispersion of the second key points.

[0138] The discreteness difference acquisition unit is used to obtain the discreteness difference of key points based on the discreteness of the first key point and the discreteness of the second key point.

[0139] According to embodiments of this disclosure, the third processing unit may include: a first rasterization unit and a first discreteness acquisition unit.

[0140] The first rasterization unit is used to rasterize the first keypoint distribution map to obtain multiple first raster regions and the number of keypoints within the multiple first raster regions.

[0141] The first discreteness obtaining unit is used to obtain the discreteness of the first key point based on the number of key points in multiple first grid regions.

[0142] According to embodiments of this disclosure, the fourth processing unit may include: a second rasterization unit and a second discreteness acquisition unit.

[0143] The second rasterization unit is used to rasterize the second keypoint distribution map to obtain multiple second raster regions and the number of keypoints within the multiple second raster regions.

[0144] The second discreteness acquisition unit is used to obtain the discreteness of the second key point based on the number of key points in multiple second grid regions.

[0145] According to embodiments of this disclosure, the difference degree acquisition submodule may include a determination unit and an acquisition unit.

[0146] A determination unit is used to determine the first weight of the dispersion difference and the second weight of the feature difference.

[0147] The unit is used to obtain the degree of difference based on the dispersion difference, the first weight, the feature difference, and the second weight.

[0148] According to embodiments of this disclosure, the detection module may include a key point detection submodule and an edge point detection submodule.

[0149] The key point detection submodule is used to perform key point detection on the first image and the second image respectively, and obtain the first key point distribution map and the second key point distribution map.

[0150] The edge point detection submodule is used to perform edge point detection on the first image and the second image respectively, and obtain the first edge point distribution map and the second edge point distribution map.

[0151] Figure 7 A block diagram of an image generation apparatus according to an embodiment of the present disclosure is shown schematically.

[0152] like Figure 7 As shown, the image generation apparatus 700 may include an acquisition module 710 and a generation module 720.

[0153] Acquisition module 710 is used to acquire data to be processed, wherein the data to be processed includes at least one of the following: text to be processed, image to be processed; and

[0154] The generation module 720 is used to process the data to be processed using a deep learning model to generate the target image. The deep learning model is trained using the deep learning model training method described above.

[0155] According to embodiments of this disclosure, this disclosure also provides a chip, an electronic device, a readable storage medium, and a computer program product.

[0156] According to another aspect of this disclosure, a chip is provided, including a training device for the deep learning model or an image generation device described above.

[0157] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0158] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described above.

[0159] According to an embodiment of this disclosure, a computer program product includes a computer program that, when executed by a processor, implements the method described above.

[0160] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0161] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0162] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0163] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as deep learning model training methods or image generation methods. For example, in some embodiments, the deep learning model training method or image generation method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the deep learning model training method or image generation method described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a deep learning model training method or an image generation method.

[0164] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0165] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0166] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0167] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0168] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0169] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0170] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0171] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training a deep learning model, comprising: performing target point detection on a first image and a second image respectively to obtain a first target point distribution map and a second target point distribution map, wherein the target points represent pixel points used to identify the pose and contour of a target object in an image, the first image is generated when an initial model is trained for an n th round, the second image is generated when the initial model is trained for an n+1 th round, n is an integer greater than 1;the first target point distribution map comprises a first key point distribution map and a first edge point distribution map;the second target point distribution map comprises a second key point distribution map and a second edge point distribution map;the key points represent pixel points used to identify the pose of the target object;the edge points represent pixel points used to represent the contour of the target object; processing the first target point distribution map and the second target point distribution map to obtain a difference degree of the first image and the second image;the difference degree comprises a key point dispersion difference and an edge point feature difference;and in response to determining that the difference degree is less than or equal to a first predetermined threshold, obtaining a trained deep learning model.

2. The method of claim 1, wherein, the processing of the first target point distribution map and the second target point distribution map to obtain a difference degree of the first image and the second image comprises: processing the first key point distribution map and the second key point distribution map to obtain a key point dispersion difference; processing the first edge point distribution map and the second edge point distribution map to obtain an edge point feature difference;and obtaining the difference degree according to the key point dispersion difference and the edge point feature difference.

3. The method of claim 2, wherein, the processing of the first edge point distribution map and the second edge point distribution map to obtain an edge point feature difference comprises: processing the first edge point distribution map and the second edge point distribution map respectively to obtain a first edge point distribution feature and a second edge point distribution feature; in a case where it is determined that the similarity of the first edge point distribution feature and the second edge point distribution feature is less than or equal to a second predetermined threshold, clustering the first edge points and the second edge points respectively to obtain a first edge entity object and a second edge entity object;and processing the first edge entity object and the second edge entity object to obtain a feature difference.

4. The method of claim 3, wherein, the processing of the first edge point distribution map and the second edge point distribution map respectively to obtain a first edge point distribution feature and a second edge point distribution feature comprises: regionally dividing the first edge point distribution map to obtain a plurality of first edge point regions and the number of edge points in the plurality of first edge point regions; fitting the number of edge points in the plurality of first edge point regions to obtain the first edge point distribution feature; regionally dividing the second edge point distribution map to obtain a plurality of second edge point regions and the number of edge points in the plurality of second edge point regions;and fitting the number of edge points in the plurality of second edge point regions to obtain the second edge point distribution feature.

5. The method of claim 3, wherein, The processing of the first edge entity object and the second edge entity object comprises: extracting first edge point features of the first edge entity object, wherein the first edge point features comprise color features and position features of edge points of the first edge entity object; extracting second edge point features of the second edge entity object, wherein the second edge point features comprise color features and position features of edge points of the second edge entity object; and processing the first edge point features and the second edge point features based on a similarity function to obtain the feature difference.

6. The method of claim 2, wherein, The processing of the first key point distribution map and the second key point distribution map comprises: processing the first key point distribution map to obtain a dispersion degree of first key points; processing the second key point distribution map to obtain a dispersion degree of second key points; and obtaining the dispersion degree difference of the key points according to the dispersion degree of the first key points and the dispersion degree of the second key points.

7. The method of claim 6, wherein, The processing of the first key point distribution map to obtain a dispersion degree of first key points comprises: performing rasterization processing on the first key point distribution map to obtain a plurality of first grid regions and a number of key points in the plurality of first grid regions; and obtaining the dispersion degree of the first key points according to the number of key points in the plurality of first grid regions.

8. The method of claim 6, wherein, The processing of the second key point distribution map to obtain a dispersion degree of second key points comprises: performing rasterization processing on the second key point distribution map to obtain a plurality of second grid regions and a number of key points in the plurality of second grid regions; and obtaining the dispersion degree of the second key points according to the number of key points in the plurality of second grid regions.

9. The method of claim 2, wherein, The obtaining of the difference degree according to the dispersion degree difference of the key points and the feature difference of the edge points comprises: determining a first weight of the dispersion degree difference and a second weight of the feature difference; and obtaining the difference degree according to the dispersion degree difference, the first weight, the feature difference and the second weight.

10. The method of claim 1, wherein, The target point detection of the first image and the second image to obtain a first target point distribution map and a second target point distribution map comprises: performing key point detection on the first image and the second image to obtain a first key point distribution map and a second key point distribution map; and performing edge point detection on the first image and the second image to obtain a first edge point distribution map and a second edge point distribution map.

11. An image generation method, comprising: obtaining to-be-processed data, wherein the to-be-processed data comprises at least one of to-be-processed text and to-be-processed image; and processing the to-be-processed data by using a deep learning model to generate a target image, wherein the deep learning model is trained by using the training method in any one of claims 1-10.

12. A training device of a deep learning model, comprising: The detection module is configured to perform target point detection on the first image and the second image respectively to obtain a first target point distribution map and a second target point distribution map, wherein the target point represents a pixel point for identifying a pose and an outline of a target object in the image, the first image is generated when the initial model is trained for the nth time, the second image is generated when the initial model is trained for the (n+1)th time, n is an integer greater than 1, the first target point distribution map includes a first key point distribution map and a first edge point distribution map, the second target point distribution map includes a second key point distribution map and a second edge point distribution map, the key point represents a pixel point for identifying the pose of the target object, and the edge point represents a pixel point for representing the outline of the target object. The processing module is configured to process the first target point distribution map and the second target point distribution map to obtain a difference degree of the first image and the second image, wherein the difference degree includes a key point dispersion difference and an edge point feature difference. The training module is configured to obtain a trained deep learning model in response to determining that the difference degree is less than or equal to a first predetermined threshold.

13. The apparatus of claim 12, wherein, The processing module includes: A first processing submodule is configured to process the first key point distribution map and the second key point distribution map to obtain a key point dispersion difference. A second processing submodule is configured to process the first edge point distribution map and the second edge point distribution map to obtain an edge point feature difference. A difference degree obtaining submodule is configured to obtain the difference degree according to the key point dispersion difference and the edge point feature difference.

14. The apparatus of claim 13, wherein, The second processing submodule includes: A first processing unit is configured to process the first edge point distribution map and the second edge point distribution map respectively to obtain a first edge point distribution feature and a second edge point distribution feature. A clustering unit is configured to cluster the first edge point and the second edge point respectively to obtain a first edge entity object and a second edge entity object in a case where a similarity between the first edge point distribution feature and the second edge point distribution feature is less than or equal to a second predetermined threshold. A second processing unit is configured to process the first edge entity object and the second edge entity object to obtain a feature difference.

15. The apparatus of claim 14, wherein, The first processing unit includes: A first region division submodule is configured to divide the first edge point distribution map into a plurality of first edge point regions and a number of edge points in the plurality of first edge point regions. A first fitting submodule is configured to fit the number of edge points in the plurality of first edge point regions to obtain the first edge point distribution feature. A second region division submodule is configured to divide the second edge point distribution map into a plurality of second edge point regions and a number of edge points in the plurality of second edge point regions. A second fitting submodule is configured to fit the number of edge points in the plurality of second edge point regions to obtain the second edge point distribution feature.

16. The apparatus of claim 14, wherein, The second processing unit includes: a first extraction subunit configured to extract first edge point features of the first edge entity object, wherein the first edge point features comprise color features and position features of edge points of the first edge entity object; a second extraction subunit configured to extract second edge point features of the second edge entity object, wherein the second edge point features comprise color features and position features of edge points of the second edge entity object; and a similarity calculation subunit configured to process the first edge point features and the second edge point features based on a similarity function to obtain the feature difference.

17. The apparatus of claim 13, wherein, The first processing sub-module comprises: a third processing unit configured to process the first key point distribution map to obtain a dispersion degree of the first key points; a fourth processing unit configured to process the second key point distribution map to obtain a dispersion degree of the second key points; and a dispersion degree difference obtaining unit configured to obtain a dispersion degree difference of the key points according to the dispersion degree of the first key points and the dispersion degree of the second key points.

18. The apparatus of claim 17, wherein, The third processing unit comprises: a first gridding unit configured to perform gridding processing on the first key point distribution map to obtain a plurality of first grid regions and a number of key points in the plurality of first grid regions; and a first dispersion degree obtaining unit configured to obtain the dispersion degree of the first key points according to the number of key points in the plurality of first grid regions.

19. The apparatus of claim 17, wherein, The fourth processing unit comprises: a second gridding unit configured to perform gridding processing on the second key point distribution map to obtain a plurality of second grid regions and a number of key points in the plurality of second grid regions; and a second dispersion degree obtaining unit configured to obtain the dispersion degree of the second key points according to the number of key points in the plurality of second grid regions.

20. The apparatus of claim 13, wherein, The difference degree obtaining sub-module comprises: a determination unit configured to determine a first weight of the dispersion degree difference and a second weight of the feature difference; and an obtaining unit configured to obtain the difference degree according to the dispersion degree difference, the first weight, the feature difference and the second weight.

21. The apparatus of claim 12, wherein, The detection module comprises: a key point detection sub-module configured to perform key point detection on the first image and the second image respectively to obtain a first key point distribution map and a second key point distribution map; and an edge point detection sub-module configured to perform edge point detection on the first image and the second image respectively to obtain a first edge point distribution map and a second edge point distribution map. 22.An image generation apparatus, comprising: an acquisition module configured to acquire to-be-processed data, wherein the to-be-processed data comprises at least one of the following: to-be-processed text, to-be-processed image; and a generation module configured to process the to-be-processed data by using a deep learning model to generate a target image, wherein the deep learning model is trained by using the training method in any one of claims 1-10. 23.A chip comprising the apparatus in any one of claims 12-22. 24.An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

25. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are for causing the computer to perform the method of any one of claims 1-11.

26. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-11.

Images