Image generation method and device

Abstract

The embodiments of this specification provide a method and device for generating an image, wherein the method includes: obtaining an image to be processed based on a target image generation request; determining label information of the image to be processed, and inputting the image to be processed and the label information corresponding to the image to be processed into an image generation model to obtain an initial image corresponding to the image to be processed; splicing the initial image and the image to be processed according to the preset splicing conditions to obtain a spliced ​​image; judging whether the spliced ​​image meets the preset requirement conditions, and if so, using the spliced ​​image as the target image, and if not, using the spliced ​​image as the image to be processed, and continuing to execute the step of determining the label information of the image to be processed, so that the image to be processed is continuously expanded and extended to improve the display effect.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and more particularly to a method for generating an image. One or more embodiments of this specification also relate to an apparatus for generating an image, a computing device, and a computer-readable storage medium. Background Art

[0002] With the continuous development of terminal technology, the popularity of smart terminals in social life is getting higher and higher, and users' requirements for browsing pictures on mobile terminals are also constantly increasing. When users want to browse pictures with a wider range after extension, they are all generated through a single picture, which cannot meet the user's need for a larger range or pictures with more picture features to meet the user's picture needs for specific scenes and improve the picture display effect. Summary of the Invention

[0003] In view of this, the present invention provides an image generation method. One or more embodiments of the present invention also involve an image generation apparatus, a computing device, and a computer-readable storage medium to address the technical shortcomings of the prior art in being unable to meet user needs for generating a wider range of images.

[0004] According to a first aspect of an embodiment of this specification, a method for generating an image is provided, comprising:

[0005] Generate a request based on the target image to obtain the image to be processed;

[0006] Determining label information of the image to be processed, and inputting the image to be processed and the label information corresponding to the image to be processed into an image generation model to obtain an initial image corresponding to the image to be processed;

[0007] Splicing the initial image and the image to be processed according to the preset splicing condition to obtain a spliced ​​image;

[0008] Determine whether the spliced ​​image meets the preset requirements. If so, use the spliced ​​image as the target image. If not, use the spliced ​​image as the image to be processed, continue to determine the label information of the image to be processed, and input the image to be processed and the label information corresponding to the image to be processed into the image generation model to obtain the initial image corresponding to the image to be processed.

[0009] According to a second aspect of the embodiments of this specification, there is provided a picture generating apparatus, including:

[0010] An acquisition module is configured to generate a request to acquire a to-be-processed image based on a target image;

[0011] an acquisition module configured to determine the label information of the image to be processed, and input the image to be processed and the label information corresponding to the image to be processed into an image generation model to obtain an initial image corresponding to the image to be processed;

[0012] a splicing module configured to splice the initial image and the image to be processed according to the preset splicing condition to obtain a spliced ​​image;

[0013] The judging module is configured to judge whether the spliced ​​picture meets the preset requirement. If so, the spliced ​​picture is used as the target picture. If not, the spliced ​​picture is used as the picture to be processed and the obtaining module is continued to be executed.

[0014] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0015] memory and processor;

[0016] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein the processor implements the steps of the image generation method when executing the computer-executable instructions.

[0017] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the image generation method are implemented.

[0018] An embodiment of the present specification implements a method and device for generating an image, wherein the image generation method includes obtaining an image to be processed based on a target image generation request, and determining label information of the image to be processed; inputting the image to be processed and the label information corresponding to the image to be processed into an image generation model to obtain an initial image corresponding to the image to be processed; splicing the initial image and the image to be processed according to preset splicing conditions to obtain the spliced ​​image, and judging whether the spliced ​​image meets preset requirements, thereby obtaining a target image; the image generation method obtains an initial image by inputting the obtained image to be processed and the label information into an image generation model, and splicing the initial image with the image to be processed to generate a spliced ​​image, and then continuously splicing to obtain the target image, so that the images to be processed can be connected through splicing to form a large-scale image including more image features, which not only improves the display effect of the image, but also meets the user's display needs for images in specific scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a system architecture diagram of a method for generating an image provided by an embodiment of this specification;

[0020] Figure 2 This is a flowchart of a method for generating an image provided by one embodiment of this specification;

[0021] Figure 3 This is a structural diagram of an image generation model in an image generation method provided in one embodiment of this specification;

[0022] Figure 4 This is a flowchart of a processing process of a method for generating an image provided by an embodiment of this specification;

[0023] Figure 5 This is a structural diagram of a picture generating device provided by an embodiment of this specification;

[0024] Figure 6 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0025] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0026] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0027] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0028] First, the terms involved in one or more embodiments of this specification are explained.

[0029] CGAN (Conditional Generative Adversarial Net, Conditional Generative Adversarial Network): can focus on one of the two main objectives of the conditional adversarial network through parameterized loss function, namely real samples or performance differences.

[0030] Generative Model (English full name: Generative Model): It is an important type of model in probability statistics and machine learning. It refers to a series of models used to randomly generate observable data. It can generate text, images, videos and other data from input data according to tasks and through model training.

[0031] The discriminative model (full name in English: Discriminative Model) is a method for modeling the relationship between unobserved data y and observed data x. It directly models the conditional probability and cannot reflect the characteristics of the training data itself, but in terms of the optimal classification between different categories, it reflects the differences between heterogeneous data.

[0032] In this specification, a method for generating an image is provided. This specification also relates to an image generating apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0033] The image generation method provided in the embodiments of this specification can be applied to any field that requires image generation, such as splicing images obtained from web pages and videos, generating large-scale images, etc. For ease of understanding, the embodiments of this specification use the application of the image generation method to the example of processing images obtained in the video field to generate target images for detailed introduction, but are not limited to this.

[0034] Then, in the case where the image generation method is applied to process images in the video field to generate target images, the target image can be understood as automatically extending the image to be processed obtained in the video frame to obtain a large range of images that meet user needs and are consistent with the obtained image to be processed.

[0035] In addition, the image generation method of the embodiments of this specification can be applied to any image display location that can present a wide range of extended images, such as web page images, advertising display images, videos and other self-media display images, etc.

[0036] See also Figure 1 , Figure 1 A system architecture diagram of a method for generating an image according to an embodiment of the present disclosure is shown.

[0037] Figure 1In the embodiment, user A watches video A through client A and obtains the image A to be processed required by the user in video A. Client A transmits the image A to be processed to the server corresponding to video A. The server receives the image A to be processed, determines the label information corresponding to the image A to be processed, and inputs the received image A to be processed and the corresponding label information into the image generation model to obtain an initial image A1, wherein the image type and content features of the initial image A1 are the same or similar to those of the image A to be processed. For example, if the image A to be processed is a starry sky image, then the initial image A1 generated by the image generation model is similar to the image type of the image A to be processed. For example, if the image A to be processed has the Big Dipper, then the initial image generated by the image generation model can be a starry sky image with Vega, Polaris, etc., and then the initial image A1 and the image A to be processed are spliced ​​according to the preset splicing conditions. When the user's needs are met, the target image is obtained, wherein the target image is a large-scale starry sky image with the Big Dipper, Vega, Polaris, etc., thereby presenting the user with a large-scale image of the same type as the image A to be processed, thereby improving the display effect of the image.

[0038] See also Figure 2 , Figure 2 A flowchart of a method for generating an image according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0039] Step 202: Generate a request based on the target image to obtain the image to be processed.

[0040] Among them, the target image generation request can be understood as a user's request to obtain a large-scale image; the image to be processed can be understood as the original image for which the user needs to expand the image range. The image to be processed can be obtained from a web page or from a video frame, and there is no limitation on the original path for obtaining the image.

[0041] Taking the application of the image generation method to enlarge the video frame image obtained in the video to generate a target image as an example, the server obtains the image to be processed based on the user's target image generation request, wherein the obtained image to be processed may include but is not limited to starry sky images, map images, constellation images, etc.

[0042] In specific implementation, the server obtains the image to be processed based on the target image generation request triggered by the user. For example, if the image to be processed obtained by the server is a starry sky image, the corresponding label information is determined to be the size of the planet in the starry sky, the color of the planet, and whether the planet has a halo belt and other information.

[0043] Step 204: Determine the label information of the image to be processed, and input the image to be processed and the label information corresponding to the image to be processed into an image generation model to obtain an initial image corresponding to the image to be processed.

[0044] Among them, the label information of the image to be processed can be understood as the information of the features in the image to be processed. For example, if the image to be processed is a starry sky image, the label information corresponding to the starry sky image can be the size of the planet, the color of the planet, whether the planet has a halo, and other information; if the image to be processed is a map image, the label information corresponding to the map image can be whether there are mountains, whether there are rivers, the density of the forest, and other information.

[0045] Furthermore, in order to determine the label information of the image to be processed, the image to be processed is input into a label recognition model. Specifically, determining the label information of the image to be processed includes:

[0046] The image to be processed is input into a label recognition model to obtain label information of the image to be processed, wherein the label recognition model outputs label information corresponding to the image to be processed.

[0047] Among them, the label recognition model can be understood as a model that can identify the feature information contained in an image by inputting an image and output it as label information. In the embodiments of this specification, there is no limitation on the type of label recognition model, and any model that can output the label information in the image is sufficient.

[0048] For example, by inputting a starry sky picture as the image to be processed into the label recognition model, the label information such as the number of planets in the starry sky picture, the size of the planets, the color of the starry sky, and whether there is halo information in the starry sky can be output.

[0049] In the embodiment of this specification, the label information in the image to be processed can be quickly determined by using the label recognition model, so that the label information corresponding to the image to be processed can be subsequently input into the image generation model to obtain the initial image.

[0050] Since the display scope of the image to be processed is limited and cannot meet the user's demand for knowledge extension in the image, and the generation of multiple single images is not continuous, it cannot improve the display effect. Therefore, the image to be processed and the label information corresponding to the image to be processed are input into the image generation model to obtain an initial image corresponding to the image to be processed, and the image to be processed is extended to improve the display effect of the image; specifically, the specific method of obtaining the initial image is as follows:

[0051] The image generation model includes an image generation network and an image identification network.

[0052] The training steps of the image generation network and the image identification network are as follows:

[0053] Obtaining a sample image training set, wherein the sample image training set includes sample images and sample labels corresponding to the sample images;

[0054] Training an initial image generation network based on the sample images and the sample labels to obtain an image generation network;

[0055] Inputting the sample image into the image generation network to obtain a sample target image;

[0056] An initial image identification network is trained based on the sample target image and the sample label to obtain an image identification network, and a trained image generation model is obtained based on the image generation network and the image identification network.

[0057] Among them, the image generation network and the image identification network can be understood as the generative model and discriminative model in the CGAN model. The generative model refers to the ability to generate text, images, videos and other data from input data through model training according to the task. The discriminative model cannot reflect the characteristics of the training data itself, but in terms of the optimal classification between different categories, it reflects the differences between heterogeneous data.

[0058] In the specific implementation, in the training of the image generation model, see Figure 3 , Figure 3 A schematic diagram showing the structure of an image generation model in an image generation method provided in one embodiment of this specification is shown;

[0059] Figure 3 (a) in the equation represents the discriminant model. Figure 3 (b) in the above diagram represents the generative model, where Figure 3 In (b), z represents the variable, y represents the condition, and y can usually be a vector or a value. Then the generated result can be expressed as x=G(z|y). Then input x into Figure 3 In the discriminant model of (a), one is to judge whether x (i.e., the input image) matches the given condition y, and the other is to judge the degree to which the generated image is a composite of the real sample.

[0060] Specifically, the image obtained by the generative model is input into the discriminative model to determine whether the input image satisfies the real sample and whether the input image matches the given conditions. The two models are trained adversarially at the same time and eventually reach a Nash equilibrium state.

[0061] During specific implementation, a sample picture training set is obtained, wherein the sample picture training set includes sample pictures and sample labels corresponding to the sample pictures, an initial picture generation network is trained based on the sample pictures and the sample labels to obtain a picture generation network, an initial picture identification network is trained based on the sample target pictures and the sample labels to obtain a picture identification network, and a picture generation model is obtained based on the picture generation network and the picture identification network, so as to input a picture to be processed and obtain an initial picture corresponding to the type of picture to be processed.

[0062] In the embodiments of this specification, through the image generation model, the obtained image to be processed can automatically generate an initial image and splice it with the image to be processed to obtain a large-scale spliced ​​image, which not only meets the user's demand for extension of the current image, but also allows the current image to continue to expand and present a better display effect to the user.

[0063] Step 206: Splicing the initial image and the image to be processed according to preset splicing conditions to obtain a spliced ​​image.

[0064] The preset splicing condition may be understood as a pre-set condition for splicing the obtained initial image with the image to be processed.

[0065] In order to quickly splice the obtained initial image with the image to be processed, specifically, before splicing the initial image with the image to be processed according to the preset splicing condition and obtaining the spliced ​​image, the process further includes:

[0066] Determining side length information of the initial image and the image to be processed;

[0067] In a case where the side length information of the initial picture is different from the side length information of the picture to be processed, the side length of the initial picture is adjusted based on the side length information of the picture to be processed.

[0068] During specific implementation, the initial image is obtained based on the image to be processed and the label information corresponding to the image to be processed, and the side length information of the initial image is determined, and the side length information of the image to be processed is determined. When the side length information of the initial image is different from the side length information of the image to be processed, the side length of the initial image is adjusted so that the side length information of the initial image is consistent with the side length information of the image to be processed. Among them, the specific method of adjusting the initial image can adopt a scaling method, or other methods that can achieve the change of the side length information of the image. The embodiments of this specification do not impose any restrictions on this.

[0069] For example, if the length of the image to be processed is a and the width is b, then the length of the initial image obtained is a and the width is c. It can be determined that the side length information of the initial image does not match the side length information of the image to be processed, that is, the width of the initial image is adjusted to b to ensure that the side length information of the initial image is the same as the side length information of the image to be processed.

[0070] In the embodiment of the present specification, by adjusting the side length information of the initial image to be the same as the side length information of the image to be processed, a complete spliced ​​image can be presented after the initial image and the image to be processed are spliced ​​together, so that further processing through the spliced ​​image can be carried out subsequently, and a continuous splicing process can be quickly achieved.

[0071] In the process of splicing the initial image to the image to be spliced, the initial image is spliced ​​according to the determined splicing conditions; specifically, the splicing conditions include the preset splicing points of the initial image and the side length information of the initial image;

[0072] Accordingly, the step of splicing the initial image and the image to be processed according to the preset splicing condition includes:

[0073] Determining the preset splicing points of the initial image and the side length information of the initial image;

[0074] Calculating a minimum distance between the initial image and the preset splicing point based on the preset splicing point and side length information of the initial image, and determining at least two to-be-spliced ​​positions of the initial image;

[0075] Based on the at least two to-be-joined positions, the initial image and the to-be-processed image are joined.

[0076] Among them, the preset splicing point can be understood as any intersection point among the intersection points of the side lengths of the image to be processed determined according to the preset splicing conditions as the initial splicing point for connection; the position to be spliced ​​can be understood as the position where the obtained initial image is spliced ​​and connected to one side of the side length of the image to be processed, and the intersection point of the initial image coincides with the preset splicing point.

[0077] During specific implementation, after receiving the image to be processed, the server determines the preset splicing points of the image to be processed and the side length information of the initial image, wherein the preset splicing point can be any splicing point in the image to be processed (which can be understood as any vertex of the image to be processed), and calculates the minimum distance between the initial image and the preset splicing point based on the preset splicing point and the side length information of the initial image, and then determines that the initial image is spliced ​​at at least two to-be-splice positions of the image to be processed. Based on the obtained to-be-splice positions, the initial image is spliced ​​on one side of the image to be processed to obtain a larger range of images.

[0078] For example, the four vertices of the image to be processed are A, B, C, and D respectively. The preset splicing point of the image to be processed determined by the server is A. The side length of the initial image is determined to be a according to the side length of the image to be processed. The preset splicing point A can make any vertex in the initial image coincide with the position to be spliced ​​determined by the preset splicing point A. In actual applications, when the preset splicing point is determined to be A, there are two positions to be spliced ​​that can coincide with the preset splicing point A. Furthermore, when the preset splicing point is determined, there are two splicing methods to achieve splicing of the initial image.

[0079] In the embodiments of this specification, by determining the preset splicing points and side length information of the image to be processed, the position of the initial image to be spliced ​​is determined to ensure that the obtained initial image is quickly spliced ​​with the image to be processed, thereby meeting the user's needs for image generation while also improving the display effect of the image.

[0080] Furthermore, the calculating the minimum distance between the initial image and the preset splicing point based on the preset splicing point and the side length information, and determining at least two to-be-spliced ​​positions of the initial image, includes:

[0081] Determining first position information corresponding to the preset splicing point based on the preset splicing point;

[0082] Determining a candidate splicing position according to the side length information of the initial image, and determining second position information corresponding to the candidate splicing position;

[0083] The minimum distance between the initial image and the preset splicing point is calculated based on the first position information and the second position information, and at least two to-be-spliced ​​positions of the initial image are determined.

[0084] The first position information may be understood as the position coordinates of a preset splicing point determined by the server, and the second position information may be understood as the position coordinates of a candidate splicing position for splicing the initial image to the image to be processed.

[0085] Based on this, based on the determined image to be processed, the candidate stitching positions include at least two, and the candidate stitching position with the smallest position coordinate distance between the candidate stitching position and the preset stitching point is determined among the at least two candidate stitching positions, and the candidate stitching position is determined as the position to be stitched of the initial image.

[0086] In actual applications, after receiving the image to be processed, the server determines the preset splicing point of the image to be processed and the side length information of the image to be processed, wherein the preset splicing point can be any splicing point in the image to be processed (which can be understood as any vertex of the image to be processed), and obtains the position coordinates of the preset splicing point based on the determined preset splicing point, and determines the position coordinates of the candidate splicing position of the initial image to be spliced ​​to the image to be processed based on the preset splicing point and the side length information of the image to be processed, calculates the minimum distance between the initial image and the preset splicing point by the breadth-first algorithm according to the position coordinates of the preset splicing point and the position coordinates of the candidate splicing position, and then determines the position to be spliced ​​of the initial image in the image to be processed, and based on the obtained position to be spliced, splices the initial image to one side of the image to be processed to obtain a larger range of images.

[0087] In the embodiments of this specification, by determining the preset splicing points and side length information of the image to be processed, the position of the initial image to be spliced ​​is determined to ensure that the obtained initial image is quickly spliced ​​with the image to be processed, thereby meeting the user's needs for image generation while also improving the display effect of the image.

[0088] When multiple positions to be stitched are obtained, the server may further determine a target position to be stitched according to a preset shape of the target image. Specifically, stitching the initial image and the image to be processed based on the at least two positions to be stitched includes:

[0089] Determining a target position to be spliced ​​among the at least two positions to be spliced ​​based on a preset target image generation shape;

[0090] The initial image and the image to be processed are spliced ​​based on the target position to be spliced.

[0091] In specific implementation, if the preset target image is a square image, when the initial image is spliced ​​to the image to be processed, the position to be spliced ​​that can generate a square image is selected as the target splicing position. In the continuous splicing process, the selection of the target splicing position is determined based on the shape of the preset target image.

[0092] In the embodiments of this specification, by determining the specific splicing position of the initial image, the initial image is accurately spliced ​​to a position that meets the splicing conditions, presenting a larger range of images to improve the display effect of the images.

[0093] Step 208: Determine whether the stitched image meets preset requirements.

[0094] Step 210: If yes, use the stitched image as the target image.

[0095] Step 212 : If not, the stitched image is used as the image to be processed and the process continues with step 204 .

[0096] The preset requirement conditions may be understood as the conditions determined by the user for the image information that meets the user's requirements.

[0097] In a specific implementation, the preset requirement condition includes that the size of the spliced ​​image is greater than or equal to a preset threshold;

[0098] Accordingly, determining whether the stitched image meets the preset requirements includes:

[0099] It is determined whether the size of the stitched image is greater than or equal to a preset threshold.

[0100] Specifically, whether the stitched picture meets a preset threshold may be determined based on the size of the stitched picture. If the size of the stitched picture is greater than or equal to the preset threshold, the stitched picture meets the preset requirement.

[0101] In the embodiment of this specification, by judging the size of the spliced ​​picture to determine whether the spliced ​​picture meets the preset requirements, not only a wider range of extended pictures can be achieved, but also the user's visual experience of the extended pictures is improved.

[0102] Furthermore, in order to make the stitched pictures meet the user's needs, a wide range of pictures are presented; specifically, before determining whether the stitched pictures meet the preset requirements, the method further includes:

[0103] Obtaining image feature information in the stitched image;

[0104] Accordingly, the step of determining whether the spliced ​​image meets a preset requirement further includes:

[0105] It is determined whether the image feature information in the spliced ​​image meets the preset feature information.

[0106] Among them, the preset feature information can be understood as the feature information contained in the extended image according to the features in different images to be processed. For example, if the image to be processed is a starry sky image, the preset feature information can be feature information with a complete Big Dipper or information with other galaxy features. In specific implementation, based on the spliced ​​image obtained by splicing the initial image and the image to be processed, and obtaining the image feature information in the spliced ​​image, it is also necessary to determine whether the spliced ​​image meets the preset requirement conditions, that is, whether it meets the preset feature information. If the spliced ​​image obtained meets the preset feature information, the spliced ​​image will be obtained as the target image; if the spliced ​​image obtained does not meet the preset feature information, the spliced ​​image will be treated as the image to be processed and continue to be spliced.

[0107] For example, taking the image to be processed as a map image as an example, the initial image obtained by inputting the image generation model is also a map-style image, and the image feature information in the map-style image is obtained, such as feature information of mountains, rivers, roads, trees, houses, etc. The initial map image is spliced ​​with the map image to be processed to obtain a spliced ​​image, and it is judged whether the spliced ​​image meets the preset feature information. If the spliced ​​image already meets the user's preset feature information for the image to be processed, that is, it meets all the preset features in the map image, then the map image of the spliced ​​image is used as the target image.

[0108] Based on this, if the spliced ​​image obtained by the user does not meet the preset feature information, the spliced ​​image will continue to be used as the image to be processed to generate the initial image and continue to splice it with the image to be processed, and the range of the image to be processed will be continuously expanded until the user's image needs are met.

[0109] In the embodiment of this specification, by judging the user's needs based on the obtained spliced ​​pictures, a wide range of pictures can be generated if the user's needs are met, so as to present a wider range of pictures to the user and expand the knowledge field of pictures.

[0110] In summary, the image generation method obtains the initial image by inputting the acquired image to be processed and label information into the image generation model, and splices the initial image with the image to be processed to generate a spliced ​​image, and then continuously splices to obtain the target image. The images to be processed can be spliced ​​together to form a large-scale image that includes more image features, which not only improves the display effect of the image, but also meets the user's display needs for images in specific scenes.

[0111] In another embodiment of the present specification, if the server still cannot meet the user's image requirements after splicing the initial images generated by the acquired images to be processed, the server will continue to process the obtained candidate images as the images to be processed; specifically, the image to be processed is the i-th image to be processed, where i is a positive integer;

[0112] Accordingly, the step of generating a request based on the target image to obtain the image to be processed and determining the tag information of the image to be processed includes:

[0113] A request is generated based on the target image to obtain an i-th image to be processed, and label information of the i-th image to be processed is determined.

[0114] The i-th to-be-processed image can be understood as the to-be-processed image obtained each time based on the target image generation request.

[0115] During specific implementation, the first picture to be processed obtained by the server, the first candidate picture obtained after inputting it into the picture generation model and splicing it with the first picture to be processed does not necessarily meet the user's needs for picture generation. Therefore, it is necessary to continue to use the first candidate picture as the second picture to be processed, input the picture generation model to obtain the second candidate picture and splice it with the second picture to be processed, and judge whether the third candidate picture obtained meets the user's needs, and repeat this cycle.

[0116] In actual applications, the i-th image to be processed is obtained based on the target image generation request, and the label information of the i-th image to be processed is determined. For example, taking i as 1, the first image to be processed obtained by the server based on the target image generation request is a map image, and the label information of the map image is determined, including road information, river information, mountain information and other information.

[0117] In the embodiments of this specification, by determining the label information of the image to be processed and inputting the corresponding label information into the image generation model to ensure that the output initial image conforms to the image style of the image to be processed, and to ensure that the initial image that meets the style conditions is spliced ​​into the image to be processed, the display effect can be improved.

[0118] In order to obtain an initial image that matches the style of the image to be processed, the image to be processed and its corresponding label information are input into the image generation model. The specific implementation method is as follows:

[0119] The step of inputting the image to be processed and the label information into an image generation model to obtain an initial image includes:

[0120] The i-th image to be processed and the label information corresponding to the i-th image to be processed are input into the image generation model to obtain an initial image.

[0121] In practical applications, the obtained i-th image to be processed and the label information corresponding to the i-th image to be processed are input into the image generation model to obtain the initial image; wherein, the image generation model is obtained by training the initial image generation network and the initial image identification network based on a large number of sample image training sets.

[0122] In the embodiments of this specification, the generated images are discriminated by the discriminant model in the CGAN model, and the output images are repeatedly determined to meet the style requirements of the images to be processed, so as to ensure that the obtained candidate images meet the user's requirements for image display, thereby also presenting a wide range of image display effects.

[0123] Furthermore, after obtaining the initial image, the initial image and the image to be processed are spliced ​​according to the preset splicing conditions. The specific implementation method is as follows:

[0124] The step of splicing the initial image and the image to be processed according to a preset splicing condition to obtain the target image includes:

[0125] Splicing the initial image and the i-th image to be processed according to the preset splicing condition to obtain an i-th spliced ​​image;

[0126] The target image is obtained based on the i-th spliced ​​image.

[0127] The preset splicing condition may be understood as a pre-set condition for splicing the obtained initial image with the image to be processed.

[0128] During specific implementation, the initial image and the i-th image to be processed are spliced ​​according to a preset initial image and the i-th image to be processed to obtain an i-th spliced ​​image, and the target image is obtained based on the i-th spliced ​​image.

[0129] In the embodiments of the present specification, by presetting splicing conditions, the initial image obtained based on the image generation model and the image to be processed are spliced ​​according to the preset splicing conditions, so that the spliced ​​image can be obtained more quickly.

[0130] Furthermore, the splicing condition includes preset splicing points of the initial image and side length information of the initial image;

[0131] Accordingly, the step of splicing the initial image with the i-th image to be processed according to the preset splicing condition includes:

[0132] Determining preset splicing points of the i pictures to be processed and side length information of the i pictures to be processed;

[0133] Calculating a minimum distance between the initial image and the preset splicing point based on the preset splicing point and the side length information, and determining a position of the initial image to be spliced;

[0134] Based on the position to be spliced, the initial image and the i-th image to be processed are spliced.

[0135] The preset splicing point can be understood as any splicable vertex of the determined i-th image to be processed, which can be set as the splicing point.

[0136] In a specific implementation, the minimum distance between the initial image and the preset splicing point is calculated based on the determined preset splicing point and the side length information, the position of the initial image to be spliced ​​is determined, and the initial image is spliced ​​with the i-th image to be processed.

[0137] In the embodiment of the present specification, the position to be spliced ​​can be determined based on the preset splicing points and the side length information to ensure that the initial image and the image to be processed are quickly spliced ​​together, thereby obtaining an enlarged image that better meets user needs.

[0138] In addition, before obtaining the target image, after splicing the initial image and the image to be processed according to a preset splicing condition, the method includes:

[0139] Determine whether the image information of the i-th spliced ​​image meets a preset requirement, wherein the preset requirement includes the preset image information of the image to be processed,

[0140] If yes, then take the i-th spliced ​​image as the target image;

[0141] If not, i is incremented by 1, and the process of generating a request based on the target image is continued to obtain the i-th image to be processed, and the label information of the i-th image to be processed is determined.

[0142] During specific implementation, after obtaining the i-th spliced ​​image, the user needs to determine whether the i-th spliced ​​image information meets the preset requirements, that is, whether it meets the preset image information of the image to be processed. If the user determines that the i-th spliced ​​image meets the preset image information of the image to be processed, the i-th spliced ​​image is used as the target image. If the user determines that the i-th spliced ​​image does not meet the preset image information of the image to be processed, i is incremented by 1, and the target image generation request is continued to be executed to obtain the i-th image to be processed, and the label information of the i-th image to be processed is determined. In the process of continuously expanding the image, if the candidate image obtained for the first time does not meet the preset requirements, the candidate image obtained for the first time can be used as the image to be processed obtained for the second time, that is, the second image to be processed when i is 2. Therefore, on the basis of continuous looping, the image to be processed can be continuously processed and expanded to obtain an image containing a larger range or more image features, until the user determines that the generated target image meets the user's preset requirements, and the above-mentioned loop expansion is stopped.

[0143] In the embodiments of this specification, by inputting the image to be processed into the image generation model, an initial image is obtained, the initial image is spliced ​​with the image to be processed, and the image to be processed is continuously expanded in a cycle so that the range of the image to be processed becomes larger and larger until it meets the user's needs for the image to be processed, and the image to be processed is continuously extended and expanded.

[0144] In summary, the image generation method obtains the initial image by inputting the acquired image to be processed and label information into the image generation model, and splices the initial image with the image to be processed to generate a spliced ​​image, and then continuously splices to obtain the target image. The images to be processed can be spliced ​​together to form a large-scale image that includes more image features, which not only improves the display effect of the image, but also meets the user's display needs for images in specific scenes.

[0145] The following combined Figure 4 Taking the application of the image generation method provided in this specification in obtaining a video frame image to generate a target image as an example, the image generation method is further described. Figure 4 A flowchart of a processing process of a picture generation method provided by an embodiment of this specification is shown, which specifically includes the following steps.

[0146] Step 402: Obtain a picture to be processed according to the video frame picture.

[0147] Step 404: Determine the label information corresponding to the image to be processed, and input the image to be processed and the label information corresponding to the image to be processed into the image generation model.

[0148] Step 406: Obtain an initial image.

[0149] Step 408: splicing the initial image with the image to be processed according to preset splicing points.

[0150] Step 410: Obtain a stitched image.

[0151] Step 412: Determine whether the stitched image meets the preset requirements.

[0152] If so, execute step 414.

[0153] Step 414: Use the stitched image as the target image.

[0154] If not, execute step 416.

[0155] Step 416: Use the spliced ​​image as the image to be processed and continue to step 404.

[0156] In the embodiments of this specification, an initial image is obtained by inputting the acquired image to be processed and label information into an image generation model, and the initial image and the image to be processed are spliced ​​together to generate a target image. This enables the images to be processed to be spliced ​​together to form a large range of images, which not only improves the display effect of the images, but also meets the user's needs for displaying images in specific scenes.

[0157] Corresponding to the above method embodiment, this specification also provides an embodiment of a picture generating device, Figure 5FIG1 shows a schematic diagram of the structure of a picture generating device provided by an embodiment of this specification. Figure 5 As shown, the device includes:

[0158] An acquisition module 502 is configured to generate a request to acquire a to-be-processed image based on a target image;

[0159] The obtaining module 504 is configured to determine the label information of the image to be processed, and input the image to be processed and the label information corresponding to the image to be processed into the image generation model to obtain an initial image corresponding to the image to be processed;

[0160] The stitching module 506 is configured to stitch the initial image and the image to be processed together according to the preset stitching conditions to obtain a stitched image.

[0161] The judging module 508 is configured to judge whether the stitched image meets the preset requirement. If so, the stitched image is used as the target image. If not, the stitched image is used as the image to be processed and the obtaining module is continued to be executed.

[0162] Optionally, the splicing module 506 is further configured to:

[0163] Determining side length information of the initial image and the image to be processed;

[0164] In a case where the side length information of the initial picture is different from the side length information of the picture to be processed, the side length of the initial picture is adjusted based on the side length information of the picture to be processed.

[0165] The splicing conditions include the preset splicing points of the initial image and the side length information of the initial image;

[0166] Optionally, the splicing module 506 is further configured to:

[0167] Determining the preset splicing points of the initial image and the side length information of the initial image;

[0168] Calculating a minimum distance between the initial image and the preset splicing point based on the preset splicing point and side length information of the initial image, and determining at least two to-be-spliced ​​positions of the initial image;

[0169] Based on the at least two to-be-joined positions, the initial image and the to-be-processed image are joined.

[0170] Optionally, the splicing module 506 is further configured to:

[0171] Determining a target position to be spliced ​​among the at least two positions to be spliced ​​based on a preset target image generation shape;

[0172] The initial image and the image to be processed are spliced ​​based on the target position to be spliced.

[0173] Optionally, the splicing module 506 is further configured to:

[0174] Determining first position information corresponding to the preset splicing point based on the preset splicing point;

[0175] Determining a candidate splicing position according to the side length information of the initial image, and determining second position information corresponding to the candidate splicing position;

[0176] The minimum distance between the initial image and the preset splicing point is calculated based on the first position information and the second position information, and at least two to-be-spliced ​​positions of the initial image are determined.

[0177] Optionally, the obtaining module 504 is further configured to: the image generation model includes an image generation network and an image identification network,

[0178] The training steps of the image generation network and the image identification network are as follows:

[0179] Obtaining a sample image training set, wherein the sample image training set includes sample images and sample labels corresponding to the sample images;

[0180] Training an initial image generation network based on the sample images and the sample labels to obtain an image generation network;

[0181] Inputting the sample image into the image generation network to obtain a sample target image;

[0182] An initial image identification network is trained based on the sample target image and the sample label to obtain an image identification network, and a trained image generation model is obtained based on the image generation network and the image identification network.

[0183] Optionally, the obtaining module 504 is further configured to:

[0184] The image to be processed is input into a label recognition model to obtain label information of the image to be processed, wherein the label recognition model outputs label information corresponding to the image to be processed.

[0185] Optionally, the determining module 508 is further configured to:

[0186] It is determined whether the size of the stitched image is greater than or equal to a preset threshold.

[0187] Optionally, the determining module 508 is further configured to:

[0188] Obtaining image feature information in the stitched image;

[0189] Optionally, the determining module 508 is further configured to:

[0190] It is determined whether the image feature information in the spliced ​​image meets the preset feature information.

[0191] The image generation method and device provided in the embodiments of this specification obtain an initial image by inputting the acquired image to be processed and label information into an image generation model, and splicing the initial image with the image to be processed to generate a spliced ​​image, and judging whether the spliced ​​image meets the preset requirements. As a result, the images to be processed can be continuously spliced ​​and connected to form a large range of images, which not only meets the user's demand for extension of the current image, but also allows the current image to continue to expand to present a better display effect to the user.

[0192] The above is a schematic diagram of an image generation device according to this embodiment. It should be noted that the technical solution of the image generation device and the technical solution of the above-mentioned image generation method are based on the same concept. For details not described in detail in the technical solution of the image generation device, please refer to the description of the technical solution of the above-mentioned image generation method.

[0193] Figure 6 6 shows a block diagram of a computing device 600 according to one embodiment of the present disclosure. Components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0194] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0195] In one embodiment of the present specification, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 6 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0196] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 600 may also be a mobile or stationary server.

[0197] The processor 620 is configured to execute the following computer-executable instructions, wherein the processor is configured to execute the computer-executable instructions, wherein the processor implements the steps of the image generation method when executing the computer-executable instructions.

[0198] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above-mentioned image generation method are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-mentioned image generation method.

[0199] An embodiment of the present specification further provides a computer-readable storage medium storing computer instructions, which implement the steps of the image generation method when executed by a processor.

[0200] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above-mentioned image generation method are based on the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-mentioned image generation method.

[0201] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0202] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0203] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0204] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0205] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for generating an image, characterized in that: include: Generate a request based on the target image to obtain the image to be processed; Determining label information of the image to be processed, and inputting the image to be processed and the label information corresponding to the image to be processed into an image generation model to obtain an initial image corresponding to the image to be processed; Splicing the initial image and the image to be processed according to a preset splicing condition to obtain a spliced ​​image; Determine whether the spliced ​​image meets the preset requirement conditions. If so, use the spliced ​​image as the target image. If not, use the spliced ​​image as the image to be processed, continue to determine the label information of the image to be processed, and input the image to be processed and the label information corresponding to the image to be processed into the image generation model to obtain the initial image corresponding to the image to be processed; The stitched image is stitched at least two to-be-stitched positions determined according to the preset stitching points of the initial image and the side length information of the initial image.

2. The image generation method according to claim 1, wherein: Before the initial image and the image to be processed are spliced ​​together according to the preset splicing condition to obtain the spliced ​​image, the method further includes: Determining side length information of the initial image and the image to be processed; In a case where the side length information of the initial picture is different from the side length information of the picture to be processed, the side length of the initial picture is adjusted based on the side length information of the picture to be processed.

3. The image generation method according to claim 1, wherein: The splicing conditions include the preset splicing points of the initial image and the side length information of the initial image; Accordingly, the step of splicing the initial image and the image to be processed according to the preset splicing condition includes: Determining the preset splicing points of the initial image and the side length information of the initial image; Calculating a minimum distance between the initial image and the preset splicing point based on the preset splicing point and side length information of the initial image, and determining at least two to-be-spliced ​​positions of the initial image; Based on the at least two to-be-joined positions, the initial image and the to-be-processed image are joined.

4. The image generation method according to claim 3, wherein: The step of splicing the initial image with the image to be processed based on the at least two positions to be spliced ​​includes: Determining a target position to be spliced ​​among the at least two positions to be spliced ​​based on a preset target image generation shape; The initial image and the image to be processed are spliced ​​based on the target position to be spliced.

5. The image generation method according to claim 3, characterized in that: The calculating the minimum distance between the initial image and the preset splicing point based on the preset splicing point and the side length information, and determining at least two to-be-spliced ​​positions of the initial image, includes: Determining first position information corresponding to the preset splicing point based on the preset splicing point; Determining a candidate splicing position according to the side length information of the initial image, and determining second position information corresponding to the candidate splicing position; The minimum distance between the initial image and the preset splicing point is calculated based on the first position information and the second position information, and at least two to-be-spliced ​​positions of the initial image are determined.

6. The image generation method according to claim 1, wherein: The image generation model includes an image generation network and an image identification network. The training steps of the image generation network and the image identification network are as follows: Obtaining a sample image training set, wherein the sample image training set includes sample images and sample labels corresponding to the sample images; Training an initial image generation network based on the sample images and the sample labels to obtain an image generation network; Inputting the sample image into the image generation network to obtain a sample target image; An initial image identification network is trained based on the sample target image and the sample label to obtain an image identification network, and a trained image generation model is obtained based on the image generation network and the image identification network.

7. The image generation method according to any one of claims 1 to 6, characterized in that: The determining of the label information of the image to be processed includes: The image to be processed is input into a label recognition model to obtain label information of the image to be processed, wherein the label recognition model outputs label information corresponding to the image to be processed.

8. The image generation method according to claim 1, wherein: The preset requirement condition includes that the size of the spliced ​​image is greater than or equal to a preset threshold; Accordingly, determining whether the stitched image meets the preset requirements includes: It is determined whether the size of the stitched image is greater than or equal to a preset threshold.

9. The image generation method according to claim 1, wherein: Before determining whether the spliced ​​image meets the preset requirement, the method further includes: Obtaining image feature information in the stitched image; Accordingly, the step of determining whether the spliced ​​image meets a preset requirement further includes: It is determined whether the image feature information in the spliced ​​image meets the preset feature information.

10. A picture generating device, characterized in that: include: An acquisition module is configured to generate a request to acquire a to-be-processed image based on a target image; an acquisition module configured to determine the label information of the image to be processed, and input the image to be processed and the label information corresponding to the image to be processed into an image generation model to obtain an initial image corresponding to the image to be processed; a splicing module configured to splice the initial image and the image to be processed according to a preset splicing condition to obtain a spliced ​​image; a judging module configured to judge whether the stitched image meets a preset requirement; if so, taking the stitched image as a target image; if not, taking the stitched image as a to-be-processed image and continuing to execute the obtaining module; The stitched image is stitched at least two to-be-stitched positions determined according to the preset stitching points of the initial image and the side length information of the initial image.

11. A computing device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein the processor implements the steps of the image generation method according to any one of claims 1 to 9 when executing the computer-executable instructions.

12. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the image generation method according to any one of claims 1 to 9.

13. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images