Image generation method and apparatus, storage medium, and electronic device

Abstract

The application discloses an image generation method and device, a storage medium and an electronic device. The method comprises the following steps: obtaining a current input drawing picture; in the case that object contour pictures corresponding to each primitive are obtained from a primitive database, the object contour pictures are spliced to obtain a candidate image, wherein the object contour picture is a reference material picture of the category to which the primitive belongs; the candidate image is corrected to obtain a first target image; in response to a view angle adjustment operation, it is determined that the first view angle is adjusted to a second view angle; the first target image and the second view angle are input into an image generation network to generate a second target image under the second view angle, wherein the image generation is a neural network for generating an image to be displayed under a specified view angle after learning from a sample image. The application solves the technical problem that the generation of a stereoscopic image effect is complicated.

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically, to an image generation method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] With the development of science and technology, more and more applications can generate stereoscopic images based on user images. However, in existing technologies, stereoscopic images are usually generated based on images uploaded by users, and the quality requirements for uploaded images are relatively high; only relatively clear images can achieve the generation of stereoscopic images.

[0003] When relying on user-uploaded images and having high image quality requirements, generating 3D images becomes more demanding, reducing user engagement and diminishing the interactive effects of 3D image generation.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides an image generation method and apparatus, storage medium and electronic device to at least solve the technical problem of cumbersome generation of stereoscopic image effects.

[0006] According to one aspect of the present invention, an image generation method is provided, comprising: acquiring a currently input drawing, wherein the drawing includes at least one primitive; when acquiring object contour maps corresponding to each of the primitives from a primitive database, stitching the object contour maps together to obtain a candidate image, wherein the object contour maps are reference material images of the category to which the primitives belong; modifying the candidate image to obtain a first target image; in response to a viewpoint adjustment operation, determining to adjust from the current first viewpoint to a second viewpoint; inputting the first target image and the second viewpoint into an image generation network to generate a second target image under the second viewpoint, wherein the image generation is a neural network for generating an image to be displayed under a specified viewpoint, obtained by learning from sample images.

[0007] According to another aspect of the present invention, an image generation apparatus is also provided, comprising: an acquisition unit for acquiring a currently input drawing, wherein the drawing includes at least one primitive; a stitching unit for stitching the object contour images to obtain a candidate image when object contour images corresponding to each of the primitives are acquired from a primitive database, wherein the object contour images are reference material images of the category to which the primitives belong; a correction unit for correcting the candidate image to obtain a first target image; a viewpoint adjustment unit for determining, in response to a viewpoint adjustment operation, to adjust from the current first viewpoint to a second viewpoint; and a generation unit for generating a second target image under the second viewpoint by using the first target image and the second viewpoint input image to generate a second target image under the second viewpoint, wherein the image generation is a neural network for generating an image to be displayed under a specified viewpoint, obtained by learning from sample images.

[0008] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the above-described image generation method at runtime.

[0009] According to another aspect of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the above-described image generation method through the computer program.

[0010] In this embodiment of the invention, the method involves acquiring a drawing image including at least one primitive as input, obtaining object contour images corresponding to each primitive from a primitive database, stitching the object contour images to obtain candidate images, correcting the candidate images to obtain a first target image, and, in response to a viewpoint adjustment operation, determining to adjust from a first viewpoint to a second viewpoint. The first target image and the second viewpoint are then input into an image generation network to generate a second target image from the second viewpoint. This method generates a first target image corresponding to the input drawing image by acquiring object contour images from the primitive database, stitching them together, and then correcting the stitched candidate images. In response to the viewpoint adjustment operation, when the first viewpoint is adjusted to the second viewpoint, an image generation network is used to generate a second target image corresponding to the first target image from the second viewpoint. This achieves optimization of the detail filling of the input drawing image and generates a second target image corresponding to the viewpoint based on the optimized first target image. The purpose of generating a two-dimensional image from different viewpoints to create a stereoscopic interactive effect of the target image is achieved, thus realizing the technical effect of generating a stereoscopic effect of the target image based on the generation of two-dimensional images from different viewpoints, thereby solving the technical problem of cumbersome stereoscopic image generation. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0012] Figure 1 This is a schematic diagram of an application environment for an optional image generation method according to an embodiment of the present invention;

[0013] Figure 2 This is a flowchart illustrating an optional image generation method according to an embodiment of the present invention;

[0014] Figure 3 This is a flowchart illustrating an optional image generation method according to an embodiment of the present invention;

[0015] Figure 4 This is a flowchart illustrating an optional image generation method according to an embodiment of the present invention;

[0016] Figure 5 This is a flowchart illustrating an optional image generation method according to an embodiment of the present invention;

[0017] Figure 6 This is a flowchart illustrating an optional image generation method according to an embodiment of the present invention;

[0018] Figure 7 This is an interactive timing diagram of an optional image generation method according to an embodiment of the present invention;

[0019] Figure 8 This is a schematic diagram of the structure of an optional image generation apparatus according to an embodiment of the present invention;

[0020] Figure 9 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] According to one aspect of the present invention, an image generation method is provided. Optionally, the image generation method may be applied to, but is not limited to, [examples of image generation methods]. Figure 1 In the illustrated environment, terminal device 102 interacts with server 112 via network 110, generates an image based on the drawing operations received by terminal device 102, and displays the image with a 3D effect on terminal device 102. Terminal device 102 sends the received drawing instructions to server 112 via network 110. Server 112 runs a database 114 and a processing engine 116. Upon receiving the drawing instructions, server 112 may generate the image by executing steps S102 to S110 sequentially, but is not limited to this step.

[0024] S102, Obtain the drawing image. The drawing image is not limited to the image corresponding to the drawing instruction; the currently input drawing image is obtained, and the drawing image includes at least one primitive. S104, Obtain candidate images. If object contour images corresponding to each primitive are obtained from the primitive database, the object contour images are stitched together to obtain candidate images. The object contour images are reference material images of the primitive category. S106, Obtain the first target image. The candidate images are corrected to obtain the first target image. If the first target image is obtained, it is not limited to sending the first target image to the terminal device 102 via network 110 for display on the terminal device 102. S108, Determine to adjust to a second viewpoint. In response to the viewpoint adjustment operation, it is determined to adjust from the current first viewpoint to a second viewpoint. S110, Generate a second target image. The first target image and the second viewpoint are input to an image generation network to generate a second target image under the second viewpoint. The image generation network is a neural network learned from sample images for generating the image to be displayed under the specified viewpoint. In the case of generating a second target image, it is not limited to sending the second target image to the terminal device 102 via network 110 so as to display the second target image from a second perspective on the terminal device 102.

[0025] Optionally, in this embodiment, the terminal device 102 can be a terminal device configured with a target client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, handheld computer, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The target client can be a client with drawing capabilities, and is not limited to audio client, video client, instant messaging client, browser client, educational client, etc. The network 110 may include, but is not limited to, wired network and wireless network. The wired network includes: local area network, metropolitan area network, and wide area network. The wireless network includes: Bluetooth, WIFI, and other networks that enable wireless communication. The server 112 can be a single server, a server cluster composed of multiple servers, or a cloud server. The above is only an example, and no limitation is made in this embodiment.

[0026] As an optional implementation method, such as Figure 2 As shown, the above image generation method includes:

[0027] S202, obtain the currently input drawing image, wherein the drawing image includes at least one graphic element.

[0028] Primitives are not limited to the pictorial elements included in the input drawing. Pictorial elements can be the same element with the same attributes distributed in different positions in the drawing, such as multiple identical water ripples distributed in a landscape painting. When the attributes are different, the pictorial elements are identified as different primitives, such as stones of different sizes and colors.

[0029] As an optional implementation, obtaining the currently input drawing image includes one of the following:

[0030] Obtain the operation trajectory formed by the touch operation triggered on the touch screen where the display interface is located, and use the picture formed by the operation trajectory as the drawing picture;

[0031] The motion trajectory of non-contact gestures is detected, and the motion trajectory is mapped onto the drawing trajectory on the display interface to form a drawing picture.

[0032] The target image is retrieved from local storage, the trajectory contained in the target image is identified, and the resulting drawing is obtained.

[0033] Drawing can be based on the operation trajectory determined by touch operations on the display interface. The display interface refers to the display interface on the terminal device corresponding to the server. When the terminal device receives a touch operation, it is not limited to converting the touch operation into an operation command. The server determines the operation trajectory based on the operation command and uses the image formed by the operation trajectory as the drawing image.

[0034] The drawn image can also be the image corresponding to the motion trajectory of a non-contact gesture received by the terminal device. Non-contact gestures are not limited to image acquisition devices of the terminal device, such as gestures captured by a camera. When the terminal device detects a non-contact gesture, it converts the gesture into a drawing instruction. The server then maps the motion trajectory of the non-contact gesture onto a canvas based on the drawing instruction, generating the drawn image.

[0035] Drawing can also be done by uploading a target image to a terminal device. When the server receives the target image, it identifies the trajectory included in the target image to obtain the drawing.

[0036] S204, after obtaining the object outline image corresponding to each graphic element from the graphic element database, the object outline images are stitched together to obtain candidate images, wherein the object outline image is a reference material image of the category to which the graphic element belongs.

[0037] The primitive database stores reference images belonging to different categories, with each category containing multiple reference images. Reference images are not limited to the object outlines corresponding to the primitive within their respective categories. The object outline includes both the outer contour lines and the internal lines of the primitive. Taking an eye as an example, the object outline stored in the primitive database includes, but is not limited to, the outer contour lines of the eye and lines such as the eyeball located inside the eye.

[0038] S206, The candidate image is corrected to obtain the first target image.

[0039] The candidate image obtained by stitching the object outline is corrected, not limited to adjusting the stitching method of the object outline, and the candidate image after adjusting the stitching method is supplemented with details according to the image style to obtain a first target image with more details.

[0040] S208, in response to the view adjustment operation, determines to switch from the current first view to the second view.

[0041] The perspective adjustment operation is not limited to the perspective adjustment operation received by the client displaying the first target image, nor is it limited to adjusting the perspective in three-dimensional space of the displayed first target image. The angle calculation bases for the first and second perspectives are the same, but the second perspective can be different relative to the canvas plane where the first perspective is located. For example, the first perspective is obtained by rotating the canvas plane corresponding to the zero-degree perspective by any angle, and the second perspective is obtained by rotating the canvas plane corresponding to the zero-degree perspective by any non-zero angle.

[0042] S210, the first target image and the second viewpoint are input to the image generation network to generate the second target image from the second viewpoint, wherein the image generation is a neural network obtained by learning from sample images to generate the image to be displayed from the specified viewpoint.

[0043] An image generation network is used to generate a second target image corresponding to the first target image from the second viewpoint, based on a first target image and a second viewpoint. Both the first and second target images are two-dimensional planar images, and their content is related. The image content of the second target image is the content of the first target image corresponding to the content in the second viewpoint.

[0044] In this embodiment, the method involves acquiring a drawing containing at least one primitive as input, obtaining object contour maps corresponding to each primitive from a primitive database, stitching the object contour maps to obtain candidate images, correcting the candidate images to obtain a first target image, and, in response to a viewpoint adjustment operation, determining a shift from a first viewpoint to a second viewpoint. The first target image and the second viewpoint are then input into an image generation network to generate a second target image from the second viewpoint. This method generates a first target image corresponding to the input drawing by acquiring object contour maps from the primitive database, stitching them together, and then correcting the stitched candidate images. In response to the viewpoint adjustment operation, when the viewpoint shifts from the first to the second viewpoint, an image generation network generates a second target image corresponding to the first target image from the second viewpoint. This achieves optimization of detail filling in the input drawing and generates a second target image corresponding to the viewpoint based on the optimized first target image. The generation of two-dimensional images from different viewpoints creates a stereoscopic interactive effect for the target image, thus achieving the technical effect of generating a stereoscopic effect based on two-dimensional images from different viewpoints, thereby solving the cumbersome technical problem of generating stereoscopic image effects.

[0045] As an optional implementation method, such as Figure 3 As shown, the above method, in which the object contour maps corresponding to each graphic element are obtained from the graphic element database, stitches the object contour maps together to obtain candidate images, including:

[0046] S302, the drawing is split to obtain at least one graphic element and the positional relationship between the graphic elements;

[0047] S304, find the object outline image that matches each graphic element in the graphic element database, wherein the similarity between the object outline image and the graphic element is greater than the first threshold, and the graphic element database stores reference material images of different categories of graphic elements;

[0048] S306. The outlines of the found objects are stitched together according to the positional relationship of the graphic elements to obtain candidate images.

[0049] Decomposing a drawn image into its primitives is not limited to using classification algorithms or models. Taking a classification model as an example, the model is trained using primitives of different categories to obtain a model with a classification accuracy exceeding a preset threshold. The trained classification model is then used to decompose the drawn image, resulting in each primitive within the image. Once each primitive is identified, its position within the image and the positional relationships between them are determined. Positional relationships indicate whether primitives are adjacent or connected by lines.

[0050] The primitive database stores object contour images of different categories. An object contour image is selected from multiple object contour images in the primitive database based on its similarity to the primitive. The object contour image corresponding to each primitive is searched sequentially in the primitive database. The primitive is replaced at its original position using the corresponding object contour image to obtain a candidate image. When replacing primitives with object contour images, the size of the object contour image is not limited to adjusting according to the size of the primitive; instead, the object contour images are stitched together at the primitive's position to obtain the candidate image.

[0051] In this embodiment of the application, the object outline map matching the graphic elements in the graphic element database is searched sequentially according to the graphic elements included in the drawing, and the drawn graphic elements are replaced with a more standardized object outline map, thereby achieving the optimization of the line standardization of the drawing.

[0052] As an optional implementation, the above-described modification of the candidate image to obtain the first target image includes:

[0053] S1, perform smoothing adjustments on the candidate image to obtain the adjusted candidate image;

[0054] S2, the adjusted candidate image is input into the style generation network to obtain the first target image, wherein the style generation network is a neural network trained using sample images to fill rendering content for different image regions.

[0055] Smoothing candidate images is not limited to smoothing the lines connecting adjacent object contours that are connected by lines. It involves connecting lines located in two separate object contours and adjusting the smoothness of the connecting lines to obtain candidate images with complete and smooth line connections.

[0056] The adjusted candidate image is input into the style generation network to obtain the first target image as output. The style generation network is used to fill in the rendering content of the input image, resulting in an output image with richer details.

[0057] As an optional implementation method, such as Figure 4 As shown, the above-mentioned input of the adjusted candidate image into the style generation network to obtain the first target image includes:

[0058] S402, perform image recognition on the adjusted candidate image to determine the first image style corresponding to the adjusted candidate image and the blank image area in the adjusted candidate image;

[0059] S404, Obtain the fill primitives that match the blank image area according to the first image style;

[0060] S406, fill the blank image area with the fill primitives to obtain the first target image.

[0061] The style generation network identifies the input image, determines its image style, and identifies blank image regions within the input image that do not contain primitives. The image style is the preset style from the style generation network that is most similar to the input image. When the input image is a line drawing, the image style is not limited to indicating the complexity of the input image or the style to which the image content belongs. When the input image is not a line drawing, the image style is not limited to indicating the drawing style of the input image, such as traditional Chinese painting, watercolor painting, or oil painting.

[0062] Taking the input image, i.e. the adjusted candidate image, as a line drawing as an example, the fill primitives to be filled in the blank image area are determined according to the complexity of the input image and / or the style of the content of the input image.

[0063] As an optional implementation method, such as Figure 5 As shown, the filling primitives determined according to the first image style to match the blank image region include:

[0064] S502, determine the default primitive type corresponding to the blank image area based on the first image style;

[0065] S504, Search for the first graphic element that corresponds to the first image style in the graphic element set corresponding to the default graphic element type;

[0066] S506, adjust the first primitive according to the blank image area to obtain the filled primitive.

[0067] Taking the adjusted candidate image as a line drawing as an example, the default primitive type corresponding to the blank image region is determined according to the image style of the adjusted candidate image. If the candidate image generated by the input style is a face image, then the blank image regions in the face image are determined to be the cheek region, forehead region, and chin region. When the number of blank image regions is greater than one, the filling primitive for each blank image region is determined sequentially. If the default primitive type for the forehead region is determined to be a texture type, then the first primitive corresponding to the first image style is searched in the texture primitive set, not limited to determining the texture primitive based on complexity. Once the texture primitive is determined, its color, size, and other attribute parameters are adjusted to obtain the filling primitive used to fill the blank image regions. The filling primitive is then used to fill the blank image regions to obtain the first target image.

[0068] In this embodiment of the application, a style generation network is used to fill and render primitives in the adjusted candidate image. Primitives are filled and rendered in blank image areas according to the image style, thereby obtaining a first target image with richer details.

[0069] As an optional implementation method, such as Figure 6 As shown, the above-mentioned network for generating a second target image from a second viewpoint and an input image from a first target image includes:

[0070] S602, perform image recognition on the first target image to determine the first primitive outline of the first target image;

[0071] S604, search the template database for the first template corresponding to the first primitive outline, wherein the first template includes the first outline corresponding to the first primitive outline and the outline image of the first outline under each view.

[0072] S606, Search for the second candidate image corresponding to the second viewpoint in the first template;

[0073] S608, the second candidate image is adjusted according to the first target image to generate the second target image from the second perspective.

[0074] A second target image is generated from a first target image using an image generation network, viewed from a second perspective. This generation is not limited to generating the second target image based on the primitive contours of the first target image. The image generation network performs image recognition on the first target image to obtain a first primitive contour, which represents the category to which the first target image belongs. Taking a human face image as an example, the first primitive contour includes the contours of facial features and the face itself. Taking a landscape image as an example, the first primitive contour includes the overall outer contour of the image and the contours of objects such as mountains, rocks, and water.

[0075] The template database stores templates of different categories, each including contour images of the corresponding contour from various viewpoints. A first template containing a first contour belonging to the same category as the first primitive contour is determined based on the first primitive contour. A second candidate image of the first contour from a second viewpoint is obtained from the first template. The second candidate image is adjusted based on the first target image to generate the second target image.

[0076] Optionally, the second candidate image is adjusted based on the first target image, but this adjustment is not limited to adjusting based on the first primitives included in the first target image:

[0077] S1, determine the second candidate image corresponding to the first primitive in the first target image in the second candidate image;

[0078] S2, adjust the second candidate primitives according to the primitive features of the first primitive to generate the second target image.

[0079] The first target image is identified as containing at least one first primitive, and the primitive features of the first primitive are determined. Primitive features include, but are not limited to, primitive color, primitive texture, and rendering method. A second candidate primitive corresponding to the first primitive is found in the second candidate image. For example, if the first target image is a face image and the first primitive is the nose, the second candidate primitive corresponding to the nose is found in the second candidate image.

[0080] If no second candidate primitive corresponding to the first primitive is found, the search continues for the second candidate primitive corresponding to the next first primitive. For example, if the first viewpoint is a zero-degree viewpoint and the second viewpoint is a 175-degree rotation of the back of the head of a face image, then the head contour of the second candidate image in the second viewpoint will not include the nose-related second candidate primitive. If no second candidate primitive is found, the adjustment of the second candidate primitive corresponding to the next first primitive continues.

[0081] After adjusting the corresponding second candidate primitives according to the primitive features of each first primitive in sequence, the adjusted second candidate image is used as the second target image to generate a second target image corresponding to the first target image in the second view.

[0082] In this embodiment, the image generation model can directly utilize template images from various perspectives and adjust them according to the first target image to conveniently and quickly generate a second target image corresponding to the first target image from a second perspective. The image content of the target image changes with the perspective adjustment operation to form a stereoscopic display effect of the target image. The convenient generation of two-dimensional images from various perspectives to present a stereoscopic image effect simplifies the stereoscopic image effect generation process by replacing the generation of stereoscopic images.

[0083] The image generation method described above, in the interaction between the target application on the terminal device and the server, generates a target image and uses two-dimensional target images from different perspectives to form a stereoscopic display effect. This interaction is not limited to, for example... Figure 7 As shown. The target application executes S702 to send a drawing instruction / image. The target application receives a touch operation or non-contact gesture to generate a corresponding drawing instruction, or selects a locally stored image to generate the target image. Upon receiving the drawing instruction or image, the server executes S704 to obtain the drawing image. The drawing image is generated based on the drawing trajectory or image indicated by the drawing instruction. Simultaneously with generating the drawing image, it is sent to the target application, causing the target application to execute S706 to display the drawing image. At this point, the drawing image is either the drawing trajectory corresponding to the touch operation or non-contact gesture, or the trajectory in the selected image.

[0084] After determining the drawing to be drawn, the server executes S708 to optimize the drawing and obtain the target image. The optimized target image is then sent to the target application, causing the target application to execute S710 to display the target image. At this point, the target image is the image obtained after optimizing and rendering the drawing trajectory or image trajectory. The target image is an image located at a zero-degree viewpoint. If the target application receives a first-viewpoint adjustment operation based on the displayed target image, it executes S712 to send the first viewpoint to the server. The first viewpoint is the viewpoint obtained by adjusting the zero-degree viewpoint. Upon determining the first viewpoint, the server executes S714 to generate the first target image and sends it to the target application. The first target image is a two-dimensional image of the target image at the first viewpoint. When the target application receives the first target image, it executes S716 to display the first target image. At this point, the target application can form a first target image where the displayed image content changes according to the first viewpoint adjustment operation at the corresponding viewpoint.

[0085] When the target application receives a second perspective adjustment operation based on the displayed target image, it executes S718 to send the second perspective to the server. The second perspective is the perspective obtained by adjusting the first perspective. Upon determining the second perspective, the server executes S720 to generate a second target image and sends it to the target application. The second target image is a two-dimensional image of the target image from the second perspective. When the target application receives the second target image, it executes S722 to display the second target image. At this time, a second target image can be formed in the target application, showing the changes in the displayed image content of the first target image under the corresponding perspective following the second perspective adjustment operation.

[0086] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0087] According to another aspect of the present invention, an image generation apparatus for implementing the above-described image generation method is also provided. For example... Figure 8 As shown, the device includes:

[0088] The acquisition unit 802 is used to acquire the currently input drawing picture, wherein the drawing picture includes at least one graphic element;

[0089] The stitching unit 804 is used to stitch together the object outline images to obtain candidate images when the object outline images corresponding to each object are obtained from the object database. The object outline images are reference material images of the category to which the object belongs.

[0090] The correction unit 806 is used to correct the candidate image to obtain the first target image;

[0091] The viewpoint adjustment unit 808 is used to respond to the viewpoint adjustment operation and determine to adjust from the current first viewpoint to the second viewpoint;

[0092] The generation unit 810 is used to input the first target image and the second viewpoint into the image generation network to generate the second target image from the second viewpoint. The image generation is a neural network that is learned from sample images and used to generate the image to be displayed from the specified viewpoint.

[0093] Optionally, the splicing unit 804 includes:

[0094] The splitting module is used to split the drawn image to obtain at least one graphic element and the positional relationship between the graphic elements;

[0095] The outline search module is used to find the outline of the object that matches each graphic element in the graphic element database. The similarity between the outline of the object and the graphic element is greater than a first threshold. The graphic element database stores reference material images of different categories of graphic elements.

[0096] The stitching module is used to stitch together the outlines of the found objects according to the positional relationship of the graphic elements to obtain candidate images.

[0097] Optionally, the above-mentioned correction unit 806 includes:

[0098] The smoothing module is used to smooth the candidate image to obtain the adjusted candidate image;

[0099] The input module is used to input the adjusted candidate image into the style generation network to obtain the first target image. The style generation network is a neural network trained using sample images to fill rendering content for different image regions.

[0100] Optionally, the above input module is further configured to perform image recognition on the adjusted candidate image, determine the first image style corresponding to the adjusted candidate image and the blank image region in the adjusted candidate image; obtain the filling primitive matching the blank image region according to the first image style; fill the blank image region with the filling primitive to obtain the first target image.

[0101] Optionally, the above input module is further configured to determine the filling primitive that matches the blank image area according to the first image style, including: determining the default primitive type corresponding to the blank image area according to the first image style; searching for the first primitive that matches the first image style in the primitive set corresponding to the default primitive type; and adjusting the first primitive according to the blank image area to obtain the filling primitive.

[0102] Optionally, the generation unit 810 includes:

[0103] The recognition module is used to perform image recognition on the first target image and determine the first primitive outline constructed by the first primitives included in the first target image.

[0104] The template lookup module is used to search the template database for a first template corresponding to the first primitive outline. The first template includes a first outline corresponding to the first primitive outline and outline images of the first outline at various viewpoints.

[0105] The candidate module is used to find a second candidate image in the first template that corresponds to the second viewpoint;

[0106] The image adjustment module is used to adjust the second candidate image based on the first target image to generate the second target image from a second perspective.

[0107] Optionally, the acquisition unit 802 includes one of the following: acquiring the operation trajectory formed by the touch operation triggered on the touch screen where the display interface is located, and using the picture formed by the operation trajectory as the drawing picture; detecting the action trajectory of a non-contact gesture, and using the picture formed by mapping the action trajectory to the drawing trajectory on the display interface as the drawing picture; acquiring a target image stored locally, recognizing the trajectory contained in the target image, and obtaining the drawing picture.

[0108] In this embodiment, the method involves acquiring a drawing containing at least one primitive as input, obtaining object contour maps corresponding to each primitive from a primitive database, stitching the object contour maps to obtain candidate images, correcting the candidate images to obtain a first target image, and, in response to a viewpoint adjustment operation, determining a shift from a first viewpoint to a second viewpoint. The first target image and the second viewpoint are then input into an image generation network to generate a second target image from the second viewpoint. This method generates a first target image corresponding to the input drawing by acquiring object contour maps from the primitive database, stitching them together, and then correcting the stitched candidate images. In response to the viewpoint adjustment operation, when the viewpoint shifts from the first to the second viewpoint, an image generation network generates a second target image corresponding to the first target image from the second viewpoint. This achieves optimization of detail filling in the input drawing and generates a second target image corresponding to the viewpoint based on the optimized first target image. The generation of two-dimensional images from different viewpoints creates a stereoscopic interactive effect for the target image, thus achieving the technical effect of generating a stereoscopic effect based on two-dimensional images from different viewpoints, thereby solving the cumbersome technical problem of generating stereoscopic image effects.

[0109] According to another aspect of the present invention, an electronic device for implementing the above-described image generation method is also provided, the electronic device being... Figure 1 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 9 As shown, the electronic device includes a memory 902 and a processor 904. The memory 902 stores a computer program, and the processor 904 is configured to execute the steps of any of the above method embodiments through the computer program.

[0110] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0111] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0112] S1, obtain the currently input drawing image, wherein the drawing image includes at least one graphic element;

[0113] S2, after obtaining the object outline image corresponding to each graphic element from the graphic element database, the object outline images are stitched together to obtain candidate images, wherein the object outline image is a reference material image of the category to which the graphic element belongs;

[0114] S3, The candidate image is corrected to obtain the first target image;

[0115] S4, in response to the view adjustment operation, determines to switch from the current first view to the second view;

[0116] S5, the first target image and the second viewpoint input image are used to generate a second target image from the second viewpoint. The image generation is a neural network that is learned from sample images and used to generate the image to be displayed from the specified viewpoint.

[0117] Alternatively, as those skilled in the art will understand, Figure 9 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 9 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 9 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 9 The different configurations shown.

[0118] The memory 902 can be used to store software programs and modules, such as the program instructions / modules corresponding to the image generation method and apparatus in this embodiment of the invention. The processor 904 executes various functional applications and data processing by running the software programs and modules stored in the memory 902, thereby realizing the image generation method described above. The memory 902 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 902 may further include memory remotely located relative to the processor 904, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 902 may be used, but is not limited to, to store information such as a first target image and a second target image. As an example, such as... Figure 9 As shown, the memory 902 may include, but is not limited to, the acquisition unit 802, stitching unit 804, correction unit 806, viewing angle adjustment unit 808, and generation unit 810 in the image generation apparatus described above. Furthermore, it may include, but is not limited to, other module units in the image generation apparatus described above, which will not be elaborated upon in this example.

[0119] Optionally, the transmission device 906 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 906 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 906 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0120] In addition, the aforementioned electronic device also includes: a display 908 for displaying the aforementioned drawn picture, the first target image, and the second target image; and a connection bus 910 for connecting the various module components in the aforementioned electronic device.

[0121] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0122] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations of the image generation aspect described above. The computer program is configured to perform the steps in any of the method embodiments described above during runtime.

[0123] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store a computer program for performing the following steps:

[0124] S1, obtain the currently input drawing image, wherein the drawing image includes at least one graphic element;

[0125] S2, after obtaining the object outline image corresponding to each graphic element from the graphic element database, the object outline images are stitched together to obtain candidate images, wherein the object outline image is a reference material image of the category to which the graphic element belongs;

[0126] S3, The candidate image is corrected to obtain the first target image;

[0127] S4, in response to the view adjustment operation, determines to switch from the current first view to the second view;

[0128] S5, the first target image and the second viewpoint input image are used to generate a second target image from the second viewpoint. The image generation is a neural network that is learned from sample images and used to generate the image to be displayed from the specified viewpoint.

[0129] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0130] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0131] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0132] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0133] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0136] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An image generation method, characterized in that, include: Obtain the currently input drawing image, wherein the drawing image includes at least one graphic element; When object outline images corresponding to each of the graphic elements are obtained from the graphic element database, the object outline images are stitched together to obtain candidate images, wherein the object outline images are reference material images of the category to which the graphic elements belong; The candidate image is corrected to obtain the first target image; In response to a viewpoint adjustment operation, the viewpoint is switched from the current first-person view to the second-person view. The first target image and the second viewpoint are input into the image generation network to generate the second target image from the second viewpoint. The image generation network is a neural network that is learned from sample images and is used to generate the image to be displayed from the specified viewpoint.

2. The method according to claim 1, characterized in that, When object contour maps corresponding to each of the aforementioned primitives are obtained from a primitive database, the object contour maps are stitched together to obtain candidate images, including: The drawn image is split to obtain at least one graphic element and the positional relationship between the graphic elements; The object outline image that matches each of the graphic elements is found in the graphic element database, wherein the similarity between the object outline image and the graphic element is greater than a first threshold, and the graphic element database stores reference material images of different categories of graphic elements; The candidate image is obtained by stitching together the outlines of the found objects according to the positional relationships of the graphic elements.

3. The method according to claim 1, characterized in that, The candidate image is corrected to obtain the first target image, including: The candidate image is smoothed to obtain the adjusted candidate image; The adjusted candidate image is input into the style generation network to obtain the first target image, wherein the style generation network is a neural network trained using sample images to fill rendering content for different image regions.

4. The method according to claim 3, characterized in that, The adjusted candidate image is input into a style generation network to obtain the first target image, including: Image recognition is performed on the adjusted candidate image to determine the first image style corresponding to the adjusted candidate image and the blank image region in the adjusted candidate image; Obtain filling primitives that match the blank image region according to the first image style; The filling primitives are filled into the blank image area to obtain the first target image.

5. The method according to claim 4, characterized in that, The step of determining the filling primitives that match the blank image region according to the first image style includes: Based on the first image style, determine the default primitive type corresponding to the blank image region; Search for the first graphic element that corresponds to the first image style in the graphic element set corresponding to the default graphic element type; The first primitive is adjusted according to the blank image area to obtain the filled primitive.

6. The method according to claim 1, characterized in that, The network generates a second target image from the first target image and the second viewpoint input image, including: Image recognition is performed on the first target image to determine the first primitive outline of the first target image; Search the template database for a first template corresponding to the first primitive outline, wherein the first template includes a first outline corresponding to the first primitive outline and outline images corresponding to the first outline from various viewpoints. Search the first template for a second candidate image that corresponds to the second viewpoint; The second candidate image is adjusted based on the first target image to generate the second target image from the second perspective.

7. The method according to any one of claims 1 to 6, characterized in that, The drawing to obtain the current input includes one of the following: The operation trajectory formed by the touch operation triggered on the touch screen where the display interface is located is obtained, and the picture formed by the operation trajectory is used as the drawing picture. The motion trajectory of a non-contact gesture is detected, and the motion trajectory is mapped onto the drawing trajectory on the display interface to form a drawing picture. Obtain the target image stored locally, identify the trajectory contained in the target image, and obtain the drawn picture.

8. An image generation apparatus, characterized in that, include: An acquisition unit is used to acquire the currently input drawing image, wherein the drawing image includes at least one graphic element; The stitching unit is used to stitch together the object outline images to obtain a candidate image when the object outline images corresponding to each of the graphic elements are obtained from the graphic element database, wherein the object outline images are reference material images of the category to which the graphic elements belong; A correction unit is used to correct the candidate image to obtain a first target image; A viewpoint adjustment unit is used to respond to viewpoint adjustment operations and determine whether to adjust from the current first viewpoint to a second viewpoint. The generation unit is used to input the first target image and the second viewpoint into the image generation network to generate a second target image from the second viewpoint. The image generation network is a neural network that is learned from sample images and is used to generate the image to be displayed from the specified viewpoint.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 7 through the computer program.

Images