Digital image generation method and device, equipment and medium

Abstract

The invention discloses a digital image generation method and device, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: obtaining a first digital image; segmenting each image frame in an image frame sequence corresponding to the first digital image to obtain a first head image and a first body image; adjusting the first head image according to an expression reasoning result to obtain a second head image; fusing the second head image with the first body image to obtain a second digital image; extracting skeleton information of the first digital image and / or the second digital image; and performing digital human generation according to the skeleton information and a two-dimensional image corresponding to the second digital image to obtain a third digital image. According to the method, the problem of sewing dislocation can be reduced, and the interaction experience is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and medium for generating a digital image. Background Art

[0002] Digital image refers to a digital image of a person that is close to a human image and is created using digital technology. It uses a variety of technical means, such as motion capture, computer graphics, graphics rendering, deep learning, speech synthesis, etc., to create a comprehensive product with multiple features such as appearance, performance ability, and interaction ability similar to that of a human.

[0003] At present, in order to make the expressions of digital images richer, the head and body of the digital human are segmented, and then the head is corrected using the results of expression reasoning. Finally, the body and the corrected head are stitched together to obtain a digital image with expressions.

[0004] However, after the body is stitched with the corrected head, there will be a problem of stitching misalignment, which affects the realism of the digital image and reduces the interactive experience. Summary of the invention

[0005] The present application provides a method, device, equipment and medium for generating a digital image, which can reduce the problem of stitching dislocation and improve the interactive experience.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions: In a first aspect, the present application provides a method for generating a digital image, comprising: obtaining a first digital image; Segmenting each image frame in the image frame sequence corresponding to the first digital image to obtain a first head image and a first body image; According to the expression inference result, adjusting the first head image to obtain a second head image; Merging the second head image with the first body image to obtain a second digital image; Extracting skeleton information of the first digital image and / or the second digital image; A digital human is generated based on the skeleton information and the two-dimensional image corresponding to the second digital image to obtain a third digital image.

[0007] Optionally, extracting skeleton information of the first digital image and / or the second digital image includes: Extract key points from the 2D image corresponding to the second digital image to obtain key point information of multiple parts; or extract key points from the 2D image corresponding to the second digital image to obtain key point information of multiple parts; generate skeleton information based on the key point information of multiple parts.

[0008] Optionally, generating a digital human according to the skeleton information and a two-dimensional image corresponding to the second digital image to obtain a third digital image includes:

[0009] Indicates the third digital image, Indicates that the parameter is Generate a digital human model, Indicates that the parameter is Digital human control model, represents a two-dimensional image corresponding to the second digital image, Represents key point information of multiple parts of the second digital image.

[0010] Optionally, the step of fusing the second head image with the first body image to obtain a second digital image includes:

[0011] Indicates the second digital image, represents the stitching model for the head image and the body image, Indicates the first head image, Indicates the first body image.

[0012] Optionally, segmenting each image frame in the image frame sequence corresponding to the first digital image to obtain a first head image and a first body image includes: Performing edge detection on each image frame in the image frame sequence corresponding to the first digital image to obtain an edge detection result, and performing face recognition on the first digital image to obtain a face recognition result; Determining a segmentation boundary according to first boundary information in the edge detection result and second boundary information in the face recognition result; The first digital image is segmented according to the segmentation boundary to obtain a first head image and a first body image.

[0013] Optionally, the expression reasoning result is determined based on audio input by the user.

[0014] Optionally, the method further includes: Get the information to be delivered; generating an action instruction according to the information to be transmitted; According to the action instruction, the third digital image is controlled to display an interactive animation to the user.

[0015] In a second aspect, the present application provides a digital image generation device, comprising: An acquisition module, used for acquiring a first digital image; A segmentation module, used for segmenting each image frame in the image frame sequence corresponding to the first digital image to obtain a first head image and a first body image; An adjustment module, configured to adjust the first head image according to the expression inference result to obtain a second head image; a fusion module, used for fusing the second head image with the first body image to obtain a second digital image; An extraction module, used to extract skeleton information of the first digital image and / or the second digital image; A generation module is used to generate a digital human according to the skeleton information and the two-dimensional image corresponding to the second digital image to obtain a third digital image.

[0016] In a third aspect, the present application provides a computing device, including a memory and a processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as described in any one of the first aspects.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method as described in any one of the first aspects.

[0018] It can be seen from the above technical solution that the present application has at least the following beneficial effects: The present application provides a method for generating a digital image, which can be applied to a processing device. The method includes: the processing device obtains a first digital image, segments each image frame in the image frame sequence corresponding to the first digital image, obtains a first head image and a first body image, and then adjusts the first head image according to the expression reasoning result to obtain a second head image, merges the second head image with the first body image to obtain a second digital image, and the second digital image may have a stitching misalignment problem. Therefore, after obtaining the second digital image, the method extracts the skeleton information of the first digital image and / or the second digital image, and then uses the skeleton information and the two-dimensional image corresponding to the second digital image to generate a digital human again to obtain a third digital image. In the process of generating a secondary digital human, the processing device can adjust the skeleton of the second digital image that may be misaligned based on the skeleton information, thereby weakening the degree of misalignment, thereby reducing the stitching misalignment problem of the digital image, improving the realism of the digital image, and improving the interactive experience.

[0019] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of features or beneficial effects means that specific technical features, technical solutions or beneficial effects are included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be realized without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in a specific embodiment that does not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for generating a digital image provided in an embodiment of the present application; Figure 2 A schematic diagram of key points of a human body provided in an embodiment of the present application; Figure 3 An architectural diagram of a dislocation repair method provided in an embodiment of the present application; Figure 4 A schematic diagram of a digital image generation device provided in an embodiment of the present application; Figure 5 A schematic diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The terms "first", "second", "third", etc. in the specification of this application and the accompanying drawings are used to distinguish different objects rather than to limit a specific order.

[0022] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0023] In order to make the description of the following embodiments clear and concise, a brief introduction to the related technology is first given: At present, when constructing digital image solutions, generative models are usually limited to driving head expressions. When it comes to the need for a complete digital image (including body movements), these solutions often take the approach of directly splicing the head and body models. However, this has a significant flaw: due to the rigid splicing, the splicing is prone to misalignment, which not only destroys the overall visual coherence of the digital image, but also causes the digital image to move unnaturally and have poor fluency when driven, thereby reducing the realism of the digital image and the interactive experience.

[0024] In view of this, the embodiment of the present application provides a method for generating a digital image, which can be executed by a processing device, which can be a terminal or a vehicle controller (such as a vehicle terminal). The terminal includes but is not limited to a smart phone, a tablet computer, a laptop computer, a personal digital assistant or a smart wearable device.

[0025] This method can extract the skeleton information of the digital image, and then use the extracted skeleton information and the two-dimensional image corresponding to the dislocated digital image to regenerate the digital human. In the process of regeneration, the skeleton information can be used to weaken the dislocation, thereby reducing the problem of dislocation of the digital image, improving the realism of the digital image, and improving the interactive experience.

[0026] When the method is applied to a processing device, it is specifically implemented in the form of a computer program. In some embodiments, the computer program may be independent, for example, an independent application program with corresponding functions (such as a client of an application program). In other embodiments, the computer program may be a functional module or a plug-in, etc., which is attached to an existing application program and runs.

[0027] In order to make the technical solution of the present application clearer and easier to understand, the technical solution provided by the embodiment of the present application is introduced below in conjunction with the accompanying drawings. Figure 1As shown in FIG. 1 , the figure is a flow chart of a method for generating a digital image provided by an embodiment of the present application, the method comprising: S101. A processing device obtains a first digital image.

[0028] The processing device can display an interactive interface to the user, and the user can select a photo in the interactive interface, or obtain a photo by taking a photo, and then the processing device can generate a first digital image based on the photo input by the user. The first digital image refers to a virtual image corresponding to the person in the photo input by the user, and the virtual image is similar to the person in the photo input by the user. The processing device can process the above photo based on a pre-trained virtual image generation model, and then obtain the corresponding first digital image. The digital image refers to an image frame sequence or video, that is, composed of multiple image frames, and the first digital image, the second digital image, the third digital image, the first head image, the second head image and the first body image are all composed of image frame sequences, and the difference is that the image frames in the image frame sequence are different.

[0029] In other embodiments, the first digital image may also be generated by other devices, and then the first digital image is transferred to the processing device by the other devices.

[0030] The present application does not specifically limit the method for obtaining the first digital image, and those skilled in the art may select the method for obtaining the first digital image based on actual needs.

[0031] S102: The processing device segments each image frame in the image frame sequence corresponding to the first digital image to obtain a first head image and a first body image.

[0032] After the processing device obtains the first digital image, it can segment each image frame in the image frame sequence corresponding to the first digital image, thereby obtaining a first head image and a first body image. The first head image refers to the image of the head in the first digital image, and the first body image refers to the image of the body part (part other than the head) in the first digital image.

[0033] In some embodiments, the processing device may first perform edge detection on each image frame in an image frame sequence corresponding to the first digital image to obtain an edge detection result, perform face recognition on the first digital image to obtain a face recognition result, and then determine a segmentation boundary based on first boundary information in the edge detection result and second boundary information in the face recognition result, and finally segment the first digital image based on the segmentation boundary to obtain a first head image and a first body image.

[0034] The processing device may use an edge detection algorithm to process the first digital image, thereby obtaining an edge detection result, and the edge detection result includes boundary information of each part of the first digital image (first boundary information). The processing device may use a face recognition algorithm to process the first digital image, thereby obtaining a face recognition result, and the face recognition result includes position information of the face part in the first digital image, and then determine the boundary information of the head (second boundary information) based on the position information of the face part, for example, determining the outer contour of the face part as the boundary information of the head.

[0035] Then, the processing device may compare the first boundary information with the second boundary information, and find target boundary information similar to the second boundary information from the first boundary information. Exemplarily, the first boundary information and the second boundary information may be represented in the form of multiple coordinates. For example, the first boundary information includes coordinates A1, A2, A3, ..., A4, A5, A6, A7, A8, A9, A10, A21, A3, A42, A53, A54, A65, A76, A8, A9, A10, A11, A21, A3, A43, A55, A12, A13, A143, A156, A167, A17, A18, A19, A20, A21, A3, A44, A57, A19, A20, A3 n The second boundary information includes coordinates B1, B2, B3, ..., B m , where n>m. Since the first boundary information refers to the boundaries of each part in the first digital image, and the second boundary information refers to the boundary of the head in the first digital image, the number of coordinates in the first boundary information is greater than the number of coordinates in the second boundary information.

[0036] Next, the processing device may extract m consecutive coordinate information from the first boundary information in sequence, and then vectorize the m consecutive coordinate information to obtain a boundary vector. For example, the processing device may vectorize the coordinate A in the first boundary information. i ~Coordinate A i+m-1 Vectorization is performed to obtain the i-th first boundary vector. When i=1, the processing device processes the coordinates A1~A m Vectorization is performed to obtain the first first boundary vector. The maximum value of i is n. In some examples, when i=n-m+1, the processing device processes the coordinate A i ~Coordinate A n Vectorization is performed. When i=n-m+2, the processing device is processing coordinate A i ~Coordinate A n And coordinate A1 is vectorized. When i=n, ​​the processing device processes coordinate A n And coordinate A1~coordinate A m-1 Vectorization is performed, that is, when i is different, the coordinates are sorted differently.

[0037] The processing device can vectorize the continuous coordinates in the second boundary information to obtain the second boundary vector. Then the processing device calculates the similarity between the second boundary vector and each first boundary vector, obtains the kth first boundary vector with the highest similarity, and then converts the coordinate A corresponding to the kth first boundary vector into k ~Coordinate A k+m-1 Respectively with coordinates B1, B2, B3...B m Find the average and get the target boundary information. For example, the target boundary information includes coordinates C1, C2, C3, ..., C m Among them, the coordinate value of the first dimension of coordinate C1 is coordinate A k The coordinate value of the first dimension of is the average value of the coordinate value of the first dimension of coordinate B1. The subsequent coordinates are similar and will not be repeated here.

[0038] After obtaining the target boundary information, the processing device can determine a segmentation boundary according to the target boundary information, and then segment the first digital image according to the segmentation boundary to obtain a first head image and a first body image.

[0039] Among them, the processing device can obtain more accurate boundaries by determining the segmentation boundaries in the above manner, and can further better segment the head and body parts of the first digital image, thereby facilitating subsequent readjustment after splicing.

[0040] S103: The processing device adjusts the first head image according to the expression inference result to obtain a second head image.

[0041] In some embodiments, the expression reasoning result can be determined based on the audio input by the user. Exemplarily, the processing device can pre-collect sample data, which includes historical audio and historical expression labels (such as happiness, sadness, anger, surprise, etc.) corresponding to the historical audio, and pre-process the collected sample data, for example, unify the audio sampling rate, remove background noise in the audio, etc.

[0042] After preprocessing the data, the processing device can use the processed sample data to perform model training to obtain an expression reasoning model. After receiving the audio input by the user, the subsequent processing device can obtain the expression reasoning result based on the expression reasoning model.

[0043] After the processing device obtains the expression inference result, it can adjust the first head image based on the expression inference result. For example, it can adjust the facial expressions, lips and other information in the first head image to obtain a head image that is more consistent with the audio input by the user, that is, the second head image.

[0044] S104: The processing device merges the second head image with the first body image to obtain a second digital image.

[0045] After the processing device obtains the second head image, since the second head image incorporates the expression information, the second head image is more vivid and real. Then the processing device can merge the second head image with the first body image to obtain the second digital image.

[0046] Among them, the processing device can use a hard splicing method to merge the first body image and the second head image, and then correct the misalignment problem caused by the hard splicing.

[0047] Exemplarily, the processing device may splice the first body image and the second head image in the following manner:

[0048] Indicates the second digital image, represents the stitching model for the head image and the body image, Indicates the first head image, Denotes the first body image. The stitching model for the head image and the body image can be obtained through training.

[0049] S105: The processing device extracts skeleton information of the first digital image and / or the second digital image.

[0050] In some embodiments, the processing device may extract key points from the two-dimensional image corresponding to the second digital image, thereby obtaining key point information of multiple parts corresponding to the second digital image, such as Figure 2 As shown in FIG. 1 , the figure is a schematic diagram of a human body key point provided by an embodiment of the present application. The processing device can generate skeleton information based on the multiple partial key point information.

[0051] In other embodiments, the processing device may obtain two-dimensional images corresponding to the second digital image at different viewing angles, each viewing angle corresponding to a set of part key point information, and then the processing device may obtain multiple sets of part key point information at multiple viewing angles. In some examples, the processing device may obtain four sets of part key point information at four viewing angles (front viewing angle, left viewing angle, right viewing angle, and rear viewing angle). The processing device then determines the skeleton information of the second digital image based on the four sets of part key point information.

[0052] Exemplarily, the processing device can establish a mapping relationship between the key point information of the parts and the skeleton information under different viewing angles. For example, in the above example, the processing device can establish four sets of mapping relationships, and then obtain four sets of skeleton information through the four sets of key point information of the parts and their corresponding mapping relationships, and then average the four sets of skeleton information respectively to obtain the final skeleton information. Determining the skeleton information of the second digital image through multiple sets of key point information of the parts can improve the accuracy of the skeleton information, so that the dislocation problem can be reduced more efficiently in the subsequent re-generation process of the digital human.

[0053] The processing device may also extract key points from the two-dimensional image corresponding to the second digital image to obtain key point information of multiple parts.

[0054] S106: The processing device generates a digital human according to the skeleton information and the two-dimensional image corresponding to the second digital image to obtain a third digital image.

[0055] After the processing device obtains the skeleton information of the first digital image and / or the second digital image, it can generate a digital human based on the skeleton information of the first digital image and / or the second digital image and the two-dimensional image corresponding to the second digital image to obtain a third digital image.

[0056] The two-dimensional image corresponding to the second digital image may be an image from a frontal perspective. In other embodiments, after obtaining the skeleton information of the first digital image and / or the second digital image, the processing device may also generate a digital human based on the skeleton information of the first digital image and / or the second digital image and the two-dimensional images from multiple perspectives corresponding to the second digital image to obtain a third digital image. Multiple perspectives have been introduced in the aforementioned embodiments and will not be described again here.

[0057] Exemplarily, the processing device may obtain the third digital image in the following manner:

[0058] Indicates the third digital image, Indicates that the parameter is Generate a digital human model, Indicates that the parameter is Digital human control model, represents a two-dimensional image corresponding to the second digital image, Represents key point information of multiple parts of the second digital image.

[0059] In other embodiments, when the processing device uses the two-dimensional images under multiple viewing angles corresponding to the second digital image, the third digital image can be obtained by:

[0060] in, represents a two-dimensional image from all viewing angles, represents the two-dimensional image at the first viewing angle, Represents the two-dimensional image at the qth viewing angle.

[0061] The digital human control model and the digital human generation model can be obtained through pre-training, and the parameters of the digital human generation model and the parameters of the digital human control model can also be obtained through pre-training.

[0062] In some embodiments, the digital human control model and the digital human generation model can be jointly trained. The sample data used in the training process can be sample dislocation images, sample skeleton information, and sample non-dislocation images, wherein the sample non-dislocation images are non-dislocation images corresponding to the sample dislocation images, that is, images corresponding to the human figures in the sample dislocation images when they are not dislocated.

[0063] like Figure 3 As shown, the figure is an architectural diagram of a dislocation repair provided by an embodiment of the present application. Among them, the digital human generation model 300 includes an encoder, a decoder and a hidden space, and the digital human control model 400 includes an encoder and a zero convolution. The processing device can input the two-dimensional image corresponding to the second digital image into the digital human generation model 300, and the processing device can also use a key point extraction algorithm to extract key points of the second digital image to obtain part key point information, and then input the part key point information into the digital human control model 400. The digital human control model 400 outputs the skeleton information of the first digital image and / or the second digital image to the digital human generation model 300. Then, the digital human generation model 300 can regenerate the digital human based on the two-dimensional image and the skeleton information, and then obtain the third digital image, so as to repair the dislocated second digital image.

[0064] It can be seen that after obtaining the second digital image, the method extracts the skeleton information of the first digital image and / or the second digital image, and then uses the skeleton information and the two-dimensional image corresponding to the second digital image to generate a digital human again to obtain a third digital image. During the secondary digital human generation process, the processing device can adjust the skeleton of the second digital image that may be misaligned based on the skeleton information, thereby reducing the degree of misalignment, thereby reducing the problem of stitching misalignment of the digital image, improving the realism of the digital image, and improving the interactive experience.

[0065] In some embodiments, the processing device may also obtain information to be transmitted, and then generate action instructions according to the information to be transmitted, and control the third digital image according to the action instructions to display an interactive animation to the user.

[0066] Among them, the information to be transmitted may refer to information to be displayed to the user, such as the broadcast information of the road condition ahead. The processing device can respond to the user's voice control to obtain the information to be transmitted. For example, the user asks "How is the road condition ahead?" The processing device can obtain the information of the road condition ahead, and generate the broadcast information of the road condition ahead based on the information of the road condition ahead. Action instructions refer to instructions for controlling the third digital image. The processing device can identify the information to be transmitted, so as to determine the action instructions to be generated in the next step. Exemplarily, a mapping relationship between different information to be transmitted and different instructions can be established in advance, and the action instructions to be generated are subsequently determined based on the pre-established mapping relationship. Interactive animation refers to the animation of the third digital image during the control of the third digital image. The method for generating a digital image in the embodiment of the present application can make the generated digital image more realistic, thereby improving the user's interactive experience.

[0067] Combination of the above Figure 1 The method for generating a digital image provided in the embodiment of the present application is introduced in detail. The device and equipment provided in the embodiment of the present application will be introduced in conjunction with the accompanying drawings.

[0068] like Figure 4 As shown in FIG. 1 , the figure is a schematic diagram of a digital image generation device provided in an embodiment of the present application, the device comprising: An acquisition module 401 is used to acquire a first digital image; A segmentation module 402 is used to segment each image frame in the image frame sequence corresponding to the first digital image to obtain a first head image and a first body image; An adjustment module 403, configured to adjust the first head image according to the expression inference result to obtain a second head image; A fusion module 404, configured to fuse the second head image with the first body image to obtain a second digital image; An extraction module 405, used to extract skeleton information of the first digital image and / or the second digital image; The generation module 406 is used to generate a digital human according to the skeleton information and the two-dimensional image corresponding to the second digital image to obtain a third digital image.

[0069] Optionally, the extraction module 405 is specifically used to extract key points from the 2D image corresponding to the second digital image to obtain multiple parts of key point information; and generate skeleton information according to the multiple parts of key point information.

[0070] Optionally, the generating module 406 is specifically configured to generate the third digital image in the following manner:

[0071] Indicates the third digital image, Indicates that the parameter is Generate a digital human model, Indicates that the parameter is Digital human control model, represents a two-dimensional image corresponding to the second digital image, Represents key point information of multiple parts of the second digital image.

[0072] Optionally, the fusion module 404 is specifically configured to generate the second digital image in the following manner:

[0073] Indicates the second digital image, represents the stitching model for the head image and the body image, Indicates the first head image, Indicates the first body image.

[0074] Optionally, the segmentation module 402 is specifically used to perform edge detection on the first digital image to obtain an edge detection result, perform face recognition on the first digital image to obtain a face recognition result; determine a segmentation boundary based on first boundary information in the edge detection result and second boundary information in the face recognition result; and segment the first digital image based on the segmentation boundary to obtain a first head image and a first body image.

[0075] Optionally, the expression reasoning result is determined based on audio input by the user.

[0076] Optionally, the device also includes a display module, and the acquisition module 401 is also used to obtain information to be transmitted; the generation module 406 is also used to generate action instructions according to the information to be transmitted; the display module is used to control the third digital image according to the action instructions to display an interactive animation to the user.

[0077] The digital image generation device according to the embodiment of the present application may correspond to the method described in the embodiment of the present application, and the above-mentioned other operations and / or functions of each module / unit of the digital image generation device are respectively to realize Figure 1 For the sake of brevity, the corresponding processes of each method in the illustrated embodiment are not described in detail here.

[0078] The present application also provides a computing device. Figure 5As shown, the figure is a schematic diagram of a computing device provided in an embodiment of the present application, and the computing device 500 includes a bus 501, a processor 502, a communication interface 503 and a memory 504. The processor 502, the memory 504 and the communication interface 503 communicate with each other through the bus 501.

[0079] The bus 501 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0080] The processor 502 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0081] The communication interface 503 is used for communicating with the outside.

[0082] The memory 504 may include a volatile memory, such as a random access memory (RAM). The memory 504 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0083] The memory 504 stores executable codes, and the processor 502 executes the executable codes to perform the aforementioned method for generating a digital image.

[0084] Specifically, in implementing Figure 4 In the case of the embodiment shown, and Figure 4 When each module or unit of the digital image generation device described in the embodiment is implemented by software, the execution Figure 4The software or program code required for the functions of each module / unit in the system may be partially or completely stored in the memory 504. The processor 502 executes the program code corresponding to each unit stored in the memory 504 to execute the aforementioned method for generating a digital image.

[0085] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned digital image generation method.

[0086] The embodiment of the present application further provides a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part.

[0087] The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0088] When the computer program product is executed by a computer, the computer executes any of the aforementioned methods for generating a digital image. The computer program product may be a software installation package, and when any of the aforementioned methods for generating a digital image is required, the computer program product may be downloaded and executed on the computer.

[0089] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0090] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A method for generating a digital image, characterized in that: The method comprises: obtaining a first digital image; Segmenting each image frame in the image frame sequence corresponding to the first digital image to obtain a first head image and a first body image; According to the expression inference result, adjusting the first head image to obtain a second head image; Merging the second head image with the first body image to obtain a second digital image; Extracting skeleton information of the first digital image and / or the second digital image; A digital human is generated based on the skeleton information and the two-dimensional image corresponding to the second digital image to obtain a third digital image.

2. The method according to claim 1, characterized in that The step of extracting skeleton information of the second digital image comprises: Extracting key points from the 2D image corresponding to the first digital image to obtain key point information of multiple parts; or extracting key points from the 2D image corresponding to the second digital image to obtain key point information of multiple parts; Skeleton information is generated according to the plurality of part key point information.

3. The method according to claim 2, characterized in that The step of generating a digital human according to the skeleton information and the two-dimensional image corresponding to the second digital image to obtain a third digital image includes: Indicates the third digital image, Indicates that the parameter is Generate a digital human model. Indicates that the parameter is Digital human control model, represents a two-dimensional image corresponding to the second digital image, Represents key point information of multiple parts of the second digital image.

4. The method according to claim 1, characterized in that: The step of fusing the second head image with the first body image to obtain a second digital image includes: Indicates the second digital image, represents the stitching model for the head image and the body image, Indicates the first head image, Indicates the first body image.

5. The method according to claim 1, characterized in that The step of segmenting each image frame in the image frame sequence corresponding to the first digital image to obtain a first head image and a first body image includes: Performing edge detection on each image frame in the image frame sequence corresponding to the first digital image to obtain an edge detection result, and performing face recognition on the first digital image to obtain a face recognition result; Determining a segmentation boundary according to first boundary information in the edge detection result and second boundary information in the face recognition result; The first digital image is segmented according to the segmentation boundary to obtain a first head image and a first body image.

6. The method according to claim 1, characterized in that The expression inference result is determined based on audio input by the user.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Get the information to be delivered; generating an action instruction according to the information to be transmitted; According to the action instruction, the third digital image is controlled to display an interactive animation to the user.

8. A digital image generation device, characterized in that: The device comprises: An acquisition module, used for acquiring a first digital image; A segmentation module, used for segmenting each image frame in the image frame sequence corresponding to the first digital image to obtain a first head image and a first body image; An adjustment module, configured to adjust the first head image according to the expression inference result to obtain a second head image; a fusion module, used for fusing the second head image with the first body image to obtain a second digital image; An extraction module, used to extract skeleton information of the first digital image and / or the second digital image; A generation module is used to generate a digital human according to the skeleton information and the two-dimensional image corresponding to the second digital image to obtain a third digital image.

9. A computing device, characterized in that including memory and processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

Images