Image generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an image generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining at least two original images, each original image at least comprises two target objects, and the target objects contained in the original images are the same; extracting at least two types of original areas of each target object in each original image; evaluating each original region to obtain a first evaluation result, and evaluating the matching degree of different types of original regions to obtain a second evaluation result; according to the first evaluation result and the second evaluation result, determining different types of target areas matched with the target objects from the original areas; and generating updated target objects according to the different types of target areas matched with the target objects, and generating a target image according to the updated target objects. A group photo with a better overall state can be automatically generated, and the user experience is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technologies, and in particular, to an image generation method, apparatus, electronic device, and storage medium. Background Art

[0002] In the case of group photos, it is often necessary to take multiple photos and confirm to obtain a group photo in which everyone is in the best state. In the prior art, through an image editing software, the people in the best state in different photos can be cut out and merged in response to the image editing operations input by the user to generate a group photo. The existing methods consume a lot of manpower and time, and the user experience is poor. Summary of the Invention

[0003] Embodiments of the present disclosure provide an image generation method, apparatus, electronic device, and storage medium, which can automatically generate a group photo with a better overall state and improve the user experience.

[0004] In a first aspect, an image generation method provided by an embodiment of the present disclosure includes:

[0005] Obtaining at least two original images, where each of the original images includes at least two target objects, and the target objects included in each of the original images are the same;

[0006] Extracting at least two types of original regions of each of the target objects in each of the original images;

[0007] Evaluating each of the original regions to obtain a first evaluation result, and evaluating the matching degree between the original regions of different types to obtain a second evaluation result;

[0008] Determining, according to the first evaluation result and the second evaluation result, different types of target regions that match each of the target objects from each of the original regions;

[0009] Generating updated ones of the target objects according to the different types of target regions that match each of the target objects, and generating a target image according to the updated target objects.

[0010] In a second aspect, an image generation apparatus provided by an embodiment of the present disclosure further includes:

[0011] An image acquisition module, configured to obtain at least two original images, where each of the original images includes at least two target objects, and the target objects included in each of the original images are the same;

[0012] A region extraction module, configured to extract at least two types of original regions of each of the target objects in each of the original images;

[0013] An evaluation module for evaluating each of the original regions to obtain a first evaluation result, and for evaluating the matching degree between the original regions of different types to obtain a second evaluation result;

[0014] A matching module for determining, according to the first evaluation result and the second evaluation result, different types of target regions that match each of the target objects from each of the original regions;

[0015] A generation module for generating updated target objects according to the different types of target regions that match each of the target objects, and generating a target image according to the updated target objects.

[0016] In a third aspect, an embodiment of the present disclosure further provides an electronic device, where the electronic device includes:

[0017] One or more processors;

[0018] A storage device for storing one or more programs,

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the image generation method according to any one of the embodiments of the present disclosure.

[0020] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, where the computer-executable instructions are used to execute the image generation method according to any one of the embodiments of the present disclosure when executed by a computer processor.

[0021] The technical solution of the embodiment of the present disclosure is to obtain at least two original images, where each original image contains at least two target objects, and the target objects included in each original image are the same; extract at least two types of original regions of each target object in each original image; evaluate each of the original regions to obtain a first evaluation result, and evaluate the matching degree between the original regions of different types to obtain a second evaluation result; determine, according to the first evaluation result and the second evaluation result, different types of target regions that match each target object from each of the original regions; generate updated target objects according to the different types of target regions that match each target object, and generate a target image according to the updated target objects.

[0022] By separately evaluating at least two types of original regions of each target object in multiple original images, and evaluating the matching degree between original regions of different types, not only can the best-performing original region of each type be determined, but also the combination of the best-performing original regions of different types can be determined. Thus, based on the results of the separate evaluation and the matching degree evaluation, the updated target objects generated can present an overall optimal state, and thus the target image composed of the updated target objects can also present an optimal state. When the technical solution of the embodiments of the present disclosure is applied to the case of group photos of multiple people, it is possible to automatically generate a group photo with a better overall state according to multiple group photos, which can improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn to scale.

[0024] Figure 1 It is a flowchart of an image generation method provided by an embodiment of the present disclosure;

[0025] Figure 2 It is a flowchart of an image generation method provided by an embodiment of the present disclosure;

[0026] Figure 3 It is a flowchart block diagram of an image generation method provided by an embodiment of the present disclosure;

[0027] Figure 4 It is a schematic structural diagram of an image generation device provided by an embodiment of the present disclosure;

[0028] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0030] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0031] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the concepts such as "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "a plurality" mentioned in this disclosure are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise clearly specified in the context, it should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of this disclosure are only for illustrative purposes, and are not used to limit the scope of these messages or information.

[0035] It can be understood that before using the technical solutions disclosed in the embodiments of this disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in this disclosure should be informed to users and the authorization of users should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0036] Figure 1 A schematic flowchart of an image generation method provided for the embodiments of this disclosure. The embodiments of this disclosure are applicable to the situation of generating a composite image based on multiple images, for example, applicable to the situation of generating the best group photo based on a group photo of multiple people. This method can be executed by an image generation device, which can be implemented in the form of software and / or hardware, and the device can be configured in an electronic device, such as configured in a mobile phone or a computer.

[0037] As Figure 1 shown, the image generation method provided in this embodiment may include:

[0038] S110. Obtain at least two original images, where each original image contains at least two target objects, and the target objects included in each original image are the same.

[0039] In the embodiments of the present disclosure, the target object may include an object and / or a biological object. Different categories of regions may be pre-divided in the target object, and the presentation effects of different categories of regions of each target object in different original images may vary. Based on the image generation method provided by the embodiments of the present disclosure, it is intended to generate a target image in which each region of each target object is presented in a good state according to the presentation situations of different regions of each target object in multiple original images.

[0040] Among them, each original image includes at least two target objects, and the target objects included in each original image are the same. Exemplarily, assuming that image 1 includes target object 1 and target object 2, and image 2 also includes target object 1 and target object 2, then image 1 and image 2 can be used as the original images. Each of these two original images contains two target objects, and the target objects included in image 1 and image 2 are the same, namely target object 1 and target object 2.

[0041] Among them, the method of obtaining at least two original images may include at least one of the following: screening each video frame in the video data to obtain at least two original images including at least two identical target objects; continuously collecting images of the same scene to obtain at least two original images including at least two identical target objects. By obtaining the original images including at least two identical target objects through the method of screening based on video data or continuous image collection, it is possible to ensure the scene similarity among the original images, which is beneficial to synthesizing a target image with a very natural presentation effect.

[0042] In addition, at least two original images may also be obtained based on other methods, where each original image includes at least two target objects, and the target objects included in each original image are the same, and no exhaustive listing is made here.

[0043] S120. Extract at least two types of original regions of each target object in each original image.

[0044] In the embodiments of the present disclosure, at least two types of original regions may be set in advance for different types of target objects. Among them, the at least two types of original regions may constitute a complete target object or a part of the target object. Exemplarily, assuming that the target object is a human body object, in the case where the at least two types of original regions include a facial region and a region other than the facial region, the two types of original regions can constitute a complete human body object; in the case where the at least two types of original regions include a facial region and a torso region, the human body object also includes other regions (such as limb regions), and at this time, the facial region and the torso region can constitute a part of the human body object.

[0045] In the embodiments of the present disclosure, based on existing image segmentation methods (such as semantic segmentation deep learning models for different category regions, etc.), at least two types of original regions of each target object in each original image can be extracted.

[0046] S130. Evaluate each original region to obtain a first evaluation result, and evaluate the matching degree between original regions of different types to obtain a second evaluation result.

[0047] In the embodiments of the present disclosure, based on existing deep learning models for aesthetic evaluation, aesthetic scores can be given to each original region to obtain a first evaluation result. In addition, scores input by users for each original region can also be received, and based on the aesthetic scores output by the model and the scores input by users, the first evaluation result can be determined. For example, the aesthetic scores output by the model and the scores input by users can be weighted to obtain the first evaluation result, etc.

[0048] In the embodiments of the present disclosure, evaluating the matching degree between original regions of different types may include: evaluating the matching degree based on at least one preset dimension such as the size, color, and pose of the original regions of different types.

[0049] In some alternative implementation manners, evaluating the matching degree between original regions of different types may include: evaluating the matching degree between original regions of different types according to the normal information of the original regions of different types.

[0050] In these alternative implementation manners, the normal information of each original region can be determined according to existing algorithms for predicting normal information based on images; and the matching degree between original regions of different types can be evaluated according to the normal information of the original regions of different types. For example, when the included angle between the normals of the original regions of different types satisfies a preset angle range, it can be considered that the matching degree between the original regions of different types is relatively high. By evaluating the matching degree of original regions of different types based on normal information, the matching degree of original regions of different types in the pose dimension can be determined.

[0051] In addition, in some implementation manners, at least two types of original regions can be preset with a first type of region that can be used to identify a target object, and other types of regions in the at least two types of original regions except the first type of region can be used as a second type of region. Correspondingly, evaluating the matching degrees between different types of original regions can include: evaluating the matching degrees between the first type of region of each target object and the second type of region of the same target object and the second type of region of other target objects respectively. In these implementation manners, by only evaluating the matching between the first type of region identifying the target object and each second type of region, it is possible to avoid evaluating the matching degrees between each type of original regions, and the matching efficiency can be improved to a certain extent.

[0052] S140. Determine different types of target regions that match each target object from each original region according to the first evaluation result and the second evaluation result.

[0053] In this embodiment, according to the first evaluation result, each original region in the better state of each target object can be determined; according to the second evaluation result, each original region in the better combination can be determined, where the original regions in the better combination can be from the same target object or from different target objects. Among them, mathematical model modeling can be performed in advance to determine different types of target regions of each target object according to the first evaluation result and the second evaluation result through the constructed data model, so that the presentation of a single type of target region of each object is in a better state, and at the same time, the combination of each target region presents a better state.

[0054] In addition, in the case where the first type of region and the second type of region are preset, determining different types of target regions that match each target object from each original region according to the first evaluation result and the second evaluation result can include: First, according to the first evaluation result, determine the first type of region with the highest score of each target object and use it as the first target region of each target object; then, according to the second evaluation result, determine the second type of regions that match the first target region of each target object and use them as the second target regions of each target object; among them, the first target region and the second target region both belong to the target regions. Thus, it is possible to retain the original regions identifying the target object on the target object, and the original regions of other types can be replaced by the original regions of this type of other target objects, so as to achieve that each target object can present an overall better state.

[0055] S150. Generate updated target objects according to different types of target regions that match each target object, and generate a target image according to the updated target objects.

[0056] In the embodiments of the present disclosure, when at least two types of original regions can form a complete target object, different types of target regions matched to each target object can be spliced and fused to obtain each updated target object; when at least two types of original regions can form a part of the target object, different types of target regions matched to each target object can be fused into other regions of the target object except the original regions to obtain each updated target object. Then, each updated target object can be fused based on an existing image fusion algorithm (such as an image fusion network model) to generate a target image.

[0057] In the technical solution of the embodiments of the present disclosure, at least two original images are obtained, where each original image contains at least two target objects and the target objects included in each original image are the same; at least two types of original regions of each target object in each original image are extracted; each original region is evaluated to obtain a first evaluation result, and the matching degree between different types of original regions is evaluated to obtain a second evaluation result; according to the first evaluation result and the second evaluation result, different types of target regions matched to each target object are determined from each original region; according to the different types of target regions matched to each target object, each updated target object is generated, and a target image is generated according to each updated target object.

[0058] By separately evaluating at least two types of original regions of each target object in multiple original images and evaluating the matching degree between different types of original regions, not only can the best-performing original region of each type be determined, but also the combination of the best-performing original regions of different types can be determined. Thus, based on the results of the separate evaluation and the matching degree evaluation, each generated updated target object can present an overall better state, and thus the target image composed of each updated target object can also present a better state. When the technical solution of the embodiments of the present disclosure is applied to the case of group photos of multiple people, it is possible to automatically generate a group photo with a better overall state according to multiple group photos, which can improve the user experience.

[0059] The embodiments of the present disclosure can be combined with each optional solution in the image generation method provided in the above embodiments. The image generation method provided in this embodiment optimizes the generation process of the target image. When the target object includes a human object, by providing a candidate clothing model, it is possible to change the clothing for the human object. In addition, by providing a candidate background region, it is possible to switch the scene. Thus, it is possible to freely switch the target image to different clothing and different scenes, which can further improve the user experience.

[0060] The image generation method provided in this embodiment, when the target object includes a human object, may further include: obtaining candidate clothing models; wherein, the candidate clothing models include three-dimensional models of clothing constructed based on each original image; in response to a clothing editing operation, determining target clothing models that match each human object from the candidate clothing models, and rendering the target clothing models onto the corresponding updated human objects.

[0061] In this embodiment, based on the existing method of generating three-dimensional models from images, clothing models can be constructed according to the images of each clothing area in the original images; and the constructed clothing models and clothing models of various styles in the preset clothing library can be used as candidate clothing models together. Among them, the candidate clothing models may include clothing models and / or accessory models. Among them, the accessory module may include, but is not limited to, models such as headgear, earrings, glasses, ties, etc.

[0062] Among them, the clothing editing operation may include operations of at least one step. For example, it may include a clothing selection operation and a clothing rendering operation, etc. For example, according to the clothing selection operation input by the user, target clothing models that match each human object can be determined from the candidate clothing models; or, alternatively, according to the characteristics of the updated human objects (such as hairstyle, face shape, body shape, etc.), the matching target clothing models can be automatically selected. Another example is that according to the clothing rendering operation, information such as the size and pose of each human object can be detected, and the clothing models can be adjusted in terms of size, distortion angle, etc. according to the detected information, and the adjusted models are rendered onto the corresponding updated human objects. Thus, clothing change for human objects can be realized, and the user experience can be improved.

[0063] The image generation method provided in this embodiment may further include: obtaining candidate background regions; wherein, the candidate background regions include the original background regions extracted from each original image; in response to a background selection operation, selecting a target background region from the candidate background regions; correspondingly, in the process of generating the target image, it may further include: generating the target image according to the target background region.

[0064] In this embodiment, similarly, based on the existing image segmentation method, the original background regions in each original image can be extracted; the extracted original background regions and the background regions of various scenes in the preset background library can be used as candidate background regions together. Among them, the target background region can be selected from the candidate background regions according to the background region selection operation input by the user. And the target background region can be fused with each updated target object to obtain the target image. Thus, scene switching can be realized, and the user experience can be further improved.

[0065] The technical solution of the embodiment of the present disclosure optimizes the generation process of the target image. When the target object includes a human object, by providing a candidate clothing model, clothing change for the human object can be realized. In addition, by providing a candidate background area, scene switching can be realized. Thus, the target image can be freely switched to different clothing and different scenes, which can further improve the user experience. The image generation method provided by the embodiment of the present disclosure and the image generation method provided by the above embodiment belong to the same general inventive concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.

[0066] The various alternative solutions in the image generation methods provided in the embodiments of the present disclosure and the above embodiments can be combined. The image generation method provided in this embodiment details the pre-processing steps of the original area and the post-processing steps of the updated target object. By analyzing the clarity and / or similarity of the original area, the original areas with high clarity and / or different performances can be screened out. By performing addition, deletion, and modification on the original area, the target object in the target image can be added, deleted, or changed. By adjusting the brightness of the updated target object, the brightness of each area of the updated target object can be made consistent, making the updated target object more realistic and natural.

[0067] Figure 2 It is a schematic flow chart of an image generation method provided by an embodiment of the present disclosure. As Figure 2 shown, the image generation method provided in this embodiment may include:

[0068] S201. Obtain at least two original images, where each original image contains at least two target objects, and the target objects included in each original image are the same.

[0069] S202. Extract at least two types of original areas of each target object in each original image.

[0070] S203. Perform clarity analysis on each original area, and screen each original area according to the result of the clarity analysis.

[0071] In this embodiment, the clarity analysis of each original area can be performed based on an existing image clarity analysis algorithm (such as a deep learning model), and the original areas with low clarity are removed. By performing clarity on the original area, the original areas with high clarity can be screened out, which can improve the presentation effect of the updated target object.

[0072] S204. Perform similarity analysis on each original area, and screen each original area according to the result of the similarity analysis.

[0073] In this embodiment, similarity analysis can be performed on each original region based on an existing image similarity analysis algorithm (such as a deep learning model), and the original regions with high similarity can be removed. By performing similarity analysis on the original regions, the original regions with different performances can be screened out, which can reduce the computational complexity of determining the matching target regions for each target object and can improve the generation efficiency of the updated target objects to a certain extent.

[0074] In addition, in this embodiment, by only performing similarity analysis on the original regions instead of on the entire target object, the influence of the overall similarity degree of the target object on the similarity degree of the original regions can be avoided, which is convenient for screening out the original regions with different performances.

[0075] S205. Perform at least one of the following operations on each original region: update, delete, and add.

[0076] In this embodiment, a single-object image of the target object can be obtained; among them, the single-object image can be considered as an image that only contains a single target object; among them, the target object may or may not be included in the original image. Furthermore, at least two types of original regions in the single-object image can be extracted. When the target object is included in the original image, the original regions extracted from the single-object image can be used to replace the original regions of the target object extracted from the original image to implement the update process of the original regions. When the target object is not included in the original image, the original regions extracted from the single-object image can be added to the original regions extracted from the original image to implement the addition process of the original regions, so that the target object can be added to the generated target image.

[0077] In addition, each original region extracted from the original image can also be deleted. For example, when a complete target object is composed of at least two types of original regions, the original regions belonging to the same target object can be deleted, so that the target object can be removed from the generated target image.

[0078] In this embodiment, there is no strict timing limit for steps S203 - S205.

[0079] S206. Evaluate each original region to obtain a first evaluation result, and evaluate the matching degree between different types of original regions to obtain a second evaluation result.

[0080] S207. Determine different types of target regions that match each target object from each original region according to the first evaluation result and the second evaluation result.

[0081] S208. Generate updated target objects according to the different types of target regions that match each target object, and generate a target image according to the updated target objects.

[0082] S209. Perform brightness analysis on each updated target object, and perform brightness adjustment on each updated target object according to the brightness analysis result.

[0083] Among them, the brightness of each target area of the target object can be analyzed according to the existing brightness analysis algorithm (such as a deep learning model) to determine the brightness situation of each target area (such as whether there is overexposure or shadow). And according to the brightness analysis result, the brightness of each target area of the target object can be adjusted to make the brightness of each target area consistent, so that the updated target object is more real and natural.

[0084] The technical solution of the embodiment of the present disclosure has described in detail the preprocessing steps of the original area and the postprocessing steps of the updated target object. By analyzing the clarity and / or similarity of the original area, the original areas with high clarity and / or different performances can be screened out. By performing addition, deletion, and modification on the original area, the target objects in the target image can be added, deleted, and changed. By performing brightness adjustment on the updated target object, the brightness of each area of the updated target object can be kept consistent, making the updated target object more real and natural. In addition, the image generation method provided by the embodiment of the present disclosure and the image generation method provided by the above embodiment belong to the same general concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.

[0085] The various alternative solutions in the image generation method provided by the embodiment of the present disclosure and the above embodiment can be combined. The image generation method provided in this embodiment has described in detail the image generation method in the case where the target object is a human object. Based on the image generation method provided in this embodiment, the best group photo can be automatically generated according to multiple group photos, which can improve the user experience.

[0086] Figure 3 It is a flowchart of an image generation method provided by an embodiment of the present disclosure. As Figure 3 shown, the image generation method provided in this embodiment may include:

[0087] S301. Obtain at least two original images, where each original image includes at least two human objects, and the human objects included in each original image are the same.

[0088] In this embodiment, at least two original images of at least two identical human objects are obtained based on methods such as video data screening or continuous image acquisition.

[0089] S302. Extract the original facial regions, original limb regions, original clothing regions, and original background regions of each human object in each original image.

[0090] In this embodiment, an identity document (ID) of each human object can be set, and the original facial region, original limb region, and original clothing region corresponding to each ID can be extracted. Among them, the original facial region can be considered as the head region, and the original limb region can be considered as the region outside the head region. Among them, the original clothing region usually overlaps with the original limb region. Among them, a three-dimensional model of the clothing can be constructed based on the image of the original clothing region as a candidate clothing model. Among them, the extracted original background region can be used as a candidate background region.

[0091] S303. Analyze the clarity of each original facial region, original limb region, and original clothing region, and screen each original facial region, original limb region, and original clothing region according to the results of the clarity analysis.

[0092] In this embodiment, the clarity of each original facial region, original limb region, and original clothing region can be analyzed based on an existing image clarity analysis algorithm (such as a deep learning model), and the regions with low clarity can be removed.

[0093] S304. Analyze the similarity of each original facial region, original limb region, and original clothing region, and screen each original facial region, original limb region, and original clothing region according to the results of the similarity analysis.

[0094] In this embodiment, the similarity of each original facial region, original limb region, and original clothing region can be analyzed based on an existing image similarity analysis algorithm (such as a deep learning model), and the regions with high similarity can be removed.

[0095] S305. Perform at least one of the following operations on each original facial region, original limb region, and original clothing region: update, delete, and add.

[0096] In this embodiment, a single-person image of a human object can be obtained; among them, the human object may or may not be included in the original image. Furthermore, the original facial region, original limb region, and original clothing region in the single-person image can be extracted.

[0097] In the case where a human object is included in the original image, at least one of the original facial region, original limb region, and original clothing region extracted from the single-person image can be used to replace the corresponding region in the original facial region, original limb region, and original clothing region of the target object extracted from the original image, so as to implement the update process of the original facial region, original limb region, and original clothing region. In the case where the target object is not included in the original image, the original facial region, original limb region, and original clothing region extracted from the single-person image can be added to the original facial region, original limb region, and original clothing region extracted from the original image, so as to implement the addition process of the original region, thereby adding a human object to the generated target image and realizing the addition of absent persons based on a group photo.

[0098] In addition, the original facial region, original limb region, and original clothing region extracted from the original image can also be deleted. For example, the original facial region, original limb region, and original clothing region belonging to the same human object can be deleted, so that the human object can be removed from the generated target image.

[0099] In this embodiment, there is no strict timing limit for steps S303 - S305.

[0100] S306. Evaluate each original facial region, original limb region, and original clothing region to obtain a first evaluation result, and evaluate the matching degree of each original facial region, original limb region, and original clothing region to obtain a second evaluation result.

[0101] In this embodiment, evaluating the original facial region may include evaluating whether the eyes are closed, whether the expression is natural, etc.; evaluating the original limb region may include evaluating whether there are afterimages, whether the body posture is good, etc.; evaluating the original clothing region may include whether there are large areas of shadow, overexposure, etc. In addition, other dimensional evaluations of the original facial region, original limb region, and original clothing region can also be included here, and no exhaustive list is made here.

[0102] In this embodiment, the matching degree of each original facial region and original limb region can also be evaluated according to the normal information of each original facial region and original limb region. So that the orientations of the original facial region and the original limb region are kept consistent to a certain extent, making the updated human object more natural.

[0103] In addition, the evaluation scores of each original facial region, original limb region, and original clothing region input by the user, as well as the evaluation scores of the matching degree of each original facial region, original limb region, and original clothing region, can also be received, so that the first evaluation result and the second evaluation result can be adjusted in combination with the user's preferences.

[0104] S307. Determine the target facial regions and target limb regions that match each human object from the original facial regions and original limb regions according to the first evaluation result and the second evaluation result.

[0105] In this embodiment, the original facial regions can be preset as the first category of regions for identifying human objects, and the original limb regions can be used as the second category of regions. Among them, according to the first evaluation result, the first category of regions with the highest scores for each human object can be determined and used as the first target regions for each human object; then, according to the second evaluation result, the second category of regions that match the first target regions of each human object can be determined and used as the second target regions for each human object; among them, the first target regions and the second target regions both belong to the target regions.

[0106] Thus, it is possible to retain the original facial regions identifying human objects on the human objects, while the original limb regions can be replaced by the original limb regions of other human objects to achieve an overall better state for each human object.

[0107] S308. Generate updated human objects according to the target facial regions and target limb regions that match each human object.

[0108] In this embodiment, the target facial regions and target limb regions that match each human object can be fused to obtain each human object.

[0109] S309. Obtain candidate clothing models; among them, the candidate clothing models include three-dimensional models of clothing constructed according to each original image.

[0110] S310. In response to a clothing editing operation, determine the target clothing models that match each human object from the candidate clothing models and render the target clothing models onto the corresponding human objects.

[0111] S311. Perform brightness analysis on the updated human objects and adjust the brightness of the updated human objects according to the brightness analysis results.

[0112] S312. Obtain candidate background regions; among them, the candidate background regions include the original background regions extracted from each original image.

[0113] S313. In response to a background selection operation, select the target background regions from the candidate background regions.

[0114] S314. Generate target images according to the human objects after brightness adjustment and the target background regions.

[0115] Among them, the human objects after brightness adjustment can be fused into the target background regions to generate target images.

[0116] The technical solution of the embodiment of the present disclosure describes in detail an image generation method in the case where the target object is a human object. Based on the image generation method provided in this embodiment, the best group photo can be automatically generated according to multiple group photos, which can improve the user experience. The image generation method provided in the embodiment of the present disclosure and the image generation method provided in the above embodiment belong to the same general concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.

[0117] Figure 4 It is a schematic structural diagram of an image generation device provided by an embodiment of the present disclosure. The image generation device provided in this embodiment is applicable to the situation of generating a composite image based on multiple images, for example, applicable to the situation of generating the best group photo based on a group photo of multiple people.

[0118] As Figure 4 shown, the image generation device provided by the embodiment of the present disclosure may include:

[0119] An image acquisition module 410, configured to acquire at least two original images, where at least two target objects are included in each original image, and the target objects included in each original image are the same;

[0120] A region extraction module 420, configured to extract at least two types of original regions of each target object in each original image;

[0121] An evaluation module 430, configured to evaluate each original region to obtain a first evaluation result, and evaluate the matching degree between the original regions of different types to obtain a second evaluation result;

[0122] A matching module 440, configured to determine different types of target regions matching each target object from each original region according to the first evaluation result and the second evaluation result;

[0123] A generation module 450, configured to generate updated target objects according to different types of target regions matching each target object, and generate a target image according to the updated target objects.

[0124] In some optional implementation manners, the evaluation module may be configured to:

[0125] Evaluate the matching degree between the original regions of different types according to the normal information of the original regions of different types.

[0126] In some optional implementation manners, when the target object includes a human object, the image generation device further includes:

[0127] A module acquisition module for acquiring candidate clothing models; wherein the candidate clothing models include three-dimensional models of clothing constructed based on each original image.

[0128] A model editing module for determining, in response to a clothing editing operation, the target clothing models matching each human object from the candidate clothing models, and rendering the target clothing models onto the corresponding updated human objects.

[0129] In some alternative implementation manners, the image generation device further includes:

[0130] A background area acquisition module for acquiring candidate background areas; wherein the candidate background areas include the original background areas extracted from each original image.

[0131] A background area selection module for selecting a target background area from the candidate background areas in response to a background selection operation.

[0132] Correspondingly, the generation module can also be used to: generate a target image according to the target background area.

[0133] In some alternative implementation manners, the image generation device can also include:

[0134] A screening module for performing at least one of the following before evaluating each original area:

[0135] Performing clarity analysis on each original area and screening each original area according to the results of the clarity analysis;

[0136] Performing similarity analysis on each original area and screening each original area according to the results of the similarity analysis.

[0137] In some alternative implementation manners, the image generation device can also include:

[0138] A brightness adjustment module for performing brightness analysis on each updated target object after generating each updated target object, and adjusting the brightness of each updated target object according to the results of the brightness analysis.

[0139] In some alternative implementation manners, the image generation device can also include:

[0140] An area processing module for performing at least one of the following on each original area after extracting at least two types of original areas of each target object in each original image: updating, deleting, and adding.

[0141] The image generation device provided by the embodiments of the present disclosure can execute the image generation method provided by any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution of the method.

[0142] It should be noted that the various units and modules included in the above device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the embodiments of the present disclosure.

[0143] Reference is made below Figure 5 to, which shows a schematic structural diagram of an electronic device (such as Figure 5 the terminal device or server in) 500 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0144] As Figure 5 shown, the electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0145] Generally, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or wirelessly to exchange data. Although Figure 5 the electronic device 500 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0146] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing device 501, the above functions defined in the image generation method of the embodiment of the present disclosure are performed.

[0147] The electronic device provided by the embodiment of the present disclosure and the image generation method provided by the above embodiment belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0148] An embodiment of the present disclosure provides a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the image generation method provided by the above embodiment is implemented.

[0149] It should be noted that the computer-readable medium described above can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a flash memory (FLASH), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0150] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0151] The above computer-readable medium can be included in the above electronic device; or it can exist separately without being assembled into the electronic device.

[0152] The above computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device is caused to:

[0153] Obtain at least two original images, where each original image contains at least two target objects, and the target objects included in each pair of original images are the same; extract at least two types of original regions of each target object in each original image; evaluate each original region to obtain a first evaluation result, and evaluate the matching degree between original regions of different types to obtain a second evaluation result; determine different types of target regions that match each target object from each original region according to the first evaluation result and the second evaluation result; generate updated target objects according to the different types of target regions that match each target object, and generate a target image according to the updated target objects.

[0154] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0155] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0156] The units involved in the embodiments of the present disclosure can be implemented in software or in hardware. Among them, the names of the units and modules do not, in some cases, constitute a limitation on the units and modules themselves.

[0157] The functions described above in this article can be performed, at least in part, by one or more hardware logic components. For example, without limitation, the exemplary types of hardware logic components that can be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and so on.

[0158] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash Memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] According to one or more embodiments of the present disclosure, an image generation method is provided, and the method includes:

[0160] Obtain at least two original images, where each of the original images contains at least two target objects, and the target objects included in each of the original images are the same;

[0161] Extract at least two types of original regions of each of the target objects in each of the original images;

[0162] Evaluate each of the original regions to obtain a first evaluation result, and evaluate the matching degree between the original regions of different types to obtain a second evaluation result;

[0163] Determine different types of target regions that match each of the target objects from the respective original regions according to the first evaluation result and the second evaluation result;

[0164] Generate each of the updated target objects according to the different types of target regions that match each of the target objects, and generate a target image according to each of the updated target objects.

[0165] According to one or more embodiments of the present disclosure, there is provided an image generation method, further including:

[0166] In some alternative implementation manners, the evaluating the matching degree between the original regions of different types includes:

[0167] Evaluate the matching degree between the original regions of different types according to the normal information of the original regions of different types.

[0168] According to one or more embodiments of the present disclosure, there is provided an image generation method, further including:

[0169] In some alternative implementation manners, when the target object includes a human object, the method further includes:

[0170] Obtain a candidate clothing model; wherein, the candidate clothing model includes a three-dimensional model of clothing constructed according to each of the original images;

[0171] In response to a clothing editing operation, determine a target clothing model that matches each of the human objects from the candidate clothing models, and render the target clothing model onto the corresponding updated human object.

[0172] According to one or more embodiments of the present disclosure, there is provided an image generation method, further including:

[0173] In some alternative implementation manners, obtain a candidate background region; wherein, the candidate background region includes an original background region extracted from each of the original images;

[0174] In response to a background selection operation, select a target background region from the candidate background regions;

[0175] Correspondingly, the process of generating the target image further includes: generating the target image according to the target background region.

[0176] According to one or more embodiments of the present disclosure, there is provided an image generation method, further including:

[0177] In some alternative implementation manners, before evaluating each of the original regions, at least one of the following is further included:

[0178] Perform clarity analysis on each of the original regions, and screen each of the original regions according to the results of the clarity analysis;

[0179] Perform similarity analysis on each of the original regions, and screen each of the original regions according to the results of the similarity analysis.

[0180] According to one or more embodiments of the present disclosure, there is provided an image generation method, further comprising:

[0181] In some alternative implementation manners, after generating each of the updated target objects, further comprising:

[0182] Perform brightness analysis on each of the updated target objects, and adjust the brightness of each of the updated target objects according to the results of the brightness analysis.

[0183] According to one or more embodiments of the present disclosure, there is provided an image generation method, further comprising:

[0184] In some alternative implementation manners, after extracting at least two types of original regions of each of the target objects in each of the original images, further comprising:

[0185] Perform at least one of the following processes on each of the original regions: update, delete, and add.

[0186] According to one or more embodiments of the present disclosure, there is provided an image generation device, the device comprising:

[0187] An image acquisition module, configured to acquire at least two original images, wherein each of the original images contains at least two target objects, and the target objects included in each of the original images are the same;

[0188] A region extraction module, configured to extract at least two types of original regions of each of the target objects in each of the original images;

[0189] An evaluation module, configured to evaluate each of the original regions to obtain a first evaluation result, and evaluate the matching degree between the original regions of different types to obtain a second evaluation result;

[0190] A matching module, configured to determine different types of target regions matching each of the target objects from each of the original regions according to the first evaluation result and the second evaluation result;

[0191] A generation module, configured to generate each of the updated target objects according to the different types of target regions matching each of the target objects, and generate a target image according to the updated target objects.

[0192] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0193] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although a number of specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0194] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms for implementing the claims.

Claims

1. An image generation method, characterized in that, it includes: obtaining at least two original images, wherein at least two target objects are included in each of the original images, and the target objects included in each of the original images are the same; extracting at least two types of original regions of each of the target objects in each of the original images; evaluating each of the original regions to obtain a first evaluation result, and evaluating the matching degree between the original regions of different types to obtain a second evaluation result; determining, according to the first evaluation result and the second evaluation result, different types of target regions that match each of the target objects from each of the original regions; generating updated each of the target objects according to the different types of target regions that match each of the target objects, and generating a target image according to the updated each of the target objects.

2. The method according to claim 1, characterized in that, the evaluating the matching degree between the original regions of different types includes: evaluating the matching degree between the original regions of different types according to the normal information of the original regions of different types.

3. The method according to claim 1, characterized in that, when the target object includes a human object, the method further includes: obtaining a candidate clothing model; wherein, the candidate clothing model includes a three-dimensional model of clothing constructed according to each of the original images; responding to a clothing editing operation, determining a target clothing model that matches each of the human objects from the candidate clothing models, and rendering the target clothing model onto the corresponding updated human object.

4. The method according to claim 1, characterized in that, the method further includes: obtaining a candidate background region; wherein, the candidate background region includes an original background region extracted from each of the original images; responding to a background selection operation, selecting a target background region from the candidate background regions; correspondingly, the process of generating the target image further includes: generating the target image according to the target background region.

5. The method according to claim 1, characterized in that, before the evaluating each of the original regions, at least one of the following is further included: performing clarity analysis on each of the original regions, and screening each of the original regions according to the result of the clarity analysis; performing similarity analysis on each of the original regions, and screening each of the original regions according to the result of the similarity analysis.

6. The method according to claim 1, characterized in that, after generating the updated each of the target objects, it further includes: performing brightness analysis on the updated each of the target objects, and adjusting the brightness of the updated each of the target objects according to the result of the brightness analysis.

7. The method according to claim 1, characterized in that, after the extracting at least two types of original regions of each of the target objects in each of the original images, it further includes: performing at least one of the following processing on each of the original regions: updating, deleting, and adding.

8. An image generation device, characterized in that, it includes: An image acquisition module, configured to acquire at least two original images, wherein each of the original images contains at least two target objects, and the target objects included in each of the original images are the same; A region extraction module, configured to extract at least two types of original regions of each of the target objects in each of the original images; An evaluation module, configured to evaluate each of the original regions to obtain a first evaluation result, and to evaluate the matching degree between the original regions of different types to obtain a second evaluation result; A matching module, configured to determine, according to the first evaluation result and the second evaluation result, different types of target regions that match each of the target objects from each of the original regions; A generation module, configured to generate updated each of the target objects according to the different types of target regions that match each of the target objects, and generate a target image according to the updated each of the target objects.

9. An electronic device, characterized in that, the electronic device includes: one or more processors; a storage device, configured to store one or more programs, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the image generation method according to any one of claims 1-7.

10. A storage medium containing computer-executable instructions, where the computer-executable instructions are used to execute the image generation method according to any one of claims 1-7 when executed by a computer processor.

Images