Image Generation Method and Apparatus, Computer-Readable Storage Medium, and Electronic Device

Abstract

The present disclosure provides an image generation method, an image generation device, a computer-readable storage medium, and an electronic device, relating to the technical field of image processing. The image generation method includes: obtaining a first color image, and determining a template of a foreground object in the first color image; splitting the first color image by color channels to obtain a plurality of first grayscale images; adjusting the grayscale values of the foreground object in the first grayscale images to obtain second grayscale images; generating a target grayscale image according to the second grayscale images, the template of the foreground object, and a target background grayscale image, and adding colors to the target grayscale image to generate a target color image, where the target color image is used as training data in the training process of a matting model. The present disclosure can expand the training data of the matting model.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technologies, and in particular, to an image generation method, an image generation device, a computer-readable storage medium, and an electronic device. Background Art

[0002] As an important means in the process of image processing, image matting has been widely used in scenarios such as image fusion and image correction.

[0003] The image matting task has high requirements for training data. Currently, there is generally a problem that the lack of training data leads to low accuracy of model processing. Summary of the Invention

[0004] The present disclosure provides an image generation method, an image generation device, a computer-readable storage medium, and an electronic device, thereby at least to some extent overcoming the problem of insufficient training data in the matting task.

[0005] According to a first aspect of the present disclosure, there is provided an image generation method, including: obtaining a first color image, and determining a template of a foreground object in the first color image; splitting the first color image by color channels to obtain a plurality of first grayscale images; adjusting the grayscale values of the foreground object in the first grayscale images to obtain second grayscale images; generating a target grayscale image according to the second grayscale images, the template of the foreground object, and a target background grayscale image, and adding colors to the target grayscale image to generate a target color image, and the target color image is used as training data in the training process of a matting model.

[0006] According to a second aspect of the present disclosure, there is provided an image generation device, including: an image acquisition module, configured to obtain a first color image and determine a template of a foreground object in the first color image; a channel splitting module, configured to split the first color image by color channels to obtain a plurality of first grayscale images; a grayscale adjustment module, configured to adjust the grayscale values of the foreground object in the first grayscale images to obtain second grayscale images; an image generation module, configured to generate a target grayscale image according to the second grayscale images, the template of the foreground object, and a target background grayscale image, and add colors to the target grayscale image to generate a target color image, and the target color image is used as training data in the training process of a matting model.

[0007] According to a third aspect of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned image generation method is implemented.

[0008] According to a fourth aspect of the present disclosure, there is provided an electronic device, including a processor; and a memory for storing one or more programs, which when executed by the processor, cause the processor to implement the above-mentioned image generation method.

[0009] In the technical solutions provided by some embodiments of the present disclosure, a template of the foreground object of the first color image is determined, the first color image is split by color channels to obtain a plurality of first grayscale images, the grayscale values of the foreground object in the first grayscale images are adjusted to obtain second grayscale images, and a target grayscale image is generated based on the second grayscale images, the template of the foreground object, and the target background grayscale image, and colors are added to the target grayscale image to generate a target color image. On the one hand, through the splitting of the color channels of the first color image, the present disclosure can determine a plurality of target color images, realizing the augmentation of training data; on the other hand, since the foreground objects of the plurality of target color images are all generated based on the first color image, using them for the training of the matting model can enhance the generalization ability and robustness of the matting model and improve the accuracy of model processing.

[0010] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure and, together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts. In the drawings:

[0012] Figure 1 shows an input-output schematic diagram of the image generation scheme according to an embodiment of the present disclosure;

[0013] Figure 2 shows a schematic diagram of the image conversion process according to an embodiment of the present disclosure;

[0014] Figure 3 schematically shows a flowchart of the image generation method according to an exemplary embodiment of the present disclosure;

[0015] Figure 4 shows a schematic diagram of splitting the channels of the first color image according to an embodiment of the present disclosure;

[0016] Figure 5 shows a schematic diagram of adjusting the grayscale value of the first grayscale image of the R channel to obtain the second grayscale image according to an embodiment of the present disclosure;

[0017] Figure 6 Schematic diagram showing channel splitting of a second color image according to an embodiment of the present disclosure;

[0018] Figure 7 Schematic diagram showing generation of a target grayscale image through image fusion according to an embodiment of the present disclosure;

[0019] Figure 8 Flowchart showing the entire process of an image generation scheme according to an embodiment of the present disclosure;

[0020] Figure 9 Block diagram schematically showing an image generation device according to an exemplary embodiment of the present disclosure;

[0021] Figure 10 Block diagram schematically showing an electronic device according to an exemplary embodiment of the present disclosure. Detailed implementation manners

[0022] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be used. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring the various aspects of the present disclosure.

[0023] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0024] The flowcharts shown in the accompanying drawings are only illustrative and do not necessarily include all steps. For example, some steps can be decomposed, while some steps can be combined or partially combined, so the actual execution order may change according to the actual situation. Additionally, all the following terms "first", "second", "third", etc. are only for the purpose of distinction and should not be construed as a limitation of the content of the present disclosure.

[0025] Figure 1 The figure shows an input-output schematic diagram of the image generation solution according to an embodiment of the present disclosure. Referring to Figure 1 , the input of the image generation solution according to an embodiment of the present disclosure may include a first color image and a second color image. Among them, the first color image is used to provide a foreground object, and the second color image is used to provide a background. The output of the image generation solution may be multiple target color images. Additionally, in some instances of applying the image generation solution of the present disclosure, only one target color image may also be generated.

[0026] The foreground object of the target color image is consistent with the foreground object of the first color image, and the background of the target color image is consistent with the background of the second color image. Additionally, in the case where the second color image is only a background, the background of the target color image is consistent with the second color image.

[0027] It should be noted that the target color images generated in the embodiments of the present disclosure are used as training data in the training process of the matting model. That is to say, the target color images can also be added to the training set of the matting model.

[0028] The image generation solution according to an embodiment of the present disclosure can be implemented by a terminal device. That is to say, the terminal device can execute each step of the following image generation method, and the following image generation device can be configured in this terminal device. Among them, the terminal device may include but is not limited to a smart phone, a tablet computer, a smart wearable device, a personal computer, a server, etc. The present disclosure does not limit the type of the terminal device.

[0029] In an embodiment where the terminal device is a smart phone or the like including a camera module, the first color image and / or the second color image may be a color image captured by the smart phone with the help of its camera module, or may be a color image obtained from other devices. In an embodiment where the terminal device is a server or the like without a camera module, the first color image and / or the second color image may be a color image obtained by the server from other devices. The present disclosure does not limit the source, content, and size of the first color image and / or the second color image.

[0030] In addition, the first color image may be an annotated image, such as an image annotated manually. It can be understood that for training samples, the annotated data can be used to train the matting model.

[0031] In some embodiments of the present disclosure, in view that the foreground objects of the first color image and the target color image are the same, the same annotation information as the first color image can be configured for the target color image, that is, the annotation information of the first color image is applied to the target color image. Thus, the generated target color image also has annotation information. When the generated target color image is added to the training set of the matting model, it is equivalent to expanding the training data. Therefore, the image generation scheme of the embodiments of the present disclosure can also solve the problems of high annotation cost and long time consumption of the training samples of the matting model.

[0032] Figure 2 shows a schematic diagram of the image conversion process of the embodiments of the present disclosure. Referring to Figure 2 , the image conversion process of the embodiments of the present disclosure includes an image conversion process for the first color image, an image conversion process for the second color image, and an image fusion process.

[0033] On the one hand, first, the terminal device can split the first color image by color channels to obtain a first grayscale image. It can be understood that the number of first grayscale images obtained by channel splitting is usually multiple.

[0034] Next, the terminal device can adjust the grayscale values of the first grayscale image to obtain a second grayscale image. Specifically, for the first grayscale image, different grayscale value adjustment methods can be adopted. That is to say, based on one first grayscale image, multiple second grayscale images corresponding to the first grayscale image can be generated.

[0035] On the other hand, the terminal device can determine the template of the foreground object of the first color image.

[0036] On yet another hand, first, the terminal device can split the second color image by color channels to obtain a third grayscale image. Similarly, the number of third grayscale images obtained by channel splitting is usually multiple.

[0037] Next, the terminal device can adjust the grayscale values of the third grayscale image to obtain a target background grayscale image. Similarly, for the third grayscale image, different grayscale value adjustment methods can be adopted. That is to say, based on one third grayscale image, multiple target background grayscale images corresponding to the third grayscale image can be generated.

[0038] When the second grayscale image, the template of the foreground object, and the target background grayscale image are determined, they can be fused to generate a target color image.

[0039] Based on the above image generation process of generating a target color image from the first color image and the second color image, on the one hand, the present disclosure can determine multiple target color images by splitting the color channels of the first color image, realizing the augmentation of training data; on the other hand, since the foreground objects of the multiple target color images are all generated based on the first color image, using them for training the matting model can enhance the generalization ability and robustness of the matting model and improve the accuracy of model processing.

[0040] Hereinafter, taking the terminal device executing the image generation method of the embodiment of the present disclosure as an example, the image processing process of the present disclosure will be described.

[0041] Figure 3 Schematically shows a flowchart of the image generation method of an exemplary embodiment of the present disclosure. Refer to Figure 3 , the image generation method may include the following steps:

[0042] S32. Obtain a first color image and determine a template of the foreground object of the first color image.

[0043] In an exemplary embodiment of the present disclosure, for the scenario of matting model training, the foreground object of the first color image may be a matting object, and the matting object refers to the object to be matted out in the image. For example, the foreground object may be a real object in the real world such as a person, an animal, a car, a building, furniture, etc. For another example, the foreground object may also be a virtual object such as an animated object or a game object in the image. The present disclosure does not limit the type of the foreground object.

[0044] The foreground object may be an object of interest specified by the user in advance. For example, if the user is interested in a portrait, the portrait can be set as the foreground object. The foreground object may also be an object determined by the terminal device through image recognition. For example, if the terminal device recognizes that the image only contains a dog, the dog can be used as the foreground object.

[0045] According to some embodiments of the present disclosure, the first color image may be a color image captured by the terminal device with the equipped camera module, or a color image drawn by a drawing software, or a color image obtained from other devices, or a frame image intercepted from a video frame. The present disclosure does not limit the source, size, etc. of the first color image.

[0046] It should be noted that the first color image mentioned in the embodiments of the present disclosure may be one of the training data for training the matting model, that is, the terminal device can obtain the first color image from the training set of the matting model. For example, the first color image can be randomly selected from the training set.

[0047] In addition, the first color image may be an annotated image, such as an image annotated manually.

[0048] After obtaining the first color image, the terminal device may determine a template (mask) of the foreground object in the first color image.

[0049] In the embodiments of the present disclosure, the template of the foreground object may be determined based on manual annotation. Alternatively, the template of the foreground object may be determined based on a rough estimation of a matting model and combined with manual adjustment. To make the annotation information of the subsequent generated target color image accurate, the annotation information of the first color image may be accurately determined in advance. That is to say, the accuracy of the template of the foreground object in the first color image affects the accuracy of the annotation information of the subsequent target color image. In the scenario where the target color image is used as the training data of the matting model, the accuracy of the annotation information of the target color image affects the processing accuracy of the matting model. Therefore, for the template of the foreground object in the first color image, the operation of manual annotation may be combined to improve the accuracy of the template.

[0050] S34. Split the first color image by color channels to obtain a plurality of first grayscale images.

[0051] In the exemplary embodiments of the present disclosure, the first color image includes a plurality of color channels. Specifically, these color channels generally include an R (red) channel, a G (green) channel, and a B (blue) channel.

[0052] After obtaining the first color image, the terminal device may split the first color image by color channels to obtain a plurality of first grayscale images. It can be understood that each first grayscale image corresponds one-to-one to the color channels of the first color image.

[0053] Reference Figure 4 , for the first color image of RGB, after channel splitting, a first grayscale image of the R channel, a first grayscale image of the G channel, and a first grayscale image of the B channel may be obtained.

[0054] S36. Adjust the grayscale value of the foreground object in the first grayscale image to obtain a second grayscale image.

[0055] In the exemplary embodiments of the present disclosure, the terminal device may adjust the grayscale value of the foreground object by using different grayscale value adjustment methods respectively to obtain a second grayscale image. It should be noted that the terminal device may adjust the grayscale value of the entire first grayscale image, or may only adjust the grayscale value of the foreground object in the first grayscale image while the grayscale value of the background area in the first grayscale image remains unchanged. The present disclosure does not limit this.

[0056] The grayscale value adjustment method may include any algorithm operation capable of changing the grayscale value, including one or a combination of adding grayscale noise, increasing or decreasing the overall grayscale value, smoothing, and sharpening.

[0057] In one embodiment, the terminal device may perform an overall increase or decrease operation on the grayscale value of the first grayscale image to obtain a second grayscale image corresponding to the first grayscale image. For example, the grayscale value is increased or decreased by 10 overall to obtain the second grayscale image. Refer to Figure 5 , Image a is Figure 4 the first grayscale image of the R channel in

[0058] In another embodiment, the terminal device may add grayscale noise to the first grayscale image to obtain a second grayscale image corresponding to the first grayscale image. For example, the grayscale noise may be randomly added. The present disclosure places no restrictions on the degree and manner of adding grayscale noise. Refer to Figure 5 , Image d is the second grayscale image obtained by adding grayscale noise to Image a.

[0059] In yet another embodiment, the terminal device may perform smoothing and / or sharpening on the first grayscale image to obtain a second grayscale image corresponding to the first grayscale image. Refer to Figure 5 , Figure e is the second grayscale image obtained by performing smoothing on Image a, and Figure f is the second grayscale image obtained by performing sharpening on Image a.

[0060] It should be understood that, on the one hand, the second grayscale images in the above embodiments are different second grayscale images determined by different grayscale value adjustment methods; on the other hand, the above grayscale value adjustment methods can be arbitrarily combined, and after the combined grayscale adjustment method is applied to the first grayscale image, a second grayscale image can also be obtained.

[0061] According to some embodiments of the present disclosure, for the multiple first grayscale images determined in step S34, the above grayscale value adjustment can be performed on each first grayscale image. If the grayscale value adjustment method includes three types, then for the three first grayscale images of the R channel, G channel, and B channel, a total of 9 (3×3) second grayscale images can be obtained.

[0062] According to some other embodiments of the present disclosure, for the multiple first grayscale images determined in step S34, the grayscale values of some of the first grayscale images can be adjusted as described above. For example, only the first grayscale image of the R channel is adjusted for the grayscale value. If there are five grayscale value adjustment methods, a total of 5 second grayscale images can be obtained. Another example is to adjust the grayscale values of the first grayscale image of the R channel and the first grayscale image of the B channel. If there are four grayscale value adjustment methods, a total of 8 (2×4) second grayscale images can be obtained.

[0063] It should be noted that the number of first grayscale images for which the grayscale value is adjusted and the number of grayscale value adjustment methods can be determined in combination with the training accuracy requirements of the matting model and the processing capabilities of the terminal device.

[0064] S38. Generate a target grayscale image based on the second grayscale image, the template of the foreground object, and the target background grayscale image, and add color to the target grayscale image to generate a target color image, which is used as training data in the training process of the matting model.

[0065] In the exemplary embodiment of the present disclosure, the target background grayscale image can be determined from the second color image.

[0066] First, the terminal device can obtain the second color image and split the second color image according to color channels to obtain multiple third grayscale images. The way of splitting the color channels is the same as the way of splitting the first color image described above. Refer to Figure 6 , for the RGB second color image, after channel splitting, a third grayscale image of the R channel, a third grayscale image of the G channel, and a third grayscale image of the B channel can be obtained.

[0067] Next, the terminal device can use the third grayscale image to determine the target background grayscale image.

[0068] According to some embodiments of the present disclosure, the terminal device can directly use the third grayscale image as the target background grayscale image.

[0069] According to some other embodiments of the present disclosure, the terminal device can adjust the grayscale value of the third grayscale image to obtain the target background grayscale image. Similar to the process of adjusting the grayscale value of the first grayscale image described above, the grayscale value adjustment method can include any algorithm operation that can change the grayscale value, including one or a combination of adding grayscale noise, increasing or decreasing the overall grayscale value, smoothing, and sharpening processing. The specific process will not be elaborated here.

[0070] After determining the second grayscale image, the template of the foreground object, and the target background grayscale image, first, the terminal device can determine the foreground object grayscale image based on the second grayscale image and the template of the foreground object. That is to say, the terminal device extracts the foreground object grayscale image from the second grayscale image by using the template of the foreground object.

[0071] Next, the terminal device fuses the foreground object grayscale image with the target background grayscale image to generate the target grayscale image.

[0072] In some embodiments of the present disclosure, the position of the foreground object grayscale image in the second grayscale image is the same as the position of the foreground object grayscale image in the target background grayscale image after fusion.

[0073] In some other embodiments of the present disclosure, the terminal device can extract the scene features of the target background grayscale image, where the scene features include the planar features of the target background grayscale image. Thus, the area where the foreground object can be placed can be screened out from the planar features. That is to say, the fusion coordinates of the foreground object grayscale image can be determined according to the scene features of the target background grayscale image. Then, the terminal device can use the fusion coordinates to superimpose the foreground object on the target background grayscale image to generate the target grayscale image.

[0074] Reference Figure 7 , the target grayscale image 74 can be generated by using the second grayscale image 71, the template 72 of the foreground object, and the target background grayscale image 73.

[0075] After determining the target grayscale image, the terminal device can add color to the target grayscale image, and this process can also be referred to as a coloring operation or a tinting operation. Thus, the target color image can be generated.

[0076] The terminal device can add color to the target grayscale image according to the grayscale value of the target grayscale image. Specifically, the terminal device can input the target grayscale image into a deep learning network, and use the non-linear mapping operation of the deep learning network to convert the grayscale value of the target grayscale image to generate the target color image corresponding to the target grayscale image. The deep learning-based coloring algorithm in the embodiments of the present disclosure can color the target grayscale image naturally and robustly. Taking a portrait as an example, the coloring process can adaptively change the clothing and hair color of the portrait, and control the portrait color to appear natural and suitable.

[0077] The foreground object of the target color image determined in the embodiments of the present disclosure corresponds to the foreground object of the first color image. The generated target color image can be used as training data in the training process of the matting model. That is to say, the terminal device can put the target color image into the training set of the matting model to expand the training set. Thus, the image generation method in the embodiments of the present disclosure can also solve the problems of high cost and long time consumption in the annotation of training samples for the matting model.

[0078] The following refers to Figure 8 to illustrate the entire process of the image generation scheme in the embodiments of the present disclosure.

[0079] In step S802, the terminal device can obtain a first color image.

[0080] In step S804, the terminal device can determine the template of the foreground object in the first color image.

[0081] In step S806, the terminal device can split the first color image by color channels to obtain a plurality of first grayscale images.

[0082] In step S808, the terminal device can adjust the grayscale values of the first grayscale images to obtain second grayscale images.

[0083] In step S810, the terminal device can obtain a second color image.

[0084] In step S812, the terminal device can split the second color image by color channels to obtain a plurality of third grayscale images.

[0085] In step S814, the terminal device can adjust the grayscale values of the third grayscale images to obtain a target background grayscale image.

[0086] In step S816, the terminal device can generate a target grayscale image according to the second grayscale image, the template of the foreground object, and the target background grayscale image.

[0087] In step S818, the terminal device can add colors to the target grayscale image to generate a target color image.

[0088] In the above steps, the order between the steps is not restricted. For example, steps S810 to S814 can be executed before steps S802 to S808, or steps S810 to S814 can be executed simultaneously with steps S802 to S808.

[0089] It should be noted that although the steps of the methods in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0090] Furthermore, in this exemplary embodiment, an image generation device is also provided.

[0091] Figure 9 A block diagram of the image generation device according to the exemplary embodiment of the present disclosure is schematically shown. Refer to Figure 9 , the image generation device 9 according to the exemplary embodiment of the present disclosure may include an image acquisition module 91, a channel splitting module 93, a grayscale adjustment module 95, and an image generation module 97.

[0092] Specifically, the image acquisition module 91 may be configured to acquire a first color image and determine a template of the foreground object in the first color image; the channel splitting module 93 may be configured to split the first color image by color channels to obtain a plurality of first grayscale images; the grayscale adjustment module 95 may be configured to adjust the grayscale values of the foreground object in the first grayscale images to obtain second grayscale images; the image generation module 97 may be configured to generate a target grayscale image based on the second grayscale images, the template of the foreground object, and a target background grayscale image, and add colors to the target grayscale image to generate a target color image, and the target color image is used as training data in the training process of the matting model.

[0093] According to the exemplary embodiment of the present disclosure, the image generation module 97 may also be configured to perform: acquiring a second color image; splitting the second color image by color channels to obtain a plurality of third grayscale images; and determining the target background grayscale image by using the third grayscale images.

[0094] According to the exemplary embodiment of the present disclosure, the process of the image generation module 97 for determining the target background grayscale image may be configured to perform: adjusting the grayscale values of the third grayscale images to obtain the target background grayscale image.

[0095] According to the exemplary embodiment of the present disclosure, the image generation module 97 may be configured to perform: determining a foreground object grayscale image according to the second grayscale images and the foreground object template; and fusing the foreground object grayscale image with the target background grayscale image to generate the target grayscale image.

[0096] According to an exemplary embodiment of the present disclosure, the image generation module 97 may be configured to perform: extracting the scene features of the target background grayscale image; determining the fusion coordinates of the foreground object grayscale image according to the scene features of the target background grayscale image; and superimposing the foreground object grayscale image on the target background grayscale image by using the fusion coordinates.

[0097] According to an exemplary embodiment of the present disclosure, the image generation module 97 may be configured to perform: adding colors to the target grayscale image according to the grayscale values of the target grayscale image to generate a target color image.

[0098] According to an exemplary embodiment of the present disclosure, the target grayscale image is input into a deep learning network, and the grayscale values of the target grayscale image are converted by using the non-linear mapping operation of the deep learning network to generate a target color image.

[0099] Since each functional module of the image generation device according to the embodiment of the present disclosure is the same as that in the above method embodiment, it will not be described in detail herein.

[0100] Figure 10 The figure shows a schematic diagram of an electronic device suitable for implementing the exemplary embodiment of the present disclosure. The terminal device according to the exemplary embodiment of the present disclosure may be configured in the form of Figure 10 It should be noted that Figure 10 The electronic device shown in the figure is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0101] The electronic device of the present disclosure at least includes a processor and a memory. The memory is used to store one or more programs. When the one or more programs are executed by the processor, the processor can implement the image generation method according to the exemplary embodiment of the present disclosure.

[0102] Specifically, as Figure 10As shown, the electronic device 100 may include: a processor 1010, an internal memory 1021, an external memory interface 1022, a Universal Serial Bus (USB) interface 1030, a charging management module 1040, a power management module 1041, a battery 1042, an antenna 1, an antenna 2, a mobile communication module 1050, a wireless communication module 1060, an audio module 1070, a sensor module 1080, a display screen 1090, a camera module 1091, an indicator 1092, a motor 1093, a button 1094, and a Subscriber Identification Module (SIM) card interface 1095, etc. The sensor module 1080 may include a depth sensor, a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0103] It can be understood that the structure schematically shown in the embodiments of the present disclosure does not constitute a specific limitation on the electronic device 100. In other embodiments of the present disclosure, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0104] The processor 1010 may include one or more processing units. For example, the processor 1010 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband processor, and / or a Neural-network Processing Unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. In addition, a memory may also be provided in the processor 1010 for storing instructions and data.

[0105] The electronic device 100 may implement a shooting function through the ISP, the camera module 1091, the video codec, the GPU, the display screen 1090, the application processor, etc. In some embodiments, the electronic device 100 may include 1 or N camera modules 1091, where N is a positive integer greater than 1. If the electronic device 100 includes N cameras, one of the N cameras is the main camera.

[0106] The internal memory 1021 can be used to store computer-executable program code, and the executable program code includes instructions. The internal memory 1021 can include a program storage area and a data storage area. The external memory interface 1022 can be used to connect to an external memory card, such as a Micro SD card, to implement the storage capacity expansion of the electronic device 100.

[0107] The present disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device.

[0108] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0109] The computer-readable storage medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0110] The computer-readable storage medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device implements the method as described in the embodiments of the present disclosure.

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0112] The units described in the embodiments of the present disclosure can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not, in some cases, constitute a limitation on the unit itself.

[0113] From the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0114] In addition, the above accompanying drawings are only schematic illustrations of the processes included in the methods according to the exemplary embodiments of the present disclosure, and are not for limiting purposes. It is easy to understand that the processes shown in the above accompanying drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0115] It should be noted that although several modules or units of devices for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided and embodied by multiple modules or units.

[0116] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed herein. The specification and examples are only illustrative, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0117] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image generation method, characterized in that, Including: Obtain a first color image and determine a template of a foreground object in the first color image; Split the first color image by color channels to obtain a plurality of first grayscale images; Adjust the grayscale values of the foreground object in the first grayscale image to obtain a second grayscale image; Generate a target grayscale image based on the second grayscale image, the template of the foreground object, and a target background grayscale image, and add color to the target grayscale image to generate a target color image, where the target color image is used as training data in the training process of a matting model.

2. The image generation method according to claim 1, wherein The image generation method further includes: Obtain a second color image; Split the second color image by color channels to obtain a plurality of third grayscale images; Determine the target background grayscale image using the third grayscale images.

3. The image generation method according to claim 2, characterized in that Determining the target background grayscale image using the third grayscale images includes: Adjust the grayscale values of the third grayscale images to obtain the target background grayscale image.

4. The image generation method according to claim 1, wherein Generating a target grayscale image based on the second grayscale image, the template of the foreground object, and a target background grayscale image includes: Determine a foreground object grayscale image based on the second grayscale image and the template of the foreground object; Fuse the foreground object grayscale image with the target background grayscale image to generate the target grayscale image.

5. The image generation method according to claim 4, wherein Fusing the foreground object grayscale image with the target background grayscale image includes: Extract the scene features of the target background grayscale image; Determine the fusion coordinates of the foreground object grayscale image according to the scene features of the target background grayscale image; Overlay the foreground object grayscale image on the target background grayscale image using the fusion coordinates.

6. The image generation method according to claim 1, wherein Adding color to the target grayscale image to generate a target color image includes: Add color to the target grayscale image according to the grayscale values of the target grayscale image to generate the target color image.

7. The image generation method according to claim 6, wherein Adding color to the target grayscale image according to the grayscale values of the target grayscale image to generate the target color image includes: Input the target grayscale image into a deep learning network, and use the non-linear mapping operation of the deep learning network to transform the grayscale values of the target grayscale image to generate the target color image.

8. An image generation device, characterized in that, Including: An image acquisition module, configured to obtain a first color image and determine a template of a foreground object in the first color image; A channel splitting module, configured to split the first color image by color channels to obtain a plurality of first grayscale images; A grayscale adjustment module, configured to adjust the grayscale values of the foreground object in the first grayscale image to obtain a second grayscale image; An image generation module, configured to generate a target grayscale image based on the second grayscale image, the template of the foreground object, and a target background grayscale image, and add color to the target grayscale image to generate a target color image, where the target color image is used as training data in the training process of a matting model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the image generation method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, Including: A processor; A memory for storing one or more programs which, when executed by the processor, cause the processor to implement the image generation method according to any one of claims 1 to 7.

Images