Image generation method, device, electronic device and storage medium
By obtaining the probability distribution information of the occlusion sample image, determining the target occlusion image and performing occlusion processing, the problem of low occlusion image generation efficiency in the prior art is solved, and the image with a specific occlusion ratio is quickly generated, which improves the generation efficiency and Control accuracy.
Patent Information
- Application Number
- CN202110247377.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-05
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-03-05
AI Technical Summary
The existing occlusion image generation methods have high trial and error costs and are inefficient, making it difficult to quickly generate images with specific occlusion ratios, especially when it is necessary to generate training sample sets of specific occlusion ratios.
By obtaining the probability distribution information of the occlusion sample image, the target occlusion image is determined, and the occlusion is occluded based on this information to generate a target image to ensure that the generated image has a specific occlusion ratio.
It reduces the cost of selecting occlusion images, improves generation efficiency, and ensures that the generated images meet the target occlusion ratio, and is suitable for image editing and model training scenarios.
Smart Images

Figure CN112967351B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to an image generating method, device, electronic device and storage medium. Background Art
[0002] In the field of image processing, through image generation or image editing, images that meet specific conditions can be generated as needed, such as occluded images, that is, the photographed object on the image appears to be occluded.
[0003] In the existing occlusion image generation process, it is necessary to generate an occlusion image with a specific occlusion ratio. Generally, a random sampling test method is used, but the trial-and-error cost of this method is very high and the efficiency is very low. Obviously, this cannot meet the image generation scenario with specific requirements on the occlusion ratio, such as generating a training sample set with a specific occlusion ratio. Therefore, how to quickly generate an occlusion image with a specific occlusion ratio is still a problem to be solved. Summary of the invention
[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide an image generating method, an apparatus, an electronic device and a storage medium.
[0005] In a first aspect, an embodiment of the present disclosure provides an image generation method, comprising:
[0006] Obtaining probability distribution information of a set of occluder sample images, wherein the probability distribution information is used to characterize the probability of occurrence of the occluder sample images under any occlusion ratio and any foreground ratio, the occlusion ratio being the ratio of the area of the occluded object occupied by the occluder on the occlusion image to the area of the occluded object, the foreground ratio being the ratio of the area of the occluder on the occlusion image to the area of the occlusion image, and the occlusion image is generated with the participation of the occluder sample images;
[0007] Determine, by using the probability distribution information, from the set of occluder sample images, a target occluder image that participates in generating an occluder image and satisfies a target occluder ratio;
[0008] Based on the foreground proportion corresponding to the target occluder image, the target occluder image is used to perform occlusion processing on the occluded object image to generate a target image.
[0009] In a second aspect, the present disclosure also provides an image generating device, including:
[0010] A probability distribution information acquisition module, used to acquire probability distribution information of a set of occluder sample images, wherein the probability distribution information is used to characterize the probability of occurrence of the occluder sample images under any occlusion ratio and any foreground ratio, the occlusion ratio being the ratio of the area of the occluded object occupied by the occluder on the occlusion image to the area of the occluded object, the foreground ratio being the ratio of the area of the occluder on the occlusion image to the area of the occlusion image, and the occlusion image is generated with the participation of the occluder sample images;
[0011] A target occluder image determination module is used to determine, from the set of occluder sample images, a target occluder image that participates in generating an occlusion image and satisfies a target occlusion ratio by using the probability distribution information;
[0012] The target image generation module is used to perform occlusion processing on the occluded object image using the target occluder image based on the foreground proportion corresponding to the target occluder image, so as to generate a target image.
[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device implements any image generation method provided by the embodiments of the present disclosure.
[0014] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computing device, the computing device implements any of the image generation methods provided by the embodiments of the present disclosure.
[0015] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has at least the following advantages: in the embodiments of the present disclosure, before generating the target image, the probability distribution information of the set of each occlusion sample image is first determined as a priori condition for generating the occlusion image, and then the target occlusion image is selected from the set of occlusion sample images according to the required target occlusion ratio using the probability distribution information, thereby achieving targeted selection of the target occlusion image. Compared with the case of randomly determining the occlusion image, the embodiments of the present disclosure reduce the trial and error cost and improve the selection efficiency of the target occlusion image. Moreover, since the probability distribution information characterizes the probability of occurrence of the occlusion sample image at any occlusion ratio and any foreground ratio, the occlusion image is occluded using the target occlusion image, and the probability of the generated target image having the target occlusion ratio is very high, thereby achieving rapid generation of images with a specific occlusion ratio, improving the generation efficiency of occlusion images, and at the same time achieving effective control over the generation of images with a specific occlusion ratio. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0018] Figure 1 A flowchart of an image generation method provided by an embodiment of the present disclosure;
[0019] Figure 2 A schematic diagram of probability distribution information of a set of occluder sample images provided by an embodiment of the present disclosure;
[0020] Figure 3 A flowchart of another image generation method provided by an embodiment of the present disclosure;
[0021] Figure 4 A schematic diagram of the structure of an image generating device provided by an embodiment of the present disclosure;
[0022] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0023] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0025] Figure 1 A flowchart of an image generation method provided in an embodiment of the present disclosure can be applied to the case of how to generate an occlusion image with a specific occlusion ratio, and the method can be executed by an image generation device. The image generation device can be implemented in software and / or hardware, and can be integrated in any electronic device with computing capabilities, such as a user terminal such as a smart phone, tablet computer, notebook or desktop computer, or a server, etc.
[0026] like Figure 1 As shown, the image generation method provided by the embodiment of the present disclosure may include:
[0027] S101. Obtain probability distribution information of a set of occluder sample images, wherein the probability distribution information is used to characterize the occurrence probability of the occluder sample images under any occlusion ratio and any foreground ratio.
[0028] In the disclosed embodiment, a sample image pool can be maintained in advance, which includes occluder sample images and obscured sample images. The occluder sample images can be used to perform occlusion processing on the obscured sample images, for example, by using a random occlusion method to generate a large number of occlusion images. Each occluder sample image can participate in the generation of multiple occlusion images, and then the occlusion ratio and foreground proportion calculation are performed on each occlusion image, so as to count the occurrence probability of the occluder sample image under any occlusion ratio and any foreground proportion (that is, when the occlusion image satisfies any occlusion ratio and any foreground proportion, the ratio between the total number of occurrences of the occluder sample images participating in the generation of the occlusion image and the total number of occlusion images), and obtain the probability distribution information of the set of occluder sample images. Among them, the occlusion ratio refers to the ratio of the area of the occluded object occupied by the occluder on the occlusion image to the area of the occluded object, and the foreground proportion refers to the ratio of the area of the occluder on the occlusion image to the area of the occlusion image.
[0029] The above probability distribution information is a probability density function of a two-dimensional joint distribution, where the two-dimensional random variables are the occlusion ratio and the foreground ratio. As an example, the probability distribution information can be implemented in the form of a matrix, and the matrix elements are the probability of occurrence of the occluder sample image under any occlusion ratio and any foreground ratio. It can be understood that since the matrix is a probability density function, the sum of all the elements of the matrix is 1.
[0030] Exemplarily, a total of N occlusion images are generated using occlusion sample images and occluded sample images, among which there are M (M value is less than N) occlusion images that satisfy the occlusion ratio of O1 and the foreground proportion of f1, that is, when the occlusion ratio is O1 and the foreground proportion is f1, the total number of occurrences of the occlusion sample images participating in the generation of the occlusion image is M, then when the occlusion ratio is O1 and the foreground proportion is f1, the probability of occurrence of the occlusion sample images is M / N. In the process of calculating the probability of occurrence of the occlusion sample images, the occlusion sample images are not distinguished.
[0031] The greater the probability of the occlusion sample image appearing at any occlusion ratio and any foreground ratio, the greater the probability of the occlusion image having the foreground ratio and the occlusion ratio generated by the occlusion sample image in the target image generation stage.
[0032] Furthermore, in the embodiments of the present disclosure, both the occluding object sample image and the occluded object sample image can be any available images, and the embodiments of the present disclosure do not make specific limitations. For example, the occluded object sample image can be a face sample image, and the occluding object sample image can be any sample image that can be used to make a face be in an occluded state.
[0033] S102 , using probability distribution information, determining, from a set of occluder sample images, target occluder images that participate in generating occluder images and that satisfy a target occluder ratio.
[0034] Among them, the target occlusion ratio is related to the occlusion ratio required for image generation, and can be set according to actual requirements. Exemplarily, after determining the target occlusion ratio, the probability of occurrence of the occlusion sample image under each foreground ratio corresponding to the target occlusion ratio can be determined in the probability distribution information, and then the target occlusion image is determined according to the occurrence probability. For example, the occlusion sample image with the highest probability of occurrence under each foreground ratio corresponding to the target occlusion ratio can be determined as the target occlusion image, or the target occlusion image can be determined comprehensively according to the required foreground ratio and the occurrence probability.
[0035] S103 . Based on the foreground proportion corresponding to the target occluder image, use the target occluder image to perform occlusion processing on the occluded object image to generate a target image.
[0036] The target image is generated based on the foreground ratio corresponding to the target occluder image, that is, the area ratio of the target occluder on the target image is kept unchanged, thereby ensuring that the target image generated based on the target occluder image can meet the required occlusion ratio with a high probability.
[0037] Optionally, based on the foreground proportion corresponding to the target occluder image, the target occluder image is used to perform occlusion processing on the occluded image to generate the target image, including:
[0038] Obtain an occluder mask image (i.e., mask image) corresponding to the target occluder image, and an occluded object mask image corresponding to the occluded object image;
[0039] Based on the foreground proportion, the occluding object mask image and the occluded object mask image corresponding to the target occluding image, the occluded object image is occluded using the target occluding object image to generate a target image.
[0040] Among them, the occluder mask image can be used to determine the area of the occluder on the target occluder image, and the occluded object mask image can be used to determine the area of the occluded object on the occluded object image. After the occluder area on the target occluder image is determined by the occluder mask image and the occluded object area on the occluded object image is determined by the occluder mask image, the occluded object can be blocked by the occluder under the constraint of the foreground proportion corresponding to the target occluded image, and finally the target image is generated. Exemplarily, the image background in the target occluder image can be used as the target background of the target image, and then the occluded object on the occluded object image determined by the occluded object mask image is pasted into the target background, and finally the occluded object on the target occluder image determined by the occluder mask image is pasted into the target background to randomly block the occluded object, thereby generating the target image. It should be noted that, regarding the background of the target image, the embodiment of the present disclosure is not specifically limited. In addition to using the image background in the target occluder image, any other available image background can also be used.
[0041] In the embodiment of the present disclosure, before generating the target image, the probability distribution information of the set of each occlusion sample image is first determined as a priori condition for generating the occlusion image, and then the target occlusion image is selected from the set of occlusion sample images based on the required target occlusion ratio using the probability distribution information, thereby achieving targeted selection of the target occlusion image. Compared with the case of randomly determining the occlusion image, the embodiment of the present disclosure reduces the trial and error cost and improves the selection efficiency of the target occlusion image. Moreover, since the probability distribution information characterizes the probability of occurrence of the occlusion sample image at any occlusion ratio and any foreground ratio, the occluded image is occluded using the target occlusion image, and the probability that the generated target image has the target occlusion ratio is very high, thereby achieving rapid generation of images with a specific occlusion ratio, improving the generation efficiency of occlusion images, and at the same time achieving effective control over the generation of images with a specific occlusion ratio.
[0042] On the basis of the above technical solution, optionally, the image generation method provided by the embodiment of the present disclosure may further include: taking the target image as a training sample, and obtaining the required image processing model based on the training sample training. That is, the technical solution provided by the embodiment of the present disclosure can be used in image editing scenarios to perform occlusion processing on the occluded image to be processed to generate a target image with a specific occlusion ratio; it can also be used in model training scenarios to generate a large number of target images with a specific occlusion ratio as training samples for model training. Compared with the situation where training samples with a specific occlusion ratio are generated by manual processing, the adoption of this solution can improve the generation efficiency of training samples, and then quickly obtain a large number of training samples that meet a certain occlusion ratio distribution, improve the training effect of the image processing model, and avoid the phenomenon that the training effect of the image processing model is poor because the training samples cannot meet the occlusion ratio distribution requirements. The image processing model may include but is not limited to an occlusion recognition or detection model, an occluded object recognition or detection model, etc., for example, a face image processing model that can be used to recognize a face in an occluded state, etc.
[0043] Optionally, the image generation method provided by the embodiment of the present disclosure further includes:
[0044] Using each occluding object sample image in the set of occluding object sample images to perform occlusion processing on each occluded object sample image, for example, using a random occlusion method to perform processing, so as to generate multiple occluding images;
[0045] According to the area of the occluded object occupied by the occluded object on each occlusion image and the area of the occluded object on each occlusion image, the occlusion ratio corresponding to the occluder sample image participating in generating each occlusion image is calculated;
[0046] According to the area of the occluder on each occluded image and the area of each image, the foreground proportion corresponding to the occluder sample image involved in generating each occluded image is calculated;
[0047] According to the calculated occlusion ratio and foreground ratio, the occurrence probability of the occlusion sample image under any occlusion ratio and any foreground ratio is statistically analyzed to obtain probability distribution information.
[0048] By pre-generating the probability distribution information of a set of occlusion sample images, a foundation can be laid for the subsequent rapid generation of images with a specific occlusion ratio.
[0049] Furthermore, according to the occlusion ratio and the foreground ratio, the occurrence probability of the occlusion sample image under any occlusion ratio and any foreground ratio is statistically analyzed to obtain probability distribution information, including:
[0050] According to the calculated prospect proportions, multiple prospect proportion intervals are obtained;
[0051] According to the calculated occlusion ratio, the occurrence probability of the occlusion sample image under any occlusion ratio in each foreground ratio interval is counted to obtain the probability distribution information.
[0052] Figure 2 This is a schematic diagram of probability distribution information of a set of occluder sample images provided in an embodiment of the present disclosure, which is used to exemplarily illustrate the probability distribution information in the embodiment of the present disclosure, but should not be understood as a specific limitation on the embodiment of the present disclosure. Figure 2 As shown, the occlusion ratio corresponding to the occlusion sample image can be expressed as O r , the foreground proportion of the occluder sample image can be expressed as F r , taking the foreground ratio as the horizontal axis and the occlusion ratio as the vertical axis. By dividing the foreground ratio into intervals, we can get multiple foreground ratio intervals, such as [F r0 , F r1 ),[F r1 , F r2 ),[F r2 , F r3 ),[F r3 , F r4 ),[F r4 , F r5 ),[F r5 , F r6 ),……[F rn-1 , F rn ], the step sizes of the foreground proportion intervals can be the same (i.e., the foreground proportion is divided at equal intervals) or different (i.e., the foreground proportion is divided at non-equal intervals), and the step size value can also be determined according to actual needs (the smaller the step size value, the higher the division accuracy of the foreground proportion interval). The embodiments of the present disclosure are not specifically limited. For example, the foreground proportion value ranges from 0 to 1, and the step size can be set to 0.1, and the foreground proportion is divided at equal intervals to obtain multiple foreground proportion intervals: [0, 0.1), [0.1, 0.2), [0.2, 0.3), [0.3, 0.4), ... [0.9, 1]. Then, for each foreground proportion interval, the probability of occurrence of the occluder sample image under any occlusion ratio is counted.
[0053] In addition, it should be noted that Figure 2 In the probability distribution information of the set of occlusion sample images shown, the visible ratio corresponding to the occlusion ratio, that is, 1-0 r , as the vertical axis. It can be determined adaptively in practical applications.
[0054] Figure 3A flowchart of another image generation method provided in an embodiment of the present disclosure is further optimized and expanded based on the above technical solution, and can be combined with the above optional implementations. Figure 3 and Figure 1 For the same operations, please refer to the description in the above embodiment, which will not be repeated below.
[0055] like Figure 3 As shown, the image generation method provided by the embodiment of the present disclosure may include:
[0056] S301. Obtain probability distribution information of a set of occluder sample images, wherein the probability distribution information is used to characterize the occurrence probability of the occluder sample images under any occlusion ratio and any foreground ratio.
[0057] S302 , using probability distribution information, determine, from a set of occluder sample images, occluder sample images that participate in generating an occluder image and that satisfy a target occluder ratio, as candidate occluder sample images.
[0058] Among them, the candidate occluder sample image is an available occluder sample image preliminarily determined based on the target occlusion ratio. Exemplarily, the candidate occluder sample image may include occluder sample images with a probability of occurrence corresponding to the target occlusion ratio and one or more foreground ratio intervals, so as to achieve preliminary screening of the occluder sample images. For example, if the required target occlusion ratio is 0.1, it can be determined based on the probability distribution information that the occluder sample image corresponding to the foreground ratio between 0.1-0.4 has a greater probability of occurrence, then the occluder sample image corresponding to the foreground ratio interval of 0.1-0.4 can be used as the candidate occluder sample image.
[0059] There is a certain regularity between the foreground ratio and the occlusion ratio. For example, when the foreground ratio is very large, the occlusion ratio of the occlusion image generated by the corresponding occlusion sample image is usually also very large, that is, it is almost impossible to generate an occlusion image with a smaller occlusion ratio using the occlusion sample image.
[0060] S303: Determine a target occluder image based on the occurrence probability of the candidate occluder sample images.
[0061] The target occluder image may be determined according to actual needs, for example, based on the occurrence probability, or based on both the required foreground proportion and the occurrence probability, to determine the target occluder image from the candidate occluder sample images.
[0062] Exemplarily, determining a target occluder image based on the occurrence probability of a candidate occluder sample image includes:
[0063] Determine the foreground proportion corresponding to the occluder sample image whose appearance probability in the candidate occluder sample image is greater than the preset probability threshold, as the target foreground proportion interval; wherein the preset probability threshold can be adaptively determined, and the embodiment of the present disclosure does not specifically limit it;
[0064] Selecting a target occluder image from occluder sample images corresponding to the target occlusion ratio and the target foreground ratio interval; for example, the target occluder image may be determined from occluder sample images corresponding to the target occlusion ratio and the target foreground ratio interval in a random or pre-specified manner; or
[0065] Exemplarily, determining the target occluder image based on the occurrence probabilities of the candidate occluder sample images includes: determining the occluder sample image with the highest occurrence probability among the candidate occluder sample images as the target occluder image.
[0066] S304: Based on the foreground proportion corresponding to the target occluder image, use the target occluder image to perform occlusion processing on the occluded object image to generate a target image.
[0067] S305 , calculating a generated occlusion ratio of the target image according to the area of the occluded object occupied by the occluded object on the target image and the area of the occluded object on the target image.
[0068] S306 : Verify the target image based on the difference between the generated occlusion ratio and the target occlusion ratio.
[0069] If the difference between the generated occlusion ratio and the target occlusion ratio is less than or equal to the preset difference threshold, the target image verification passes, that is, the currently generated target image meets the requirements, and the target image can continue to be used in subsequent image processing; if the difference between the generated occlusion ratio and the target occlusion ratio is greater than the preset difference threshold, the target image verification fails, that is, the currently generated target image does not meet the requirements, and the target image generation operation needs to be repeated.
[0070] Exemplarily, the absolute value of the difference between the generated occlusion ratio and the target occlusion ratio can be calculated as the difference between the generated occlusion ratio and the target occlusion ratio, and the quotient between the generated occlusion ratio and the target occlusion ratio can also be calculated as the difference between the generated occlusion ratio and the target occlusion ratio. The embodiments of the present disclosure are not specifically limited, and each difference calculation method corresponds to setting a corresponding difference threshold. The value of the difference threshold can also be determined adaptively.
[0071] Taking the calculation of the absolute value of the difference between the generated occlusion ratio and the target occlusion ratio as an example, the generated occlusion ratio can be expressed as s , the target occlusion ratio can be expressed as O r The absolute value of the difference between the two can be expressed as |Os -O r |, and then compare the absolute value with the difference threshold α. The value of the difference threshold α can be set to 10 -3 The smaller the difference threshold value is, the higher the accuracy of the generated occlusion ratio is.
[0072] Continuing with the above example, if |O s -O r |<α, that is, the generated occlusion ratio is within a reasonable error range, the target image is verified, and then the target image is output for storage; if |O s -O r |≥α, that is, the generated occlusion ratio exceeds the reasonable error range and the target image verification fails, then the target image generation operation is re-executed until the generated occlusion ratio calculated based on the newly generated target image is within the reasonable error range.
[0073] In the disclosed embodiment, the target foreground ratio interval corresponding to the occlusion sample images is initially screened by utilizing the probability distribution information of the set of occlusion sample images and the required target occlusion ratio. Then, the target occlusion image is further determined from the occlusion sample images corresponding to the target occlusion ratio and the target foreground ratio interval, thereby achieving targeted selection of the target occlusion image and improving the selection efficiency of the target occlusion image. Furthermore, by verifying the target image, it is ensured that the generated target image meets the occlusion ratio requirement.
[0074] Figure 4 A structural schematic diagram of an image generating device provided for an embodiment of the present disclosure can be applicable to the situation of how to generate an occlusion image with a specific occlusion ratio. The device can be implemented using software and / or hardware and can be integrated on any electronic device with computing capabilities, such as user terminals such as smartphones, tablet computers, notebooks or desktops, or servers, etc.
[0075] like Figure 4 As shown, the image generation device 400 provided in the embodiment of the present disclosure may include a probability distribution information acquisition module 401, a target occluder image determination module 402 and a target image generation module 403, wherein:
[0076] The probability distribution information acquisition module 401 is used to acquire the probability distribution information of the set of occluder sample images, wherein the probability distribution information is used to characterize the probability of occurrence of the occluder sample images under any occlusion ratio and any foreground ratio. The occlusion ratio is the ratio of the area of the occluded object occupied by the occluder on the occlusion image to the area of the occluded object. The foreground ratio refers to the ratio of the area of the occluder on the occlusion image to the area of the occlusion image. The occlusion image is generated with the participation of the occluder sample images.
[0077] A target occluder image determination module 402 is used to determine, from a set of occluder sample images, a target occluder image that participates in generating an occluder image and satisfies a target occluder ratio by using probability distribution information;
[0078] The target image generation module 403 is used to perform occlusion processing on the occluded object image using the target occluder image based on the foreground proportion corresponding to the target occluder image, so as to generate the target image.
[0079] Optionally, the target occluder image determination module 402 includes:
[0080] A candidate occluder sample image determination unit is used to determine, from the set of occluder sample images, an occluder sample image that participates in the generation of an occluder image and satisfies a target occluder ratio, as a candidate occluder sample image, by using probability distribution information;
[0081] The target occluder image determining unit is used to determine the target occluder image based on the occurrence probability of the candidate occluder sample image.
[0082] Optionally, the target occluder image determining unit includes:
[0083] A target foreground ratio interval determination subunit is used to determine the foreground ratio corresponding to the occluder sample images whose appearance probability is greater than a preset probability threshold in the candidate occluder sample images as the target foreground ratio interval;
[0084] A first target occluder image determination subunit is used to select a target occluder image from occluder sample images corresponding to a target occlusion ratio and a target foreground ratio interval; or
[0085] Optionally, the target occluder image determining unit includes:
[0086] The second target occluder image determination subunit is used to determine the occluder sample image with the highest probability of appearing in the candidate occluder sample images as the target occluder image.
[0087] Optionally, the target image generation module 403 includes:
[0088] A mask image acquisition unit, used to acquire an occluder mask image corresponding to the target occluder image, and an occluded object mask image corresponding to the occluded object image;
[0089] The target image generating unit is used to perform occlusion processing on the occluded object image using the target occluder image based on the foreground proportion, the occluder mask image and the occluded object mask image corresponding to the target occlusion image, so as to generate the target image.
[0090] Optionally, the image generating device 400 provided in the embodiment of the present disclosure further includes:
[0091] A generated occlusion ratio calculation module is used to calculate the generated occlusion ratio of the target image according to the area of the occluded object occupied by the occluded object on the target image and the area of the occluded object on the target image;
[0092] The target image verification module is used to verify the target image based on the difference between the generated occlusion ratio and the target occlusion ratio.
[0093] Optionally, the image generating device 400 provided in the embodiment of the present disclosure further includes:
[0094] The occlusion image generation module is used to perform occlusion processing on the occluded object sample image using the occlusion object sample image to generate multiple occlusion images;
[0095] An occlusion ratio calculation module is used to calculate the occlusion ratio corresponding to the occlusion object sample image participating in generating each occlusion image according to the area of the occluded object occupied by the occluded object on each occlusion image and the area of the occluded object on each occlusion image;
[0096] A foreground ratio calculation module is used to calculate the foreground ratio corresponding to the occluder sample image involved in generating each occluder image according to the area of the occluder on each occluder image and the area of each image;
[0097] The probability distribution information determination module is used to obtain probability distribution information by statistically analyzing the occurrence probability of the occlusion sample image under any occlusion ratio and any foreground ratio according to the calculated occlusion ratio and foreground ratio.
[0098] Optionally, the probability distribution information determination module includes:
[0099] A foreground proportion interval determination unit, used for obtaining a plurality of foreground proportion intervals according to the calculated foreground proportion;
[0100] The probability distribution information determination unit is used to count the occurrence probability of the occlusion object sample image under any occlusion ratio in each foreground proportion interval according to the calculated occlusion ratio, so as to obtain probability distribution information.
[0101] Optionally, the image generating device 400 provided in the embodiment of the present disclosure further includes:
[0102] The model training module is used to use the target image as a training sample and obtain the required image processing model based on the training sample.
[0103] The image generation device provided in the embodiment of the present disclosure can execute any image generation method provided in the embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. The contents not described in detail in the embodiment of the device of the present disclosure can refer to the description in any method embodiment of the present disclosure.
[0104] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure is used to exemplify an electronic device that implements the image generation method provided in an embodiment of the present disclosure. The electronic device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, wearable electronic devices, servers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and occupation scope of the embodiments of the present disclosure.
[0105] like Figure 5 As shown, the electronic device 500 includes one or more processors 501 and a memory 502 .
[0106] The processor 501 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 500 to perform desired functions.
[0107] The memory 502 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and the processor 501 may run the program instructions to implement the image generation method provided by the embodiment of the present disclosure, and may also implement other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
[0108] The image generation method provided by the embodiment of the present disclosure may include: obtaining probability distribution information of a set of occluder sample images, wherein the probability distribution information is used to characterize the probability of occurrence of the occluder sample images under any occlusion ratio and any foreground ratio, the occlusion ratio is the ratio of the area of the occluded object occupied by the occluder on the occlusion image to the area of the occluded object, the foreground ratio refers to the ratio of the area of the occluder on the occlusion image to the area of the occlusion image, and the occlusion image is generated by the occluder sample images; using the probability distribution information, determining the target occluder image that meets the target occlusion ratio from the set of occluder sample images; based on the foreground ratio corresponding to the target occluder image, using the target occluder image to perform occlusion processing on the occluded image to generate the target image. It should be understood that the electronic device 500 can also execute other optional implementation schemes provided by the embodiment of the method of the present disclosure.
[0109] In one example, the electronic device 500 may further include: an input device 503 and an output device 504 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0110] In addition, the input device 503 may also include, for example, a keyboard, a mouse, and the like.
[0111] The output device 504 can output various information to the outside, including the determined distance information, direction information, etc. The output device 504 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.
[0112] Of course, to simplify, Figure 5 Only some of the components related to the present disclosure in the electronic device 500 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 500 may also include any other appropriate components.
[0113] In addition to the above-mentioned methods and devices, the embodiments of the present disclosure also provide a computer program product, which includes a computer program or computer program instructions. When the computer program or computer program instructions are executed by a computing device, the computing device implements any image generation method provided by the embodiments of the present disclosure.
[0114] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user electronic device, partially on the user electronic device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device.
[0115] In addition, the embodiments of the present disclosure may also provide a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a computing device, the computing device implements any image generation method provided by the embodiments of the present disclosure.
[0116] Among them, the image generation method provided by the embodiment of the present disclosure may include: obtaining probability distribution information of a set of occluder sample images, wherein the probability distribution information is used to characterize the probability of occurrence of the occluder sample images under any occlusion ratio and any foreground ratio, the occlusion ratio is the ratio of the area of the occluded object occupied by the occluder on the occlusion image to the area of the occluded object, and the foreground ratio refers to the ratio of the area of the occluder on the occlusion image to the area of the occlusion image, and the occlusion image is generated by the occluder sample images; using the probability distribution information, determining the target occluder image that meets the target occlusion ratio from the set of occluder sample images; based on the foreground ratio corresponding to the target occluder image, using the target occluder image to perform occlusion processing on the occluded image to generate the target image. It should be understood that when the computer program instructions are executed by the computing device, they can also enable the computing device to implement other optional implementation schemes provided by the embodiment of the method of the present disclosure.
[0117] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0118] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0119] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An image generation method, characterized in that: include: Obtaining probability distribution information of a set of occluder sample images, wherein the probability distribution information is used to characterize the occurrence probability of the occluder sample images at any occlusion ratio in each foreground ratio interval, the occlusion ratio is the ratio of the area of the occluded object occupied by the occluder on the occlusion image to the area of the occluded object, the foreground ratio is the ratio of the area of the occluder on the occlusion image to the area of the occlusion image, and the occlusion image is generated with the participation of the occluder sample images; Determine, by using the probability distribution information, from the set of occluder sample images, a target occluder image that participates in generating an occluder image and satisfies a target occluder ratio; Based on the foreground proportion corresponding to the target occluder image, the target occluder image is used to perform occlusion processing on the occluded object image to generate a target image.
2. The method according to claim 1, characterized in that: The method of using the probability distribution information to determine, from the set of occluder sample images, a target occluder image that participates in generating an occluder image and satisfies a target occluder ratio includes: Using the probability distribution information, determine, from the set of occluder sample images, occluder sample images that participate in generating an occluder image and that satisfy a target occluder ratio as candidate occluder sample images; The target occluder image is determined based on the occurrence probability of the candidate occluder sample image.
3. The method according to claim 2, characterized in that The step of determining the target occluder image based on the occurrence probability of the candidate occluder sample image comprises: Determine the foreground proportion corresponding to the occluder sample images whose occurrence probability is greater than a preset probability threshold in the candidate occluder sample images as the target foreground proportion interval; Selecting the target occluder image from the occluder sample images corresponding to the target occlusion ratio and the target foreground ratio interval; or The step of determining the target occluder image based on the occurrence probability of the candidate occluder sample image comprises: The occluder sample image with the highest occurrence probability among the candidate occluder sample images is determined as the target occluder image.
4. The method according to claim 1, characterized in that The method of performing occlusion processing on the occluded object image based on the foreground proportion corresponding to the target occluder image and generating the target image by using the target occluder image includes: Acquire an occluder mask image corresponding to the target occluder image and an occluded object mask image corresponding to the occluded object image; Based on the foreground proportion corresponding to the target occluder image, the occluder mask image and the occluded object mask image, the occluded object image is occluded using the target occluder image to generate the target image.
5. The method according to any one of claims 1 to 4, characterized in that: After performing occlusion processing on the occluded image based on the foreground proportion corresponding to the target occluder image and generating the target image using the target occluder image, the method further includes: Calculating a generated occlusion ratio of the target image according to the area of the occluded object occupied by the occluded object on the target image and the area of the occluded object on the target image; The target image is verified based on a difference between the generated occlusion ratio and the target occlusion ratio.
6. The method according to claim 1, characterized in that Also includes: Using the obstruction sample image to perform occlusion processing on the obstructed object sample image to generate multiple occlusion images; Calculate the occlusion ratio corresponding to the occlusion sample image participating in generating each occlusion image according to the area of the occluded object occupied by the occluder on each occlusion image and the area of the occluded object on each occlusion image; Calculating the foreground proportion corresponding to the occluder sample image involved in generating each occluded image according to the area of the occluder on each occluded image and the area of each occluded image; According to the occlusion ratio and the foreground proportion, the occurrence probability of the occlusion object sample image under any occlusion ratio and any foreground proportion is statistically analyzed to obtain the probability distribution information.
7. The method according to claim 6, characterized in that The method of obtaining the probability distribution information by performing statistics on the occurrence probability of the occlusion sample image under any occlusion ratio and any foreground ratio according to the occlusion ratio and the foreground ratio includes: According to the prospect proportion, a plurality of prospect proportion intervals are obtained; According to the occlusion ratio, the occurrence probability of the occlusion sample image under any occlusion ratio in each foreground proportion interval is counted to obtain the probability distribution information.
8. The method according to claim 1, characterized in that Also includes: The target image is used as a training sample, and the required image processing model is obtained by training based on the training sample.
9. An image generating device, characterized in that: include: A probability distribution information acquisition module, used to acquire probability distribution information of a set of occluder sample images, wherein the probability distribution information is used to characterize the occurrence probability of the occluder sample images under any occlusion ratio in each foreground ratio interval, the occlusion ratio is the ratio of the area of the occluded object occupied by the occluder on the occlusion image to the area of the occluded object, the foreground ratio refers to the ratio of the area of the occluder on the occlusion image to the area of the occlusion image, and the occlusion image is generated with the participation of the occluder sample images; A target occluder image determination module is used to determine, from the set of occluder sample images, a target occluder image that participates in generating an occlusion image and satisfies a target occlusion ratio by using the probability distribution information; The target image generation module is used to perform occlusion processing on the occluded object image using the target occluder image based on the foreground proportion corresponding to the target occluder image, so as to generate a target image.
10. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device implements the image generation method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the image generation method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Method for reliably generating video abstraction in complex scene
CN103413330A
Human face shielding degree recognition method and device, electronic equipment and readable storage medium
CN111414879A
Improvements relating to face recognition
EP3428843A1
