Image processing method and device, storage medium and electronic equipment

Abstract

The invention discloses an image processing method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring a depth interval of a target object in a first image; obtaining a first probability density that the interval upper limit of the target object in the depth interval meets the effective display condition, and obtaining a second probability density that the interval lower limit of the target object in the depth interval meets the effective display condition; on the basis of the probability density difference between the first probability density and the second probability density, N ordered depth points are determined from a depth interval, and the probability density difference values corresponding to every two adjacent depth points are the same; according to the ordered N depth points, performing image sampling of different depths on the target object from the first image to obtain N image planes corresponding to the ordered N depth points; and performing image synthesis by using the N image planes to obtain a synthesized second image. The technical problem that the image processing effect is poor is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the computer field, in particular to an image processing method and device, a storage medium and an electronic device. BACKGROUND

[0002] The multi-plane sampling technology is an image processing manner for synthesizing a complete image by sampling multiple image planes at different depths, so as to improve the display effect of the image.

[0003] In the related art, in the process of multi-plane sampling of a target object, the multiple images sampled at different depths cannot effectively display the content of the target object, which results in poor quality of the finally synthesized image, for example, the synthesized image has defects such as blurred ghosting and artifacts, and thus the image processing effect is poor.

[0004] Therefore, the related art has the technical problem of poor image processing effect. SUMMARY

[0005] Embodiments of the present application provide an image processing method, device, storage medium and electronic device to at least solve the technical problem of low image processing efficiency in the related art.

[0006] According to an aspect of an embodiment of the present application, an image processing method is provided, comprising: obtaining a depth interval in which a target object is located in a first image; obtaining a first probability density that an upper limit of the depth interval of the target object satisfies an effective display condition, and obtaining a second probability density that a lower limit of the depth interval of the target object satisfies the effective display condition; determining N ordered depth points from the upper limit of the depth interval based on a probability density difference between the first probability density and the second probability density, wherein the probability density difference of adjacent two depth points in the N ordered depth points is the same, and N is a positive integer; performing image sampling at different depths in the first image according to the N ordered depth points, to obtain N image planes corresponding to the N ordered depth points; and performing image synthesis using the N image planes to obtain a second synthesized image.

[0007] According to another aspect of the embodiments of the present application, an image processing apparatus is also provided, which comprises: a first obtaining unit, configured to obtain a depth interval in which a target object is located in a first image; a second obtaining unit, configured to obtain a first probability density that an upper limit of the depth interval satisfies an effective display condition, and obtain a second probability density that a lower limit of the depth interval satisfies the effective display condition; a first determining unit, configured to determine, based on a probability density difference between the first probability density and the second probability density, N ordered depth points from the depth interval, wherein adjacent two depth points in the N ordered depth points correspond to a same probability density difference value, and N is a positive integer; a second determining unit, configured to sample images at different depths in the first image according to the N ordered depth points, to obtain N image planes corresponding to the N ordered depth points; and a synthesizing unit, configured to perform image synthesis using the N image planes, to obtain a second image after synthesis.

[0008] As an optional solution, the first determining unit comprises: a first determining module, configured to determine, based on the probability density difference, N ordered probability densities, wherein adjacent two probability densities in the N ordered probability densities correspond to a same probability density difference value, and a first probability density in the N ordered probability densities is the second probability density; and a second determining module, configured to sequentially obtain depth points corresponding to the N ordered probability densities, to obtain the N ordered depth points.

[0009] As an optional solution, the first determining module comprises: an obtaining sub-module, configured to obtain a probability density average difference, wherein the probability density average difference is obtained by subtracting the second probability density from the first probability density and then dividing the result by N; and a determining sub-module, configured to determine, based on the probability density average difference and the first probability density, the N ordered probability densities, wherein an i-th probability density in the N ordered probability densities is obtained by adding, to the first probability density, a product of the probability density average difference and i-1, i being a positive integer less than or equal to N.

[0010] As an optional solution, the second determining module comprises: a matching sub-module, configured to match, using a depth center distance of the target object in the first image and an inverse error function, depth points corresponding to the N ordered probability densities, to obtain the N ordered depth points, wherein an i-th depth point corresponding to an i-th probability density in the N ordered probability densities is obtained by adding, to the depth center distance, a first output value, the first output value being obtained by multiplying a first preset parameter by a second output value, the second output value being an output value obtained by inputting the i-th probability density into the inverse error function, i being a positive integer less than or equal to N.

[0011] As an optional solution, the first obtaining unit comprises a first obtaining module configured to obtain an upper limit of an interval and a lower limit of the interval in which the target object is located in the first image, wherein the target object is displayed in the first image at a first view angle; and a synthesizing unit comprising a synthesizing module configured to perform new view angle synthesis using the N image planes to obtain a second image, wherein the target object is displayed in the second image at a second view angle.

[0012] As an optional solution, the synthesizing module comprises an input sub-module configured to input the N image planes, the first image and the camera parameters into a view angle synthesis model to obtain the second image output by the view angle synthesis model, wherein the view angle synthesis model is a neural network model pre-trained for new view angle synthesis, and the camera parameters are used to indicate the view angle parameters corresponding to the second view angle.

[0013] As an optional solution, the second obtaining unit comprises a second obtaining module configured to obtain a first distance difference between the upper limit of the interval and a depth center distance of the target object in the first image; and a third obtaining module configured to input the first distance difference into an error function to obtain a third output value, and adjust the third output value using a second preset parameter to obtain a first probability density; and the second obtaining unit further comprises a fourth obtaining module configured to obtain a second distance difference between the lower limit of the interval and the depth center distance; and a fifth obtaining module configured to input the second distance difference into the error function to obtain a fourth output value, and adjust the fourth output value using the second preset parameter to obtain a second probability density.

[0014] According to still another aspect of the embodiments of the present application, a computer program product is provided, which includes computer programs / instructions stored in a computer readable storage medium. A processor of a computer device reads the computer programs / instructions from the computer readable storage medium, and the processor executes the computer programs / instructions, so that the computer device performs the image processing method as above.

[0015] According to still another aspect of the embodiments of the present application, an electronic device is also provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the image processing method as above through the computer program.

[0016] In the embodiments of the present application, according to the depth interval in which the target object is located in the first image, the first probability density that the upper limit of the depth interval of the target object satisfies the effective display condition and the second probability density that the lower limit of the depth interval of the target object satisfies the effective display condition are determined, and based on the probability density difference between the first probability density and the second probability density, the N ordered depth points are determined from the upper limit of the depth interval according to the standard of equal interval division of the probability density (i.e., the adjacent two probability density difference values are the same). According to the above-mentioned N ordered depth points, image sampling at different depths is performed in the first image to obtain the corresponding N image planes, wherein the N image planes are used for image synthesis to obtain the second image.

[0017] In this way, the N image planes corresponding to the N depth points are sampled according to the standard of equal interval division of the probability density, wherein in the sampling process of the N depth points, for the first region (such as the center of the target object) in which the probability density of the target object satisfying the effective display condition is large and the second region (such as the edge of the target object) in which the probability density of the target object satisfying the effective condition is small, since the probability density is equally divided (i.e., the integral of the probability of the target object satisfying the effective display condition on the depth distance is equal), the depth distance difference of two samplings for the former first region will be small, which is equivalent to dense sampling, and the depth distance difference of two samplings for the latter second region will be large, which is equivalent to sparse sampling, thereby achieving the purpose of dense sampling in the region close to the center of the target object (large probability density) and sparse sampling in the region far from the center of the target object (small probability density), so that the multiple images sampled at different depths can effectively display the content of the target object, thereby realizing the technical effect of improving the image quality based on the image synthesis processing of the above-mentioned multiple images, and solving the technical problem of poor image processing effect in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings used to provide further understanding of the present application and constitute a part of the present application, the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute improper limitation on the present application. In the drawings:

[0019] Figure 1 is a schematic diagram of an application environment of an optional image processing method according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of the flow of an optional image processing method according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an optional image processing method according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of an optional image processing method according to an embodiment of the application;

[0023] Figure 5 is a schematic diagram of an optional image processing method according to an embodiment of the application;

[0024] Figure 6 is a schematic diagram of an optional image processing device according to an embodiment of the application;

[0025] Figure 7 is a structural schematic diagram of an optional electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0026] In order to enable persons skilled in the art to better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] According to an aspect of an embodiment of the present application, an image processing method is provided. Optionally, as an optional implementation, the above-mentioned image processing method can be applied in, but is not limited to, an environment as shown in Figure 1 , which can include, but is not limited to, a client 102 and a server 112. The client 102 can include, but is not limited to, a display 104, a processor 106 and a memory 108. The server 112 includes a database 114 and a processing engine 116.

[0029] The specific process can be as follows:

[0030] In step S102, the client 102 obtains an image processing request, wherein the image processing request is used to request multi-plane sampling in the first image and to synthesize a second image based on the multi-plane sampling results;

[0031] In steps S104-S106, client 102 sends an image processing request to server 112;

[0032] In step S108, server 112 responds to the image processing request, obtains the depth range of the target object in the first image, and obtains the first probability density that the upper limit of the depth range satisfies the effective display condition and the second probability density that the lower limit of the depth range satisfies the effective display condition.

[0033] In step S110, server 112 determines N ordered depth points from the depth interval based on the probability density difference between the first probability density and the second probability density, wherein the probability density difference between two adjacent depth points in the ordered N depth points is the same.

[0034] In step S112, server 112 performs image sampling at different depths in the first image according to the ordered N depth points to obtain N image planes corresponding to the ordered N depth points; using the N image planes, image synthesis is performed to obtain the synthesized second image.

[0035] Steps S114-S116: The second image and its image information are sent to the client 102 via the network 110. The processor 106 in the client 102 is used to receive the second image and process its image information, and to store the processed image data in the memory 108.

[0036] remove Figure 1 Beyond the examples shown, the above steps can be completed independently by the client or server, or collaboratively by both, such as by client 102 executing steps S108 to S112, thereby reducing the processing load on server 112. Client 102 includes, but is not limited to, laptops, tablets, desktop computers, smart TVs, etc., and this application does not limit the specific implementation of client 102. Server 112 can be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0037] Alternatively, as an optional implementation, such as Figure 2 As shown, image processing methods can be executed by electronic devices, such as... Figure 1 The client or server shown includes the following specific steps:

[0038] S202, Obtain the depth range of the target object in the first image;

[0039] S204, obtaining a first probability density that the upper limit of the depth interval of the target object satisfies the effective display condition, and obtaining a second probability density that the lower limit of the depth interval of the target object satisfies the effective display condition; S206, determining, based on a probability density difference between the first probability density and the second probability density, N ordered depth points from the depth interval, wherein adjacent two depth points in the N ordered depth points correspond to the same probability density difference value, and N is a positive integer;

[0040] S208, performing image sampling at different depths in the first image according to the N ordered depth points to obtain N image planes corresponding to the N ordered depth points;

[0041] S210, performing image synthesis using the N image planes to obtain a synthesized second image.

[0042] Optionally, in the embodiment, the image processing method can be applied in a new view synthesis scene of an image, but is not limited to this. In the scene, N depth values in the depth interval are determined based on the probability density of the depth interval and the related depth position (such as the upper limit of the depth interval and the lower limit of the depth interval) of the target object in the first image, and then the first image is divided into N image planes based on the N depth values. Then, based on the N image planes, the first image corresponding to the N image planes, and the camera parameters, new view synthesis is performed to synthesize a second image of a new view. The camera parameters indicate the view parameters of the new view and are used to determine the new view. The information redundancy of the multi-plane image can be inhibited, the information effectiveness of the plane image can be improved, and thus the algorithm accuracy and visual effect of the view synthesis can be improved.

[0043] It should be noted that the image processing method can also be applied in other image processing scenes requiring multi-plane sampling, such as image resolution improvement processing and image quality improvement processing, but is not limited to this. The embodiment is not limited in this regard.

[0044] Optionally, in the embodiment, the target object can be, but is not limited to, a subject object in the first image. The depth interval of the target object in the first image can be obtained by a computer vision algorithm, such as edge detection, corner detection, and the like, to obtain the depth information of the target object in the first image, including the upper limit of the depth interval (the farthest depth of the target object from the camera when the first image is captured), the lower limit of the depth interval (the closest depth of the target object from the camera when the first image is captured), and the depth center distance of the target object (the distance of the center position of the target object from the camera when the first image is captured).

[0045] It should be noted that the probability of the target object being effectively displayed in the image plane corresponding to different depths in the first image is different, and the closer the distance from the depth center of the target object, the higher the probability of effective display.

[0046] It can be understood that the probability of the target object being effectively displayed in the first image conforms to the standard normal distribution modeling rule. As shown in formula (1), d is the depth d of the first image, D is the depth center distance of the target object in the first image, f(d) is the probability of the target object being effectively displayed at depth d, and the effective display condition of the target object at d is used to indicate that the target object is effectively displayed at depth d, which can be but is not limited to indicating that the target object completely appears at depth d.

[0047]

[0048] Optionally, in the embodiment, the cumulative distribution function is used to study the distribution characteristics of the above-mentioned probability f(d). In the embodiment, the probability density of the target object satisfying the effective display condition at each depth point in the depth interval can be but is not limited to calculated and determined based on the cumulative distribution function, including the first probability density of the target object satisfying the effective display condition at the upper limit of the interval of the depth interval, and the second probability density of the target object satisfying the effective display condition at the lower limit of the interval of the depth interval.

[0049] Optionally, in the embodiment, the cumulative distribution function can be but is not limited to as shown in formula (2), wherein Φ(x) is the probability density corresponding to depth x, and erf(x) is the error function.

[0050]

[0051] Optionally, in the embodiment, the error function erf(x) can be but is not limited to as shown in formula (3):

[0052]

[0053] It can be understood that for the first probability density cdf max of the target object at the upper limit d max of the interval of the depth interval, the calculation formula is For the second probability density cdf min of the target object at the lower limit d min of the interval of the depth interval, the calculation formula is

[0054] It should be noted that the first probability density is an upper bound of the cumulative distribution function, the second probability density is a lower bound of the cumulative distribution function, the probability density between the upper bound and the lower bound of the cumulative distribution function is sampled at equal intervals to obtain N probability densities, any two adjacent probability densities in the N probability densities have the same difference value, and then N depth points corresponding to the N probability densities are determined based on the N probability densities, wherein the relationship between the depth point and the probability density is shown in formula (4), wherein d i is the i-th depth point, D is the depth center distance of the target object in the first image, cdf i is the i-th probability density, erf -1 (x) is the inverse error function, and the inverse error function is the inverse function of the error function.

[0055]

[0056] It can be understood that the N depth points are ordered, wherein for the depth region a1 close to the depth center distance D and the depth region a2 far from the depth center distance D, since f(d) corresponding to the depth region a1 is larger and f(d) corresponding to the depth region a2 is smaller, and the probability density interval corresponding to the two is the same, that is, Φ(x) is the same, and since the sampling distance interval corresponding to the depth region a1 is smaller than the sampling distance interval corresponding to the depth region a2, and then dense sampling is achieved in the place with large probability density and sparse sampling is achieved in the place with small probability density, so that sampling of the depth space according to the probability density is realized.

[0057] Optionally, in the embodiment, according to the N depth points, image sampling of different depths is performed in the first image to obtain N image planes corresponding to the ordered N depth points, and image synthesis is performed using the N image planes to obtain a synthesized second image.

[0058] According to the embodiment provided in the present application, the N image planes corresponding to the N depth points of the target object are sampled according to the standard of equally spaced probability density. During the sampling of the N depth points, for the first region (such as the center of the target object) with large probability density satisfying the effective display condition of the target object and the second region (such as the edge of the target object) with small probability density satisfying the effective condition, since the probability density is equally spaced (i.e., the integral of the probability of the target object satisfying the effective display condition over the depth distance is equal), the depth distance difference of the two samplings for the former first region will be small, which is equivalent to dense sampling, and the depth distance difference of the two samplings for the latter second region will be large, which is equivalent to sparse sampling, thereby achieving the purpose of dense sampling in the region close to the center of the target object (large probability density) and sparse sampling in the region far from the center of the target object (small probability density), so that the multiple images sampled at different depths can effectively display the content of the target object, thereby realizing the technical effect of improving the image quality based on the synthesis processing of the above multiple images.

[0059] As an optional solution, the ordered N depth points are determined from the depth interval based on the probability density difference between the first probability density and the second probability density, comprising:

[0060] S1, determining the ordered N probability densities based on the probability density difference, wherein the difference between adjacent two probability densities in the ordered N probability densities is the same, and the first probability density in the N probability densities is the second probability density;

[0061] S2, sequentially obtaining the depth points corresponding to the ordered N probability densities to obtain the ordered N depth points.

[0062] Optionally, in the present embodiment, the probability density difference can be but is not limited to indicating the probability density between the upper bound and the lower bound of the cumulative distribution function, wherein the upper bound of the cumulative distribution function is the first probability density cdf max , and the lower bound of the cumulative distribution function is the second probability density cdf min .

[0063] Optionally, in the present embodiment, the ordered N probability densities are determined from the probability density between the upper bound and the lower bound of the cumulative distribution function, wherein the difference between adjacent two probability densities is the same, and the first probability density is the second probability density cdf min .

[0064] It should be noted that the depth points corresponding to the ordered N probability densities are obtained to obtain N depth points, wherein one depth point corresponds to one probability density, and one probability density corresponds to one probability density, that is, one depth point corresponds to one probability density.

[0065] Through the embodiments provided in the present application, by equally dividing the probability density, the purpose of densely sampling in the area close to the center of the target object (large probability density) and sparsely sampling in the area far from the center of the target object (small probability density) is achieved, so that the multiple images sampled at different depths can effectively display the content of the target object, thereby achieving the technical effect of improving the image quality based on the above multiple image synthesis processing.

[0066] As an optional solution, the ordered N probability densities are determined based on the probability density difference, including:

[0067] S1, obtaining a probability density average difference, wherein the probability density average difference is obtained by subtracting the second probability density from the first probability density and then dividing by N;

[0068] S2, determining the ordered N probability densities according to the probability density average difference and the first probability density, wherein the i-th probability density in the ordered N probability densities is obtained by adding the probability density average difference multiplied by i-1 to the first probability density, i is a positive integer less than or equal to N.

[0069] Optionally, in the present embodiment, the calculation formula of the probability density average difference is shown in formula (5), wherein Δcdf is the probability density average difference, cdf max is the first probability density, cdf min is the second probability density.

[0070] Δcdf=(cdf max -cdf min ) / N (5)

[0071] Optionally, in the present embodiment, the calculation formula of the i-th probability density is shown in formula (6), wherein cdf i is the i-th probability density, cdf min is the second probability density, and Δcdf is the probability density average difference.

[0072] cdf i =cdf min +Δcdf*(i-1) (6)

[0073] It can be understood that N is the total number of image planes to be sampled, and i is a positive integer less than N.

[0074] As an optional solution, the depth points corresponding to the ordered N probability densities are sequentially obtained, to obtain the ordered N depth points, including:

[0075] S1, matching the N ordered probability densities respectively corresponding depth points by using the depth center distance of the target object in the first image and the inverse error function, to obtain N ordered depth points, wherein the i-th depth point corresponding to the i-th probability density in the N ordered probability densities is obtained by adding the first output value to the depth center distance, the first output value is obtained by using the first preset parameter multiplied by the second output value, the second output value is an output value obtained by inputting the i-th probability density into the inverse error function, and i is a positive integer less than or equal to N.

[0076] Optionally, in the embodiment, the i-th depth point is obtained in the manner shown in formula 7, wherein d i is the depth value corresponding to the i-th depth point, D is the depth center distance of the target object in the first image, is the first preset parameter, cdf i is the i-th probability density, erf -1 (x) is the inverse error function, and the inverse error function is the inverse function of the error function erf(x).

[0077]

[0078] It can be understood that in the above formula (7), d corresponds to the first output value, corresponds to the first preset parameter, and erf -1 (cdf i ) corresponds to the second output value.

[0079] As an optional solution, the depth interval in which the target object is located in the first image is obtained, including:

[0080] S1, obtaining the upper limit and the lower limit of the interval in which the target object is located in the first image, wherein the target object is displayed in the first image at a first view angle;

[0081] Using N image planes, image synthesis is performed to obtain a second image after synthesis, including:

[0082] S2, using N image planes, new view angle synthesis is performed to obtain a second image, wherein the target object is displayed in the second image at a second view angle.

[0083] Optionally, in the embodiment, the depth interval in which the target object is located in the first image can include, but is not limited to, obtaining the upper limit of the interval and the lower limit of the interval in which the target object is located in the first image. The upper limit of the interval and the lower limit of the interval can be obtained by, but are not limited to, a trained neural network model (such as a convolutional neural network CNN, a generative adversarial network GAN). For example, the trained neural network model can be used to perform depth estimation tasks for the first image, and output the upper limit and the lower limit of the interval in which the target object is located in the first image. It should be noted that in the depth estimation task, the trained neural network model can output the depth center distance of the target object in the first image in addition to the upper limit and the lower limit of the interval.

[0084] Optionally, in the embodiment, the upper limit and the lower limit of the interval and the depth center distance can be obtained by, but are not limited to, using image segmentation and semantic segmentation technology to separate the target object (subject) in the first image from the background, and then applying monocular vision or binocular vision to the segmented target object to calculate the depth information of the target object to obtain the upper limit and the lower limit of the interval and the depth center distance.

[0085] Optionally, in the embodiment, N image planes are used to perform new view synthesis to obtain a second image, in which the target object is displayed at a second view angle.

[0086] Optionally, in the embodiment, the new view synthesis process is used to indicate that the existing image is sampled by using computer vision technology and / or computer graphics technology, and a new view image is created by combining the input camera parameters.

[0087] It can be understood that in the new view synthesis process, the multiple image planes obtained based on the multi-plane sampling, the original images corresponding to the multiple image planes, and the camera parameters are used to perform image synthesis to obtain a new view image, wherein the camera parameters indicate the view angle parameters of the new view and are used to determine the new view.

[0088] Through the embodiments provided in the present application, the target object is displayed at a first view angle in the first image and at a second view angle in the second image, and the second image is obtained by performing new view synthesis based on the N image planes obtained by sampling after the equal interval division of the probability density. The target object is densely sampled in the region close to the center (high probability density) and sparsely sampled in the region far from the center (low probability density), so that the multiple images obtained by sampling at different depths can effectively display the content of the target object, thereby achieving the technical effect of improving the quality of new view synthesis of the image.

[0089] As an optional solution, a second image is obtained by using N image planes to perform new view synthesis, comprising:

[0090] S1, inputting the N image planes, the first image and the camera parameters into a view synthesis model to obtain a second image output by the view synthesis model, wherein the view synthesis model is a pre-trained neural network model for new view synthesis, and the camera parameters are used to indicate the view parameters corresponding to the second view.

[0091] Optionally, in the embodiment, the view synthesis model is a pre-trained neural network model for new view synthesis. For example, the view synthesis model can be, but is not limited to, a view synthesizer SVSMI, and can also be, but is not limited to, other types of neural network models, such as a convolutional neural network (CNN) and a generative adversarial network (GAN) model, and the embodiment does not make additional limitations on this.

[0092] It can be understood that the N image planes P obtained by sampling, the first image I and the camera parameters C representing the new view are input into the view synthesizer SVSMI to obtain the image I out :

[0093] I out =SVSMI(P,I,C)

[0094] As an optional solution, a first probability density that an upper limit of the depth interval of the target object satisfies the effective display condition is obtained, comprising:

[0095] Obtaining a first distance difference between the upper limit of the interval and the distance between the depth center and the target object in the first image;

[0096] Inputting the first distance difference into an error function to obtain a third output value, and adjusting the third output value using a second preset parameter to obtain the first probability density;

[0097] A second probability density that a lower limit of the depth interval of the target object satisfies the effective display condition is obtained, comprising:

[0098] Obtaining a second distance difference between the lower limit of the interval and the distance between the depth center;

[0099] Inputting the second distance difference into an error function to obtain a fourth output value, and adjusting the fourth output value using a second preset parameter to obtain the second probability density.

[0100] Optionally, in the embodiment, assuming that the upper limit of the depth interval of the target object is d max , the lower limit of the depth interval is d min , and the distance between the depth center in the first image is D, then the first distance difference is (d max-D), the second distance difference is (d min -D).

[0101] Optionally, in the embodiment, the calculation formulas of the first probability density and the second probability density are shown in formulas (8) and (9), where cdf max is the first probability density, cdf min is the second probability density, and erf(x) is an error function.

[0102]

[0103] It can be understood that in the formula (8), the corresponds to the third output value, in the formula (9), the corresponds to the fourth output value, and 0.5 and 1 in the formulas (8) and (9) correspond to the second preset parameter.

[0104] As an optional solution, the image processing method is applied to a Gaussian multi-plane sampling scenario applied to new view synthesis. In the scenario, for a general image, a subject (i.e., the target object, also referred to as a target or a target object hereinafter) is generally included, such as a person in a portrait, a table in an object photograph, or a mountain in a landscape. The subject is at a certain distance from the camera and also has a certain distance from the background, so it presents an approximately unimodal distribution in the depth space. Therefore, although the target has a thickness that cannot be covered by a single depth plane, the distribution of the entire effective content of the target in the depth space can be approximately regarded as a unimodal distribution, with the most effective content at the central depth of the subject and gradually decreasing effective content away from the central depth. This is very similar to the shape of a Gaussian distribution, with the probability density being the largest near the central position and gradually decreasing away from the central position. Therefore, multi-plane sampling with the central position being the depth of the subject and the sampling probability being the probability density of the Gaussian distribution can make the limited multi-planes cut as much target content as possible.

[0105] Based on the image processing method, the embodiment utilizes the priori that a photographed target generally presents an approximately unimodal distribution in the depth space, samples the depth planes near the target according to the probability density of the Gaussian distribution, thereby avoiding sampling planes from sampling redundant and invalid depth content, achieving accurate and efficient new view synthesis, and improving the quality of view synthesis.

[0106] Further, a new view synthesis method based on the image processing method is illustrated in a schematic diagram as shown in FIG. 6. Figure 3As shown, including: based on the input reference image data, the target depth corresponding Gaussian distribution probability density, sampling target in the reference image of discrete multi-depth plane, and as a new view synthesizer input, to get the new view synthesis results.

[0107] Specifically, the depth upper and lower bounds [d min ,d max ] of the reference image and the target center depth D (the center depth of the target in the reference image) are obtained. min and d max Do not need to be too accurate but need to cover the real depth range of the target, and the target center depth D also does not need to be too accurate but needs to basically meet its position in the upper and lower bounds. The above three parameters are hyperparameters, which can be manually adjusted according to the content of the input image.

[0108] According to the target center depth D of the target, the probability density of the depth distribution of the target is modeled according to the standard normal distribution:

[0109]

[0110] Where f(d) represents the probability of the target appearing completely at depth d, wherein f(d) is maximum at the estimated target center depth D and gradually decreases away from the target center depth D.

[0111] According to the above obtained probability density function, it is hoped to densely sample at high probability and sparsely sample at low probability. To this end, first calculate the upper and lower bounds [cdf min ,cdf max ] of the cumulative distribution function Φ(d) of f(d) in the depth interval [d min ,d max ]:

[0112]

[0113] Where is the error function, and its relationship with the integral cumulative distribution function Φ of the standard normal distribution is Then sample equally in the interval [cdf min ,cdf max ] to obtain the value of the cumulative distribution function of the desired sampling depth value i=1,2,…,N, where N is the total number of planes to be sampled, which is set to 24 in the example of the present scheme. Then the corresponding depth value can be obtained according to the value of cdf:

[0114]

[0115] Where, erf -1(x) is the inverse function of erf(x). When d i With d i+1 When near the center of the target, f(d) i ) and f(d i+1 The size is relatively large, because at this time Since f(d) is in the interval [d] i ,d i+1 The value of ] is relatively large, so when sampling CDF at equal intervals, d i With d i+1 The interval is relatively small; similarly, when d i With d i+1 When the distance from the center of the target is far, d i With d i+1 The spacing is relatively large. Each time, dense sampling is performed where the probability density is high, and sparse sampling is performed where the probability density is low, thus achieving sampling of the depth space according to the probability density.

[0116] Based on the N depth values ​​{d} obtained above i} N We can obtain N planes P:{z=d} perpendicular to the new viewpoint. i} N .

[0117] The N depth planes obtained above, the input reference image I, and the camera parameters C representing the new viewpoint are input into the viewpoint synthesizer SVSMI to obtain the corresponding new viewpoint image I. out :

[0118] I out =SVSMI(P,I,C)

[0119] SVSMI is a pre-trained neural network for synthesizing new perspectives. It can directly output images from the new perspectives by receiving a reference image I, N depth planes, and camera parameters C representing the new perspectives.

[0120] It is understandable that by using the above image processing methods, we can take advantage of the prior knowledge that the target generally has a near-unimodal distribution in depth space, and sample near the target with a Gaussian distribution density to achieve efficient multi-depth plane sampling, thereby achieving high-quality new perspective synthesis.

[0121] To further illustrate, such as Figure 4 As shown, where, Figure 4 The left sub-image in the image is the input reference image, with the figure displayed in the image from a side view. Figure 4The top right sub-image in the image is the result of conventional multi-plane sampling and perspective synthesis. The person is displayed in the image from a frontal perspective. There are blurry ghostings at the person's left temple and right sleeve, while there are artifacts near the right side of the green screen. Figure 4 The lower right sub-image is the result of multi-plane sampling and viewpoint synthesis based on the above image processing method. Because the effective area of ​​the person is accurately sampled, the above-mentioned blurring, ghosting and artifact phenomena are avoided.

[0122] To illustrate further, in Figure 5 The paper further demonstrates the differences in the effectiveness of depth plane sampling between existing conventional multi-plane sampling and viewpoint synthesis methods and multi-plane sampling and viewpoint synthesis methods based on the aforementioned image processing methods. Figure 5 The processing result of the existing technology's ordinary multi-plane sampling and view synthesis method for the left subgraph. Figure 5 The right sub-image in the figure shows the processing result corresponding to the multi-plane sampling and viewpoint synthesis method based on the above image processing method. Both methods sample 24 planes, but the former only starts to capture the subject from the 10th depth plane, with the first 9 planes capturing only air, resulting in wasted depth planes. This embodiment (the latter) begins to capture the subject from the 7th plane, effectively suppressing noise. The former mixes the subject and green screen in the same depth plane in the last plane, while this embodiment (the latter) captures the center depth of the subject in the 10th plane, using the last plane only to represent the green screen, thus avoiding confusion between the subject and the green screen in depth space and providing strong depth guidance for subsequent new viewpoint synthesis.

[0123] It should be noted that in this embodiment, the target to be photographed is modeled as a near-unimodal distribution. In fact, bimodal or even trimodal distributions can also be modeled using multiple Gaussian distributions, thereby utilizing their probability density to sample the depth plane. As long as the method uses Gaussian distribution density to sample the depth plane, it should still not exceed the protection scope of this patent. The novel perspective synthesizer SVSMI used in this solution can actually be replaced with various other novel perspective synthesizers or trained neural network models, as long as its input depends on the pre-sampled multiple depth planes, it should still not exceed the protection scope of this patent.

[0124] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0125] It should be noted that, for the foregoing method embodiments, for the purpose of simple description, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0126] According to another aspect of the embodiments of the present application, an image processing apparatus for implementing the image processing method described above is also provided. As shown in the figure, Figure 6 The apparatus comprises:

[0127] A first obtaining unit 602 is configured to obtain a depth interval in which a target object is located in a first image;

[0128] A second obtaining unit 604 is configured to obtain a first probability density that an upper limit of the depth interval of the target object satisfies an effective display condition, and obtain a second probability density that a lower limit of the depth interval of the target object satisfies the effective display condition;

[0129] A first determining unit 606 is configured to determine, based on a probability density difference between the first probability density and the second probability density, N ordered depth points from the depth interval, wherein adjacent two of the N ordered depth points have the same probability density difference value, and N is a positive integer;

[0130] A second determining unit 608 is configured to perform image sampling at different depths in the first image according to the N ordered depth points, to obtain N image planes corresponding to the N ordered depth points;

[0131] A synthesizing unit 610 is configured to perform image synthesis using the N image planes, to obtain a second image after synthesis.

[0132] As an optional solution, the first determining unit 606 comprises:

[0133] A first determining module is configured to determine, based on the probability density difference, N ordered probability densities, wherein adjacent two of the N ordered probability densities have the same probability density difference value, and the first probability density of the N probability densities is the second probability density;

[0134] A second determining module is configured to sequentially obtain depth points corresponding to the N ordered probability densities, to obtain the N ordered depth points.

[0135] As an optional solution, the first determining module comprises:

[0136] The acquisition sub-module is configured to acquire a probability density mean difference, where the probability density mean difference is obtained by subtracting the second probability density from the first probability density and then dividing the result by N;

[0137] The determination sub-module is configured to determine the ordered N probability densities according to the probability density mean difference and the first probability density, where the i-th probability density in the ordered N probability densities is obtained by adding the probability density mean difference multiplied by i-1 to the first probability density, i is a positive integer less than or equal to N.

[0138] As an optional solution, the second determination module includes:

[0139] The matching sub-module is configured to match the respective depth points corresponding to the ordered N probability densities by using the depth center distance of the target object in the first image and the inverse error function, to obtain the ordered N depth points, where the i-th depth point corresponding to the i-th probability density in the ordered N probability densities is obtained by adding the first output value to the depth center distance, the first output value is obtained by multiplying the first preset parameter by the second output value, the second output value is an output value obtained by inputting the i-th probability density into the inverse error function, and i is a positive integer less than or equal to N.

[0140] As an optional solution, the first acquisition unit 602 includes:

[0141] The first acquisition module is configured to acquire an upper limit and a lower limit of an interval in which the target object is located in the first image, where the target object is displayed in the first image at the first view angle.

[0142] The synthesis unit 610 includes:

[0143] The synthesis module is configured to perform new view angle synthesis by using the N image planes, to obtain the second image, where the target object is displayed in the second image at the second view angle.

[0144] As an optional solution, the synthesis module includes:

[0145] The input sub-module is configured to input the N image planes, the first image, and the camera parameters into the view angle synthesis model, to obtain the second image output by the view angle synthesis model, where the view angle synthesis model is a neural network model that is pre-trained and used for new view angle synthesis, and the camera parameters are used to indicate the view angle parameters corresponding to the second view angle.

[0146] As an optional solution, the second acquisition unit 604 includes a second acquisition module configured to acquire a first distance difference between the upper limit of the interval and the depth center distance of the target object in the first image.

[0147] The third obtaining module is configured to input the first distance difference into an error function to obtain a third output value, and adjust the third output value using a second preset parameter to obtain a first probability density;

[0148] The second obtaining unit 604 further includes:

[0149] The fourth obtaining module is configured to obtain a second distance difference between the lower limit of the interval and the depth center distance.

[0150] The fifth obtaining module is configured to input the second distance difference into an error function to obtain a fourth output value, and adjust the fourth output value using a second preset parameter to obtain a second probability density.

[0151] According to still another aspect of the embodiments of the present application, an electronic device for implementing the image processing method is provided, which can be but is not limited to the client 102 or the server 112 shown in FIG. 1. In this embodiment, the electronic device is taken as the client 102 for example, and as shown in FIG. 7, the electronic device includes a memory 702 and a processor 704, the memory 702 stores a computer program, and the processor 704 is configured to execute the steps in any of the above method embodiments through the computer program. Figure 1 Figure 7 The processor 704 is configured to execute the steps in any of the above method embodiments through the computer program.

[0152] Optionally, in this embodiment, the electronic device can be located in at least one of the network devices in the computer network.

[0153] Optionally, in this embodiment, the processor can be configured to execute the following steps through the computer program:

[0154] S1, obtaining a depth interval in which a target object is located in a first image;

[0155] S2, obtaining a first probability density that a upper limit of the depth interval of the target object satisfies an effective display condition, and obtaining a second probability density that a lower limit of the depth interval of the target object satisfies the effective display condition; S3, determining N ordered depth points from the depth interval based on a probability density difference between the first probability density and the second probability density, wherein a probability density difference value of adjacent two depth points in the N ordered depth points is same, and N is a positive integer;

[0156] S4, performing image sampling at different depths in the first image according to the N ordered depth points to obtain N image planes corresponding to the N ordered depth points;

[0157] S5, performing image synthesis using the N image planes to obtain a second image after synthesis.

[0158] Optionally, those skilled in the art can understand that,​Figure 7 The structure shown is for illustrative purposes only. Figure 7 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 7 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 7 The different configurations shown.

[0159] The memory 702 can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and apparatus in this embodiment. The processor 704 executes various functional applications and data processing by running the software programs and modules stored in the memory 702, thereby implementing the aforementioned image processing method. The memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 702 may further include memory remotely located relative to the processor 704, and these remote memories can be connected to electronic devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 702 may be used, but is not limited to, to store information such as a first probability density and a second probability density. As an example, such as... Figure 7 As shown, the memory 702 may include, but is not limited to, the first acquisition unit 602, the second acquisition unit 604, the first determination unit 606, the second determination unit 608, and the synthesis unit 610 in the image processing device. Furthermore, it may include, but is not limited to, other module units in the image processing device, which will not be elaborated upon in this example.

[0160] Optionally, the transmission device 706 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 706 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 706 is a radio frequency (RF) module, used for wireless communication with the Internet.

[0161] In addition, the aforementioned electronic device also includes: a display 708 for displaying information such as a first probability density and a second probability density; and a connection bus 710 for connecting various module components in the aforementioned electronic device.

[0162] In other embodiments, the above-mentioned client or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which can be a distributed system formed by the plurality of nodes connected in the form of network communication. Among them, the nodes can constitute a peer-to-peer network, and any form of computing device, such as a server, a client, and an electronic device, can become a node in the blockchain system by joining the peer-to-peer network.

[0163] According to an aspect of the present application, a computer program product is provided, which includes computer programs / instructions containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a detachable medium. When the computer program is executed by a central processing unit, various functions provided by the embodiments of the present application are executed.

[0164] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0165] It should be noted that the computer system of the electronic device is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0166] The computer system includes a central processing unit (CPU), which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) or programs loaded from a storage part to a random access memory (RAM). In the random access memory, various programs and data required for system operation are also stored. The central processing unit, the read-only memory, and the random access memory are connected to each other through a bus. An input / output interface (I / O interface) is also connected to the bus.

[0167] The following components are connected to the input / output interface: an input section including a keyboard, a mouse, etc.; an output section including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a local area network card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the input / output interface as necessary. A removable media such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive as necessary, so that a computer program read out from the removable media is installed in the storage section as necessary.

[0168] In particular, according to embodiments of the present application, the processes described in the various method flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section, and / or installed from a removable media. When the computer program is executed by the central processing unit, various functions defined in the system of the present application are performed.

[0169] According to an aspect of the present application, a computer readable storage medium is provided, from which a processor of a computer device reads computer instructions, the processor executes the computer instructions to cause the computer device to perform the method provided in the various optional implementation manners described above.

[0170] Optionally, in the present embodiment, the computer readable storage medium described above can be configured to store a computer program for executing the following steps:

[0171] S1, obtaining a depth interval in which a target object is located in a first image;

[0172] S2, obtaining a first probability density that an upper limit of the depth interval of the target object satisfies an effective display condition, and obtaining a second probability density that a lower limit of the depth interval of the target object satisfies the effective display condition;

[0173] S3, determining, based on a probability density difference between the first probability density and the second probability density, N ordered depth points from the depth interval, wherein a probability density difference value corresponding to any two adjacent depth points in the N ordered depth points is the same, and N is a positive integer;

[0174] S4, performing image sampling at different depths in the first image according to the N ordered depth points, to obtain N image planes corresponding to the N ordered depth points.

[0175] S5, using the N image planes, performing image synthesis to obtain a second synthesized image.

[0176] Optionally, in the embodiment, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by instructing the hardware related to the electronic device through a program, and the program can be stored in a computer readable storage medium, and the storage medium can include a flash disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0177] The serial numbers of the embodiments of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0178] The integrated units in the above-mentioned embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the above-mentioned computer readable storage medium. Based on such understanding, the technical solutions of the application essentially or the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the embodiments of the application.

[0179] In the above-mentioned embodiments of the application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0180] In the several embodiments provided by the present application, it should be understood that the recorded client can be implemented by other means. Among them, the above-mentioned device embodiment is only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0181] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0182] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0183] The above is only the preferred embodiment of the present application, and it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should also be considered as the protection scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises the following steps: obtaining a depth interval in which a target object is located in a first image; obtaining a first probability density that an upper limit of the depth interval of the target object satisfies an effective display condition and a second probability density that a lower limit of the depth interval of the target object satisfies the effective display condition; determining N ordered depth points from the depth interval based on a probability density difference between the first probability density and the second probability density, wherein adjacent two depth points in the N ordered depth points correspond to the same probability density difference value, and N is a positive integer; performing image sampling at different depths in the first image according to the N ordered depth points to obtain N image planes corresponding to the N ordered depth points; performing image synthesis using the N image planes to obtain a second synthesized image.

2. The method of claim 1, wherein, The step of determining the N ordered depth points from the depth interval based on the probability density difference between the first probability density and the second probability density comprises the following steps: determining N ordered probability densities based on the probability density difference, wherein adjacent two probability densities in the N ordered probability densities correspond to the same probability density difference value, and a first probability density in the N ordered probability densities is the second probability density; obtaining the N ordered depth points by sequentially obtaining depth points corresponding to the N ordered probability densities.

3. The method of claim 2, wherein, The step of determining the N ordered probability densities based on the probability density difference comprises the following steps: obtaining a probability density average difference, wherein the probability density average difference is obtained by subtracting the second probability density from the first probability density and then dividing the result by N; determining the N ordered probability densities according to the probability density average difference and the first probability density, wherein an i-th probability density in the N ordered probability densities is obtained by adding the probability density average difference multiplied by i-1 to the first probability density, i is a positive integer less than or equal to N.

4. The method of claim 2, wherein, The step of obtaining the N ordered depth points by sequentially obtaining depth points corresponding to the N ordered probability densities comprises the following steps: matching the depth points corresponding to the N ordered probability densities by using a depth center distance of the target object in the first image and an inverse error function to obtain the N ordered depth points, wherein an i-th depth point corresponding to an i-th probability density in the N ordered probability densities is obtained by adding a first output value to the depth center distance, the first output value is obtained by multiplying a first preset parameter by a second output value, and the second output value is an output value obtained by inputting the i-th probability density into the inverse error function, i is a positive integer less than or equal to N.

5. The method of claim 1, wherein the step of obtaining a depth interval in which a target object is located in a first image comprises the following steps: obtaining an upper limit and a lower limit of the depth interval in which the target object is located in the first image, wherein the target object is displayed in the first image at a first viewing angle; the step of performing image synthesis using the N image planes to obtain a second synthesized image comprises the following steps: The N image planes are used to perform new view synthesis to obtain the second image, wherein the target object is displayed in the second image at a second view angle.

6. The method of claim 5, wherein, The using the N image planes to perform new view synthesis to obtain the second image comprises: The N image planes, the first image and camera parameters are input into a view synthesis model to obtain the second image output by the view synthesis model, wherein the view synthesis model is a pre-trained neural network model used for the new view synthesis, and the camera parameters are used to indicate view angle parameters corresponding to the second view angle.

7. The method of any one of claims 1-6, wherein The obtaining the first probability density that the upper limit of the depth interval of the target object satisfies the effective display condition comprises: Obtaining a first distance difference between the upper limit of the depth interval and a depth center distance of the target object in the first image; The first distance difference is input into an error function to obtain a third output value, and the third output value is adjusted using a second preset parameter to obtain the first probability density; The obtaining the second probability density that the lower limit of the depth interval of the target object satisfies the effective display condition comprises: Obtaining a second distance difference between the lower limit of the depth interval and the depth center distance; The second distance difference is input into the error function to obtain a fourth output value, and the fourth output value is adjusted using the second preset parameter to obtain the second probability density.

8. An image processing apparatus characterized by comprising: comprises: The first acquisition unit is configured to acquire a depth interval in which a target object is located in a first image. The second acquisition unit is configured to acquire a first probability density that an upper limit of the depth interval of the target object satisfies an effective display condition, and acquire a second probability density that a lower limit of the depth interval of the target object satisfies the effective display condition. The first determination unit is configured to determine, based on a probability density difference between the first probability density and the second probability density, N ordered depth points from the depth interval, wherein a probability density difference value of adjacent two depth points in the N ordered depth points is the same, and N is a positive integer. The second determination unit is configured to perform image sampling at different depths in the first image according to the N ordered depth points to obtain N image planes corresponding to the N ordered depth points. The synthesis unit is configured to use the N image planes to perform image synthesis to obtain a second image after synthesis.

9. A computer readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the program is run by an electronic device to execute the method in any one of claims 1-7.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the method in any one of claims 1-7.

11. An electronic device comprising a memory and a processor, characterized in that The memory stores a computer program, and the processor is configured to execute the method in any one of claims 1-7 by the computer program.

Images