Image processing method and device, electronic equipment and medium

Abstract

The embodiment of the invention discloses an image processing method and device, electronic equipment and a medium, and relates to the technical field of artificial intelligence. And performing segmented compression on the second depth range of the second image of the current frame in combination with a preset compression algorithm to obtain a third depth range, and further generating a third image corresponding to the second viewpoint. According to the process, after the second depth range of the second image is obtained, the depth values of part of pixel points are processed in a targeted manner through segmented compression, the problem of depth value mutation caused by scene mutation is effectively avoided, it is ensured that the depth values of key objects in the image are stable in 3D conversion, the film watching discomfort of a user is reduced, the 3D video watching experience is improved, and the user experience is improved. And meanwhile, full-automatic and low-cost conversion from the 2D video to the 3D video is realized, and the method is suitable for 3D processing of massive 2D video resources.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to an image processing method, apparatus, electronic device, and medium. Background Technology

[0002] With the development of image processing technology, the core requirement in the field of converting 2D video to 3D video is to transform 2D video content into video with a 3D visual effect. The principle is to allow the viewer's eyes to receive images from different viewpoints, and to form a stereoscopic effect through binocular parallax.

[0003] Currently, in the related technologies for 2D video to 3D image processing, there are two main methods to determine the depth value of each pixel in each frame of the 2D video from the current viewpoint. One method is to manually set the depth value, and the other is to use a depth estimation algorithm. After determining the depth value of each pixel, multiple frames from another viewpoint are obtained based on these depth values ​​to achieve a 3D stereoscopic effect.

[0004] However, manually setting the depth values ​​of each pixel requires a significant investment of manpower and is costly, making it difficult to apply to scenarios involving the conversion of massive amounts of 2D video to 3D video, and resulting in low image processing efficiency. On the other hand, directly using depth estimation algorithms to obtain the depth values ​​of each pixel in each frame of the image from the current viewpoint can lead to changes in the depth range of the current frame compared to the previous frame when the scene suddenly changes (e.g., the camera switches from indoors to outdoors, or the maximum depth changes from the wall to the sky). This causes changes in the depth values ​​of all objects in the current frame, resulting in abrupt changes in depth values ​​and affecting the stability of the 3D effect. Summary of the Invention

[0005] This application provides an image processing method, apparatus, electronic device, and medium to solve the problems in related technologies. By acquiring the second depth range of the second image, and using the first depth range of the first image and preset key depth values, the depth values ​​of some pixels in the second depth range are specifically compressed, effectively avoiding the problem of depth value mutation caused by scene changes, ensuring the stability of the depth values ​​of key objects in the image during the 3D conversion process, improving the stability of the 3D effect, and realizing fully automatic and low-cost conversion of 2D video to 3D video, which is suitable for 3D image processing of massive 2D video resources.

[0006] In a first aspect, embodiments of this application provide an image processing method, the method comprising: determining a first depth range of a first image and a second depth range of a second image, wherein the first image is a frame preceding the second image, and the first image and the second image are two-dimensional images corresponding to a first viewpoint, the first depth range including a first depth value of each pixel in the first image after compression, and the second depth range including a second depth value of each pixel in the second image; performing segmented compression on the second depth range based on the first depth range and preset key depth values, combined with a preset compression algorithm, to obtain a compressed third depth range, the third depth range including a third depth value of each pixel in the second image after compression; and generating a third image corresponding to the second image based on the third depth value of each pixel, the third image being a two-dimensional image corresponding to the second viewpoint.

[0007] In some embodiments, the second depth range is segmented and compressed based on a first depth range and a preset key depth value, combined with a preset compression algorithm, to obtain a compressed third depth range. This includes: dividing the second depth range based on the preset key depth value to determine a first sub-depth range, a second sub-depth range, and a third sub-depth range within the second depth range; compressing the first sub-depth range and the second sub-depth range respectively based on the preset compression algorithm and the first depth range to obtain a compressed first sub-depth range and a compressed second sub-depth range; and obtaining a third depth range based on the third sub-depth range, the compressed first sub-depth range, and the compressed second sub-depth range.

[0008] In some embodiments, dividing the second depth range based on a preset key depth value and determining the first sub-depth range, the second sub-depth range, and the third sub-depth range within the second depth range includes: determining a second target depth value for a target pixel within the second depth range, wherein the target pixel is the pixel corresponding to the first target depth value in the first depth range; determining the depth range within the second depth range that is less than the preset key depth value as the first sub-depth range; determining the depth range within the second depth range that is greater than or equal to the preset key depth value and less than the second target depth value as the second sub-depth range; and determining the depth range within the second depth range that is greater than or equal to the second target depth value as the third sub-depth range.

[0009] In some embodiments, based on a preset compression algorithm and a first depth range, the first sub-depth range and the second sub-depth range are compressed to obtain the compressed first sub-depth range and the compressed second sub-depth range, respectively. This includes: determining a first available depth range in the first depth range and defining the first available depth range as the compressed first sub-depth range, wherein the first available depth range is the depth range in the first depth range that is less than a preset key depth value; determining a second available depth range in the first depth range, and based on the second available depth range, a second target depth value, and a preset key depth value, combined with the preset compression algorithm, determining the compressed second sub-depth range, wherein the second available depth range is the depth range in the first depth range that is greater than or equal to the preset key depth value and less than the second target depth value.

[0010] In some embodiments, generating a third image corresponding to the second image based on the third depth value of each pixel, wherein the third image is a two-dimensional image corresponding to the second viewpoint includes: determining the second abscissa of each pixel in the second viewpoint based on the first abscissa of each pixel in the first viewpoint and the third depth value of each pixel, combined with a depth rendering algorithm; determining the first pixel value of each pixel in the first abscissa as the second pixel value of each pixel in the second abscissa; and generating the third image based on the second pixel value.

[0011] In some embodiments, generating a third image corresponding to the second image based on the third depth value of each pixel, wherein the third image is a two-dimensional image corresponding to the second viewpoint includes: smoothing the third depth value of each pixel based on a preset smoothing algorithm to obtain a smoothed third smoothed depth value of each pixel; and generating a third image corresponding to the second image based on the third smoothed depth value and a depth rendering algorithm.

[0012] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising: The determining unit is used to determine a first depth range of a first image and a second depth range of a second image. The first image is a frame image preceding the second image. The first image and the second image are two-dimensional images corresponding to a first viewpoint. The first depth range includes the first depth value of each pixel in the first image after compression processing, and the second depth range includes the second depth value of each pixel in the second image. The compression unit is used to perform segmented compression on the second depth range based on the first depth range and the preset key depth value, combined with the preset compression algorithm, to obtain the compressed third depth range. The third depth range includes the third depth value of each pixel in the second image after compression. The generation unit is used to generate a third image corresponding to the second image based on the third depth value of each pixel and in combination with a depth rendering algorithm. The third image is a two-dimensional image corresponding to the second viewpoint.

[0013] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor, wherein, when the processor runs the computer program, it performs the method described in any embodiment of the first aspect.

[0014] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described in any embodiment of the first aspect.

[0015] Fifthly, embodiments of this application provide a computer program product including a computer program that, when executed by a processor, performs the method described in any embodiment of the first aspect.

[0016] This application provides an image processing method that utilizes the first depth range of the previous frame's first image and preset key depth values, combined with a preset compression algorithm, to segmentally compress the second depth range of the current frame's second image, obtaining a third depth range, and then generating a third image corresponding to the second viewpoint. This process, after obtaining the second depth range of the second image, selectively processes the depth values ​​of certain pixels through segmented compression, effectively avoiding depth value abrupt changes caused by scene shifts. This ensures that the depth values ​​of key objects in the image remain stable during 3D conversion, reducing user viewing discomfort and improving the 3D video viewing experience. Simultaneously, it achieves fully automatic and low-cost conversion from 2D to 3D video, suitable for the 3D conversion of massive 2D video resources.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are merely embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort, and this application can be applied to other similar scenarios based on the provided drawings.

[0019] Figure 1 A flowchart illustrating an image processing method provided in an embodiment of this application; Figure 2 A flowchart illustrating the second image processing method provided in this application embodiment; Figure 3 A flowchart illustrating the third image processing method provided in this application embodiment; Figure 4 A schematic diagram of a display imaging model provided in an embodiment of this application; Figure 5 A flowchart illustrating the fourth image processing method provided in this application embodiment; Figure 6 This application provides a function graph corresponding to a third depth range in its embodiments.

[0020] Figure 7 This application provides an embodiment of an ideal function graph processed by a preset smoothing algorithm. Figure 8 This application provides a function graph corresponding to a smoothed third depth range as an embodiment of the present application; Figure 9 A flowchart for converting 2D video to 3D video is provided as an embodiment of this application; Figure 10 A schematic diagram of a 3D video viewing technology provided in this application embodiment; Figure 11 A general flowchart of an image processing scheme provided in an embodiment of this application; Figure 12 A scene diagram of a 2D video in a real-world scenario provided for an embodiment of this application; Figure 13 The overall depth map of a frame in a scene of a 2D video obtained by a depth estimation algorithm provided in this application embodiment; Figure 14 This application provides an embodiment of the overall depth map of a frame in a scene of a 2D video under actual conditions, obtained by using a preset compression algorithm and a preset smoothing algorithm. Figure 15 This is a schematic diagram of the structure of an image processing apparatus 1500 provided in an embodiment of this application; Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. The described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] It should be noted that the terms "system," "device," "unit," and / or "module" used in this application are methods of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they can be replaced by other expressions.

[0023] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0024] With the development of image processing technology, the core requirement in the field of converting 2D video to 3D video is to transform 2D video content into video with a 3D visual effect.

[0025] The principle behind converting 2D video to 3D video is to allow the viewer's two eyes to receive images from different viewpoints, creating a stereoscopic effect through binocular parallax. Specifically, the viewer's left eye receives the image from a first viewpoint, while the right eye receives the image from a second viewpoint; the resulting binocular parallax creates the 3D visual effect.

[0026] The key to converting 2D video to 3D video is generating an image from a second viewpoint based on the image from the first viewpoint. The generation of the image from the second viewpoint is based on the direct calculation of the depth values ​​of each pixel in the image.

[0027] Currently, in the related technologies for 2D video to 3D image processing, there are two main methods to determine the depth value of each pixel in each frame of the 2D video from the current viewpoint. One method is to layer the 2D video according to different depths of field and manually set the depth value of each pixel. The other method is to use a depth estimation algorithm (such as Depth Anything, MIDAS, etc., a monocular depth estimation algorithm) to uniformly predict the depth value of each pixel. After determining the depth value of each pixel, multiple frames of images from another viewpoint are obtained based on these depth values ​​to achieve a 3D stereoscopic effect.

[0028] However, manually setting the depth values ​​of each pixel requires a significant investment of manpower and is costly, making it difficult to apply to scenarios involving the conversion of massive amounts of 2D video to 3D video, and resulting in low image processing efficiency. On the other hand, directly using depth estimation algorithms to obtain the depth values ​​of each pixel in each frame of the image from the current viewpoint is susceptible to changes when the scene suddenly changes (e.g., the camera switches from indoors to outdoors, or the maximum depth changes from the wall to the sky). This alters the depth range of the current frame compared to the previous frame, causing changes in the depth values ​​of all objects in the current frame, leading to abrupt changes in depth values ​​and affecting the stability of the 3D effect. Such abrupt changes can also cause significant discomfort to users, resulting in symptoms such as dizziness.

[0029] To address the aforementioned issues, this application proposes an image processing method for 2D video to 3D video conversion scenarios. By utilizing a preset compression algorithm, the depth values ​​of each pixel in the previous frame image from a first viewpoint, and preset key depth values, the depth values ​​of each pixel in the current frame image from the first viewpoint are segmented and compressed. This allows the generation of an image from a second viewpoint using the segmented compressed pixel depth values ​​of the current frame image from the first viewpoint. This ensures that when the scene in the current frame changes compared to the previous frame, the depth values ​​of key objects in the current frame image remain unchanged or change according to the preset compression algorithm. This effectively avoids depth value abrupt changes caused by scene shifts, ensuring consistency in the user's viewing experience and reducing viewing discomfort.

[0030] The image processing method provided in this application will be described in detail below with reference to the accompanying drawings.

[0031] Figure 1 A flowchart illustrating an image processing method provided in an embodiment of this application is shown. Figure 1 As shown, the image processing method includes steps 101-103.

[0032] Step 101: Determine the first depth range of the first image and the second depth range of the second image.

[0033] In the embodiments of this application, the first image is the image preceding the second image, the first image and the second image are two-dimensional images corresponding to the first viewpoint, the first depth range includes the first depth value of each pixel in the first image after compression, and the second depth range includes the second depth value of each pixel in the second image.

[0034] The first viewpoint refers to the viewpoint from which the user's left eye observes the image.

[0035] Specifically, this application first utilizes a depth estimation algorithm, such as a monocular depth estimation algorithm, to determine the original first depth range of the first image and the second depth range of the second image by identifying scene elements (scene elements can be set according to actual conditions, and are not limited in this embodiment). The original first depth range and the second depth range are the depth ranges after normalization processing following the determination of the depth range of each pixel by the depth estimation algorithm, so that the original first depth range and the second depth range are within the [0-1] interval.

[0036] It is understood that after obtaining the original first depth range of the first image, this application can use the previous frame image of the first image in combination with the preset compression algorithm and preset key depth value mentioned in step 102 to obtain the segmented compressed first depth range, which can be used as a reference benchmark for the segmented compression of the second depth range of the second image, maintaining the continuity of the depth values ​​between frames and avoiding sudden changes in depth values ​​caused by scene changes.

[0037] Step 102: Based on the first depth range and the preset key depth value, and combined with the preset compression algorithm, the second depth range is segmented and compressed to obtain the compressed third depth range, wherein the third depth range includes the third depth value of each pixel in the second image after compression.

[0038] In the embodiments of this application, the depth value typically corresponds to a distance in actual space (such as meters, millimeters, etc.) and is used to describe the vertical distance from a point or pixel on the surface of an object to the camera imaging plane.

[0039] The preset critical depth value can be a threshold set manually by R&D personnel based on experience, or it can be a threshold determined by averaging the depth values ​​of historical near-field objects. It can be adjusted according to actual conditions, and is not limited in this embodiment. The preset critical depth value represents the boundary of the depth range of most near-field objects (such as people, main scenery objects, etc.) that are easily noticed by the user in the lens, to ensure that the depth value of near-field objects remains stable when the scene changes, and to prioritize the continuity of the user's visual focus. This application takes a preset critical depth value of 40% as an example.

[0040] The principle of the preset compression algorithm is to combine preset key depth values ​​and the first depth range of the first image in the previous frame, and then segment the depth values ​​of each pixel in the second depth range of the second image in the current frame. Different sub-depth ranges within the second depth range are processed differently to obtain the compressed depth values ​​of each sub-depth range, thus obtaining the third depth range. This ensures that when the second depth range changes compared to the first depth range (e.g., the maximum depth value of the second depth range is greater than the maximum depth value of the first depth range, or vice versa), the depth values ​​of nearby objects that are easily noticed remain stable without abrupt changes, while the depth values ​​of distant objects with larger depth values ​​that are less easily noticed are proportionally and adaptively adjusted, reducing viewing discomfort for the user.

[0041] The third depth range refers to the range formed by the third depth values ​​of all pixels after the second image has been segmented and compressed.

[0042] Step 103: Generate a third image corresponding to the second image based on the third depth value of each pixel.

[0043] In the embodiments of this application, the third depth value is the depth value corresponding to each pixel point in the third depth range. The third image is a two-dimensional image corresponding to the second viewpoint.

[0044] The second viewpoint refers to the viewing point of the image observed by the user's right eye.

[0045] Specifically, after obtaining the third depth value of each pixel, this application can use a depth rendering algorithm to obtain a third image. After obtaining the third image, i.e., the image observed by the user's right eye, the left eye receives the second image, and the right eye receives the third image, creating a parallax between the two eyes, thus producing a 3D visual perception.

[0046] In this application, the depth rendering algorithm refers to the depth image-based rendering (DIBR) algorithm.

[0047] In summary, according to the image processing method proposed in this application, a first depth range of a first image and a second depth range of a second image are determined; based on the first depth range and preset key depth values, combined with a preset compression algorithm, the second depth range is segmented and compressed to obtain a compressed third depth range, wherein the third depth range includes the third depth value of each pixel in the second image after compression; based on the third depth value of each pixel, a third image corresponding to the second image is generated, thereby ensuring the stability of the depth values ​​of key objects in the current frame image when the scene changes compared to the previous frame, effectively avoiding the problem of depth value abrupt changes caused by scene changes, thus ensuring the consistency of the user's viewing experience and reducing user viewing discomfort.

[0048] As one possible implementation method, Figure 2 A flowchart of the second image processing method is shown. Based on the above embodiments, the second depth range is segmented and compressed using a first depth range and a preset key depth value, combined with a preset compression algorithm, to obtain a compressed third depth range, including the following steps: Step 201: Based on the preset key depth value, the second depth range is divided to determine the first sub-depth range, the second sub-depth range, and the third sub-depth range within the second depth range.

[0049] In the embodiments of this application, the application can use preset key depth values ​​to divide the second depth range of the second frame image, and perform differentiated compression processing on different sub-depth ranges of the second depth range.

[0050] This application divides a second depth range based on a preset key depth value. Determining the first, second, and third sub-depth ranges within the second depth range may include: determining a second target depth value for a target pixel within the second depth range, where the target pixel is the pixel corresponding to the first target depth value in the first depth range; defining the depth range within the second depth range that is less than the preset key depth value as the first sub-depth range; defining the depth range within the second depth range that is greater than or equal to the preset key depth value and less than the second target depth value as the second sub-depth range; and defining the depth range within the second depth range that is greater than or equal to the second target depth value as the third sub-depth range. The first target depth value is the maximum depth value within the first depth range.

[0051] Specifically, this application defines a preset key depth value as The second target depth value is (It should be noted that this value is the second depth value of the pixel corresponding to the maximum depth value in the first depth range, and this value may change depending on the actual situation. It is not limited in this embodiment.) The pixel coordinates of each pixel in the second image are ( x , y (), x , y ) use uniformly X express; x Here are the x-coordinates of each pixel. y The ordinates of each pixel are: This represents the second depth value for each pixel within the second depth range. The first sub-depth range is... The second sub-depth range is The third sub-depth range is .

[0052] Step 202: Based on the preset compression algorithm and the first depth range, the first sub-depth range and the second sub-depth range are compressed respectively to obtain the first sub-depth range and the second sub-depth range after compression.

[0053] In embodiments of this application, the application can determine a first available depth range within a first depth range, and define the first available depth range as a first sub-depth range after compression processing. The first available depth range is the depth range within the first depth range that is less than a preset key depth value. The application can also determine a second available depth range within the first depth range, and based on the second available depth range, a second target depth value, and a preset key depth value, combined with a preset compression algorithm, determine a second sub-depth range after compression processing. The second available depth range is the depth range within the first depth range that is greater than or equal to the preset key depth value and less than the second target depth value.

[0054] Specifically, according to the preset compression algorithm, when the depth range in the second depth range is less than the preset key depth value... When the depth range is greater than or equal to the preset key depth value (i.e., the first sub-depth range), the first available depth range in the first depth range is directly determined as the first sub-depth range after compression; when the depth range in the second depth range is greater than or equal to the preset key depth value... And less than the second target depth value (i.e., the second sub-depth range), based on the second available depth range in the first depth range, the preset key depth value, and the second target depth value, adaptive compression processing is performed to calculate the compressed second sub-depth range; when the depth range in the second depth range is greater than or equal to the second target depth value... (i.e., the third sub-depth range), in this case, no compression processing is performed on the third sub-depth range. The specific preset compression algorithm function is as follows:

[0055] in, These are the x and y coordinates of each pixel in the image. Time corresponding The first available depth range, Time corresponding For the second available depth range, This represents the first depth value of each pixel within the first depth range. Time corresponding This indicates that the second depth value within the second depth range is greater than or equal to... The depth range of the depth value.

[0056] Step 203: Based on the third sub-depth range, the first sub-depth range after compression, and the second sub-depth range after compression, the third depth range is obtained.

[0057] In the embodiments of this application, the third sub-depth range is a second depth range that is greater than or equal to the second target depth value. The depth range.

[0058] This application can combine a first sub-depth range (after compression) and a second sub-depth range (after compression) with a third sub-depth range to obtain the final third depth range. .

[0059] In summary, this application designs a segmented compression algorithm to perform segmented compression processing on different depth ranges in the second frame image, which can effectively avoid the depth abruptness problem caused by related technologies.

[0060] As one possible implementation method, Figure 3 A flowchart of the third image processing method is shown. Based on the above embodiments, a third image corresponding to the second image is generated based on the third depth value of each pixel, including the following steps: Step 301: Based on the first horizontal coordinate of each pixel in the first viewpoint and the third depth value of each pixel, and combined with the depth rendering algorithm, determine the second horizontal coordinate of each pixel in the second viewpoint.

[0061] In the embodiments of this application, the depth rendering algorithm (i.e., the DIBR algorithm) calculates the second abscissa of each pixel in the third image from the second viewpoint by using the first abscissa of each pixel in the second image from the first viewpoint, the third depth range obtained from the second image, and parallax. (Parallax refers to the difference in the position of an object in the field of view due to the different viewpoint positions of the observers, such as...) Figure 4 As shown, Figure 4 A schematic diagram of a display imaging model provided in this application is shown, and the specific parallax formula is as follows:

[0062] in Indicates parallax. Indicates the distance between a person's eyes. Indicates the distance between the person and the monitor. Represents a point in the scene The depth of field from the viewer (i.e., the pixels in the third depth range in this application) (third depth value) Indicates the depth of field range between the in-screen and out-of-screen areas of a scene. for The x-coordinate of the point on the left viewpoint (i.e., the first x-coordinate under the first viewpoint). for The x-coordinate of the point on the right viewpoint (i.e., the second x-coordinate under the second viewpoint). The x-coordinate of the point on the left viewpoint and the x-coordinate on the right viewpoint They are as follows:

[0063]

[0064] in yes The x-coordinate of the point on the point where the eyes converge (the fixation point).

[0065] Specifically, based on the first x-coordinate of each pixel in the first viewpoint. and the third depth value of each pixel By combining depth rendering algorithms, the second x-coordinate of each pixel in the second viewpoint can be determined. .

[0066] Step 302: Determine the first pixel value of each pixel at the first horizontal coordinate as the second pixel value at the second horizontal coordinate.

[0067] In the embodiments of this application, the first pixel value of each pixel on the first horizontal coordinate is the pixel value of each pixel in the second image (i.e., the lower left image of the left viewpoint) under the first viewpoint, and the second pixel value of each pixel on the second horizontal coordinate is the pixel value of each pixel in the third image (i.e., the lower right image of the right viewpoint) under the second viewpoint.

[0068] Specifically, after determining the pixel values ​​of each pixel in the left image, the horizontal coordinates of each pixel in the right image are obtained using the parallax formula. The remaining coordinates can be directly reused from the coordinate values ​​of the corresponding pixels in the left image and are directly determined from the remaining coordinates of each pixel in the left image.

[0069] In the process of generating the right image from the left image, the right image is not generated completely randomly. Based on the principle of binocular parallax, it is directly translated from most of the pixel values ​​in the left image. Therefore, the pixel value of a certain pixel in the left image corresponds to the pixel value of the same pixel in the right image only with a change in its horizontal coordinate. Thus, this application can directly determine the pixel value of each pixel in the right image based on the first horizontal coordinate of the pixel in the left image and the second horizontal coordinate of the pixel in the right image. That is, the pixel values ​​in the right image and the pixel values ​​in the left image are highly correlated in content, and the coordinate translation is only caused by the viewpoint switching. The first pixel value of each pixel in the left image at the first horizontal coordinate can be directly used as the second pixel value of the corresponding pixel in the right image at the second horizontal coordinate.

[0070] Step 303: Generate a third image based on the second pixel value.

[0071] In the embodiments of this application, the second pixel value represents the pixel value of each pixel in the right image, and the third image represents the right image. By generating the right image using the pixel values ​​of each pixel in the right image obtained by the above-mentioned preset compression algorithm, the abrupt change in the depth value of each pixel in the image can be avoided, ensuring that the 3D effect of the same object remains unchanged, thereby preventing dizziness for the user.

[0072] In summary, this application generates the right image from the left image based on a segmented compression algorithm, which can ensure that the depth values ​​of each pixel in the image do not change abruptly, thus ensuring that the generated 3D effect does not change abruptly and improving the user experience.

[0073] As one possible implementation method, Figure 5 A flowchart of the fourth image processing method is shown. Based on the above embodiments, a third image corresponding to the second image is generated based on the third depth value of each pixel, including the following steps: Step 401: Based on a preset smoothing algorithm, the third depth value of each pixel is smoothed to obtain the smoothed third smoothed depth value of each pixel.

[0074] In the embodiments of this application, the image depth distribution (i.e., the depth value of each pixel in the image) is generally a linear function. The new image depth distribution (i.e., the third depth range) obtained after processing with a preset compression algorithm in this application has depth values ​​within... and At these two points, a clear inflection point appears, causing the depth values, which are divided into three segments, to change inconsistently. Figure 6 As shown, Figure 6 This application provides a function graph corresponding to a third depth range, from... Figure 6 As can be seen, in this case, the change in depth value will have two obvious anomalies, making the depth change of the compressed middle section significantly greater than that of the two sections before and after.

[0075] To address the aforementioned depth "anomaly" problem, this application designs a function whose depth changes rapidly in the first half and slowly in the second half, serving as the activation function for depth values ​​(i.e., a pre-defined smoothing algorithm). The physical meaning of this function is: for "near" locations with smaller depth values, the depth change is more pronounced, resulting in a stronger 3D effect and easier perception; while for "far" locations with larger depth values, the depth change is smaller, and the perception of depth changes is less noticeable. The ideal function after processing with the pre-defined smoothing algorithm is as follows: Figure 7 As shown, Figure 7 This application provides an embodiment of an ideal function graph processed by a preset smoothing algorithm. The preset smoothing algorithm can be designed using the following formula:

[0076] in, The design philosophy is to ensure that the overall function image of the activation function remains as consistent as possible after being modified by a preset compression algorithm. Since the image depth is uniformly compressed to the [0, 1] interval, to ensure that the domain and range of the piecewise function do not change after smoothing, limit The points are (0, 0) and (1, 1). Using the above formula, the specific preset smoothing algorithm can be obtained as follows:

[0077] The new depth distribution after segmentation and compression (i.e., the third depth range) Input into the preset smoothing algorithm In the middle, we obtained The graph of the function is as follows Figure 8 As shown, Figure 8This application provides a function graph corresponding to a smoothed third depth range in an embodiment. Therefore, in the same shot, when the depth range of consecutive frames changes abruptly, the depth value of foreground objects that are easily noticed can be kept stable, ensuring consistency in the user's viewing experience.

[0078] Step 402: Based on the third smoothing depth value and the depth rendering algorithm, generate the third image corresponding to the second image.

[0079] In embodiments of this application, the third smoothing depth range obtained in step 401 can be used. And the third smoothing depth value of each pixel after smoothing and compression. Generate a third image.

[0080] Specifically, similar to steps 301 to 303, the first horizontal coordinate of each pixel in the second image from the first viewpoint is used. and the third smoothing depth value for each pixel By combining this with a depth rendering algorithm, the second x-coordinate of each pixel in the third image from the second viewpoint can be obtained. After determining the first pixel value of each pixel at the first viewpoint, this application determines the second pixel value of each pixel at the second viewpoint based on the first pixel value of each pixel at its first horizontal coordinate; the second pixel value can be used to generate a third image, wherein the third image is a two-dimensional image corresponding to the second viewpoint.

[0081] To further explain the entire process of converting 2D video to 3D video, please refer to... Figure 9 , Figure 9 This document provides a flowchart for converting 2D video to 3D video, as illustrated in an embodiment of this application. Converting 2D video to 3D video involves setting each frame of the 2D video as the left image (i.e., the second image from the first viewpoint in this application), and then... Figure 9 The process shown generates the right image (i.e., the third image from the second viewpoint in this application). When the left and right images are played simultaneously, a 3D effect can be achieved using the principle of binocular parallax. This process includes the following core steps: When converting 2D video to 3D video, this application first uses a single frame image from the 2D video as the left image. Then, a depth estimation algorithm is used to estimate the depth of the left image, obtaining the depth value of each pixel (corresponding to the third depth value of each pixel in the third depth range of this application or the third smoothing depth value in the third smoothing depth range of this application), thereby generating a depth map of the left image. To prevent inter-frame depth flickering, this application can also perform inter-frame smoothing processing on the depth map obtained from depth estimation using a depth temporal stabilizer. If a single frame image in the 2D video contains embedded subtitles, this application needs to use subtitle matting technology to identify the subtitle region and use a subsequent DIBR algorithm to assign a fixed parallax (e.g., setting a minimum depth value to make the subtitles appear to float).

[0082] Subsequently, based on the parallax formula, and using the left image and its generated depth map as a basis, the pixel values ​​of each pixel in the right image are derived using the DIBR algorithm (corresponding to the depth rendering algorithm in this application). However, the DIBR algorithm may produce occlusion areas that are invisible to the left eye but visible to the right eye. In this case, an occlusion repair algorithm (such as ProPainter) can be called to fill in and complete these areas. If there are high requirements for the output video resolution, such as requiring high-resolution output, super-resolution technology can be used to improve the image quality. Finally, a usable right image is generated, which works in conjunction with the left image to form a 3D parallax effect.

[0083] The key steps in the above process are explained in detail below: Depth estimation: Using deep learning techniques as input to the left image, the depth values ​​of each pixel in the left image are predicted. For example, monocular depth algorithms such as Depth Anything and MIDAS can be used to predict the depth values ​​of each pixel in the left image. It should be noted that the depth estimation in this application requires the use of preset compression and smoothing algorithms, which can be found in the following references. Figures 1 to 7 The embodiments shown will not be described in detail here.

[0084] Depth timing stabilizer: In order to solve the problem of inter-frame depth flicker caused by single-frame input depth estimation, a depth timing stabilizer is introduced to alleviate the problem of inter-frame depth jump and ensure the stability of depth estimation.

[0085] Subtitle matting: When subtitles are embedded in a video scene, depth estimation algorithms cannot perfectly predict the depth of the subtitle area. Subtitle matting can be used to identify the subtitle area and a fixed parallax (e.g., 0) can be assigned to the area during the DIBR stage. If it is necessary to keep the subtitles floating at the forefront of the image, the depth of the subtitle area can be set to the minimum value.

[0086] DIBR: refers to the DIBR algorithm, which combines the disparity formula and uses the depth value of each pixel in the left image to infer the pixel value of each pixel in the right image.

[0087] Video Inpainting (Occlusion Repair Algorithm): Images generated by the DIBR algorithm may contain areas that are not visible to the left eye but are visible to the right eye. These areas cannot be inferred by the DIBR algorithm and need to be restored by occlusion repair algorithms, such as ProPainter.

[0088] Super-resolution: In order to meet the output requirements of 4K or higher resolution, the video processed by the above steps is subjected to super-resolution processing to improve the video quality.

[0089] It is understandable that in the 2D to 3D video conversion process, depth temporal stabilizer, subtitle keying, video inpainting (occlusion repair), and super-resolution are image optimization steps used to improve the 3D video effect. In this embodiment, whether to enable optimization methods such as depth temporal stabilizer, subtitle keying, video inpainting, and super-resolution is not mandatory and can be flexibly selected according to actual needs.

[0090] Furthermore, to help with a better understanding, Figure 10 This diagram illustrates the principle of a 3D video viewing technology, a format used to display stereoscopic images. For example... Figure 10 As shown, the images for the left and right eyes are placed side by side in the same image. The left eye image (corresponding to the second image in this application) is located on the left and the right eye image (corresponding to the third image in this application) is located on the right. The video presents a 3D effect. The fundamental principle is to allow the viewer's left eye to receive the left image and the right eye to receive the right image, so that the two eyes form a parallax, thereby creating a 3D visual sensation.

[0091] In 2D to 3D video conversion, the most fundamental step is depth estimation, the quality of which directly affects the subsequent 3D video effect. Figure 11 The overall flowchart of the image processing scheme proposed in this application shows that the depth values ​​of each pixel in the predicted left image (corresponding to the second depth values ​​of each pixel in the second depth range in this application) are segmented and compressed based on a segmented compression function (corresponding to the preset compression algorithm in this application) to obtain a segmented compressed third depth range. Then, the third depth values ​​of each pixel in the third depth range are smoothed using an activation function (corresponding to the preset smoothing algorithm in this application) to obtain a third smoothed depth value. The right image is then obtained based on the third smoothed depth value using a depth rendering algorithm, thereby avoiding the problem of abrupt changes in the depth values ​​of each pixel when the scene changes in consecutive frames. Figure 12This illustration shows a scene diagram of a 2D video in a real-world scenario, as provided in the embodiments of this application. It can be seen that as the camera moves, the sky may appear outside the left window. At this time, the depth range between consecutive frames in the 2D video will change, that is, the maximum depth value within the depth range will change. Therefore, the overall depth map of a certain frame image obtained directly using the depth estimation algorithm is shown below. Figure 13 As shown, Figure 13 This is a depth map of a frame in a 2D video scene obtained by a depth estimation algorithm. When the farthest object changes, its depth value changes, and the depth values ​​of each pixel also change significantly, leading to a noticeable visual distortion. However, the depth map processed by this application is as follows: Figure 14 As shown, Figure 14 This embodiment of the application provides an overall depth map of a frame in a 2D video scene under actual conditions, obtained through a preset compression algorithm and a preset smoothing algorithm. When the farthest object changes, the depth value of the farthest object changes, but the depth value of each pixel does not change significantly. After processing the depth values ​​of each pixel in this way, a 2D-to-3D conversion process is performed to generate the right-hand image without abrupt changes. In this way, the overall depth of the video does not change abruptly, thus greatly improving the user's viewing experience.

[0092] In summary, the image processing method provided in this application for converting 2D video to 3D video involves obtaining the depth values ​​of each pixel in the left image, and then performing segmented compression and smoothing processing on these pixel depth values ​​based on preset compression and smoothing algorithms. This avoids abrupt changes in the depth values ​​of pixels in consecutive frames when the scene changes. Subsequently, the depth values ​​of each pixel in the right image are predicted based on the smoothed depth values ​​to generate a 3D video, thus ensuring a stable 3D effect for near-field objects. This method is suitable for fully automated, low-cost conversion of massive amounts of 2D video to 3D video, and can alleviate discomfort for viewers watching 3D videos.

[0093] To implement the above embodiments, this application also provides an image processing apparatus. Figure 15 This is a schematic diagram of the structure of an image processing apparatus 1500 provided in an embodiment of this application. Figure 15 As shown, the device includes: The determining unit 1501 is used to determine a first depth range of a first image and a second depth range of a second image. The first image is a frame image preceding the second image. The first image and the second image are two-dimensional images corresponding to a first viewpoint. The first depth range includes the first depth value of each pixel in the first image after compression processing. The second depth range includes the second depth value of each pixel in the second image. Compression unit 1502 is used to perform segmented compression on the second depth range based on the first depth range and the preset key depth value, combined with the preset compression algorithm, to obtain the compressed third depth range. The third depth range includes the third depth value of each pixel in the second image after compression. The generation unit 1503 is used to generate a third image corresponding to the second image based on the third depth value of each pixel and in combination with a depth rendering algorithm. The third image is a two-dimensional image corresponding to the second viewpoint.

[0094] In some embodiments of this application, the compression unit 1502 is configured to: divide a second depth range based on a preset key depth value to determine a first sub-depth range, a second sub-depth range, and a third sub-depth range within the second depth range; compress the first sub-depth range and the second sub-depth range respectively based on a preset compression algorithm and the first depth range to obtain a compressed first sub-depth range and a compressed second sub-depth range; and obtain a third depth range based on the third sub-depth range, the compressed first sub-depth range, and the compressed second sub-depth range.

[0095] In some embodiments of this application, the compression unit 1502 is configured to: determine a second target depth value of a target pixel in a second depth range, wherein the target pixel is the pixel corresponding to the first target depth value in a first depth range; determine a depth range in the second depth range that is less than a preset key depth value as a first sub-depth range; determine a depth range in the second depth range that is greater than or equal to the preset key depth value and less than the second target depth value as a second sub-depth range; and determine a depth range in the second depth range that is greater than or equal to the second target depth value as a third sub-depth range.

[0096] In some embodiments of this application, the compression unit 1502 is configured to: determine a first available depth range within a first depth range, and define the first available depth range as a first sub-depth range after compression processing, wherein the first available depth range is a depth range within the first depth range that is less than a preset key depth value; determine a second available depth range within the first depth range, and based on the second available depth range, a second target depth value, and a preset key depth value, and in conjunction with a preset compression algorithm, determine a second sub-depth range after compression processing, wherein the second available depth range is a depth range within the first depth range that is greater than or equal to the preset key depth value and less than the second target depth value.

[0097] In some embodiments of this application, the generation unit 1503 is configured to: determine the second horizontal coordinate of each pixel in a second viewpoint based on the first horizontal coordinate of each pixel in a first viewpoint and the third depth value of each pixel, combined with a depth rendering algorithm; determine the first pixel value of each pixel in the first horizontal coordinate as the second pixel value of each pixel in the second horizontal coordinate; and generate a third image based on the second pixel value.

[0098] In some embodiments of this application, the generation unit 1503 is further configured to: smooth the third depth value of each pixel based on a preset smoothing algorithm to obtain the smoothed third smoothed depth value of each pixel; and generate a third image corresponding to the second image based on the third smoothed depth value and a depth rendering algorithm.

[0099] The methods and apparatus provided in the embodiments of this application have been described above. To implement the functions of the methods provided in the embodiments of this application, the electronic device may include a hardware structure and software modules, and may implement the above functions in the form of a hardware structure, software modules, or a hardware structure plus software modules. One of the above functions may be executed in the form of a hardware structure, software modules, or a hardware structure plus software modules.

[0100] Figure 16 This is a block diagram illustrating an electronic device 1600 for implementing the above-described image processing method according to an exemplary embodiment. For example, the electronic device 1600 may be a mobile phone, computer, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0101] Reference Figure 16 The electronic device 1600 may include a communication interface 1601, capable of interacting with other devices; a processor 1602, connected to the communication interface 1601 to interact with other devices, used to execute the methods provided by one or more of the above-described technical solutions when running a computer program; and a memory 1603, on which the computer program is stored. Specifically, the specific processing procedure of the processor 1602 can refer to the image processing method described in the above embodiments of this application.

[0102] Of course, in practical applications, the various components in electronic device 1600 are coupled together through bus system 1604. It can be understood that bus system 1604 is used to realize the connection and communication between these components. In addition to a data bus, bus system 1604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 16 The general labeled all buses as Bus System 1604.

[0103] The memory 1603 in this embodiment is used to store various types of data to support the operation of the electronic device 1600. Examples of such data include any computer program used to operate on the electronic device 1600.

[0104] The methods disclosed in the embodiments of this application can be applied to processor 1602, or implemented by processor 1602. Processor 1602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 1602 or by instructions in the form of software. The processor 1602 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 1602 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 1603. Processor 1602 reads the information in memory 1603 and completes the steps of the aforementioned method in conjunction with its hardware.

[0105] In an exemplary embodiment, the electronic device 1600 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0106] Embodiments of this application also propose a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the image processing method described in the above embodiments of this application.

[0107] Embodiments of this application also provide a computer program product, including a computer program that is executed by a processor using the image processing method described in the above embodiments of this application.

[0108] Embodiments of this application also propose a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, it causes the electronic device to perform the image processing method described in the above embodiments of this application.

[0109] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0110] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0111] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0112] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (control method), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic device, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0113] It should be understood that various parts of the embodiments of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0114] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0115] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc.

[0116] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An image processing method, characterized in that, The method includes: A first depth range of a first image and a second depth range of a second image are determined. The first image is the previous frame of the second image. The first image and the second image are two-dimensional images corresponding to a first viewpoint. The first depth range includes the first depth value of each pixel in the first image after compression. The second depth range includes the second depth value of each pixel in the second image. Based on the first depth range and the preset key depth value, and combined with the preset compression algorithm, the second depth range is segmented and compressed to obtain the compressed third depth range. The third depth range includes the third depth value of each pixel in the second image after compression. Based on the third depth value of each pixel, a third image corresponding to the second image is generated, and the third image is a two-dimensional image corresponding to the second viewpoint.

2. The method according to claim 1, characterized in that, Based on the first depth range and a preset key depth value, and combined with the preset compression algorithm, the second depth range is segmented and compressed to obtain a compressed third depth range, which includes: Based on the preset key depth value, the second depth range is divided to determine the first sub-depth range, the second sub-depth range and the third sub-depth range in the second depth range. Based on the preset compression algorithm and the first depth range, the first sub-depth range and the second sub-depth range are compressed respectively to obtain the first sub-depth range and the second sub-depth range after compression. The third depth range is obtained based on the third sub-depth range, the first sub-depth range after compression, and the second sub-depth range after compression.

3. The method according to claim 2, characterized in that, The step of dividing the second depth range based on the preset key depth value and determining the first sub-depth range, the second sub-depth range, and the third sub-depth range within the second depth range includes: Determine the second target depth value of the target pixel in the second depth range, wherein the target pixel is the pixel corresponding to the first target depth value in the first depth range; The depth range within the second depth range that is smaller than the preset key depth value is defined as the first sub-depth range; The depth range in the second depth range that is greater than or equal to the preset key depth value and less than the second target depth value is defined as the second sub-depth range; The depth range within the second depth range that is greater than or equal to the second target depth value is defined as the third sub-depth range.

4. The method according to claim 2, characterized in that, The step of compressing the first sub-depth range and the second sub-depth range based on the preset compression algorithm and the first depth range to obtain the compressed first sub-depth range and the compressed second sub-depth range includes: Determine a first available depth range within the first depth range, and define the first available depth range as the first sub-depth range after compression. The first available depth range is the depth range within the first depth range that is less than the preset key depth value. A second available depth range is determined within the first depth range. Based on the second available depth range, the second target depth value, and the preset key depth value, and in conjunction with the preset compression algorithm, the second sub-depth range after compression is determined. The second available depth range is the depth range within the first depth range that is greater than or equal to the preset key depth value and less than the second target depth value.

5. The method according to claim 1, characterized in that, The step of generating a third image corresponding to the second image based on the third depth value of each pixel, wherein the third image is a two-dimensional image corresponding to the second viewpoint, includes: Based on the first horizontal coordinate of each pixel in the first viewpoint and the third depth value of each pixel, combined with the depth rendering algorithm, the second horizontal coordinate of each pixel in the second viewpoint is determined; The first pixel value of each pixel at the first horizontal coordinate is determined as the second pixel value of each pixel at the second horizontal coordinate; The third image is generated based on the second pixel value.

6. The method according to claim 1, characterized in that, The step of generating a third image corresponding to the second image based on the third depth value of each pixel, wherein the third image is a two-dimensional image corresponding to the second viewpoint, includes: Based on a preset smoothing algorithm, the third depth value of each pixel is smoothed to obtain the smoothed third smoothed depth value of each pixel. Based on the third smoothing depth value and the depth rendering algorithm, a third image corresponding to the second image is generated.

7. An image processing apparatus, characterized in that, The device includes: The determining unit is used to determine a first depth range of a first image and a second depth range of a second image, wherein the first image is a frame image preceding the second image, and the first image and the second image are two-dimensional images corresponding to a first viewpoint, the first depth range includes the first depth value of each pixel in the first image after compression, and the second depth range includes the second depth value of each pixel in the second image. The compression unit is used to perform segmented compression on the second depth range based on the first depth range and the preset key depth value, combined with the preset compression algorithm, to obtain a compressed third depth range, wherein the third depth range includes the third depth value of each pixel in the second image after compression. The generation unit is used to generate a third image corresponding to the second image based on the third depth value of each pixel and in combination with a depth rendering algorithm. The third image is a two-dimensional image corresponding to the second viewpoint.

8. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images