Image processing method, apparatus and electronic device

Abstract

The application discloses an image processing method, device and electronic equipment. The method comprises the following steps: obtaining a first image, wherein the first image is a two-dimensional image; processing the first image to obtain a first view and a second view, wherein the first view corresponds to a first viewpoint, the second view corresponds to a second viewpoint, and the image quality corresponding to the first view is lower than the image quality corresponding to the second view; and the first view and the second view are used to realize three-dimensional image output.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus and electronic device. Background Technology

[0002] Naked-eye 3D refers to the process of outputting a 2D image as a 3D image without the aid of external tools such as polarized glasses, thus achieving a stereoscopic visual effect. For example, based on a 2D image, a left-eye view and a right-eye view are generated, and then the left-eye view and the right-eye view are combined to obtain a 3D image.

[0003] To provide users with high-definition 3D output, it is usually necessary to generate high-resolution left and right eye views, which results in a large amount of data processing required for generating the left and right eye views.

[0004] Therefore, there is an urgent need for a technical solution that can reduce the amount of naked-eye 3D data processing. Summary of the Invention

[0005] In view of the above, this application provides an image processing method, apparatus, and electronic device, as follows:

[0006] An image processing method, comprising:

[0007] Obtain a first image, wherein the first image is a two-dimensional image;

[0008] The first image is processed to obtain a first view and a second view, the first view corresponding to a first viewpoint and the second view corresponding to a second viewpoint, and the image quality corresponding to the first view is lower than the image quality corresponding to the second view.

[0009] The first view and the second view are used to achieve three-dimensional image output.

[0010] The above method, preferably, involves processing the first image to obtain a first view and a second view, including:

[0011] The first image is processed at least according to a first resolution to obtain a first view corresponding to a first viewpoint;

[0012] The first image is processed according to the second resolution to obtain a second view corresponding to the second viewpoint;

[0013] The first resolution is lower than the second resolution.

[0014] The above method, preferably, involves processing the first image at least according to a first resolution to obtain a first view corresponding to a first viewpoint, including:

[0015] The first image is processed to obtain a first initial view corresponding to a first viewpoint;

[0016] The first processing area in the first initial view is processed according to the first resolution;

[0017] The second processing area in the first initial view is processed according to the second resolution;

[0018] The first processing area and the second processing area together form a first view corresponding to the first viewpoint.

[0019] The above method, preferably, involves processing the first image to obtain a first view and a second view, including:

[0020] The first image is processed to obtain a first initial view corresponding to a first viewpoint. The first initial view is then filled with content according to a first filling method to obtain the first view.

[0021] The first image is processed to obtain a second initial view corresponding to a second viewpoint. The second initial view is then filled with content according to a second filling method to obtain a second view.

[0022] The amount of content filled by the first filling method is less than the amount of content filled by the second filling method.

[0023] The above method, preferably, involves filling the first initial view with content according to a first filling method to obtain a first view, including:

[0024] According to the first filling method, the pixels of the background area in the first image are used to fill the content of the first filling area in the first initial view;

[0025] According to the first filling method, the pixels of the foreground region and the background region in the first image are used to fill the second filling region in the first initial view to obtain the first view;

[0026] The first filling area and the second filling area together constitute the area that needs to be filled in the first initial view.

[0027] The above method, preferably, involves processing the first image to obtain a first view and a second view, including:

[0028] Obtain the image parameters of the first image;

[0029] When the image parameters meet the processing conditions, a first view corresponding to a first viewpoint is generated according to the first image in a first generation method; and a second view corresponding to a second viewpoint is generated according to the first image in a second generation method.

[0030] If the image parameters do not meet the processing conditions, a first view corresponding to a first viewpoint and a second view corresponding to a second viewpoint are generated according to the first image and the second generation method.

[0031] The image quality corresponding to the first generation method is lower than that corresponding to the second generation method.

[0032] In the above method, preferably, the first content in the first view is output to the first viewpoint through an output device, and the second content in the first view is output to the second viewpoint through the output device, wherein the first content and the second content constitute the first view;

[0033] Furthermore, the second view is output to the second viewpoint via the output device;

[0034] The above method, preferably, involves processing the first image to obtain a first view and a second view, including:

[0035] The first image is processed using a first image generation model to obtain a first view corresponding to a first viewpoint;

[0036] The first image is processed using a second image generation model to obtain a second view corresponding to a second viewpoint;

[0037] The first image generation model is trained based on a first input sample and a first output sample, wherein the first input sample is a two-dimensional image sample and the first output sample is a first view sample corresponding to the first viewpoint; the second image generation model is trained based on a second input sample and a second output sample, wherein the second input sample is a two-dimensional image sample and the second output sample is a second view sample corresponding to the second viewpoint; the image quality of the first view sample is lower than that of the second view sample.

[0038] An image processing apparatus, comprising:

[0039] A two-dimensional acquisition unit is used to acquire a first image, wherein the first image is a two-dimensional image;

[0040] A view acquisition unit is used to process the first image to obtain a first view and a second view, wherein the first view corresponds to a first viewpoint and the second view corresponds to a second viewpoint, and the image quality corresponding to the first view is lower than the image quality corresponding to the second image.

[0041] The first view and the second view are used to achieve three-dimensional image output.

[0042] An electronic device, comprising:

[0043] A memory used to store computer programs and the data generated by the execution of said computer programs;

[0044] A processor is configured to execute the computer program to: obtain a first image, the first image being a two-dimensional image; process the first image to obtain a first view and a second view, the first view corresponding to a first viewpoint and the second view corresponding to a second viewpoint, wherein the image quality corresponding to the first view is lower than the image quality corresponding to the second view; wherein the first view and the second view are used to implement three-dimensional image output.

[0045] As can be seen from the above technical solutions, in the image processing method, apparatus and electronic device disclosed in this application, when generating a view of two viewpoints for a two-dimensional image, the view of one viewpoint is processed with low quality to reduce the amount of data processing for the view. However, due to the perceptual filling principle of the brain, that is, the human brain will automatically fill in the missing content of the other viewpoint based on the content of one viewpoint, even if the image quality of one viewpoint is reduced in this application, the effect of the three-dimensional image output will not be worse. Therefore, this application can reduce the amount of data processing for three-dimensional output without affecting the three-dimensional output effect. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of an image processing method provided in Embodiment 1 of this application;

[0048] Figure 2 This is an example diagram illustrating the implementation of 3D image output based on a first view and a second view in an embodiment of this application.

[0049] Figure 3 , Figure 4 , Figure 5These are partial flowcharts of an image processing method provided in Embodiment 1 of this application;

[0050] Figure 6 This is a schematic diagram of the structure of an image processing device provided in Embodiment 2 of this application;

[0051] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this application;

[0052] Figure 8 This is an example diagram illustrating the processing of crosstalk regions in an embodiment of this application;

[0053] Figure 9 This is an example diagram illustrating the processing based on depth parameters in the embodiments of this application;

[0054] Figure 10 This is an example diagram showing the output of left and right views in an embodiment of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] refer to Figure 1 The diagram shown is a flowchart illustrating the implementation of an image processing method according to Embodiment 1 of this application. This method can be applied to electronic devices capable of image processing, such as computers or servers. The technical solution in this embodiment is mainly used to reduce the amount of data processing required for 3D output.

[0057] The method in this embodiment may include the following steps:

[0058] Step 101: Obtain the first image, which is a two-dimensional image.

[0059] The first image is a two-dimensional image that needs to be output in three dimensions.

[0060] In one implementation, the first image is a two-dimensional image captured by an image acquisition device such as a camera. For example, an outdoor scene photograph taken by a user using a mobile phone.

[0061] In another implementation, the first image is a two-dimensional image generated by an image processing tool. For example, a line drawing, such as a comic, created by a user using a drawing tool.

[0062] Step 102: Process the first image to obtain a first view and a second view. The image quality of the first view is lower than that of the second view. The first view and the second view are used to realize the three-dimensional image output.

[0063] In this view, the first view corresponds to the first viewpoint, and the second view corresponds to the second viewpoint. For example, the first view corresponds to the left-eye viewpoint, and the second view corresponds to the right-eye viewpoint.

[0064] In one implementation, at least some areas in the first view have a lower image resolution than the second view.

[0065] In another implementation, at least some areas in the first view contain less image content than the corresponding areas in the second view.

[0066] For example, such as Figure 2 As shown, L represents the image content of the left eye view and R represents the image content of the right eye view. Through the cylindrical lens of the display screen, L is output to the left eye viewpoint and R is output to the right eye viewpoint, thereby achieving a three-dimensional output effect.

[0067] In one implementation, the image quality corresponding to the first view is lower than that corresponding to the second view; specifically, the overall image quality of the first view is lower than the overall image quality of the second view. Figure 2 For example, the overall image quality of all L's image content is lower than that of all R's image content. Since the human brain automatically fills in the missing content of one viewpoint based on the content of another, even if the overall image quality of the first view is lower, the higher overall image quality of the second view allows the human brain to automatically fill in the missing content of the first view based on the image content of the second view when outputting to the human eye. This does not affect the 3D output effect, and reduces the amount of data processing in this process.

[0068] In another implementation, the image quality corresponding to the first view is lower than that corresponding to the second view. Specifically, the image quality of the image region in the first view corresponding to the second viewpoint is lower than that of the image region in the second view corresponding to the second viewpoint. Figure 2For example, the image quality of the image content located slightly to the right in L is lower than that of the image content located slightly to the right in R. However, because the human brain automatically fills in the missing content from one viewpoint based on the content from another, even if the image quality of the image region in the first view corresponding to the second viewpoint is lower, the higher image quality of the image region in the second view allows the human brain to automatically fill in the corresponding image content in the first view based on the image content in the second view, without affecting the 3D output effect. This process reduces the amount of data processing.

[0069] As can be seen from the above technical solutions, in the image processing method provided in Embodiment 1 of this application, when generating views from two viewpoints for a two-dimensional image, the view from one viewpoint is processed with low quality to reduce the amount of data processing for the view. However, due to the perceptual filling principle of the brain, that is, the human brain will automatically fill in the missing content of the other viewpoint based on the content of one viewpoint, even if the image quality of one viewpoint is reduced in this embodiment, the effect of the three-dimensional image output will not be worse. Therefore, this embodiment can reduce the amount of data processing for the three-dimensional output without affecting the three-dimensional output effect.

[0070] Furthermore, if the image quality of the image region in the first view corresponding to the second viewpoint is lower than that of the image region in the second view corresponding to the second viewpoint, the image quality of the image region in the second view corresponding to the first viewpoint can also be lower than that of the image region in the first view corresponding to the first viewpoint. In this case, the overall image quality of the first view is similar to that of the second view. However, if the image quality of the image region in the second view corresponding to the first viewpoint is similar to that of the image region in the first view corresponding to the first viewpoint, then the overall image quality of the first view is lower than that of the second view. Since the human brain automatically fills in the missing content of one viewpoint based on the content of another viewpoint, even if the image quality of the image region in the second view corresponding to the first viewpoint is lower, because the image quality of the image region in the first view corresponding to the first viewpoint is higher, when output to the human eye, the human brain will automatically fill in the corresponding image content in the second view based on the image content of the image region in the first view corresponding to the first viewpoint, without affecting the 3D output effect, and this process reduces the amount of data processing.

[0071] In one implementation, step 102, when processing the first image to obtain the first view and the second view, can be achieved in the following way: Figure 3 As shown:

[0072] Step 301: Process the first image at least according to a first resolution to obtain a first view corresponding to a first viewpoint.

[0073] In the first view, at least a portion of the area has a resolution of the first resolution, and the amount of data processed in the first view corresponds to the first resolution.

[0074] Step 302: Process the first image according to the second resolution to obtain a second view corresponding to the second viewpoint.

[0075] In this context, the resolution of the second view is the second resolution, and the data processing volume of the second view corresponds to the second resolution. The first resolution is lower than the second resolution, which makes the data processing volume of the first view lower than that of the second view. However, since the human brain automatically fills in the missing content of one viewpoint based on the content of another viewpoint, even if the image resolution of the first view is relatively low, the image resolution of the second view is relatively high. When the image is output to the human eye, the human brain will automatically fill in the image content of the first view based on the high-resolution image content of the second view. Thus, while achieving a three-dimensional output effect, the data processing volume is reduced.

[0076] It should be noted that the execution order between steps 301 and 302 can be as follows: Figure 3 As shown, step 302 can be executed first and then step 301, or steps 301 and 302 can be executed simultaneously. Different technical solutions achieved by different execution orders of steps 301 and 302 are all within the protection scope of this application.

[0077] In one embodiment, in step 301, the first image can be processed first, such as viewpoint image conversion, to obtain image content corresponding to the first viewpoint, i.e., the first initial view. Then, all areas to be processed in the first initial view are processed according to the first resolution, such as rendering and content filling, to obtain the first view with the first resolution, while the resolution of the second view is the second resolution, so that the data processing amount of the first view is lower than that of the second view. Thus, while achieving the three-dimensional output effect, the data processing amount is reduced.

[0078] In another embodiment, in step 301, the first image may be processed first to obtain a first initial view corresponding to a first viewpoint. Then, the first processing area in the first initial view is processed according to a first resolution, such as rendering and content filling. At the same time, the second processing area in the first initial view is processed according to a second resolution. The first processing area and the second processing area together form the first view corresponding to the first viewpoint.

[0079] In other words, in this embodiment, some areas in the first initial view are processed with a lower first resolution, while other areas in the first initial view are still processed with a higher second resolution. As a result, some areas in the first view have a lower first resolution, while other areas have a second resolution. The second view also has a second resolution. Therefore, the amount of data processing for the first view is lower than that for the second view. Thus, while achieving the three-dimensional output effect, the amount of data processing is reduced.

[0080] For example, the first processing region is the area in the first initial view that is at least adjacent to the view corresponding to the second viewpoint. In practical applications, the first processing region is a crosstalk region in the first initial view where there is crosstalk between the views corresponding to the second viewpoint, and the crosstalk between the two views occurs in the adjacent areas of the two views. Based on this, this embodiment performs low-resolution processing on the crosstalk region to reduce image quality. However, when achieving 3D output, since the effect of 3D imaging is dominated by the high-resolution second view, this embodiment can reduce the amount of data processing while also reducing the impact of crosstalk.

[0081] It should be noted that the crosstalk region is related to the hardware configuration of the output device. Therefore, in this embodiment, the crosstalk rate can be determined based on the hardware configuration of the electronic device, and thus the crosstalk region can be determined.

[0082] Furthermore, in this embodiment, when processing the first image according to the second resolution to obtain a second view corresponding to the second viewpoint, the first image can be processed first to obtain a second initial view corresponding to the second viewpoint. Then, the third processing area in the second initial view is processed according to the first resolution, and the fourth processing area in the second initial view is processed according to the second resolution. The third processing area and the fourth processing area constitute the second view. At this time, the image quality of the image area in the first view corresponding to the second viewpoint is lower than the image quality of the image area in the second view corresponding to the second viewpoint, and the image quality of the image area in the second view corresponding to the first viewpoint is lower than the image quality of the image area in the first view corresponding to the first viewpoint.

[0083] Specifically, crosstalk typically occurs in the region corresponding to another viewpoint in the current view. Therefore, in addition to causing crosstalk regions in the first initial view that correspond to crosstalk between the viewpoint and the second viewpoint, crosstalk regions in the second initial view may also exist that correspond to crosstalk between the viewpoint and the first viewpoint. In this embodiment, the crosstalk regions in the second initial view can be processed using a lower first resolution, while other regions in the second initial view can still be processed using a higher second resolution. As a result, some regions in the second view have a lower first resolution, while other regions have a second resolution. Because the human brain automatically fills in the missing content of one viewpoint with the content of another, even if the image quality of the image region in the first view corresponding to the second viewpoint is low, the higher image quality of the image region in the second view will allow the brain to automatically fill in the corresponding image content in the first view based on the image content of the second view. Similarly, even if the image quality of the image region in the second view corresponding to the first viewpoint is low, the higher image quality of the image region in the first view will allow the brain to automatically fill in the corresponding image content in the second view based on the image content of the first view. Therefore, by processing the crosstalk regions in both views with low resolution, the 3D visual effect of each view at the corresponding viewpoint is not affected, and the amount of data processing can be further reduced.

[0084] For example, the first processing area is the boundary area between the foreground and background areas in the first initial view. In practical applications, the first processing area is the jagged area in the first initial view. Jagged areas exist at the boundary between the foreground image content and the background image content. Based on this, this embodiment performs low-resolution processing on the jagged areas to reduce image quality. However, when achieving 3D output, since the effect of 3D imaging is dominated by the high-resolution second view, this embodiment can reduce the amount of data processing while also reducing the impact of crosstalk.

[0085] In one implementation, step 102, when processing the first image to obtain the first view and the second view, can be achieved in the following way: Figure 4 As shown:

[0086] Step 401: Process the first image to obtain a first initial view corresponding to the first viewpoint, and fill the first initial view with content according to the first filling method to obtain the first view.

[0087] The first filling method corresponds to the first content filling amount, which refers to the amount of content filled per unit area in the first filling method.

[0088] Step 402: Process the first image to obtain a second initial view corresponding to the second viewpoint, and fill the second initial view with content according to the second filling method to obtain the second view.

[0089] The second fill method corresponds to the second content fill amount, which refers to the amount of content filled per unit area in the second fill method. The content fill amount corresponding to the first fill method is less than the content fill amount in the second method, making the data processing volume of the first view lower than that of the second view. Thus, while achieving the 3D output effect, the data processing volume is reduced.

[0090] It should be noted that the areas to be filled in the first and second initial views are the missing areas between the foreground and background regions in the first image. To achieve 3D output, these missing areas need to be filled with content. For example, rasterization or similar methods can be used to fill these missing areas with pixels from both the foreground and background regions.

[0091] The execution order between steps 401 and 402 can be as follows: Figure 4 As shown, step 402 can be executed first and then step 401, or steps 401 and 402 can be executed simultaneously. Different technical solutions achieved by different execution orders of steps 401 and 402 are all within the protection scope of this application.

[0092] In one embodiment, the first filling method is a blurry filling method, and the second filling method is a realistic filling method. The second filling method contains more details than the first filling method. Therefore, the first view contains fewer details than the second view. Thus, even though this embodiment reduces the image quality of the first view by decreasing the details, the 3D imaging effect is dominated by the high-resolution second view, and this does not affect the 3D output effect. This achieves the goal of reducing data processing volume without affecting the 3D output effect.

[0093] In another embodiment, the first filling method is to fill the area to be filled by dividing it into regions using pixels from different regions; the second filling method is to fill the area to be filled by using pixels from multiple regions.

[0094] In a specific implementation, this embodiment can use pixels from the background area of ​​the first image to fill the first fill area in the first initial view according to the first fill method; and use pixels from the foreground and background areas of the first image to fill the second fill area in the first initial view according to the first fill method, to obtain the first view. Then, using pixels from the foreground and background areas of the first image, all fill areas in the second initial view are filled according to the second fill method, to obtain the second view.

[0095] The first and second fill areas here constitute the area that needs to be filled in the first initial view.

[0096] In this implementation, the first filling region is the boundary area between the foreground and background regions within the area to be filled in the first initial view. In practical applications, the first filling region is the jagged area in the first initial view. A jagged area exists at the boundary between the image content of the foreground region and the image content of the background region. Therefore, in this embodiment, for other areas to be filled outside the jagged area, content is filled using pixels from both the foreground and background regions through rasterization or other methods. For the jagged area, pixels from the background region are used for filling; for example, corresponding pixels are selected from the background region to fill the jagged area. In this implementation, the image content of the foreground region is not increased, and the data processing volume of filling with pixels from the background region is significantly lower than that of filling with pixels from both the foreground and background regions, with significantly lower implementation complexity. Thus, even though this embodiment reduces the image quality of the first view by decreasing the content of the foreground region, the 3D imaging effect is dominated by the high-resolution second view, which does not affect the 3D output effect, thereby achieving the goal of reducing data processing volume without affecting the 3D output effect.

[0097] In another embodiment, the first filling method is a blurry filling method that uses pixels from different regions to fill the area to be filled in sections; the second filling method is a realistic filling method that uses pixels from multiple regions to fill the area to be filled. Based on this, in this embodiment, for areas in the first initial view other than jagged areas, pixels from the foreground and background regions are used to fill in details through rasterization or other methods. However, for jagged areas, pixels from the background region in the first image are used for content filling in a blurry manner. Therefore, this embodiment reduces the detail of the content being filled in the first view, reduces image content in the foreground region to eliminate jagged edges, and uses a less complex filling method. Even if the image quality of the first view is reduced in this embodiment, the 3D imaging effect is dominated by the high-resolution second view, so it will not affect the 3D output effect. Thus, the goal of reducing data processing volume without affecting the 3D output effect is achieved.

[0098] In one implementation, step 102, when processing the first image to obtain the first view and the second view, can be achieved in the following way: Figure 5 As shown:

[0099] Step 501: Obtain the image parameters of the first image.

[0100] The image parameters may include at least one of depth parameters, texture parameters, frequency parameters, and attention parameters. Attention refers to the degree of attention the human eye pays to different regions of an image. Attention can be identified using a deep learning model.

[0101] Step 502: Determine whether the image parameters meet the processing conditions. If the image parameters meet the processing conditions, proceed to step 503. If the image parameters do not meet the processing conditions, proceed to step 504.

[0102] Step 503: Based on the first image, generate a first view corresponding to the first viewpoint according to a first generation method; based on the first image, generate a second view corresponding to the second viewpoint according to a second generation method.

[0103] In one embodiment, the first generation method includes: processing with a first resolution, and / or, a first filling method. The second generation method corresponds to the first generation method. If the first generation method includes processing with a first resolution, the second generation method includes processing with a second resolution; if the first generation method includes a first filling method, the second generation method includes a second filling method.

[0104] The image quality corresponding to the first generation method is lower than that corresponding to the second generation method.

[0105] Step 504: Based on the first image, generate a first view corresponding to the first viewpoint and a second view corresponding to the second viewpoint according to the second generation method.

[0106] For example, the first generation method includes: processing a first image to obtain a first initial view corresponding to a first viewpoint; then, processing the entire area of ​​the first initial view or only a first processing area in the first initial view according to a first resolution, and filling the first initial view with content according to a first filling method to obtain a first view corresponding to the first viewpoint.

[0107] The second generation method includes: processing the first image to obtain a second initial view corresponding to a second viewpoint; then processing the second initial view according to a second resolution, and filling the second initial view with content according to a second filling method to obtain a second view corresponding to the second viewpoint. As a result, the image quality of the obtained first view is lower than that of the second view.

[0108] Taking depth as an example, the depth parameter represents the detected image depth in the first image, and it also has a corresponding confidence value, which represents the reliability of the detected image depth. Based on this, the image parameter can satisfy the processing condition as follows: the confidence value corresponding to the depth parameter is less than or equal to the confidence threshold. That is, when the reliability of the detected image depth in the first image is low, the image quality of one viewpoint is reduced. For example, the entire image region or only the image region with greater depth in the first initial view is rendered at a low resolution to obtain the first view; or, in the first initial view, the image region with greater depth is filled with blur to obtain the first view; and so on. Therefore, even though the image quality of one viewpoint is reduced by lowering the resolution and blurring in this embodiment, the 3D imaging effect is dominated by the view of the other viewpoint with higher image quality, so the 3D output effect is not affected. This achieves the goal of reducing data processing volume without affecting the 3D output effect.

[0109] Taking attention as an example, the attention level characterizes the degree to which each image region in the first image is noticed by the human eye. Based on this, the processing condition for attention level to be satisfied can be: the attention level of some image regions in the first image is less than or equal to the attention level threshold. That is, when some image regions in the first image have low attention levels, the image quality of one viewpoint is reduced. For example, the image regions with low attention levels in the first initial view are rendered at a low resolution to obtain the first view; or, in the first initial view, the image regions with low attention levels are filled with content using a blur fill method to obtain the first view; and so on. Therefore, even though the image quality of one viewpoint is reduced by lowering the resolution and blur fill in this embodiment, since the effect of 3D imaging is dominated by the view of another viewpoint with higher image quality, it will not affect the 3D output effect, thus achieving the goal of reducing the amount of data processing without affecting the 3D output effect.

[0110] In one implementation, during the process of outputting a three-dimensional image, a first content in a first view is output to a first viewpoint via an output device, a second content in the first view is output to a second viewpoint via an output device, and a second view is output to a second viewpoint via an output device.

[0111] The first content and the second content together constitute the first view. That is, a portion of the content in the first view and the second view... Figure 1 The image content output to the second viewpoint is increased, while the image content output to the first viewpoint is reduced. This means increasing the resolution of the second view and decreasing the resolution of the first view. Therefore, in this embodiment, by increasing the image resolution of the second view and decreasing the image resolution of the first view during the output process, a high-resolution 3D output effect is achieved while limiting the increase in data processing volume. However, since the 3D imaging effect is dominated by the view from the second viewpoint with higher image quality, the method of reducing the image resolution of the first viewpoint in this embodiment will not affect the 3D output effect. Thus, the goal of reducing the data processing volume without affecting the 3D output effect is achieved.

[0112] In one embodiment, the second content in the first view can be output to a second viewpoint through a cylindrical lens in the display, thereby increasing the resolution of the second view and reducing the resolution of the first view.

[0113] In one implementation, step 102, when processing the first image to obtain the first view and the second view, can be achieved in the following way:

[0114] The first image is processed using a first image generation model to obtain a first view corresponding to a first viewpoint;

[0115] The first image is processed using a second image generation model to obtain a second view corresponding to a second viewpoint;

[0116] The first image generation model is trained based on a first input sample and a first output sample. The first input sample is a two-dimensional image sample, and the first output sample is a first view sample corresponding to a first viewpoint. The second image generation model is trained based on a second input sample and a second output sample. The second input sample is a two-dimensional image sample, and the second output sample is a second view sample corresponding to a second viewpoint. The image quality of the first view sample is lower than that of the second view sample.

[0117] It should be noted that both the first and second image generation models can be deep learning-based models. The two-dimensional image samples in the first input sample and the two-dimensional image samples in the second input sample can be the same or different. Accordingly, the first and second image generation models can be trained independently or jointly. For example, the first and second image generation models can be jointly trained using the first and second view samples corresponding to the same two-dimensional image sample; or, different two-dimensional image samples can be used to train the first and second image generation models separately.

[0118] Therefore, even if a low-image-quality view is generated using a low-image-quality model in this embodiment, the 3D imaging effect is dominated by the view from another viewpoint with higher image quality, so the 3D output effect will not be affected. This achieves the goal of reducing the amount of data processing without affecting the 3D output effect.

[0119] refer to Figure 6 This is a schematic diagram of an image processing device provided in Embodiment 2 of this application. This device can be configured in an electronic device capable of image processing, such as a computer or server. The technical solution in this embodiment is mainly used to reduce the amount of data processing required for 3D output.

[0120] The device in this embodiment may include the following units:

[0121] The two-dimensional acquisition unit 601 is used to acquire a first image, wherein the first image is a two-dimensional image;

[0122] The view acquisition unit 602 is used to process the first image to obtain a first view and a second view, wherein the first view corresponds to a first viewpoint and the second view corresponds to a second viewpoint, and the image quality corresponding to the first view is lower than the image quality corresponding to the second image.

[0123] The first view and the second view are used to achieve three-dimensional image output.

[0124] As can be seen from the above technical solutions, in the image processing device provided in Embodiment 2 of this application, when generating views from two viewpoints for a two-dimensional image, the view from one viewpoint is processed with low quality to reduce the amount of data processing for the view. Since the brain's perceptual filling principle, that is, the human brain will automatically fill in the missing content of the other viewpoint based on the content of one viewpoint, even if the image quality of one viewpoint is reduced in this embodiment, the effect of the three-dimensional image output will not be worse. Therefore, this embodiment can reduce the amount of data processing for three-dimensional output without affecting the three-dimensional output effect.

[0125] In one implementation, the view acquisition unit 602 is specifically configured to: process the first image at least according to a first resolution to obtain a first view corresponding to a first viewpoint; process the first image according to a second resolution to obtain a second view corresponding to a second viewpoint; wherein the first resolution is lower than the second resolution.

[0126] Specifically, when the view acquisition unit 602 processes the first image at least according to a first resolution to obtain a first view corresponding to a first viewpoint, it is configured to: process the first image to obtain a first initial view corresponding to the first viewpoint; process a first processing region in the first initial view according to the first resolution; and process a second processing region in the first initial view according to the second resolution; wherein the first processing region and the second processing region constitute the first view corresponding to the first viewpoint.

[0127] In one implementation, the view acquisition unit 602 is specifically configured to: process the first image to obtain a first initial view corresponding to a first viewpoint; fill the first initial view with content according to a first filling method to obtain a first view; process the first image to obtain a second initial view corresponding to a second viewpoint; fill the second initial view with content according to a second filling method to obtain a second view; wherein the content filling amount corresponding to the first filling method is less than the content filling amount corresponding to the second filling method.

[0128] Specifically, when the view acquisition unit 602 fills the first initial view with content according to the first filling method to obtain the first view, it is used to: fill the first filling area in the first initial view with content using the pixels of the background area in the first image according to the first filling method; and fill the second filling area in the first initial view with content using the pixels of the foreground area and the background area in the first image according to the first filling method to obtain the first view; wherein the first filling area and the second filling area constitute the area in the first initial view that needs to be filled.

[0129] In one implementation, the view acquisition unit 602 is specifically configured to: acquire image parameters of the first image; if the image parameters satisfy the processing conditions, generate a first view corresponding to a first viewpoint according to a first generation method based on the first image; generate a second view corresponding to a second viewpoint according to a second generation method based on the first image; if the image parameters do not satisfy the processing conditions, generate a first view corresponding to the first viewpoint and a second view corresponding to the second viewpoint according to the second generation method based on the first image; wherein the image quality corresponding to the first generation method is lower than the image quality corresponding to the second generation method.

[0130] In one implementation, a first content in the first view is output to the first viewpoint via an output device, and a second content in the first view is output to the second viewpoint via the output device, wherein the first content and the second content constitute the first view; and the second view is output to the second viewpoint via the output device.

[0131] In one implementation, the view acquisition unit 602 is specifically configured to: process the first image using a first image generation model to obtain a first view corresponding to a first viewpoint; process the first image using a second image generation model to obtain a second view corresponding to a second viewpoint; wherein the first image generation model is trained based on a first input sample and a first output sample, the first input sample being a two-dimensional image sample, and the first output sample being a first view sample corresponding to the first viewpoint; the second image generation model is trained based on a second input sample and a second output sample, the second input sample being a two-dimensional image sample, and the second output sample being a second view sample corresponding to the second viewpoint; the image quality of the first view sample is lower than the image quality of the second view sample.

[0132] It should be noted that the specific implementation of each unit in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0133] refer to Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this application. The electronic device may include the following structure:

[0134] Memory 701 is used to store computer programs and data generated by the execution of said computer programs;

[0135] The processor 702 is configured to execute the computer program to: obtain a first image, the first image being a two-dimensional image; process the first image to obtain a first view and a second view, the first view corresponding to a first viewpoint and the second view corresponding to a second viewpoint, wherein the image quality corresponding to the first view is lower than the image quality corresponding to the second view; wherein the first view and the second view are used to implement three-dimensional image output.

[0136] Additionally, electronic devices may include output devices, such as displays, or the electronic device and the output device may be independently configured but connected. The output device is used to output a first view and a second view to achieve 3D image output.

[0137] As can be seen from the above technical solutions, in the electronic device provided in Embodiment 3 of this application, when generating views from two viewpoints for a two-dimensional image, the view from one viewpoint is processed with low quality to reduce the amount of data processing for the view. However, due to the perceptual filling principle of the brain, that is, the human brain will automatically fill in the missing content of the other viewpoint based on the content of one viewpoint, even if the image quality of one viewpoint is reduced in this embodiment, the effect of the three-dimensional image output will not be worse. Therefore, this embodiment can reduce the amount of data processing for three-dimensional output without affecting the three-dimensional output effect.

[0138] Taking a display that enables naked-eye 3D (three-dimensional) image output as an example, the following describes several implementation schemes of this application in specific applications:

[0139] In one implementation scheme, this scheme proposes that, based on the principle that the human brain automatically fills in the missing content of one viewpoint with the content of another viewpoint, an appropriately low resolution can be used for one viewpoint in the generated left and right views, or some details can be blurred and filled in. This can not only obtain excellent naked-eye 3D display effect, but also reduce computing power consumption.

[0140] Specifically, in the process of generating the left and right views, this solution generates a view with a relatively lower resolution for one of the viewpoints (i.e., the first viewpoint), or uses a simpler filling algorithm to restore the color of the occluded area (i.e. the area that needs to be filled) caused by human eye parallax, without generating complex details. The brain is then used to automatically complete the content of the other viewpoint, ultimately resulting in a high-quality naked-eye 3D effect.

[0141] In another implementation, when using generative artificial intelligence (AIGC) to process 3D left and right view generation tasks, the current approach to addressing the issues of insufficient resolution and low detail accuracy in one of the generated viewpoints often involves increasing network complexity and adding more sophisticated preprocessing and post-processing methods, thus consuming more computing power to improve the quality of the generated left and right views. Therefore, a solution with lower resource consumption and almost no impact on the quality of the generated results is needed.

[0142] In view of this, this solution proposes that during the generation of the left and right views, using an appropriately low resolution for any one of the views (i.e., the first view), or no longer generating high-precision details, can significantly reduce computing power consumption while still achieving high-quality naked-eye 3D effects.

[0143] For example, during the generation of left and right views by AIGC, it was discovered that appropriately reducing the resolution and detail of one of the viewpoints in the left and right views does not affect the final naked-eye 3D effect. Specifically, this can be achieved by using a high-quality generation model and a relatively low-quality generation model. That is, generating a view with a relatively lower resolution for one of the viewpoints, or using a lighter network to replace a complex network so that high-precision detail generation is no longer required. Ultimately, this reduces the consumption of computing resources while still obtaining a high-quality 3D effect.

[0144] In another implementation, since the core problem in current naked-eye 3D imaging is crosstalk rate, a high crosstalk rate will bring a poor visual experience and dizziness, and crosstalk mostly occurs at the edges.

[0145] In view of this, this solution proposes to utilize the perceptual filling principle of the human brain (to a certain extent, if the resolution and detail fidelity of the images seen by the two eyes are different, the brain can form a visual perception with a resolution and detail fidelity that is closer to the actual effect through perceptual filling). In naked-eye 3D imaging, for areas with high crosstalk, the resolution and detail fidelity of the image are reduced, so that the corresponding imaging area is dominated by images with high definition and high fidelity details, thereby reducing the impact of crosstalk.

[0146] For example, such as Figure 8As shown in the diagram, the shaded areas represent regions with reduced resolution and detail fidelity. It is evident that this solution reduces the resolution and detail fidelity (by blurring) of the image regions in the left view corresponding to the right viewpoint and the right view corresponding to the left viewpoint—regions with high crosstalk—thus reducing the data processing load for both views. Thus, given a fixed hardware crosstalk rate, it utilizes the principle of human perception to reduce the impact of crosstalk, and consequently, lowers the computational requirements for content rendering.

[0147] In another implementation, the current method for generating new perspectives in naked-eye 3D imaging generally involves: first, generating depth information from a 2D image; then, determining the area requiring inpainting based on the depth information; and finally, performing inpainting. If the depth information generation is inaccurate, the subsequent errors will be amplified, affecting the 3D imaging effect.

[0148] like Figure 9 As shown, this scheme utilizes the perceptual filling principle of the human brain. When there is insufficient confidence in judging depth information, the resolution and fidelity of the corresponding inpainting can be reduced according to the level of information, allowing the human eye to perceive the corresponding area primarily from the original 2D image, resulting in better performance. For areas of the image with estimated depth that are far away, the human eye pays less attention, so the resolution and detail fidelity of the inpainting can also be reduced, saving computational power. Therefore, this scheme can improve 3D visual perception while saving computational power.

[0149] In another implementation, current naked-eye 3D imaging methods involve projecting images of different areas of the screen onto the left and right eyes of the viewer using methods such as gratings and lenses. This results in images with parallax seen by the left and right eyes, creating stereoscopic vision. With this method, each eye perceives only half the resolution of the screen; for example, if the screen is 8k, each eye can only see 4k.

[0150] This solution utilizes the perceptual filling principle of the human brain to change the method of naked-eye 3D gratings, lenses, etc., from uniformly distributing the resolution to the left and right eyes to uneven distribution (such as 2:1). As a result, the resolution of stereoscopic vision will not decrease by 50%, but only by 33%.

[0151] like Figure 10 As shown in the diagram, this solution improves the lens structure, allowing the right eye to see more pixel content, thereby maximizing the perceived 3D resolution given a screen resolution.

[0152] In addition, this solution can also be applied to multi-view naked-eye 3D imaging.

[0153] In another implementation, a new viewpoint view is currently generated from a 2D image using a rasterization method. However, due to the principle of rasterization, jagged edges are generated at each boundary of the viewpoint.

[0154] In light of this, this solution proposes to utilize the perceptual filling principle of the human brain to eliminate jagged edges at the boundary between the subject and the background, for example, by filling the jagged areas with the background color. This does not affect the naked-eye 3D effect, and there is no increase in detail to the subject, thus mitigating the 3D effect degradation caused by inconsistent left and right viewpoints.

[0155] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0156] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0157] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0158] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, comprising: Obtain a first image, wherein the first image is a two-dimensional image; The first image is processed to obtain a first view and a second view, the first view corresponding to a first viewpoint and the second view corresponding to a second viewpoint, and the image quality corresponding to the first view is lower than the image quality corresponding to the second view. The first view and the second view are used to achieve three-dimensional image output; The first image is processed to obtain a first view and a second view, including: The first image is processed to obtain a first initial view corresponding to a first viewpoint; The first processing area in the first initial view is processed according to the first resolution; The second processing area in the first initial view is processed according to the second resolution; Wherein, the first processing area and the second processing area constitute a first view corresponding to the first viewpoint; The first image is processed according to the second resolution to obtain a second view corresponding to the second viewpoint; The first resolution is lower than the second resolution.

2. An image processing method, comprising: Obtain a first image, wherein the first image is a two-dimensional image; The first image is processed to obtain a first view and a second view, the first view corresponding to a first viewpoint and the second view corresponding to a second viewpoint, and the image quality corresponding to the first view is lower than the image quality corresponding to the second view. The first view and the second view are used to achieve three-dimensional image output; The first image is processed to obtain a first view and a second view, including: The first image is processed to obtain a first initial view corresponding to a first viewpoint. According to a first filling method, the pixels of the background area in the first image are used to fill the first filling area in the first initial view. According to the first filling method, the pixels of the foreground area and the background area in the first image are used to fill the second filling area in the first initial view to obtain the first view. The first filling area and the second filling area constitute the area to be filled in the first initial view. The first image is processed to obtain a second initial view corresponding to a second viewpoint. The second initial view is then filled with content according to a second filling method to obtain a second view. The amount of content filled by the first filling method is less than the amount of content filled by the second filling method.

3. An image processing method, comprising: Obtain a first image, wherein the first image is a two-dimensional image; The first image is processed to obtain a first view and a second view, the first view corresponding to a first viewpoint and the second view corresponding to a second viewpoint, and the image quality corresponding to the first view is lower than the image quality corresponding to the second view. The first view and the second view are used to achieve three-dimensional image output; The first image is processed to obtain a first view and a second view, including: Obtain the image parameters of the first image; When the image parameters meet the processing conditions, a first view corresponding to a first viewpoint is generated according to the first image in a first generation method; and a second view corresponding to a second viewpoint is generated according to the first image in a second generation method. If the image parameters do not meet the processing conditions, a first view corresponding to a first viewpoint and a second view corresponding to a second viewpoint are generated according to the first image and the second generation method. The image quality corresponding to the first generation method is lower than that corresponding to the second generation method.

4. The method according to any one of claims 1-3, wherein the first content in the first view is output to the first viewpoint via an output device, and the second content in the first view is output to the second viewpoint via the output device, wherein the first content and the second content constitute the first view; Furthermore, the second view is output to the second viewpoint via the output device.

5. An image processing apparatus, comprising: A two-dimensional acquisition unit is used to acquire a first image, wherein the first image is a two-dimensional image; A view acquisition unit is used to process the first image to obtain a first view and a second view, wherein the first view corresponds to a first viewpoint and the second view corresponds to a second viewpoint, and the image quality corresponding to the first view is lower than the image quality corresponding to the second view. The first view and the second view are used to achieve three-dimensional image output; The first image is processed to obtain a first view and a second view, including: The first image is processed to obtain a first initial view corresponding to a first viewpoint; The first processing area in the first initial view is processed according to the first resolution; The second processing area in the first initial view is processed according to the second resolution; Wherein, the first processing area and the second processing area constitute a first view corresponding to the first viewpoint; The first image is processed according to the second resolution to obtain a second view corresponding to the second viewpoint; The first resolution is lower than the second resolution.

6. An electronic device, comprising: A memory used to store computer programs and the data generated by the execution of said computer programs; A processor is configured to execute the computer program to: obtain a first image, wherein the first image is a two-dimensional image; The first image is processed to obtain a first view and a second view, the first view corresponding to a first viewpoint and the second view corresponding to a second viewpoint, and the image quality corresponding to the first view is lower than the image quality corresponding to the second view; wherein, the first view and the second view are used to realize three-dimensional image output; The first image is processed to obtain a first view and a second view, including: The first image is processed to obtain a first initial view corresponding to a first viewpoint; The first processing area in the first initial view is processed according to the first resolution; The second processing area in the first initial view is processed according to the second resolution; Wherein, the first processing area and the second processing area constitute a first view corresponding to the first viewpoint; The first image is processed according to the second resolution to obtain a second view corresponding to the second viewpoint; The first resolution is lower than the second resolution.

Images