Stereoscopic (3D) digital enhancement processing method for two-dimensional (2D) images
By setting vanishing points or vertical vanishing lines on the two-dimensional image and adjusting the object width using geometric perspective, the problem of weakening the three-dimensional effect of binocular parallax is solved, and the effect of simplifying processing and significantly enhancing the three-dimensional sense is achieved.
Patent Information
- Application Number
- CN202280095251.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-28
- Publication Date
- 2024-11-26
AI Technical Summary
Binocular parallax weakens the three-dimensional effect in two-dimensional images. The prior art is difficult to significantly enhance the three-dimensional information by simply adjusting brightness, color, saturation and other parameters. Traditional image processing methods are complex and difficult to process in real time.
By setting a vanishing point or vertical vanishing line on the image, geometric perspective enhances the three-dimensional effect. Objects close to the vanishing line become visually distant and objects far away from the vanishing line become closer, and the horizontal width of the object is adjusted to enhance the three-dimensional sense.
The image processing process is simplified and the three-dimensional sense of two-dimensional images is significantly enhanced. It is suitable for various scenes and resolutions. The processed images are more visually three-dimensional and are suitable for a variety of display devices.
Smart Images

Figure CN119032376A_ABST
Abstract
Description
Technical Field of the Invention
[0001] The present invention belongs to the field of digital image processing. Specifically, the present invention enhances the stereoscopic effect of a two-dimensional image by a digital image processing method. Background of the Invention
[0002] The stereoscopic information of an image includes binocular parallax, eye focus, convergence, motion parallax, aerial perspective (bringing changes in color and brightness), object occlusion, geometric perspective and other information. Among all the above information, binocular parallax is the strongest stereoscopic information. Due to the horizontal distance between a person's eyes, when viewing a scene, there is a difference in the image formation on the retinas of the two eyes, and the brain uses this information (binocular parallax) to form a sense of stereo. Most stereoscopic movies, stereoscopic televisions and other stereoscopic display devices achieve stereoscopic effects by using binocular parallax. However, when viewing a flat two-dimensional image, the eyes will focus on each pixel on a plane, and because of the existence of binocular parallax, the brain can perceive that this is a two-dimensional image on a plane. As the strongest stereoscopic information, binocular parallax weakens the stereoscopic effect brought about by other stereoscopic information in the two-dimensional image. The present invention aims to produce and strengthen the stereoscopic effect by enhancing other stereoscopic information when viewing a two-dimensional image with the naked eye. SUMMARY OF THE INVENTION
[0003] Binocular parallax weakens other stereo information in a two-dimensional image. There are some methods that can produce and enhance the stereo effect when viewing a two-dimensional image on a flat display by enhancing other stereo information in the two-dimensional image. The stereo information of a two-dimensional image displayed on a traditional device (such as a traditional TV) can be enhanced by adjusting brightness, color, saturation, contrast, etc., but the effect of adjusting these parameters is not obvious. Among all the stereo information in a two-dimensional image, geometric perspective is very strong and important information. When painting, relying on geometric perspective can be very effective in enhancing the stereo effect. When using geometric perspective, it is very important to determine the vanishing point (see Figure 1 ). Many traditional image processing methods for enhancing or generating a three-dimensional effect are relatively complex. These traditional methods usually take a long time to process, and therefore are more difficult to process in real time. The present invention proposes a method of setting a vanishing point or a vertical vanishing line on an image and using the vanishing point or the vertical vanishing line to enhance geometric perspective. By applying this method, the three-dimensional effect of a two-dimensional image can be enhanced.
[0004] In the geometric perspective of an image, there can be one or more vanishing points. Setting only one vanishing point can simplify the image processing process. The vanishing point can be replaced by a vertical vanishing line, which can also simplify the image processing process. Figure 2 For the two-dimensional image shown, the vertical vanishing line can be set at the horizontal center of the image. In this case, the vertical vanishing line is the center line of the image.
[0005] When viewed by a person, a distant object usually appears smaller in the field of vision than a closer object. Compared to other objects in the image, objects close to the vertical vanishing line of the image are usually farthest from the viewer, therefore, the image processing method proposed in the present invention reduces the horizontal width of objects close to the vertical vanishing line relative to their initial horizontal width on the image. Compared to other objects in the image, objects far from the vertical vanishing line of the image are usually closest to the viewer, therefore, the present invention increases the horizontal width of objects far from the vertical vanishing line relative to their initial horizontal width on the image. When the vertical vanishing line is also the center line of the image, on the image, the left and right edges of the image are farthest from the vertical vanishing line. Therefore, when the vertical vanishing line is also the center line of the image, the image processing method proposed in the present invention reduces the horizontal width of objects close to the center line of the image relative to their initial horizontal width on the image, and at the same time, this method increases the horizontal width of objects close to the left and right edges of the image relative to their initial horizontal width on the image. This method has two characteristics: 1. By setting the vertical vanishing line of geometric perspective, the three-dimensional information of the image is enhanced; 2. Objects close to the vertical vanishing line of the image are visually farther away from the audience, and objects far from the vertical vanishing line of the image are visually closer to the audience, so the geometric perspective of the processed image is enhanced. The present invention is applicable and effective to various scenes and film sources, such as movies, sports games, variety shows, TV dramas, fashion shows, etc.
[0006] Figure 3-6 Four examples of this image processing method are shown (viewed from above the screen).
[0004]
[0005]
[0006] introduces an image processing method for setting a vertical vanishing line. This method can also be extended to set a vanishing point instead of a vertical vanishing line. For example, in Fig.15 In this method, the vanishing point is set at the center point c of the image, and the image is divided into five regions (d12, d3, d4, d5, d6) for processing. When the vanishing point is set, this method shrinks the area near the vanishing point in the horizontal and vertical directions. For example, Fig.15 In , the vanishing point is in area d12, so area d12 is reduced after processing. When setting the vanishing point, this method makes the area far from the vanishing point larger in the horizontal and vertical directions. For example, in Fig.15 In the figure, region d6 is the region at the edge and farthest from the vanishing point, so region d6 increases after processing. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 Schematic diagram of the vanishing point. Figure 2 Shown is a rectangular image and its center line. If the image is rectangular and fills the entire screen, the center line of the image is also the center line of the screen. Figure 3-6 Shown are four examples of this image processing method. Figure 7 The original image is shown. To make the example easier to understand, assume Figure 3-7 The image fills the entire screen. Figure 3-7 The perspectives in the diagram are all looking down at the screen, with the screen facing the bottom of the diagram and the audience watching the screen at the bottom of the diagram. Figure 3-7 In , C represents the center line of the image, L represents the left edge of the image, and R represents the right edge of the image. Figure 3-7 The viewing angles in the figure are all looking down at the screen. Figure 3-7 The center line C, left edge L and right edge R in the figure are also from a top-down perspective, so the three line segments C, L and R are Figure 3-7 Shown as dots. Figure 3 For the first example. Figure 3 In the image, the vertical vanishing line is set at the center line C. After processing, the horizontal width of the image is reduced in the area close to the vertical vanishing line, and the horizontal width of the image is increased in the area far from the vertical vanishing line. Because the original image is displayed on the screen, after processing, the horizontal width of the image near the vertical vanishing line ( Figure 3 The area at the center line C in the image appears to be located behind the screen and farther from the viewer's eyes. At the same time, because the vertical vanishing line of the image is set at the center line C, the left edge L and the right edge R are at Figure 3 is farthest from the vertical vanishing line. Figure 3 This method makes the horizontal width of the image close to the left edge L and the right edge R (in Figure 3 The area farthest from the vertical vanishing line in the image is enlarged, so after processing, the areas near the left and right edges appear to be in front of the screen and closer to the audience's eyes. Figure 3 In , when the original image is processed at the horizontal 1 / 4 and 3 / 4 points from the left edge, the horizontal magnification is set to 1. Therefore, at the horizontal 1 / 4 and 3 / 4 points from the left edge, the horizontal width of the image after processing remains unchanged relative to the initial horizontal width, and these two points on the image still appear to be on the screen. Figure 3 In , the processed image is symmetrical in shape, appearing to be behind the screen at the horizontal center and in front of the screen at the left and right edges. Figure 7 In the example, the center line C, the left edge L, and the right edge R all appear to be on the screen. Figure 3 In the , the horizontal magnification decreases linearly from L to C and from R to C, and at the horizontal 1 / 4 and 3 / 4 points from the left edge, the horizontal magnification is 1. Figure 3 , horizontal magnification is greatest at L and R and smallest at C. Figure 4 For the second example. Figure 4In , the vertical vanishing line of the image is set at the center line C, and L and R represent the left and right edges of the image respectively. Figure 4 The horizontal magnification of all regions is less than or equal to 1. Figure 5 For the third example. Figure 5 In , the vertical vanishing line of the image is set at the center line C, and L and R represent the left and right edges of the image respectively. Figure 5 The horizontal magnification of all regions is greater than or equal to 1. Figure 6 This is the fourth example. Figure 6 In , the processed image is a tilted plane. Figure 6 In , C represents the center line of the image, and L and R represent the left and right edges of the image, respectively. Figure 6 In the case where the processed image is an inclined plane, one of the left or right edges has the smallest horizontal magnification in the entire image, and the other of the left or right edges has the largest horizontal magnification in the entire image. for Figure 3-6 In any of the four examples shown, the vertical vanishing line may not be set at the center line of the image. Figure 8 As shown in Figure 1, an example of processing eight regions of an image to enhance stereo information is shown. Figure 8 In the method, the original image is divided into 8 regions with the same horizontal width. This method can also set some or all regions in the original image to different horizontal widths, and the number of regions may not be 8. Fig. 9 As shown, Figure 8 An example of adding a step to process 8 regions of the image to enhance the stereo information is shown in FIG. 1 . The additional step is to round up or down the number of horizontal pixels when calculating the horizontal width of each region after processing. Fig.10 The image processing shown is at the camera or video recording end. Fig.11 The figure shows the image processing in the post-production of a TV program. Fig.12 The figure shows the image processing in the TV program receiving link of the TV set or set-top box. Fig.13 The figure shows the image processing in the movie projection process. Fig.14 Shown is the image processing of the finished TV program. Fig.15 As shown, it is an image processing by setting the vanishing point to enhance the three-dimensional information, and Fig.15 The image fills the entire screen. Fig.15In , the vanishing point of the image is set at the center point c of the image, and the image is divided into 5 areas (d12, d3, d4, d5, d6) for processing. DETAILED DESCRIPTION exist Figure 8 In the example shown, the original image is horizontally divided into 8 regions of the same horizontal width, and each region in the image is processed separately. The vertical vanishing line of the image is set at the center line. The 4 regions on the left are symmetrical to the 4 regions on the right with the vertical vanishing line as the boundary. When processing, the horizontal width of a region can be widened, narrowed, or maintained. Within each region, the widening or narrowing process is applied evenly, and each region retains its information after processing. Figure 8 In the example shown, the difference in the number of horizontal pixels that any two adjacent regions are widened or narrowed is the same (except for the two regions immediately adjacent to the center line, since the two center regions have the same horizontal width after processing). Figure 8 In , the number of pixels is width*height (such as 1920*1080), and n is a positive integer used to represent the change in the number of horizontal pixels. The value n / 3 in the second area from the left means that the horizontal width of the second area in the original image has increased by n / 3 pixels after processing. Similarly, any positive value in the area means that the area has been widened after processing, and any negative value in the area means that the area has been narrowed after processing. Figure 8 Of the 8 areas shown, two areas next to the vertical vanishing line (set at the center line in this example) have a value of -n, which means that the horizontal width of the two areas has been reduced by n pixels each. In the 8 areas (from left to right), the number of horizontal pixels widened in each area is n, n / 3, -n / 3, -n, -n, -n / 3, n / 3, and n, respectively. With the center line as the boundary, the 4 areas on the left are grouped together, and the 4 areas on the right are grouped together. In any of the 4 areas in the two groups, the difference in the number of horizontal pixels widened or narrowed between any two adjacent areas is 2n / 3 pixels, so in Figure 8 In , the horizontal magnification changes linearly from the vertical vanishing line to the left and right edges of the image. Figure 8 In the example shown, the sum of the horizontal pixels widened in all regions is equal to the sum of the horizontal pixels narrowed in all regions, so the total number of horizontal pixels of the entire image remains unchanged after processing. Fig. 9 As shown, when this method is applied, the processed horizontal width of each region can be converted to an integer by rounding up or down. Fig. 9 In the figure, rounding is applied and the number of horizontal pixels widened in each region (from left to right) after rounding is n, [n / 3], -[n / 3], -n, -n, -[n / 3], [n / 3], and n. For example, the resolutions of standard definition TV, high definition TV, 4K ultra high definition TV, and 8K ultra high definition TV are 720*576, 1920*1080, 3840*2160, and 7680*4320, respectively. The horizontal pixel counts are 720, 1920, 3840, and 7680, respectively. When processing images of different resolutions, this method can apply the same combination of horizontal magnifications to achieve the same effect on the output (processed) image. Figure 8 In the example shown, k is a preset parameter, and n in
[0024] (used to represent the change in the number of horizontal pixels) can be calculated by dividing the number of horizontal pixels by k. For example, the number of horizontal pixels of a high-definition television is 1920, and when performing image processing, n can be calculated by n=1920 / k. exist Figure 8 In the example shown, the original image is divided into 8 regions of the same horizontal width, and the horizontal magnification increases from less than 1 to greater than 1 from the vertical vanishing line (set at the center line in this example) to the left and right edges of the image. When this method is applied, the original image can be divided into any number of regions to be processed, and the number of regions can be odd or even. The vertical vanishing line can be set at the center line of the image or at other positions. Before processing, any two regions in the original image can have the same or different horizontal widths, and the arrangement of the regions can be symmetrical or asymmetrical. The change in horizontal magnification from the vertical vanishing line to either the left or right edge of the image can be linear or nonlinear. After the region is processed, the minimum change in the horizontal width of the region can be 0. The sum of the widened horizontal pixels in all regions may be equal to or not equal to the sum of the narrowed horizontal pixels in all regions, and the horizontal width of the entire image may or may not be changed after the entire image is processed. The original image can be divided into regions of arbitrary shapes, and the arrangement of the regions can be horizontal or non-horizontal. exist Figure 3-6 The four examples shown show different positions and settings of the vertical vanishing line (eg, whether the vertical vanishing line is located at the center line of the image), and different settings of the horizontal magnification. A vertical treatment of geometric perspective can also be incorporated into this method by replacing vertical vanishing lines with vanishing points. Fig.15 An example of setting a vanishing point is shown in , and the vanishing point is set at the center point c of the image. When setting a vanishing point, the original image is also divided into multiple regions to be processed, and the regions can be of any shape. When setting a vanishing point, this method shrinks the area close to the vanishing point in the horizontal and vertical directions, and this method enlarges the area far from the vanishing point in the horizontal and vertical directions. The vanishing point can be set at the center point of the image or at other locations. After applying the image processing method of the present invention, if there is a need to adjust the vertical or horizontal size of the processed image, the processed image can be evenly enlarged or reduced in size as a whole, and the enhanced three-dimensional effect can be retained. like Figure 10-14 As shown, this image processing method can be used in the camera link, the intermediate image and video production link, the display device, the finished image and video, etc.
Claims
1. An image processing method, the method comprising: Select a position on the image and set a vertical vanishing line; Divide the image into multiple regions; Reduce the horizontal width of the area near the vertical vanishing line; as well as Increase the horizontal width of the area away from the vertical vanishing line.
2. The image processing method according to claim 1, characterized in that: The vertical vanishing line of the image is located at the center line of the image.
3. The image processing method according to claim 1, characterized in that: The vertical vanishing line of the image is not located at the center line of the image.
4. The image processing method according to claim 1, characterized in that: In the original image all regions have the same horizontal width.
5. The image processing method according to claim 1, characterized in that: The horizontal widths of some or all regions in the original image are different.
6. The image processing method according to claim 1, characterized in that: The total horizontal width of the image remains unchanged after image processing.
7. The image processing method according to claim 1, characterized in that: The total horizontal width of the image changes after image processing.
8. The image processing method according to claim 1, characterized in that: The horizontal magnification changes linearly from the vertical vanishing line of the image to the left and right edges of the image.
9. The image processing method according to claim 1, characterized in that: The change in horizontal magnification from the vertical vanishing line of the image to the left and right edges of the image is nonlinear.
10. The image processing method according to claim 1, characterized in that: The change in horizontal magnification is linear from the vertical vanishing line of the image to one of the left or right edge of the image, and non-linear from the vertical vanishing line of the image to the other of the left or right edge of the image.
11. An image processing method, the method comprising: Select a location on the image and set a vanishing point; Divide the image into multiple regions; Reduce the area near the vanishing point horizontally and vertically; as well as Increase the area away from the vanishing point both horizontally and vertically.
12. The image processing method according to claim 11, characterized in that: The vanishing point of an image is at the center of the image.
13. The image processing method according to claim 11, characterized in that: The vanishing point of an image is not located at the center of the image.
