Naked-eye 3D display method and system based on parallax

Abstract

The present invention relates to a parallax-based naked-eye 3D display method and system. The method comprises: obtaining left and right views, and calculating the distance difference between the left and right views according to the binocular vision stereo principle to obtain a parallax map; calculating depth information according to the parallax map, and performing color processing on the objects in the parallax map according to the depth information to obtain a color distribution map; using image segmentation technology to process the color distribution map, connecting the contours of the objects in the color distribution map, and marking the closed object contours; performing three-dimensional color processing on the parallax map according to the closed object contour, and assigning the processed parallax map to the left and right views, which can improve the naked eye's depth resolution ability for objects, optimize the overall segmentation efficiency of elements, enhance the depth information of the picture and realize the naked eye's depth perception.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a parallax-based naked-eye 3D display method and system. Background Art

[0002] The physical world we live in is a three-dimensional space. In the process of evolution, humans have developed a binocular imaging vision system that adapts to the environment to achieve three-dimensional spatial perception of objects in the three-dimensional world. Humans can generally perceive the physical world through vision, smell, hearing and other means, of which vision accounts for more than 80% of the total information perceived by humans. In the early 19th century, people began to create stereoscopic images. By the mid-20th century, filmmakers began to use special binocular photography technology to shoot movies and use special 3D glasses to present realistic 3D effects when showing movies. In recent years, with the rapid development of computer technology and television technology, 3D technology has also been further developed and applied. For example, people have been able to use 3D printing technology to produce realistic 3D models and objects, use naked-eye 3D technology to achieve 3D TVs that do not require glasses, and apply 3D images and effects in virtual reality and augmented reality technologies. Among them, 3D cinema technology often uses binocular special photography and other technologies to shoot two perspectives of the picture, corresponding to the left and right eyes of the person, and then synthesizes a 3D movie. The cinema is equipped with a 3D projection system, including a 3D projector, 3D screen, 3D glasses, etc.; while the naked-eye 3D display technology uses adaptive grating technology to convert 2D images into 3D images, and controls the transmission of light so that each pixel only sends light to the left eye or the right eye.

[0003] However, traditional naked-eye 3D technology requires a specific angle to achieve the 3D effect, which brings certain inconveniences. It is necessary to choose the appropriate position and angle when watching to get the best visual experience. Therefore, traditional naked-eye 3D technology has the problem of poor naked-eye depth perception. Summary of the invention

[0004] Based on this, in order to solve the above technical problems, a parallax-based naked-eye 3D display method and system are provided, which can improve the naked eye's depth resolution ability for objects and achieve the effect of naked-eye depth perception.

[0005] A parallax-based naked-eye 3D display method, the method comprising:

[0006] Obtaining left and right views, and calculating the distance difference between the left and right views according to the binocular vision stereo principle to obtain a disparity map;

[0007] Calculating depth information according to the disparity map, and performing color processing on the object in the disparity map according to the depth information to obtain a color distribution map;

[0008] Processing the color distribution map using image segmentation technology, connecting the contours of objects in the color distribution map, and marking the contours of closed objects;

[0009] According to the closed object outline, the disparity map is subjected to three-dimensional color processing, and the processed disparity map is assigned to the left and right views.

[0010] In one embodiment, the distance difference between the left and right views is calculated according to the binocular vision stereo principle to obtain a disparity map, including:

[0011] Determining pixel positions of the same object in the left and right views;

[0012] According to the binocular vision stereo principle, the disparity of the same object is calculated through the pixel positions to obtain a disparity map.

[0013] In one of the embodiments, performing color processing on the object in the disparity map according to the depth information to obtain a color distribution map includes:

[0014] Determining each continuous color distribution area of ​​the object in the disparity map;

[0015] Each of the continuous color distribution areas is converted into separated colors of different depths according to the depth information to obtain a color distribution map.

[0016] In one embodiment, the color distribution map is processed using image segmentation technology to connect the contours of objects in the color distribution map and mark the closed object contours, including:

[0017] According to the color distribution map, the separated colors at different depths are connected in outline respectively, and the outline of the closed object is marked;

[0018] Objects at different depths are segmented based on the closed object contours.

[0019] In one embodiment, obtaining the left and right views includes:

[0020] The left and right views are photographed by using a drone device, and the left and right views are acquired by the drone device.

[0021] A parallax-based naked-eye 3D display system, the system comprising:

[0022] A disparity map acquisition module is used to acquire left and right views, and calculate the distance difference between the left and right views according to the binocular vision stereo principle to obtain a disparity map;

[0023] A color processing module, used to calculate depth information according to the disparity map, and perform color processing on the object in the disparity map according to the depth information to obtain a color distribution map;

[0024] An image segmentation module, used to process the color distribution map using image segmentation technology, connect the contours of objects in the color distribution map, and mark the contours of closed objects;

[0025] The three-dimensional processing module is used to perform three-dimensional color processing on the disparity map according to the closed object outline, and assign the processed disparity map to the left and right views.

[0026] The above-mentioned parallax-based naked-eye 3D display method and system obtains a parallax map according to the stereoscopic principle of binocular vision, and then performs color processing and image segmentation on the parallax map through depth information, thereby performing three-dimensional color processing. It can improve the naked eye's depth resolution ability for objects, optimize the overall segmentation efficiency of elements, enhance the depth information of the picture and realize the naked eye's depth perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A diagram showing an application environment of a parallax-based naked-eye 3D display method in one embodiment;

[0028] Figure 2 is a schematic flow chart of a parallax-based naked-eye 3D display method in one embodiment;

[0029] Figure 3 A schematic diagram of image changes in a parallax-based naked-eye 3D display method in one embodiment;

[0030] Figure 4 is a structural block diagram of a parallax-based naked-eye 3D display system in one embodiment;

[0031] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] The parallax-based naked-eye 3D display method provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown. Figure 1As shown, the application environment includes a computer device 110. The computer device 110 can obtain the left and right views, and calculate the distance difference between the left and right views according to the binocular vision stereo principle to obtain a disparity map; the computer device 110 can calculate the depth information according to the disparity map, and perform color processing on the objects in the disparity map according to the depth information to obtain a color distribution map; the computer device 110 can use image segmentation technology to process the color distribution map, connect the objects in the color distribution map with contours, and mark the closed object contours; the computer device 110 can perform three-dimensional color processing on the disparity map according to the closed object contour, and assign the processed disparity map to the left and right views. Among them, the computer device 110 can be, but is not limited to, various personal computers, laptops, smart phones, robots, unmanned aerial vehicles, tablet computers and other devices.

[0034] In one embodiment, Figure 2 As shown, a parallax-based naked-eye 3D display method is provided, comprising the following steps:

[0035] Step 202 , obtain the left and right views, and calculate the distance difference between the left and right views according to the binocular vision stereo principle to obtain a disparity map.

[0036] The computer device can obtain the left and right views, corresponding to the images seen by the left and right eyes of the human eye. Then, the computer device can calculate the distance difference between the left and right views based on the binocular stereo principle, thereby obtaining the most appropriate left and right views and obtaining a disparity map.

[0037] Step 204 , depth information is calculated according to the disparity map, and color processing is performed on the objects in the disparity map according to the depth information to obtain a color distribution map.

[0038] Depth information can be used to represent the distance information between each object in the disparity map and the image acquisition device. By calculating the depth information through the disparity map, the distance between each object in the disparity map and the image acquisition device can be obtained.

[0039] The computer device can perform color processing on the objects in the disparity map according to the depth information. Specifically, the computer device can assign different colors to objects at different distances in the disparity map according to the depth information, for example, the closer the object, the darker the color, and the farther the object, the lighter the color, thereby obtaining a color distribution map.

[0040] Step 206, using image segmentation technology to process the color distribution map, connect the contours of the objects in the color distribution map, and mark the closed object contours.

[0041] The image segmentation technology can implement segmentation operations on objects of the same type in the image. In this embodiment, the computer device can use the image segmentation technology to process the color distribution map, specifically connecting the contours of objects of the same color in the color distribution map, obtaining closed object contours and marking them for further processing.

[0042] Step 208 , performing three-dimensional color processing on the disparity map according to the closed object outline, and assigning the processed disparity map to the left and right views.

[0043] The outline of the same closed object has the same color and the same depth information. The computer device can perform three-dimensional color processing on the disparity map according to the outline of the closed object, and obtain a naked-eye 3D image by fusing the left and right views.

[0044] Specifically, the computer device can assign pixel intensity to the original grayscale image according to the outline of the closed object and distinguish it with different colors, so that the image has a third dimension; using the discrete color distribution and the outline of the closed object, the original grayscale image is assigned a corresponding disparity map, and the color is determined according to the depth information of different objects. The closer the distance to the observer, the brighter the color. In this way, the visual effect of depth perception is achieved, and the depth information of different objects is made clearer and obvious, thereby enhancing the viewing experience.

[0045] In this embodiment, the computer device obtains a disparity map based on the binocular vision stereo principle, and then performs color processing and image segmentation on the disparity map through depth information, thereby performing three-dimensional color processing, which can improve the naked eye's depth resolution ability for objects, optimize the overall segmentation efficiency of elements, enhance the depth information of the picture and realize the depth perception of the naked eye. A parallax-based naked eye 3D display method provided in this embodiment first uses the parallax color distribution of dual-viewing angles to segment the picture elements as a whole, and then assigns different colors according to the original depth in the picture of the segmentation result. The existing naked eye 3D display technology requires the use of grating technology and has certain requirements on the viewing angle of the viewer. In this embodiment, only the color-assigned picture is needed to achieve the effect of naked eye depth perception.

[0046] In one embodiment, a parallax-based naked-eye 3D display method provided may also include a process of calculating the difference between left and right views to obtain a parallax map, the specific process including: determining the pixel position of the same object in the left and right views; and calculating the parallax of the same object through the pixel position according to the binocular vision stereo principle to obtain a parallax map.

[0047] The computer device can identify objects in the left and right views and mark the same object. Then, the computer device can determine the pixel position of the same object in the left and right views, calculate the difference in pixel position of the same object based on the binocular vision stereo principle, and thus calculate the difference distance to obtain a disparity map. Specifically, the computer device can perform disparity estimation based on the obtained left and right views, that is, calculate the difference between different objects in the left and right views, and obtain a disparity map by calculating the disparity, thereby more accurately measuring the distance difference between different objects.

[0048] In one embodiment, a parallax-based naked-eye 3D display method provided may further include a color processing process, the specific process including: determining each continuous color distribution area of ​​an object in a parallax map; converting each continuous color distribution area into separated colors of different depths according to depth information to obtain a color distribution map.

[0049] The computer device can make changes to the color distribution in the disparity map. Specifically, the computer device can determine the continuous color distribution areas of the objects in the disparity map, and convert the continuous color distribution into separated colors of different depths, so as to better segment and distinguish the image elements as a whole. By discretizing the color distribution, the depth levels of different objects in the image can be judged more accurately, thereby achieving better depth resolution.

[0050] In one embodiment, a parallax-based naked-eye 3D display method provided may also include a process of image segmentation, the specific process including: according to a color distribution map, connecting the separated colors of different depths with their contours respectively, marking the contours of closed objects; and segmenting objects of different depths based on the contours of closed objects.

[0051] Due to occlusion, insufficient contrast and other reasons, some pixel disparity estimation errors occur, resulting in the possibility of noise in the color distribution map containing depth information. Computer equipment can calculate the depth based on the difference in disparity, divide the corresponding depth level, divide different object elements, further remove small connected areas, finely divide the elements on the image, obtain closed contours, and obtain a complete and smooth depth layer, that is, connect the separated colors of different depths with contours, and mark the closed object contours. Among them, different contour connected domains not only represent different image elements, but also reflect different depth layers.

[0052] After the color distribution of the disparity map is discretized, the image segmentation technology is applied to obtain the closed object contours according to the discrete color distribution map; based on these contours, the image elements are further segmented to achieve accurate distinction of objects at different depths. Closed contours can avoid problems such as missed segmentation or mis-segmentation, making the depth perception effect more accurate.

[0053] By using contour extraction technology and allocating colors based on depth information, objects at different depths appear in different colors, thus forming a sense of layering visually and realizing naked eye depth perception; the dual-perspective parallax color distribution ensures that the overall image elements can be effectively segmented.

[0054] In one embodiment, a parallax-based naked-eye 3D display method provided may further include a process of acquiring left and right views, and the specific process includes: using a drone device to shoot the left and right views, and acquiring the left and right views through the drone device.

[0055] In one embodiment, in a parallax-based naked-eye 3D display method, the image processing change process is as follows: Figure 3 As shown, the computer device can use the drone to obtain the F1 left view and the F2 right view, wherein the F1 left view corresponds to the image seen by the left eye of the human eye, and the F2 right view corresponds to the image seen by the right eye of the human eye;

[0056] Next, the computer device can calculate the distance difference between the left and right views taken by the drone based on the principle of binocular vision stereo, that is, based on the left and right views obtained by aerial photography, perform disparity estimation, that is, calculate the difference between different objects in the left and right views, and obtain a disparity map by calculating the disparity;

[0057] The computer device can change the color distribution in the disparity map, convert the continuous color distribution into separated colors of different depths, and discretize the color distribution to better segment and distinguish the image elements as a whole;

[0058] After the color distribution of the disparity map is discretized, the computer device can obtain the closed object contours according to the discrete color distribution map by applying image segmentation technology; based on these contours, the image elements are further segmented to achieve accurate distinction of objects at different depths;

[0059] Computer equipment can assign pixel intensity to the original grayscale image based on the connected domain and distinguish them with different colors, giving the image a third dimension; using discrete color distribution and closed contours, the original grayscale image is assigned a corresponding disparity map, and the color is determined based on the depth information of different objects. The closer the distance to the observer, the brighter the color.

[0060] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0061] In one embodiment, Figure 4 As shown, a parallax-based naked-eye 3D display system is provided, comprising: a parallax map acquisition module 410, a color processing module 420, an image segmentation module 430 and a three-dimensional processing module 440, wherein:

[0062] The disparity map acquisition module 410 is used to acquire the left and right views, and calculate the distance difference between the left and right views according to the binocular vision stereo principle to obtain a disparity map;

[0063] A color processing module 420 is used to calculate depth information according to the disparity map, and perform color processing on the objects in the disparity map according to the depth information to obtain a color distribution map;

[0064] An image segmentation module 430 is used to process the color distribution map using image segmentation technology, connect the contours of objects in the color distribution map, and mark the contours of closed objects;

[0065] The three-dimensional processing module 440 is used to perform three-dimensional color processing on the disparity map according to the closed object outline, and assign the processed disparity map to the left and right views.

[0066] In one embodiment, the disparity map acquisition module 410 is further used to determine the pixel positions of the same object in the left and right views; according to the binocular vision stereo principle, the disparity of the same object is calculated through the pixel positions to obtain the disparity map.

[0067] In one embodiment, the color processing module 420 is further used to determine each continuous color distribution area of ​​the object in the disparity map; and convert each continuous color distribution area into separated colors of different depths according to the depth information to obtain a color distribution map.

[0068] In one embodiment, the image segmentation module 430 is further used to connect the separated colors at different depths according to the color distribution map, mark the closed object contours, and segment the objects at different depths based on the closed object contours.

[0069] In one embodiment, the disparity map acquisition module 410 is further used to shoot left and right views using a drone device, and acquire the left and right views through the drone device.

[0070] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a naked-eye 3D display method based on parallax is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0071] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0072] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of a parallax-based naked-eye 3D display method when executing the computer program.

[0073] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a parallax-based naked-eye 3D display method are implemented.

[0074] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0075] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0076] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A parallax-based naked eye 3D display method, characterized in that: The method comprises: Obtaining left and right views, and calculating the distance difference between the left and right views according to the binocular vision stereo principle to obtain a disparity map; Calculating depth information according to the disparity map, and performing color processing on the object in the disparity map according to the depth information to obtain a color distribution map; The color distribution map is processed using image segmentation technology, and the objects in the color distribution map are connected in outline, and the closed object outline is marked, including: according to the color distribution map, the separated colors of different depths are connected in outline respectively, and the closed object outline is marked; objects of different depths are segmented based on the closed object outline; According to the closed object outline, the disparity map is subjected to three-dimensional color processing, and the processed disparity map is assigned to the left and right views.

2. The parallax-based naked-eye 3D display method according to claim 1, characterized in that: The method of calculating the distance difference between the left and right views according to the binocular vision stereo principle to obtain a disparity map includes: Determining pixel positions of the same object in the left and right views; According to the binocular vision stereo principle, the disparity of the same object is calculated through the pixel positions to obtain a disparity map.

3. The parallax-based naked-eye 3D display method according to claim 1, characterized in that: The step of performing color processing on the object in the disparity map according to the depth information to obtain a color distribution map includes: Determining each continuous color distribution area of ​​the object in the disparity map; Each of the continuous color distribution areas is converted into separated colors of different depths according to the depth information to obtain a color distribution map.

4. The parallax-based naked-eye 3D display method according to claim 1, characterized in that: The obtaining of the left and right views comprises: The left and right views are photographed by using a drone device, and the left and right views are acquired by the drone device.

5. A parallax-based naked-eye 3D display system, characterized in that: The system comprises: A disparity map acquisition module is used to acquire left and right views, and calculate the distance difference between the left and right views according to the binocular vision stereo principle to obtain a disparity map; A color processing module, used to calculate depth information according to the disparity map, and perform color processing on the object in the disparity map according to the depth information to obtain a color distribution map; An image segmentation module is used to process the color distribution map using image segmentation technology, connect the contours of the objects in the color distribution map, and mark the closed object contours; it is also used to connect the separated colors of different depths according to the color distribution map, mark the closed object contours; and segment the objects of different depths based on the closed object contours; The three-dimensional processing module is used to perform three-dimensional color processing on the disparity map according to the closed object outline, and assign the processed disparity map to the left and right views.

6. The parallax-based naked-eye 3D display system according to claim 5, characterized in that: The disparity map acquisition module is also used to determine the pixel positions of the same object in the left and right views; and according to the binocular vision stereo principle, the disparity of the same object is calculated through the pixel positions to obtain the disparity map.

7. The parallax-based naked-eye 3D display system according to claim 5, characterized in that: The color processing module is further used to determine each continuous color distribution area of ​​the object in the disparity map; and convert each of the continuous color distribution areas into separated colors of different depths according to the depth information to obtain a color distribution map.

8. The parallax-based naked-eye 3D display system according to claim 5, characterized in that: The disparity map acquisition module is also used to use a drone device to shoot left and right views, and acquire the left and right views through the drone device.

Images