3D terminal

Abstract

The present application relates to a 3D terminal, comprising: a housing including a first end portion and a second end portion; a display screen disposed in the housing; a 3D photographing device configured to acquire a 3D image of an object to be photographed, including a first color camera disposed at the first end portion and a second color camera disposed at the second end portion. In this way, the distance between the two color cameras can be set by making full use of the size of the 3D terminal itself. Compared with the traditional double cameras arranged adjacent to each other, the 3D photographing effect or the 3D display effect is more in line with the real three-dimensional scene seen by the user.

Description

Technical Field

[0001] The present application relates to a 3D terminal. Background Art

[0002] Currently, some 3D terminals implement 3D shooting by arranging adjacent dual cameras on one side.

[0003] There are at least the following problems in the related art:

[0004] The distance between the dual cameras is limited, making it difficult to obtain a parallax image with a reasonable parallax value, which has an adverse impact on the 3D shooting effect or 3D display effect. For example, from the perspective of the user (who is both the shooter and the viewer), it does not conform to the real three-dimensional scene seen by the user. In addition, due to the limited distance between the dual cameras, the distance or range for 3D shooting is also limited. Summary of the Invention

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary is not a comprehensive review, nor is it intended to identify key / important elements or delineate the scope of protection of these embodiments, but rather serves as a preamble to the subsequent detailed description.

[0006] In one aspect, a 3D terminal is provided, which is characterized by including: a housing including a first end and a second end; a display screen disposed in the housing; a 3D shooting device configured to acquire a 3D image of an object to be shot, including a first color camera disposed at the first end and a second color camera disposed at the second end.

[0007] In this way, the distance between the two color cameras can be set by making full use of the size of the 3D terminal itself. Compared with the traditional adjacent dual cameras, the 3D shooting effect or 3D display effect is more in line with the real three-dimensional scene seen by the user (who is both the shooter and the viewer). In addition, since the distance between the two color cameras is increased compared with the traditional adjacent dual cameras, the distance or range for 3D shooting is also expanded.

[0008] In some embodiments, the housing includes a front side of the housing and a back side of the housing, the display screen is disposed on the front side of the housing, and the 3D shooting device is disposed on the back side of the housing.

[0009] In some embodiments, the 3D terminal further includes a depth camera configured to acquire depth information of the object to be shot, and the depth camera and the first color camera are disposed in the same camera module.

[0010] In some embodiments, the housing further includes a first side and a second side disposed between the first end and the second end; the first color camera is disposed at a corner where the first side intersects the first end, and the second color camera is disposed at a corner where the first side intersects the second end.

[0011] In some embodiments, the 3D terminal further includes a telescopic portion disposed at the second end of the housing and received inside the housing, and the second color camera is disposed in the telescopic portion, and the telescopic portion is configured to move the second color camera by telescoping.

[0012] By this setting method, the selection range of the distance between the dual cameras can be further expanded to obtain a more reasonable parallax value, so as to obtain a better 3D shooting effect or 3D display effect. Based on the setting method in which the distance between the two color cameras is adjustable, the distance between the two color cameras can be adjusted according to the object to be photographed, so as to obtain a better 3D shooting effect or 3D display effect.

[0013] In some embodiments, the 3D terminal further includes a pivoting portion pivotally connected to the second end of the housing and receivable inside the housing, and the second color camera is disposed in the pivoting portion, and the pivoting portion is configured to move the second color camera by pivoting.

[0014] In some embodiments, the back surface of the housing has a recessed portion, and the recessed portion includes a first recessed area and a second recessed area deeper than the first recessed area. The second color camera is disposed on the pivoting portion, so that when the pivoting portion is received, the second color camera is received inwardly in the second recessed area.

[0015] By this setting method, the selection of the distance between the dual cameras can be optimized without affecting the visual flatness and aesthetics of the back surface of the 3D terminal housing, so as to obtain a more reasonable parallax value, so as to obtain a better 3D shooting effect or 3D display effect.

[0016] In some embodiments, the first color camera and the second color camera are located in the same plane.

[0017] In some embodiments, the display screen is a multi-view autostereoscopic 3D display screen, and the 3D terminal further includes a 3D processing device configured to render and display the acquired 3D image in the multi-view autostereoscopic 3D display screen.

[0018] In some embodiments, the multi-view autostereoscopic 3D display screen includes a plurality of composite pixels, each of the plurality of composite pixels includes a plurality of composite sub-pixels, and each of the plurality of composite sub-pixels includes a plurality of sub-pixels corresponding to a plurality of viewpoints.

[0019] In some embodiments, the 3D terminal further includes an eye tracking device configured to determine the spatial position of the user's eyes; and a 3D processing device configured to determine the viewpoint where the eyes are located based on the spatial position of the user's eyes, and render the sub-pixels corresponding to the viewpoint in each composite sub-pixel based on the acquired 3D image.

[0020] In some embodiments, the 3D terminal further includes a camera adjustment unit configured to adjust the shooting parameters of the first color camera and the second color camera to adjust the 3D rendering effect of the acquired 3D image in real time.

[0021] In some embodiments, the camera adjustment unit is further configured to present an operable camera adjustment icon on the multi-viewpoint autostereoscopic 3D display screen.

[0022] In some embodiments, the 3D image is a captured 3D image or a 3D image obtained by framing a to-be-captured image.

[0023] Using such a 3D terminal, it is possible to achieve 3D shooting of "viewing, shooting, and adjusting simultaneously" or "what you see is what you get", that is, to adjust the 3D rendering effect of the acquired 3D image in real time.

[0024] The above general description and the following description are only exemplary and explanatory, and are not used to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] One or more embodiments are illustrated by corresponding drawings. These exemplary illustrations and the drawings do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements. The drawings do not constitute a scale limitation, and:

[0026] Figure 1 is a schematic rear view of a 3D terminal provided by an embodiment of the present disclosure. Among them, the first color camera is disposed at the corner where the first side and the first end intersect, and the second color camera is disposed at the corner where the first side and the second end intersect;

[0027] Figure 2 is a schematic rear view of another 3D terminal provided by an embodiment of the present disclosure. Among them, the first color camera is disposed at the corner where the first side and the first end intersect, and the second color camera is disposed in a telescopic part. The telescopic part is disposed at the second end of the housing and is housed inside the housing;

[0028] Figure 3A is a schematic rear view of another 3D terminal provided by an embodiment of the present disclosure. Among them, the first color camera is disposed at the corner where the first side and the first end intersect, and the second color camera is disposed in a pivoting part. The pivoting part is pivotally connected to the second end of the housing and can be housed inside the housing;

[0029] Figure 3B is Figure 3A a side schematic view of the 3D terminal shown as viewed from the second side;

[0030] Figure 4 is a front schematic view of another 3D terminal provided by an embodiment of the present disclosure;

[0031] Figure 5 is Figure 4 a structural schematic view of the 3D terminal shown;

[0032] Figure 6 is Figure 4 a structural schematic view of the eye tracking device of the 3D terminal shown;

[0033] Figure 7 is Figure 4 a structural schematic view of the eye tracking device of the 3D terminal shown.

[0034] Reference numerals:

[0035] 100: 3D terminal; 110: housing; 111: first end; 112: second end; 113: first side; 114: second side; 120: 3D shooting device; 121: first color camera; 122: second color camera; 123: depth camera; 124: telescopic part; P1: extending path; 125: pivot part; P2: pivoting path; 126: recessed part; 127: first recessed area; 128: second recessed area; 200: 3D terminal; 220: 3D shooting device; 221: first color camera; 222: second color camera; 223: depth camera; 230: eye tracking device; 231: eye tracker; 231a: first black and white camera; 231b: second black and white camera; 231c: black and white camera; 231d: depth camera; 232: eye tracking image processor; 233: eye tracking data interface; 240: multi-viewpoint autostereoscopic 3D display screen; CP: composite pixel; CSP: composite sub-pixel; SP: sub-pixel; 250: camera adjustment unit; OBJ: adjustment object; 260: signal interface; 270: 3D processing device; 280: processor; 281: register; 282: GPU. Detailed implementation manners

[0036] In order to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of explanation, numerous details are provided to give a thorough understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be shown in a simplified manner to simplify the drawings.

[0037] The embodiments of the present disclosure provide a 3D terminal, which can be configured as a smart cellular phone, a tablet computer, a wearable device, a laptop computer, a ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc.

[0038] It should be understood that in the description of the embodiments of the present disclosure, the orientation or positional relationship indicated by terms such as "front", "rear", "left", "right", "upper", "lower", "horizontal", "vertical", "inner", "outer", "top", "bottom", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the embodiments of the present disclosure and simplifying the description, rather than indicating or implying that the device or component referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation on the embodiments of the present disclosure.

[0039] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the defined features.

[0040] In addition, different examples for implementing the embodiments of the present disclosure are disclosed below. For the purpose of simplifying the description, the components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the embodiments of the present disclosure. In addition, the embodiments of the present disclosure may reuse the reference numerals in different examples. This reuse is for the purpose of simplification and clarity and does not itself indicate the relationship between the components and settings involved.

[0041] In this article, "naked-eye 3D display" refers to a technology in which a user can observe 3D display images on a flat display without wearing glasses for 3D display, including but not limited to "parallax barrier", "lenticular lens", "directional backlight" technologies.

[0042] In this article, "multi-viewpoint" has its conventional meaning in the art, meaning that different images displayed by different pixels or sub-pixels of the display screen can be viewed at different positions (viewpoints) in space. In this article, multi-viewpoint will mean at least 3 viewpoints.

[0043] As used herein, "grating" has a broad interpretation in the art, including but not limited to "parallax barrier" gratings and "lens" gratings such as "cylindrical lens" gratings.

[0044] As used herein, "lens" or "lens grating" has its conventional meaning in the art, for example including cylindrical lenses and spherical lenses.

[0045] A conventional "pixel" refers to the smallest display unit in terms of resolution for a 2D display or when a 2D display is being shown.

[0046] However, in some embodiments herein, the "composite pixel" as referred to when applied to multi-viewpoint technology in the field of autostereoscopic 3D displays refers to the smallest display unit when the autostereoscopic 3D display provides multi-viewpoint display, but it does not exclude that a single composite pixel for multi-viewpoint technology may include or be presented as multiple pixels of a 2D display. As used herein, unless specifically stated as a composite pixel or 3D pixel for a "3D display" or "multi-viewpoint" application, a pixel will refer to the smallest display unit for a 2D display. Similarly, when describing the "composite sub-pixel" of an autostereoscopic 3D display for multi-viewpoints, it will refer to the composite sub-pixel of a single color presented in the composite pixel when the autostereoscopic 3D display provides multi-viewpoint display. As used herein, a sub-pixel in a "composite sub-pixel" will refer to the smallest display unit of a single color, which is often corresponding to a viewpoint.

[0047] See Figure 1 , which schematically shows the back of the 3D terminal 100 provided by an embodiment of the present disclosure. The 3D terminal 100 includes a housing 110, a display screen (not shown) disposed on the front of the housing 110, and a 3D shooting device 120 disposed on the back of the housing 110. The housing 110 has opposite first and second ends 111 and 112, and a first side 113 and a second side 114 disposed between the first end 111 and the second end 112, and the second side 114 is opposite to the first side 113. The first end 111, the second end 112, the first side 113, and the second side 114 together define the contour of the housing 110.

[0048] In Figure 1 the illustrated embodiment, the 3D shooting device 120 includes a first color camera 121 and a second color camera 122; wherein, the first color camera 121 is disposed at the corner where the first side 113 intersects with the first end 111, and the second color camera 122 is disposed at the corner where the first side 113 intersects with the second end 112. It can also be envisioned that the two color cameras are respectively disposed at the corners where the second side 114 intersects with the first end 111 and with the second end 112. The two color cameras are flush-mounted in the same plane.

[0049] The first color image is captured by the first color camera 121, and the second color image is captured by the second color camera 122. Since the two color cameras have a spatial position difference, such as a spacing, the two captured color images have a parallax. When the user (who is both the photographer and the viewer) views these two images, the brain uses the parallax of the two color images to restore the depth information of the formed image, that is, the brain superimposes and reconstructs the observed image information to form an image with three-dimensional effects such as front and back, up and down, left and right, near and far.

[0050] Exemplarily, when the photographer takes the horizontal direction of shooting along the vertical direction shown, that is, along the first side 113 or the second side 114, a left parallax image is obtained through the first color camera 121, and a right parallax image is obtained through the second color camera 122. Figure 1 When the photographer takes the horizontal direction of shooting along the vertical direction shown, that is, along the first side 113 or the second side 114, a left parallax image is obtained through the first color camera 121, and a right parallax image is obtained through the second color camera 122.

[0051] Exemplarily, when the photographer takes the horizontal direction of shooting along the horizontal direction shown, that is, along the first end 111 or the second end 112, an upper parallax image is obtained through the first color camera 121, and a lower parallax image is obtained through the second color camera 122. Figure 1 When the photographer takes the horizontal direction of shooting along the horizontal direction shown, that is, along the first end 111 or the second end 112, an upper parallax image is obtained through the first color camera 121, and a lower parallax image is obtained through the second color camera 122.

[0052] In the case of using dual cameras for 3D shooting, or in the case of binocular stereoscopic vision, only when the parallax value of the parallax image is reasonable can a better 3D shooting effect or 3D display effect be obtained. In the parallax image, the magnitude of the parallax value is related to parameters such as the camera focal length, the distance between the dual cameras, and the distance between the object to be photographed and the camera; among them, the camera focal length and the distance between the object to be photographed and the camera are mainly determined by the actual shooting situation. Therefore, the distance between the dual cameras is the key factor affecting the size of the parallax. Through the setting method described above, the distance between the two color cameras can be set by making full use of the size of the 3D terminal itself. Compared with the traditional adjacent dual cameras, the 3D shooting effect or 3D display effect is more in line with the real three-dimensional scene seen by the user.

[0053] In some embodiments, the two color cameras may be wide-angle color cameras.

[0054] In some embodiments, the 3D shooting device 120 further includes a depth camera 123, and the depth camera 123 and the first color camera 121 are disposed in the same camera module. It can also be conceived that the depth camera 123 and the second color camera 122 are disposed in the same camera module. It can be conceived that the camera module may further include other cameras. The depth camera 123 is configured to obtain the depth information of the object to be photographed, which includes point cloud data conforming to the resolution of the depth camera 123. Two color images captured by the two color cameras are synthesized into a synthesized color image, and the synthesized depth information of the object to be photographed is obtained through the synthesized color image. The synthesized depth information in the synthesized color image is adjusted according to the depth information obtained by the depth camera 123. Generally, the resolution of the two color cameras is higher than that of the depth camera, but the synthesized depth information of the synthesized color image obtained by the two color cameras is less accurate than the depth information obtained by the depth camera. In this case, the synthesized color image can be divided into multiple regions to be adjusted according to the resolution of the depth camera 123, and the point cloud data included in the depth information obtained by the depth camera 123 is used to adjust the synthesized depth information of the corresponding regions to be adjusted in the synthesized color image. The adjustment methods include but are not limited to: directly adjusting based on the point cloud data included in the depth information obtained by the depth camera 123, adjusting at a certain ratio, adjusting by a certain value, or adjusting the synthesized depth information of the corresponding regions to be adjusted in the synthesized color image in other ways.

[0055] In some embodiments, the depth camera 123 may be a structured light camera or a time-of-flight (TOF) camera.

[0056] See Figure 2 , which schematically shows the back of another 3D terminal 100 provided by an embodiment of the present disclosure. Figure 2 The shown 3D terminal 100 and Figure 1 The difference between the shown 3D terminals is that the 3D terminal 100 further includes a telescopic part 124. The telescopic part 124 is disposed at the second end 112 of the housing 110 and is housed inside the housing 110. The second color camera 122 is located in the telescopic part 124. The telescopic part 124 can extend from the second end 112 of the housing 110 along the extending path P1, or can be retracted into the housing 110 along the path opposite to the extending path P1. The telescopic part 124 is configured to realize the displacement of the second color camera 122 by telescoping.

[0057] In some embodiments, the telescopic part 124 together with the second color camera 122 can directly extend from the second end 112 of the housing 110 to a fixed extended position. In other embodiments, the telescopic part 124 together with the second color camera 122 can also extend from the second end 112 of the housing 110 to more than one extended position; the more than one extended position can be a multi-stage position or a continuous position.

[0058] In some embodiments, the front surface of the main body of the telescopic part 124 (the side of the telescopic part farther from the front of the 3D terminal) is arranged flush with the back surface of the housing 110; or rather, the front surface of the main body of the telescopic part 124 forms a part of the back surface of the housing 100. That is to say, in both the extended and retracted states, the second color camera 122 is exposed outside the housing 100, so that it can be used for 3D shooting in both states. In other embodiments, the telescopic part 124 together with the second color camera 122 is completely located inside the housing 110 in the retracted state. That is to say, the second color camera 122 can only be used for shooting in the extended state.

[0059] In some embodiments, the ways to trigger the telescopic movement of the telescopic part 124 include but are not limited to: manually pressing, manually sliding, controlling through an application program (APP), controlling through voice (password), etc.

[0060] Through this setting method, the selection range of the distance between the two cameras can be further expanded to obtain a more reasonable parallax value, so as to obtain a better 3D shooting effect or 3D display effect.

[0061] In other embodiments not shown, the telescopic part may include an extension mechanism, such as a hinge-type or elastic-type extension mechanism, so that after the telescopic part extends, the distance between the second color camera disposed therein and the first color camera disposed at the first end can be further increased.

[0062] See Figure 3A , which schematically shows the back surface of another 3D terminal 100 provided by the embodiments of the present disclosure. Figure 3A The shown 3D terminal 100 and Figure 1The difference of the 3D terminal shown is that the 3D terminal 100 further includes a pivoting part 125. The pivoting part 125 is pivotally connected to the second end 112 of the housing 110 and can be received inside the housing 110. The second color camera 122 is located in the pivoting part 125 and is protrudingly provided on the pivoting part 125. The pivoting part 125 can be flipped out from the second end 112 of the housing 110 along the pivoting path P2, and can also be flipped back into the housing 110 along the path opposite to the pivoting path P2. In the flipped-out state, the protrudingly provided second color camera 122 is flush with the first color camera 121 and arranged in the same plane. In the flipped-back state, the back surface of the main body of the pivoting part 125 (the side surface that is farther from the front of the 3D terminal in the flipped-back state) is flush with the back surface of the housing 110; or rather, the back surface of the main body of the pivoting part 125 forms a part of the back surface of the housing 100. The pivoting part 125 is configured to achieve the displacement of the second color camera 122 through pivoting.

[0063] Figure 3B shown Figure 3A A side schematic view of the 3D terminal shown. Through Figure 3B it can be seen more clearly the structures of the pivoting part 125 and the housing 110 and the pivoting path P2. The housing 110 has a recessed part on the back surface, including a first recessed area 127 and a second recessed area 128. The depth of the second recessed area 128 recessed into the housing is deeper than the depth of the first recessed area 127 recessed into the housing. When the pivoting part 125 together with the second color camera 122 provided therein is received in the housing 110, that is, in the flipped-back state, the second color camera 122 is received face-inward in the deeper second recessed area 128, and the main body of the pivoting part 125 is received in the shallower first recessed area 127.

[0064] In some embodiments, the ways to trigger the pivoting of the pivoting part 125 include but are not limited to: manually pressing, controlling through an application program (APP), controlling through voice (password), etc.

[0065] Through this setting method, it is possible to optimize the selection of the double-camera spacing without affecting the visual flatness and aesthetics of the back surface of the 3D terminal, so as to obtain a more reasonable parallax value, and thus obtain a better 3D shooting effect or 3D display effect.

[0066] In some other embodiments not shown, another 3D terminal is provided, which is different from Figure 3A and Figure 3B the 3D terminal shown in that the 3D terminal is provided with a notch at the corner where the first side intersects the second end, and the pivoting part can be rotated out to the left from the notch and can be rotated back to the notch and received inside the housing. In this case, the pivoting part forms the corner where the first side and the second end of the 3D terminal intersect.

[0067] In some other embodiments not shown, the pivot portion may include an extension mechanism, such as a hinge-type or elastic-type extension mechanism, so that after the pivot portion is turned out or rotated out, the distance between the second color camera disposed therein and the first color camera disposed at the first end can be further increased.

[0068] See Figure 4 , taking a tablet computer as an example, schematically showing the front of another 3D terminal 200 provided by an embodiment of the present disclosure. As can be seen from Figure 4 , on the front of the 3D terminal 200, there are a multi-view autostereoscopic 3D display screen 240 and an eye tracking device 230.

[0069] The multi-view autostereoscopic 3D display screen 240 provides i viewpoints for the user, where i≥3, enabling the user to see the 3D rendering effect from different positions. The eye tracking device 230 is configured to determine the spatial position of the user's eyes. The 3D terminal 200 uses the eye tracking device 230 to present a 3D image that conforms to the spatial position of the user's eyes to the user through the multi-view autostereoscopic 3D display screen 240. The 3D image is a captured 3D image or a 3D image obtained by framing a to-be-captured image.

[0070] In some embodiments, the 3D terminal 200 further includes a camera adjustment unit, which is configured to adjust the shooting parameters of the two color cameras of the 3D shooting device, so as to realize real-time adjustment of the 3D rendering effect of the acquired 3D image. The shooting parameters include but are not limited to: the depth of field, contrast, saturation, sharpness, white balance, sensitivity, metering mode, focusing mode, aperture, shutter, etc. of the object to be photographed.

[0071] In some embodiments, as Figure 4 shown, the camera adjustment unit includes a touch-adjustable module 250, and is further configured to present an operable camera adjustment icon in the multi-view autostereoscopic 3D display screen. By way of explanation and not limitation, by moving the slider of the camera adjustment icon, the depth of field of the adjustment object OBJ can be adjusted. The adjustment object OBJ can be the entire object to be photographed or a part of the object to be photographed.

[0072] In Figure 4 the example shown, by clicking on the multi-view autostereoscopic 3D display screen 240 of the 3D terminal 200, a part of the object to be photographed is selected as the adjustment object OBJ, and by moving the slider of the camera adjustment icon, the depth of field of the selected adjustment object OBJ is adjusted.

[0073] Using the 3D terminal described above, it is possible to achieve "shooting while watching and adjusting" or "what you see is what you get" 3D shooting, that is, real-time adjustment of the 3D rendering effect of the acquired 3D image.

[0074] In some embodiments, the camera adjustment unit may further include a setting module optionally presented in the multi-viewpoint autostereoscopic 3D display screen 240, through which, for example, it is possible to set to automatically or manually adjust the synthetic depth-of-field information of the synthetic color image based on the depth-of-field information obtained by the depth camera, whether to turn on the flash, and so on.

[0075] Figure 5 shows Figure 4 a schematic structural diagram of the 3D terminal 200 shown. Referring to Figure 5 , the 3D terminal 200 includes a multi-viewpoint autostereoscopic 3D display screen 240, a 3D shooting device 220, an eye-tracking device 230, a 3D processing device 270, a signal interface 260, and a processor 280. The multi-viewpoint autostereoscopic 3D display screen 240 is communicatively connected to the 3D processing device 270; the eye-tracking device 230 and the signal interface 260 are respectively communicatively connected to the 3D processing device 270; the 3D shooting device 220 and the signal interface 260 are respectively communicatively connected to the processor 280; the 3D shooting device 220 may also be communicatively connected to the 3D processing device 270; the eye-tracking device 230 may also be communicatively connected to the processor 280.

[0076] In some embodiments, the 3D processing device is communicatively connected to the driving device of the multi-viewpoint autostereoscopic 3D display screen.

[0077] The multi-viewpoint autostereoscopic 3D display screen 240 is provided on the front of the 3D terminal 200 and is provided in the housing. The multi-viewpoint autostereoscopic 3D display screen 240 may include a display panel and a grating covering the display panel. The multi-viewpoint autostereoscopic 3D display screen 240 may include m columns and n rows, that is, m×n composite pixels CP, and thus defines an m×n display resolution.

[0078] In some embodiments, the m×n display resolution may be a resolution of high definition (HD) or above full high definition (FHD), including but not limited to, 1280×720, 1920×1080, 1920×1200, 2048×1280, 2560×1440, 3840×2160, etc.

[0079] By way of explanation and not limitation, each composite pixel CP includes a plurality of composite sub-pixels CSP, and each composite sub-pixel is composed of i same-color sub-pixels corresponding to i viewpoints, where i≥3. In Figure 5 the example shown, i = 6, but it is conceivable that i is other values; each composite pixel CP includes three composite sub-pixels CSP, and each composite sub-pixel CSP is composed of 6 same-color sub-pixels SP corresponding to 6 viewpoints (i = 6). The three composite sub-pixels CSP respectively correspond to three colors, namely red (R), green (G), and blue (B). In Figure 5In the illustrated embodiment, the composite sub-pixels CSP in each composite pixel CP are arranged in a single column form, and the sub-pixels SP in each composite sub-pixel CSP are arranged in a single row form. However, it can be conceived that the composite sub-pixels in the composite pixel have other arrangement manners or the sub-pixels in the composite sub-pixel have other arrangement manners.

[0080] The 3D shooting device 220 may include a first color camera 221 and a second color camera 222, and may further include a depth camera 223. For the basic structure and working mode of the 3D shooting device 220, please refer to the above description of Figure 1 、 Figure 2 、 Figure 3A and Figure 3B which will not be elaborated here.

[0081] Exemplarily, the first color image captured by the first color camera 221 and the second color image captured by the second color camera 222 have the same image resolution, which is consistent with the display resolution defined by the multi-view autostereoscopic 3D display screen 240 through the composite pixels CP, including but not limited to, 1280×720, 1920×1080, 1920×1200, 2048×1280, 2560×1440, 3840×2160, etc.

[0082] The processor 280 may include a register 281 and a GPU (Graphics Processing Unit) 282. The register 281 may be configured to temporarily store instructions, data, and addresses. Exemplarily, the register 281 may be configured to receive information about the display requirements of the multi-view autostereoscopic 3D display screen 240. The GPU 282 may be configured to process 3D images; for example, synthesize two color images captured by the 3D shooting device 220 and calculate their parallax values, adjust the synthetic depth information of the synthetic color image by using the depth information obtained by the depth camera 223, etc.

[0083] The 3D processing device 270 is configured to render and display the acquired 3D image on the multi-view autostereoscopic 3D display screen 240. The 3D image may be acquired by the 3D shooting device 220.

[0084] In some embodiments, the 3D terminal 200 may be provided with one or more 3D processing devices 270. When one 3D processing device 270 is provided, the one 3D processing device simultaneously processes the rendering of the sub-pixels SP of each composite sub-pixel CSP of each composite pixel CP of the autostereoscopic 3D display screen 240; when more than one 3D processing device 270 is provided, they process the rendering of the sub-pixels SP of each composite sub-pixel CSP of each composite pixel CP of the autostereoscopic 3D display screen 240 in parallel, serially, or in a combination of serial and parallel. Those skilled in the art will understand that more than one 3D processing device may be allocated and process multiple rows and columns of composite pixels or composite sub-pixels of the autostereoscopic 3D display screen in other ways, which fall within the scope of the embodiments of the present disclosure.

[0085] In some embodiments, the 3D processing device 270 is an FPGA or an ASIC chip or an FPGA or ASIC chipset.

[0086] The eye tracking device 230 is configured to determine the spatial position of the user's eyes. The eye tracking device 230 is communicatively connected to the 3D processing device 270. Thus, the 3D processing device 270 can directly receive the eye tracking data containing the spatial position of the user's eyes and determine the viewing point where the eyes are located based on the spatial position of the eyes, and render the sub-pixels SP corresponding to the determined viewing point in each composite sub-pixel CSP based on the acquired 3D image. As mentioned above, the 3D image may be a captured 3D image or a 3D image obtained by framing a to-be-captured image. By way of explanation and not limitation, the eye tracking device 230 may also be communicatively connected to the processor 280.

[0087] By way of explanation and not limitation, determining the viewing point from the spatial position of the eyes may also be implemented by the eye tracking image processor of the eye tracking device. In this case, the 3D processing device directly receives the eye tracking data containing the viewing point where the user's eyes are located.

[0088] Exemplarily, as Figure 4 shown, the spatial positions of the user's two eyes are tracked by the eye tracking device 230, and it is determined by the 3D processing device that the left eye is at the first viewing point and the right eye is at the fifth viewing point. The sub-pixels SP corresponding to the first viewing point and the fifth viewing point in each composite sub-pixel CSP of each composite pixel CP of the multi-view autostereoscopic 3D display screen 240 are rendered based on the 3D image acquired by the 3D shooting device 220.

[0089] By way of explanation and not limitation, the 3D shooting device 220 may also be communicatively connected to the 3D processing device 270, and the 3D processing device 270 performs operations such as synthesizing two color images, calculating the disparity value, and adjusting the synthesized depth-of-field information of the synthesized color image using the depth-of-field information acquired by the depth-of-field camera 223.

[0090] By way of explanation and not limitation, the 3D shooting device 220 may also be provided with an integrated image processor, and perform operations such as synthesis of two color images, calculation of disparity values, and adjustment of the synthesized depth information of the color image using the depth information obtained by the depth camera 223 through the integrated image processor.

[0091] The signal interface 260 is configured to receive a 3D signal containing a 3D image. The signal interface 260 may be a MIPI, mini-MIPI interface, LVDS interface, min-LVDS interface or Display Port interface.

[0092] In some embodiments, the 3D terminal 200 may further include a codec, configured to decompress, encode and decode the compressed 3D signal and send the decompressed 3D signal to the 3D processing device 270 via the signal interface 260.

[0093] In some embodiments, the 3D terminal 200 may further include a format adjuster, configured to adjust the format of the 3D image contained in the 3D signal, such as the size.

[0094] In the above-mentioned 3D terminal 200, since the display resolution defined by the multi-view autostereoscopic display screen 240 through the composite pixels CP is consistent with the image resolution of the color images captured by the two color cameras of the 3D shooting device 220, and the composite sub-pixels CSP are set corresponding to the viewpoints by their sub-pixels SP, the rendering of the display screen can be realized in a "point-to-point" manner, greatly reducing the calculation amount. In contrast, the transmission and display of images or videos of conventional multi-view autostereoscopic displays are still based on 2D display panels, which not only have problems of resolution reduction and a sharp increase in rendering calculation amount, but may also have problems of multiple format adjustments and image or video display adaptation.

[0095] Figure 6 and Figure 7 respectively show two schematic structural diagrams of the eye tracking device 230.

[0096] As Figure 6 shown, the eye tracking device 230 includes an eye tracker 231, an eye tracking image processor 232 and an eye tracking data interface 233. The eye tracker 231 includes a first black-and-white camera 231a and a second black-and-white camera 231b. The first black-and-white camera 231a is configured to capture a first black-and-white image, and the second black-and-white camera 231b is configured to capture a second black-and-white image. When the eye tracking device 231 is disposed on the front of the 3D terminal 200, the shooting objects of the first black-and-white camera and the second black-and-white camera are the user's face.

[0097] In some embodiments, the eye tracking data interface 233 of the eye tracking device 230 is communicatively connected to the 3D processing device 270 of the 3D terminal 200; thus, the 3D processing device 270 can directly receive eye tracking data. In other embodiments, the eye tracking image processor 232 of the eye tracking device 230 may be communicatively connected to the processor 280 of the 3D terminal 200; thus, the eye tracking data can be transmitted from the processor 280 to the 3D processing device 270 through the eye tracking data interface 233.

[0098] Optionally, the eye tracker 231 is further provided with an infrared emission device (not shown). When the first or second black-and-white camera is working, the infrared emission device is configured to selectively emit infrared light to provide supplementary lighting when the ambient light is insufficient, for example, during night shooting, so as to capture the first or second black-and-white image that can identify the user's face and eyes even under weak ambient light conditions.

[0099] The captured first black-and-white image and second black-and-white image are transmitted to the eye tracking image processor 152. Exemplarily, the eye tracking image processor 152 is configured to have a visual recognition function, such as a face recognition function, and is configured to recognize a face and both eyes based on at least one of the two black-and-white images and determine the spatial positions of both eyes based on the positions of both eyes present in the two black-and-white images. In some embodiments, the first black-and-white camera and the second black-and-white camera are the same black-and-white camera. In other embodiments, the first black-and-white camera and the second black-and-white camera may also be different black-and-white cameras. In this case, in order to determine the spatial positions of both eyes, the first black-and-white image and the second black-and-white image can be calibrated or corrected.

[0100] In some embodiments, the first black-and-white camera and the second black-and-white camera may be wide-angle black-and-white cameras.

[0101] As Figure 7 shown, the eye tracking device 230 includes an eye tracker 231, an eye tracking image processor 232, and an eye tracking data interface 233. The eye tracker 231 includes a black-and-white camera 231c and a depth camera 231d. The black-and-white camera 231c is configured to capture a black-and-white image of the user's face, and the depth camera 231d is configured to obtain depth information of the user's face.

[0102] The captured black-and-white image and the obtained depth information are transmitted to the eye tracking image processor 232. Exemplarily, the eye tracking image processor 232 is configured to have a visual recognition function, such as a face recognition function, and is configured to recognize a face and its both eyes based on the black-and-white image, and determine the spatial positions of both eyes based on the black-and-white image and the obtained depth information.

[0103] In some embodiments, the depth camera 231d is a structured light camera or a TOF camera.

[0104] In some embodiments, the black and white camera 231c is a wide-angle black and white camera.

[0105] In the embodiments of the present disclosure, the components of the device are described in the form of functional modules / units. It can be envisioned that multiple functional modules / units are implemented in one or more "combined" functional modules / units and / or one or more software and / or hardware. It can also be envisioned that a single functional module / unit is implemented by a combination of multiple sub-functional modules or sub-units and / or multiple software and / or hardware. The division of functional modules / units can be only a logical function division. In a specific implementation manner, multiple modules / units can be combined or integrated into another system. In addition, the connections of the modules, units, devices, systems and their components described herein include direct or indirect connections, covering feasible electrical, mechanical, and communication connections, especially including wired or wireless connections between various interfaces, including but not limited to HDMI, radar, USB, WiFi, and cellular networks.

[0106] The exemplary device / apparatus of the present invention has been specifically shown and described with reference to the above embodiments, which are only examples of the best mode for implementing the device / apparatus. Those skilled in the art can understand that various changes can be made to the embodiments of the device / apparatus described herein when implementing the device / apparatus without departing from the spirit and scope of the present invention defined in the appended claims. The appended claims are intended to define the scope of the device / apparatus, so devices / apparatus falling within these claims and their equivalents can be covered.

Claims

1. A 3D terminal, characterized in that, Comprising: A housing including a first end and a second end, wherein the housing includes a front side of the housing and a back side of the housing; A display screen disposed on the front side of the housing, wherein the display screen is a multi-view autostereoscopic 3D display screen, the multi-view autostereoscopic 3D display screen includes a plurality of composite pixels, each of the plurality of composite pixels includes a plurality of composite sub-pixels, and each of the plurality of composite sub-pixels includes a plurality of same-color sub-pixels corresponding to a plurality of viewpoints; A 3D shooting device disposed on the back side of the housing and configured to shoot a 3D image of an object to be shot, including a first color camera disposed at the first end and a second color camera disposed at the second end; An eye tracking device disposed on the front side of the housing and configured to determine the spatial position of the user's eyes; A 3D processing device configured to render and display the acquired 3D image on the multi-view autostereoscopic 3D display screen, wherein the 3D processing device is configured to determine the viewpoint where the eyes are located according to the spatial position of the user's eyes, and based on the acquired 3D image, render the same-color sub-pixels corresponding to the viewpoint in each of the composite sub-pixels, wherein the acquired 3D image is a 3D image shot by the 3D shooting device or a 3D image obtained by framing a to-be-shot image; A camera adjustment unit configured to adjust the shooting parameters of the first color camera and the second color camera to adjust the 3D rendering effect of the acquired 3D image in real time, and the shooting parameters include the depth of field of the acquired 3D image; Wherein the camera adjustment unit is further configured to present an operable camera adjustment icon on the multi-view autostereoscopic 3D display screen, the camera adjustment icon includes a slider, and by moving the slider of the camera adjustment icon, the depth of field of an adjustment object in the 3D image of the object to be shot is adjusted in real time.

2. The 3D terminal according to claim 1, wherein It further includes a depth camera configured to acquire the depth information of the object to be shot, and the depth camera and the first color camera are disposed in the same camera module.

3. The 3D terminal according to claim 1 or 2, wherein The housing further includes a first side and a second side disposed between the first end and the second end; The first color camera is disposed at the corner where the first side intersects the first end, and the second color camera is disposed at the corner where the first side intersects the second end.

4. The 3D terminal according to claim 3, wherein It further includes a telescopic part disposed at the second end of the housing and housed inside the housing, the second color camera is disposed in the telescopic part, and the telescopic part is configured to shift the second color camera by telescoping.

5. The 3D terminal according to claim 3, characterized in that, It further includes a pivoting part pivotally connected to the second end of the housing and capable of being housed inside the housing, the second color camera is disposed in the pivoting part, and the pivoting part is configured to shift the second color camera by pivoting.

6. The 3D terminal according to claim 5, wherein, The back of the housing has a recessed portion, the recessed portion includes a first recessed area and a second recessed area deeper than the first recessed area, and the second color camera is disposed on the pivot portion, so that when the pivot portion is received, the second color camera is received inwardly in the second recessed area.

7. The 3D terminal according to claim 6, characterized in that, The first color camera and the second color camera are located in the same plane.

Images