Electronic device, display equipment and stereoscopic imaging method thereof

Abstract

The invention provides an electronic device, a display device and a stereo imaging method thereof. The electronic device comprises a display unit, an input unit, an artificial intelligence frame filling model, a detection unit and an imaging unit. The input unit is used for obtaining a first original image and a second original image. The first original image is shot at a first angle, and the second original image is shot at a second angle. The first angle is different from the second angle. The artificial intelligence frame filling model is used for generating a plurality of frames of intermediate generation images. The detection unit is used for detecting a viewing angle of a user relative to the display unit. The imaging unit is used for displaying one of the first original image, the plurality of intermediate generated images and the second original image on the display unit according to the viewing angle.

Description

TECHNICAL FIELD

[0001] The present application relates to an electronic device and a control method thereof, and particularly to an electronic device, a display device and a stereoscopic imaging method thereof. BACKGROUND

[0002] Nowadays, 3D cameras have gradually matured, but in order to view the photos taken by the 3D cameras, the users have to additionally purchase related devices, such as naked-eye 3D screens or 3D glasses.

[0003] In order to make 3D images more popular, the industry is committed to developing stereoscopic display technology that can be applied to general displays. SUMMARY

[0004] The present application relates to an electronic device, a display device and a stereoscopic imaging method thereof, which generates generated images corresponding to multiple viewing angles by using artificial intelligence frame interpolation technology, and presents a picture on a display unit according to the viewing angle of a user, so that the user feels that he is watching around the stereoscopic foreground object.

[0005] According to an aspect of the present application, a stereoscopic imaging method is provided. The stereoscopic imaging method includes the following steps. A first original image and a second original image are obtained. The first original image is taken at a first angle, and the second original image is taken at a second angle. The first angle is different from the second angle. A plurality of intermediate generated images are generated by using an artificial intelligence frame interpolation model. A viewing angle of a user relative to a display unit is detected. According to the viewing angle, one of the first original image, the plurality of intermediate generated images and the second original image is presented on the display unit.

[0006] According to another aspect of the present application, an electronic device is provided. The electronic device includes a display unit, an input unit, an artificial intelligence frame interpolation model, a detection unit and an imaging unit. The input unit is used to obtain a first original image and a second original image. The first original image is taken at a first angle, and the second original image is taken at a second angle. The first angle is different from the second angle. The artificial intelligence frame interpolation model is used to generate a plurality of intermediate generated images. The detection unit is used to detect a viewing angle of a user relative to the display unit. The imaging unit is used to present one of the first original image, the plurality of intermediate generated images and the second original image on the display unit according to the viewing angle.

[0007] According to still another aspect of the present application, a display device is provided. The display device includes a display unit, an input unit, an artificial intelligence frame filling model, and an imaging unit. The input unit is configured to obtain a first original image and a second original image. The first original image is captured at a first angle. The second original image is captured at a second angle. The first angle is different from the second angle. The artificial intelligence frame filling model is configured to generate a plurality of intermediate generated images. The imaging unit is configured to present the first original image, the plurality of intermediate generated images, and the second original image in the display unit in turn. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to make the above and other objects, features and advantages of the present application more comprehensible, specific embodiments accompanied by drawings are described in detail as follows:

[0009] Figures 1A-1B An example of a stereoscopic imaging method according to an embodiment of the present application is illustrated.

[0010] Figure 2 A diagram showing a relationship between a foreground object and a user is shown.

[0011] Figure 3 A block diagram of an electronic device according to an embodiment is shown.

[0012] Figure 4 A flowchart of a stereoscopic imaging method according to an embodiment is shown.

[0013] Figure 5 An example of step S120 is illustrated.

[0014] Figures 6-7 An example of step S130 is illustrated.

[0015] Figure 8 An example of steps S150-S160 is illustrated.

[0016] Figure 9 A block diagram of a display device 200 according to another embodiment of the present application is shown.

[0017] Figure 10 A flowchart of a stereoscopic imaging method according to an embodiment is shown.

[0018] Figure 11 An example of step S260 is illustrated.

[0019] REFERENCE NUMERALS:

[0020] 100: electronic device

[0021] 110, 210: display unit

[0022] 120, 220: input unit

[0023] 130,230: Artificial Intelligence Frame Interpolation Model

[0024] 140, 240: Storage Units

[0025] 150: Detection Unit

[0026] 160, 260: Imaging unit

[0027] 200: Display device

[0028] 900: 3D Camera

[0029] A01, A02, VA: Viewing Angle

[0030] A1: First Angle

[0031] A65: Second Angle

[0032] FM: Footage shot in front of the screen

[0033] IM01, IM02: Images

[0034] IM1: First Original Image

[0035] IM2~IM64: Intermediate generated images

[0036] IM65: Second Original Image

[0037] O1: Foreground object

[0038] S120, S130, S150, S160, S220, S230, S260: Steps

[0039] U1: User Detailed Implementation

[0040] The technical terms used in this specification are based on common terminology in the field. Where this specification provides further explanation or definition of certain terms, the interpretation of those terms shall be based on the explanation or definition provided in this specification. Each embodiment of the present invention has one or more technical features. Where feasible, those skilled in the art may selectively implement some or all of the technical features in any embodiment, or selectively combine some or all of the technical features in the multiple embodiments.

[0041] Please refer to Figures 1A-1B Examples illustrate a stereoscopic imaging method according to an embodiment of the present invention. For example... Figure 1A As shown, user U1 can view the display unit 110 from viewing angle A01, and the display unit 110 displays the image IM01 of the foreground object O1 corresponding to viewing angle A01. Figure 1BAs shown, the user U1 can watch the display unit 110 at an angle A02, and the display unit 110 presents an image IM02 of the foreground object O1 corresponding to the angle A02.

[0042] Please refer to Figure 2 which shows a diagram of the relationship between the foreground object O1 and the user U1. Once the user U1 changes the viewing angle, the display unit 110 will also present the image of the foreground object O1 at a different angle accordingly. It is as if the user U1 is watching around the three-dimensional foreground object O1.

[0043] Please refer to Figure 3 which shows a block diagram of the electronic device 100 according to an embodiment. The electronic device 100 includes a display unit 110, an input unit 120, an artificial intelligence frame filling model 130, a storage unit 140, a detection unit 150, and an imaging unit 160. The display unit 110 is used to display a picture, such as a liquid crystal display panel, an OLED display panel, or an electronic paper display panel.

[0044] The input unit 120 is used to receive data, such as a transmission port, a wireless transmission module, or a wired network transmission module.

[0045] The artificial intelligence frame filling model 130 is used to generate a picture, such as a SpatialParallax-AI system, which can be implemented by a circuit, a circuit board, a storage device storing program code, or a chip.

[0046] The detection unit 150 is used to track and detect faces, such as a convolutional neural network (CNN), a long short-term memory network (LSTM), a recurrent neural network (RNN), a generative adversarial network (GAN), and a radial basis function network (RBFN), which can be implemented by a circuit, a circuit board, a storage device storing program code, or a chip.

[0047] The imaging unit 160 is used to present the generated picture on the display unit 110, which can be implemented by a circuit, a circuit board, a storage device storing program code, or a chip.

[0048] The chip is, for example, a central processing unit (CPU), or other programmable general purpose or special purpose micro control unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar components or combinations thereof.

[0049] The storage unit 140 is, for example, any type of fixed or mobile random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar components or combinations thereof, and is used to store a plurality of modules or various application programs executable by the processor.

[0050] In the present embodiment, the artificial intelligence frame filling technology is used to generate generated images corresponding to a plurality of viewing angles, and the display unit 110 presents a picture according to the viewing angle of the user U1, so that the user U1 feels that he is watching around the three-dimensional foreground object O1. The operation of each component will be described in detail below in conjunction with a flowchart.

[0051] Referring to Figure 4 , a flowchart of a stereoscopic imaging method according to an embodiment is shown. The stereoscopic imaging method includes steps S120, S130, S150, and S160.

[0052] Referring to Figure 5The example illustrates step S120. In step S120, input unit 120 obtains a first original image IM1 and a second original image IM65. The first original image IM1 is captured at a first angle A1, and the second original image IM65 is captured at a second angle A65. The first angle A1 is different from the second angle A65. The first original image IM1 and the second original image IM65 are, for example, a left-eye image and a right-eye image captured by a 3D camera 900.

[0053] Next, please refer to Figures 6 and 7, which illustrate step S130. In step S130, the AI ​​frame interpolation model 130 generates several intermediate generated images IM2 to IM64. The AI ​​frame interpolation model 130 generates one output image each time based on two input images. Figure 6 As shown, the AI ​​frame interpolation model 130 obtains the intermediate generated image IM33 based on the first original image IM1 and the second original image IM65. The AI ​​frame interpolation model 130 then obtains the intermediate generated image IM17 based on the first original image IM1 and the intermediate generated image IM33. The AI ​​frame interpolation model 130 then obtains the intermediate generated image IM49 based on the intermediate generated image IM33 and the second original image IM65.

[0054] like Figure 7 As shown, the AI ​​frame interpolation model 130 then obtains the intermediate generated image IM9 based on the first original image IM1 and the intermediate generated image IM17. The AI ​​frame interpolation model 130 then obtains the intermediate generated image IM25 based on the intermediate generated image IM17 and the intermediate generated image IM33. The AI ​​frame interpolation model 130 then obtains the intermediate generated image IM41 based on the intermediate generated image IM33 and the intermediate generated image IM46. The AI ​​frame interpolation model 130 then obtains the intermediate generated image IM57 based on the intermediate generated image IM49 and the second original image IM65. And so on, the AI ​​frame interpolation model 130 repeats this process to obtain intermediate generated images IM2 to IM64.

[0055] In step S130, the AI ​​frame interpolation model 130 first segments out the foreground object O1 and generates various angles for the foreground object O1, instead of interpolating the entire image. This invention generates various angles only for the foreground object O1, enabling the foreground object O1 to rotate relative to the background.

[0056] Then, please refer to Figure 8 The example illustrates steps S150 to S160. In step S150, the detection unit 150 detects the viewing angle VA of the user U1 relative to the display unit 110. In this step, the detection unit 150 captures an image FM (shown on the screen of the display unit 110) in front of the display unit 110.Figure 3 The camera tracks a face to obtain the viewing angle VA. Since the image capture range of the screen FM is fixed, the viewing angle VA can be determined based on the position of the face.

[0057] Next, in step S160, the imaging unit 160 displays one of the following on the display unit 110: the first original image IM1, the intermediate generated images IM2 to IM64, and the second original image IM65, based on the viewing angle VA. The imaging unit 160 will only select one image for imaging. Once the user U1 moves and changes the viewing angle VA, the imaging unit 160 will replace it with another image according to the new viewing angle VA. It is as if the user U1 is viewing around the three-dimensional foreground object O1.

[0058] Please refer to Figure 9 This diagram illustrates a block diagram of a display device 200 according to another embodiment of the present invention. The display device 200 includes a display unit 210, an input unit 220, an artificial intelligence frame interpolation model 230, a storage unit 240, and an imaging unit 260. The display unit 210 is used to display images, such as a liquid crystal display panel, an OLED display panel, or an electronic paper display panel. The input unit 220 is used to receive data, such as a transmission port, a wireless transmission module, or a wired network transmission module. The artificial intelligence frame interpolation model 230 is used to generate images, such as a SpatialParallax-AI system, which can be implemented by a circuit, a circuit board, a storage device storing program code, or a chip. The imaging unit 260 is used to present the generated images on the display unit 210, and can be implemented by a circuit, a circuit board, a storage device storing program code, or a chip. The storage unit 240 is, for example, any type of fixed or removable memory or hard disk.

[0059] In this embodiment, artificial intelligence frame interpolation technology is used to generate images corresponding to multiple viewing angles, and the images from various angles are automatically looped and played, making the user U1 feel that the three-dimensional foreground object O1 is being rotated and displayed. A flowchart is provided below to explain in detail the operation of each component.

[0060] Please refer to Figure 10 The diagram illustrates a flowchart of a stereoscopic imaging method according to an embodiment. The stereoscopic imaging method includes steps S220, S230, and S260.

[0061] Please refer to Figure 5The example illustrates step S220. In step S220, input unit 220 obtains a first original image IM1 and a second original image IM65. The first original image IM1 was captured from a first angle A1, and the second original image IM65 was captured from a second angle A65. The first angle A1 is different from the second angle A65. This step is similar to step S120 above, and the similarities will not be repeated.

[0062] Next, please refer to Figures 6-7 The example illustrates step S230. In step S230, the artificial intelligence frame interpolation model 230 generates intermediate generated images IM2 to IM64. This step is similar to step S130 described above, and the similarities will not be repeated.

[0063] Then, please refer to Figure 11 The example illustrates step S260. In step S260, the imaging unit 260 sequentially presents the first original image IM1, the intermediate generated images IM2-IM64, and the second original image IM65 to the display unit 210. For example, the imaging unit 260 sequentially presents the first original image IM1, the intermediate generated images IM2-IM64, and the second original image IM65, and then sequentially presents the second original image IM65, the intermediate generated images IM64-IM2, and the first original image IM1 in reverse order. The foreground object O1 presented by the display unit 210 will rotate and be displayed by swinging left and right.

[0064] The above-described invention provides different features for implementing some embodiments or examples of the invention. Specific examples of components and configurations described above (e.g., mentioned values ​​or names) are provided to simplify / illustrate some embodiments of the invention. Of course, these components and configurations are merely examples and are not intended to be limiting. Furthermore, reference numerals and / or letters may be repeated in various embodiments of the invention. This repetition is for simplicity and clarity and does not in itself indicate a relationship between the various embodiments and / or configurations discussed. Although the invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Any person skilled in the art can make some modifications and improvements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the invention should be defined by the claims.

Claims

1. A stereo imaging method, comprising: A first original image and a second original image are obtained, wherein the first original image is taken from a first angle and the second original image is taken from a second angle, the first angle being different from the second angle; A multi-frame interpolation image is generated using an artificial intelligence frame interpolation model. Detecting a user's viewing angle relative to a display unit; and Based on the viewing angle, one of the first original image, the plurality of intermediate generated images, and the second original image is presented on the display unit.

2. The stereoscopic imaging method as described in claim 1, characterized in that, In the step of obtaining the first original image and the second original image, the first original image and the second original image are a left-eye image and a right-eye image captured by a 3D camera.

3. The stereoscopic imaging method as described in claim 1, characterized in that, In the step of generating the multiple intermediate generated images, the artificial intelligence frame interpolation model generates one output image each time based on two input images.

4. The stereoscopic imaging method as described in claim 3, characterized in that, In the step of generating the multiple intermediate generated images, the artificial intelligence frame interpolation model is repeatedly executed to obtain the multiple intermediate generated images.

5. The stereoscopic imaging method as described in claim 1, characterized in that, In the step of generating the multiple intermediate generated images, the artificial intelligence frame interpolation model generates images for a foreground object.

6. In the stereoscopic imaging method of claim 1, in the step of detecting the viewing angle of the user relative to the display unit, a face is tracked in a front-view image captured by the display unit to obtain the viewing angle.

7. The stereoscopic imaging method as described in claim 1, characterized in that, The step of detecting the user's viewing angle relative to the display unit is performed after the step of generating the plurality of intermediate generated images.

8. An electronic device comprising: One display unit; An input unit is used to acquire a first original image and a second original image, wherein the first original image is captured at a first angle and the second original image is captured at a second angle, the first angle being different from the second angle; An artificial intelligence frame interpolation model is used to generate images between multiple frames; A detection unit is used to detect a user's viewing angle relative to the display unit; as well as An imaging unit is used to display one of the first original image, the plurality of intermediate generated images and the second original image on the display unit according to the viewing angle.

9. The electronic device as claimed in claim 8, characterized in that, The first original image and the second original image are a left-eye image and a right-eye image taken by a 3D camera.

10. The electronic device as claimed in claim 8, characterized in that, This AI-powered frame interpolation model generates one output image each time based on two input images.

11. The electronic device as claimed in claim 10, characterized in that, The AI-powered frame interpolation model is executed repeatedly to obtain the multiple intermediate generated images.

12. The electronic device as claimed in claim 8, characterized in that, This AI-powered frame interpolation model generates frames for a foreground object.

13. The electronic device of claim 8, wherein the detection unit tracks a face in front of a screen of the display unit to obtain the viewing angle.

14. A display device, comprising: One display unit; An input unit is used to acquire a first original image and a second original image, wherein the first original image is captured at a first angle and the second original image is captured at a second angle, the first angle being different from the second angle; An AI-powered frame interpolation model is used to generate inter-frame images; and An imaging unit is used to sequentially display the first original image, the plurality of intermediate generated images, and the second original image on the display unit.

Images