A stereoscopic shooting and display device and method

Through short-pitch dual camera system and image processing technology, stereo display is directly realized on the 2D display device, solving the three-dimensional effect and cost problems of the existing stereo camera device, and improving image quality and user experience.

CN115314698BActive Publication Date: 2024-12-06SHENZHEN FAJIA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210767648.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-01
Publication Date
2024-12-06
Estimated Expiration
2042-07-01

AI Technical Summary

Technical Problem

Due to the large spacing between dual cameras, the existing stereo camera devices have poor three-dimensional effects, serious ghosting, poor image quality, and the naked-eye stereo display method requires precise optical film processing, which increases cost and difficulty, making it difficult to promote.

Method used

Using a short-pitch dual camera system, combined with a control system and an image processing system, the initial image is acquired through spectroscopic, concentrating, shielding or rastering devices, and a solid image is generated through the image processing system, and the stereoscopic display is realized directly on the 2D display device, reducing the Manufacturing difficulty and cost.

Benefits of technology

It solves the three-dimensional effect problem caused by dual camera spacing, reduces the equipment size and manufacturing difficulty, avoids complex optical film processing, improves the comfort and effect of three-dimensional display, reduces costs, and promotes the popularization of technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115314698B_ABST
    Figure CN115314698B_ABST
Patent Text Reader

Abstract

The present invention relates to a stereoscopic shooting and display device and method, including: a short-distance dual-imaging system, a control system, an image processing system and a stereoscopic image display system; the short-distance dual-imaging system is used to collect images and output an initial image; the control system is connected to the short-distance dual-imaging system and the image processing system, and is used to receive the initial image, and processes the initial image in combination with the image processing system to obtain a stereoscopic image; the stereoscopic image display system is connected to the control system, and is used to receive and display the stereoscopic image. The present invention adopts a short-distance dual-imaging system, which can reduce the difficulty of manufacturing alignment correction and reduce the size of the equipment, and the stereoscopic image display system does not need to be complicatedly processed, which can reduce costs. In addition, the present invention is based on the physical perspective principle of the human eye, which makes viewing comfortable, does not cause discomfort such as dizziness, and does not have the problem of perspective difference and ghosting in the stereoscopic image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of stereoscopic display, and more specifically, to a stereoscopic shooting and display device and method. Background Art

[0002] Conventional stereo camera devices can be found in Figure 1 The block diagram of a traditional stereo camera is composed of one or more (inclusive) stereo dual camera modules, a display screen that can display real-time captured stereo images, an image calculation method adapted to the naked-eye 3D, and a digital control circuit. The design method of this traditional stereo camera uses the pupil distance simulated by the human eye, and the distance between the two cameras is generally designed to be between 18mm and 70mm. The dual cameras have a large span and a small viewing angle difference in design, which will lead to shortcomings such as unclear three-dimensional effect and severe ghosting.

[0003] The aforementioned conventional design method will cause the integrated circuit board and the external structure of the camera to occupy a large area because the distance between the two cameras is large, so that there is enough space to place more than two (inclusive) cameras.

[0004] In addition to the requirements for configuration space, due to the requirements of the stereo camera for image interlacing, the relative positions of the two cameras must reach a certain accuracy so that the captured stereo images will have good effects. The effect here refers to the requirements when the left and right stereo images are generally placed side by side. If a certain accuracy is not achieved, for example, the top and bottom are not in a horizontal position, or there is a skew on the left and right, the stereo image will have a large parallax on the left and right pictures, the image will not be clear, and the top and bottom will be blurred due to interlacing, resulting in poor image quality.

[0005] In addition, the current three-dimensional stereoscopic display method utilizes the principle of human eye visual angle difference. Whether it is viewed with glasses or without glasses, the three-dimensional stereoscopic display method based on the parallax principle will cause viewers to feel uncomfortable if they view images of poor quality, which is also one of the reasons for dizziness when viewing three-dimensional images.

[0006] The aforementioned three-dimensional display screens or stereoscopic display devices, especially those without glasses, need to be laminated with a layer of optical film to have the ability of stereoscopic display. Whether it is the material of the columnar grating or the slit grating optical film, it needs to be processed by precise and special methods to obtain a high-quality stereoscopic display screen, which invisibly increases the cost and the barrier to popularization. This is also one of the many reasons why this technology and product have been promoted for many years but are still difficult to popularize.

[0007] Since the aforementioned precision optical film bonding has a relatively high threshold, such as equipment investment, production personnel training, optical film material investment, etc., there is a conventional optical film process. Through simple bonding auxiliary tools, individual users can attach optical films to the surface of the liquid crystal display screen of personal electronic devices to achieve the effect of naked-eye stereoscopic display. However, this process deviates from the physical conditions of near-field optics (Near Field Optics) and cannot truly achieve a good stereoscopic display effect.

[0008] In addition to the stereo display screen, dual cameras, control circuits, etc. mentioned above, a special process flow has been formed on the conventional stereo 3D camera system, and the volume is also larger than that of ordinary single-camera cameras or handheld devices such as mobile phones and tablet computers, which require a lot of space. For current electronic product developers, there are great design difficulties in terms of space layout and cost control. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide a stereoscopic shooting and display device and method in view of the defects of the prior art.

[0010] The technical solution adopted by the present invention to solve the technical problem is: constructing a stereoscopic shooting and display device, including: a short-distance dual-capture imaging system, a control system, an image processing system and a stereoscopic image display system;

[0011] The short-spacing dual-imaging system is used to collect images and output initial images;

[0012] The control system is connected to the short-distance dual-imaging system and the image processing system, and is used to receive the initial image, and process the initial image in conjunction with the image processing system to obtain a stereoscopic image;

[0013] The stereoscopic image display system is connected to the control system and is used for receiving the stereoscopic image and displaying the stereoscopic image.

[0014] In the stereoscopic shooting and display device of the present invention, the short-distance dual-imaging system includes: any one of a first composition mode, a second composition mode, a third composition mode, a fourth composition mode and a fifth composition mode;

[0015] The first composition mode, the second composition mode, the third composition mode, the fourth composition mode and the fifth composition mode all adopt dual lenses, and the distance between the two lenses is less than a preset value.

[0016] In the stereoscopic shooting and display device of the present invention, the first component mode includes: a first lens, a second lens, a light splitting device, a light focusing device and an image sensor;

[0017] The first lens and the second lens are respectively arranged in the light collecting direction of the light splitting device, and are used to obtain light and transmit it to the light splitting device;

[0018] The light splitting device is arranged on the sensing surface of the image sensor, and is used to transmit the received light to the sensing surface of the image sensor after light splitting processing;

[0019] The light focusing device is arranged between the light splitting device and the image sensor, and is used to focus the light output by the light splitting device on the sensing surface of the image sensor;

[0020] The image sensor is used to convert light to obtain the initial image.

[0021] In the stereoscopic shooting and display device of the present invention, the second composition mode includes: a first lens, a second lens, a shielding device, a focusing device and an image sensor;

[0022] The first lens and the second lens are arranged in the light incident direction of the shielding device to obtain light; the first lens and the second lens have image circles of different focal lengths and different sizes;

[0023] The light focusing device is arranged between the shielding device and the image sensor, and is used to focus the light passing through the shielding device onto the sensing surface of the image sensor;

[0024] The image sensor is used to convert light to obtain the initial image.

[0025] In the stereoscopic shooting and display device of the present invention, the third composition mode includes: a first lens, a second lens, a shielding device, a focusing device and an image sensor;

[0026] The first lens and the second lens are arranged in the light incident direction of the shielding device to obtain light; the first lens and the second lens have the same focal length and the same size of image circle;

[0027] The light focusing device is arranged between the shielding device and the image sensor, and is used to focus the light passing through the shielding device onto the sensing surface of the image sensor;

[0028] The image sensor is used to convert light to obtain the initial image.

[0029] In the stereoscopic shooting and display device of the present invention, the fourth component mode includes: a first lens, a second lens, a shielding device, a grating device, a focusing device and an image sensor;

[0030] The first lens and the second lens are arranged in the light incident direction of the shielding device to obtain light;

[0031] The grating device is built into the shielding device and is used to filter the light output by the first lens and the second lens;

[0032] The light focusing device is arranged between the shielding device and the image sensor, and is used to focus the light passing through the shielding device onto the sensing surface of the image sensor;

[0033] The image sensor is used to convert light to obtain the initial image.

[0034] In the stereoscopic shooting and display device of the present invention, the fifth component mode includes: a first module and a second module;

[0035] The first module includes: a first lens, a first light focusing device and a first image sensor; the second module includes: a second lens, a second light focusing device and a second image sensor;

[0036] The first lens is configured to capture light and transmit it to the first light focusing device;

[0037] The first light focusing device is disposed between the first lens and the first image sensor, and is used to focus the light passing through the first lens onto the sensing surface of the first image sensor;

[0038] The first image sensor is used to convert light to obtain a first image;

[0039] The second lens is configured to capture light and transmit it to the second light focusing device;

[0040] The second light focusing device is disposed between the second lens and the second image sensor, and is used to focus the light passing through the second lens onto the sensing surface of the second image sensor;

[0041] The second image sensor is used to convert the light to obtain a second image;

[0042] The first image and the second image form the initial image.

[0043] In the stereoscopic shooting and display device of the present invention, the control system includes: an image processing unit, a digital signal processing unit, an artificial intelligence computing unit and a central computing control unit;

[0044] The image processing unit, the digital signal processing unit, and the artificial intelligence computing unit are connected to the central computing control unit, and the image processing unit and the digital signal processing unit are used to convert the initial image output by the short-spacing dual-imaging system to obtain a standard image;

[0045] The central operation control unit is used to control the image processing unit, the digital signal processing unit and the artificial intelligence operation unit to perform three-party work, and processes the standard image in combination with the image processing system to obtain a stereoscopic image.

[0046] In the stereoscopic shooting and display device of the present invention, the image processing system includes: a marking unit, a segmentation unit, a frame deformation calculation unit and a fusion unit;

[0047] The marking unit is used to mark the position of the image center of the standard image;

[0048] The segmentation unit is used to perform image segmentation on the standard image according to the position mark to obtain a foreground image, a background image and a background image;

[0049] The frame deformation calculation unit is used to perform frame deformation calculation on the background image to obtain the background to be fused;

[0050] The fusion unit is used to fuse the foreground image, the background image and the background to be fused to obtain a stereoscopic image.

[0051] The present invention also provides a stereoscopic shooting and display method, comprising the following steps:

[0052] Capture images through a short-distance dual-imaging system and output the initial image;

[0053] The control system receives the initial image and processes the initial image in conjunction with the image processing system to obtain a stereoscopic image;

[0054] The stereoscopic image display system displays the stereoscopic image.

[0055] The implementation of the stereoscopic shooting and display device and method of the present invention has the following beneficial effects: the stereoscopic shooting and display device includes: a short-distance dual-imaging system, a control system, an image processing system, and a stereoscopic image display system; the short-distance dual-imaging system is used to collect images and output the initial image; the control system is connected to the short-distance dual-imaging system and the image processing system, and is used to receive the initial image, and processes the initial image in combination with the image processing system to obtain a stereoscopic image; the stereoscopic image display system is connected to the control system, and is used to receive the stereoscopic image and display the stereoscopic image. The present invention adopts a short-distance dual-imaging system, which can reduce the difficulty of manufacturing alignment correction and reduce the size of the equipment, and the stereoscopic image display system does not need to be complicatedly processed, which can reduce costs. In addition, the present invention is based on the physical perspective principle of the human eye, which makes viewing comfortable, does not cause discomfort such as dizziness, and does not have the problem of perspective difference and ghosting in the stereoscopic image. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0057] Figure 1 This is the block diagram of a traditional stereo camera;

[0058] Figure 2 It is a principle block diagram of the stereoscopic shooting and display device provided by the present invention;

[0059] Figure 3 It is a structural schematic diagram of a first embodiment of a short-distance dual-imaging system provided by the present invention;

[0060] Figure 4 It is a structural schematic diagram of a second embodiment of a short-distance dual-imaging system provided by the present invention;

[0061] Figure 5 It is a structural schematic diagram of a third embodiment of a short-distance dual-imaging system provided by the present invention;

[0062] Figure 6 1 is a schematic structural diagram of a fourth embodiment of a short-distance dual-imaging system provided by the present invention;

[0063] Figure 7 1 is a schematic structural diagram of a fifth embodiment of a short-distance dual-imaging system provided by the present invention;

[0064] Figure 8 It is a principle block diagram of the control system provided by the present invention;

[0065] Fig. 9 It is a principle block diagram of the image processing system provided by the present invention;

[0066] Fig.10Schematic diagram of the left and right image ranging principle provided by an embodiment of the present invention;

[0067] Fig.11 is a schematic diagram of folding provided by an embodiment of the present invention;

[0068] Fig.12 It is a flow chart of the stereoscopic shooting and display method provided by the present invention. DETAILED DESCRIPTION

[0069] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0070] refer to Figure 2 , which is a principle block diagram of an optional embodiment of the stereoscopic shooting and display device provided by the present invention.

[0071] like Figure 2 As shown, the stereoscopic shooting and display device includes: a short-distance dual-imaging system 10, a control system 20, an image processing system 30, and a stereoscopic image display system 40. Among them, the short-distance dual-imaging system 10 is used to capture images and output the initial image; the control system 20 is connected to the short-distance dual-imaging system 10 and the image processing system 30, and is used to receive the initial image, and process the initial image in combination with the image processing system 30 to obtain a stereoscopic image; the stereoscopic image display system 40 is connected to the control system 20, and is used to receive the stereoscopic image and display the stereoscopic image. It should be noted that in the embodiment of the present invention, the stereoscopic image display system 40 can adopt a flat (2D) display device. That is, the present invention does not need to adopt a dedicated stereo camera device (such as a three-dimensional stereo device device with stereoscopic camera and display functions with the same functions as an independent camera) to present a stereoscopic image on a 2D display device.

[0072] The present invention can solve the problem of high parallax angle requirements and accurate calculations required to obtain a good stereoscopic image effect by using a short-distance dual-capture imaging system 10, and can also reduce the difficulty of manufacturing alignment correction and the size of the camera module. For stereoscopic display, the present invention can directly use a 2D display device to achieve stereoscopic display without complicated processing, greatly reducing the difficulty and cost of the manufacturing process.

[0073] In addition, the present invention makes viewing comfortable based on the perspective principle of the physical properties of the human eye, and does not cause discomfort such as dizziness, nor does it have the viewing angle difference and ghosting problems that often occur in stereoscopic images. It can obtain a better stereoscopic display effect, and at the same time, it will not reduce or affect the original plane (2D) display quality.

[0074] Furthermore, since the present invention adopts a 2D display device, it is easy to produce and has a high yield rate, thereby greatly reducing the cost of the entire system and equipment. For the entire stereoscopic display and camera industry, it can be easily promoted and enhance the user's experience of stereoscopic images.

[0075] Optionally, in an embodiment of the present invention, the short-spacing dual imaging system 10 includes: any one of a first composition mode, a second composition mode, a third composition mode, a fourth composition mode, and a fifth composition mode. That is, the short-spacing dual imaging system 10 of the present invention can be implemented by using any one of the first composition mode, the second composition mode, the third composition mode, the fourth composition mode, and the fifth composition mode. Among them, the first composition mode, the second composition mode, the third composition mode, the fourth composition mode, and the fifth composition mode all use dual lenses, and the spacing between the two lenses is less than a preset value. Optionally, in an embodiment of the present invention, the preset value is less than or equal to 15 mm.

[0076] Specifically, refer to Figure 3 , is a schematic structural diagram of a first embodiment of a short-spacing dual-imaging system 10 provided by the present invention.

[0077] In this embodiment, the short-distance dual-imaging system 10 adopts the first composition mode. Figure 3 As shown, the first component mode includes: a first lens 11, a second lens 12, a light splitting device 13, a light focusing device 14 and an image sensor 15.

[0078] The first lens 11 and the second lens 12 are respectively arranged in the light-collecting direction of the spectroscopic device 13, and are used to obtain light and transmit it to the spectroscopic device 13. The spectroscopic device 13 is arranged on the sensing surface of the image sensor 15, and is used to transmit the received light to the sensing surface of the image sensor 15 after spectroscopic processing. The focusing device 14 is arranged between the spectroscopic device 13 and the image sensor 15, and is used to aggregate the light output by the spectroscopic device 13 on the sensing surface of the image sensor 15. The image sensor 15 is used to convert the light to obtain an initial image.

[0079] like Figure 3 As shown, two paths of light enter the light splitting device 13 after passing through the first lens 11 and the second lens 12 respectively. After being split by the light splitting device 13, they are integrated on the sensing surface of the image sensor 15 by the focusing device 14 and integrated into an image.

[0080] Optionally, in the embodiment of the present invention, the light splitting device 13 may be a prism, and the light focusing device 14 may be a microlens. Further, in the embodiment of the present invention, the design of the first lens 11 and the second lens 12 and the matching of the image sensor 15 may be in a common manner, and the present invention does not specifically limit them.

[0081] refer to Figure 4 , is a schematic structural diagram of a second embodiment of a short-spacing dual-imaging system 10 provided by the present invention.

[0082] In this embodiment, the short-distance dual-imaging system 10 adopts the second composition mode. Figure 4 As shown, the second composition mode includes: a first lens 11, a second lens 12, a shielding device 16, a focusing device 14 and an image sensor 15.

[0083] The first lens 11 and the second lens 12 are arranged in the light incident direction of the shielding device 16 to obtain light. The focusing device 14 is arranged between the shielding device 16 and the image sensor 15 to focus the light passing through the shielding device 16 on the sensing surface of the image sensor 15. The image sensor 15 is used to convert the light to obtain an initial image.

[0084] like Figure 4 As shown, after the two light rays pass through the first lens 11 and the second lens 12 respectively and enter the shielding device 16, they are integrated on the sensing surface of the image sensor 15 by the focusing device 14 and integrated into an image. In this embodiment, the first lens 11 and the second lens 12 have different focal lengths and different sizes of image circles.

[0085] Optionally, in the embodiment of the present invention, the shielding device 16 can adopt a lens shield commonly used in lens design. By setting the lens shield, stray light can be effectively prevented from entering, thereby improving the imaging quality. Optionally, in the embodiment of the present invention, the focusing device 14 can adopt a microlens. Further, in the embodiment of the present invention, the design method of the first lens 11 and the second lens 12 and the matching of the image sensor 15 can adopt a common method, and the present invention does not specifically limit it.

[0086] refer to Figure 5 , is a schematic structural diagram of a third embodiment of a short-spacing dual-imaging system 10 provided by the present invention.

[0087] In this embodiment, the short-distance dual-imaging system 10 adopts the third composition mode. Figure 5 As shown, the third composition mode includes: a first lens 11, a second lens 12, a shielding device 16, a focusing device 14 and an image sensor 15.

[0088] The first lens 11 and the second lens 12 are arranged in the light-entering direction of the shielding device 16 to obtain light. The focusing device 14 is arranged between the shielding device 16 and the image sensor 15 to focus the light passing through the shielding device 16 on the sensing surface of the image sensor 15. The image sensor 15 is used to convert the light to obtain an initial image.

[0089] like Figure 5 As shown, after the two light rays pass through the first lens 11 and the second lens 12 respectively and enter the shielding device 16, they are integrated on the sensing surface of the image sensor 15 by the focusing device 14 and integrated into an image. In this embodiment, the first lens 11 and the second lens 12 have the same focal length and the same size of image circle.

[0090] Optionally, in the embodiment of the present invention, the shielding device 16 can adopt a lens shield commonly used in lens design. By setting the lens shield, stray light can be effectively prevented from entering, thereby improving the imaging quality. Optionally, in the embodiment of the present invention, the focusing device 14 can adopt a microlens. Further, in the embodiment of the present invention, the design method of the first lens 11 and the second lens 12 and the matching of the image sensor 15 can adopt a common method, and the present invention does not specifically limit it.

[0091] refer to Figure 6 , which is a schematic structural diagram of a fourth embodiment of the short-spacing dual-imaging system 10 provided by the present invention.

[0092] In this embodiment, the short-distance dual-imaging system 10 adopts the fourth composition mode. Figure 6 As shown, the fourth component mode includes: a first lens 11, a second lens 12, a shielding device 16, a grating device 17, a focusing device 14 and an image sensor 15.

[0093] The first lens 11 and the second lens 12 are arranged in the light incident direction of the shielding device 16 to obtain light. The grating device 17 is built into the shielding device 16 to filter the light output by the first lens 11 and the second lens 12. The focusing device 14 is arranged between the shielding device 16 and the image sensor 15 to focus the light passing through the shielding device 16 on the sensing surface of the image sensor 15. The image sensor 15 is used to convert the light to obtain an initial image.

[0094] like Figure 6 As shown, after the two paths of light pass through the first lens 11 and the second lens 12 respectively and enter the shielding device 16, they first pass through the built-in grating device 17, and then are integrated on the sensing surface of the image sensor 15 by the focusing device 14, and an image is integrated. In this embodiment, by setting the grating device 17, the stray light entering the lens can be effectively filtered, and the utilization rate of light can be effectively improved. Optionally, in this embodiment, the first lens 11 and the second lens 12 can use the same focal length and the same size of the image circle, or two image circles with different focal lengths can be used.

[0095] refer to Figure 7 , which is a structural diagram of a fifth embodiment of a short-spacing dual-imaging system 10 provided by the present invention.

[0096] In this embodiment, the short-distance dual-imaging system 10 adopts the fifth composition mode. Figure 7 As shown, the fifth component mode includes: a first module 71 and a second module 72 .

[0097] The first module 71 includes: a first lens 11 , a first light focusing device 711 and a first image sensor 712 . The second module 72 includes: a second lens 12 , a second light focusing device 721 and a second image sensor 722 .

[0098] The first lens 11 is configured to acquire light and transmit it to the first light-condensing device 711; the first light-condensing device 711 is configured between the first lens 11 and the first image sensor 712, and is configured to aggregate the light passing through the first lens 11 onto the sensing surface of the first image sensor 712; the first image sensor 712 is configured to convert the light to obtain a first image. The second lens 12 is configured to acquire light and transmit it to the second light-condensing device 721; the second light-condensing device 721 is configured between the second lens 12 and the second image sensor 722, and is configured to aggregate the light passing through the second lens 12 onto the sensing surface of the second image sensor 722; the second image sensor 722 is configured to convert the light to obtain a second image. The first image and the second image form an initial image.

[0099] like Figure 7 As shown, the fifth composition mode is composed of two independent lens modules, wherein the two independent lens modules are voice coil motors with autofocus. The two independent lens modules can be bonded in any manner. Among them, the first module 71 and the second module 72 can adopt the same design according to the system design requirements, or they can also adopt different designs, or they can also adopt a design in which one is configured with a color camera and the other is configured with a black and white grayscale camera.

[0100] like Figures 3 to 7 In the embodiment shown, the micro lens used in the present invention is a common design of the lens module, and the light can be maximized by setting the micro lens. Figures 3 to 7 In any of the embodiments, the distance between the first lens 11 and the second lens 12 can be shortened to a preset value or below.

[0101] In the embodiment of the present invention, by adopting a short-pitch imaging method, in addition to significantly reducing the size and volume of the product, the high-precision same-position calibration required in the dual-camera production process can also be avoided, greatly improving the product production yield. In addition, for the existing conventional technologies and methods commonly used to calculate depth, such as Time of Flight technology and 3D Structure Light technology, whether the former uses time difference or the latter is based on trigonometric functions, lasers are used as references for calibration. Therefore, there are environmental restrictions on imaging, module parts costs, triangulation and laser calculation time, and high requirements for electronic hardware systems, etc. All of these cannot be compared with the advantages of the present invention, which are relatively low cost, short calculation time, and relatively low electronic hardware conditions.

[0102] refer to Figure 8 , is a principle block diagram of the control system 20 provided by the present invention. Figure 8 As shown, the control system 20 includes: an image processing unit 22 , a digital signal processing unit 23 , an artificial intelligence computing unit 24 and a central computing control unit 21 .

[0103] The image processing unit 22, the digital signal processing unit 23, and the artificial intelligence operation unit 24 are connected to the central operation control unit 21. The image processing unit 22 and the digital signal processing unit 23 are used to convert and process the initial image output by the short-distance dual-imaging system 10 to obtain a standard image; the central operation control unit 21 is used to control the image processing unit 22, the digital signal processing unit 23 and the artificial intelligence operation unit 24 to perform three-party work, and process the standard image in combination with the image processing system 30 to obtain a stereoscopic image.

[0104] Furthermore, if Figure 8 As shown, the control system 20 also includes: a power control unit 25, a UI interface unit 26 (wherein UI: User Interface), an internal memory unit 27, an external memory unit 28, a temporary register unit 29, a wireless connection unit 211 and a display drive control unit 210.

[0105] Among them, the power control unit 25 is used to realize power control of the control system 20, the UI interface unit 26 is used to realize interaction with the user, the internal memory unit 27, the external memory unit 28 and the register unit 29 are used to realize data storage, the wireless connection unit 211 is used to realize wireless communication between the control system 20 and external devices, and the display drive control unit 210 is used to realize display drive control between the stereoscopic image display system 40.

[0106] like Figure 8As shown, when capturing images, when the user issues an image capturing instruction to the central operation control unit 21 through the UI interface unit 26, the short-distance dual-imaging system 10 immediately converts the optical signal into an electrical signal to obtain an initial image, and connects to the image processing unit 22 through the control channel of the short-distance dual-imaging system 10, converts the initial image into an international standard image format (such as YUV422, etc.) through the image processing unit 22 and the digital signal processing unit 23, and temporarily stores the converted standard image in a memory unit (any one or more of the internal memory unit 27, the external memory unit 28, and the temporary storage unit 29). The image processing unit 22 is also used to provide the short-distance dual-imaging system 10 with imaging parameter values ​​such as shutter speed, white balance, exposure time, and basic color standards.

[0107] In the process of image processing, in addition to software calculation, hardware adaptation and electrical signal realization of circuit functions are also required. Among them, the processing of electrical signals is controlled and coordinated by the central operation control unit 21. After completing the image acquisition, the central operation control unit 21 outputs instructions to control the image processing to start execution, and performs three-party operations with the digital signal processing unit 23 and the image processing unit 22, and combines with the image processing system 30 to perform image processing. Moreover, after completing the image processing, the obtained stereoscopic image can be stored in the internal memory unit 27 and / or the external memory unit 28, or directly displayed by the stereoscopic image display system 40, or the obtained stereoscopic image can be transmitted to a third-party system or device in real time through a wireless network through the wireless connection unit 211.

[0108] refer to Fig. 9 , is a principle block diagram of the image processing system 30 provided by the present invention. Fig. 9 As shown, the image processing system 30 includes: a marking unit 31 , a segmentation unit 32 , a frame deformation calculation unit 33 and a fusion unit 34 .

[0109] In some embodiments, the marking unit 31 is used to mark the position of the image center of the standard image. The segmentation unit 32 is used to segment the standard image according to the position mark to obtain a foreground image, a background image and a background image; the frame deformation calculation unit 33 is used to calculate the frame deformation of the background image to obtain the background to be fused; the fusion unit 34 is used to fuse the foreground image, the background image and the background to be fused to obtain a stereoscopic image.

[0110] Specifically, the image source with depth information is first calculated to obtain the relationship between the depth and distance in the image; the position of the image center is marked according to the relationship between the depth and distance in the image; specifically, after the image source with depth information (for example, the left and right image arrangement (SBS)) is obtained by the short-spacing dual-capture imaging system 10, the artificial intelligence computing unit 24 can be used to calculate and calculate the distance according to the left and right image distance measurement principle (such as Fig.10 As shown in the figure), the depth and distance relationship in the image is calculated, and the source image is marked according to the depth and distance relationship in the image. The specific calculation formula is as follows:

[0111] (1).

[0112] Therefore, through formula (1), we can get:

[0113] (2).

[0114] Where Z represents the depth, T is the distance between the left and right lenses, x l is the center point of the left lens ( Figure 2 The coordinates are expressed as ( c xl , c yl ), x r is the center point of the right lens ( Figure 2 The coordinates are expressed as ( c xr , c yr ), f represents the focal length of the lens. x l and x r All are known, so Z can be calculated based on formula (2).

[0115] Among them, artificial intelligence operations can be calculated using the artificial intelligence deep learning CNN (Convolutional Neural Networks) algorithm library, and the accuracy of marking can be improved by calculating multiple images multiple times.

[0116] Furthermore, after completing the position mark of the center of the image, the segmentation unit 32 segments the image area where the mark is located, so that a foreground image, a background image and a background image can be obtained. Optionally, the division of the foreground image, the background image and the background image can be divided according to the calculated depth value and the relative distance from the lens. Among them, the depth value with a small relative distance (close distance) from the lens is the foreground image (with the position mark as a reference), the depth value with a medium relative distance from the lens is the background image, and the depth value with the largest relative distance from the lens is the background image. Among them, the large, medium and small relative distances between the depth value and the lens are determined to distinguish the front, back and background images, and the present invention does not make specific limitations.

[0117] Optionally, in an embodiment of the present invention, after obtaining the foreground image and the background image, the foreground image and the background image are cached. That is, after completing the image segmentation and segmenting the foreground image and the background image, the foreground image and the background image do not need to be processed, but the foreground image and the background image are stored in a cache for subsequent image fusion.

[0118] In the embodiment of the present invention, after the background image is segmented, it is necessary to perform frame deformation calculation on the background image to obtain a background image that can be fused (ie, a background to be fused).

[0119] Optionally, in an embodiment of the present invention, the background image includes: an image that can be adapted to the folding edge, an image that cannot be adapted to the folding edge but can be optimized for the border, and an image that cannot be adapted to the folding edge and cannot be optimized for the border.

[0120] Among them, for images that are not suitable for folding and cannot be optimized for borders, the present invention does not perform any processing on the background image, but directly uses the image that is not suitable for folding and cannot be optimized for borders as the background to be fused. For images that are not suitable for folding but can be optimized for borders, the present invention performs HSV conversion on the image that is not suitable for folding but can be optimized for borders to obtain an HSV image. Specifically, HSV (Hue Saturation Value), H represents color hue, S represents saturation, and V represents brightness. Among them, the HSV image is the background to be fused. For images that can adapt to folding, the present invention needs to perform folding first and then perform border optimization processing.

[0121] In some embodiments, the present invention may process the adaptable folded edge image by the following method: that is, the border of the adaptable folded edge image is calculated to obtain the size to be cropped; the adaptable folded edge image is cropped according to the size to be cropped to obtain the cropped border; the cropped border is perspective transformed to obtain the transformed image; the transformed image is HSV converted to obtain the HSV image; the HSV image is the fusion background.

[0122] Among them, for images with adaptable folding edges, in an embodiment of the present invention, the border size is first calculated, and then the border of the image is cropped according to the calculated border width and height, and then the coordinates of the border of the cropped image are perspective transformed, and finally the RGB of the border after perspective transformation is converted to HSV to obtain an HSV image.

[0123] Specifically, Fig.11 As shown, the initial value is 1 / 8 of the smaller side of the image as the width w of the border, the initial value of the angle α is 45° by default, and the height of the border is calculated according to the trigonometric formula h=w * tan(Radians(α)) + 0.5. The border of the image is cut out according to the calculated border width and height (such as HGFE in the schematic diagram), and the perspective transformation is performed using the general formula for perspective transformation (as shown below):

[0124] (3).

[0125] The border HGFE coordinates are transformed into the border ABFE coordinates, and the same method is used for the other three borders. After completing the perspective change, the RGB of the border image is converted to HSV to obtain the HSV image. Furthermore, after completing the HSV conversion, V can also be processed with histogram equalization to optimize the brightness of the image.

[0126] In some embodiments, after completing the histogram equalization process on V, the background to be fused, the rear image, and the foreground image are sequentially pasted back to the source image to obtain a stereoscopic image. The source image here is the image before segmentation.

[0127] Furthermore, in some embodiments, when displaying a stereoscopic image, the central operation control unit 21 is electrically connected to the stereoscopic image display system 40 through the display drive control unit 210, and displays system startup information, such as a startup screen, system information, etc. When entering the camera preview mode or the photo mode, the initial image acquired by the short-distance dual-capture imaging system 10 is displayed on the stereoscopic image display system 40 after image processing. Among them, 2D plane image display can be realized by setting the interface function, and 3D stereoscopic image display can also be realized.

[0128] Furthermore, in the integrated control system 20 with the artificial intelligence computing unit 24, the accuracy of image processing can be improved through the establishment and training of artificial intelligence models. Among them, any integrated circuit functional module with such an integrated circuit is also within the scope of the invention.

[0129] In some embodiments, the stereoscopic image display system 40 may include a display, wherein the type of display may include but is not limited to a color liquid crystal display, an OLED (Organic Light-Emitting Diode) display, or even a new type of Micro LED, Mini LED display, laser display, and any display device that can display in color or grayscale can be applied to the creation of the present invention.

[0130] The stereoscopic shooting and display system of the present invention breaks through the technical bottleneck of the long distance between the dual cameras of the traditional stereoscopic shooting camera and the image acquisition and image processing method matched with this dual camera design. The image processing results according to the present invention are displayed on a general display screen to achieve the function of naked eye stereoscopic display.

[0131] refer to Fig.12 , is a flow chart of an optional embodiment of a stereoscopic shooting and display method provided by the present invention. The stereoscopic shooting and display method can be implemented by the stereoscopic shooting and display disclosed by the present invention.

[0132] Specifically, Fig.12 As shown, the stereoscopic shooting and display method comprises the following steps:

[0133] Step S101 : Capture images through the short-distance dual-imaging system 10 and output an initial image.

[0134] Step S102: the control system 20 receives an initial image, and processes the initial image in conjunction with the image processing system 30 to obtain a stereoscopic image.

[0135] Step S103 : The stereoscopic image display system 40 displays the stereoscopic image.

[0136] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0137] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0138] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM (Compact Disc Read-Only Memory), or any other form of storage medium known in the art.

[0139] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot limit the scope of protection of the present invention. All equivalent changes and modifications made to the scope of the claims of the present invention should fall within the scope of the claims of the present invention.

Claims

1. A stereoscopic shooting and display device, characterized in that: include: Short-distance dual-capture imaging system, control system, image processing system and stereoscopic image display system; The short-spacing dual-imaging system is used to collect images and output initial images; The distance between the two lenses in the short-distance dual-image capturing system is less than a preset value; The short-distance dual-imaging system includes: any one of the second composition mode, the third composition mode, and the fourth composition mode; the second composition mode, the third composition mode, and the fourth composition mode all use dual lenses, and the distance between the two lenses is less than a preset value; the preset value is less than or equal to 15 mm; The second composition mode includes: a first lens, a second lens, a shielding device, a focusing device and an image sensor; the first lens and the second lens are arranged in the light incident direction of the shielding device, and are used to obtain light; the first lens and the second lens have different focal lengths and image circles of different sizes; the focusing device is arranged between the shielding device and the image sensor, and is used to focus the light passing through the shielding device on the sensing surface of the image sensor; the image sensor is used to convert the light to obtain the initial image; The third composition mode includes: a first lens, a second lens, a shielding device, a focusing device and an image sensor; the first lens and the second lens are arranged in the light incident direction of the shielding device, and are used to obtain light; the first lens and the second lens have the same focal length and the same size of image circle; the focusing device is arranged between the shielding device and the image sensor, and is used to focus the light passing through the shielding device on the sensing surface of the image sensor; the image sensor is used to convert the light to obtain the initial image; The fourth composition mode includes: a first lens, a second lens, a shielding device, a grating device, a focusing device and an image sensor; the first lens and the second lens are arranged in the light incident direction of the shielding device, and are used to obtain light; the grating device is built into the shielding device, and is used to filter the light output by the first lens and the second lens; the focusing device is arranged between the shielding device and the image sensor, and is used to focus the light passing through the shielding device on the sensing surface of the image sensor; the image sensor is used to convert the light to obtain the initial image; The control system is connected with the short-distance dual-imaging system and the image processing system, and is used to receive the initial image, and process the initial image in combination with the image processing system to obtain a stereoscopic image; the control system includes: an image processing unit, a digital signal processing unit, an artificial intelligence operation unit and a central operation control unit; the image processing unit, the digital signal processing unit and the artificial intelligence operation unit are connected with the central operation control unit, and the image processing unit and the digital signal processing unit are used to convert and process the initial image output by the short-distance dual-imaging system to obtain a standard image; the central operation control unit is used to control the image processing unit, the digital signal processing unit and the artificial intelligence operation unit to perform a three-party work, and process the standard image in combination with the image processing system to obtain a stereoscopic image; The image processing system comprises: a marking unit, a segmentation unit, a frame deformation calculation unit and a fusion unit; the marking unit is used to mark the position of the image center of the standard image; the segmentation unit is used to segment the standard image according to the position mark to obtain a foreground image, a background image and a background image; the frame deformation calculation unit is used to calculate the frame deformation of the background image to obtain a background to be fused; the fusion unit is used to fuse the foreground image, the background image and the background to be fused to obtain a stereo image; the background image comprises: an image that can be adapted to the folding edge, an image that cannot be adapted to the folding edge but can be optimized for the frame, and an image that cannot be adapted to the folding edge and cannot be optimized for the frame; the width and height of the frame of the image that can be adapted to the folding edge satisfy: h=w * tan(Radians(α)) + 0.5; w is the width of the frame, h is the height of the frame, and α=45°; Wherein, if the background image is an adaptable folded edge image, the frame deformation calculation of the background image to obtain the background to be fused includes: performing frame calculation on the adaptable folded edge image to obtain the size to be cropped; cropping the adaptable folded edge image according to the size to be cropped to obtain a cropped frame; performing perspective transformation on the cropped frame to obtain a transformed image; performing HSV conversion on the transformed image to obtain an HSV image; the HSV image is the background to be fused; If the background image is an image that cannot be adapted to the folded edge but can be optimized for the border, then performing border deformation calculation on the background image to obtain the background to be fused includes: performing HSV conversion on the background image to obtain an HSV image; the HSV image is the background to be fused; If the background image is an image that is not suitable for folding and cannot be optimized for the border, performing border deformation calculation on the background image to obtain the background to be fused includes: using the background image as the background to be fused; The stereoscopic image display system is connected to the control system and is used to receive and display the stereoscopic image; the stereoscopic image display system adopts a flat display device.

2. A stereoscopic shooting and display method, characterized in that: The following steps are involved: Capture images through a short-distance dual-imaging system and output the initial image; The distance between the two lenses in the short-distance dual-image capturing system is less than a preset value; The short-distance dual-imaging system includes: any one of the second composition mode, the third composition mode, and the fourth composition mode; the second composition mode, the third composition mode, and the fourth composition mode all use dual lenses, and the distance between the two lenses is less than a preset value; the preset value is less than or equal to 15 mm; The second composition mode includes: a first lens, a second lens, a shielding device, a focusing device and an image sensor; the first lens and the second lens are arranged in the light incident direction of the shielding device, and are used to obtain light; the first lens and the second lens have different focal lengths and image circles of different sizes; the focusing device is arranged between the shielding device and the image sensor, and is used to focus the light passing through the shielding device on the sensing surface of the image sensor; the image sensor is used to convert the light to obtain the initial image; The third composition mode includes: a first lens, a second lens, a shielding device, a focusing device and an image sensor; the first lens and the second lens are arranged in the light incident direction of the shielding device, and are used to obtain light; the first lens and the second lens have the same focal length and the same size of image circle; the focusing device is arranged between the shielding device and the image sensor, and is used to focus the light passing through the shielding device on the sensing surface of the image sensor; the image sensor is used to convert the light to obtain the initial image; The fourth composition mode includes: a first lens, a second lens, a shielding device, a grating device, a focusing device and an image sensor; the first lens and the second lens are arranged in the light incident direction of the shielding device, and are used to obtain light; the grating device is built into the shielding device, and is used to filter the light output by the first lens and the second lens; the focusing device is arranged between the shielding device and the image sensor, and is used to focus the light passing through the shielding device on the sensing surface of the image sensor; the image sensor is used to convert the light to obtain the initial image; The control system receives the initial image, and processes the initial image in combination with the image processing system to obtain a stereoscopic image; the control system includes: an image processing unit, a digital signal processing unit, an artificial intelligence operation unit and a central operation control unit; the image processing unit, the digital signal processing unit and the artificial intelligence operation unit are connected to the central operation control unit, and the image processing unit and the digital signal processing unit are used to convert and process the initial image output by the short-distance dual-imaging system to obtain a standard image; the central operation control unit is used to control the image processing unit, the digital signal processing unit and the artificial intelligence operation unit to perform a three-party work, and process the standard image in combination with the image processing system to obtain a stereoscopic image; The image processing system comprises: a marking unit, a segmentation unit, a frame deformation calculation unit and a fusion unit; the marking unit is used to mark the position of the image center of the standard image; the segmentation unit is used to segment the standard image according to the position mark to obtain a foreground image, a background image and a background image; the frame deformation calculation unit is used to calculate the frame deformation of the background image to obtain a background to be fused; the fusion unit is used to fuse the foreground image, the background image and the background to be fused to obtain a stereo image; the background image comprises: an image that can be adapted to the folding edge, an image that cannot be adapted to the folding edge but can be optimized for the frame, and an image that cannot be adapted to the folding edge and cannot be optimized for the frame; the width and height of the frame of the image that can be adapted to the folding edge satisfy: h=w * tan(Radians(α)) + 0.5; w is the width of the frame, h is the height of the frame, and α=45°; Wherein, if the background image is an adaptable folded edge image, the frame deformation calculation of the background image to obtain the background to be fused includes: performing frame calculation on the adaptable folded edge image to obtain the size to be cropped; cropping the adaptable folded edge image according to the size to be cropped to obtain a cropped frame; performing perspective transformation on the cropped frame to obtain a transformed image; performing HSV conversion on the transformed image to obtain an HSV image; the HSV image is the background to be fused; If the background image is an image that cannot be adapted to the folded edge but can be optimized for the border, then performing border deformation calculation on the background image to obtain the background to be fused includes: performing HSV conversion on the background image to obtain an HSV image; the HSV image is the background to be fused; If the background image is an image that is not suitable for folding and cannot be optimized for the border, performing border deformation calculation on the background image to obtain the background to be fused includes: using the background image as the background to be fused; The stereoscopic image display system displays the stereoscopic image; the stereoscopic image display system adopts a flat display device.