Three-dimensional display system, three-dimensional display method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a three-dimensional display system, a three-dimensional display method, an electronic device and a storage medium, and relate to the technical field of display imaging. The three-dimensional display system divides the light signal of a shooting target into two paths through a shooting lens to form a first parallax image and a second parallax image of the shooting target, processes the second parallax image through a processing unit to obtain a third parallax image matched with the first parallax image, fuses the first parallax image and the third parallax image to obtain a 3D frame picture image, and finally projects the 3D frame picture image to the visual field range of a person object through an optical conversion device arranged on the surface of a display unit. The external shooting lens and the optical conversion device enable a device to realize a real-time naked-eye three-dimensional display process, and reduce the system complexity of the three-dimensional display system.

Description

Technical Field

[0001] This invention relates to the field of display imaging technology, and more particularly to three-dimensional display systems, three-dimensional display methods, electronic devices, and storage media. Background Technology

[0002] Glasses-free 3D display, also known as naked-eye 3D display, allows users to obtain more information about objects or people compared to 2D displays. Glasses-free 3D display achieves a stereoscopic visual effect by displaying three-dimensional graphics on a flat surface without the aid of external tools such as polarized glasses.

[0003] In related technologies, 3D display systems for naked-eye 3D displays all utilize imaging equipment to generate 2D video or images, which are then sent to another device for 3D display processing to create a 3D viewing effect. The process of achieving 3D display requires the joint operation of multiple devices, making the 3D display system quite complex. Summary of the Invention

[0004] The main objective of this application is to propose a three-dimensional display system, a three-dimensional display method, an electronic device, and a storage medium, thereby reducing the complexity of the three-dimensional display system.

[0005] To achieve the above objectives, a first aspect of this application provides a three-dimensional display system, including: a first camera, a shooting lens, a processing unit, and a three-dimensional display module;

[0006] The shooting lens includes a prism, which is placed above the first camera and divides the lens surface of the first camera into a first region and a second region. This allows a first initial light signal of the shooting target to reach the first region and form a first parallax image in the first region. The second initial light signal of the shooting target passes through the prism and enters the second region to form a second parallax image. The second parallax image has a parallax with the first parallax image.

[0007] The processing unit is electrically connected to the first camera and is used to receive the first parallax image and the second parallax image, and to perform image processing on the second parallax image based on the first parallax image to obtain a third parallax image; the processing unit is also used to generate 3D frame images using the first parallax image and the third parallax image.

[0008] The three-dimensional display module includes a display unit and an optical conversion device. The optical conversion device is disposed on the display plane of the display unit and is used to convert the light beam emitted from the display unit into a left eye light beam emitted to the left eye of the subject and a right eye light beam emitted to the right eye of the subject to achieve three-dimensional display.

[0009] The display unit is electrically connected to the processor. The display unit is used to receive and display the 3D frame image so that the displayed 3D frame image is displayed in three dimensions after passing through the optical conversion device.

[0010] In one embodiment, when the processing unit performs image processing on the second disparity image based on the first disparity image to obtain a third disparity image, it performs the following steps:

[0011] If the second disparity image is a mirror image of the first disparity image, perform mirror restoration processing on the second disparity image to obtain a third disparity image that matches the first disparity image;

[0012] Alternatively, if the second disparity image is a rotated image of the first disparity image, the second disparity image is rotated and restored to obtain a third disparity image that matches the first disparity image.

[0013] Alternatively, if the second disparity image is a scaled-down image of the first disparity image, the second disparity image is mirrored and magnified to obtain a third disparity image that matches the first disparity image.

[0014] Alternatively, if the second disparity image is a magnified version of the first disparity image, the second disparity image is mirrored and scaled down to obtain a third disparity image that matches the first disparity image.

[0015] In one embodiment, the processing unit performs the following steps when generating a 3D frame image using the first parallax image and the third parallax image:

[0016] The third parallax image is processed into a first frame image in 3D format, and the first parallax image is processed into a second frame image in 3D format; the first frame image and the second frame image are left-right format images of each other;

[0017] The first frame image and the second frame image are merged and stitched together to obtain the 3D frame image in 3D format.

[0018] In one embodiment, processing the third parallax image into a first frame image in 3D format includes:

[0019] The transformed and restored third disparity image is compared with the first disparity image to obtain a comparison result characterizing the degree of overlap between the images;

[0020] Based on the comparison results, the third disparity image is calibrated to obtain a calibrated fourth disparity image, so that the overlap between the calibrated fourth disparity image and the first disparity image meets the preset requirements.

[0021] The calibrated fourth parallax image is processed into the first frame image in 3D format.

[0022] In one embodiment, the step of comparing the transformed and restored third disparity image with the second disparity image to obtain a comparison result characterizing the degree of overlap between the images includes:

[0023] Identify target features in the third parallax image to obtain the first target corner point of the target feature in the third parallax image;

[0024] Identify the target features in the first parallax image to obtain the second target corner point of the target features in the first parallax image;

[0025] The third disparity image and the first disparity image are overlaid, and the first target corner point and the second target corner point are virtually overlapped for comparison. The comparison result representing the overlap between the images is obtained based on the overlap between the corner points.

[0026] In one embodiment, processing the third parallax image into a first frame image in 3D format includes:

[0027] The redundant portion in the third disparity image compared to the first disparity image is determined to obtain the first redundant portion;

[0028] The first redundant part is cropped out from the third parallax image, and the remaining third parallax image is processed into the first frame image in 3D format;

[0029] Alternatively, processing the first parallax image into a second frame image in 3D format includes:

[0030] The redundant portion in the first parallax image that is different from the third parallax image is determined to obtain the second redundant portion;

[0031] The second redundant portion is cropped from the first parallax image, and the remaining second parallax image is processed into a second frame image in 3D format.

[0032] In one embodiment, the first frame image includes a plurality of first frame images of a preset width arranged in a first arrangement order; the second frame image includes a plurality of second frame images of the preset width arranged in a second arrangement order.

[0033] The step of fusing and stitching the first frame image with the second frame image to obtain a 3D frame image in 3D format includes:

[0034] According to the first arrangement order and the second arrangement order, the first frame image and the second frame image are alternately arranged to obtain the 3D frame image.

[0035] In one embodiment, the shooting lens includes:

[0036] The outer casing is a columnar structure with a first opening at the top and a second opening at the bottom, the second opening being close to the plane of the first camera for accommodating the first camera; the first camera plane is the plane containing the lens surface of the first camera.

[0037] A prism is disposed above the first camera; the prism is fitted to one side of the interior of the housing, so that a first cavity is formed between the inner wall of the other side of the housing and the prism, thereby allowing the first initial light signal of the target to pass through the first opening, and enter the first camera after passing through the first cavity, and the second initial light signal of the target to pass through the first opening, and enter the first camera after passing through the prism.

[0038] The prism includes: a first prism surface, and an incident surface and an exit surface arranged sequentially along the second initial optical signal optical path;

[0039] The first prism surface is perpendicular to the plane of the first camera, and the extended surface of the first prism surface divides the lens surface of the first camera into a first region and a second region.

[0040] The first region is used to receive the first initial light signal, and the second region is used to receive the light signal emitted from the light-emitting surface, so that the target being photographed forms a first parallax image in the first region and a second parallax image corresponding to the first parallax image in the second region.

[0041] In one embodiment, the prism further includes a reflective surface;

[0042] The angle between the light-incident surface and the reflective surface forms a first angle, and the angle between the first prism surface and the light-emitting surface forms a second angle.

[0043] The first angle causes the second initial light signal to enter the prism from the light-incident surface to obtain the first light signal, the first light signal to be reflected by the reflective surface to obtain the second light signal, the second light signal to be emitted from the light-out surface to obtain the target light signal, and the second angle causes the target light signal to reach the second region on the lens surface.

[0044] In one embodiment, the longest line segment perpendicular to the boundary line between the lens surface of the first camera and the first region and the second region is defined as the first line segment, and the first line segment forms a first endpoint at the edge of the first region;

[0045] The outer casing further includes: a first outer casing wall having a first height disposed opposite to the prism, the first outer casing wall including a first outer casing edge;

[0046] The first endpoint and the first outer shell edge constitute the first boundary surface of the effective shooting range of the first camera;

[0047] The extended surface of the first prism constitutes the second boundary surface of the effective shooting range of the first camera;

[0048] The first boundary surface and the second boundary surface define the effective shooting range of the first camera, so that a first initial beam of the shooting target within the effective shooting range enters the first region; and a second initial beam of the shooting target within the effective shooting range enters the second region through the prism.

[0049] In one embodiment, the intersection of the first line segment with the boundary line of the first region and the second region is the second endpoint, and the boundary line is the projection line of the first prism surface on the lens surface;

[0050] The shooting target includes a first shooting boundary point and a second shooting boundary point; the first initial light signal includes a first boundary light signal emitted from the first shooting boundary point and a second boundary light signal emitted from the second shooting boundary point; the second initial light signal includes a third boundary light signal emitted from the first shooting boundary point and a fourth boundary light signal emitted from the second shooting boundary point.

[0051] The first boundary light signal passes through the first cavity to reach the first endpoint, and the second boundary light signal passes through the first cavity to reach the second endpoint, so as to form the first parallax image in the first region;

[0052] The third boundary light signal enters the prism from the light-incident surface, passes through the reflective surface and the light-exiting surface in sequence, and reaches the first arrival point in the second region; the fourth boundary light signal enters the prism from the light-incident surface, passes through the reflective surface and the light-exiting surface in sequence, and reaches the second arrival point in the second region; the first arrival point is located at the mirror position or the first adjacent mirror position of the first endpoint, and the second arrival point is located at the mirror position or the second adjacent mirror position of the second endpoint, so as to form a second parallax image in the second region that is mirrored or mirror-scaled with the first parallax image.

[0053] In one embodiment, the first line segment forms a third endpoint at the edge of the second region, the position of the first arrival point is the third endpoint, and the position of the second arrival point is the second endpoint, so as to form a first parallax image in the first region and a second parallax image that is a mirror image of the first parallax image in the second region.

[0054] In one embodiment, the system further includes a second camera connected to the processor;

[0055] The second camera is used to acquire facial images of a person and send the facial images to the processing unit;

[0056] The processing unit is used to obtain the eyeball position information of the person object based on the facial image, and the eyeball position information is used to characterize the position of the person object's eyeballs.

[0057] The processing unit is further configured to adjust the preset width according to the eye position information, so that the first parallax image is emitted to the left eye of the subject in a first direction, and the third parallax image is emitted to the right eye of the subject in a second direction.

[0058] In one embodiment, the processing unit performs the following steps when it obtains the eye position information of a person based on the facial image:

[0059] The facial image is used to detect the eye region based on a preset detector to obtain the eye socket position information;

[0060] The facial image is converted into a grayscale image, and the grayscale image is binarized to obtain a first preprocessed image;

[0061] The first preprocessed image is subjected to erosion and dilation processing, and noise in the image is removed to obtain a second preprocessed image. The position of the circular region representing the eyeball of the human subject is extracted in the second preprocessed image using a circular structuring element to obtain the eyeball position information of the human subject.

[0062] In one embodiment, the optical conversion device is a three-dimensional display conversion film, which is a slit-type liquid crystal grating film, a columnar lens film, or a light-pointing film.

[0063] To achieve the above objectives, a second aspect of this application provides a three-dimensional display method applied to a three-dimensional display system as described in any one of the first aspects, the method comprising:

[0064] Acquire the first parallax image formed by the first initial light signal of the shooting target arriving in the first region, and acquire the second parallax image formed by the second initial light signal of the shooting target entering the second region after passing through the prism;

[0065] The second disparity image is mirrored based on the first disparity image to obtain a third disparity image; the third disparity image has a disparity with the first disparity image.

[0066] A 3D frame image is obtained using the first parallax image and the third parallax image;

[0067] The optical conversion device is used to display the 3D frame image in three dimensions.

[0068] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0069] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0070] The three-dimensional display system, three-dimensional display method, electronic device, and storage medium proposed in this application embodiment include: a three-dimensional display system comprising: a camera, a shooting lens, a processing unit, and a three-dimensional display module; a prism in the shooting lens is placed above the camera to divide the lens surface of the camera into a first region and a second region; a first initial light signal of the shooting target arrives at the first region to form a first parallax image in the first region, and a second initial light signal of the shooting target enters the second region after passing through the prism to form a second parallax image; the processing unit performs image processing on the second parallax image according to the first parallax image to obtain a third parallax image, and generates a 3D frame image using the first parallax image and the third parallax image; the display unit receives and displays the 3D frame image, and an optical conversion device is disposed on the display plane of the display unit to display the 3D frame image in three dimensions. In this embodiment, the 3D display system splits the light signal of the target into two paths through the camera lens, forming a first parallax image and a second parallax image of the target. The second parallax image is then processed by the processing unit to obtain a third parallax image that matches the first parallax image. The first and third parallax images are then fused to obtain a 3D frame image. Finally, the 3D frame image is projected onto the field of view of the subject using an optical conversion device mounted on the surface of the display unit. The use of an external camera lens and optical conversion device enables a single device to achieve real-time naked-eye 3D display, reducing the system complexity of the 3D display system. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of a three-dimensional display system provided in an embodiment of the present invention.

[0072] Figure 2a This is a physical schematic diagram of a three-dimensional display system provided in another embodiment of the present invention.

[0073] Figure 2b yes Figure 2a Another schematic diagram of the three-dimensional display system.

[0074] Figure 3 yes Figure 2a Schematic diagrams of the 3D display system from different angles.

[0075] Figure 4 This is a schematic diagram of a prism in a three-dimensional display system provided in another embodiment of the present invention.

[0076] Figure 5 for Figure 4 The six views of a medium prism.

[0077] Figure 6 This is a schematic diagram showing the positional relationship between the first prism surface of the prism and the first camera in a three-dimensional display system provided by another embodiment of the present invention.

[0078] Figure 7 This is a schematic diagram of the structure of the camera lens in a three-dimensional display system provided in another embodiment of the present invention.

[0079] Figure 8 This is an optical path diagram of the camera lens in a three-dimensional display system provided in another embodiment of the present invention.

[0080] Figure 9 This is yet another optical path diagram of the camera lens in a three-dimensional display system provided in another embodiment of the present invention.

[0081] Figure 10 This is an imaging schematic diagram of the camera lens in a three-dimensional display system provided in another embodiment of the present invention.

[0082] Figure 11 This is a schematic diagram of the process by which a processing unit in a three-dimensional display system, according to another embodiment of the present invention, performs image processing to obtain a third parallax image.

[0083] Figure 12 This is a schematic diagram showing the left-right mirror image of the second parallax image and the first parallax image in a three-dimensional display system provided by another embodiment of the present invention.

[0084] Figure 13 This is a schematic diagram showing the second parallax image and the first parallax image being inverted in a three-dimensional display system provided in another embodiment of the present invention.

[0085] Figure 14 This is a schematic diagram of a second parallax image being a scaled-down image of the first parallax image in a three-dimensional display system provided by another embodiment of the present invention.

[0086] Figure 15 This is a schematic diagram of a second parallax image being an enlarged image of the first parallax image in a three-dimensional display system provided by another embodiment of the present invention.

[0087] Figure 16 This is a flowchart of the process of generating 3D frame images by the processing unit in a three-dimensional display system provided in another embodiment of the present invention.

[0088] Figure 17 This is a flowchart illustrating the process by which the processing unit in a three-dimensional display system, according to another embodiment of the present invention, processes a third parallax image into a first frame image in 3D format.

[0089] Figure 18 This is a schematic diagram of the corner position in a three-dimensional display system provided in another embodiment of the present invention.

[0090] Figure 19 This is a schematic diagram of image overlap in a three-dimensional display system provided in another embodiment of the present invention.

[0091] Figure 20 This is a schematic diagram of the fourth parallax image in a three-dimensional display system provided in another embodiment of the present invention.

[0092] Figure 21 This is a schematic diagram of the alternating arrangement of the first frame image and the second frame image in a three-dimensional display system provided by another embodiment of the present invention.

[0093] Figure 22 This is a schematic diagram of an optical conversion device in a three-dimensional display system provided in another embodiment of the present invention.

[0094] Figure 23 This is a schematic diagram of the second camera in a three-dimensional display system provided in another embodiment of the present invention.

[0095] Figure 24 This is a schematic diagram of adjusting a preset width in a three-dimensional display system provided in another embodiment of the present invention.

[0096] Figure 25 This is a schematic diagram of the process by which the processing unit in a three-dimensional display system obtains the eye position information of a person based on facial image analysis, according to another embodiment of the present invention.

[0097] Figure 26 This is a flowchart of a three-dimensional display method provided in another embodiment of the present invention.

[0098] Figure 27 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0099] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0100] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0101] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0102] Glasses-free 3D display, also known as naked-eye 3D display, allows users to obtain more information about objects or people compared to 2D displays. Glasses-free 3D display achieves a stereoscopic visual effect by displaying three-dimensional graphics on a flat surface without the aid of external tools such as polarized glasses.

[0103] In related technologies, 3D display systems for naked-eye 3D displays all utilize imaging equipment to generate 2D video or images, which are then sent to another device for 3D display processing to create a 3D viewing effect. The process of achieving 3D display requires the joint operation of multiple devices, making the 3D display system quite complex.

[0104] Based on this, embodiments of the present invention provide a three-dimensional display system, a three-dimensional display method, an electronic device, and a storage medium. The three-dimensional display system splits the light signal of the target into two paths through a camera lens to form a first parallax image and a second parallax image of the target. The second parallax image is then processed by a processing unit to obtain a third parallax image that matches the first parallax image. The first and third parallax images are then fused to obtain a 3D frame image. Finally, an optical conversion device disposed on the surface of the display unit projects the 3D frame image onto the field of vision of the subject. The use of an external camera lens and optical conversion device enables a single device to achieve real-time naked-eye three-dimensional display, reducing the system complexity of the three-dimensional display system.

[0105] The present invention provides a three-dimensional display system, a three-dimensional display method, an electronic device, and a storage medium, which are specifically described through the following embodiments. First, the three-dimensional display system in the embodiments of the present invention is described.

[0106] Reference Figure 1 In this application embodiment, a three-dimensional display system 10 includes: a first camera 200, a shooting lens 100, a processing unit 300, and a three-dimensional display module 400, wherein the three-dimensional display module 400 includes a display unit 410 and an optical conversion device 420.

[0107] In one embodiment, the first camera 200, the processing unit 300, and the display unit 410 may be functional components of the same electronic device. This electronic device utilizes the first camera 200 to perform photo or video recording functions, the processing unit 300 to perform computational processing, and the display unit 410 to display the image. The three-dimensional display system 10 of this embodiment can be obtained by installing a camera lens 100 and an optical conversion device 420 for three-dimensional display on this electronic device.

[0108] Electronic devices can be mobile phones, tablets, laptops, handheld computers, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc. Non-mobile electronic devices can be network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. The embodiments of this application do not impose specific limitations.

[0109] Reference Figure 2a and Figure 2b The first camera 200 is mounted on an electronic device, which in this embodiment is a mobile phone. In other embodiments, the electronic device can be a tablet, camera, or other devices with shooting capabilities. The first camera 200 is mounted on the upper left corner of the back of the electronic device. Light from outside the electronic device enters the first camera 200, and then the first camera 200 collects the light to form an image. It is understood that... Figure 2a and Figure 2b The mounting position of the first camera 200 in the illustrated embodiment of the electronic device is merely illustrative. In some other embodiments, the first camera 200 may be mounted in other locations on the electronic device. The first camera 200 may be mounted in the upper center or upper right corner of the back of the electronic device, etc. This embodiment does not limit the mounting position of the first camera 200.

[0110] Meanwhile, the first camera 200 can be an ultra-wide-angle lens, a wide-angle lens, a main camera lens, a telephoto lens, a 2x telephoto lens, a monocular lens, an infrared lens, a depth-sensing lens, etc. This embodiment does not limit the type of the first camera 200. Furthermore, the number of first cameras 200 can also be N, where N is an integer greater than or equal to 1. Figure 2 illustrates this with an example of three cameras. If the electronic device contains multiple first cameras, one camera is selected as the target camera based on the shooting requirements; this target camera is the first camera 200 in this embodiment. For example, the main camera among multiple cameras can be selected as the target camera. It is understood that this embodiment does not limit the number of first cameras 200.

[0111] Reference Figure 2a and Figure 2bThe camera lens 100 is mounted above the first camera (the area is outlined by the dotted line in the figure). The camera lens 100 includes a prism 110 and a housing 120 (shown by the black frame in the figure). The housing 120 is a columnar structure with a first opening at the top. Figure 2b This is a schematic diagram showing the view from the perspective of the electronic device, perpendicular to the first opening, combined with... Figure 2b It can be seen that in this columnar structure, the prism 110 is attached to one side of the interior of the outer shell 120. Figure 2b (As shown in the upper part), the prism 110 is positioned above the camera 200, and a first cavity is formed between the inner wall of the other side of the housing 120 and the prism 110. The housing 120 has a second opening at its lower part, close to the camera plane, for accommodating the first camera. This embodiment does not specifically limit the size of the second opening, only requiring that it be able to accommodate the reference... Figure 2b The diagram shows that the second opening can accommodate the camera 200. Here, the camera plane is defined as the plane on which the lens surface of the camera 200 is located.

[0112] In one embodiment, refer to Figure 3 The prism 110 is placed above the first camera 200, dividing the lens surface of the first camera 200 into a first region Q1 and a second region Q2. The first initial light signal of the target arrives at the first region Q1 and forms a first parallax image in the first region. The second initial light signal of the target enters the second region Q2 after passing through the prism and forms a second parallax image. There is a parallax between the second parallax image and the first parallax image.

[0113] Reference Figure 3 The prism 110 is fitted to one side of the interior of the housing 120. The housing 120 has a first opening at the top, which is illustrated as K1 in the figure. A first cavity is formed between the inner wall of the other side of the housing 120 and the prism 110. Simultaneously, the prism 110 is positioned above the first camera 200, and the housing 120 has a second opening at the bottom, which is illustrated as K2 in the figure. The discontinuous area of ​​the second opening K2 represents the area accommodating the first camera.

[0114] In this embodiment, prism 110 is fitted to the lens surface of the first camera 200. The first prism surface 111 of prism 110 is perpendicular to the first camera 200. The extension surface of the first prism surface 111 towards the first camera 200 divides the lens surface of the first camera 200 into a first region Q1 and a second region Q2. The first edge L1 of prism 110 is in contact with the lens surface of the first camera 200, or at least very close to it. In one embodiment, the length of the first edge L1 needs to be greater than or equal to the diameter of the first camera 200.

[0115] Figure 3The target AB is located within the shooting range of the first camera 200. Target AB emits two light signals: a first initial light signal S1, which passes through the first opening K1, through the first cavity, and enters the first region Q1 of the first camera 200. Simultaneously, the other light signal from target AB is a second initial light signal S2, which passes through the first opening K1, through the light-incident surface 113 of the prism 110, and exits through the light-exit surface 115 of the prism 110, entering the second region Q2 of the first camera 200. In the diagram, S2' represents the second initial light signal S2 exiting through the light-exit surface 115. It should be understood that the light paths of the first and second initial light signals S1 and S2 in the diagram are for illustrative purposes only and do not define the actual light paths.

[0116] As can be seen from the above, the first initial light signal S1 received in the first region Q1 and the light signal S2' emitted from the light-emitting surface received in the second region Q2 are light signals of the same target under small parallax. Therefore, the target forms a first parallax image in the first region Q1 and a second parallax image corresponding to the first parallax image in the second region Q2. The first parallax image and the second parallax image obtained at this time can be used for three-dimensional display and can produce a good display effect.

[0117] Next, the detailed structure of prism 110 is described. In one embodiment, Figure 4 This is a schematic diagram of a prism in one embodiment. Figure 5 for Figure 4 The six views of a prism. The six views of a prism include: front view, left view, right view, top view, bottom view, and rear view.

[0118] Reference Figure 4 The prism 110 includes: a first prism surface 111, a second prism surface 112, an incident light surface 113, a reflecting surface 114, an exiting light surface 115, a first inclined surface 116, and a second inclined surface 117.

[0119] In this configuration, the first prism surface 111 intersects with both the incident light surface 113 and the emitting light surface 115. The other end of the incident light surface 113 intersects with the reflecting surface 114 to form a second edge L2. The other end of the emitting light surface 115 intersects with the second prism surface 112 to form a first edge L1. The other end of the second prism surface 112 intersects with the reflecting surface 114. Furthermore, the angle between the incident light surface 113 and the reflecting surface 114 forms a first angle, the angle between the first prism surface 111 and the emitting light surface 115 forms a second angle, and the angle between the second prism surface 112 and the reflecting surface 114 forms a third angle.

[0120] Figure 4The intermediate prism 110 further includes a first inclined plane 116 and a second inclined plane 117, wherein both the first inclined plane 116 and the second inclined plane 117 are perpendicular to the first prism surface 111. It can be understood that the prism 110 of this embodiment can be obtained by cutting the prism-like structure formed by the first prism surface 111, the second prism surface 112, the incident light surface 113, the reflecting surface 114, and the emitting light surface 115 through the first inclined plane 116 and the second inclined plane 117. Assuming that the two endpoints of the first edge L1 are the first vertex D1 and the second vertex D2, and the two endpoints of the second edge L2 are the third vertex D3 and the fourth vertex D4, then the first vertex D1 and the third vertex D3 form the first inclined plane, and the second vertex D2 and the fourth vertex D4 form the second inclined plane. The first inclined plane 116 passes through the first inclined plane, and the second inclined plane 117 passes through the second inclined plane.

[0121] In addition, combined Figure 5 and Figure 4 For cost reasons, the lengths of the first edge L1 and the second edge L2 are set to be different. Therefore, the first inclined plane 116 and the second inclined plane 117 cut the prism 110 into the shape shown below. Figure 5 The illustrated structure is trapezoidal, but this does not mean that the prism 110 in this embodiment can only be trapezoidal. If the lengths of the first edge L1 and the second edge L2 are the same, the front view of the prism becomes rectangular. This embodiment does not specifically limit the length relationship between the first edge L1 and the second edge L2.

[0122] Figure 6 This is a schematic diagram showing the positional relationship between the first prism surface and the first camera in one embodiment.

[0123] Reference Figure 6 In this embodiment, the first camera is circular in shape. Since the first prism is perpendicular to the first camera plane, that is, perpendicular to the lens surface, the projection of the first prism onto the lens surface is a line, namely projection line L3 in the figure. Projection line L3 divides the lens surface into a first region Q1 and a second region Q2. In other words, projection line L3 is the boundary line between the first region Q1 and the second region Q2.

[0124] In this embodiment, the projection line L3 is located exactly at the center line of the lens surface. Therefore, the first region Q1 and the second region Q2 are the same size, meaning that the projection line L3 bisects the lens surface. However, this embodiment does not limit whether the projection line L3 must be located at the center line of the lens surface. The projection line L3 can deviate from the center line. This deviation will affect the accuracy of the first and second parallax images, thereby affecting the imaging effect of the subsequent 3D display. Generally speaking, the closer the projection line L3 is to the center line of the lens surface, the better the accuracy of the first and second parallax images, and the better the imaging effect of the subsequent 3D display. It is understood that this embodiment only requires that this deviation be within an acceptable range, that is, combined with... Figure 6 The projection line L3 can be deviated to the left or right within a certain range.

[0125] Figure 6 The diagram also shows that the first edge L1 generates multiple line segments perpendicular to the boundary line (i.e., projection line L3) on the lens surface, with the longest line segment selected as the first line segment, which passes through the center of the lens surface. In this embodiment, the intersection point formed by the first line segment and the edge of the first region Q1 is defined as the first endpoint C1, the intersection point formed by the first line segment and the projection line L3 is defined as the second endpoint C2, and the intersection point formed by the first line segment and the edge of the second region Q2 is defined as the third endpoint C3.

[0126] Figure 7 This is a schematic diagram of the structure of the camera lens in one embodiment.

[0127] Reference Figure 3 and Figure 7 The outer casing 120 includes: a first outer casing wall W1 having a first height h1 disposed opposite to the prism, the first outer casing wall W1 including a first outer casing edge R1, wherein the first outer casing edge R1 is the edge of the first outer casing wall W1 in the first opening K1 region. (Refer to...) Figure 6 The first outer shell edge R1 can be a straight line or a curve. The first outer shell edge R1 contains a first vertex M1. When the first outer shell edge R1 is a straight line, the first vertex M1 can be any position on the first outer shell edge R1. When the first outer shell edge R1 is a curve, the first vertex M1 is the position with the highest peak value on the first outer shell edge R1. The first height h1 is defined as the vertical distance between the first vertex M1 and the camera plane.

[0128] Reference Figure 7 Because the outer casing 120 obstructs the view of the camera 200, the field of view of the camera 200 changes, and its shooting range also changes accordingly. In this embodiment, the shooting range of the camera 200 under the influence of the outer casing 120 is referred to as the effective shooting range (shown in the shaded area in the figure).

[0129] In this system, the first endpoint C1 of the lens surface and the first outer shell edge R1 form the first boundary surface of the effective field of view of the camera 200. The extended surface of the first prism surface 111 forms the second boundary surface of the effective field of view of the camera 200. The first and second boundary surfaces then define the effective shooting range of the camera, as shown in the shaded area in the figure. Within the effective shooting range, the first initial light signal S1 of the target enters the first region Q1, and its second initial light signal S2 enters the second region Q2 through the prism 110. It is evident that the effective shooting range is related to both the height and position of the first outer shell wall W1.

[0130] Figure 8 This is an optical path diagram of the camera lens in one embodiment.

[0131] In the figure, it is assumed that the shooting target AB includes two boundary points, namely the first shooting boundary point A and the second shooting boundary point B. Since the shooting target AB contains a first initial light signal and a second initial light signal, for the two boundary points, the first initial light signal S1 includes: the first boundary light signal g1 emitted from the first shooting boundary point A and the second boundary light signal g2 emitted from the second shooting boundary point B. The second initial light signal S2 includes: the third boundary light signal g3 emitted from the first shooting boundary point A and the fourth boundary light signal g4 emitted from the second shooting boundary point B.

[0132] Analysis of the optical path diagram shows that the first boundary light signal g1, within the effective imaging range, passes through the first cavity and reaches the first endpoint C1 of the first region Q1. The first camera 200 images the first imaging boundary point A at the first endpoint C1. Simultaneously, the second boundary light signal g2, within the effective imaging range, passes through the first cavity and travels along the first prism surface 111 to reach the second endpoint C2 of the first region. The first camera 200 images the second imaging boundary point B at the second endpoint C2. Therefore, referring to... Figure 10 The first region Q1 will form a first parallax image about the shooting target AB.

[0133] On the other hand, since the second initial light signal S2 enters the prism 110 from the light-incident surface 113 to obtain the first light signal, the first light signal is reflected by the reflecting surface 114 to obtain the second light signal, and the second light signal is emitted from the light-outcrystal surface 115 to obtain the target light signal, the target light signal reaches the second region on the lens surface. When the second initial light signal S2 is the third boundary light signal g3, the first light signal is the first boundary point first light signal g31, the second light signal is the first boundary point second light signal g32, and the target light signal is the first boundary point target light signal g33; when the second initial light signal S2 is the fourth boundary light signal g4, ​​the first light signal is the second boundary point first light signal g41, the second light signal is the second boundary point second light signal g42, and the target light signal is the second boundary point target light signal g43. Therefore, the optical path is described as follows:

[0134] The third boundary light signal g3 enters the prism 110 from the incident surface 113. Since it is not incident perpendicularly to the incident surface 113, the third boundary light signal g3 is refracted after passing through the incident surface 113. The refraction changes the propagation direction of the third boundary light signal g3, generating the first boundary point first light signal g31. The first boundary point first light signal g31 continues to propagate inside the prism 110 and is reflected by the reflecting surface 114. The reflection changes the propagation direction of the first boundary point first light signal g31, generating the first boundary point second light signal g32. The first boundary point second light signal g32 continues to propagate inside the prism 110 and reaches the exit surface 115. The figure illustrates this by taking the example that the first boundary point second light signal g32 just exits from the first edge. Therefore, the first boundary point second light signal g32 passes through the exit surface 115 and reaches the first arrival point F1 of the second region.

[0135] Simultaneously, the fourth boundary light signal g4 enters the prism 110 from the incident surface 113. Since it is not incident perpendicularly to the incident surface 113, the fourth boundary light signal g4 is refracted after passing through the incident surface 113. The refraction changes the propagation direction of the fourth boundary light signal g4, ​​generating the second boundary point first light signal g41. The second boundary point first light signal g41 continues to propagate inside the prism 110 and is reflected by the reflecting surface 114. The reflection changes the propagation direction of the second boundary point first light signal g41, generating the second boundary point second light signal g42. The second boundary point second light signal g42 continues to propagate inside the prism 110 and reaches the exit surface 115. Since it is not incident perpendicularly to the exit surface 115, the exit surface 115 is refracted at the contact point. The refraction changes the propagation direction of the second boundary point second light signal g42, generating the second boundary point target light signal g43. Therefore, the second boundary point target light signal g43 passes through the exit surface 115 and reaches the second arrival point F2 of the second region.

[0136] As can be seen from the above, referring to Figure 9 If the second light signal g32 at the first boundary point is not emitted from the first edge, it will be refracted at the light-emitting surface 115 to generate the target light signal g33 at the first boundary point. Then the first arrival point F1' of the target light signal g33 at the first boundary point will be closer to the second endpoint C2.

[0137] Reference Figure 9The fourth boundary light signal g4 passes directly through the incident light surface, and the first boundary point target light signal g33 is obtained by refraction of the first boundary point second light signal g32 through the exit light surface. Therefore, in this embodiment, when the second initial light signal passes through the incident light surface, it can be refracted or directly transmitted to generate the first light signal; similarly, when the second light signal passes through the exit light surface, it can be refracted or directly transmitted to generate the target light signal. It is understood that the transmission method of the second initial light signal through the incident light surface or the second light signal through the exit light surface can be designed by adjusting the refractive index, the first included angle, and / or the second included angle of the prism 110; this embodiment does not specifically limit this.

[0138] Analysis of the optical path diagram shows that the third boundary light signal g3 eventually reaches the first arrival point F1 of the second region Q2. The first camera 200 images the first shooting boundary point A at the first arrival point F1, denoted as A'. Simultaneously, the fourth boundary light signal g4 eventually reaches the second arrival point F2 of the second region Q2. The first camera 200 images the second shooting boundary point B at the second arrival point F2, denoted as B'. Therefore, referring to... Figure 10 The second region Q2 will form a second parallax image with respect to the target AB. Simultaneously, the second parallax image formed by the second region Q2 is a mirror image of the first parallax image, or a mirror image of the first parallax image scaled up.

[0139] In one embodiment, reference is made to Figure 10 The first arrival point F1 is located at a mirror position or a first adjacent mirror position of the first endpoint C1, and the second arrival point F2 is located at a mirror position or a second adjacent mirror position of the second endpoint C2, so as to form a second parallax image in the second region that is a mirror image or a mirror-scaled version of the first parallax image. The mirror position of the first endpoint C1 is the third endpoint C3, and the mirror position of the second endpoint C2 is itself. Therefore, the first mirror position is a small area on the side of the third endpoint C3 facing the second endpoint C2, and the second mirror position is a small area on the side of the second endpoint C2 facing the third endpoint C3. The size of this small area affects the scaling value of the image in the second region Q2. Therefore, the size of the small area is selected according to the desired effect or prism processing technology. It is understood that if the second parallax image formed in the second region Q2 is a mirror image of the mirror-scaled first parallax image, then during subsequent 3D display, the second parallax image needs to be magnified accordingly to match the size of the first parallax image. This embodiment does not limit the positions of the first arrival point F1 and the second arrival point F2.

[0140] Mirror scaling refers to a situation where the second parallax image is a mirror image of the first parallax image, but with a certain scaling ratio. This is because the second initial light signal undergoes refraction and reflection through a prism, resulting in a mirror image relationship between the target light signal received in the second region and the first initial light signal received in the first region. If the first arrival point F1 is located at the first nearest mirror position of the first endpoint C1, or the second arrival point F2 is located at the second nearest mirror position of the second endpoint C2, the imaging distance of the light signal in the second region is reduced, causing the second parallax image in the second region to have a scaling characteristic compared to the first parallax image. The specific scaling ratio depends on the angle design of the incident and / or exit surfaces and the refractive index of the prism material.

[0141] In one embodiment, in order to achieve the desired shooting effect, the refractive index, first included angle and / or second included angle of the prism 110 are adjusted so that the position of the first arrival point F1 is the third endpoint C3 and the position of the second arrival point F2 is the second endpoint C2, so as to form a first parallax image in the first region Q1 and a second parallax image that is a mirror image of the first parallax image in the second region Q2.

[0142] Because the focal lengths of the first cameras vary, in one embodiment, the length of the second edge is set according to the focal length of the first camera. The length of the second edge is positively correlated with the focal length of the first camera; that is, the larger the focal length, the longer the second edge. This is because if the focal length is too large and the second edge is too short, it will cause frame occlusion, affecting the imaging effect. For example, in one embodiment, if the focal length is 24mm, the second edge can be set to 41mm; if the focal length is 26mm, the second edge can be set to 46.7mm. It is understood that the correlation between the focal length and the length of the second edge can be obtained experimentally, and this embodiment does not impose specific limitations on it.

[0143] As can be seen from the above, if the camera lens only uses a plane mirror for reflection, a very large plane mirror is required to ensure that the image captured by the mobile phone lens is complete. This will result in an excessively large device size, which is not suitable as an external lens for mobile phones. Therefore, the embodiment of this application utilizes a prism structure design to solve the problem of excessively large plane mirror size. Only a very small volume is needed to make the captured image fully reflective and obtain a complete image image, which makes the design more portable and practical.

[0144] In one embodiment, the electronic device can perform landscape or portrait shooting. The camera lens can be mounted vertically or horizontally, and the specific mounting method can be set according to requirements. Different mounting positions will result in different first and second parallax images. (Refer to...) Figure 1As shown in Figure 2, during portrait shooting, the camera lenses are mounted vertically, and the lens surface is divided vertically into a first region and a second region. The resulting first and second parallax images are vertically mirrored. If the camera is switched to portrait mode, the lens surface is divided horizontally to obtain a first and second region, and the resulting first and second parallax images are horizontally mirrored. Alternatively, in one embodiment, during portrait shooting, the camera lenses are mounted horizontally, and the lens surface is divided horizontally to obtain a first and a second region. The resulting first and second parallax images are horizontally mirrored. If the camera is switched to landscape mode, the lens surface is divided vertically to obtain a first and a second region, and the resulting first and second parallax images are vertically mirrored.

[0145] As can be seen from the above, the embodiments of this application can directly obtain two sets of images with parallax about the target by combining the shooting lens and the first camera, namely the first parallax image and the second parallax image. The image processing process of the processing unit is described below.

[0146] In one embodiment, the processing unit 300 is located inside the electronic device, as shown in Figure 2 and... Figure 3 The images are not shown in the diagram. The processing unit 300 and the first camera 200 are electrically connected within the electronic device. For example, the processing unit 300 obtains image data captured by the first camera 200 by accessing its internal data interface. In this embodiment, the processing unit 300 obtains a first parallax image and a second parallax image captured by the first camera 200 through the camera data interface. Then, it performs image processing on the second parallax image based on the first parallax image to obtain a third parallax image. Subsequently, the processing unit 300 also uses the first parallax image and the third parallax image to generate a 3D frame image.

[0147] In one embodiment, reference is made to Figure 11 When the processing unit 300 performs image processing on the second disparity image based on the first disparity image to obtain the third disparity image, it performs the following steps:

[0148] Step S1110: If the second disparity image is a mirror image of the first disparity image, perform mirror restoration processing on the second disparity image to obtain a third disparity image that matches the first disparity image.

[0149] In one embodiment, the second disparity image is an image that is mirrored horizontally, vertically, or after multiple mirroring operations with the first disparity image. Therefore, it is necessary to perform image geometric transformation to mirror and flip the second disparity image to obtain the third disparity image. (Refer to...) Figure 12The second disparity image is shown as a left-right mirror image of the first disparity image. Since images are stored in the form of pixels, geometric transformations of the image can be achieved by changing the spatial position of the pixels. Specifically, a mapping relationship is established between the pixels of the original image and the pixels of the transformed image, and the geometric transformation of the image is achieved through this mapping relationship.

[0150] In one embodiment, if only one mirroring is performed, the second parallax image is mirrored and flipped using the `flip()` function in OpenCV to obtain the third parallax image. The calling relationship is represented as: `dst = cv2.flip(src, flipCode)`, where `src` represents the second parallax image, and `flipCode` represents the flipping direction (if `flipCode` is 0, it is flipped with the X-axis as the axis of symmetry; if `flipCode` > 0, it is flipped with the Y-axis as the axis of symmetry; if `flipCode` < 0, it is flipped simultaneously along both the X and Y axes). In this embodiment, `flipCode` is 0, indicating that the second parallax image is flipped vertically to obtain the third parallax image; `flipCode` > 0, indicating that the second parallax image is flipped horizontally to obtain the third parallax image; `flipCode` < 0, indicating that the second parallax image is flipped horizontally and then vertically to obtain the third parallax image.

[0151] Step S1120: If the second disparity image is a rotated image of the first disparity image, perform rotation recovery processing on the second disparity image to obtain a third disparity image that matches the first disparity image.

[0152] In one embodiment, the second parallax image is an image rotated relative to the first parallax image. (See also...) Figure 13 The second parallax image is an inverted version of the first parallax image; that is, the second parallax image is an inverted version of the first parallax image after being rotated 180°. Therefore, a geometric transformation is needed to rotate the second parallax image by 90 degrees to obtain the third parallax image. It is understandable that the rotation can be clockwise or counterclockwise depending on the actual needs; the diagram uses a 90° clockwise rotation as an example.

[0153] In one embodiment, a second disparity image is read and processed by calling image rotation-related functions in OpenCV, such as the getRotationMatrix2D() and wrapAffine() functions, to obtain a third disparity image. The image rotation-related functions in OpenCV can achieve rotation around the center of the image.

[0154] For example, first calling the getRotationMatrix2D() function is represented as:

[0155] M=cv2.getRotationMatrix2D((cols / 2,rows / 2),90,1)

[0156] The function's parameters are as follows: M represents the rotation parameter, cols and rows represent the width and height of the mirrored flipped image of the second parallax image, represents the rotation degree as 180, and scale represents the number of rotation pixels.

[0157] Then call the wrapAffine() function, which is represented as:

[0158] rotated=cv2.warpAffine(src,M,(cols,rows))

[0159] The function's parameters are as follows: src represents the mirrored flipped image of the second parallax image, and M represents the aforementioned rotation parameters.

[0160] Step S1130: If the second disparity image is a scaled-down image of the first disparity image, perform mirror magnification processing on the second disparity image to obtain a third disparity image that matches the first disparity image.

[0161] Step S1140: If the second disparity image is a magnified image of the first disparity image, perform mirror reduction processing on the second disparity image to obtain a third disparity image that matches the first disparity image.

[0162] In one embodiment, the second parallax image is a scaled-down or magnified version of the first parallax image, i.e., a reference image. Figure 10 The first arrival point F1 is located at the first nearest mirror position of the first endpoint C1, or the second arrival point F2 is located at the second nearest mirror position of the second endpoint C2, thereby forming a second parallax image in the second region that is a mirror scale of the first parallax image. Here, mirror scaling refers to the second parallax image being a mirror image of the first parallax image, but with a certain reduction or magnification ratio. This is because the second initial light signal, after refraction and reflection by the prism, results in a mirror inversion between the target light signal received in the second region and the first initial light signal received in the first region. If the first arrival point F1 is located at the first nearest mirror position of the first endpoint C1, or the second arrival point F2 is located at the second nearest mirror position of the second endpoint C2, the imaging distance of the light signal in the second region is reduced, causing the second parallax image in the second region to have a scaling characteristic compared to the first parallax image. The specific reduction or magnification ratio depends on the angle design of the incident and / or exit surfaces and the refractive index of the prism material.

[0163] In one embodiment, reference is made to Figure 14 The second parallax image is a scaled-down version of the first parallax image, as referenced. Figure 15If the second parallax image is a magnified version of the first parallax image, then... Figure 14 The process involves enlarging and resizing the second parallax image to obtain the third parallax image, where the size of the third parallax image is the same as that of the first parallax image. Figure 15 The process involves reducing the size of the second disparity image to obtain the third disparity image, where the size of the third disparity image is the same as that of the first disparity image.

[0164] In the above embodiments, the width and height of the scaled image will change. A horizontal scaling factor controls the scaling of the image width; a value of 1 keeps the image width unchanged. A vertical scaling factor controls the scaling of the image height; a value of 1 keeps the image height unchanged. If the horizontal and vertical scaling factors are not equal, the width-to-height ratio of the scaled image will change, causing image distortion. Therefore, in this embodiment, to maintain the width-to-height ratio of the generated third parallax image consistent with that of the second parallax image, the horizontal and vertical scaling factors are set to be equal.

[0165] In one embodiment, the scaling process in steps S1130 and S1140 above is implemented based on OpenCV scaling. Image scaling is mainly achieved by calling the resize() function, specifically represented as: def = resize(src, dsize, dst = None, fx = None, fy = None, interpolation = None), where the parameters in the function have the following meanings: src represents the second disparity image, dsize represents the size of the first disparity image, and dsize = Size(round(fx*src.cols), round(fy*src.rows)), where fx and fy are the width and height directions in the second disparity image. The scaling factor is 't' in the 't' direction; 'fx' is the scaling factor in the 'width' direction. If it's 0, it will be calculated according to '(double)dsize.width / src.cols'; 'fy' is the scaling factor in the 'height' direction. If it's 0, it will be calculated according to '(double)dsize.height / src.rows'; 'interpolation' specifies the interpolation method. Since pixels need to be recalculated after image scaling, the interpolation method needs to be specified. Common interpolation methods include: nearest neighbor interpolation, bilinear interpolation, resampling using pixel region relationships, bicubic interpolation within a 4x4 pixel neighborhood, or Lanczos interpolation within an 8x8 pixel neighborhood.

[0166] It is understood that the use of OpenCV to process the second disparity image to obtain the third disparity image in this embodiment of the invention does not constitute a limitation on this embodiment.

[0167] In one embodiment, for step S1140 above, if the second parallax image is a magnified image of the first parallax image, refer to... Figure 15 It can be seen that the completeness of the target in the second parallax image is worse than that in the first parallax image. That is, the target may not be fully displayed in the second parallax image. Therefore, the target in the obtained third parallax image will have a certain deviation from that in the first parallax image. This deviation can be made within an acceptable range by adjusting the prism.

[0168] As described above, the processing unit processes the second disparity image based on the first disparity image to obtain a third disparity image that matches the first disparity image. The process by which the processing unit generates a 3D frame image using the first and third disparity images is described below. (Refer to...) Figure 16 The process of generating 3D frame images includes:

[0169] Step S1610: Process the third parallax image into a first frame image in 3D format, and process the first parallax image into a second frame image in 3D format.

[0170] In one embodiment, reference is made to Figure 17 The process of processing the third parallax image into the first frame image in 3D format includes:

[0171] Step S1611: Compare the transformed and restored third disparity image with the first disparity image to obtain the comparison result representing the degree of overlap between the images.

[0172] In one embodiment, step S1611 includes the following steps:

[0173] First, target features in the third disparity image are identified, yielding the first target corner point in the third disparity image. Then, target features in the first disparity image are identified, yielding the second target corner point in the first disparity image. Next, the third and first disparity images are overlaid, and the first and second target corner points are virtually compared. The degree of overlap between the corner points is used to obtain a comparison result representing the degree of overlap between the images.

[0174] In one embodiment, the first target corner point is the position of a preset location of the target in the third parallax image, and the second target corner point is the position of the preset location of the target in the first parallax image. (Refer to...) Figure 18In this example, the preset position of the target in the image is set to the center of the person's forehead. The center of the person's forehead in the third parallax image is positioned as the first target corner point, and the center of the person's forehead in the first parallax image is positioned as the second target corner point. It is understood that the setting of the first and second target corner points can be preset. For example, when the target is a person, more obvious feature points of the person can be selected, such as eyes, forehead, nose, or mouth; when the target is a landscape, specific landscape content can be selected, such as houses, vehicles, trees, roads, mountains, or rivers. Alternatively, the second target corner point can be randomly selected in the first parallax image, and then the pixel point most similar to the second target corner point can be calculated in the third parallax image. This embodiment does not specifically limit this.

[0175] In one embodiment, reference is made to Figure 19 ,Will Figure 18 The first and third disparity images are overlaid. During the overlay process, the distance between the first and second target corner points is determined, and the degree of overlap between the corner points is obtained based on the distance. Since the degree of overlap can be used to characterize the overlap relationship between the first and third disparity images, the comparison result representing the degree of overlap between the images is obtained based on the degree of overlap between the corner points. It is understandable that the correspondence between the distance between two corner points and the degree of overlap can be stored in advance as a chart according to actual needs, and the degree of overlap corresponding to the distance can be obtained by looking up the table when needed.

[0176] Step S1612: Perform image calibration on the third disparity image based on the comparison results to obtain the calibrated fourth disparity image, so that the overlap between the calibrated fourth disparity image and the first disparity image meets the preset requirements.

[0177] In one embodiment, if the overlap does not meet a preset requirement, the third disparity image is adjusted by translation. During the translation process, the overlap is continuously calculated until the overlap meets the preset requirement. When the overlap meets the preset requirement after the translation, the third disparity image becomes the fourth disparity image. (Refer to...) Figure 20When the movement process meets the preset requirements, the third parallax image is shown by the thick solid line in the figure. The third parallax image contains two parts: an overlapping part Q3 and a non-overlapping part Q4 between it and the first parallax image. The first parallax image includes the overlapping part Q3 and the non-overlapping part Q5. Since the third parallax image and the first parallax image are images of the same target from different perspectives, they have a high degree of similarity. The overlap degree ensures that the main parts of the two images overlap as required. For the user, the influence of the image boundary is low. Therefore, the main part of the third parallax image (i.e., the overlapping part Q3) and the edge part of the first parallax image (i.e., the non-overlapping part Q5) are merged together to obtain the fourth parallax image. The size of the fourth parallax image is the same as that of the first parallax image.

[0178] Step S1613: Process the calibrated fourth parallax image into the first frame image in 3D format.

[0179] In one embodiment, for subsequent 3D display, the fourth parallax image is sequentially segmented into multiple first frame images of preset widths according to a preset width. These multiple first frame images of preset widths are then arranged in a first arrangement order to obtain a first frame image containing multiple first frame images. (Refer to...) Figure 20 The first frame image here is an image sequence, where the first frame images are arranged in a first arrangement order. The first arrangement order can be from left to right, counting from 1 to N, where N is an integer greater than or equal to 1. The first arrangement order can be set according to actual needs.

[0180] Meanwhile, in one embodiment, the first parallax image is sequentially segmented into multiple second frame images of preset widths according to a preset width. These multiple second frame images of preset widths are then arranged in a second arrangement order to obtain a second frame image containing multiple second frame images. The first arrangement order can be set according to actual needs. The second arrangement order can also be set according to actual needs.

[0181] It is understandable that the first and second frames are left-right images of each other, meaning one image targets the left eye and the other targets the right eye. Furthermore, the preset width can be adjusted according to actual needs.

[0182] Step S1620: Merge and stitch the first frame image with the second frame image to obtain a 3D frame image in 3D format.

[0183] In one embodiment, the fusion and stitching process involves alternating the first frame image and the second frame image according to the first arrangement order and the second arrangement order to obtain a 3D frame image.

[0184] Reference Figure 21The first frame image is used as the left image projected to the left eye, and the second frame image is used as the right image projected to the right eye. By alternating the left and right images, a 3D frame image can be obtained.

[0185] The above process yields 3D frame images for display. The following describes the process of displaying the 3D frame images in three dimensions.

[0186] The display unit 410 is electrically connected to the processing unit 300 and is used to receive and display 3D frame images sent by the processing unit 300. In this embodiment, the display unit 410 can be a display screen of an electronic device. The optical conversion device 420 is placed on the display plane of the display unit, which can convert two-dimensional 3D frame images into three-dimensional displays. The optical conversion device 420 can convert the light beam emitted from the display unit 410 into a left-eye light beam emitted to the left eye of the subject and a right-eye light beam emitted to the right eye of the subject, thereby realizing three-dimensional display.

[0187] In one embodiment, according to the naked-eye 3D display technology, the optical conversion device 420 may be a slit-type liquid crystal grating device, a lenticular lens, or a pointing light source device, etc.

[0188] In one embodiment, the optical conversion device 420 is a slit-type liquid crystal grating device. The technical principle is that after adding a slit-type grating in front of the screen of the display unit 410, the image that should be seen by the left eye will be emitted to the user's left eye, and at this time the opaque stripe will block the right eye; similarly, the image that should be seen by the right eye will be emitted to the user's right eye, and at this time the opaque stripe will block the left eye. By separating the visible images of the left and right eyes, the user can see 3D images, and the 3D frame image displayed on the display unit 410 is converted into three-dimensional display content.

[0189] In one embodiment, reference is made to Figure 22 The optical conversion device 420 is a cylindrical lens. Its technical principle is to project the corresponding pixels of the left and right eyes into the left and right eyes respectively through the refraction principle of the lens, so as to achieve image separation and enable the user to see 3D images. It converts the left and right images in the 3D frame image displayed on the display unit 410 into the three-dimensional display content seen by the left and right eyes respectively.

[0190] In one embodiment, the optical conversion device 420 is a directional light source device. Its technical principle is to use two sets of LEDs, along with a fast-response LCD panel and driving method, to allow 3D content to enter the viewer's left and right eyes in a sorted manner, creating parallax and thus allowing the human eye to perceive a 3D three-dimensional effect.

[0191] In one embodiment, the optical conversion device 420 can be made into a film and attached above the screen of the display unit 410. The optical conversion film can be a slit-type liquid crystal grating film, a lenticular lens film, or a directional light source film, etc., that is, an optical conversion film with a specially structured slit-type liquid crystal grating, lenticular lens, or directional light source device. It can be in the form of a mobile phone screen protector and can be set to different sizes and specifications according to different display unit 410 models (e.g., mobile phones or tablets). In use, the optical conversion film is attached to the screen of the display unit 410. When displaying 3D frame images, the mobile phone screen protector converts them into a three-dimensional form for display, achieving the effect of naked-eye 3D display. This realizes the entire process of naked-eye 3D display using a single device, including image acquisition, image processing, and image display, expanding the application scenarios of electronic devices.

[0192] Because users have different viewing distances and angles, in one embodiment, to achieve a better display effect, the displayed content is adjusted by obtaining the user's viewing distance. In this embodiment, reference is made to... Figure 23 The 3D display system also includes a second camera 500, which is electrically connected to the processing unit 300. The second camera 500 is used to acquire facial images of a person and send these images to the processing unit 300. The second camera 500 can be a front-facing camera on an electronic device, similar to the first camera 200, and connected to the processing unit 300 via a camera interface. The advantage of using a front-facing camera is that it eliminates the need for additional sensors or cameras, resulting in low cost and suitability for existing devices such as smartphones, tablets, and 3D training terminals.

[0193] Then, the processing unit 300 obtains the eye position information of the person based on the facial image analysis. This eye position information is used to characterize the position of the person's eyes. It should be noted that the facial image in this embodiment can be obtained by directly recognizing the user's face using a front-facing camera. In one embodiment, the facial image is obtained by cropping an image acquired by the front-facing camera. For example, the terminal acquires an image through the front-facing camera, which includes the user's face but may also contain other debris that could interfere with eye recognition. Therefore, this embodiment of the invention crops the image to remove the user's facial area, thereby improving the accuracy of recognition.

[0194] In the above embodiment, the processing unit adjusts the preset width based on the eye position information, that is, adjusts the preset width of the first frame image in the first frame image and the second frame image in the second frame image. Changing the preset width changes the number of the first frame image and the second frame image. (Refer to...) Figure 24The purpose of adjusting the preset width is to ensure that, under different viewing distances, the first parallax image is projected onto the left eye of the subject in the first direction, and the third parallax image is projected onto the right eye of the subject in the second direction.

[0195] In one embodiment, reference is made to Figure 25 When the processing unit obtains the eye position information of the person based on the facial image parsing, it performs the following steps:

[0196] Step S2510: Detect the eye region of the face image based on a preset detector to obtain the eye socket position information.

[0197] Step S2520: Convert the facial image into a grayscale image and perform binarization on the grayscale image to obtain the first preprocessed image.

[0198] Step S2530: Perform erosion and dilation processing on the first preprocessed image and remove noise from the image to obtain the second preprocessed image. Use circular structuring elements to extract the position of the circular region representing the eyeball of the human subject in the second preprocessed image to obtain the eyeball position information of the human subject.

[0199] In the above embodiments, the orbital position information refers to the position of the user's orbit. The orbit refers to the frame formed by the edges of the eyelids on the face; it is a quadrangular pyramid-shaped bony cavity that houses the eyeball and other tissues. There is one orbit on each side, symmetrical to each other, and the depth of an adult orbit is approximately 4 to 5 cm. It is understood that in the embodiments of the present invention, the orbital position information of both eyes of the user can be obtained, or only the orbital position information of one eye can be obtained, and the orbital position information of the other eye can be obtained based on prior knowledge. This embodiment does not impose specific limitations on this.

[0200] It should be noted that in this embodiment of the invention, facial image recognition is achieved through image processing to obtain the required eye socket position information. Specifically, this embodiment of the invention performs face eye region detection on the facial image based on a preset detector to obtain eye socket position information, converts the facial image into a grayscale image, and performs binarization processing on the grayscale image to obtain a first preprocessed image. The first preprocessed image is further subjected to erosion and dilation processing, and noise in the image is removed to obtain a second preprocessed image. Since the human eyeball is round, the position of the circular region representing the user's eyeball in the second preprocessed image is extracted using circular structuring elements to obtain the user's eyeball position information.

[0201] In one embodiment, the present invention processes facial images based on OpenCV and uses a cascaded classifier as a detector for face recognition. The cascaded classifier is based on Local Binary Pattern (LBP) features and Haar-like features. The LBP and HAAR features are used to implement classifier data trained on specific targets. This data can be saved, loaded, and effectively used for object recognition. Each pixel in the image can be encoded by the LBP operator. After extracting the original LBP operator from the image, the original LBP features are still the image. Therefore, this embodiment of the invention is based on a trained LBP feature cascade detector. When in use, it calls the relevant face detection cascade analyzer data. After cropping the face region, it takes the upper half and then divides the upper half into two equal parts, namely the left and right parts of the eyes. Then, it crops the eye region according to the proportion of the upper half occupied by the eyes, thus completing the selection and labeling of the eye region. In addition, it also uses the eye cascade detector in OpenCV to realize eye detection. The detected eye object sub-image is cached as a template so that when the detector cannot detect the eye region, the template image prepared above can be used to complete the matching of the eye region. When performing eye localization, the labeled eye region is binarized to obtain the outline of the eye and thus the orbital position information. To determine the position of the eyeballs, we can use a circular structuring element to perform an opening operation (erosion followed by dilation) on the image. At this point, noise still exists in the central circular region, which needs to be removed first. Then, we can extract the eyeball position using the circular structuring element to obtain the eyeball position information. For example, the center of the circular structure can be used as the eyeball position, thus providing the eyeball position information.

[0202] It is understood that the use of OpenCV to process facial images to obtain eye socket position information in this embodiment of the invention does not constitute a limitation on this embodiment.

[0203] For example, by combining the above methods, the effect of 3D display at different distances and / or angles can be determined through multiple measurements, and a preset width adjustment relationship table based on different angles and / or distances can be established. In actual use, the preset width that needs to be adjusted can be obtained by directly querying the relationship table.

[0204] As described above, the three-dimensional display system of this application embodiment includes: a camera, a shooting lens, a processing unit, and a three-dimensional display module; a prism in the shooting lens is placed above the camera to divide the lens surface of the camera into a first region and a second region; a first initial light signal of the shooting target arrives at the first region to form a first parallax image in the first region, and a second initial light signal of the shooting target enters the second region after passing through the prism to form a second parallax image; the processing unit performs image processing on the second parallax image according to the first parallax image to obtain a third parallax image, and generates a 3D frame image using the first parallax image and the third parallax image; the display unit receives and displays the 3D frame image, and an optical conversion device is disposed on the display plane of the display unit to display the 3D frame image in three dimensions.

[0205] In some embodiments, since the camera lens can be mounted on an electronic device, the first camera, the second camera, the processing unit, and the display unit can be functional components of the same electronic device. Therefore, the processing unit can also obtain the target installation error of the camera lens in history; and determine the cropping parameters according to the magnitude of the target installation error, wherein the cropping parameters are used to crop the excess parts in the third parallax image or the first parallax image.

[0206] In some embodiments, obtaining the target installation error of historically installed shooting lenses includes: obtaining device model information of electronic devices; obtaining multiple sample installation errors, wherein the sample installation errors are the installation errors of multiple sample devices when installing shooting lenses under the device model information; calculating the variance of the multiple sample installation errors; if the variance is greater than a preset variance threshold, selecting the maximum error value from the multiple sample installation errors and using the maximum error value as the target installation error of installing shooting lenses; if the variance is less than or equal to the preset variance threshold, calculating the average error value of the multiple sample installation errors and using the average error value as the target installation error of installing shooting lenses.

[0207] For example, in this application embodiment, the cropping of the third parallax image or the second parallax image can be achieved by determining preset cropping parameters. Specifically, in this application embodiment, the cropping parameters can be obtained by the installation error during the installation of the shooting lens.

[0208] For example, in this application embodiment, the target installation error of the camera lens can be obtained historically. The target installation error is the error of the corresponding camera when installing the camera lens, such as the installation error of a certain target camera when connecting the camera lens. It can be understood that the installation error can be a specific parameter, such as displacement difference, indicating how far the camera lens will offset from the camera after installation.

[0209] For example, after obtaining the target installation error, the processing unit in this embodiment can determine the cropping parameters based on the magnitude of the target installation error. Different target installation errors can yield corresponding cropping parameters. These cropping parameters are used to crop the excess portion of the first or third disparity image. There are various types of cropping parameters. For example, cropping parameters can include horizontal cropping parameters, which characterize how much image needs to be cropped in the horizontal direction, and vertical cropping parameters, which characterize how much image needs to be cropped in the vertical direction.

[0210] The steps for obtaining multiple sample installation errors can be as follows: obtain the device model information of the electronic device; obtain multiple sample installation errors, where the sample installation error is the installation error of multiple sample devices when installing the shooting lens under the device model information; calculate the variance of the multiple sample installation errors; if the variance is greater than a preset variance threshold, select the maximum error value from the multiple sample installation errors and use the maximum error value as the target installation error for installing the shooting lens; if the variance is less than or equal to the preset variance threshold, calculate the average error value of the multiple sample installation errors and use the average error value as the target installation error for installing the shooting lens.

[0211] For example, in this embodiment of the application, the target installation error can be determined based on the device model of different electronic devices. First, the device model information of the electronic device can be obtained. This device model information represents the device model of the current electronic device. If the electronic device is a mobile phone, the device model information is the specific mobile phone model information, or it can represent the brand to which the device model belongs.

[0212] In practical applications, the applicant found that the stability of installation errors is related to the device model. Since the camera lens is generally detachably connected to the device and operated by the user, installation errors are difficult to control. Taking a mobile phone as an example, for a certain model A, the camera lens designed for this model has good fit and stability with the phone's camera, resulting in relatively stable installation errors after the user installs the lens. For another model B, the camera lens designed for this model has poor fit and stability with the phone's camera, leading to generally unstable installation errors after the user installs the lens. Therefore, this application's embodiments design an adaptive algorithm to match different cropping parameters according to different device models, aiming to minimize the image cropping range while ensuring image quality, thereby maximizing the 3D imaging effect.

[0213] In this embodiment, multiple sample installation errors can be pre-stored. These sample installation errors refer to the installation errors when installing a camera lens on multiple sample devices with the same device model information. It is understood that this embodiment can establish a sample database to provide data support for obtaining the target installation error. The sample database stores the installation errors of multiple electronic devices of different models when connecting to a camera lens, and there are also multiple installation errors for electronic devices of the same model when connecting to a camera lens. Therefore, in this embodiment, after obtaining the device model information of the current electronic device, it will be matched with the sample database to find multiple sample installation errors for that device model.

[0214] For example, in this embodiment, the error of the electronic device under the current device model is processed. Specifically, this includes variance processing. By calculating the variance of the installation errors of multiple samples, the dispersion of the error under this device model can be obtained. In this embodiment, a preset variance threshold is set, and the variance threshold is used to determine whether the error dispersion exceeds a preset range.

[0215] Specifically, if the variance exceeds a preset variance threshold, it indicates that the electronic device of that model is unstable when connected to the shooting lens, resulting in large error variations. Therefore, this embodiment selects the maximum error value from multiple sample installation errors and uses it as the target installation error for the shooting lens. It is important to understand that by setting the maximum error value as the target installation error, it ensures that even with large variations in installation error, cropping can be used to ensure that the cropped third parallax image remains essentially consistent with the first parallax image, thereby improving the image quality of the 3D frame.

[0216] Specifically, if the variance is less than or equal to a preset variance threshold, it indicates that the electronic device of that model is installed relatively stably when connected to the shooting lens, resulting in small error variations. Therefore, this embodiment calculates the average error value of multiple sample installation errors and uses the average error value as the target installation error for installing the shooting lens. It should be understood that by setting the average error value as the target installation error, it is ensured that with relatively stable installation and small error variations, a larger 3D image can be guaranteed through cropping. The cropped third parallax image is also basically consistent with the first parallax image, thereby improving the image quality and image quality of the 3D frame image.

[0217] In this embodiment, the 3D display system splits the light signal of the target into two paths through the camera lens, forming a first parallax image and a second parallax image of the target. The second parallax image is then processed by the processing unit to obtain a third parallax image that matches the first parallax image. The first and third parallax images are then fused to obtain a 3D frame image. Finally, the 3D frame image is projected onto the field of view of the subject using an optical conversion device mounted on the surface of the display unit. The use of an external camera lens and optical conversion device enables a single device to achieve real-time naked-eye 3D display, reducing the system complexity of the 3D display system.

[0218] In addition, this application also provides a three-dimensional display method.

[0219] Figure 26 This is an optional flowchart of the three-dimensional display method provided in the embodiments of the present invention. Figure 26 The method may include, but is not limited to, steps S2610 to S2640. It is also understood that this embodiment... Figure 26 The order of steps S2610 to S2640 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0220] Step S2610: Acquire a first parallax image formed by the first initial light signal of the target arriving in the first region, and acquire a second parallax image formed by the second initial light signal of the target entering the second region after passing through the prism.

[0221] Step S2620: Perform mirror image processing on the second disparity image based on the first disparity image to obtain a third disparity image, wherein the third disparity image has disparity with the first disparity image.

[0222] Step S2630: Obtain 3D frame images using the first parallax image and the third parallax image.

[0223] Step S2640: Use optical conversion devices to display the 3D frame image in three dimensions.

[0224] The specific implementation method of the three-dimensional display method in this embodiment is basically the same as the specific implementation method of the three-dimensional display system described above, and will not be repeated here.

[0225] This invention also provides an electronic device, comprising:

[0226] At least one memory;

[0227] At least one processor;

[0228] At least one program;

[0229] The program is stored in a memory, and the processor executes the at least one program to implement the three-dimensional display method described above. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0230] Please see Figure 27 , Figure 27 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0231] The processor 2701 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0232] The memory 2702 can be implemented in the form of ROM (Read-Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 2702 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 2702 and is called and executed by the processor 2701 to execute the three-dimensional display method of the embodiments of this invention.

[0233] The 2703 input / output interface is used to implement information input and output.

[0234] The communication interface 2704 is used to enable communication and interaction between this device and other devices. Communication can be achieved via wired means (e.g., USB, Ethernet cable) or wireless means (e.g., mobile network, Wi-Fi, Bluetooth).

[0235] Bus 2705 transmits information between various components of the device (e.g., processor 2701, memory 2702, input / output interface 2703, and communication interface 2704);

[0236] The processor 2701, memory 2702, input / output interface 2703 and communication interface 2704 are connected to each other within the device via bus 2705.

[0237] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described three-dimensional display method.

[0238] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0239] The three-dimensional display system, three-dimensional display method, electronic device, and storage medium proposed in this invention embodiment include: a camera, a shooting lens, a processing unit, and a three-dimensional display module; a prism in the shooting lens is placed above the camera, dividing the lens surface of the camera into a first region and a second region; a first initial light signal of the target arrives at the first region and forms a first parallax image in the first region; a second initial light signal of the target passes through the prism and enters the second region to form a second parallax image; the processing unit performs image processing on the second parallax image based on the first parallax image to obtain a third parallax image, and generates a 3D frame image using the first parallax image and the third parallax image; the display unit receives and displays the 3D frame image, and an optical conversion device is disposed on the display plane of the display unit to display the 3D frame image in three dimensions. In this embodiment, the 3D display system splits the light signal of the target into two paths through the camera lens, forming a first parallax image and a second parallax image of the target. The second parallax image is then processed by the processing unit to obtain a third parallax image that matches the first parallax image. The first and third parallax images are then fused to obtain a 3D frame image. Finally, the 3D frame image is projected onto the field of view of the subject using an optical conversion device mounted on the surface of the display unit. The use of an external camera lens and optical conversion device enables a single device to achieve real-time naked-eye 3D display, reducing the system complexity of the 3D display system.

[0240] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0241] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0242] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0243] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0244] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0245] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0246] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0247] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0248] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0249] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0250] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A three-dimensional display system, characterized in that, include: The first camera, the capturing lens, the processing unit, and the 3D display module; The shooting lens includes a prism, which is placed above the first camera and divides the lens surface of the first camera into a first region and a second region. This allows a first initial light signal of the shooting target to reach the first region and form a first parallax image in the first region. The second initial light signal of the shooting target passes through the prism and enters the second region to form a second parallax image. The second parallax image has a parallax with the first parallax image. The processing unit is electrically connected to the first camera and is used to receive the first parallax image and the second parallax image, and to perform image processing on the second parallax image based on the first parallax image to obtain a third parallax image; the processing unit is also used to generate 3D frame images using the first parallax image and the third parallax image. The three-dimensional display module includes a display unit and an optical conversion device. The optical conversion device is disposed on the display plane of the display unit and is used to convert the light beam emitted from the display unit into a left eye light beam emitted to the left eye of the subject and a right eye light beam emitted to the right eye of the subject to achieve three-dimensional display. The display unit is electrically connected to the processor, and the display unit is used to receive and display the 3D frame image so that the displayed 3D frame image is displayed in three dimensions after passing through the optical conversion device; The camera lens includes: a housing, which is a columnar structure with a first opening at the top and a second opening at the bottom, the second opening being close to the plane of a first camera to accommodate the first camera; the first camera plane is the plane containing the lens surface of the first camera; a prism, which is disposed above the first camera; the prism is fitted to one side of the interior of the housing, such that a first cavity is formed between the inner wall of the other side of the housing and the prism, allowing the first initial light signal of the target to pass through the first opening, through the first cavity, and into the first camera, and the second initial light signal of the target to pass through the first opening, through the prism, and into the first camera; the prism includes: a first prism surface, and an incident light surface and an exit light surface sequentially arranged along the light path of the second initial light signal; the first prism... The first prism surface is perpendicular to the plane of the first camera. The extended surface of the first prism surface divides the lens surface of the first camera into a first region and a second region. The first region is used to receive the first initial light signal, and the second region is used to receive the light signal emitted from the light-emitting surface, so that the target forms a first parallax image in the first region and a second parallax image corresponding to the first parallax image in the second region. The prism also includes a reflective surface. The angle between the incident surface and the reflective surface forms a first angle, and the angle between the first prism surface and the light-emitting surface forms a second angle. The first angle causes the second initial light signal to enter the prism from the incident surface to obtain a first light signal, the first light signal to be reflected by the reflective surface to obtain a second light signal, the second light signal to be emitted from the light-emitting surface to obtain a target light signal, and the second angle causes the target light signal to reach the second region of the lens surface. Let the longest line segment perpendicular to the boundary line between the lens surface of the first camera and the first region and the second region be the first line segment, and let the first line segment form a first endpoint at the edge of the first region; the housing further includes: a first housing wall with a first height disposed opposite to the prism, the first housing wall including a first housing edge; the first endpoint and the first housing edge constitute a first boundary surface of the effective shooting range of the first camera; the extended surface of the first prism surface constitutes a second boundary surface of the effective shooting range of the first camera; the first boundary surface and the second boundary surface define the effective shooting range of the first camera, so that the first initial beam of the shooting target within the effective shooting range enters the first region; the second initial beam of the shooting target within the effective shooting range enters the second region through the prism.

2. The three-dimensional display system according to claim 1, characterized in that, When the processing unit performs image processing on the second disparity image based on the first disparity image to obtain a third disparity image, it executes the following steps: If the second disparity image is a mirror image of the first disparity image, perform mirror restoration processing on the second disparity image to obtain a third disparity image that matches the first disparity image; Alternatively, if the second disparity image is a rotated image of the first disparity image, the second disparity image is rotated and restored to obtain a third disparity image that matches the first disparity image. Alternatively, if the second disparity image is a scaled-down image of the first disparity image, the second disparity image is mirrored and magnified to obtain a third disparity image that matches the first disparity image. Alternatively, if the second disparity image is a magnified version of the first disparity image, the second disparity image is mirrored and scaled down to obtain a third disparity image that matches the first disparity image.

3. The three-dimensional display system according to claim 1, characterized in that, When the processing unit generates a 3D frame image using the first parallax image and the third parallax image, it performs the following steps: The third parallax image is processed into a first frame image in 3D format, and the first parallax image is processed into a second frame image in 3D format; the first frame image and the second frame image are left-right format images of each other; The first frame image and the second frame image are merged and stitched together to obtain the 3D frame image in 3D format.

4. The three-dimensional display system according to claim 3, characterized in that, The step of processing the third parallax image into a first frame image in 3D format includes: The transformed and restored third disparity image is compared with the first disparity image to obtain a comparison result characterizing the degree of overlap between the images; Based on the comparison results, the third disparity image is calibrated to obtain a calibrated fourth disparity image, so that the overlap between the calibrated fourth disparity image and the first disparity image meets the preset requirements. The calibrated fourth parallax image is processed into the first frame image in 3D format.

5. The three-dimensional display system according to claim 4, characterized in that, The step of comparing the transformed and restored third disparity image with the second disparity image to obtain a comparison result characterizing the degree of overlap between the images includes: Identify target features in the third parallax image to obtain the first target corner point of the target feature in the third parallax image; Identify the target features in the first parallax image to obtain the second target corner point of the target features in the first parallax image; The third disparity image and the first disparity image are overlaid, and the first target corner point and the second target corner point are virtually overlapped for comparison. The comparison result representing the overlap between the images is obtained based on the overlap between the corner points.

6. The three-dimensional display system according to claim 3, characterized in that, The step of processing the third parallax image into a first frame image in 3D format includes: The redundant portion in the third disparity image compared to the first disparity image is determined to obtain the first redundant portion; The first redundant part is cropped out from the third parallax image, and the remaining third parallax image is processed into the first frame image in 3D format; Alternatively, processing the first parallax image into a second frame image in 3D format includes: The redundant portion in the first parallax image that is different from the third parallax image is determined to obtain the second redundant portion; The second redundant portion is cropped from the first parallax image, and the remaining second parallax image is processed into a second frame image in 3D format.

7. The three-dimensional display system according to claim 3, characterized in that, The first frame image includes a plurality of first frame images of preset width arranged in a first arrangement order; the second frame image includes a plurality of second frame images of the preset width arranged in a second arrangement order. The step of fusing and stitching the first frame image with the second frame image to obtain a 3D frame image in 3D format includes: According to the first arrangement order and the second arrangement order, the first frame image and the second frame image are alternately arranged to obtain the 3D frame image.

8. The three-dimensional display system according to claim 1, characterized in that, The intersection of the first line segment with the boundary line of the first region and the second region is the second endpoint, and the boundary line is the projection line of the first prism surface on the lens surface; The shooting target includes a first shooting boundary point and a second shooting boundary point; the first initial light signal includes a first boundary light signal emitted from the first shooting boundary point and a second boundary light signal emitted from the second shooting boundary point; the second initial light signal includes a third boundary light signal emitted from the first shooting boundary point and a fourth boundary light signal emitted from the second shooting boundary point. The first boundary light signal passes through the first cavity to reach the first endpoint, and the second boundary light signal passes through the first cavity to reach the second endpoint, so as to form the first parallax image in the first region; The third boundary light signal enters the prism from the light-incident surface, passes through the reflective surface and the light-exiting surface in sequence, and reaches the first arrival point in the second region; the fourth boundary light signal enters the prism from the light-incident surface, passes through the reflective surface and the light-exiting surface in sequence, and reaches the second arrival point in the second region; the first arrival point is located at the mirror position or the first adjacent mirror position of the first endpoint, and the second arrival point is located at the mirror position or the second adjacent mirror position of the second endpoint, so as to form a second parallax image in the second region that is mirrored or mirror-scaled with the first parallax image.

9. The three-dimensional display system according to claim 8, characterized in that, The first line segment forms a third endpoint at the edge of the second region, the position of the first arrival point is the third endpoint, and the position of the second arrival point is the second endpoint, so as to form a first parallax image in the first region and a second parallax image that is a mirror image of the first parallax image in the second region.

10. The three-dimensional display system according to claim 7, characterized in that, The system also includes a second camera, which is connected to the processor; The second camera is used to acquire facial images of a person and send the facial images to the processing unit; The processing unit is used to obtain the eyeball position information of the person object based on the facial image, and the eyeball position information is used to characterize the position of the person object's eyeballs. The processing unit is further configured to adjust the preset width according to the eye position information, so that the first parallax image is emitted to the left eye of the subject in a first direction, and the third parallax image is emitted to the right eye of the subject in a second direction.

11. The three-dimensional display system according to claim 10, characterized in that, The processing unit performs the following steps when it obtains the eye position information of a person based on the facial image: The facial image is used to detect the eye region based on a preset detector to obtain the eye socket position information; The facial image is converted into a grayscale image, and the grayscale image is binarized to obtain a first preprocessed image; The first preprocessed image is subjected to erosion and dilation processing, and noise in the image is removed to obtain a second preprocessed image. The position of the circular region representing the eyeball of the human subject is extracted in the second preprocessed image using a circular structuring element to obtain the eyeball position information of the human subject.

12. The three-dimensional display system according to any one of claims 1 to 11, characterized in that, The optical conversion device is a three-dimensional display conversion film, which is a slit-type liquid crystal grating film, a columnar lens film, or a light source pointing film.

13. A three-dimensional display method, characterized in that, Applied to a three-dimensional display system as described in any one of claims 1 to 12, the method comprises: Acquire the first parallax image formed by the first initial light signal of the shooting target arriving in the first region, and acquire the second parallax image formed by the second initial light signal of the shooting target entering the second region after passing through the prism; The second disparity image is mirrored based on the first disparity image to obtain a third disparity image; the third disparity image has a disparity with the first disparity image. A 3D frame image is obtained using the first parallax image and the third parallax image; The optical conversion device is used to display the 3D frame image in three dimensions.

14. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the three-dimensional display method of claim 13.

15. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the three-dimensional display method of claim 13.

Images