A 3D picture processing method and device, a storage medium and an electronic device
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- YIBIN XGIMI OPTOELECTRONIC CO LTD
- Filing Date
- 2022-01-25
- Publication Date
- 2026-07-10
AI Technical Summary
In existing technologies, 3D video processing requires processing the left and right eye views separately, which involves a huge workload and is difficult to simplify efficiently.
The 2D video frame is divided into multiple layers of images, and the left and right eye views are shifted by the lens offset and rotation of the projection device, which simplifies the processing flow and only requires processing a single frame.
By using hardware to create left and right eye view offsets, the 3D video processing process is simplified and processing efficiency is improved.
Smart Images
Figure CN116567440B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video, and more specifically, to a 3D image processing method, apparatus, storage medium, and electronic device. Background Technology
[0002] 3D video typically refers to a video created by combining the left-eye view and the right-eye view. When a 3D video is presented, the viewer watches the displayed image (such as a projected image). The images from the left-eye view and the right-eye view are filtered and fed into the viewer's left and right eyes respectively. Because of the distance difference between the left and right eyes, the human senses misjudge the distance, thus creating a 3D effect.
[0003] It should be understood that 3D videos contain numerous sub-images corresponding to the left and right eye views. In related technologies, the left and right eye views need to be processed separately, which is a huge workload. Overcoming these problems has become a challenging issue of concern to those skilled in the art. Summary of the Invention
[0004] The purpose of this application is to provide a 3D image processing method, apparatus, storage medium, and electronic device to at least partially improve the above-mentioned problems.
[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:
[0006] In a first aspect, embodiments of this application provide a 3D image processing method, the method comprising:
[0007] Divide the 2D video frames into N layers, where N is greater than or equal to 2, according to the preset number of layers required.
[0008] The N layer images are arranged and combined to obtain the 3D image set corresponding to the 2D video frame. The 3D image set includes M sub-images, where M is greater than or equal to 4N-2. The total playback time of the at least M sub-images on the projection device is equal to the preset period. The i-th layer image corresponds to the i-th and M / 2+2-i-th sub-images respectively. The 3D image set is used for playback on the projection device. The lens offset angle of the projection device is a preset offset. The lens of the projection device rotates according to the preset period.
[0009] Secondly, embodiments of this application provide a 3D image processing apparatus, the apparatus comprising:
[0010] The processing unit is used to divide the 2D video frame into N layers according to the preset number of layers required, so as to obtain N layer images, where N is greater than or equal to 2.
[0011] An arrangement unit is used to arrange and combine N layer images to obtain a 3D image set corresponding to the 2D video frame. The 3D set includes M sub-images, where M is greater than or equal to 4N-2. The total playback time of the at least M sub-images on the projection device is equal to the preset period. The i-th layer image corresponds to the i-th and M / 2+2-i-th sub-images respectively. The 3D image set is used for playback on the projection device. The lens offset angle of the projection device is a preset offset angle, and the lens of the projection device rotates according to a preset period.
[0012] Thirdly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method.
[0013] Fourthly, embodiments of this application provide an electronic device, the electronic device comprising: a processor and a memory, the memory being used to store one or more programs; when the one or more programs are executed by the processor, the above-described method is implemented.
[0014] Compared to existing technologies, the 3D image processing method, apparatus, storage medium, and electronic device provided in this application divide a 2D video frame into N layer images (N greater than or equal to 2) by a preset number of layers. These N layer images are then arranged and combined to obtain a 3D image set corresponding to the 2D video frame. The 3D image set includes M sub-images (M greater than or equal to 4N-2), and the total playback time of at least M sub-images on the projection device is equal to a preset period. The i-th layer image corresponds to the i-th and M / 2+2-i-th sub-images. The 3D image set is used for playback on the projection device, where the lens offset angle is a preset offset angle, and the lens rotates according to a preset period. Because the lens of the projection device rotates while maintaining the preset offset angle, the left and right eye views are offset through hardware. Therefore, in the 3D image processing process, only one image needs to be processed for each single frame, simplifying the 3D video processing process and improving the efficiency of 3D image processing.
[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 A cross-sectional structural schematic diagram of the rotating optical path adjustment device 100 provided in the embodiments of this application;
[0018] Figure 2 This is a schematic diagram of the light trajectory provided in an embodiment of this application;
[0019] Figure 3 This is one of the schematic diagrams of light trajectories provided in the embodiments of this application;
[0020] Figure 4 A schematic diagram illustrating the viewing distance provided in an embodiment of this application;
[0021] Figure 5 A time division diagram provided for an embodiment of this application;
[0022] Figure 6 This is a schematic diagram illustrating the relationship between time segments and layer images provided in the embodiments of this application;
[0023] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;
[0024] Figure 8 A flowchart illustrating the 3D image processing method provided in an embodiment of this application;
[0025] Figure 9 A schematic diagram of permutation and combination relationships provided for embodiments of this application;
[0026] Figure 10 A schematic diagram of the sub-steps of S102 provided in the embodiments of this application;
[0027] Figure 11 One of the flowcharts of the 3D image processing method provided in the embodiments of this application;
[0028] Figure 12 One of the flowcharts of the 3D image processing method provided in the embodiments of this application;
[0029] Figure 13 This is a schematic diagram of a 3D image processing device provided in an embodiment of this application.
[0030] In the diagram: 10-Processor; 11-Memory; 12-Bus; 13-Communication interface; 100-Rotating optical path adjustment device; 110-Housing; 120-Mounting cylinder; 210-Light beam; 140-Optical element; 401-Processing unit; 402-Arrangement unit. Detailed Implementation
[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0032] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0033] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0034] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0035] In the description of this application, it should be noted that the terms "upper", "lower", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of this application is usually placed in. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.
[0036] In the description of this application, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.
[0037] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0038] This application provides a projection device, which includes, as described above... Figure 1 The rotating optical path adjustment device 100 shown is shown. Figure 1 This is a cross-sectional structural schematic diagram of the rotating optical path adjustment device 100 provided in this embodiment. Please refer to the attached diagram. Figure 1 This embodiment provides a rotating optical path adjustment device 100, and correspondingly, a projection device is provided.
[0039] It should be understood that projection devices also include projection components, such as lenses, to achieve projection.
[0040] The rotating optical path adjustment device 100 includes a housing 110, a mounting cylinder 120, and an optical element 140. The mounting cylinder 120 is installed inside the housing 110 and is rotatable relative to the housing 110 around its own axis. The optical element 140 can be a lens. The optical element 140 is installed inside the mounting cylinder 120 and is radially tilted relative to the mounting cylinder 120. This allows the optical element 140 to rotate 360° relative to the housing 110 with the mounting cylinder 120. During this rotation, the tilt position of the optical element 140 changes, causing the exit position of the light ray 210 incident along the axial direction of the mounting cylinder 120 to change after passing through the optical element 140. Because the optical element 140 rotates 360°, the image formed by the light ray 210 passing through the optical element 140 will continuously move along a circle with a radius equal to the distance the light ray 210 deviates from its path (e.g., ...). Figure 2 As shown in the diagram, the pixel grid is thus infinitely subdivided, thereby reducing the pixelation and improving the visual experience. Furthermore, since this optical path adjustment device is rotary, compared to the existing technology that uses a chain to repeatedly rotate it at a certain angle, it avoids fatigue damage and failure, helping to adapt to the ever-increasing demands of frame rates.
[0041] It should be understood that the image formed by the light 210 passing through the optical element 140 will move continuously along a circle with the radius of the distance of the light 210 deviating from its position. When the light 210 moves to different positions, the distance of its projection point relative to the viewer is different.
[0042] Please refer to Figure 3 Because the lens is rotating and affected by its refractive index n, thickness T, and offset angle α, the offset light rays rotate with an offset distance R as the radius, forming a circle with a diameter of 2R. Through timing control, the content of the same layer is displayed when the light rays are at either end of the diameter offset, combined with the visual persistence effect of the human eye. This allows the viewer to see overlapping images with an offset distance of 2R. Once these overlapping images with a distance of 2R are obtained, by using certain methods (such as color filtering, polarized light filtering, and frequency filtering) to send the two overlapping images to the viewer's left and right eyes respectively, the viewer can misjudge the image distance. This visually creates the illusion of different layers of depth.
[0043] like Figure 4 As shown, there is a certain distance between the left and right eyes of a movie viewer. When the information received by the left and right eyes is image 1 and image 2 respectively, the perceived distance between the viewer and the image is X + the distance to the wall. The calculation of X is a standard trigonometric function calculation, which will not be elaborated here. Based on the above derivation process, a mapping relationship between the depth of field X and the lens tilt angle α can be established. The lens tilt angle α is determined according to the actual required value of X.
[0044] In one possible implementation, the projection device also includes an encoder used to capture the optical path deflection angle. It should be understood that the encoder operates on the same principle as the encoder used in servo motors, and is divided into electromagnetic encoders and optical encoders. An optical encoder has a circular grating that rotates with the motor shaft that drives the mounting cylinder 120. The angle of the motor in one revolution is determined by accumulating the number of gratings. An electromagnetic encoder determines the angle of the motor in one revolution by using the pressure difference generated by the magnetic material when displacement and angular changes occur. Identifying the rotation angle helps to subsequently display different content as needed within a single frame.
[0045] Furthermore, by adjusting the rotation speed of the optical path adjustment device 100, the playback time of one frame is made the same as the time it takes for the optical path adjustment device 100 to rotate one revolution. For example... Figure 5 As shown, the time it takes for the light to deflect one full circle with radius R is the playback time of one frame. Therefore, the playback time of one frame can be further subdivided. Different content is displayed at different deflection positions, resulting in a shift in the left and right views of each layer. For example... Figure 5As shown, different layer contents can be displayed when the light is at different deflection positions. For example, when the light is deflected to the far left and far right, the layer with the greatest depth of field is displayed. As the deflection position changes, layers with medium depth of field, shallow depth of field, and minimum depth of field can be displayed. Understandably, the depth of field of the layers with the greatest depth of field, medium depth of field, shallow depth of field, and minimum depth of field decreases in that order.
[0046] Please refer to Figure 6 , Figure 6 This is a schematic diagram of time division provided for an embodiment of this application. For example... Figure 6 As shown, the playback time of one frame (i.e., the time it takes for the rotating optical path adjustment device 100 to rotate one revolution) is divided into time periods A to L. Optionally, in time periods A and G, the DMD chip of the projection device only reflects the pattern of layer 1. The reflected pattern of the DMD chip is the same in time periods A and G, but after the rotation of the rotating optical path adjustment device 100, the two images in time periods A and G are actually projected with a left-right offset effect, thus creating a difference between the left-eye view and the right-eye view. Figure 6 As shown, layers 1, 2, and 3 are displayed during their respective time periods. For example, layer 2 is displayed during periods B and F, and its left and right offset is shorter than that of layer 1. The difference in left and right eye view offsets between layer 1 and layer 2 provides the necessary condition for viewers to mistakenly perceive that the content displayed in layers 1 and 2 is different from their actual distance after the projection. Similarly, layers 3, 4, etc., can also be displayed during other time periods.
[0047] It should be understood that in order to provide viewers with a 3D viewing experience, a single 2D frame needs to be divided into several sub-frames, requiring a higher display frequency from the DMD chip. For example, if a single 2D frame is divided into 12 sub-frames, and the original 2D video frame rate is 60 frames per second, the DMD chip needs to display 720 different frames within the same timeframe. The maximum flip frequency of a DMD chip can reach over 1000Hz, which meets this frequency requirement.
[0048] It should be understood that in order to achieve a 3D viewing effect, the original 2D video frames need to be split, and the split sub-frames need to be combined together and projected onto a projection device for playback, so that users can enjoy 3D viewing. How to quickly and efficiently acquire 3D images suitable for the projection device described in this application's embodiments has become a pressing problem to be solved.
[0049] This application provides an electronic device, which may be a computer device or a server device. Please refer to... Figure 7This is a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12. The processor 10 is used to execute executable modules, such as computer programs, stored in the memory 11.
[0050] Processor 10 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the 3D image processing method can be completed through integrated logic circuits in the hardware or software instructions within processor 10. The aforementioned processor 10 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0051] The memory 11 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.
[0052] Bus 12 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. Figure 7 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus 12 or one type of bus 12.
[0053] The memory 11 is used to store programs, such as programs corresponding to a 3D image processing device. The 3D image processing device includes at least one software functional module that can be stored in the memory 11 in the form of software or firmware or embedded in the operating system (OS) of the electronic device. After receiving an execution instruction, the processor 10 executes the program to implement the 3D image processing method.
[0054] Possibly, the electronic device provided in this application embodiment also includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus. The electronic device can interact with the projection device through the communication interface 13, thereby transmitting the processed 3D image to the projection device so that the projection device can project the 3D image.
[0055] It should be understood that, Figure 7 The structure shown is only a partial schematic diagram of the electronic device; the electronic device may also include components that are larger than... Figure 7 The more or fewer components shown, or having the same Figure 7 The different configurations shown. Figure 7 The components shown can be implemented using hardware, software, or a combination thereof.
[0056] The 3D image processing method provided in this application embodiment can be applied to, but is not limited to, [various applications]. Figure 7 For the specific process of the electronic devices shown, please refer to [link / reference]. Figure 8 The 3D image processing methods include S102 and S106, which are described in detail below.
[0057] S102, divide the 2D video frames into N layers according to the preset number of layers required, and obtain N layer images.
[0058] Where N is greater than or equal to 2.
[0059] It should be understood that each layer of the image corresponds to a different depth of field. Because a difference in the left and right eye view offsets of different layers is required to achieve a 3D viewing effect when the user observes the image, at least two layers are needed, such as Layer 1 and Layer 2 mentioned above, where the left and right eye view offsets of Layer 1 and Layer 2 create a difference.
[0060] S106, Arrange and combine N layer images to obtain a set of 3D images corresponding to 2D video frames.
[0061] The 3D set includes M sub-pictures, where M is greater than or equal to 4N-2. The total playback time of at least M sub-pictures on the projection device is equal to a preset period. The i-th layer image corresponds to the i-th and M / 2+2-i-th sub-pictures respectively. The 3D picture set is used for playback on the projection device. The lens offset angle of the projection device is a preset offset angle. The lens of the projection device rotates according to a preset period, where 1≤i≤N.
[0062] Please refer to Figure 9 , Figure 9 This is a schematic diagram of permutations and combinations provided in an embodiment of this application. If the preset period is divided into... Figure 6 The time periods A to L are shown, and time periods A to L are respectively related to... Figure 9The light path offset regions A to L are shown. It should be understood that the preset period is the time it takes for the lens to rotate one full circle. When N=3, this includes the first, second, and third layer images. It should be understood that the depth of field decreases sequentially from the first to the Nth layer image.
[0063] It should be noted that, Figure 9 The preset cycle division results and number of layers shown are for reference only and are not intended as limitations.
[0064] Please refer to Figure 6 In area A, the projected light rays deflect to the left by the maximum distance, and in area G, they deflect to the right by the maximum distance. It should be understood that the interval between the projected images in area A and area G is the largest, so areas A and G can be grouped as group 1, used to project the first layer image with the greatest depth of field. The interval between the projected images in area B and area F is smaller than the interval between the projected images in area A and area G, so areas B and F can be grouped as group 2, used to project the second layer image with a relatively deep depth of field. The interval between the projected images in area C and area E is smaller than the interval between the projected images in area B and area F, so areas C and E can be grouped as group 3, used to project the third layer image with the shallowest depth of field.
[0065] refer to Figure 6 and Figure 9 When the preset period is divided into 12 zones, A to L, because different sub-screens need to be projected in different zones, the sub-screen played in the x-th zone is the x-th sub-screen. This should be understood. Figure 6 and Figure 9 In the division results shown, M equals 12. Zone G is the 7th zone, and the sub-picture played in zone G is the 7th sub-picture, 12 / 2+2-1=7.
[0066] In one possible implementation, when playing the first sub-frame, the lens's projected light is shifted to the left or right by the maximum distance, that is, when playing the first sub-frame, it is in area A or area G of a preset period, thereby ensuring that the first sub-frame is the sub-frame with the maximum depth of field.
[0067] It should be understood that the i-th layer image corresponds to the i-th and M / 2+2-i-th sub-images, respectively. The i-th and M / 2+2-i-th areas belong to the same group; for example, areas A and G belong to group 1, areas B and F belong to group 2, and areas C and E belong to group 3. The same layer image is projected onto the same group (e.g., group 1, group 2, and group 3). It should be understood that the i-th and M / 2+2-i-th areas are symmetrical about the left and right sides of area D.
[0068] It should be understood that different layers correspond to different groups, and the superposition of all layers constitutes the 3D image. In one possible implementation, whether it's animation or video post-processing, the image will be divided into several layers. After arranging the depth-of-field order of the layers in the software, the video creator will group them according to the layer order, corresponding to the depth of field (in this example, the depth-of-field relationship is: Group 1 > Group 2 > Group 3). After arranging the groups, the layer content can then be arranged according to the corresponding time zone within each group. Following this logic, with existing video and animation production technology, 3D time-series images can be automatically arranged according to the layer order without manual arrangement.
[0069] It should be understood that conventional 3D video production requires processing two images, a left-eye view and a right-eye view. In this embodiment, because the projection device's lens rotates while maintaining a preset offset angle, the left and right-eye views are offset through hardware. Therefore, in the 3D image processing, only one image needs to be processed for each frame, which simplifies the 3D video processing process and improves the efficiency of 3D image processing.
[0070] In summary, this application provides a 3D image processing method. It divides a 2D video frame into N layer images (N > 2) based on a preset number of layers. These N layer images are then arranged to form a 3D image set corresponding to each 2D video frame. The 3D image set includes M sub-images (M > 4N - 2), and the total playback time of at least M sub-images on the projection device is equal to a preset period. The i-th layer image corresponds to the i-th and M / 2+2-i-th sub-images. The 3D image set is used for playback on the projection device. The projection device's lens offset angle is a preset offset angle, and the lens rotates according to a preset period. Because the projection device's lens rotates while maintaining the preset offset angle, the left and right eye views are offset through hardware. Therefore, in the 3D image processing process, only one image needs to be processed for each single frame, simplifying the 3D video processing process and improving the efficiency of 3D image processing.
[0071] exist Figure 8 Building upon the previous method, this application also provides a possible implementation for accurately obtaining N layer images in step S102. Please refer to [link / reference]. Figure 10 S102 includes S102-1 and S102-2, which are described in detail below.
[0072] S102-1, Determine the depth of field range corresponding to each layer based on the maximum depth of field information and the required number of layers.
[0073] Optionally, the maximum depth of field information can be set in advance according to user needs, or it can be determined based on the layer depth of the 2D video frame.
[0074] It should be understood that when determining the maximum depth of field information and the required number of layers (i.e., the specific value of N), the depth of field range corresponding to each layer can be determined.
[0075] S102-2, divide the 2D video frames according to the depth range corresponding to each layer to obtain N layer images.
[0076] It should be understood that after determining the depth range corresponding to each layer, the image content in the corresponding depth range of the 2D video frame is divided and extracted to obtain the corresponding layer image.
[0077] exist Figure 10 Based on this, regarding how to obtain maximum depth-of-field information, this application embodiment also provides a possible implementation method, please refer to... Figure 11 The 3D image processing method also includes: S101, which is described in detail below.
[0078] S101 identifies the layer depth of 2D video frames to obtain maximum depth information.
[0079] Optionally, depth information of each pixel in a 2D video frame can be obtained to filter out the maximum depth information.
[0080] In one possible implementation, regarding how to facilitate the projection device to project and play the acquired 3D image set, this application embodiment also provides a possible implementation method, please refer to... Figure 12 The 3D image processing methods also include: S103, S104, S105 and S106, which are described in detail below.
[0081] S103 determines the preset offset angle based on the maximum depth of field information.
[0082] It should be understood that to achieve the maximum depth of field viewing effect, the left and right offset of the first layer image needs to reach the target offset. The magnitude of the left and right offset is related to the offset angle. Understandably, the larger the offset angle, the greater the left and right offset between area A and area G. Therefore, the preset offset angle can be determined based on the maximum depth of field information.
[0083] It should be understood that as the lens rotates with the mounting tube 120 degrees, the position of the projected image on the projection wall changes accordingly. For example... Figure 4As shown, when a viewer's left and right eyes receive information from image 1 and image 2 respectively, the perceived distance between the viewer and the image is X + the distance to the wall. It should be understood that image 1 and image 2 are on the same image layer; for example, area A corresponds to image 1, and area G corresponds to image 2. It should also be understood that the distance from the viewer to the wall is fixed, and the magnitude of the perceived depth information is determined by X. (Reference) Figure 4 It can be seen that the value of X is closely related to the distance between screen 1 and screen 2; when the distance between screen 1 and screen 2 is large, the value of X is larger. It should be understood that the maximum depth of field minus the distance to the wall is the maximum value of X. After determining the maximum value of X, the interaction point for the left and right eyes can be determined. By drawing lines connecting the left and right eyes to the interaction point, the maximum distance between screen 1 and screen 2 can be determined. It should be understood that the maximum distance is twice the light ray offset distance R, and the light ray offset distance is closely related to the offset angle α. Therefore, after determining the maximum depth of field information, the offset angle α can be determined based on the distance to the wall, the distance between the viewer's left and right eyes, and the angle between the line connecting the viewer's left and right eyes and the projection wall.
[0084] It should be understood that after obtaining the preset offset angle, the preset offset angle can be transmitted to the projector so that the lens of the projector remains at the preset offset angle when playing the 3D image set.
[0085] S104, determine the total number of sub-screens M based on N.
[0086] It should be understood that the larger N is, the larger M is required, and the larger M is, the higher the requirements for the DMD chip. In order not to increase the burden on the DMD chip, the total number of sub-screens M needs to be adjusted adaptively according to the value of N.
[0087] S105, determine the projection frequency of the projection device based on M and the preset period.
[0088] It should be understood that the projection frequency is the frequency at which the projection device switches between projected sub-images. Optionally, the obtained projection frequency can be transmitted to the projection device so that the projection device switches between projected sub-images according to the projection frequency, and the projection device projects M sub-images sequentially within a preset period.
[0089] In one possible implementation, in order to enhance the richness of the content of each layer, this application embodiment also provides a possible implementation, please refer to the following.
[0090] When i is not equal to 1, the i-th layer image includes a first part and a second part. The first part of the i-th layer image corresponds to the i-th and M / 2+2-i-th sub-images, and the second part of the i-th layer image corresponds to the i+M / 2-th and M+2-i-th sub-images.
[0091] It should be understood that the first and second parts of an image on the same layer have the same depth information, and therefore require the same left and right offset during projection. For example... Figure 9 As shown, group 2 corresponds to areas B and F, and group 4 corresponds to areas H and L. (Refer to...) Figure 6 The left and right offset of the projected images in areas B and F is the same as that in areas H and L. Similarly, group 3 corresponds to areas C and E, and group 5 corresponds to areas I and K. The left and right offset of the projected images in areas C and E is the same as that in areas I and K.
[0092] It should be understood that the first and second parts of the same layer image can be projected at different times to enhance the user's viewing experience.
[0093] It should be noted that, Figure 6 and Figure 9 The sub-screens corresponding to areas D and J are not shown in the image. Areas D and J can be left blank or display other depth-of-field layer images, which is not limited here.
[0094] To achieve a 3D effect, it is necessary to intervene in the images received by the left and right eyes during projection. This can be done through methods such as color filtering, polarization valves, or frame rate differentiation, as detailed below.
[0095] Possibly, color filtering refers to a method where the user wears glasses with a red filter for the left eye and a blue filter for the right. The left and right eye views are respectively a red image and a blue image. The color filters remove the corresponding colors from the image, causing a difference in the images received by the left and right eyes.
[0096] Polarized light projection is likely the most common method for 3D movie projection. This method involves using two projectors to project images onto the same point. Each projector has polarizers that filter the light into horizontally polarized and vertically polarized light, respectively. The left and right eyes of the user's glasses are positioned opposite the horizontal and vertical polarizers, respectively. Therefore, the eye behind the horizontal polarizer can only perceive the horizontally polarized image, and the same applies to the vertical polarizer, resulting in a difference in the images received by the left and right eyes.
[0097] Frame rate differentiation is often used in displays. For example, in a 60-frame-per-second video, odd-numbered frames display the left-eye view, and even-numbered frames display the right-eye view. The user wears glasses that switch on and off at a certain frequency. The left eye receives the odd-numbered frames, and the right eye receives the even-numbered frames, resulting in a difference in the images received by the left and right eyes.
[0098] Please see Figure 13 , Figure 13 The present application provides a 3D image processing device, which is optionally applied to the electronic device described above.
[0099] The 3D image processing device includes a processing unit 401 and an arrangement unit 402.
[0100] The processing unit 401 is used to divide the 2D video frame into N layers according to the preset number of layers required, so as to obtain N layer images, where N is greater than or equal to 2.
[0101] Arrangement unit 402 is used to arrange and combine N layer images to obtain a set of 3D images corresponding to 2D video frames. The 3D set includes M sub-images, where M is greater than or equal to 4N-2. The total playback time of at least M sub-images on the projection device is equal to a preset period. The i-th layer image corresponds to the i-th and M / 2+2-i-th sub-images respectively. The 3D image set is used for playback on the projection device. The lens offset angle of the projection device is a preset offset angle, and the lens of the projection device rotates according to a preset period.
[0102] Optionally, the processing unit 401 is further configured to determine the depth range corresponding to each layer based on the maximum depth information and the required number of layers; and to divide the 2D video frames according to the depth range corresponding to each layer to obtain N layer images.
[0103] Optionally, the processing unit 401 may execute S101 to S105 as described above, and the arrangement unit 402 may execute S106 as described above.
[0104] It should be noted that the 3D image processing apparatus provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effects. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments.
[0105] This application also provides a storage medium storing computer instructions and programs, which, when read and executed, perform the 3D image processing method described above. The storage medium may include memory, flash memory, registers, or a combination thereof.
[0106] The following provides an electronic device, which may be a computer device or a server device, such as... Figure 7 As shown, the above-described 3D image processing method can be implemented. Specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 may be a CPU. The memory 11 is used to store one or more programs, and when one or more programs are executed by the processor 10, the 3D image processing method of the above embodiment is executed.
[0107] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0108] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0109] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0110] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
[0111] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this application. No reference numerals in the claims should be construed as limiting the scope of the claims.
Claims
1. A 3D image processing method, characterized in that, The method includes: Divide the 2D video frames into N layers, where N is greater than or equal to 2, according to the preset number of layers required. The N layer images are arranged and combined to obtain the 3D image set corresponding to the 2D video frame. The 3D image set includes M sub-images, where M is greater than or equal to 4N-2. The total playback time of the M sub-images on the projection device is equal to a preset period. The i-th layer image corresponds to the i-th and M / 2+2-i-th sub-images respectively. The 3D image set is used for playback on the projection device. The lens offset angle of the projection device is a preset offset angle. The lens of the projection device rotates according to a preset period. The step of dividing the 2D video frames into N layers to obtain N layer images includes: The depth of field range corresponding to each layer is determined based on the maximum depth of field information and the required number of layers. The 2D video frames are divided according to the depth of field range corresponding to each layer to obtain N layer images.
2. The 3D image processing method as described in claim 1, characterized in that, Before determining the depth-of-field range corresponding to each layer based on the maximum depth-of-field information and the required number of layers, the method further includes: The layer depth of the 2D video frame is identified to obtain the maximum depth of field information.
3. The 3D image processing method as described in claim 1, characterized in that, After dividing the 2D video frames according to the depth of field range corresponding to each layer to obtain N layer images, the method further includes: The preset offset angle is determined based on the maximum depth of field information, and the preset offset angle is used to adjust the lens offset angle of the projection device; The total number of sub-screens M is determined based on N; The projection frequency of the projection device is determined based on M and the preset period, where the projection frequency is the frequency at which the projection device switches sub-screens.
4. The 3D image processing method as described in claim 1, characterized in that, When i is not equal to 1, the i-th layer image includes a first part and a second part. The first part of the i-th layer image corresponds to the i-th and M / 2+2-i-th sub-pictures, and the second part of the i-th layer image corresponds to the i+M / 2-th and M+2-i-th sub-pictures.
5. The 3D image processing method as described in claim 1, characterized in that, When playing the first sub-picture, the maximum distance by which the projected light from the lens shifts to the left or right.
6. A 3D image processing device, characterized in that, The device includes: The processing unit is used to divide the 2D video frame into N layers according to the preset number of layers required, so as to obtain N layer images, where N is greater than or equal to 2. An arrangement unit is used to arrange and combine N layer images to obtain a 3D image set corresponding to the 2D video frame. The 3D image set includes M sub-images, where M is greater than or equal to 4N-2. The total playback time of the M sub-images on the projection device is equal to a preset period. The i-th layer image corresponds to the i-th and M / 2+2-i-th sub-images respectively. The 3D image set is used for playback on the projection device. The lens offset angle of the projection device is a preset offset angle, and the lens of the projection device rotates according to a preset period. The step of dividing the 2D video frame into N layer images by a preset number of layers includes: determining the depth range corresponding to each layer based on the maximum depth information and the required number of layers; and dividing the 2D video frame into N layer images based on the depth range corresponding to each layer.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1-5.
8. An electronic device, characterized in that, include: Processor and memory, the memory being used to store one or more programs; When the one or more programs are executed by the processor, the method as described in any one of claims 1-5 is implemented.