3D Stereoscopic Visual Perception Enhancement Optimization Processing Method Based on Digital Twin 3D Application

Abstract

The present invention provides a 3D stereoscopic visual sense enhancement and optimization processing method based on digital twin three-dimensional applications. A 3D stereoscopic plug-in is added to the digital twin application server. By emitting rays from the lens and colliding with objects, the depth of the objects is obtained, and the actual distance and screen occupancy ratio between the core object and the lens are judged. The rotation value (focus point) and the optimal spacing (interpupillary distance) of the dual lenses are automatically adjusted to enhance the stereoscopic effect and meet the comfortable visual sense experience of users. The images captured by the dual lenses in real time are rendered onto two high-definition dynamic texture maps; through the image pixel odd-even column interval arrangement algorithm, the two high-definition dynamic texture maps are fused into one image; the fused image is drawn to the top layer of the view through Widget; the application is started and browsed on a polarized display. On the basis of not affecting the original function interaction of the three-dimensional application, the experiencer can wear polarized glasses to see a clear and comfortable 3D stereoscopic effect.

Description

Technical Field

[0001] The present invention relates to the field of 3D vision technology, and particularly to a 3D stereoscopic visual sense enhancement optimization processing method based on digital twin three-dimensional applications, a 3D stereoscopic visual sense enhancement optimization processing device based on digital twin three-dimensional applications, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the construction of digital twin three-dimensional models, the traditional application methods for making 3D stereoscopic scene images mainly include the following several types:

[0003] I. Traditional three-dimensional stereoscopic technology (stereoscopic simulation technology)

[0004] Directly use a stereoscopic software to open a picture for wire drawing without image extraction.

[0005] Make a layered diagram of a planar diagram in Photoshop and then import it into a stereoscopic software for single-layer wire drawing.

[0006] This method is one of the most common technologies in the stereoscopic market. Because it is relatively simple and easy to train, it has been widely promoted in the training industry. However, whether a good stereoscopic effect can be produced still depends on the operator's understanding and skills.

[0007] II. Quasi-realistic 3D technology

[0008] This is an upgrade and improvement of the traditional three-dimensional stereoscopic technology, aiming to simplify the production process and improve the realistic effect and drawing efficiency of stereoscopic paintings.

[0009] III. Digital real-scene 3D technology

[0010] A technology further developed on the basis of the quasi-realistic 3D technology, including three core technologies: edge optimization technology, 3D data optimization technology, and grating calibration technology.

[0011] Edge optimization technology: Analyze and process a picture according to the actual picture effect of the picture, make the edge of each image smoothly transition in the depth direction, eliminate the flaky feeling, and make the picture contour more smooth, real, and natural.

[0012] 3D data optimization technology: On the basis of the edge optimization technology, according to the actual stereoscopic requirements of the picture, the local and overall stereoscopic data can be freely adjusted, or adjusted according to the actual proportion of the picture, so as to obtain a realistic stereoscopic effect.

[0013] Grating calibration technology: A technology for testing grating data, which aims to eliminate light waves by testing gratings. This technology can not only quickly test the data of qualified gratings, but also test the error range of gratings with poor straightness, laying a good foundation for the grating correction technology.

[0014] With the increasing demand for 3D applications and visual enhancement, ordinary 3D applications are difficult to meet the impact experience brought by the immersive visual feeling. The traditional application methods for making 3D stereoscopes often require high R & D costs, complex production processes, and there are many problems in the generated applications in terms of visual feeling, such as weak stereoscopic effect and poor picture quality, which are difficult to meet the requirements of users for immersive and comfortable visual experience.

[0015] Therefore, there is an urgent need for an application method that can automatically convert and optimize the 3D stereoscopic visual feeling of ordinary 3D applications to promote the technological progress and wide application in related fields. Summary of the Invention

[0016] In order to solve the technical problems existing in the prior art, the present invention provides the following technical solutions:

[0017] On the one hand, a 3D stereoscopic visual feeling enhancement and optimization processing method based on digital twin 3D applications is provided. This method is implemented by an electronic device and includes:

[0018] S1. Activate the 3D stereoscopic plug-in in the background of the 3D camera, and obtain the depth information of the target object through the dual-lens scene capture component;

[0019] S2. According to the depth information of the target object, calculate the target focus point and target rotation value of the dual-lens scene capture component according to the preset rotation value and interpupillary distance function of the lens, and control the dual-lens scene capture component to face the target object;

[0020] S3. Capture the corresponding scene image through the dual-lens scene capture component;

[0021] S4. Render the scene image captured by the dual-lens scene capture component onto two preset high-definition dynamic texture maps;

[0022] S5. Through the image pixel odd-even column interval arrangement algorithm, fuse the two high-definition dynamic texture maps into one image, and draw the fused image to the top layer of the view through Widget to obtain a 3D stereoscopic visual feeling enhanced image;

[0023] S6. Send the 3D stereoscopic visual feeling enhanced image to the 3D terminal.

[0024] Preferably, the depth information of the target object includes the actual distance between the target object and the lens and the screen occupancy ratio.

[0025] Preferably, the 3D terminal is polarized glasses.

[0026] Preferably, after obtaining the depth information of the target object, it further includes:

[0027] Calculate the actual distance of the target object from the lens based on the depth information of the target object, and calculate the scene depth of the target object based on the actual distance.

[0028] Control the lens axis of the dual-lens scene capture component to be mapped to the corresponding scene depth through the lens axis component.

[0029] On the other hand, a 3D stereoscopic visual sense enhancement optimization processing device based on digital twin three-dimensional applications is provided. This device is applied to the 3D stereoscopic visual sense enhancement optimization processing method based on digital twin three-dimensional applications. The device includes: a 3D stereoscopic plug-in and a 3D terminal. Among them, the 3D stereoscopic plug-in is deployed on the digital twin application server, and the 3D stereoscopic plug-in includes:

[0030] A dual-lens scene capture component for simulating the left and right eyes to obtain the depth information of the target object; and capturing the left and right eye scene images of the corresponding target object.

[0031] A processor for calculating the target focus point and target rotation value of the dual-lens scene capture component according to the depth information of the target object and the rotation value and interpupillary distance function of the preset lens.

[0032] A lens spacing and rotation adjustment module for controlling the dual-lens scene capture component to face the target object.

[0033] A dual-lens screen rendering component for rendering the scene images captured by the dual-lens scene capture component onto two preset high-definition dynamic textures.

[0034] A left and right eye screen fusion module for fusing the two high-definition dynamic textures into one image through the image pixel odd and even column interval arrangement algorithm, and drawing the fused image to the top layer of the view through Widget to obtain a 3D stereoscopic visual sense enhanced image.

[0035] The dual-lens scene capture component, the lens spacing and rotation adjustment module, the dual-lens screen rendering component, and the left and right eye screen fusion module are respectively communicatively connected to the processor.

[0036] The processor is further configured to:

[0037] Calculate the actual distance of the target object from the lens based on the depth information of the target object, and calculate the scene depth of the target object based on the actual distance.

[0038] The 3D stereoscopic plug-in further includes:

[0039] A lens axis component for controlling the lens axis of the dual-lens scene capture component to be mapped to the corresponding scene depth.

[0040] The lens axis assembly is communicatively connected to the processor.

[0041] On the other hand, an electronic device is provided, which includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the methods in the above-mentioned 3D stereoscopic visual sense enhancement optimization processing method based on digital twin three-dimensional applications is implemented.

[0042] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by the processor to implement any one of the methods in the above-mentioned 3D stereoscopic visual sense enhancement optimization processing method based on digital twin three-dimensional applications.

[0043] The beneficial effects brought by the technical solutions provided in the embodiments of the present invention at least include:

[0044] In the digital twin application server of the present invention, a 3D stereo plug-in is added, and a new scene capture function object is developed in the plug-in to solve the problem of poor image quality. It internally includes a dual-lens scene capture component that simulates the human eyes and captures two scene images of the left and right eyes in real-time rendering. By emitting rays from the lens and colliding with the object, the depth of the object is obtained, the actual distance and the screen occupancy ratio between the core object and the lens are judged, and the rotation value (focus point) and the optimal distance (interpupillary distance) of the dual-lens are automatically adjusted to enhance the three-dimensional sense and meet the user's comfortable visual experience.

[0045] The present invention renders the images captured by the dual-lens in real-time onto two high-definition dynamic texture maps; through the image pixel odd-even column interval arrangement algorithm, the two high-definition dynamic texture maps are fused into one image; the fused image is drawn to the top layer of the view through Widget; the application is started and browsed on a polarized display, and on the basis of not affecting the original function interaction of the three-dimensional application, the experiencer can see a clear and comfortable 3D stereo effect by wearing polarized glasses. Description of the Drawings

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0047] Figure 1 It is a block diagram of a 3D stereoscopic visual sense enhancement optimization processing device provided by an embodiment of the present invention;

[0048] Figure 2It is a schematic diagram showing the distribution of the depth of the lens scene provided by an embodiment of the present invention;

[0049] Figure 3 It is a flowchart of a 3D stereoscopic visual sense enhancement optimization processing method based on digital twin 3D applications provided by an embodiment of the present invention;

[0050] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0051] Next, the technical solutions in the present invention will be described with reference to the accompanying drawings.

[0052] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of the word "example" is intended to present concepts in a specific manner. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two can be selected.

[0053] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings they express are the same. "(of)", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings they express are the same.

[0054] In the embodiments of the present invention, sometimes subscripts such as W1 may be miswritten as non-subscript forms such as W1. When their differences are not emphasized, the meanings they express are the same.

[0055] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0056] Regarding the application systems / software, etc. of the digital twin application server, the present invention does not make any limitations. Users can select the corresponding digital twin application platforms or systems by themselves, such as visualization digital twin platforms like 51aes.

[0057] Traditional methods for constructing 3D images on a digital twin platform, such as oblique photography modeling: Oblique photography modeling uses a drone equipped with an oblique photography system to take multi-angle, multi-directional, and multi-perspective photos, obtaining a large amount of image data, and generating a 3D model through image processing, registration, stitching and other technologies. This method can truly reflect the ground object situation, accurately obtain object-side texture information, and is suitable for 3D modeling of large-scale scenes. Oblique photography modeling has advantages such as high efficiency, high precision, low cost, and flexibility, and is particularly suitable for scenarios such as smart cities and complex terrains. Or laser point cloud modeling: Use 3D laser scanning or photogrammetry and other technologies to obtain the spatial 3D data of real-world objects, and then use modeling software to build a high-precision 3D model with the point cloud data as a reference. This method can achieve measurement accuracy at the millimeter level or even the micron level, and is widely used in fields such as building modeling, structural modeling, and industrial design modeling. Laser point cloud modeling has advantages such as high-precision measurement, high-efficiency modeling, and suitability for complex scenes.

[0058] There are various traditional methods for constructing 3D images on a digital twin platform, and each method has its unique advantages and applicable scenarios. When selecting a specific method, it is necessary to comprehensively consider factors such as project requirements, ease of data acquisition, and modeling accuracy requirements.

[0059] The present invention proposes an image optimization processing system for a 3D camera lens, which can perform 3D optimization processing on 2D images captured by a dual lens based on a 3D stereo plug-in deployed on a digital twin application server, enhancing the 3D visual effect.

[0060] When making 3D stereo images, the application of lenses and polarized glasses plays a crucial role.

[0061] The production of 3D stereo images is based on the parallax principle of the human eye, that is, the two eyes of a person are separated by about 5 centimeters, and when looking at anything, the angles of the two eyes are not exactly the same. This slight difference is captured by the retina and transmitted to the brain, and the brain uses this difference to generate the depth of near and far, thus generating a three-dimensional sense. When making 3D stereo images, it is necessary to use two side-by-side lenses (such as movie camera or projector lenses) to represent the left and right eyes of a person respectively, and synchronously shoot or project two images with a slight horizontal parallax.

[0062] Next, the application of polarized glasses becomes particularly important. Polarized glasses change the incident natural light into polarized light in a specific direction through special materials and structures. Usually, the polarization axes of the left and right polarizing lenses are perpendicular to each other. When the audience wears polarized glasses to watch a 3D image, the left eye can only see the left image, and the right eye can only see the right image. In this way, through the binocular convergence function, the left and right images are superimposed on the retina, and the brain nerves generate a three-dimensional stereo visual effect.

[0063] This principle is widely used in 3D movies or projection displays. When showing, two film strips are loaded into the left and right projectors respectively, and two polarizing filters with polarization axes at 90 degrees to each other are installed in front of the projection lens. After the audience wears polarized glasses, they can watch 3D images with a three-dimensional sense.

[0064] The lens is used to capture or project images with horizontal parallax, while the polarized glasses ensure that these images can be correctly received by the audience's left and right eyes, respectively, so as to synthesize a three-dimensional visual effect in the brain. This combined application makes it possible to produce and watch 3D stereoscopic images.

[0065] The present invention proposes a 3D stereoscopic visual enhancement optimization processing device based on a digital twin three-dimensional application, the device comprising: a 3D stereo plug-in and a 3D terminal, wherein the 3D stereo plug-in is deployed on a digital twin application server, and the 3D stereo plug-in comprises:

[0066] The dual-lens scene capture component is used to simulate the left and right eyes to obtain the depth information of the target object, including: the actual distance between the target object and the lens and the screen ratio; and to capture the left and right eye scene images of the corresponding target object;

[0067] A processor, configured to calculate a target focus point and a target rotation value of the dual-lens scene capture assembly according to the target object depth information and a rotation value of a preset lens and an eye distance function;

[0068] A lens spacing and rotation adjustment module, used to control the dual-lens scene capture assembly to face the target object;

[0069] A dual-lens image rendering component, used for rendering the scene image captured by the dual-lens scene capturing component onto two preset high-definition dynamic maps;

[0070] The left-eye and right-eye image fusion module is used to fuse the two high-definition dynamic maps into one image through an image pixel odd-even column spacing algorithm, and draw the fused image to the top layer of the view through a widget to obtain a 3D stereoscopic vision enhanced image;

[0071] The dual-lens scene capture component, the lens spacing and rotation adjustment module, the dual-lens image rendering component and the left-right eye image fusion module are respectively connected to the processor for communication.

[0072] The present invention develops a common three-dimensional application program based on the Unreal Engine (a digital twin application server, which is connected to a 3D terminal through a communication port such as a USB port, etc., and can transmit 3D digital images). After adding a 3D stereo plug-in and running the application, the 3D stereo plug-in will automatically convert and optimize it into a 3D stereoscopic visual application (2D movie visual before processing and 3D movie visual after processing).

[0073] Develop a new scene capture function object in the 3D stereo plug-in (to solve the problem of poor image quality). It internally contains a dual-lens scene capture component that simulates the human eyes and renders two scene images of the left and right eyes in real time.

[0074] Since the closer an object is to the lens, the stronger its 3D sense, and the 3D sense of objects more than 10 meters away from the lens is weak. To solve this problem, the scene depth (the distance between the object and the lens) is divided into 5 depth levels and mapped onto a virtual depth axis with the lens facing directly forward. Objects that require different levels of 3D visual effects are inserted into the corresponding depth level slots to enhance the 3D visual effect.

[0075] Specific application functions and interactions of each plug-in:

[0076] 1. Dual-lens scene capture component

[0077] ‌Application function‌:

[0078] Simulate the left and right eyes of the human eye to capture the scene images of the target object respectively.

[0079] Obtain the actual distance between the target object and the lens, as well as the proportion of the target object in the picture.

[0080] Provide high-precision image data as the basis for subsequent processing.

[0081] ‌Interaction‌:

[0082] Receive control instructions from the processor to adjust the orientation and focus point of the lens.

[0083] Send the captured left and right eye scene images to the dual-lens picture rendering component.

[0084] 2. Processor

[0085] ‌Application function‌:

[0086] According to the target object depth information (actual distance and picture proportion) provided by the dual-lens scene capture component, calculate the target focus point and target rotation value.

[0087] Apply the preset lens rotation value and interpupillary distance function to ensure that the dual-lens can accurately align with the target object.

[0088] Control the lens spacing and rotation adjustment module to achieve precise adjustment of the lens.

[0089] For example:

[0090] 1) Basic parameter setting

[0091] ‌Interpupillary distance (D)‌: Assume the average interpupillary distance of a person is 65mm.

[0092] Lens focal length range (f_min, f_max): Assume that the focal length of the lens can be adjusted between 10 mm and 50 mm.

[0093] Proportion of the screen occupied (P): The proportion of the target object in the screen, expressed as a percentage.

[0094] Actual distance between the target object and the lens (Z): In meters.

[0095] Rotation value of the preset lens and eye distance function: This function may be an empirical formula or a calculation based on geometric optics, used to calculate the rotation value according to the eye distance, focal length, and object distance. For simplicity, assume it is a linear function.

[0096] 2) Example data

[0097] Actual distance between the target object and the lens (Z): 2 meters;

[0098] Proportion of the screen occupied (P): 50% (i.e., the target object occupies half of the screen);

[0099] Calculate the target focus point: In photography and optics, the focus point is usually related to the focal length of the lens and the object distance. However, in this simplified example, the actual physical focus point will not be directly calculated. Instead, assume that by adjusting the focal length of the lens, the target object can be clearly imaged in the screen. Therefore, an "equivalent focal length" can be estimated based on the proportion of the screen occupied, so that the target object occupies the specified proportion in the screen.

[0100] Assume that when the focal length is f, the proportion of the target object in the screen is exactly 50%. Then, the focal length at this time can be used as the equivalent focal length of the target focus point. Since this is a simplified model, factors such as lens distortion and aberration are not considered. For simplicity of calculation, assume an inverse relationship between the proportion of the screen occupied and the focal length (in reality, this relationship may be more complex):

[0101] P = k / (Z × f),

[0102] where k is a constant. Since P = 50% (i.e., 0.5) and Z = 2 meters are known, this equation can be solved to find f:

[0103] 0.5 = k / (2 × f),

[0104] Assume k = 0.01 (this is just an assumption for simplicity of calculation), then:

[0105] f = 0.01 meters = 10 mm.

[0106] Obviously, this focal length matches the previously set focal length range (10mm to 50mm). Therefore, in practical applications, this calculation needs to be adjusted according to the actual focal length range of the lens. In this example, the maximum value (50mm) within the focal length range may be selected as the target focus point because this will make the target object as large as possible in the frame.

[0107] 3) Calculate the target rotation value

[0108] The calculation of the rotation value may be more complex because it depends on the specific configuration of the lens, the eye distance, and the position of the target object. However, for simplicity, it can be assumed that the rotation value is related to the eye distance and the relative position of the target object to the lens.

[0109] Construct a linear function to calculate the rotation value (θ):

[0110] θ = k2 × (D / Z)

[0111] where k2 is a constant. Similarly, for simplicity of calculation, it can be assumed that k2 = 1 (this is just an example). Then:

[0112] θ = 0.0325 radians.

[0113] In this simplified example, based on the depth information (distance and frame occupancy) of the target object, as well as the preset lens parameters and functions, the target focus point (approximately 10mm) and the target rotation value (about 1.86 degrees) of the dual-lens scene capture component are calculated. It should be noted that this calculation is based on a series of simplifications and assumptions, and more complex models and calculation methods may be required in practical applications to obtain accurate results.

[0114] ‌ Interaction ‌:

[0115] Receive the depth information of the dual-lens scene capture component.

[0116] Send control instructions to the lens spacing and rotation adjustment module.

[0117] Send the calculated focus point and rotation value to the dual-lens frame rendering component (optional, for further optimizing the rendering effect).

[0118] 3. Lens Spacing and Rotation Adjustment Module

[0119] ‌ Application Function ‌:

[0120] According to the instructions of the processor, adjust the spacing and rotation angle of the lenses in the dual-lens scene capture component.

[0121] Ensure that the dual lenses can accurately align with the target object to obtain high-quality image data.

[0122] ‌ Interaction ‌:

[0123] Receive the control instructions of the receiving processor.

[0124] Control the lens of the dual-lens scene capture component to adjust the spacing and rotation.

[0125] 4. Dual-lens frame rendering component

[0126] ‌Application function‌:

[0127] Render the left and right eye scene images captured by the dual-lens scene capture component onto two preset high-definition dynamic texture maps.

[0128] Optimize the rendering effect according to the focus point and rotation value (if provided) of the processor.

[0129] Provide high-quality rendered images, providing a basis for subsequent image fusion.

[0130] ‌Interaction‌:

[0131] Receive the left and right eye scene images of the dual-lens scene capture component.

[0132] Receive the focus point and rotation value of the processor (optional).

[0133] Send the rendered high-definition dynamic texture maps to the left and right eye frame fusion module.

[0134] 5. Left and right eye frame fusion module

[0135] ‌Application function‌:

[0136] Through the image pixel odd-even column interval arrangement algorithm, fuse the two high-definition dynamic texture maps into one image. Specifically:

[0137] The detailed implementation steps of fusing the two high-definition dynamic texture maps into one image through the image pixel odd-even column interval arrangement algorithm are as follows:

[0138] Step 1: Prepare the high-definition dynamic texture maps

[0139] ‌Obtain the texture maps‌: The administrator pre-sets two high-definition dynamic texture maps with the same size and resolution, denoted as texture map A and texture map B respectively. These two texture maps should be the left and right eye views obtained from the dual-lens scene capture component.

[0140] ‌Check the format‌: Confirm whether the formats (such as RGB, RGBA, etc.) and pixel depths (such as 8-bit, 16-bit, etc.) of the two texture maps are the same. If not, format conversion or depth adjustment is required.

[0141] Step 2: Initialize the fused image

[0142] Create a new image: Create a new blank image, denoted as the composite image, according to the size and format of Texture A (or Texture B, as they are of the same size).

[0143] Set initial values: If the image has an alpha channel (such as in RGBA format), ensure that the alpha channel of the composite image is initialized to fully opaque (or set as needed).

[0144] Step 3: Apply the pixel odd-even column interval arrangement algorithm

[0145] Traverse pixels: Traverse the pixels of Texture A and Texture B row by row. For each row, traverse the pixels column by column.

[0146] Odd-even column determination:

[0147] For the odd-column pixels in the current row, take the pixel value at the corresponding position from Texture A and assign it to the pixel at the same position in the composite image.

[0148] For the even-column pixels in the current row, take the pixel value at the corresponding position from Texture B and assign it to the pixel at the same position in the composite image.

[0149] Handle boundaries: If the width of the image is odd, the last column of pixels can be chosen to take values from Texture A or Texture B as needed, or undergo special processing (such as taking the average, interpolation, etc.).

[0150] Step 4: Optimization and adjustment

[0151] Smoothing processing: To mitigate the potential issue of uneven image edges caused by the odd-even column interval arrangement, a slight smoothing filter can be applied to the composite image.

[0152] Color adjustment: Adjust the color balance, brightness, contrast, etc. of the composite image as needed to achieve a better visual effect.

[0153] Stereo effect enhancement: If the goal is to enhance the stereo visual perception, specific stereo enhancement algorithms, such as parallax adjustment and depth perception enhancement, can be considered for application to the composite image.

[0154] Step 5: Save and output

[0155] Save the image: Save the composite image in the desired file format (such as JPEG, PNG, etc.).

[0156] Output for display: Output the composite image to a display device, such as a screen, projector, etc., for users to view.

[0157] Through the above steps, applying the image pixel odd-even column interval arrangement algorithm, two high-definition dynamic texture maps are fused into an image with a specific visual effect.

[0158] Ensure that the fused image has the effect of enhanced 3D stereoscopic visual sense.

[0159] Draw the fused image to the top layer of the view through the Widget for the user to view.

[0160] Interaction:

[0161] Receive two high-definition dynamic texture maps from the dual-lens frame rendering component.

[0162] Apply the image pixel odd-even column interval arrangement algorithm for image fusion.

[0163] Send the fused image to the view layer for display.

[0164] Interaction process:

[0165] The dual-lens scene capture component captures the left and right eye scene images and depth information of the target object.

[0166] The processor calculates the target focus point and rotation value based on the depth information, and controls the lens spacing and rotation adjustment module to adjust the lens.

[0167] The dual-lens frame rendering component renders the captured image onto the high-definition dynamic texture map (optionally, optimizing the rendering effect according to the focus point and rotation value).

[0168] The left and right eye frame fusion module fuses the two texture maps into an image with enhanced 3D stereoscopic visual sense through an algorithm, and draws it to the top layer of the view for the user to view.

[0169] Through such a detailed component division and interaction process design, it can be ensured that the system can efficiently achieve the enhanced effect of 3D stereoscopic visual sense and provide an immersive visual experience for users.

[0170] The processor is further configured to:

[0171] Calculate the actual distance between the target object and the lens according to the depth information of the target object, and calculate the scene depth of the target object according to the actual distance.

[0172] The 3D stereo plug-in further includes:

[0173] A lens axis component for controlling the lens axis of the dual-lens scene capture component to be mapped to the corresponding scene depth;

[0174] The lens axis component is communicatively connected to the processor.

[0175] Here, the scene depth (the distance of an object from the camera) is divided into 5 depth levels and mapped onto a virtual depth axis with the camera facing straight ahead. Objects that require different degrees of 3D visual perception are placed in the corresponding depth level slots to enhance the 3D visual perception.

[0176] As Figure 2 shown, with the camera VP_0 as the reference, the scene depth advances 2 depths forward, namely VP_1 and VP_2. By controlling the camera axis component, the camera axis of the dual-camera scene capture component can be mapped to the corresponding scene depth according to the calculated scene depth. It is specifically selected according to the target object.

[0177] By emitting rays from the camera and colliding with the object, the object depth is obtained, and the actual distance and the proportion of the core object in the frame are judged. The rotation value (focus point) and the optimal distance (interocular distance) of the dual cameras are automatically adjusted to enhance the stereoscopic effect and meet the user's comfortable visual experience.

[0178] Based on the application of the above device, an embodiment of the present invention provides a 3D stereoscopic visual perception enhancement optimization processing method based on digital twin three-dimensional applications. This method can be implemented by an electronic device, which can be a terminal or a server. As Figure 3 shown in the flowchart of the 3D stereoscopic visual perception enhancement optimization processing method based on digital twin three-dimensional applications, the processing flow of this method can include the following steps:

[0179] S1. Activate the 3D stereoscopic plug-in in the 3D camera background, and obtain the depth information of the target object through the dual-camera scene capture component;

[0180] S2. According to the depth information of the target object, calculate the target focus point and the target rotation value of the dual-camera scene capture component according to the preset rotation value and interocular distance function of the camera, and control the dual-camera scene capture component to face the target object;

[0181] S3. Capture the corresponding scene image through the dual-camera scene capture component;

[0182] S4. Render the scene image captured by the dual-camera scene capture component onto two preset high-definition dynamic texture maps;

[0183] S5. Through the image pixel odd-even column interval arrangement algorithm, fuse the two high-definition dynamic texture maps into one image, and draw the fused image to the top layer of the view through Widget to obtain a 3D stereoscopic visual perception enhanced image;

[0184] S6. Send the 3D stereoscopic visual perception enhanced image to the 3D terminal.

[0185] When specifically implemented, in combination with the above device application description, the world position of the object in the three-dimensional space is obtained as the focus point and input into the function provided by the dual-lens component. The function performs three-dimensional matrix mathematical operations on the dual-lens component to calculate the target rotation value of the dual-lens, and then sets it into the dual-lens component to drive the lens to rotate so that the lens faces the target point.

[0186] Render the images captured by the lens in real time onto two high-definition dynamic texture maps;

[0187] Fuse the two high-definition dynamic texture maps into one image through the image pixel odd-even column interval arrangement algorithm;

[0188] Draw the fused image onto the top layer of the view through Widget;

[0189] Start the application and view it on the polarized display. On the basis of not affecting the original function interaction of the three-dimensional application, the experiencer wearing polarized glasses can see a clear and comfortable 3D stereoscopic effect.

[0190] Figure 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 4 shown, the electronic device may include the above-mentioned Figure 3 3D stereoscopic visual sense enhancement and optimization processing device based on digital twin three-dimensional applications. Optionally, the electronic device 410 may include a first processor 2001.

[0191] Optionally, the electronic device 410 may further include a memory 2002 and a transceiver 2003.

[0192] Among them, the first processor 2001, the memory 2002, and the transceiver 2003 may be connected through a communication bus, for example.

[0193] Next, in combination with Figure 4 specific introductions will be made to the various components of the electronic device 410:

[0194] Among them, the first processor 2001 is the control center of the electronic device 410, which can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or can be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0195] Optionally, the first processor 2001 can execute various functions of the electronic device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0196] In a specific implementation, as an embodiment, the first processor 2001 can include one or more CPUs, such as Figure 4 CPU0 and CPU1 shown in

[0197] In a specific implementation, as an embodiment, the electronic device 410 can also include multiple processors, such as Figure 4 the first processor 2001 and the second processor 2004 shown in

[0198] Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0199] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 4 not shown) of the electronic device 410. The embodiments of the present invention do not make specific limitations on this.

[0200] The transceiver 2003 is used to communicate with a network device or with a terminal device.

[0201] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 not shown separately). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0202] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 4 not shown) of the electronic device 410. The embodiments of the present invention do not make specific limitations on this.

[0203] It should be noted that Figure 4 the structure of the electronic device 410 shown in does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0204] In addition, the technical effects of the electronic device 410 may refer to the technical effects of the 3D stereoscopic visual sense enhancement optimization processing method based on digital twin three-dimensional applications described in the above method embodiments, and will not be elaborated here.

[0205] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0206] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0207] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that contains one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0208] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. Additionally, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood with reference to the context before and after.

[0209] In the present invention, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0210] It should be understood that in various embodiments of the present invention, the magnitudes of the sequence numbers of the above processes do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0211] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0212] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the devices, apparatuses, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0213] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0214] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0215] In addition, the functional units in various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0216] If the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0217] As described above, the above are only specific implementation manners of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application, characterized in that: The method comprises: S1. Activate the 3D stereo plug-in in the 3D camera background, and obtain the depth information of the target object through the dual-lens scene capture component, including: the actual distance between the target object and the lens and the screen ratio; S2, calculating the target focus point and the target rotation value of the dual-lens scene capture component according to the depth information of the target object, according to the rotation value of the preset lens and the eye distance function, and controlling the dual-lens scene capture component to face the target object; Calculate the target focus point: Estimate an "equivalent focal length" based on the screen ratio so that the target object occupies a specified proportion in the screen. When the focal length is f, it is assumed that the screen ratio is inversely proportional to the focal length: P = k / (Z × f), Where k is a constant, Z is the actual distance between the target object and the lens, and P is the screen ratio; Calculate the target rotation value: Construct a linear function to calculate the rotation value (θ): θ = k2 × (D / Z), Where k2 is a constant, and D is the average eye distance of a person; S3, capturing a corresponding scene image through the dual-lens scene capture component; S4, rendering the scene image captured by the dual-lens scene capture component onto two preset high-definition dynamic maps; S5. By using an image pixel odd-even column spacing algorithm, the two high-definition dynamic maps are merged into one image, and the merged image is drawn to the top layer of the view through a widget to obtain a 3D stereoscopic vision enhanced image; S6: Send the 3D stereoscopic vision enhanced image to a 3D terminal.

2. The 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application according to claim 1 is characterized in that: The 3D terminal is polarized glasses.

3. The 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application according to claim 1 is characterized in that: After obtaining the depth information of the target object, it also includes: Calculating the actual distance of the target object from the lens according to the target object depth information, and calculating the scene depth of the target object according to the actual distance; Through the lens axis component, the lens axis of the dual-lens scene capture component is controlled to be mapped to the corresponding scene depth.

4. A 3D stereoscopic vision enhancement optimization processing device based on digital twin three-dimensional application, which is applied to the 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application as described in any one of claims 1 to 3, characterized in that: The device includes: A 3D stereo plug-in and a 3D terminal, wherein the 3D stereo plug-in is deployed on a digital twin application server, and the 3D stereo plug-in includes: A dual-lens scene capture component is used to simulate the left and right eyes to obtain the depth information of the target object; and to capture the left and right eye scene images corresponding to the target object; The processor is used to calculate the target focus point and the target rotation value of the dual-lens scene capture component according to the depth information of the target object, the rotation value of the preset lens and the eye distance function; calculate the target focus point: estimate an "equivalent focal length" according to the screen ratio so that the target object occupies a specified proportion in the screen, and when the focal length is f, it is assumed that the screen ratio is inversely proportional to the focal length: P = k / (Z × f), Where k is a constant, Z is the actual distance between the target object and the lens, and P is the screen ratio; Calculate the target rotation value: Construct a linear function to calculate the rotation value (θ): θ = k2 × (D / Z), Where k2 is a constant, and D is the average eye distance of a person; A lens spacing and rotation adjustment module, used to control the dual-lens scene capture assembly to face the target object; A dual-lens image rendering component, used for rendering the scene image captured by the dual-lens scene capturing component onto two preset high-definition dynamic maps; The left-eye and right-eye image fusion module is used to fuse the two high-definition dynamic maps into one image through an image pixel odd-even column spacing algorithm, and draw the fused image to the top layer of the view through a widget to obtain a 3D stereoscopic vision enhanced image; The dual-lens scene capture component, the lens spacing and rotation adjustment module, the dual-lens image rendering component and the left-right eye image fusion module are respectively connected to the processor for communication.

5. The 3D stereoscopic vision enhancement optimization processing device based on digital twin three-dimensional application according to claim 4 is characterized in that: The processor is further configured to: According to the target object depth information, the actual distance of the target object from the lens is calculated, and the scene depth of the target object is calculated according to the actual distance.

6. The 3D stereoscopic vision enhancement optimization processing device based on digital twin three-dimensional application according to claim 4 is characterized in that: The 3D stereo plug-in also includes: A lens axis component, used to control the lens axis of the dual-lens scene capture component to be mapped to the corresponding scene depth; The lens axis assembly is communicatively connected to the processor.

7. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 3 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 3.

Images