3D stereoscopic vision enhancement optimization processing method based on digital twinborn three-dimensional application

Abstract

The invention provides a digital twinning three-dimensional application-based 3D visual sense enhancement optimization processing method, which comprises the following steps of: adding a 3D plug-in in a digital twinning application server, acquiring the depth of an object through collision between rays emitted from a lens and the object, judging the actual distance between a core object and the lens and a picture ratio, and determining the visual sense of the object according to the actual distance between the core object and the lens. And the rotation value (focus point) and the optimal distance (eye distance) of the double lenses are automatically adjusted, so that the stereoscopic impression is enhanced, and the comfortable visual experience of a user is met. Rendering the images captured by the double lenses in real time to the two high-definition dynamic chartlets; fusing the two high-definition dynamic chartlets into one image through an image pixel odd-even column interval arrangement algorithm; the fused image is drawn to the top layer of the view through Widget; and the application is started and browsed on the polarization display, so that an experiencer can see a clear and comfortable 3D stereoscopic effect by wearing the polarization glasses on the basis of not influencing the original function interaction of the three-dimensional application.

Description

Technical Field

[0001] The present invention relates to the field of 3D vision technology, and in particular to a 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional applications, a 3D stereoscopic vision enhancement optimization processing device based on digital twin three-dimensional applications, an electronic device and a computer-readable storage medium. Background Art

[0002] In the construction of digital twin 3D models, the traditional application methods for producing 3D stereoscopic scene images mainly include the following: 1. Traditional 3D technology (stereoscopic simulation technology) Directly use the 3D software to open the picture and draw the lines without cutting out the image.

[0003] Make a layered plan in Photoshop, then import it into 3D software to outline a single layer.

[0004] This method is the most common technology in the 3D market. It is widely promoted in the training industry because it is relatively simple and easy to train. However, whether a good 3D effect can be produced depends on the operator's understanding and skills.

[0005] 2. Virtual Reality 3D Technology This is an upgrade and improvement of traditional three-dimensional technology, aiming to simplify the production process and improve the realism and drawing efficiency of stereoscopic paintings.

[0006] 3. Digital Reality 3D Technology The technology further developed based on the virtual reality 3D technology includes three core technologies: edge optimization technology, 3D data optimization technology and raster correction technology.

[0007] Edge optimization technology: Analyze and process the image based on its actual picture effect, so that the edge of each image smoothly transitions in the depth direction, eliminating the sense of flakes and making the image outline more smooth, realistic and natural.

[0008] 3D data optimization technology: Based on edge optimization technology, according to the actual 3D requirements of the picture, the local and overall 3D data can be freely adjusted, and can also be adjusted according to the actual proportion of the picture to obtain a realistic 3D effect.

[0009] Grating calibration technology: a technology for testing grating data, which can eliminate light waves by testing the grating. This technology can not only quickly test the data of qualified gratings, but also test the error range of gratings with poor straightness, laying a good foundation for grating correction technology.

[0010] With the growing demand for 3D applications and visual enhancement, ordinary 3D applications are difficult to meet the impact of immersive visual experience. Traditional 3D application methods often require high R&D costs and complex production processes, and the generated applications have many visual problems, such as weak 3D sense and poor image quality, which are difficult to meet users' requirements for immersive and comfortable visual experience.

[0011] Therefore, there is an urgent need for an application method that can automatically convert ordinary three-dimensional applications and optimize 3D stereoscopic vision to promote technological progress and widespread application in related fields. Summary of the invention

[0012] In order to solve the technical problems existing in the prior art, the present invention provides the following technical solutions: On the one hand, a 3D stereoscopic vision enhancement optimization processing method based on a digital twin three-dimensional application is provided, the method is implemented by an electronic device, and the method includes: S1. Activate the 3D stereo plug-in in the 3D camera background and obtain the depth information of the target object through the dual-lens scene capture component; S2, calculating the target focus point and the target rotation value of the dual-lens scene capture component according to the depth information of the target object, according to the rotation value of the preset lens and the eye distance function, and controlling the dual-lens scene capture component to face the target object; S3, capturing a corresponding scene image through the dual-lens scene capture component; S4, rendering the scene image captured by the dual-lens scene capture component onto two preset high-definition dynamic maps; S5. By using an image pixel odd-even column spacing algorithm, the two high-definition dynamic maps are merged into one image, and the merged image is drawn to the top layer of the view through a widget to obtain a 3D stereoscopic vision enhanced image; S6: Send the 3D stereoscopic vision enhanced image to a 3D terminal.

[0013] Preferably, the target object depth information includes: the actual distance between the target object and the lens and the screen ratio.

[0014] Preferably, the 3D terminal is polarized glasses.

[0015] Preferably, after acquiring the depth information of the target object, the method further includes: Calculating the actual distance of the target object from the lens according to the target object depth information, and calculating the scene depth of the target object according to the actual distance; Through the lens axis component, the lens axis of the dual-lens scene capture component is controlled to be mapped to the corresponding scene depth.

[0016] On the other hand, a 3D stereoscopic vision enhancement optimization processing device based on a digital twin three-dimensional application is provided, and the device is applied to a 3D stereoscopic vision enhancement optimization processing method based on a digital twin three-dimensional application. The device includes: a 3D stereo plug-in and a 3D terminal, wherein the 3D stereo plug-in is deployed on a digital twin application server, and the 3D stereo plug-in includes: A dual-lens scene capture component is used to simulate the left and right eyes to obtain the depth information of the target object; and to capture the left and right eye scene images corresponding to the target object; A processor, configured to calculate a target focus point and a target rotation value of the dual-lens scene capture assembly according to the target object depth information and a rotation value of a preset lens and an eye distance function; A lens spacing and rotation adjustment module, used to control the dual-lens scene capture assembly to face the target object; A dual-lens image rendering component, used for rendering the scene image captured by the dual-lens scene capturing component onto two preset high-definition dynamic maps; The left-eye and right-eye image fusion module is used to fuse the two high-definition dynamic maps into one image through an image pixel odd-even column spacing algorithm, and draw the fused image to the top layer of the view through a widget to obtain a 3D stereoscopic vision enhanced image; The dual-lens scene capture component, the lens spacing and rotation adjustment module, the dual-lens image rendering component and the left-right eye image fusion module are respectively connected to the processor for communication.

[0017] The processor is further configured to: According to the target object depth information, the actual distance of the target object from the lens is calculated, and the scene depth of the target object is calculated according to the actual distance.

[0018] The 3D stereo plug-in also includes: A lens axis component, used to control the lens axis of the dual-lens scene capture component to be mapped to the corresponding scene depth; The lens axis assembly is communicatively connected to the processor.

[0019] On the other hand, an electronic device is provided, comprising: a processor; and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned 3D stereoscopic vision enhancement optimization processing methods based on digital twin three-dimensional applications is implemented.

[0020] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned 3D stereoscopic vision enhancement optimization processing methods based on digital twin three-dimensional applications.

[0021] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: The present invention adds a 3D stereo plug-in to the digital twin application server, develops a new scene capture function object in the plug-in to solve the problem of poor image quality, and contains a dual-lens scene capture component to simulate human eyes, and renders and captures two scene images of the left and right eyes in real time. By colliding with the object through the rays emitted from the lens, the depth of the object is obtained, the actual distance between the core object and the lens and the proportion of the screen are determined, and the rotation value (focus point) and the optimal spacing (eye distance) of the dual lenses are automatically adjusted to enhance the stereoscopic effect and meet the user's comfortable visual experience.

[0022] The present invention renders the images captured by dual lenses in real time onto two high-definition dynamic maps; merges the two high-definition dynamic maps into one image through an image pixel odd-even column spacing algorithm; draws the merged image to the top layer of the view through a widget; starts the application and browses it on a polarized display, and without affecting the original functional interaction of the three-dimensional application, the experiencer can see a clear and comfortable 3D stereoscopic effect by wearing polarized glasses. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a block diagram of a 3D stereoscopic vision enhancement optimization processing device based on a digital twin three-dimensional application provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the distribution of a lens scene depth provided by an embodiment of the present invention; Figure 3 It is a flow chart of a 3D stereoscopic vision enhancement optimization processing method based on a digital twin three-dimensional application provided by an embodiment of the present invention; Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0026] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0027] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0028] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0029] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0030] The present invention does not limit the application system / software of the digital twin application server, and users can choose the corresponding digital twin application platform or system by themselves, such as 51aes and other visual digital twin platforms.

[0031] Traditional methods of building 3D images on digital twin platforms, such as oblique photography modeling: Oblique photography modeling uses drones equipped with oblique photography systems to shoot at multiple angles, directions, and perspectives to obtain massive image data, and then generates a three-dimensional model through image processing, registration, splicing and other technical processes. This method can truly reflect the situation of ground objects and obtain object texture information with high precision, which is suitable for three-dimensional modeling of large-scale scenes. Oblique photography modeling has the advantages of high efficiency, high precision, low cost and flexibility, and is particularly suitable for scenes such as smart cities and complex terrain. Or laser point cloud modeling: Use technologies such as three-dimensional laser scanning or photogrammetry to obtain spatial three-dimensional data of real targets, and then use modeling software to build a high-precision three-dimensional model with point cloud data as a reference. This method can achieve millimeter-level or even micron-level measurement accuracy and is widely used in building modeling, structural modeling, industrial design modeling and other fields. Laser point cloud modeling has the advantages of high-precision measurement, high-efficiency modeling and applicability to complex scenes.

[0032] There are many traditional methods for building 3D images on the digital twin platform, each with its own unique advantages and applicable scenarios. When choosing a specific method, it is necessary to comprehensively consider factors such as project requirements, the difficulty of data acquisition, and modeling accuracy requirements.

[0033] The present invention proposes an image optimization processing system for a 3D camera lens, which can perform 3D optimization processing on 2D images captured by dual lenses based on a 3D stereo plug-in deployed on a digital twin application server to enhance the 3D visual effect.

[0034] When creating 3D stereo images, the use of lenses and polarized glasses plays a vital role.

[0035] The production of 3D stereo images is based on the parallax principle of the human eye, that is, the two eyes are about 5 cm apart, and when looking at anything, the angles of the two eyes are not exactly the same. This tiny difference is captured by the retina and transmitted to the brain, which uses this difference to produce depth of distance, thereby creating a sense of three-dimensionality. When producing 3D stereo images, two parallel lenses (such as movie camera or projector lenses) are needed to represent the left and right eyes of a person, and to simultaneously shoot or project two images with slight horizontal parallax.

[0036] Next, the application of polarized glasses is particularly important. Polarized glasses use special materials and structures to convert incident natural light into polarized light in a specific direction. Usually, the polarization axes of the left and right polarizers are perpendicular to each other. When viewers wear polarized glasses to watch 3D images, the left eye can only see the left image, and the right eye can only see the right image. In this way, the left and right images are superimposed on the retina through the binocular convergence function, and the brain nerves produce a three-dimensional visual effect.

[0037] This principle is widely used in 3D movies or projection displays. When showing, two film strips are loaded into the left and right projectors respectively, and two polarizing filters with polarization axes at 90 degrees to each other are installed in front of the projection lens. After the audience wears polarized glasses, they can watch 3D images with a three-dimensional sense.

[0038] The lens is used to capture or project images with horizontal parallax, while the polarized glasses ensure that these images can be correctly received by the audience's left and right eyes, respectively, so as to synthesize a three-dimensional visual effect in the brain. This combined application makes it possible to produce and watch 3D stereoscopic images.

[0039] The present invention proposes a 3D stereoscopic visual enhancement optimization processing device based on a digital twin three-dimensional application, the device comprising: a 3D stereo plug-in and a 3D terminal, wherein the 3D stereo plug-in is deployed on a digital twin application server, and the 3D stereo plug-in comprises: The dual-lens scene capture component is used to simulate the left and right eyes to obtain the depth information of the target object, including: the actual distance between the target object and the lens and the screen ratio; and to capture the left and right eye scene images of the corresponding target object; A processor, configured to calculate a target focus point and a target rotation value of the dual-lens scene capture assembly according to the target object depth information and a rotation value of a preset lens and an eye distance function; A lens spacing and rotation adjustment module, used to control the dual-lens scene capture assembly to face the target object; A dual-lens image rendering component, used for rendering the scene image captured by the dual-lens scene capturing component onto two preset high-definition dynamic maps; The left-eye and right-eye image fusion module is used to fuse the two high-definition dynamic maps into one image through an image pixel odd-even column spacing algorithm, and draw the fused image to the top layer of the view through a widget to obtain a 3D stereoscopic vision enhanced image; The dual-lens scene capture component, the lens spacing and rotation adjustment module, the dual-lens image rendering component and the left-right eye image fusion module are respectively connected to the processor for communication.

[0040] The present invention develops a common three-dimensional application program based on the Unreal Engine (a digital twin application server, which is connected to a 3D terminal through a communication port such as a USB port, etc., and can transmit 3D digital images). After adding a 3D stereo plug-in and running the application, the 3D stereo plug-in will automatically convert and optimize it into a 3D stereoscopic visual application (2D movie visual before processing and 3D movie visual after processing).

[0041] Develop a new scene capture function object in the 3D stereo plug-in (to solve the problem of poor image quality), which contains a dual-lens scene capture component to simulate human eyes and capture two scene images of the left and right eyes in real time; The closer the object is to the lens, the stronger the 3D sense is, and the 3D sense is weaker for objects beyond 10 meters from the lens. To solve this problem, the scene depth (the distance of the object from the lens) is divided into 5 depth levels and mapped to a virtual depth axis facing the front of the lens. Objects that require different degrees of 3D vision are placed in the corresponding depth level slots to enhance the 3D vision.

[0042] Specific application functions and interactions of each plug-in: 1. Dual-lens scene capture component ‌APP FEATURES‌: Simulate the left and right eyes of the human eye to capture scene images of the target object respectively.

[0043] Get the actual distance between the target object and the lens, as well as the proportion of the target object in the picture.

[0044] Provide high-precision image data, which provides a basis for subsequent processing.

[0045] ‌Interaction‌: Receive control instructions from the processor and adjust the direction and focus of the lens.

[0046] The captured left and right eye scene images are sent to the dual-lens image rendering component.

[0047] 2. Processor ‌APP FEATURES‌: According to the depth information (actual distance and screen ratio) of the target object provided by the dual-lens scene capture component, the target focus point and target rotation value are calculated.

[0048] Apply the preset lens rotation value and eye distance function to ensure that the dual cameras can accurately align with the target object.

[0049] Control the lens spacing and rotation adjustment module to achieve precise adjustment of the lens.

[0050] for example: 1) Basic parameter setting ‌Eye distance (D): Assume that the average eye distance is 65mm.

[0051] ‌Lens focal length range (f_min, f_max)‌: Assume that the focal length of the lens can be adjusted between 10mm and 50mm.

[0052] ‌Screen ratio (P)‌: The ratio of the target object in the screen, expressed as a percentage.

[0053] ‌ Actual distance of the target object from the camera (Z) ‌: in meters.

[0054] ‌ Preset lens rotation value and eye distance function ‌: This function may be an empirical formula or a calculation based on geometric optics, which is used to calculate the rotation value based on eye distance, focal length and object distance. For simplicity, it is assumed to be a linear function.

[0055] 2) Sample Data ‌ Actual distance between the target object and the lens (Z) ‌: 2 meters; ‌Screen ratio (P): 50% (i.e. the target object occupies half of the screen); Calculate the target focus point: In photography and optics, the focus point is usually related to the focal length of the lens and the object distance. However, in this simplified example, the actual physical focus point will not be calculated directly, but it is assumed that by adjusting the focal length of the lens, the target object can be clearly imaged in the picture. Therefore, an "equivalent focal length" can be estimated based on the picture ratio so that the target object occupies a specified proportion in the picture.

[0056] Assuming that when the focal length is f, the target object occupies exactly 50% of the image, the focal length at this time can be used as the equivalent focal length of the target focus point. Since this is a simplified model, factors such as lens distortion and aberration are not considered. To simplify the calculation, it is assumed that the image occupancy rate is inversely proportional to the focal length (in practice, this relationship may be more complicated): P = k / (Z × f), where k is a constant. Since we know that P = 50% (or 0.5) and Z = 2 meters, we can solve this equation to find f: 0.5=k / (2×f), Assume k=0.01 (this is just an assumption made to simplify the calculation), then: f=0.01m=10mm.

[0057] Obviously, this focal length matches the focal length range set previously (10mm to 50mm). Therefore, in practice, this calculation needs to be adjusted according to the actual focal length range of the lens. In this example, the maximum value in the focal length range (50mm) may be selected as the target focus point, because this will make the target object as large as possible in the frame.

[0058] 3) Calculate the target rotation value The calculation of the rotation value can be more complex, as it depends on the specific configuration of the lens, the eye distance, and the position of the target object. However, for simplicity, it can be assumed that the rotation value is related to the eye distance and the relative position of the target object to the lens.

[0059] Construct a linear function to calculate the rotation value (θ): θ = k2 × (D / Z) Where k2 is a constant. Similarly, to simplify the calculation, we can assume k2=1 (this is just an example). Then: θ = 0.0325 radians.

[0060] In this simplified example, the target focus point (approximately 10mm) and target rotation value (approximately 1.86 degrees) of the dual-lens scene capture component are calculated based on the depth information (distance and screen ratio) of the target object and the preset lens parameters and functions. It should be noted that this calculation is based on a series of simplifications and assumptions, and more complex models and calculation methods may be required in actual applications to obtain accurate results.

[0061] ‌Interaction‌: Receive depth information from a dual-lens scene capture component.

[0062] Send control instructions to the lens spacing and rotation adjustment module.

[0063] Send the calculated focus point and rotation values ​​to the dual-lens image rendering component (optional, used to further optimize the rendering effect).

[0064] 3. Lens spacing and rotation adjustment module ‌APP FEATURES‌: According to the instructions of the processor, the spacing and rotation angle of the lenses in the dual-lens scene capture component are adjusted.

[0065] Ensure that the dual lenses can accurately aim at the target object and obtain high-quality image data.

[0066] ‌Interaction‌: Receive control instructions from the processor.

[0067] Controls the pitch and rotation of the lenses of a dual-lens scene capture component.

[0068] 4. Dual-lens image rendering component ‌APP FEATURES‌: Render the left and right eye scene images captured by the dual-lens scene capture component onto two preset high-definition dynamic maps.

[0069] Optimizes rendering based on the processor's focus point and rotation values ​​(if provided).

[0070] Provide high-quality rendered images, which provide a basis for subsequent image fusion.

[0071] ‌Interaction‌: Receive left and right eye scene images from a dual-lens scene capture component.

[0072] Receives focus and rotation values ​​from a processor (optional).

[0073] The rendered high-definition dynamic map is sent to the left and right eye image fusion module.

[0074] 5. Left and right eye image fusion module ‌APP FEATURES‌: Through the image pixel odd-even column spacing algorithm, two high-definition dynamic maps are merged into one image. Specifically: The detailed implementation steps of merging two high-definition dynamic maps into one image through the image pixel odd-even column spacing algorithm are as follows: Step 1: Prepare HD dynamic textures ‌Get stickers‌: The administrator pre-sets two high-definition dynamic stickers of the same size and resolution, which are respectively recorded as sticker A and sticker B. These two stickers should be the left and right eye views obtained from the dual-lens scene capture component.

[0075] ‌Check Format‌: Check whether the formats (such as RGB, RGBA, etc.) and pixel depths (such as 8-bit, 16-bit, etc.) of the two textures are consistent. If they are inconsistent, format conversion or depth adjustment is required.

[0076] Step 2: Initialize the fused image ‌Create New Image‌: Create a new blank image based on the size and format of Texture A (or Texture B, since they are the same size), denoted as the fused image.

[0077] ‌Set Initial Value‌: If the image has an alpha channel (e.g. RGBA format), make sure the fused image's alpha channel is initialized to fully opaque (or set as desired).

[0078] Step 3: Apply the pixel odd-even column spacing algorithm ‌Iterate over pixels‌: Iterate over the pixels of both tile A and tile B by row. For each row, iterate over the pixels by column.

[0079] ‌Parity column judgment‌: For the odd-numbered column pixels in the current row, the pixel values ​​at the corresponding positions are taken from map A and assigned to the pixels at the same positions in the fused image.

[0080] For the pixels in the even columns of the current row, the pixel values ​​at the corresponding positions are taken from map B and assigned to the pixels at the same positions in the fused image.

[0081] ‌Processing boundaries‌: If the width of the image is an odd number, the last column of pixels can be selected from map A or map B as needed, or special processing (such as averaging, interpolation, etc.) can be performed.

[0082] Step 4: Optimization and Adjustment ‌Smoothing‌: In order to alleviate the problem of uneven image edges caused by the arrangement of odd and even columns, a slight smoothing filter can be performed on the fused image.

[0083] ‌Color Adjustment‌: Adjust the color balance, brightness, contrast, etc. of the fused image as needed to achieve better visual effects.

[0084] ‌Stereoscopic effect enhancement‌: If the goal is to enhance stereoscopic vision, you can consider applying specific stereoscopic enhancement algorithms to the fused image, such as parallax adjustment, depth perception enhancement, etc.

[0085] Step 5: Save and export ‌Save Image‌: Save the fused image in the desired file format (such as JPEG, PNG, etc.).

[0086] ‌Output display‌: Output the fused image to a display device, such as a screen, projector, etc., for users to watch.

[0087] Through the above steps, the image pixel odd-even column spacing algorithm is applied to merge the two high-definition dynamic maps into an image with a specific visual effect.

[0088] Ensure that the fused image has an enhanced 3D stereoscopic visual effect.

[0089] The fused image is drawn to the top layer of the view through the widget for the user to view.

[0090] ‌Interaction‌: Receives two high-definition dynamic textures from the dual-lens image rendering component.

[0091] Image fusion is performed using an algorithm that arranges image pixels in odd and even columns.

[0092] The fused image is sent to the view layer for display.

[0093] Interaction process: The dual-lens scene capture component captures left and right eye scene images and depth information of the target object.

[0094] The processor calculates the target focus point and rotation value according to the depth information, and controls the lens spacing and rotation adjustment module to adjust the lens.

[0095] The Dual-Camera Rendering component renders the captured image onto a high-definition dynamic texture (optionally optimizing the rendering based on focus point and rotation values).

[0096] The left-eye and right-eye image fusion module uses an algorithm to fuse the two maps into a 3D stereoscopic vision-enhanced image, and draws it to the top layer of the view for users to watch.

[0097] Through such detailed component division and interaction process design, it can be ensured that the system can efficiently achieve the enhanced effect of 3D stereoscopic vision and provide users with an immersive visual experience.

[0098] The processor is further configured to: According to the target object depth information, the actual distance of the target object from the lens is calculated, and the scene depth of the target object is calculated according to the actual distance.

[0099] The 3D stereo plug-in also includes: A lens axis component, used to control the lens axis of the dual-lens scene capture component to be mapped to the corresponding scene depth; The lens axis assembly is communicatively connected to the processor.

[0100] Here, the scene depth (the distance of the object from the lens) is divided into 5 depth levels and mapped to a virtual depth axis facing the front of the lens. Objects that require different degrees of 3D vision are placed in the corresponding depth level slots to enhance the 3D vision.

[0101] like Figure 2 As shown, the scene depth is based on lens VP_0 and is advanced by 2 depths, namely VP_1 and VP_2. By controlling the lens axis component, the lens axis of the dual-lens scene capture component can be mapped to the corresponding scene depth according to the calculated scene depth. The specific selection depends on the target object.

[0102] By emitting rays from the lens and colliding with objects, the depth of the object is obtained, the actual distance between the core object and the lens and the proportion of the screen are judged, and the rotation value (focusing point) and the optimal distance (eye distance) of the dual lenses are automatically adjusted to enhance the three-dimensional effect and meet the user's comfortable visual experience.

[0103] Based on the application of the above-mentioned device, the embodiment of the present invention provides a 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application, which can be implemented by an electronic device, which can be a terminal or a server. Figure 3 The flowchart of the 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application is shown. The processing flow of the method may include the following steps: S1. Activate the 3D stereo plug-in in the 3D camera background and obtain the depth information of the target object through the dual-lens scene capture component; S2, calculating the target focus point and the target rotation value of the dual-lens scene capture component according to the depth information of the target object, according to the rotation value of the preset lens and the eye distance function, and controlling the dual-lens scene capture component to face the target object; S3, capturing a corresponding scene image through the dual-lens scene capture component; S4, rendering the scene image captured by the dual-lens scene capture component onto two preset high-definition dynamic maps; S5. By using an image pixel odd-even column spacing algorithm, the two high-definition dynamic maps are merged into one image, and the merged image is drawn to the top layer of the view through a widget to obtain a 3D stereoscopic vision enhanced image; S6: Send the 3D stereoscopic vision enhanced image to a 3D terminal.

[0104] During the specific implementation, combined with the above device application description, the three-dimensional world position of the object is obtained as the focusing point and passed into the function provided by the dual-lens component. The function performs three-dimensional matrix mathematical operations on the dual-lens component, calculates the target rotation value of the dual-lens, and then sets it in the dual-lens component to drive the lens rotation so that the lens faces the target point.

[0105] Render the images captured by the camera in real time onto two high-definition dynamic maps; Through the image pixel odd-even column spacing algorithm, two high-definition dynamic maps are merged into one image; Draw the fused image to the top layer of the view through the Widget; Launch the application and browse it on a polarized display. Without affecting the original functional interaction of the three-dimensional application, the user can see the clear and comfortable 3D stereoscopic effect by wearing polarized glasses.

[0106] Figure 4 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 4 As shown, the electronic device may include the above Figure 3 The 3D stereoscopic vision enhancement optimization processing device based on digital twin three-dimensional application is shown. Optionally, the electronic device 410 may include a first processor 2001.

[0107] Optionally, the electronic device 410 may further include a memory 2002 and a transceiver 2003 .

[0108] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0109] Combine the following Figure 4 The components of the electronic device 410 are described in detail: The first processor 2001 is the control center of the electronic device 410, and may be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or may be application specific integrated circuits (ASICs), or may be one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0110] Optionally, the first processor 2001 can execute various functions of the electronic device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0111] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0112] In a specific implementation, as an embodiment, the electronic device 410 may also include multiple processors, such as Figure 4 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0113] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0114] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently and access the first processor 2001 through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0115] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0116] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0117] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0118] It should be noted that Figure 4 The structure of the electronic device 410 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or fewer components than those shown in the figure, or combine certain components, or arrange the components differently.

[0119] In addition, the technical effects of the electronic device 410 can refer to the technical effects of the 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional applications described in the above method embodiment, and will not be repeated here.

[0120] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0121] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0122] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0123] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0124] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0125] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0126] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0127] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0128] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0131] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0132] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application, characterized in that: The method comprises: S1. Activate the 3D stereo plug-in in the 3D camera background and obtain the depth information of the target object through the dual-lens scene capture component; S2, according to the depth information of the target object, according to the rotation value of the preset lens and the eye distance function, calculating the target focus point and the target rotation value of the dual-lens scene capture component, and controlling the dual-lens scene capture component to face the target object; S3, capturing a corresponding scene image through the dual-lens scene capture component; S4, rendering the scene image captured by the dual-lens scene capture component onto two preset high-definition dynamic maps; S5. By using an image pixel odd-even column spacing algorithm, the two high-definition dynamic maps are merged into one image, and the merged image is drawn to the top layer of the view through a widget to obtain a 3D stereoscopic vision enhanced image; S6: Send the 3D stereoscopic vision enhanced image to a 3D terminal.

2. The 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application according to claim 1 is characterized in that: Target object depth information, including: the actual distance between the target object and the lens and the screen ratio.

3. The 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application according to claim 1 is characterized in that: The 3D terminal is polarized glasses.

4. The 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application according to claim 2 is characterized in that: After obtaining the depth information of the target object, it also includes: Calculating the actual distance of the target object from the lens according to the target object depth information, and calculating the scene depth of the target object according to the actual distance; Through the lens axis component, the lens axis of the dual-lens scene capture component is controlled to be mapped to the corresponding scene depth.

5. A 3D stereoscopic vision enhancement optimization processing device based on digital twin three-dimensional application, the device is applied to the 3D stereoscopic vision enhancement optimization processing method based on digital twin three-dimensional application according to any one of claims 1 to 4, characterized in that: The device includes: A 3D stereo plug-in and a 3D terminal, wherein the 3D stereo plug-in is deployed on a digital twin application server, and the 3D stereo plug-in includes: A dual-lens scene capture component is used to simulate the left and right eyes to obtain the depth information of the target object; and to capture the left and right eye scene images corresponding to the target object; A processor, configured to calculate a target focus point and a target rotation value of the dual-lens scene capture assembly according to the target object depth information and a rotation value of a preset lens and an eye distance function; A lens spacing and rotation adjustment module, used to control the dual-lens scene capture assembly to face the target object; A dual-lens image rendering component, used for rendering the scene image captured by the dual-lens scene capturing component onto two preset high-definition dynamic maps; The left-eye and right-eye image fusion module is used to fuse the two high-definition dynamic maps into one image through an image pixel odd-even column spacing algorithm, and draw the fused image to the top layer of the view through a widget to obtain a 3D stereoscopic vision enhanced image; The dual-lens scene capture component, the lens spacing and rotation adjustment module, the dual-lens image rendering component and the left-right eye image fusion module are respectively connected to the processor for communication.

6. The 3D stereoscopic vision enhancement optimization processing device based on digital twin three-dimensional application according to claim 5 is characterized in that: The processor is further configured to: According to the target object depth information, the actual distance of the target object from the lens is calculated, and the scene depth of the target object is calculated according to the actual distance.

7. The 3D stereoscopic vision enhancement optimization processing device based on digital twin three-dimensional application according to claim 5 is characterized in that: The 3D stereo plug-in also includes: A lens axis component, used to control the lens axis of the dual-lens scene capture component to be mapped to the corresponding scene depth; The lens axis assembly is communicatively connected to the processor.

8. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 4 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 4.

Images