A 3D model dynamic loading method using Unity engine

Abstract

The application relates to the technical field of model loading, in particular to a 3D model dynamic loading method using a Unity engine, which comprises the following steps: obtaining an OSGB file of a three-dimensional model reconstructed from image data, importing the OSGB file into the Unity engine after format conversion, and constructing a tree index structure by using a file naming rule, wherein each node is associated with a corresponding mesh model and a map image; obtaining all nodes located within the field of view of a main camera at each moment during the runtime of the Unity engine; calculating the field of view fading degree, visual richness, visual saliency, motion coefficient and rendering evaluation value of each node, optimizing a node evaluation function of an LOD algorithm, and dynamically scheduling and loading the mesh model and the map image of a corresponding detail level according to the evaluation result of the node to perform rendering. The application can guarantee high-performance rendering, and improve the realism and immersion of a scene picture.

Description

Technical Field

[0001] This application relates to the field of model loading technology, specifically to a method for dynamically loading 3D models using the Unity engine. Background Technology

[0002] With the rapid development of smart cities, digital twins, and entertainment games, large-scale, high-precision 3D models have become a core element in building immersive virtual environments. However, the Unity engine requires enormous computing power to load and render 3D models. To achieve smooth real-time loading of large-scale real-world models, multi-level detail technology can dynamically adjust rendering precision, thereby reducing the load on real-time rendering.

[0003] Traditional multi-level detail techniques typically rely solely on the static geometric relationship of spatial distance between nodes and the main camera to adjust the level of detail. This fails to effectively distinguish the inherent differences in visual importance within a scene, resulting in excessively high rendering precision being allocated to objects that are close but have simple visual textures, leading to a waste of computing power. Conversely, objects that are far away but have complex structures and rich details appear blurry due to insufficient precision. This misalignment between rendering resource allocation and the focus of human visual perception means that limited computing power is not used to improve the image quality of the areas that users care about most. At the same time, it cannot perceive and respond to the strong attraction of dynamic elements in the scene to the user's visual attention. This results in the same loading and rendering precision for dynamic and static areas at the same viewing distance. Consequently, dynamic areas suffer from detail distortion because they are not given higher rendering precision, which in turn reduces the realism of the scene and affects the dynamic loading effect of the model. Summary of the Invention

[0004] To address the aforementioned technical issues, a method for dynamically loading 3D models using the Unity engine is provided to resolve the existing problems.

[0005] The solution to the technical problem in this application is to provide a method for dynamically loading 3D models using the Unity engine, including the following steps:

[0006] Obtain the OSGB file of the 3D model reconstructed from the image data, convert the format and import it into the Unity engine, and build a tree-like index structure using file naming rules, where each node is associated with the corresponding mesh model and texture image; when the Unity engine is running, obtain all nodes within the field of view of the main camera at each time point;

[0007] For each node within the field of view of the main camera at each time point, analyze the spatial distance relationship between the mesh model of each node and the main camera, as well as the proportion of the field of view of the mesh model that is not occluded, and calculate the field of view fading degree of each node.

[0008] The visual richness of each node is calculated by evaluating the number and shape differences of different connected components corresponding to the texture image of each node, as well as the complexity of the color information contained in the texture image. Combined with the visual fading degree, the visual saliency of each node is obtained.

[0009] By analyzing the displacement changes of pixels in the texture image of the same node at adjacent time points, and the number of pixels with displacement, the motion coefficients of each node are calculated. Combined with the dynamic changes of the main camera at adjacent time points and the visual saliency, the rendering evaluation value of each node is calculated, and the node evaluation function of the LOD algorithm is optimized. Based on the evaluation results of nodes within the field of view of the main camera, the corresponding mesh model and texture image at the detail level are dynamically scheduled and loaded for rendering.

[0010] Preferably, the tree-like index structure is a quadtree structure.

[0011] Preferably, the calculation of the visual fade degree of each node includes:

[0012] Obtain the bounding box of the mesh model corresponding to each node; calculate the distance between the center position of each face in the bounding box and the main camera, and select the face corresponding to the minimum distance as the view plane;

[0013] The field of view is divided into multiple sub-regions. Line of sight occlusion is detected in each sub-region. The ratio of the number of sub-regions without line of sight occlusion to the total number of sub-regions in the field of view is calculated as the unoccluded percentage. The product of the area of ​​the field of view and the unoccluded percentage is used as the field of view percentage of each node.

[0014] Calculate the distance between the center of the bounding box of each node and the main camera;

[0015] The degree of visual field blurring is positively correlated with distance and negatively correlated with the proportion of the visual field.

[0016] Preferably, the calculation of the visual richness of each node includes:

[0017] Extract all connected components from the grayscale image corresponding to the texture image, and calculate the Hu moment value of each connected component;

[0018] Calculate the degree of dispersion of Hu moments for all connected components in a grayscale image;

[0019] Calculate the image entropy of the pixel values ​​of all pixels in each color channel of the texture image corresponding to each node; calculate the mean of the image entropy of all color channels as the average image entropy;

[0020] Count the number of all connected components in a grayscale image;

[0021] The visual richness is the result of fusing the number of images, the degree of dispersion, and the average image entropy.

[0022] Preferably, the specific process of the fusion is as follows: the product of the number, the degree of dispersion, and the average image entropy is used as the visual richness.

[0023] Preferably, the visual salience is the ratio of the visual richness to the visual field dilution.

[0024] Preferably, the calculation of the motion coefficients of each node includes:

[0025] Input the texture image corresponding to the same node at each time step and the previous time step into the optical flow algorithm to obtain the displacement vector of each pixel and calculate the magnitude of the displacement vector;

[0026] Obtain the segmentation threshold of the modulus of all pixels in the texture image corresponding to each node within the field of view of the main camera at each time point, and mark the pixels with a modulus greater than or equal to the segmentation threshold as moving pixels;

[0027] Statistical analysis of the percentage of moving pixels in the texture image corresponding to each node within the field of view of the main camera at each time point;

[0028] The motion coefficient is the normalized result of the product of the average value of the magnitude of all moving pixels in the texture image and the proportion of their number.

[0029] Preferred, the first At any given moment, it is within the field of view of the main camera. Rendering evaluation value of each node The calculation formula is: ,in, For the first At any given moment, it is within the field of view of the main camera. The rendering evaluation value of each node. For the first At any given moment, it is within the field of view of the main camera. The visual saliency of each node, For the first At any given moment, it is within the field of view of the main camera. The motion coefficients of each node As a preset weighting factor, For the first At any given moment, it is within the field of view of the main camera. The distance from each node to the main camera For the first At any given moment, it is within the field of view of the main camera. The distance from each node to the main camera.

[0030] The preferred and optimized formula for the node evaluation function is as follows: ,in, The evaluation results for the nodes, This represents the distance from the center of the node's bounding box to the main camera. Let be the side length of the node. This is the rendering evaluation value for the node.

[0031] Preferably, the dynamic scheduling and loading of the mesh model and texture image corresponding to the detail level for rendering includes: if the evaluation result of each node within the field of view of the main camera at each time is less than 1, then the detail level of the node is subdivided to load and render the descendant nodes; if the evaluation result is greater than or equal to 1, then the detail level of the node is maintained and rendered.

[0032] This application has at least the following beneficial effects:

[0033] This application loads nodes within the main camera's field of view in real time, significantly reducing computational load, avoiding unnecessary resource loading and rendering, and improving frame rate and overall performance. Calculating the field-of-view fading of each node effectively distinguishes between nodes that are close but have a small visible area and those that are far but have a large visible area by quantifying the distance between the node and the camera and the proportion of the mesh model that is not occluded in the field of view. This reflects the importance of each node within the field of view, more accurately assesses the spatial visual influence of nodes on the screen, and prioritizes rendering resources for nodes with a larger visible area, avoiding over-allocation of high precision to a model that is close but actually occluded or occupies only a small portion, thus improving rendering efficiency. Calculating the visual richness of each node allows for greater focus on detailed areas by quantifying the structural complexity and color information of textures, thereby improving visual realism while maintaining high performance. Obtaining the visual saliency of each node identifies truly visually compelling focal points in the scene from multiple dimensions, ensuring that rendering is prioritized for areas with these focal points, significantly improving the rendering quality of areas where user attention is focused. Surface quality; calculating the motion coefficients of each node, the beneficial effect of which is to dynamically perceive and capture the motion state of the node by considering the displacement of pixels of the same node in the texture image at adjacent time points when the main camera is stationary, so as to ensure that dynamic elements can be presented with more refined models, avoid motion blur or loss of details, and enhance the interactive experience; calculating the rendering evaluation value of each node, the beneficial effect of which is to prevent over-adjustment of details for static or low-speed nodes by combining the dynamic changes of the main camera, and to evaluate the possibility that the node will become the focus of the user's visual attention due to its own motion characteristics and texture characteristics by comprehensively considering visual and motion characteristics, effectively preserving dynamic details and improving the realism and immersion of the scene; optimizing the node evaluation function of the LOD algorithm, dynamically scheduling and loading the mesh model and texture image of the corresponding detail level for rendering based on the evaluation results of nodes within the field of view of the main camera, the beneficial effect of which is to accurately direct limited computing power to the area of ​​most user visual attention based on the importance of nodes to visual impact, and solve the problem of detail distortion caused by insufficient loading accuracy in dynamic areas, significantly improving the visual fidelity of the image and the realism of the scene while ensuring real-time rendering performance. Attached Figure Description

[0034] The following section provides a more detailed description of a method for dynamically loading 3D models using the Unity engine, in conjunction with the accompanying drawings.

[0035] Figure 1 A flowchart illustrating the steps of a method for dynamically loading 3D models using the Unity engine, as provided in this application embodiment;

[0036] Figure 2A flowchart illustrating the steps of the method for obtaining the visual saliency of each node provided in the embodiments of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description, in conjunction with the accompanying drawings and implementation examples, provides a method for dynamically loading 3D models using the Unity engine. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0039] Please see Figure 1 The diagram illustrates a flowchart of a method for dynamically loading 3D models using the Unity engine, according to an embodiment of this application. The method includes the following steps:

[0040] Step 1: Obtain the OSGB file of the 3D model reconstructed from the image data, convert the format and import it into the Unity engine, and build a tree-like index structure using file naming rules, where each node is associated with the corresponding mesh model and texture image; when the Unity engine is running, obtain all nodes within the field of view of the main camera at each time point.

[0041] With the development of virtual reality technology, the real world can be realistically simulated in computers, merging humans and computers and placing people in the real world. This technology can realistically simulate real-world environments, allowing users to immerse themselves in the virtual environment through a combination of multiple senses, while also being highly interactive. To construct virtual environments that closely resemble reality, people have been constantly exploring how to create more realistic 3D models. Oblique photogrammetry technology can quickly and efficiently acquire rich image and other data information, and use computing clusters to reconstruct high-precision real-world 3D models, making the creation of large-scale, high-fidelity virtual environments possible.

[0042] Based on the above analysis, the five-lens tilt camera mounted on the drone was used to collect image data of the target scene area, and the image data was input into ContextCapture software to generate a textured 3D model.

[0043] In this embodiment, the ground resolution of the five-lens tilt camera is required to be within 5 cm, the image forward overlap is above 80%, and the lateral overlap is above 70%. As for other implementation methods, the implementer can set them according to the actual situation. The process of using ContextCapture software to construct a real-scene 3D model is a well-known technology and will not be described in detail here.

[0044] Since the models built by ContextCapture software are usually high-precision but static meshes that lack interactivity and dynamic behavior, while Unity engine, as a powerful, integrated real-time interactive content creation platform and game engine, can achieve dynamic interaction, importing a real-world 3D model into Unity can transform a static, non-interactive model into a dynamic, interactive object that can be integrated into a virtual world with complex rules, thereby achieving virtual-real synchronization and real-time roaming.

[0045] Secondly, loading and rendering 3D models in virtual scenes requires enormous computing power. Therefore, how to load and render tilted 3D models quickly and smoothly is a challenge. To achieve real-time and smooth loading of large-scale real-world models, Level of Detail (LOD) technology can solve the model loading problem by reducing the complexity of the model. Its purpose is to dynamically adjust the rendering precision according to the spatial relationship between the model and the observer. Specifically, 3D models closer to the observer are rendered with higher precision, while those farther away are rendered with lower precision, thereby reducing the overall rendering pressure and optimizing the virtual scene experience.

[0046] The LOD algorithm based on a quadtree structure recursively divides the entire scene into four quadrants, forming a multi-level tree structure. The root node represents the coarsest global model, while the child nodes represent model blocks of a more refined level in the region corresponding to their parent node. During rendering, a preset evaluation criterion is used to determine whether the level of detail of each node meets the current requirements. When the parent node cannot meet the precision requirements, four higher-precision child nodes are loaded and the parent node is replaced for rendering. Conversely, when the precision provided by a child node far exceeds the requirements, it will be replaced by its parent node to merge rendering batches and reduce the computational load.

[0047] Since the Unity engine only supports FBX files and not OSGB files, the textured 3D model is exported using ContextCapture software. The generated OSGB file is then batch converted using osgb2fbx software to obtain an FBX file supported by the Unity engine, which contains the mesh model and the corresponding texture image.

[0048] It should be noted that the osgb2fbx software uses well-known techniques for format conversion, which will not be elaborated here; secondly, the ContextCapture software does not export a large OSGB file, but thousands of small OSGB files, each representing a node in a quadtree. The naming of these files themselves encodes their position in the quadtree. In addition, each OSGB file is accompanied by a texture image of its own small region.

[0049] By loading FBX files using the Unity platform and parsing the naming rules of the filenames, a quadtree index structure is constructed in the program logic, and each mesh model and texture image is mapped to a node of the quadtree.

[0050] It should be noted that the construction of quadtree index structures is a well-known technique, and will not be elaborated upon here.

[0051] When navigating using the Unity engine, the built-in view frustum culling method of Unity is used to obtain all nodes within the field of view of the main camera at each moment;

[0052] It should be noted that in the Unity engine, the main camera represents the player's field of view in the virtual scene, and the position of the main camera and some models may change dynamically during the roaming process. Therefore, updating the screen in real time at a frequency of 60fps / s is an alternative implementation method that implementers can set according to their actual situation. Secondly, the view frustum culling method is a well-known technology and will not be described in detail here.

[0053] At this point, we have obtained all the nodes within the field of view of the main camera at each time step, as well as the corresponding mesh models and texture images.

[0054] Step 2: For each node within the field of view of the main camera at each time point, analyze the spatial distance relationship between the mesh model of each node and the main camera, as well as the proportion of the field of view of the mesh model that is not obstructed, and calculate the field of view fading degree of each node.

[0055] Since each node in the quadtree corresponds to a different entity object in the 3D scene, and these entities have different geometric dimensions in space, even if they are within the same viewing distance of the main camera, their visual proportion in the field of view is not consistent. Considering that the entities represented by the mesh model usually have multiple faces, and some faces may not be visible to the main camera due to their orientation, we calculate the field of view fading degree by analyzing the proportion of the unoccluded area of ​​the mesh model corresponding to the node, as well as the distance of the node from the main camera, to preliminarily assess the rendering accuracy of the node. Specifically:

[0056] For each node within the main camera's field of view at any given time, the bounding box of the corresponding mesh model for each node is obtained using the mesh rendering component of the Unity engine.

[0057] In this embodiment, the bounding box type is AABB (Axis Aligned Bounding Box). The acquisition of the bounding box is a well-known technique and will not be described in detail here.

[0058] Calculate the distance between the center of each face in the bounding box and the main camera, and select the face with the minimum distance as the field of view face;

[0059] In this embodiment, the distance is calculated by measuring the Euclidean distance between the center position of each surface and the main camera. Euclidean distance is a well-known technique and will not be described in detail here.

[0060] The field of view is divided into multiple sub-regions. Using ray collision detection technology, a ray is emitted from the location of the main camera to each sub-region to detect line-of-sight occlusion. The ratio of the number of sub-regions without line-of-sight occlusion to the total number of sub-regions in the field of view is counted as the unoccluded percentage.

[0061] In this embodiment, the field of view is evenly divided into 12 sub-regions. As for other implementation methods, the implementer can set them according to the actual situation. Secondly, the ray collision detection method built into the Unity engine is a well-known technology and will not be described in detail here. By detecting whether the ray will collide with other objects before reaching the sub-region, it is possible to determine whether there is visual obstruction in the sub-region.

[0062] The product of the area of ​​the field of view and the proportion of the unobstructed area is used as the field of view proportion of each node.

[0063] At each time point, the degree of field of view of each node within the field of view of the main camera is positively correlated with distance and negatively correlated with field of view percentage;

[0064] It should be noted that a positive correlation means that the dependent variable increases as the independent variable increases and decreases as the independent variable decreases, while a negative correlation means that the dependent variable decreases as the independent variable increases and increases as the independent variable decreases.

[0065] In this embodiment, the square of the distance between the center of the bounding box of each node and the main camera is calculated, and the ratio of this distance to the field of view ratio is used as the field of view dilution of each node within the field of view of the main camera at each time. In other implementations, the ratio of the normalized distance to the normalized field of view ratio can be used as the field of view dilution of each node within the field of view of the main camera at each time.

[0066] It should be noted that the 3D spatial coordinate system used is Unity's default 3D spatial coordinate system. Therefore, the construction of the bounding box, the center position of each face, and the calculation of distances are all well-known techniques and will not be elaborated here.

[0067] It should be noted that, when calculating the ratio, to avoid the denominator being 0, a preset value greater than 0 is added to the denominator. In this embodiment, the preset value greater than 0 is 0.01. In other implementations, the implementer can set it according to the actual situation. Secondly, the field of view represents the surface of the mesh model corresponding to the node that is of interest at each time point in the view. The larger the area of ​​the field of view, the wider the area covered by the mesh model in the picture, and the greater the impact on the user's visual perception. The larger the unobstructed proportion, the higher the degree to which the field of view is not obstructed by objects in front at that time point in the view. The larger the obtained field of view proportion, the more the mesh model is visible in the picture. The greater the visibility, the higher the visual importance, and the more precise the rendering is required to improve the realism and detail of the image. The farther the node is from the main camera, the larger the square of the distance, indicating that the entity represented by the mesh model corresponding to the node is at a far distance. The resulting field-of-view fade reflects the degree to which the mesh model corresponding to the node needs to be faded at the user's visual perception level. The smaller the value, the closer the mesh model is to the user and the larger the display area of ​​the mesh model. Therefore, a higher loading precision and rendering resources should be set to improve image quality. Conversely, a larger value indicates a weaker importance, and more fading is required to improve rendering efficiency.

[0068] Thus, the degree of field of view dilution of each node within the field of view of the main camera at each time point is obtained.

[0069] Step 3: Evaluate the number and shape differences of different connected components corresponding to the texture images of each node, as well as the complexity of the color information contained in the texture images, calculate the visual richness of each node, and combine it with the visual fading degree to obtain the visual saliency of each node.

[0070] Furthermore, entities with distinctive appearance features, such as building outlines, doors and windows with large color differences, billboards, and graffiti, usually have higher visual appeal and therefore require higher loading accuracy. On the other hand, areas with uniform color and gentle texture, such as grass, wasteland, and water surfaces, usually have lower visual appeal and can have lower loading accuracy.

[0071] Furthermore, the flowchart of the method for obtaining the visual saliency of each node provided in the embodiments of this application is as follows: Figure 2 As shown.

[0072] First, perform connected component analysis on the texture image corresponding to the node, specifically:

[0073] For each node within the field of view of the main camera at each time point, its corresponding texture image is converted into a grayscale image, and all connected components in the grayscale image are extracted;

[0074] In this embodiment, a weighted average method is used for grayscale processing to obtain a grayscale image; a region growing method is used to extract connected components. Both the weighted average method and the region growing method are well-known techniques and will not be described in detail here.

[0075] Secondly, determine whether multiple connected components could be the same object, such as bricks or windows. If they are the same object, although there may be a large number of them, the overall visual salience will be low due to the lack of shape variation. Since the size of the same object is consistent, the structural complexity is calculated by analyzing the shape differences of different connected components and the number of connected components. Specifically:

[0076] The Hu moment function is used to calculate the Hu moment value of each connected component; and the dispersion of the Hu moment values ​​of all connected components in the grayscale image is calculated.

[0077] It should be noted that the Hu moment function is a well-known technique and will not be elaborated here. Since the Hu moment function can calculate 7 Hu moment values ​​for each connected component, forming a moment vector, the coefficient of variation of the moment vectors of all connected components in the same dimension is calculated, and the mean of the coefficients of variation in all dimensions is used as the degree of dispersion of the grayscale image. The calculation of the coefficient of variation is a well-known technique and will not be elaborated here. As other implementation methods, implementers can use other methods of existing technology, such as information entropy, etc. This embodiment does not impose any special restrictions on this.

[0078] Calculate the image entropy of the pixel values ​​of all pixels in each color channel of the texture image corresponding to each node; calculate the mean of the image entropy of all color channels as the average image entropy;

[0079] It should be noted that the texture image is an RGB image, and each pixel has pixel values ​​in three color channels: R, G, and B. Secondly, the calculation of image entropy is a well-known technique, which will not be elaborated here.

[0080] The number of all connected components in the grayscale image is counted, and the product of the number, the degree of dispersion, and the average image entropy is used as the visual richness of each node within the field of view of the main camera at each time step.

[0081] It should be noted that the more connected components there are, the more objects with significant grayscale differences exist in the texture image, reflecting the greater the number of mutually separated objects in the texture image, and thus the greater the complexity of the internal structure of the texture image. The greater the degree of discreteness, the more obvious the shape differences between different connected components, reflecting the greater diversity of object shapes contained in the texture image. The greater the average image entropy, the more color information the texture image contains. The greater the visual richness, the more complex the structure and the richer the colors of the mesh model corresponding to the node, the higher the visual appeal, and the more high-precision rendering is needed to display image details and improve image quality.

[0082] Furthermore, based on visual fade and visual richness, visual salience is determined, specifically as follows:

[0083] The ratio of visual richness to visual fading is used as the visual salience of each node within the field of view of the main camera at each time point.

[0084] It should be noted that the greater the visual saliency, the more important the texture image corresponding to the node is in the picture, with rich structural and color details, and the closer it is to the main camera. Therefore, it needs to be rendered with high precision.

[0085] Thus, the visual salience of each node within the field of view of the main camera at each time point is obtained.

[0086] Step 4: Calculate the motion coefficients of each node by analyzing the displacement changes of pixels in the texture image of the same node at adjacent time points and the number of pixels with displacement. Combine this with the dynamic changes of the main camera at adjacent time points and the visual saliency to calculate the rendering evaluation value of each node. Optimize the node evaluation function of the LOD algorithm. Based on the evaluation results of nodes within the main camera's field of view, dynamically schedule and load the corresponding detail level mesh model and texture image for rendering.

[0087] In current game and film applications, to enhance scene realism, some environmental models, such as flowers, trees, and water surfaces, often dynamically change with wind or water flow. Considering that while visual saliency can assess node importance, it only analyzes static scenes and cannot effectively capture the visual importance of dynamic elements, this paper analyzes the displacement changes between the texture images corresponding to the same node at adjacent time points and calculates motion coefficients, specifically:

[0088] Input the texture image corresponding to the same node at each time step and the previous time step into the optical flow algorithm to obtain the displacement vector of each pixel and calculate the magnitude of the displacement vector;

[0089] In this embodiment, the optical flow algorithm adopts the KTL optical flow method. The KTL optical flow method and the calculation of the modulus are well-known technologies and will not be described in detail here. As other implementation methods, implementers may adopt other methods of existing technology, such as the Lucas-Kanade method, etc. This embodiment does not impose any special restrictions on this.

[0090] It should be noted that the magnitude of the displacement vector reflects the displacement change of the pixel between adjacent moments in the texture image; the larger the magnitude, the greater the displacement change of the pixel.

[0091] Obtain the segmentation threshold of the modulus of all pixels in the texture image corresponding to each node within the field of view of the main camera at each time point;

[0092] In this embodiment, the Otsu threshold segmentation method is used to obtain the segmentation threshold. The Otsu threshold segmentation method is a well-known technology and will not be described in detail here. As other implementation methods, implementers may use other methods of the prior art, such as cross-validation. This embodiment does not impose any special restrictions on this.

[0093] Pixels with a modulus greater than or equal to the segmentation threshold are marked as moving pixels;

[0094] Statistical analysis of the percentage of moving pixels in the texture image corresponding to each node within the field of view of the main camera at each time point;

[0095] The normalized product of the average value of the modulus of all moving pixels in the texture image and the proportion of their number is used as the motion coefficient of each node within the field of view of the main camera at each time step.

[0096] In this embodiment, the sigmoid function is used for normalization. The sigmoid function is a well-known technique and will not be described in detail here.

[0097] It should be noted that the larger the percentage of the number, the higher the proportion of the area where the texture image is moving, that is, there is a large area where significant movement has occurred; the larger the average value, the more intense the average movement amplitude of all moving pixels; the larger the obtained motion coefficient, the more the mesh model part corresponding to the node not only has a large area in motion, but also the amplitude of the movement is very significant, making it easier to attract user attention and increasing its visual importance.

[0098] It should be noted that the motion coefficient is set to 1 at the initial moment of the roaming.

[0099] Furthermore, based on the above analysis, and using visual saliency and motion coefficients, the rendering evaluation value is calculated as follows:

[0100]

[0101] in, For the first At any given moment, it is within the field of view of the main camera. The rendering evaluation value of each node. For the first At any given moment, it is within the field of view of the main camera. The visual saliency of each node, For the first At any given moment, it is within the field of view of the main camera. The motion coefficients of each node As a preset weighting factor, For the first At any given moment, it is within the field of view of the main camera. The distance from each node to the main camera For the first At any given moment, it is within the field of view of the main camera. The distance from each node to the main camera;

[0102] In this embodiment, the preset weight factor is set to 2. Since the normalized motion coefficient ranges from 0.5 to 1, increasing the motion coefficient by setting the preset weight factor makes... The value range is between 1 and 2, so as to improve the rendering accuracy of nodes and avoid calculation errors when the main camera is stationary.

[0103] It should be noted that when the main camera is stationary, the motion changes of nodes in the texture image are entirely due to the node's own movement, such as object movement. In this case, a high motion coefficient truly reflects the dynamic visual appeal of the node, and the mesh model corresponding to that node requires higher precision rendering to ensure the visual experience of stationary nodes. However, when the main camera is moving, the motion changes of nodes in the texture image are mainly due to camera movement, and the motion coefficient cannot reflect the node's own movement. In this case, the node's motion characteristics are not analyzed to avoid errors. Therefore, the higher the obtained rendering evaluation value, the more likely the node is to become the focus of the user's visual attention due to its own motion and texture characteristics, and the more precise the rendering is required to ensure the display effect of the image.

[0104] Furthermore, the formula for the node evaluation function in the traditional quadtree-based LOD algorithm is as follows:

[0105]

[0106] in, This represents the distance from the center of the node's bounding box to the main camera. Let d be the side length of the node, where d is the width of the node's bounding box. A constant used to control the fineness of the entire LOD loading process. The larger the value, the higher the level of detail required for the current scene to be drawn.

[0107] It should be noted that the LOD algorithm based on quadtrees is a well-known technology and will not be elaborated upon here.

[0108] Therefore, based on the rendering evaluation values, the node evaluation function of the quadtree-based LOD algorithm is optimized. The node evaluation function is as follows:

[0109]

[0110] in, The evaluation results for the nodes, This represents the distance from the center of the node's bounding box to the main camera. Let be the side length of the node. This is the rendering evaluation value for the node;

[0111] Based on the node evaluation function, the evaluation result of each node is calculated as follows:

[0112]

[0113] in, For the first At any given moment, it is within the field of view of the main camera. Evaluation results of each node For the first At any given moment, it is within the field of view of the main camera. The distance from the center of the bounding box of each node to the main camera. For the first At any given moment, it is within the field of view of the main camera. The side length of each node, For the first At any given moment, it is within the field of view of the main camera. The rendering evaluation value of each node;

[0114] For the Each node, if the evaluation result If the value is less than 1, the node needs further subdivision. This involves traversing the node's four child nodes and determining if the child node's precision has reached the finest level of the quadtree structure. If the child node is at the finest level, it is loaded and displayed directly; otherwise, the child node is iterated and evaluated until the evaluation result of the descendant node meets the precision requirement or reaches the finest level of the quadtree structure. If the evaluation result... If the value is greater than or equal to 1, then the node does not need to be subdivided and the currently loaded precision level remains unchanged. When the main camera moves or the scene changes dynamically, the Unity engine loads and renders the nodes within the main camera's field of view in real time as needed, thus achieving dynamic loading of the overall 3D model.

[0115] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this application, without departing from the content of the technical solution of this application, shall fall within the protection scope of the technical solution of this application.

Claims

1. A method for dynamically loading 3D models using the Unity engine, characterized in that, The method includes the following steps: Obtain the OSGB file of the 3D model reconstructed from the image data, convert the format and import it into the Unity engine, and build a tree-like index structure using file naming rules, where each node is associated with the corresponding mesh model and texture image; when the Unity engine is running, obtain all nodes within the field of view of the main camera at each time point; For each node within the field of view of the main camera at each time point, analyze the spatial distance relationship between the mesh model of each node and the main camera, as well as the proportion of the field of view of the mesh model that is not occluded, and calculate the field of view fading degree of each node. The visual richness of each node is calculated by evaluating the number and shape differences of different connected components corresponding to the texture image of each node, as well as the complexity of the color information contained in the texture image. Combined with the visual fading degree, the visual saliency of each node is obtained. By analyzing the displacement changes of pixels in the texture image of the same node at adjacent time points and the number of pixels that have displacement, the motion coefficient of each node is calculated. Combined with the dynamic changes of the main camera at adjacent time points and the visual saliency, the rendering evaluation value of each node is calculated, and the node evaluation function of the LOD algorithm is optimized. Based on the evaluation results of nodes within the field of view of the main camera, the corresponding mesh model and texture image at the detail level are dynamically scheduled and loaded for rendering. No. At any given moment, it is within the field of view of the main camera. Rendering evaluation value of each node The calculation formula is: ,in, For the first At any given moment, it is within the field of view of the main camera. The rendering evaluation value of each node. For the first At any given moment, it is within the field of view of the main camera. The visual saliency of each node, For the first At any given moment, it is within the field of view of the main camera. The motion coefficients of each node As a preset weighting factor, For the first At any given moment, it is within the field of view of the main camera. The distance from each node to the main camera For the first At any given moment, it is within the field of view of the main camera. The distance from each node to the main camera.

2. The method for dynamically loading 3D models using the Unity engine as described in claim 1, characterized in that, The tree-like index structure is a quadtree structure.

3. The method for dynamically loading 3D models using the Unity engine as described in claim 1, characterized in that, The calculation of the field-of-view fading degree for each node includes: Obtain the bounding box of the mesh model corresponding to each node; calculate the distance between the center position of each face in the bounding box and the main camera, and select the face corresponding to the minimum distance as the view plane; The field of view is divided into multiple sub-regions. Line of sight occlusion is detected in each sub-region. The ratio of the number of sub-regions without line of sight occlusion to the total number of sub-regions in the field of view is calculated as the unoccluded percentage. The product of the area of ​​the field of view and the unoccluded percentage is used as the field of view percentage of each node. Calculate the distance between the center of the bounding box of each node and the main camera; The degree of visual field blurring is positively correlated with distance and negatively correlated with the proportion of the visual field.

4. The method for dynamically loading 3D models using the Unity engine as described in claim 1, characterized in that, The calculation of the visual richness of each node includes: Extract all connected components from the grayscale image corresponding to the texture image, and calculate the Hu moment value of each connected component; Calculate the degree of dispersion of Hu moments for all connected components in a grayscale image; Calculate the image entropy of the pixel values ​​of all pixels in each color channel of the texture image corresponding to each node; calculate the mean of the image entropy of all color channels as the average image entropy; Count the number of all connected components in a grayscale image; The visual richness is the result of fusing the number of images, the degree of dispersion, and the average image entropy.

5. The method for dynamically loading 3D models using the Unity engine as described in claim 4, characterized in that, The specific process of the fusion is as follows: the product of the number, the degree of dispersion, and the average image entropy is used as the visual richness.

6. The method for dynamically loading 3D models using the Unity engine as described in claim 1, characterized in that, The visual salience is the ratio of the visual richness to the visual field dilution.

7. The method for dynamically loading 3D models using the Unity engine as described in claim 1, characterized in that, The calculation of the motion coefficients of each node includes: Input the texture image corresponding to the same node at each time step and the previous time step into the optical flow algorithm to obtain the displacement vector of each pixel and calculate the magnitude of the displacement vector; Obtain the segmentation threshold of the modulus of all pixels in the texture image corresponding to each node within the field of view of the main camera at each time point, and mark the pixels with a modulus greater than or equal to the segmentation threshold as moving pixels; Statistical analysis of the percentage of moving pixels in the texture image corresponding to each node within the field of view of the main camera at each time point; The motion coefficient is the normalized result of the product of the average value of the magnitude of all moving pixels in the texture image and the proportion of their number.

8. The method for dynamically loading 3D models using the Unity engine as described in claim 1, characterized in that, The optimized node evaluation function formula is as follows: ,in, The evaluation results for the nodes, This represents the distance from the center of the node's bounding box to the main camera. Let be the side length of the node. This is the rendering evaluation value for the node.

9. A method for dynamically loading 3D models using the Unity engine as described in claim 1, characterized in that, The dynamic scheduling and loading of mesh models and texture images corresponding to the level of detail for rendering includes: if the evaluation result of each node within the field of view of the main camera at each time point is less than 1, then the level of detail of that node is subdivided to load and render the descendant nodes; if the evaluation result is greater than or equal to 1, then the level of detail of that node is maintained and rendered.

Images