Data processing method and device for scene visualization

Abstract

The invention discloses a data processing method and device for scene visualization, and the method comprises the steps: determining a scheduling scene type when a scene visualization demand is received, determining a corresponding scheduling mechanism according to the scheduling scene type, scheduling a corresponding node set according to the scheduling mechanism, and carrying out the loading display, so as to complete the visualization process of a scene, the scheduling mechanism is an internal and external memory scheduling mechanism based on a leap-over type detail level model, and the scheduling mechanism at least comprises a direct scheduling mechanism, a scheduling mechanism of a father node substitution method or a scheduling mechanism of a knapsack problem solving method.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and more specifically, to a data processing method and device for scene visualization. Background Art

[0002] In scene construction, the scene visualization system is an indispensable part for the demand for real-time display of scene data. With the continuous development of computer graphics, a type of scene visualization system represented by 3D rendering engines has appeared in front of people. They provide great support for scene visualization and achieve the goal of visualizing various scenes. In various scene visualization systems, in order to meet the needs of large-scale scene visualization and keep the system running smoothly, they usually have the function of scheduling the internal and external memory of scene model data.

[0003] The existing scene visualization system has a way of scheduling internal and external memory for large-scale scene data, such as the internal and external memory scheduling mechanism implemented by the 3D rendering engine through DatabasePager. As for the existing 3D rendering engine, although it can schedule the internal and external memory data of the scene model and complete the scene rendering task, its internal and external memory scheduling mechanism for the level of detail model will load too many intermediate level models during the scheduling process, increasing the amount of data loaded and the time for users to wait for the display of the high-resolution level of detail model (LOD), thereby reducing the user experience.

[0004] Therefore, how to improve user experience in the process of internal and external memory data scheduling implemented by the scene visualization system is a problem that needs to be urgently solved in this application. Summary of the invention

[0005] In view of this, the present application discloses a data processing method and device for scene visualization, aiming to achieve an optimal balance between operating efficiency and rendering effect of the scene visualization system, thereby improving user experience in the process of internal and external memory data scheduling implemented by the scene visualization system.

[0006] In order to achieve the above purpose, the disclosed technical solution is as follows:

[0007] In a first aspect, the present application discloses a data processing method for scene visualization, the method comprising:

[0008] When a scene visualization requirement is received, the scheduling scene type is determined;

[0009] Determine a corresponding scheduling mechanism according to the scheduling scenario type;

[0010] The corresponding node set is scheduled to be loaded and displayed according to the scheduling mechanism to complete the visualization process of the scene; wherein the scheduling mechanism is an internal and external memory scheduling mechanism based on a leap-forward detail level model; the scheduling mechanism at least includes a direct scheduling mechanism, a parent node replacement method scheduling mechanism or a knapsack problem solving method scheduling mechanism.

[0011] Preferably, when receiving a scene visualization requirement, determining the scheduling scene type includes:

[0012] When a scene visualization requirement is received, if the expected node set is directly loaded within the time limit, the scheduling scene type is determined to be a scheduling scene type for loading the expected node set;

[0013] If the expected node set cannot be directly loaded within the time limit, the scheduling scenario type is determined to be a scheduling scenario type for generating a substitute node set for the expected node set.

[0014] Preferably, determining a corresponding scheduling mechanism according to the scheduling scenario type includes:

[0015] If the scheduling scenario type is a scheduling scenario type for loading an expected node set, determining a scheduling mechanism corresponding to the scheduling mechanism, a direct scheduling mechanism;

[0016] If the scheduling scenario type is a scheduling scenario type for generating a replacement node set for an expected node set, its corresponding scheduling mechanism is determined to be a scheduling mechanism of a parent node replacement method or a scheduling mechanism of a knapsack problem solving method.

[0017] Preferably, the process of scheduling the corresponding node set to load and display according to the scheduling mechanism to complete the visualization process of the scene includes:

[0018] If the scheduling mechanism is a direct scheduling mechanism, all intermediate levels are crossed to load and display the expected node set to complete the visualization of the scene;

[0019] If the scheduling mechanism is a parent node replacement method, an ancestor node set is generated, and the ancestor node set is loaded and displayed as a replacement node set to complete the visualization of the scene; or,

[0020] If the scheduling mechanism is a scheduling mechanism of the knapsack problem solving method, the optimal set of replacement nodes is selected for loading and display to complete the visualization of the scene.

[0021] Preferably, if the scheduling mechanism is a scheduling mechanism of the parent node replacement method, an ancestor node set is generated, and the ancestor node set is loaded and displayed as a replacement node set to complete the visualization of the scene, including:

[0022] The parent nodes of the nodes in the expected node set are combined into a set;

[0023] Determine whether the set only has a root node;

[0024] If yes, then end the loading and display process;

[0025] If not, process the illegal nodes to generate a set of legal scheduling nodes;

[0026] Counting the load of the legal scheduling node set, and determining whether the load is less than or equal to the load that can be loaded by the alternative legal scheduling node set within the limited time;

[0027] If the load amount is less than or equal to the amount that the substitute legal scheduling node set can load within the time limit, the generated legal scheduling node set is loaded and displayed as the substitute node set to complete the visualization of the scene;

[0028] If the load is greater than what the alternative legal scheduling node set can load within the time limit, the generated legal scheduling node set is used as the expected node set, and the step of combining the expected nodes with the parent nodes of the middle nodes to form a set is performed.

[0029] Preferably, if the scheduling mechanism is a scheduling mechanism of a knapsack problem solving method, selecting an optimal set of replacement nodes for loading and displaying to complete the visualization of the scene includes:

[0030] Obtaining a preset legal scheduling node set by offline processing, and using the preset legal scheduling node set as an item template set;

[0031] When the scene visualization system is running, the expected node set and node state table are generated according to the visibility calculation and detail level calculation;

[0032] Generating a legal scheduling node set as an input of a multi-select knapsack problem from the item template set in real time according to the expected node set and the node state table;

[0033] The legal scheduling node set is solved to obtain the optimal replacement node set and load it for display to complete the visualization of the scene.

[0034] A second aspect of the present application discloses a data processing device for scene visualization, the device comprising:

[0035] A first determining unit, configured to determine a scheduling scene type when a scene visualization requirement is received;

[0036] A second determining unit, configured to determine a corresponding scheduling mechanism according to the scheduling scenario type;

[0037] A visualization unit is used to schedule the corresponding node set to be loaded and displayed according to the scheduling mechanism to complete the visualization process of the scene; wherein the scheduling mechanism is an internal and external memory scheduling mechanism based on a leap-forward detail level model; the scheduling mechanism at least includes a direct scheduling mechanism, a parent node replacement method scheduling mechanism or a knapsack problem solving method scheduling mechanism.

[0038] Preferably, the first determining unit includes:

[0039] A first determination module is used to, when receiving a scene visualization requirement, determine that the scheduling scene type is a scheduling scene type for loading the expected node set if the expected node set is directly loaded within a limited time;

[0040] The second determination module is used to determine the scheduling scenario type as a scheduling scenario type for generating a substitute node set for the expected node set if the expected node set cannot be directly loaded within the limited time.

[0041] Preferably, the second determining unit includes:

[0042] A third determination module is used to determine the direct scheduling mechanism corresponding to the scheduling scenario type if the scheduling scenario type is a scheduling scenario type for loading an expected node set;

[0043] The fourth determination module is used to determine that the corresponding scheduling mechanism is a scheduling mechanism of a parent node replacement method or a scheduling mechanism of a knapsack problem solving method if the scheduling scenario type is a scheduling scenario type of generating a replacement node set for an expected node set.

[0044] Preferably, the visualization unit comprises:

[0045] A first visualization module, for loading and displaying the desired node set across all intermediate levels to complete the visualization of the scene if the scheduling mechanism is a direct scheduling mechanism;

[0046] A second visualization module is used to generate an ancestor node set if the scheduling mechanism is a parent node replacement method scheduling mechanism, and load and display the ancestor node set as a replacement node set to complete the visualization of the scene; or,

[0047] The third visualization module is used to select the optimal set of replacement nodes for loading and display if the scheduling mechanism is a scheduling mechanism of the knapsack problem solving method, so as to complete the visualization of the scene.

[0048] It can be seen from the above technical solution that the present application discloses a data processing method and device for scene visualization. When a scene visualization demand is received, the scheduling scene type is determined, the corresponding scheduling mechanism is determined according to the scheduling scene type, and the corresponding node set is scheduled for loading and display according to the scheduling mechanism to complete the scene visualization process, wherein the scheduling mechanism is an internal and external memory scheduling mechanism based on a leap-forward detail level model, and the scheduling mechanism includes at least a direct scheduling mechanism, a parent node replacement method scheduling mechanism, or a knapsack problem solving method scheduling mechanism.

[0049] Through the above scheme, since the scheduling mechanism is an internal and external memory scheduling mechanism based on the leap-forward detail level model, the corresponding scheduling mechanism is determined according to the scheduling scene type, and the corresponding node set is scheduled for loading and display according to the scheduling mechanism, which reduces the data loading amount and user waiting time. At the same time, this application studies the node scheduling optimization problem, and through two alternative node set generation methods, namely the parent node replacement method or the knapsack problem solving method, fully considers factors such as the external memory device reading capacity, model data loading amount, rendering effect after loading, and loading time, and more accurately selects the detail level model, so that the scene visualization system achieves the best balance between operating efficiency and rendering effect, thereby improving the user experience in the process of internal and external memory data scheduling implemented by the scene visualization system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0051] Figure 1 A flowchart of a data processing method for scene visualization disclosed in an embodiment of the present application;

[0052] Figure 2 This is an example diagram of the node scheduling optimization problem disclosed in the embodiment of the present application;

[0053] Figure 3 This is a flow chart of the parent node replacement method disclosed in the embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of a data processing device for scene visualization disclosed in an embodiment of the present application;

[0055] Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0058] As can be seen from the background technology, the existing scene visualization system has a method for scheduling internal and external memory of large-scale scene data, such as the internal and external memory scheduling mechanism implemented by the three-dimensional rendering engine through DatabasePager. As for the existing three-dimensional rendering engine, although it can perform scene model internal and external memory data scheduling and complete the scene rendering task, its level of detail model internal and external memory scheduling mechanism will load too many intermediate level models during the scheduling process, increasing the amount of data loaded and the time for users to wait for the display of high-resolution LOD models, thereby reducing the user experience. Therefore, how to improve the user experience in the process of internal and external memory data scheduling implemented by the scene visualization system is a problem that needs to be solved urgently in this application.

[0059] In order to solve the above problems, the present application discloses a data processing method and device for scene visualization. Since the scheduling mechanism is an internal and external memory scheduling mechanism based on a leap-forward detail level model, the corresponding scheduling mechanism is determined according to the scheduling scene type, and the corresponding node set is scheduled for loading and display according to the scheduling mechanism, which reduces the data loading amount and user waiting time. At the same time, the present application studies the node scheduling optimization problem. Through two alternative node set generation methods, namely the parent node replacement method or the knapsack problem solving method, the external memory device reading capacity, the model data loading amount, the rendering effect after loading, the loading time and other factors are fully considered to more accurately select the detail level model, so that the scene visualization system can achieve the best balance between operating efficiency and rendering effect, thereby improving the user experience in the process of internal and external memory data scheduling implemented by the scene visualization system. The specific implementation method is specifically described through the following embodiments.

[0060] refer to Figure 1As shown, a data processing method for scene visualization disclosed in an embodiment of the present application is provided. The data processing method for scene visualization mainly includes the following steps:

[0061] S101: When a scene visualization requirement is received, a scheduling scene type is determined.

[0062] The specific process of determining the scheduling scenario type is shown in A1-A2.

[0063] A1: When a scene visualization requirement is received, if the expected node set is directly loaded within the time limit, the scheduling scene type is determined to be the scheduling scene type for loading the expected node set.

[0064] Among them, the time limit can be 5 seconds, 1 minute, etc. The time limit is set according to the actual situation and this application does not make any specific limitation.

[0065] A2: If the expected node set cannot be directly loaded within the time limit, the scheduling scenario type is determined to be a scheduling scenario type for generating an alternative node set for the expected node set.

[0066] S102: Determine a corresponding scheduling mechanism according to the scheduling scenario type.

[0067] In S102, if the scheduling scenario type is a scheduling scenario type for loading an expected node set, its corresponding scheduling mechanism is determined to be a direct scheduling mechanism; if the scheduling scenario type is a scheduling scenario type for generating an alternative node set for an expected node set, its corresponding scheduling mechanism is determined to be a scheduling mechanism of a parent node replacement method or a scheduling mechanism of a knapsack problem solving method.

[0068] The scheduling mechanism of the present application can read the serialized files generated in advance using an offline processing method, and organize the nodes into a detailed level node tree structure when the scene visualization system is initialized, so that the scene visualization system can obtain the ability to know the hierarchical relationship of each node in the entire scene without loading and parsing the node files one by one, thereby achieving the goal of cross-level loading of the LOD model.

[0069] S103: Scheduling the corresponding node set for loading and display according to the scheduling mechanism to complete the visualization process of the scene; wherein the scheduling mechanism is an internal and external memory scheduling mechanism based on the leap-forward detail level model; the scheduling mechanism at least includes a direct scheduling mechanism, a parent node replacement method scheduling mechanism or a knapsack problem solving method scheduling mechanism.

[0070] The scheduling mechanism in this application is a memory scheduling mechanism for the internal and external memory of the level of detail model that has the ability to load and display LOD across levels, that is, a leapfrogging memory scheduling mechanism for the level of detail model. Under the premise of considering the reading capacity of the external memory device, the LOD model in the scene is scheduled by skipping some LOD levels without loading, and the most suitable LOD model within the current reading capacity of the external memory device is found for loading and display, so as to achieve the goal of reducing the amount of external memory data loaded and the user's waiting time. Among them, LOD refers to the allocation of resources for object rendering based on the position and importance of the nodes of the object model in the display environment, reducing the number of faces and detail of non-important objects, thereby obtaining efficient rendering operations.

[0071] The core of achieving the above goals is how to select the most appropriate LOD model for loading and display within the current external memory device reading capacity. Especially when the target level LOD model cannot be directly loaded and displayed, how to generate a replacement node set according to the leap rules of the internal and external memory scheduling mechanism of the leap-level detail model becomes the core part of achieving the above goals.

[0072] To address this issue, this patent application abstracts it into an optimization problem. It studies how to more accurately select the LOD model under known external memory reading speed, taking into full consideration factors such as the external memory device reading capacity, LOD model loading amount, rendering effect after loading, and loading time, so as to achieve the best balance between operating efficiency and rendering effect of the scene visualization system, thereby improving the user experience.

[0073] Before defining the node scheduling optimization problem, the following two concepts need to be clarified:

[0074] (Definition 1) Expected node set:

[0075] In a scene composed of n nodes (Tiles), after the visibility calculation and detail level calculation are performed according to the current viewpoint information, the set P consisting of the LOD nodes of each tile that meet the calculation results is i The set P = {P1, P2, P3, ..., P n}, called the expected node set.

[0076] (Definition 2) Legal scheduling node set:

[0077] In the tree graph G = (V, E) consisting of all LOD nodes of a Tile, for a LOD node set U = {u1, u2, u3, ..., u n}, In the tree, the node type of all nodes can only be one of the root node, leaf node and non-leaf node (non-root node). V is the set of vertices in the tree graph G; E is the set of edges in the tree graph G. The LOD node set U is a legal scheduling node set for this Tile if and only if all nodes ui in U meet the legal conditions of the corresponding node type.

[0078] The legal conditions corresponding to the three node types are as follows:

[0079] Legal conditions for the root node: u i It is the root node, and there is only this root node in the LOD node set, that is, U = {u i}.

[0080] Legal conditions for leaf nodes: u i is a leaf node, and u i All the sibling nodes of are in the LOD node set U. That is, for u i The set of all sibling nodes (V x ) of any node (vx), then vx∈V x And vx∈U.

[0081] Legal conditions for non-leaf nodes (non-root nodes): u i is a non-leaf node (non-root node), and there is no descendant node of it in the LOD set U. i The set of all descendant nodes of y ) in any node (vy), vy∈V y and

[0082] Combining the definition of the expected node set and the legal scheduling node set, the node scheduling optimization problem can be defined as follows:

[0083] (Definition 3) Node scheduling optimization problem: In a node scheduling, when it is impossible to completely load all nodes in the expected node set, find a legal scheduling node set for each visible tile so that the revenue V generated after loading the nodes in these sets is maximized. That is, for each visible tile, the set A consisting of all legal scheduling node sets i ={A i1 , A i2 , A i3 , ..., A ij}, from each A i Select at most one legal scheduling node set A ik, so that the total node load in this scheduling is less than or equal to the current maximum allowed load S, and the benefits V generated by all loaded nodes are maximized. The specific mathematical description is shown in formula (1):

[0084]

[0085] Among them, m is the total number of currently visible tiles; y ij =0,y ij The jth legal scheduling node set in the i-th tile is not selected in this scheduling, otherwise y ij =1,y ij = 1 means the jth legal scheduling node set in the i-th Tile is selected for loading, and no other legal scheduling node sets in the same Tile are loaded. The total amount of loading required to load the nodes in this set is w ij , the revenue generated is v ij ; The benefit v of a node set ij Refers to the rendering effect after the nodes in the collection are loaded. It can be defined according to the specific situation, such as the total number of facets of the LOD model corresponding to all nodes in the collection.

[0086] An example of the node scheduling optimization problem mentioned above is Figure 2 shown. Figure 2 Shown are examples only.

[0087] Assume that after visibility calculation in the scene, there are only two visible tiles, Tile1 and Tile2, and assume that the expected node set of Tile1 is A = {6,7,8,5}, and the expected node set of Tile2 is B = {6,7,4}. Assume that node 5 and its child nodes do not pass the visibility judgment:

[0088] In a node scheduling, the current maximum allowed load is 15. When it is impossible to completely load all nodes in the expected node set and (the total load is 32), it is necessary to find a legal scheduling node set for the two tiles to replace the expected node set. There are multiple legal scheduling node sets for each of the two tiles, but in the current case, there are 6 legal scheduling node sets that can replace the expected node set for Tile1 and 4 for Tile2. Tile1 and Tile2 need to select at most one from them, so that the total load of the nodes in the selected set does not exceed the current maximum allowed load, and the benefit generated by the loaded nodes is the largest. Among the 24 possible situations, Tile1 chooses C = {6,7,4,5} and Tile2 chooses D = {1}, which generates the largest benefit (15). Therefore, in this node scheduling, the set and should be selected for loading display.

[0089] The node scheduling optimization problem is simple to describe, but it is difficult to find the optimal solution. The main reason is that in the actual scheduling process, the number of visible tiles and the number of nodes in each tile will not be as small as in the above example. The number of legal scheduling node sets composed of nodes is very large. Solving the node scheduling optimization problem requires extremely long running time and huge storage space. If it is directly integrated into a large-scale scene visualization system for solution, the real-time performance of the system will inevitably not be met. How to solve the node scheduling optimization problem while meeting the real-time requirements has become the core issue of realizing the internal and external memory scheduling mechanism of the leap-forward level of detail model.

[0090] In order to solve the problems existing in the scheduling mechanism of the existing 3D rendering engine, this application proposes the concept of a leapfrogging memory scheduling mechanism for the internal and external memory of the detail level model. The core idea is to know the relationship between each node through the pre-established detail level node tree, and combine the system external memory reading speed and the expected node set loading amount and other information, and skip some intermediate level LOD nodes according to the leaping rules and not load them, so as to achieve the goal of reducing the number of loaded LOD nodes, alleviating data loading pressure, and reducing the waiting time for displaying high-resolution LOD models. Among them, how to formulate the leapfrogging rules for the memory scheduling mechanism of the leapfrogging detail level model is the most core part of achieving the above goals.

[0091] In the actual scheduling process, there are two situations: the first situation is that the expected node set can be directly loaded within the time limit; the second situation is that when the expected node set cannot be directly loaded within the time limit, some alternative nodes are selected for loading and display;

[0092] For the first case, all intermediate levels are skipped and the expected node set is directly loaded and displayed. The second case is the focus of this application. It is necessary to generate a replacement node set for the expected node set according to the formulated skipping rules to achieve the goal of selecting the most appropriate LOD model to load and display within the current external memory device reading capacity.

[0093] For the determination of the crossing rules in the second case, this application proposes two solutions: one is the parent node replacement method, which generates an ancestor node set through the expected node set and loads and displays it as a replacement node set. This method is simple to implement, has a small amount of calculation, and can quickly and efficiently generate a replacement node set; the other is the knapsack problem solving method, which considers the rendering effect after the node is loaded and selects the best replacement node set for loading and display.

[0094] It should be noted that the node scheduling optimization problem is very similar to the description of the multi-select knapsack problem. Therefore, the idea of ​​solving the multi-select knapsack problem can be used to solve the node scheduling optimization problem by treating the tile as a group in the multi-select knapsack problem and treating all the legal scheduling node sets of each tile as items in the group. In this way, the optimization problem can be converted into a multi-select knapsack problem for solution, and the optimal set of replacement nodes can be calculated for loading and display under the premise of meeting the real-time requirements.

[0095] After further research on the knapsack problem solution method, it was found that although the idea of ​​solving the multi-select knapsack problem can be used to solve the node scheduling optimization problem, the number of legal scheduling nodes for items to be selected in the multi-select knapsack problem is very large, and it is very difficult to solve the large-scale multi-select knapsack problem in real time. In the case where the optimal solution cannot be obtained, this application solves the "suboptimal solution" of the node scheduling optimization problem in a certain way: Under the premise of meeting the real-time requirements of the scene visualization system, an approximate solution close to the optimal solution is found.

[0096] To this end, this application adopts an offline processing method with the goal of reducing the number of legal scheduling node sets used as input for the multi-choice knapsack problem. For each Tile, a small number of legal scheduling node sets are pre-generated to replace the original huge number of legal scheduling node sets, and then the optimal solution is obtained from the smaller number of legal scheduling node sets and loaded and displayed as a replacement node set.

[0097] It should be noted that there are two key points in implementing the knapsack problem solution method. The first is how to generate a small number of legal scheduling node sets offline. The second is how to process the legal scheduling node sets generated offline according to the system operation status, and generate the legal scheduling node sets to be selected as the input of the multi-selection knapsack problem in real time.

[0098] Specifically, the corresponding node set is scheduled to be loaded and displayed according to the scheduling mechanism to complete the visualization process of the scene, as shown in B1-B3.

[0099] B1: If the scheduling mechanism is a direct scheduling mechanism, all intermediate levels are crossed to load and display the expected node set to complete the visualization of the scene.

[0100] B2: If the scheduling mechanism is a parent node replacement method, an ancestor node set is generated, and the ancestor node set is loaded and displayed as a replacement node set to complete the visualization of the scene; or,

[0101] B3: If the scheduling mechanism is the scheduling mechanism of the knapsack problem solving method, select the optimal set of replacement nodes for loading and display to complete the visualization of the scene.

[0102] Among them, the parent node replacement method proposed in this application generates an alternative legal scheduling node set according to the following method: starting from the original expected node set, the set composed of the parent nodes of all nodes in the expected node set that cannot be directly loaded is regarded as a new expected node set, and its load s1 is calculated to determine whether s1 is less than or equal to v·t. If s1 is less than or equal to v·t, it indicates that the alternative legal scheduling node set can be loaded within the limit time, and this set is directly used as the final alternative node set; otherwise, the parent nodes in the set are continued to form a new expected node set for calculation until the load s1 of the alternative legal scheduling node set is less than or equal to v·t or the root node is found. The whole process is as follows Figure 3 shown.

[0103] S301: The father nodes of the nodes in the expected node set are grouped into a set.

[0104] S302: Determine whether the set has only a root node, if so, execute S303, if not, execute S304.

[0105] S303: End the loading and display process.

[0106] S304: Process illegal nodes to generate a set of legal scheduling nodes.

[0107] S305: Count the load s1 of the legal scheduling node set;

[0108] S306: Determine whether the load s1 is less than or equal to the amount that the set of alternative legal scheduling nodes can load within the time limit, that is, determine whether s1 is less than or equal to v·t. If yes, execute S307; if no, execute S308.

[0109] S307: The generated legal scheduling node set is loaded and displayed as a replacement node set to complete the visualization of the scene.

[0110] S308: Use the generated legal scheduling node set as the expected node set, and return to execute S301.

[0111] Specifically, if the scheduling mechanism is a scheduling mechanism of the knapsack problem solving method, the process of selecting the optimal set of replacement nodes for loading and displaying to complete the visualization of the scene is shown in C1-C4:

[0112] C1: Obtain a preset legal scheduling node set (ie, a pre-prepared legal scheduling node set) through offline processing, and use the preset legal scheduling node set as an item template set.

[0113] In C1, a small number of legal scheduling node sets are preset through offline processing as item template sets.

[0114] C2: The expected node set and node status table generated based on visibility calculation and level of detail calculation when the scene visualization system is running.

[0115] C3: Based on the expected node set and the node status table, a legal scheduling node set is generated from the item template set in real time as the input of the multi-choice knapsack problem.

[0116] In C3, when the scene visualization system is running, the expected node set generated by visibility calculation and detail level calculation, as well as the node state table, are combined to generate the legal scheduling node set as the input of the multi-choice knapsack problem from each item template in real time.

[0117] C4: Solve the legal scheduling node set, obtain the optimal replacement node set and load it for display to complete the scene visualization.

[0118] In C4, the result of solving the multi-selection knapsack problem is used as the solution for the current LOD node scheduling to complete the generation of the optimal replacement node set.

[0119] The knapsack problem solving method of the present application is to find an approximate solution close to the optimal solution while satisfying the real-time performance of the scene visualization system. The present application proposes the concepts of "item template" and "item", and uses a small number of legal scheduling node sets preset by offline processing as a set of "item templates". When the scene visualization system is running, the expected node set generated by visibility calculation and detail level calculation, as well as the node status table, are combined to generate a legal scheduling node set from each "item template" in real time as the input of the multi-select knapsack problem. The result of solving the multi-select knapsack problem is used as the current LOD node scheduling solution to complete the generation of the alternative node set.

[0120] The present application realizes a cross-level detail model internal and external memory scheduling mechanism. The scheduling mechanism proposed in the present application effectively solves the problems existing in the level-by-level scheduling mechanism of the existing three-dimensional rendering engine. The scheduling mechanism of the present patent can read the serialized files generated in advance using the offline processing method, and organize the nodes into a level-of-detail node tree structure when the scene visualization system is initialized, so that the scene visualization system can obtain the ability to know the hierarchical relationship of each node in the entire scene without loading and parsing the node files one by one, thereby achieving the goal of cross-level loading of the LOD model.

[0121] In the face of the situation where all nodes in the expected node set cannot be loaded directly, this application studies the leap rules in the internal and external memory scheduling mechanism of the leap-forward detail level model, and proposes two alternative node set generation methods: the parent node replacement method and the knapsack problem solving method. Among them, the parent node replacement method can quickly and efficiently generate alternative node sets, and has the characteristics of small computational complexity and simple implementation. The knapsack problem solving rule studies the node scheduling optimization problem, and converts the node scheduling optimization problem into a multi-selection knapsack problem for solution in a certain way. Under the premise of considering the reading capacity of the external memory device, a set of alternative nodes with the best display effect of the current loading is generated.

[0122] This application aims to improve the shortcomings of the scheduling mechanism of the existing 3D rendering engine that can only load and display LOD models layer by layer, and endow the scene visualization system with the ability to load and display LOD models across levels. On this basis, in order to determine the specific number of detail levels that need to be crossed in a scheduling process, two schemes are proposed for the crossing rules of the internal and external memory scheduling mechanism of the leap-forward detail level model. The scene visualization system can complete the work of loading and rendering the LOD model according to these two crossing rules, thereby achieving the goal of large-scale scene visualization.

[0123] This application takes the internal and external memory scheduling mechanism of the detail level model as the research object, and proposes the concept of a leapfrog detail level model internal and external memory scheduling mechanism based on a thorough study of the two aspects of the internal and external memory organization form and scheduling mechanism of the scene model. Compared with the existing three-dimensional rendering engine's layer-by-layer detail level model internal and external memory scheduling mechanism, the internal and external memory scheduling mechanism of this application can load the detail level model across levels, reducing the amount of data loaded and the user's waiting time. At the same time, this application studies the node scheduling optimization problem and proposes two alternative node set generation methods, which fully consider factors such as the external memory device reading capacity, the amount of model data loaded, the rendering effect after loading, and the loading time, and more accurately select the detail level model, so that the scene visualization system can achieve the best balance between operating efficiency and rendering effect, thereby improving the user experience.

[0124] The beneficial effects of the embodiments of the present application are as follows: Since the scheduling mechanism is an internal and external memory scheduling mechanism based on the leap-forward detail level model, the corresponding scheduling mechanism is determined according to the scheduling scene type, and the corresponding node set is scheduled for loading and display according to the scheduling mechanism, which reduces the amount of data loaded and the user waiting time. At the same time, the present application studies the node scheduling optimization problem, and through two alternative node set generation methods, namely the parent node replacement method or the knapsack problem solving method, fully considers factors such as the external memory device reading capacity, the model data loading amount, the rendering effect after loading, and the loading time, and more accurately selects the detail level model, so that the scene visualization system achieves the best balance between operating efficiency and rendering effect, thereby improving the user experience in the process of internal and external memory data scheduling implemented by the scene visualization system.

[0125] Based on the above embodiments Figure 1 A data processing method for scene visualization is disclosed, and an embodiment of the present application also discloses a data processing device for scene visualization, such as Figure 4 As shown, the data processing device for scene visualization includes:

[0126] A first determining unit 401 is configured to determine a scheduling scene type when a scene visualization requirement is received;

[0127] A second determining unit 402 is used to determine a corresponding scheduling mechanism according to the scheduling scenario type;

[0128] The visualization unit 403 is used to schedule the corresponding node set to load and display according to the scheduling mechanism to complete the visualization process of the scene; wherein the scheduling mechanism is an internal and external memory scheduling mechanism based on the leap-forward detail level model; the scheduling mechanism at least includes a direct scheduling mechanism, a parent node replacement method scheduling mechanism or a knapsack problem solving method scheduling mechanism.

[0129] Furthermore, the first determining unit 401 includes:

[0130] A first determination module is used to, when receiving a scene visualization requirement, determine that the scheduling scene type is a scheduling scene type for loading the expected node set if the expected node set is directly loaded within a limited time;

[0131] The second determination module is used to determine the scheduling scenario type as a scheduling scenario type for generating a substitute node set for the expected node set if the expected node set cannot be directly loaded within the limited time.

[0132] Further, the second determining unit 402 includes:

[0133] A third determination module is used to determine the direct scheduling mechanism corresponding to the scheduling scenario type if the scheduling scenario type is a scheduling scenario type for loading an expected node set;

[0134] The fourth determination module is used to determine that the corresponding scheduling mechanism is a scheduling mechanism of a parent node replacement method or a scheduling mechanism of a knapsack problem solving method if the scheduling scenario type is a scheduling scenario type of generating a replacement node set for an expected node set.

[0135] Furthermore, the visualization unit 403 includes:

[0136] A first visualization module, for loading and displaying the desired node set across all intermediate levels to complete the visualization of the scene if the scheduling mechanism is a direct scheduling mechanism;

[0137] The second visualization module is used to generate an ancestor node set if the scheduling mechanism is a parent node replacement method scheduling mechanism, and load and display the ancestor node set as a replacement node set to complete the visualization of the scene; or,

[0138] The third visualization module is used to select the optimal set of replacement nodes for loading and display if the scheduling mechanism is a scheduling mechanism of the knapsack problem solving method, so as to complete the visualization of the scene.

[0139] Furthermore, the second visualization module includes:

[0140] A composition submodule is used to form a set of parent nodes of nodes in the expected node set;

[0141] The first judgment submodule is used to judge whether the set only has the root node;

[0142] End submodule, used to end the loading and display process if yes;

[0143] A generation submodule, for processing illegal nodes to generate a set of legal scheduling nodes if no;

[0144] The first judgment submodule is used to count the load of the legal scheduling node set and judge whether the load is less than or equal to the load of the alternative legal scheduling node set within the limited time;

[0145] A visualization submodule is used to load and display the generated legal scheduling node set as the replacement node set if the loading amount is less than or equal to the replacement legal scheduling node set that can be loaded within the limited time, so as to complete the visualization of the scene;

[0146] The execution submodule is used to, if the load amount is greater than the alternative legal scheduling node set that can be loaded within the time limit, use the generated legal scheduling node set as the expected node set and execute the step of combining the parent node of the expected node with the middle node to form a set.

[0147] Furthermore, the third visualization module includes:

[0148] An acquisition submodule, used to acquire a preset legal scheduling node set by an offline processing method, and use the preset legal scheduling node set as an item template set;

[0149] A calculation submodule, used for generating an expected node set and a node state table according to visibility calculation and detail level calculation when the scene visualization system is running;

[0150] A determination submodule, for generating a legal scheduling node set as an input of a multi-select knapsack problem from an item template set in real time according to the expected node set and the node state table;

[0151] The solving submodule is used to solve the legal scheduling node set, obtain the optimal replacement node set and load and display it to complete the visualization of the scene.

[0152] The beneficial effects of the embodiments of the present application are as follows: Since the scheduling mechanism is an internal and external memory scheduling mechanism based on the leap-forward detail level model, the corresponding scheduling mechanism is determined according to the scheduling scene type, and the corresponding node set is scheduled for loading and display according to the scheduling mechanism, which reduces the amount of data loaded and the user waiting time. At the same time, the present application studies the node scheduling optimization problem, and through two alternative node set generation methods, namely the parent node replacement method or the knapsack problem solving method, fully considers factors such as the external memory device reading capacity, the model data loading amount, the rendering effect after loading, and the loading time, and more accurately selects the detail level model, so that the scene visualization system achieves the best balance between operating efficiency and rendering effect, thereby improving the user experience in the process of internal and external memory data scheduling implemented by the scene visualization system.

[0153] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the data processing method for scene visualization as described above.

[0154] The present application also provides an electronic device, whose structural diagram is as follows: Figure 5 As shown, it specifically includes a memory 501 and one or more instructions 502, wherein the one or more instructions 502 are stored in the memory 501 and are configured to be executed by one or more processors 503 to execute the one or more instructions 502 to perform the above-mentioned data processing method for scene visualization.

[0155] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0156] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0157] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0158] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0159] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0160] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method for scene visualization, characterized in that: The method comprises: When a scene visualization requirement is received, the scheduling scene type is determined; Determine a corresponding scheduling mechanism according to the scheduling scenario type; The corresponding node set is scheduled to be loaded and displayed according to the scheduling mechanism to complete the visualization process of the scene; wherein the scheduling mechanism is an internal and external memory scheduling mechanism based on a leap-forward detail level model; the scheduling mechanism at least includes a direct scheduling mechanism, a parent node replacement method scheduling mechanism or a knapsack problem solving method scheduling mechanism.

2. The method according to claim 1, characterized in that When a scene visualization requirement is received, determining a scheduling scene type includes: When a scene visualization requirement is received, if the expected node set is directly loaded within the time limit, the scheduling scene type is determined to be a scheduling scene type for loading the expected node set; If the expected node set cannot be directly loaded within the time limit, the scheduling scenario type is determined to be a scheduling scenario type for generating a substitute node set for the expected node set.

3. The method according to claim 1, characterized in that The determining a corresponding scheduling mechanism according to the scheduling scenario type includes: If the scheduling scenario type is a scheduling scenario type for loading an expected node set, determining a direct scheduling mechanism corresponding to the scheduling scenario type; If the scheduling scenario type is a scheduling scenario type for generating a replacement node set for an expected node set, its corresponding scheduling mechanism is determined to be a scheduling mechanism of a parent node replacement method or a scheduling mechanism of a knapsack problem solving method.

4. The method according to claim 3, characterized in that The process of scheduling the corresponding node set to load and display according to the scheduling mechanism to complete the visualization of the scene includes: If the scheduling mechanism is a direct scheduling mechanism, all intermediate levels are crossed to load and display the expected node set to complete the visualization of the scene; If the scheduling mechanism is a parent node replacement method, an ancestor node set is generated, and the ancestor node set is loaded and displayed as a replacement node set to complete the visualization of the scene; or, If the scheduling mechanism is a scheduling mechanism of the knapsack problem solving method, the optimal set of replacement nodes is selected for loading and display to complete the visualization of the scene.

5. The method according to claim 4, characterized in that If the scheduling mechanism is a scheduling mechanism of the parent node replacement method, an ancestor node set is generated, and the ancestor node set is loaded and displayed as a replacement node set to complete the visualization of the scene, including: The father nodes of the nodes in the expected node set are combined into a set; Determine whether the set only has a root node; If yes, then end the loading and display process; If not, process the illegal nodes to generate a set of legal scheduling nodes; Counting the load of the legal scheduling node set, and determining whether the load is less than or equal to the load that can be loaded by the alternative legal scheduling node set within the limited time; If the load amount is less than or equal to the amount that the substitute legal scheduling node set can load within the time limit, the generated legal scheduling node set is loaded and displayed as the substitute node set to complete the visualization of the scene; If the load is greater than what the alternative legal scheduling node set can load within the time limit, the generated legal scheduling node set is used as the expected node set, and the step of combining the expected node with the parent node of the middle node to form a set is performed.

6. The method according to claim 4, characterized in that If the scheduling mechanism is a scheduling mechanism of the knapsack problem solving method, the optimal set of replacement nodes is selected for loading and display to complete the visualization of the scene, including: Obtaining a preset legal scheduling node set by offline processing, and using the preset legal scheduling node set as an item template set; When the scene visualization system is running, the expected node set and node state table are generated according to the visibility calculation and detail level calculation; Generating a legal scheduling node set as an input of a multi-select knapsack problem from the item template set in real time according to the expected node set and the node state table; The legal scheduling node set is solved to obtain the optimal replacement node set and load it for display to complete the visualization of the scene.

7. A data processing device for scene visualization, characterized in that: The device comprises: A first determining unit, configured to determine a scheduling scene type when a scene visualization requirement is received; A second determining unit, configured to determine a corresponding scheduling mechanism according to the scheduling scenario type; A visualization unit is used to schedule the corresponding node set to be loaded and displayed according to the scheduling mechanism to complete the visualization process of the scene; wherein the scheduling mechanism is an internal and external memory scheduling mechanism based on a leap-forward detail level model; the scheduling mechanism at least includes a direct scheduling mechanism, a parent node replacement method scheduling mechanism or a knapsack problem solving method scheduling mechanism.

8. The device according to claim 7, characterized in that The first determining unit includes: A first determination module is used to, when receiving a scene visualization requirement, determine that the scheduling scene type is a scheduling scene type for loading the expected node set if the expected node set is directly loaded within a limited time; The second determination module is used to determine the scheduling scenario type as a scheduling scenario type for generating a substitute node set for the expected node set if the expected node set cannot be directly loaded within the limited time.

9. The device according to claim 7, characterized in that The second determining unit includes: A third determination module is used to determine the direct scheduling mechanism corresponding to the scheduling scenario type if the scheduling scenario type is a scheduling scenario type for loading an expected node set; The fourth determination module is used to determine that the corresponding scheduling mechanism is a scheduling mechanism of a parent node replacement method or a scheduling mechanism of a knapsack problem solving method if the scheduling scenario type is a scheduling scenario type of generating a replacement node set for an expected node set.

10. The device according to claim 9, characterized in that The visualization unit comprises: A first visualization module, for loading and displaying the desired node set across all intermediate levels to complete the visualization of the scene if the scheduling mechanism is a direct scheduling mechanism; A second visualization module is used to generate an ancestor node set if the scheduling mechanism is a parent node replacement method scheduling mechanism, and load and display the ancestor node set as a replacement node set to complete the visualization of the scene; or, The third visualization module is used to select the optimal set of replacement nodes for loading and display if the scheduling mechanism is a scheduling mechanism of the knapsack problem solving method, so as to complete the visualization of the scene.

Images