A Method and System for Implementing Interactive Retrieval of Tens of Millions of Instances Based on a Real-Time 3D Engine

Abstract

This invention discloses a method and system for interactive retrieval of tens of millions of instances based on a real-time 3D engine. Multiple independent instances are created in DCC (Digital Content Creation) software, with instance data stored independently. These instances are merged into a single set instance and imported into the 3D engine to create a 3D data tree. A local portion of the set instance is selected, and its UV coordinates are read. The local UV axis number is obtained, which represents its XY value in the 3D data tree. Based on the local XY value, the instance data contained in the Z value of that local portion in the 3D data tree is retrieved. This method can be applied to data retrieval and positioning of massive amounts of instances, accurately locating data positions directly in the 3D data tree for efficient data retrieval. For interactive feedback, the CPU load is shifted to the more powerful GPU, allowing more CPU resources to execute complex logical tasks.

Description

Technical Field

[0001] This invention relates to the field of digital content creation technology, specifically to a method and system for interactive retrieval of tens of millions of instances based on a real-time 3D engine. Background Technology

[0002] Most current solutions for real-time interactive querying of instance information are based on treating instances as independent entities. Multiple independent entities are placed and rendered for separate queries, using a ray-mapping algorithm to distinguish the ID (Identity Document, unique code) or name of the individual entity within the interacting instance. This ID or name then serves as the necessary path to enter the next level of information block, thereby retrieving relevant information about the interacting individual entity. Current solutions for individual entities cannot support a large number of instances because each instance rendering requires the CPU (central processing unit) to process the instance's rendering information and extract it from memory to video memory for the GPU (graphics processing unit) to perform rendering calculations. Since this process occurs for every frame of the screen, and the minimum standard for stable program operation is 30 frames per second, fewer CPU submissions result in better overall performance and more stable program operation. Actual testing shows that under current hardware conditions, a submission count approaching 10,000 is sufficient to cause a severe performance impact.

[0003] In the text information interface, to generate a text list corresponding to an instance, all instances need to be traversed, and the instance names of the instances need to be written to the list. During a query, each character of the instance names needs to be traversed based on the names in the list until an instance that completely matches the name is found. In the text information interface, to query the specific information of an instance in the list, the existing instance names need to be traversed character by character in sequence, comparing them with the names in the list. The time complexity of this process is O(f(n)). The more instances there are, the more processing steps there are, and the more time-consuming the traversal process becomes. Summary of the Invention

[0004] To address this issue, the present invention provides a method and system for interactive retrieval of tens of millions of instances based on a real-time 3D engine, in order to solve the problems of existing real-time interactive query of instance information failing to meet the needs of massive instance retrieval and the information query taking too long.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] According to a first aspect of the present invention, a method for interactive retrieval of tens of millions of instances based on a real-time 3D engine is proposed, the method comprising:

[0007] Create multiple independent instances, and store the instance data of each instance independently.

[0008] Multiple independent instances are sequentially arranged in a UV coordinate system. The UV coordinate data of each instance after arrangement is obtained and the UV axis number is recorded. Multiple instances are then merged into a set instance.

[0009] Import the collection instances and instance data into the 3D engine, display the collection instances, and create a three-dimensional data tree corresponding to the collection instances. In the three-dimensional data tree, the X-axis corresponds to the U-axis number of each instance, the Y-axis corresponds to the V-axis number of each instance, and the Z-axis corresponds to the instance data of each instance.

[0010] Select a local area in the set instance and read its UV coordinates. Obtain the local UV axis number based on the local UV coordinates. The local UV axis number is its XY value in the 3D data tree. Read the instance data contained in the Z value of the local area in the 3D data tree based on the local XY value.

[0011] Furthermore, multiple independent instances are created, and the instance data of each independent instance is stored independently, specifically including:

[0012] Multiple original instances are created in the DCC software, each instance using independent data and becoming an independent instance;

[0013] The instance data of each independent instance is stored outside the software in a preset format and is matched one-to-one with each instance in a preset correspondence method.

[0014] Furthermore, the multiple independent instances are sequentially arranged in a UV coordinate system, the UV coordinate data of each instance is obtained and the UV axis number is recorded, and the multiple instances are merged into a set instance, specifically including:

[0015] Set a digital reserved capacity in binary format based on the number of instances, and calculate the unit span value based on the reserved capacity;

[0016] According to the order of each independent instance, each instance is compiled in the UV coordinate two-dimensional space with the span value as the unit. The values ​​of U and V are floating-point numbers between 0 and 1. The U-axis number and V-axis number of each instance are obtained, and the UV coordinates of each instance are calculated based on the UV-axis number and span value of each instance. The UV coordinate data of each instance is modified from the original default value to the compiled UV coordinate value.

[0017] Multiple instances are merged into a collection instance, and the UV axis numbers and instance data of each instance are generated into a data file and exported in sequence.

[0018] Further, a specific area within the set instance is selected and its UV coordinates are read. The UV axis number of the local area is obtained based on the local UV coordinates, specifically including:

[0019] The UV coordinates of a local area within the collection instance are read using the ray detection function of the 3D engine.

[0020] The local UV axis number is calculated based on the UV coordinates and span values.

[0021] Furthermore, the method also includes:

[0022] The local UV coordinates are passed as parameters to the graphics shader. The GPU calculates the local position and displays the local area in a special way to achieve interactive feedback.

[0023] Furthermore, the method also includes:

[0024] The DCC software was further developed to enable it to read external instance data, modify instance UVs, merge instances, and export data.

[0025] According to a second aspect of the present invention, a system for interactive retrieval of tens of millions of instances based on a real-time 3D engine is proposed, the system comprising:

[0026] The DCC module is used to create multiple independent instances, and the instance data of each independent instance is stored independently.

[0027] Multiple independent instances are sequentially arranged in a UV coordinate system. The UV coordinate data of each instance after arrangement is obtained and the UV axis number is recorded. Multiple instances are then merged into a set instance.

[0028] The 3D engine module is used to import the collection instances and instance data into the 3D engine, display the collection instances, and create a three-dimensional data tree corresponding to the collection instances. In the three-dimensional data tree, the X-axis corresponds to the U-axis number of each instance, the Y-axis corresponds to the V-axis number of each instance, and the Z-axis corresponds to the instance data of each instance.

[0029] Select a local area in the set instance and read its UV coordinates. Obtain the local UV axis number based on the local UV coordinates. The local UV axis number is its XY value in the 3D data tree. Read the instance data contained in the Z value of the local area in the 3D data tree based on the local XY value.

[0030] According to a third aspect of the present invention, a computer storage medium is provided, the computer storage medium containing one or more program instructions, the one or more program instructions being executed by a real-time 3D engine-based system for retrieving and interacting with tens of millions of instances as described above.

[0031] The present invention has the following advantages:

[0032] This invention proposes a method and system for interactive retrieval of tens of millions of instances based on a real-time 3D engine. The method involves creating multiple independent instances, with each instance's data stored independently. These independent instances are then sequentially arranged in a UV coordinate system. The UV coordinate data of each instance is obtained, and the UV axis number is recorded. Multiple instances are then merged into a single set instance. This set instance and its data are imported into the 3D engine, displayed, and a corresponding 3D data tree is created. In this 3D data tree, the X-axis corresponds to the U-axis number of each instance, the Y-axis corresponds to the V-axis number, and the Z-axis corresponds to the instance data. A specific area within the set instance is selected, and its UV coordinates are read. The UV axis number of this area is obtained based on its UV coordinates; this UV axis number represents its XY value in the 3D data tree. Finally, the instance data contained in the Z-value of this area in the 3D data tree is retrieved based on its XY value. While fully retaining the functionality and scalability of the original solution, the instance limit can be significantly increased, making it applicable to the functional requirements of massive instances. In the data retrieval and positioning process, it avoids the need to traverse and check each instance one by one, and directly and accurately locates the data position in the 3D data tree, enabling efficient data reading. In terms of interactive feedback, the CPU load is transferred to the more powerful GPU, allowing more CPU resources to execute complex logical tasks. Attached Figure Description

[0033] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0034] Figure 1 This is a flowchart illustrating a method for interactive retrieval of tens of millions of instances based on a real-time 3D engine, as provided in Embodiment 1 of the present invention.

[0035] Figure 2 A detailed implementation flowchart of a method for interactive retrieval of tens of millions of instances based on a real-time 3D engine, provided in Embodiment 1 of the present invention;

[0036] Figure 3 This is a demonstration diagram of the effect of a method for interactive retrieval of tens of millions of instances based on a real-time 3D engine provided in Embodiment 1 of the present invention. Detailed Implementation

[0037] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Example 1

[0039] This embodiment proposes a method for interactive retrieval of tens of millions of instances based on a real-time 3D engine. To preserve the independence between instances and reduce the CPU submission count of individual instances, it is necessary to merge all independent instances into a collection instance, while storing additional information of the instances externally, separate from the instance itself. The collection instance is essentially equivalent to an independent instance, except that it is merged after being processed by DCC (Digital Content Creation) software. During the compilation process, the information of each part and the externally stored information are indexed. This utilizes the necessary attribute of instances in the 3D engine: "UVmap" (planar two-dimensional coordinate mapping, where U and V represent two-dimensional spatial axes, with the same meaning as XYZ). After merging independent instances, the collection instance is created through compilation. The UV attribute of the collection instance retains its independence before merging, allowing the real-time 3D engine to distinguish the instance information of a local part by extracting the UV attribute value of that part of the collection instance. This is equivalent to extracting an independent instance from the collection instance.

[0040] like Figure 1 As shown, the method specifically includes:

[0041] S100: Create multiple independent instances, and store the instance data of each independent instance independently.

[0042] (1) Create original instances in DCC software, and make each instance an independent instance by using independent data.

[0043] In DCC, instances are composed of data, and multiple instances can share a single set of data. Independent data refers to instances that do not share data. Here, "data" specifically refers to the data that constitutes an instance, such as the point, line, and surface composition of a model class instance, its memory location, UV information, etc.

[0044] (2) Store additional information of the instance in any form externally. The external data must correspond to the instance in advance.

[0045] Additional information refers to data unrelated to DCC, such as its name, category, application method, operating instructions, and any other recordable content. Once stored externally, it becomes external data.

[0046] There are multiple ways to map external data to instances, such as mapping by name, mapping by name sorting, or mapping by spatial location sorting. In short, all feasible methods are used to map external data to instances one by one.

[0047] External data may not only be in list form, but may also be presented in the form of tree diagrams, 2D tables, etc., which requires additional logic to match them one by one.

[0048] (3) Secondary development of DCC software to enable it to read external data, modify instance UV, merge instances, and export data.

[0049] 1) There are many ways to develop DCC software. Taking Blender as an example, it uses Python for secondary development.

[0050] 2) External data can be read using Python's built-in CSV read / write module, which writes external CSV data into memory as an array.

[0051] 3) Mainstream programming languages ​​such as C#, C++, Java, and JavaScript all support reading and writing CSV files.

[0052] 4) The modification methods vary depending on the DCC software's own API (Application Programming Interface).

[0053] Using Blender as an example, how to call it in Python:

[0054] bpy.context.active_object.data.uv_layers.data[index].uv

[0055] This allows you to modify the UV data in the "loop" field at the current index of the active UV layer under the current active_object. "loop" is a data string.

[0056] 5) Exporting data also uses the CSV module built into the programming language to write the data information to an external CSV file.

[0057] S200. The multiple independent instances are sequentially arranged in the UV coordinate system. The UV coordinate data of each instance after arrangement is obtained and the UV axis number is recorded. The multiple instances are then merged into a set instance.

[0058] Specifically:

[0059] 1) Set the reserved capacity to a number conforming to binary format, such as 32 / 64 / 128 / 2048 / 4096, according to the number of instances. The target capacity value is the square of the number.

[0060] The quantity setting is mainly for optimization. Smaller needs can use smaller reserved capacity to optimize performance. For example, if you need 1,000 instances, you can reserve 32, and if you need 10,000,000 instances, you need to reserve 4,096.

[0061] Because GPU computing is involved here, the numerical requirements are quite strict. The value must be 2 to the power of n, where 32 is not the minimum value, but depends on the support of the 3D engine. 4096 is not the maximum value, but depends on the performance of the CPU and the maximum capacity supported by the GPU.

[0062] 2) Assuming the capacity is set to the square of 128, then calculate 1 / 128 as the span between values, since the UV attribute is a floating-point number between 0 and 1.

[0063] 3) Based on the order of each independent instance, the UV data of the instance is laid out in the space between 0 and 1 on both UV axes, using the span value as the unit.

[0064] Each dimension of the UV information is a floating-point value in the range of 0 to 1. The span value is the minimum spacing to evenly fill this 2-dimensional space under the current maximum capacity, and it is calculated as 1 / capacity.

[0065] Taking a capacity of 128 as an example, the span value is 1 / 128 = 0.0078125. Considering the convenience and performance optimization when writing shaders, the final value of each dimension of UV should be reduced by 1 / 2 of the span value, which is 0.0078125 - 0.00390625.

[0066] 4) Read the external data corresponding to each instance, and calculate the UV integer number of this instance (assuming it is from 1 to 128, number U = 5, number V = 127, indicating that this instance is the 128*5+127=767th one compiled, and the UV data on the instance is U = (1 / 128*5)-(1 / 128 / 2), V = (1 / 128*127)-(1 / 128 / 2)). Write the UV data, UV number, external data, and other necessary data (such as spatial coordinates, instance size, hierarchical relationship, instance name, etc.) in the 767th line of the data to be exported.

[0067] 5) After all instances have been processed, merge all instances into a collection instance and export it along with the data to be exported. Merging instances is also an operation of the DCC software and is a basic function. Taking Blender as an example, in Python, you can complete the merge by calling bpy.ops.object.join().

[0068] S300. Import the collection instance and instance data into the 3D engine, display the collection instance, and create a three-dimensional data tree corresponding to the collection instance. In the three-dimensional data tree, the X-axis corresponds to the U-axis number of each instance, the Y-axis corresponds to the V-axis number of each instance, and the Z-axis corresponds to the instance data of each instance.

[0069] S400. Select a local part of the set instance and read its UV coordinates. Obtain the UV axis number of the local part based on the UV coordinates. The UV axis number of the local part is its XY value in the three-dimensional data tree. Read the instance data contained in the Z value of the local part in the three-dimensional data tree based on the XY value of the local part.

[0070] Specifically:

[0071] 1. Place the processed files in the 3D engine resource directory.

[0072] 2. Develop a 3D engine program and process the prepared data according to the following steps:

[0073] 1) Organize the imported 2D data into 3D data so that the external data stored in the Z dimension (where X dimension is the UV number U, Y dimension is the UV number V) can be read quickly.

[0074] 2) Use ray detection to read the UV data of a certain part of the collection instance, and calculate this data into a UV number.

[0075] Raycasting is a fundamental function of 3D engines. After confirming the ray vector, intersecting instances are usually quickly identified by establishing the BVH (Bounding Volume Hierarchy) of the current 3D space. Once the instance is obtained, the underlying instructions are immediately called to find the actual location where the ray intersects with the instance. This location points to some physical memory locations (usually three), which store various local information of this instance, including UV information. The specific UV coordinates of the intersection location are obtained by interpolation, and the values ​​are transferred to a specific memory location in memory associated with this ray.

[0076] After the instances are merged, they are like welded into one, but the coordinate data within the instances remains unchanged, and local areas of the instances can be distinguished by the coordinates.

[0077] 3) Write a Shader (graphics shader), pass UV data into the Shader, and let the Shader use the UV property of the collection instance to handle local appearance in a special way to achieve the purpose of interaction.

[0078] UV mapping is essentially a coordinate mapping method used to map 3D spatial coordinates to 2D spatial coordinates. You can think of it this way: clothes are three-dimensional when worn, but flat when cut out and laid flat. However, every edge and pattern on the clothes is still there; the only difference is their spatial position when worn or laid flat. They can be mapped to each other in a one-to-one correspondence way.

[0079] The special local processing methods are very diverse. This is because the 3D position is obtained through UV coordinates, and any operation can be performed in this space, such as highlighting, stroking, deformation, color change, semi-transparency, etc.

[0080] 4) Using the calculated UV number, read the data block pointed to by this UV number in the 3D data. This data is additional data for the current interactive instance.

[0081] Furthermore, by leveraging additional data and existing interactive feedback, we can develop extended functionalities, such as developing a device list dropdown list, or displaying the device in a tree-like or network structure instead of a list, or directly manipulating other instances, or collecting information into a database for big data analysis, and so on.

[0082] Figure 2 This document illustrates a detailed implementation flowchart of a method for interactive retrieval of tens of millions of instances based on a real-time 3D engine, as described in this embodiment. The method includes the following optional steps when sequentially compiling instances: allocating parent-child hierarchies of instances according to data format, reading the parent-child hierarchical relationship of instances, determining whether the current instance is a child instance, and if so, recording the instance coordinates, recording the instance size, incrementing the compilation sequence number by 1, and recording the instance number; if it is not a child instance but a parent instance, recording the average coordinates of all subsets, calculating the total size of all subsets, incrementing the compilation sequence number by 1, and recording the instance number.

[0083] Figure 3 This illustration shows the effect of a method for interactive retrieval of tens of millions of instances based on a real-time 3D engine, as shown in this embodiment. Multiple instances were created in DCC, and 10 instances were displayed. 2 Taking the reserved capacity as an example, each instance is compiled in a UV coordinate system, and the compiled set of instance data files is exported. The set of instances is then imported into the 3D engine for display. In the 3D data tree, the X and Y axes correspond to the UV number data of the instance, and the Z axis contains the instance data. The data position can be accurately located directly in the 3D data tree, enabling efficient data reading. In terms of interactive feedback, the CPU load is shifted to the more powerful GPU, allowing more CPU resources to execute complex logical tasks.

[0084] Example 2

[0085] Corresponding to Embodiment 1 above, this embodiment proposes a system for interactive retrieval of tens of millions of instances based on a real-time 3D engine. The system includes:

[0086] The DCC module is used to create multiple independent instances, and the instance data of each independent instance is stored independently.

[0087] Multiple independent instances are sequentially arranged in a UV coordinate system. The UV coordinate data of each instance after arrangement is obtained and the UV axis number is recorded. Multiple instances are then merged into a set instance.

[0088] The 3D engine module is used to import the collection instances and instance data into the 3D engine, display the collection instances, and create a three-dimensional data tree corresponding to the collection instances. In the three-dimensional data tree, the X-axis corresponds to the U-axis number of each instance, the Y-axis corresponds to the V-axis number of each instance, and the Z-axis corresponds to the instance data of each instance.

[0089] Select a local area in the set instance and read its UV coordinates. Obtain the local UV axis number based on the local UV coordinates. The local UV axis number is its XY value in the 3D data tree. Read the instance data contained in the Z value of the local area in the 3D data tree based on the local XY value.

[0090] The functions performed by each component in the interactive retrieval system for tens of millions of instances based on a real-time 3D engine provided in this embodiment of the invention have been described in detail in the above embodiment 1, so they will not be repeated here.

[0091] Example 3

[0092] Corresponding to the above embodiments, this embodiment proposes a computer storage medium containing one or more program instructions, which are used by a real-time 3D engine-based system for retrieving and interacting with tens of millions of instances to execute the method described in Embodiment 1 above.

[0093] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for implementing a hundred-million-level instance retrieval interaction based on an instant 3D engine, characterized in that, The method comprises: Making a plurality of independent instances, and independently storing instance data of each independent instance; Arranging the plurality of independent instances in sequence in a UV coordinate system, obtaining UV coordinate data of each instance after arrangement while recording UV axis numbers, and merging the plurality of instances into a collection instance; Importing the collection instance and the instance data into a 3D engine, displaying the collection instance, and creating a three-dimensional data tree corresponding to the collection instance, wherein an X axis of the three-dimensional data tree corresponds to a U axis number of each instance, a Y axis corresponds to a V axis number of each instance, and a Z axis corresponds to instance data of each instance; Selecting a certain local part in the collection instance and reading UV coordinates thereof, obtaining a UV axis number of the local part according to the UV coordinates, wherein the UV axis number of the local part is an XY value in the three-dimensional data tree, and reading instance data contained in a Z value of the local part in the three-dimensional data tree according to the XY value.

2. The method of claim 1, wherein the method is implemented based on an instant 3D engine. Making a plurality of independent instances, and independently storing instance data of each independent instance, specifically comprising: Making a plurality of original instances in a DCC software, each instance using independent data and becoming an independent instance; Storing instance data of each independent instance outside the software in a preset form, and one-to-one corresponding to each instance in a preset corresponding manner.

3. The method of claim 1, wherein the method is implemented based on an instant 3D engine. Arranging the plurality of independent instances in sequence in a UV coordinate system, obtaining UV coordinate data of each instance after arrangement while recording UV axis numbers, and merging the plurality of instances into a collection instance, specifically comprising: Setting a digital reserved capacity conforming to a binary system according to the number of instances, and calculating a unit span value according to the reserved capacity; Arranging each instance in a UV coordinate two-dimensional space in the order of each independent instance, taking the span value as a unit, taking U and V values as floating-point numbers between 0 and 1, obtaining U axis numbers and V axis numbers of each instance, and calculating UV coordinates of each instance according to the UV axis numbers of each instance and the span value, and modifying the UV coordinate data of each instance from original default values to the UV coordinate values after arrangement; Merging the plurality of instances into a collection instance, and generating data files in sequence according to the UV axis numbers and instance data of each instance and exporting the data files.

4. The method of claim 1, wherein the method is implemented based on an instant 3D engine. Selecting a certain local part in the collection instance and reading UV coordinates thereof, obtaining a UV axis number of the local part according to the UV coordinates, specifically comprising: Reading the UV coordinates of the local part in the collection instance through a ray detection function of the 3D engine; Calculating the UV axis number of the local part according to the UV coordinates and the span value.

5. The method of claim 1, wherein the method is implemented based on an instant 3D engine. The method further comprises: Transmitting the UV coordinates of the local part to a graphics shader as a parameter, obtaining a local position through calculation by a GPU, and specially displaying the local part to realize interactive feedback.

6. The method of claim 2, wherein the method is implemented by a real-time 3D engine-based 100 million-level instance retrieval interaction method. The method further comprises: Secondarily developing the DCC software to have the functions of reading external instance data, modifying instance UV, merging instances, and exporting data.

7. A system for implementing an interactive retrieval of millions of instances based on an instant 3D engine, characterized in that, The system comprises: A DCC module for making a plurality of independent instances, and independently storing instance data of each independent instance; sequentially arranging the multiple independent instances in a UV coordinate system, obtaining UV coordinate data of each instance after the arrangement and recording UV axis numbers, and merging the multiple instances into one set instance; a 3D engine module for importing the set instance and instance data into a 3D engine, displaying the set instance, and creating a three-dimensional data tree corresponding to the set instance, wherein an X axis of the three-dimensional data tree corresponds to U axis numbers of each instance, a Y axis corresponds to V axis numbers of each instance, and a Z axis corresponds to instance data of each instance; selecting a certain part in the set instance and reading UV coordinates of the part, obtaining UV axis numbers of the part according to the UV coordinates of the part, the UV axis numbers of the part being XY values of the part in the three-dimensional data tree, and reading instance data contained in a Z value of the part in the three-dimensional data tree according to the XY values of the part.

8. A computer storage medium, characterized in that The computer storage medium contains one or more program instructions for being executed by a system based on a real-time 3D engine and capable of implementing retrieval interaction of ten million instances, and the one or more program instructions are used to implement the method according to any one of claims 1-6.

Images