Reality model file access methods, systems, devices, media, and program products

Abstract

This invention discloses a method, system, device, medium, and program product for accessing real-world model files. The method includes: first, packaging the real-world model files into a preset format file, which includes a file header area, a file index area, and a data body area; then, according to a user request, performing index access on the preset format file to obtain the access result. The index access includes single-file index access and hierarchical index access. This invention achieves random access to a single model tile file and continuous data range reading based on the LOD level without decompressing or expanding the file, significantly reducing file system pressure and improving the efficiency of real-world model file access.

Description

Technical Field

[0001] This invention relates to the field of model data processing technology, and in particular to a method, system, device, medium and program product for accessing real-world model files. Background Technology

[0002] With the development of digital twins, smart cities, and the digitalization of natural resources, technologies such as oblique photogrammetry and laser scanning are widely used in 3D reality modeling. The 3D reality model data generated by these technologies is typically organized in a layered tile format, such as 3DTiles (containing .json, .b3dm, .i3dm, .cmpt, etc.) or OSGB format. The 3D reality models generated by this technology usually contain spatial structure description files and massive amounts of model data files (such as .b3dm, .i3dm, .cmpt, .osgb, etc.).

[0003] However, existing 3D reality data generally suffers from the following problems in engineering applications:

[0004] 1. Severe file fragmentation: City-level or regional-level real-world projects often contain millions or even tens of millions of small files, which can easily lead to the exhaustion of file system inode resources and significantly reduce the operating system's response performance in scenarios such as directory traversal and file retrieval.

[0005] 2. Low transmission and deployment efficiency: During data copying, network transmission, or cloud upload, a large number of small files will introduce extremely high I / O scheduling and protocol overhead, making the actual transmission speed far lower than the theoretical bandwidth of the disk or network.

[0006] 3. Inconvenient project delivery and use: Existing project delivery usually relies on compression methods such as ZIP and TAR. Users must completely decompress the files before use, which is a complicated and time-consuming process and is not conducive to online browsing and on-demand loading.

[0007] 4. Common archiving formats are not suitable for 3D spatial access modes: Formats such as ZIP and TAR are not optimized for the hierarchical and spatial continuity of 3D spatial data. For example, the central directory of a ZIP file is located at the end of the file. In network streaming scenarios, it is usually necessary to download the complete index to locate the data, making it difficult to support continuous range reading based on hierarchy (LOD).

[0008] Therefore, there is an urgent need for a single-file storage and access solution tailored to the characteristics of 3D reality models, in order to solve the problems of excessive file quantity, low access efficiency, and inability to load on demand in existing technologies. Summary of the Invention

[0009] In view of this, embodiments of the present invention provide a method, system, device, medium, and program product for accessing real-world model files, which at least partially solves the problems existing in the prior art.

[0010] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0011] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0012] According to a first aspect of the present invention, a method for accessing reality model files is provided, the method comprising:

[0013] The real-world model file is packaged into a preset format file, which includes a file header area, a file index area, and a data body area.

[0014] Based on the user's request, the preset format file is indexed and accessed to obtain the access result. The indexing access includes single-file indexing access and hierarchical indexing access.

[0015] Furthermore, the file header area includes a fixed identifier field, a version number field, a maximum LOD level field, a reserved bit field, a global bounding box field, a file index table field, a level index table field, and a reserved byte segment;

[0016] The file index area includes a path hash value field, a byte offset field, a data byte length field, an original file path string field, and a corresponding LOD level field.

[0017] The data body area is used to store model data files in LOD hierarchical order.

[0018] Further, the preset format file is indexed and accessed to obtain the access results, including:

[0019] When the index access is a single file index access, read the first preset number of bytes of the file header area of ​​the preset format file;

[0020] Based on the aforementioned preset number of bytes, the fixed identifier field of the preset format file is validated;

[0021] After the verification is successful, the access path hash corresponding to the logical path of the file to be accessed is calculated based on the logical path of the file to be accessed entered by the user.

[0022] Based on the file index table field in the file header area, read the file index area of ​​the preset format file;

[0023] Within the file index area, the target file is matched for the access path hash;

[0024] Based on the byte offset field and data byte length field of the file index area, the storage location information of the target file in the data body area is obtained, and the storage location information includes the absolute byte offset and byte length;

[0025] Based on the storage location information of the target file, the target file data is read from the data body area.

[0026] Furthermore, based on the user's request, the preset format file is indexed and accessed to obtain the access results, including:

[0027] When the index access is the hierarchical index access, the hierarchical index table field in the file header area of ​​the preset format file is parsed to obtain the hierarchical data index table;

[0028] Based on the hierarchical data index table, the target hierarchical range is determined according to the user's access intent;

[0029] Based on the target hierarchical interval, the hierarchical offset of the target hierarchical interval in the data volume area is calculated, and the hierarchical offset includes a start offset and an end offset;

[0030] Based on the level offset, the target level data corresponding to the target level interval is read from the data volume area and output.

[0031] Furthermore, according to the user's request, the preset format file is indexed and accessed to obtain the access result, which also includes:

[0032] After matching the target file corresponding to the access path hash, the target file is checked for hash collision based on the original file path string field in the file index area.

[0033] Furthermore, according to the user's request, the preset format file is indexed and accessed to obtain the access result, which also includes:

[0034] For the target level data to be accessed corresponding to the downloaded target level range, the file index area is used to parse and manage the cache of each level of data to be accessed locally.

[0035] According to a second aspect of the present invention, a reality model file access system is provided, the system comprising:

[0036] The model file packaging module is used to package real-world model files into a preset format file, which includes a file header area, a file index area, and a data body area.

[0037] The model file access module is used to perform index access on the preset format file according to the user request and obtain the access result. The index access includes single file index access and hierarchical index access.

[0038] According to a third aspect of the present invention, a reality model file access device is provided, the device comprising: a processor and a memory;

[0039] The memory is used to store one or more program instructions;

[0040] The processor is configured to run one or more program instructions to perform the steps of a real-world model file access method as described in any of the preceding claims.

[0041] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, wherein when executed by a processor, the computer program implements the steps of a real-world model file access method as described in any of the preceding claims.

[0042] According to a fifth aspect of the present invention, a computer program product is provided, the computer program product comprising a computing program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to implement the steps of a real-world model file access method as described in any of the preceding claims.

[0043] This invention provides a method, system, device, medium, and program product for accessing real-world model files. The method includes: first, packaging the real-world model file into a preset format file, the preset format file including a file header area, a file index area, and a data body area; then, according to a user request, performing index access on the preset format file to obtain the access result, the index access including single-file index access and hierarchical index access. This invention achieves random access to a single model tile file and continuous data range reading based on LOD levels without decompressing or expanding the file, significantly reducing file system pressure and improving the efficiency of real-world model file access. Attached Figure Description

[0044] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0045] Figure 1A flowchart illustrating a method for accessing real-world model files provided in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the structure of a real-scene model file access system provided in an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the file structure of a preset format file provided in an embodiment of the present invention. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0049] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] Figure 1 A flowchart of a method for accessing reality model files according to an embodiment of the present invention is shown.

[0051] like Figure 1 As shown, the method for accessing reality model files according to an embodiment of the present invention may include steps S100 and S200.

[0052] In step S100, the real-scene model file is packaged into a preset format file.

[0053] Specifically, the above steps include:

[0054] The default format file (.3drp) consists of a header, a file index, and a data body, from beginning to end. Figure 3 The file structure of the preset format file is shown.

[0055] The file header is located at the beginning of the file and has a fixed length of 512 bytes. It is used to quickly obtain global metadata and index location information. Its structure is defined in byte order as shown in Table 1:

[0056] Table 1. File Header Fields Table

[0057] Field Name Length (Byte) type describe Fixed identifier field MagicNumber 4 Byte[4] Fixed identifier: 0x33445250 (ASCII: 3DRP) Version number field 2 Uint16 Version number MaxLevel, the maximum level of order (LOD) field 1 Uint8 The maximum number of LOD levels contained in the data Reserved bit field 1 Byte Reserved bits for byte alignment Global bounding box field BoundingBox 48 Double[6] Global bounding box (minX, minY, minZ, maxX, maxY, maxZ) FileIndexTable field 8 Uint64 File index table, total length of the file index area (Length) LevelIndexTable field 256 Uint64

[32] The hierarchical index table supports a fixed range of 0-31 levels, with each value representing the total number of bytes of data at that level. Note: If a level has no data, the value is 0. Reserved byte segment Extra 192 Byte

[192] Reserved bytes

[0058] The embodiments of the present invention use a fixed-length file header design, which allows the client to complete format recognition, spatial range determination and hierarchical distribution analysis by only reading the beginning part of the file.

[0059] The file index area follows the file header area and is used to describe the location and attribute information of each packaged file in the data body area. The file index area consists of several file index entries in sequence, and its structure is defined in byte order as shown in Table 2:

[0060] Field Name Length (Byte) type describe PathHash field 8 Uint64 The hash value of the original file path string is calculated using the XXHash64 algorithm for fast location. Byte offset field DataOffset 8 Uint64 The absolute byte offset of the file in the data body area Data byte length field DataLength 4 Uint32 Byte length of the corresponding data in this file The PathLength field represents the length of the original file path string in bytes. 2 Uint16 Byte length of the original file path string PathString field PathLength Byte[] The original file path string is used to handle hash collisions and restore the path. Level field of the corresponding LOD 1 Uint8 LOD level to which this file belongs

[0061] The total length of the above file index area is determined by the FileIndexLength field in the file header area.

[0062] The data body area follows the file index area and is used to sequentially store all packaged model data files (binary raw content).

[0063] The key constraints of the data volume area are: 1. Data in the data volume area must be physically arranged strictly according to the LOD hierarchy order; 2. Files within the same level can be stored in any order; 3. Levels must not be mixed or overlapped.

[0064] In other words, the data volume area first stores all file data of Level 0 continuously; then it stores all file data of Level 1 continuously; and so on, up to the maximum level MaxLevel.

[0065] The data body area of ​​this embodiment adopts the above-described physical layout, which provides a structural basis for subsequent hierarchical continuous range reading.

[0066] Next, in step S200, according to the user request, the preset format file is indexed and accessed to obtain the access result. The indexing access includes single file indexing access and hierarchical indexing access.

[0067] Specifically, the above steps include:

[0068] When index access is a single-file index access, the first 512 bytes of the file header area of ​​the preset format file are read. Based on these 512 bytes, the fixed identifier field (0x33445250) of the preset format file is validated to confirm the file format is valid. After successful validation, the access path hash corresponding to the logical path of the file to be accessed (e.g., Data / 0 / 2 / 0.b3dm) is calculated using the XXHash64 algorithm, for example, the hash value of "Data / 0 / 2 / 0.b3dm" is 287959845153183541. Then, based on the file header area... The FileIndexLength field in the file index table reads the complete file index area of ​​a file in a preset format. Within the file index area, the corresponding PathHash is matched to determine the target file. If necessary, hash collision checks are performed on the target file based on the original file path string field in the file index area. The storage location information of the target file in the data body area is obtained based on the byte offset field DataOffset and the data byte length field DataLength in the file index area. The storage location information includes the absolute byte offset and byte length. Finally, based on the storage location information of the target file, the corresponding data is directly read via HTTP RangeRequest or local random read.

[0069] When index access is a hierarchical index access, the LevelIndexTable field in the header area of ​​the pre-formatted file is parsed to obtain the hierarchical data index table. Based on the hierarchical data index table, the target hierarchical range is determined according to the user's access intent, for example, level m to level n (starting from level 0). Based on the target hierarchical range, the hierarchical offset of the target hierarchical range in the data body area is calculated. The hierarchical offset includes a start offset and an end offset. The formula for calculating the start offset of level m is as follows: The formula for calculating the end offset of the nth layer is: Based on the hierarchical offset and the continuous HTTP Range requests initiated by the client, read and output the target hierarchical data corresponding to the target hierarchical range from the data body area.

[0070] For the data to be accessed corresponding to the target level range after downloading, the file index area is used to parse and manage the data cache of each level of the data to be accessed locally.

[0071] The embodiments of the present invention, through the above-described hierarchical index access method, can efficiently complete the hierarchical data retrieval without traversing a large number of file indexes.

[0072] In addition, embodiments of the present invention also provide a real-world model file access system. Figure 2This diagram illustrates the structure of a reality model file access system, which specifically includes:

[0073] The model file packaging module is used to package real-world model files into preset format files, which include a file header area, a file index area, and a data body area.

[0074] The model file access module is used to index and access files in a preset format according to user requests, and obtain the access results. Index access includes single file index access and hierarchical index access.

[0075] In addition, embodiments of the present invention also provide a reality model file access device, the device comprising: a processor and a memory; the memory being used to store one or more program instructions; the processor being used to execute one or more program instructions to perform the steps of a reality model file access method as described above.

[0076] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a real-world model file access method as described above.

[0077] In addition, embodiments of the present invention also provide a computer program product, which includes computer program instructions that, when executed by a processor, implement the steps of a real-world model file access method as described above.

[0078] The real-world model file access method, system, device, medium, and program products provided in this invention package the originally scattered real-world model files into a single independent file (example extension: .3drp, i.e., 3D Reality Package) by defining a unified binary file structure, and introduce two types of indexing mechanisms within the file: file-level indexing and level-level indexing. This invention enables random access to a single model tile file, continuous data range reading based on LOD levels, batch acquisition of multiple consecutive LOD level data, and efficient support for streaming access methods such as HTTP Range Requests, without decompressing or expanding the file.

[0079] Compared with the prior art, the present invention has the following advantages:

[0080] The embodiments of the present invention can significantly reduce the pressure on the file system: integrate massive fragmented files into a single file for storage, avoid the problem of innode resource exhaustion, and improve the efficiency of data deployment and management.

[0081] The embodiments of the present invention can improve network transmission and loading performance: by combining hierarchical indexing and physical contiguous storage, the client can obtain complete hierarchical data with a very small number of range requests.

[0082] The embodiments of the present invention can accommodate both random access and batch access: while supporting precise positioning of single tiles, it also supports efficient preloading and caching by LOD level.

[0083] The parsing process of this invention is simple and reliable: the fixed-length file header and clear secondary index structure enable the parser to make access decisions without decompressing or scanning the entire file.

[0084] This invention is particularly suitable for the efficient application of large-scale 3D reality models in scenarios such as network transmission, online browsing, and engineering delivery.

[0085] In this embodiment of the invention, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in this embodiment of the invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in this embodiment of the invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the storage medium and, in conjunction with its hardware, completes the steps of the above methods. The storage medium can be memory, for example, volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM).The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory. Those skilled in the art will recognize that the functions described in the above examples can be implemented using a combination of hardware and software. When applied software, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers. Although the invention has been described in detail above with general description and specific embodiments, modifications or improvements can be made to it, which will be apparent to those skilled in the art. Therefore, such modifications or improvements made without departing from the spirit of the invention are all within the scope of protection claimed by the invention.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, or alterations made by those skilled in the art using the disclosed technical content shall fall within the protection scope of the present invention.

Claims

1. A method for accessing real-world model files, characterized in that, The method includes: The real-world model file is packaged into a preset format file, which includes a file header area, a file index area, and a data body area. Based on the user's request, the preset format file is indexed and accessed to obtain the access result. The indexing access includes single-file indexing access and hierarchical indexing access.

2. The method for accessing a real-world model file according to claim 1, characterized in that, The file header area includes a fixed identifier field, a version number field, a maximum LOD level field, a reserved bit field, a global bounding box field, a file index table field, a level index table field, and a reserved byte segment; The file index area includes a path hash value field, a byte offset field, a data byte length field, an original file path string field, and a corresponding LOD level field. The data body area is used to store model data files in LOD hierarchical order.

3. The method for accessing a real-world model file according to claim 2, characterized in that, Based on the user's request, the preset format file is indexed and accessed to obtain the access results, including: When the index access is a single file index access, read the first preset number of bytes of the file header area of ​​the preset format file; Based on the aforementioned preset number of bytes, the fixed identifier field of the preset format file is validated; After the verification is successful, the access path hash corresponding to the logical path of the file to be accessed is calculated based on the logical path of the file to be accessed entered by the user. Based on the file index table field in the file header area, read the file index area of ​​the preset format file; Within the file index area, the target file is matched for the access path hash; Based on the byte offset field and data byte length field of the file index area, the storage location information of the target file in the data body area is obtained, and the storage location information includes the absolute byte offset and byte length; Based on the storage location information of the target file, the target file data is read from the data body area.

4. The method for accessing a real-world model file according to claim 2, characterized in that, Based on the user's request, the preset format file is indexed and accessed to obtain the access results, including: When the index access is the hierarchical index access, the hierarchical index table field in the file header area of ​​the preset format file is parsed to obtain the hierarchical data index table; Based on the hierarchical data index table, the target hierarchical range is determined according to the user's access intent; Based on the target hierarchical interval, the hierarchical offset of the target hierarchical interval in the data volume area is calculated, and the hierarchical offset includes a start offset and an end offset; Based on the level offset, the target level data corresponding to the target level interval is read from the data volume area and output.

5. The method for accessing a real-world model file according to claim 3, characterized in that, According to the user's request, the preset format file is indexed and accessed to obtain the access result, which also includes: After matching the target file corresponding to the access path hash, the target file is checked for hash collision based on the original file path string field in the file index area.

6. The method for accessing a real-world model file according to claim 4, characterized in that, According to the user's request, the preset format file is indexed and accessed to obtain the access result, which also includes: For the target level data to be accessed corresponding to the downloaded target level range, the file index area is used to parse and manage the cache of each level of data to be accessed locally.

7. A system for accessing real-world model files, characterized in that, The system includes: The model file packaging module is used to package real-world model files into a preset format file, which includes a file header area, a file index area, and a data body area. The model file access module is used to perform index access on the preset format file according to the user request and obtain the access result. The index access includes single file index access and hierarchical index access.

8. A device for accessing real-world model files, characterized in that, The device includes: a processor and a memory; The memory is used to store one or more program instructions; The processor is configured to run one or more program instructions to perform the steps of a real-world model file access method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a real-world model file access method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes computer program instructions that, when executed by a processor, implement the steps of a real-world model file access method as described in any one of claims 1 to 6.

Images