3D cultural relic digital model construction method and system for constructing online museum

Through NeRF model and voxelization processing technology, the problems of high cost and insufficient model quality of traditional 3D cultural relics digital modeling equipment are solved, and high-quality 3D cultural relics models are generated on low-cost equipment, and the rendering efficiency is improved.

CN119091094AInactive Publication Date: 2024-12-06SICHUAN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411176831.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional 3D cultural relics digital modeling equipment is costly, the model quality is insufficient, and different modeling and scanning technologies are not suitable for small or fine objects, resulting in insufficient model accuracy and equipment limitations.

Method used

The NeRF model is used to combine image information and voxelization processing to establish an initial reconstruction model through image information, extract 3D feature maps, calculate the location and size of high-frequency regions, perform model reconstruction, generate high-quality 3D cultural relics digital models, and use partitioning Server management improves rendering efficiency.

Benefits of technology

It reduces equipment costs, improves model quality and rendering efficiency, and can digitize cultural relics with the help of mobile phones or cameras, reduces storage space usage, and improves user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091094A_ABST
    Figure CN119091094A_ABST
Patent Text Reader

Abstract

The invention discloses a 3D cultural relic digital model construction method and system for constructing an online museum, and relates to the field of digital model construction, and the method comprises the steps: obtaining the image information of each target sub-scene, and building an initial reconstruction model corresponding to each target sub-scene; inputting the coordinates of each 3D point in the initial reconstruction model into a NeRF model for processing to obtain the radiation field intensity and color of each 3D point; extracting a plurality of 3D feature maps from each voxel of each initial reconstruction model; calculating the feature change rate of local voxels to obtain the position and size of a high-frequency region in each 3D feature map; determining each reconstructed initial reconstruction model as a 3D cultural relic target model of a single object or a large-scale scene; the low-frequency region is represented by fewer voxel points, and the high-frequency region is represented by more voxels, so that the memory is optimized to the maximum extent, the storage space of the whole model is reduced, and the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of digital model construction, and more specifically, to a method and system for constructing a 3D cultural relic digital model for building an online museum. Background Art

[0002] The existing 3D digital modeling of cultural relics is mainly based on traditional three-dimensional reconstruction technologies such as 3D scanning. Through data collection, relevant point cloud data sets are established, and three-dimensional reconstruction and mapping optimization are used to form a 3D digital model of cultural relics. The principle is: by using high-precision optical sensors, the appearance, texture and color information of cultural relics are captured at multiple angles and directions. This information is then converted into point cloud data to form a point cloud model containing a large number of discrete points; then the point cloud data processing algorithm is used to reconstruct the discrete point cloud into a continuous surface model, such as triangulation and surface reconstruction, and then the model processed by the point cloud data is texture mapped and material adjusted to finally generate a more realistic three-dimensional model.

[0003] Disadvantages of traditional construction of 3D cultural relics exhibition halls:

[0004] High equipment cost: 3D scanning equipment and related software are usually expensive, which may limit the use of 3D digital modeling by some small institutions or individuals.

[0005] Model quality issues: In actual operations, problems such as insufficient model accuracy and inability to accurately restore details may occur. Although there are some ways to improve these problems, they often require more professional knowledge and more work.

[0006] Technical limitations: Different 3D modeling and scanning technologies have their own characteristics and limitations. For example, the modeling method used for large objects may not be suitable for small or delicate objects. The physical limitations of the scanner (such as measurement range and accuracy) may also affect the quality of the final model. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for constructing a 3D digital model of cultural relics for building an online museum, so as to solve the problems existing in the above-mentioned background technology.

[0008] The above technical objectives of the present invention are achieved through the following technical solutions:

[0009] In a first aspect, the present application provides a method for constructing a 3D digital model of cultural relics for building an online museum, comprising the following specific steps:

[0010] Divide the scene according to the elements in the scene to be modeled to obtain multiple target sub-scenes containing a single object or a large-scale scene;

[0011] Acquire image information of each target sub-scene from different angles, and establish an initial reconstruction model corresponding to each target sub-scene based on the image information, wherein the image information includes a picture set and a video;

[0012] For each initial reconstruction model, the coordinates of each 3D point in the initial reconstruction model are input into the preset NeRF model for processing to obtain the radiation field intensity and color of each 3D point;

[0013] voxelize each initial reconstruction model, and extract multiple 3D feature maps from each voxel of each initial reconstruction model based on the radiation field intensity and color of each 3D point in each initial reconstruction model;

[0014] Based on multiple 3D feature maps in each initial reconstruction model, the position and size of the high-frequency area in each 3D feature map are obtained by calculating the feature change rate of the local voxel;

[0015] The corresponding initial reconstruction model is reconstructed using the position and size of each high-frequency area, and each initial reconstruction model after reconstruction is determined as a 3D cultural relic target model of a single object or a large-scale scene.

[0016] Based on the above technical solution, the present invention can also be improved as follows.

[0017] Furthermore, the initial reconstruction model corresponding to each target sub-scene is established according to the image information, specifically:

[0018] The image information is decoded and frame extracted using ffmpeg to obtain the corresponding image set. The image set is processed by COLMAP to obtain the camera position and posture information of image information at different angles, and the three-dimensional point cloud data of each target sub-scene is obtained. An initial reconstruction model corresponding to each target sub-scene is established based on the three-dimensional point cloud data.

[0019] Furthermore, the actual color of each 3D point from different angles is obtained according to the color of each 3D point, specifically:

[0020]

[0021] In the formula, C predict represents the actual color presented at different angles, T(s) represents the probability that the light is not blocked before point s, σ(s) represents the probability density of light hitting the particle at point s, and C(s) represents the color emitted by the particle at point s.

[0022] Furthermore, the above NeRF model includes an input layer, a hidden layer and an output layer.

[0023] Furthermore, the above-mentioned extraction of multiple 3D feature maps from each voxel of each initial reconstruction model also includes:

[0024] The dynamic voxelization method is used to retain the original point and voxel information of each 3D point.

[0025] Furthermore, multiple 3D feature maps are extracted from each voxel of each initial reconstruction model, specifically:

[0026] The voxel data after voxelization is processed from different scales, and the voxel data is processed by convolution kernels or pooling layers of different sizes to capture spatial features at different levels. The 3D feature map includes spatial features.

[0027] Furthermore, for the position and size of the high-frequency area in each 3D feature map, RoI pooling operation is performed in the high-frequency area to extract high-frequency precise features of the region of interest from the 3D feature map.

[0028] In a second aspect, the present application provides a system for constructing a 3D digital model of cultural relics for building an online museum, which is applied to a method for constructing a 3D digital model of cultural relics for building an online museum in any one of the first aspects, including:

[0029] The first module is used to divide the scene according to the elements in the scene to be modeled, and obtain multiple target sub-scenes containing a single object or a large-scale scene;

[0030] The second module is used to obtain image information of each target sub-scene from different angles, and establish an initial reconstruction model corresponding to each target sub-scene according to the image information, the image information includes a picture set and a video;

[0031] The third module is used for inputting the coordinates of each 3D point in the initial reconstruction model into a preset NeRF model for processing, so as to obtain the radiation field intensity and color of each 3D point;

[0032] The fourth module is used to voxelize each initial reconstruction model, and extract multiple 3D feature maps from each voxel of each initial reconstruction model based on the radiation field intensity and color of each 3D point in each initial reconstruction model;

[0033] A fifth module is used to obtain the position and size of the high-frequency area in each 3D feature map by calculating the feature change rate of the local voxel based on the multiple 3D feature maps in each initial reconstruction model;

[0034] The sixth module is used to reconstruct the corresponding initial reconstruction model using the position and size of each high-frequency area, and determine each reconstructed initial reconstruction model as a 3D cultural relic target model of a single object or a large-scale scene.

[0035] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods in the first aspect when executing the computer program.

[0036] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable a computer to execute any one of the methods in the first aspect.

[0037] Compared with the prior art, the present invention has at least the following beneficial effects:

[0038] In this application, the traditional NeRF model is applied to the reconstruction of the exhibition hall, but the training speed is slow, and NeRF has major defects such as slow inference speed. That is, if a V100 GPU graphics card is used, it takes 1 minute to render a high-definition image. The traditional NeRF model occupies a large storage space and memory. For example, for 1024*1024*1024 resolution voxels, after model quantization and pruning, the storage space still reaches 300M. In addition, for some cultural relics with complex textures and colors, how to accurately reproduce and protect this information in the 3D model is a challenge, which requires not only precise hardware equipment, but also professional processing technology and methods. In this solution, through the detection of high-frequency areas of the 3D voxel model in the system of the entire virtual exhibition hall, fewer voxel points are used to represent the low-frequency areas, and more voxels are used to represent the high-frequency areas, so as to maximize the optimization of memory, reduce the storage space of the entire model, and improve the user experience.

[0039] In this application, traditional 3D reconstruction technology focuses on geometric expression rather than realistic rendering, which means that even if the reconstructed geometric accuracy is very high, the realistic rendering effect may not be obtained. The NeRF model pays more attention to realistic rendering and can render objects at any perspective in a new perspective synthesis manner, thereby simulating a 3D effect, without the need to use 3D printing technology to process the missing parts of its 3D reconstruction; at the same time, the Nerf model can use a mobile phone or camera to digitize exhibits, reducing the cost of cloud museums; the process is simple, only the neural network needs to be built and trained to process the input images and parameters, avoiding complicated steps; in the construction process, in order to increase the system construction speed without reducing the accuracy of the model, the detection of high-frequency areas of the 3D voxel model is used, and in order to improve the user experience and reduce the display time of the exhibition hall, the method of combining partitioned server management with NeRF is used. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0041] Figure 1 A method flow chart of a construction method in an embodiment of the present invention;

[0042] Figure 2 A connection diagram of a system for constructing an embodiment of the present invention;

[0043] Figure 3 Schematic diagram of the connection of electronic equipment in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0045] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0047] Embodiment 1: In view of the problems of high equipment cost, low model quality, technical limitations, etc. in the current establishment of 3D cultural relics exhibition halls, as well as the slow training speed and large memory usage of traditional NeRF models and methods, this embodiment provides a method for constructing a 3D cultural relics digital model for building an online museum, such as Figure 1 As shown, the specific steps include:

[0048] S1, dividing the scene according to the elements in the scene to be modeled, and obtaining multiple target sub-scenes containing a single object or a large-scale scene.

[0049] Among them, the elements in the scene to be modeled can be classified, for example, into single objects and large-scale scenes. A camera device can be used to shoot the single object or large-scale scene to be modeled to obtain a picture set or video, shooting as many angles as possible, and the more complex the background texture, the more conducive to the refinement of the modeling.

[0050] S2, acquiring image information of each target sub-scene from different angles, and establishing an initial reconstruction model corresponding to each target sub-scene according to the image information, wherein the image information includes a picture set and a video.

[0051] Optionally, the initial reconstruction model corresponding to each target sub-scene is established according to the image information, specifically:

[0052] The image information is decoded and frame extracted using ffmpeg to obtain the corresponding image set. The image set is processed by COLMAP to obtain the camera position and posture information of image information at different angles, and the three-dimensional point cloud data of each target sub-scene is obtained. An initial reconstruction model corresponding to each target sub-scene is established based on the three-dimensional point cloud data.

[0053] Among them, the image information is decoded and framed by ffmpeg to extract the image set, and then converted into a format that can be processed by NeRF; specifically, COLMAP is usually used for 3D reconstruction and camera positioning of images; in NeRF, COLMAP can be used to generate an initial 3D reconstruction model of the scene; first, the image set is processed by COLMAP to extract the camera's position and attitude information, and estimate the 3D point cloud in the scene; then, these camera poses and 3D point cloud data can be used as input to the NeRF model, and the 3D point cloud generated by COLMAP is used as the initialization data of the NeRF model, which helps NeRF learn the depth and color information of the scene faster.

[0054] Among them, COLMAP (Image-based 3D Reconstruction and Camera Positioning Software) is a tool for image processing, 3D reconstruction and camera positioning; COLMAP uses advanced image feature extraction and matching algorithms such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features), etc. Based on image sets of different qualities, COLMAP supports sparse reconstruction and dense reconstruction to calculate the camera position and 3D point cloud.

[0055] S3, for each initial reconstruction model, input the coordinates of each 3D point in the initial reconstruction model into a preset NeRF model for processing to obtain the radiation field intensity and color of each 3D point.

[0056] Among them, NeRF (Neural Radiance Fields) is a neural network model for 3D scene reconstruction and rendering. NeRF learns a function that represents the entire 3D scene. The function accepts the coordinates of a 3D point as input and outputs the radiation field intensity and color of the point. This representation function is used to capture the geometric and color information in the scene. The core of NeRF is an eight-layer MLP neural network, which is used to approximate the representation function. Each layer is a fully connected MLP layer, which contains activation functions, weights, and biases. The structure of the eight-layer MLP network is as follows: Input layer: Receives the coordinates of 3D points as input. Hidden layers 1-6: Each hidden layer contains 256 neurons and uses ReLU (Rectified Linear Unit) as the activation function. Output layer: Outputs the radiation field intensity and color of each 3D point in the scene.

[0057] Optionally, the NeRF model includes an input layer, a hidden layer and an output layer.

[0058] Optionally, the actual color of each 3D point presented from different angles is obtained according to the color of each 3D point, specifically:

[0059]

[0060] In the formula, C predict represents the actual color presented at different angles, T(s) represents the probability that the light is not blocked before point s, σ(s) represents the probability density of light hitting the particle at point s, and C(s) represents the color emitted by the particle at point s.

[0061] Among them, the coordinates of 3D points are usually encoded as the input of the network. In order to handle inputs of different scales, the coordinates are usually mapped to vectors of fixed dimensions, which can be achieved by splicing or other methods; during the rendering process, NeRF samples camera rays. These rays are emitted from the camera position along the line of sight, pass through the 3D points in the scene, and then arrive at the image plane, sampling a series of 3D points for each ray; for each sampled point, forward propagation is performed through the MLP network to obtain an estimate of the intensity and color of the radiation field. This step calculates the intensity and color of the light passing through each point in the scene.

[0062] S4, voxelizing each initial reconstructed model, and extracting multiple 3D feature maps from each voxel of each initial reconstructed model based on the radiation field intensity and color of each 3D point in each initial reconstructed model.

[0063] Among them, in the entire virtual exhibition hall system, each model contains a large number of voxels, which will cause the entire model to occupy a large storage space and a slow user experience. Therefore, we proposed a 3D voxel model high-frequency detection algorithm, which uses fewer voxel points to represent low-frequency areas and more voxels to represent high-frequency areas.

[0064] Optionally, the above step of extracting multiple 3D feature maps from each voxel of each initial reconstructed model further includes:

[0065] The dynamic voxelization method is used to retain the original point and voxel information of each 3D point.

[0066] Optionally, multiple 3D feature maps are extracted from each voxel of each initial reconstruction model, specifically:

[0067] The voxel data after voxelization is processed from different scales, and the voxel data is processed by convolution kernels or pooling layers of different sizes to capture spatial features at different levels. The 3D feature map includes spatial features.

[0068] Optionally, for the position and size of the high-frequency area in each 3D feature map, a RoI pooling operation is performed in the high-frequency area to extract high-frequency precise features of the region of interest from the 3D feature map.

[0069] S5, based on the multiple 3D feature maps in each initial reconstruction model, the position and size of the high-frequency area in each 3D feature map are obtained by calculating the feature change rate of the local voxel.

[0070] Among them, voxel fusion includes the following specific steps: Voxelization: Voxelization is the process of discretizing the continuous three-dimensional space into small cubes (voxels). In 3D object detection, voxelization helps to convert unstructured point cloud data into a regular form suitable for CNNs processing.

[0071] Voxel feature extraction: The voxelized data is subjected to feature extraction through a 3D convolutional neural network to generate a 3D feature map containing rich spatial information. This step is the basis for subsequent high-frequency area detection.

[0072] High-frequency region detection: Design an algorithm to automatically identify and mark high-frequency regions in the 3D feature map. Calculate the feature change rate of local voxels to predict the location and size of high-frequency regions.

[0073] Dynamic voxel allocation: In order to optimize storage and computational efficiency, a dynamic voxelization method can be used to retain all original point and voxel information, avoid information loss due to downsampling, and ensure the stability of the detection results.

[0074] Multi-scale voxel processing: Process voxel data at different scales to capture spatial features at different levels. For example, using convolution kernels or pooling layers of different sizes to process voxel data enables more accurate detection of high-frequency areas.

[0075] Spatial Proximity Enhancement: Voxels are sorted by the Hilbert curve technique to preserve spatial proximity, which helps improve the accuracy and efficiency of subsequent high-frequency region detection.

[0076] RoI pooling operation: Apply RoI (Region of Interest) pooling operation on the detected high-frequency area to accurately extract the features of the region of interest from the 3D feature map to prepare for further classification or regression tasks.

[0077] Specifically, the entire NeRF training process involves using known 3D point clouds and corresponding colors as supervisory signals to optimize the parameters of the MLP network by minimizing the difference between the rendering results and the real image. This enables the network to learn to capture the details and structure of the scene, thereby achieving high-quality 3D scene reconstruction and rendering.

[0078] S6, reconstructing the corresponding initial reconstruction model using the position and size of each high-frequency area, and determining each reconstructed initial reconstruction model as a 3D cultural relic target model of a single object or a large-scale scene.

[0079] The following steps may also be included:

[0080] 1. Divide the exhibition hall into multiple nodes, and divide each node into a primary node and a secondary node.

[0081] 2. The master node performs preliminary rendering of the entire scene and distributes different partition data to different slave node servers.

[0082] 3. The master node server merges the regional models of each slave node, and for the current user's observation direction, the master node server calculates the light path and determines the model perspective of the user's perspective.

[0083] 4. Calculate the model angle voxel information of each partition based on NeRF’s rendering neural network.

[0084] Embodiment 2: This embodiment of the present application provides a system for constructing a 3D digital model of cultural relics for building an online museum, which is applied to a method for constructing a 3D digital model of cultural relics for building an online museum in any one of Embodiment 1, such as Figure 2 As shown, including:

[0085] The first module is used to divide the scene according to the elements in the scene to be modeled, and obtain multiple target sub-scenes containing a single object or a large-scale scene.

[0086] The second module is used to obtain image information of each target sub-scene from different angles, and establish an initial reconstruction model corresponding to each target sub-scene based on the image information, and the image information includes picture sets and videos.

[0087] The third module is used to input the coordinates of each 3D point in the initial reconstruction model into a preset NeRF model for processing, so as to obtain the radiation field intensity and color of each 3D point.

[0088] The fourth module is used to voxelize each initial reconstruction model, and extract multiple 3D feature maps from each voxel of each initial reconstruction model based on the radiation field intensity and color of each 3D point in each initial reconstruction model.

[0089] The fifth module is used to obtain the position and size of the high-frequency area in each 3D feature map by calculating the feature change rate of the local voxel based on the multiple 3D feature maps in each initial reconstruction model.

[0090] The sixth module is used to reconstruct the corresponding initial reconstruction model using the position and size of each high-frequency area, and determine each reconstructed initial reconstruction model as a 3D cultural relic target model of a single object or a large-scale scene.

[0091] Embodiment 3: This embodiment of the present application provides an electronic device, such as Figure 3 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any method in Embodiment 1 is implemented.

[0092] Embodiment 4: The embodiment of the present application provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable a computer to execute any method in Embodiment 1.

[0093] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a 3D digital model of cultural relics for building an online museum, characterized in that: The specific steps include: Divide the scene according to the elements in the scene to be modeled to obtain multiple target sub-scenes containing a single object or a large-scale scene; Acquire image information of each of the target sub-scenes from different angles, and establish an initial reconstruction model corresponding to each of the target sub-scenes according to the image information, wherein the image information includes a picture set and a video; For each of the initial reconstruction models, the coordinates of each 3D point in the initial reconstruction model are input into a preset NeRF model for processing to obtain the radiation field intensity and color of each 3D point; voxelize each of the initial reconstructed models, and extract multiple 3D feature maps from each voxel of each initial reconstructed model based on the radiation field intensity and color of each 3D point in each initial reconstructed model; Based on multiple 3D feature maps in each initial reconstruction model, the position and size of the high-frequency area in each 3D feature map are obtained by calculating the feature change rate of the local voxel; The corresponding initial reconstruction model is reconstructed using the position and size of each high-frequency area, and each initial reconstruction model after reconstruction is determined as a 3D cultural relic target model of a single object or a large-scale scene.

2. A method for constructing a 3D cultural relic digital model for building an online museum according to claim 1, characterized in that: The initial reconstruction model corresponding to each target sub-scene is established according to the image information, specifically: The image information is decoded and frame extracted using ffmpeg to obtain a corresponding image set, and the image set is processed by COLMAP to obtain the camera position and posture information of image information at different angles, and the three-dimensional point cloud data of each target sub-scene is obtained. An initial reconstruction model corresponding to each target sub-scene is established based on the three-dimensional point cloud data.

3. The method for constructing a 3D cultural relic digital model for building an online museum according to claim 1, characterized in that: According to the color of each 3D point, the actual color of each 3D point from different angles is obtained, specifically: In the formula, C predict represents the actual color presented at different angles, T(s) represents the probability that the light is not blocked before point s, σ(s) represents the probability density of light hitting the particle at point s, and C(s) represents the color emitted by the particle at point s.

4. The method for constructing a 3D cultural relic digital model for building an online museum according to claim 1, characterized in that: The NeRF model includes an input layer, a hidden layer and an output layer.

5. The method for constructing a 3D cultural relic digital model for building an online museum according to claim 1, characterized in that: The step of extracting a plurality of 3D feature maps from each voxel of each initial reconstruction model further includes: The dynamic voxelization method is used to retain the original point and voxel information of each 3D point.

6. A method for constructing a 3D digital model of cultural relics for building an online museum according to claim 1, characterized in that: The multiple 3D feature maps are extracted from each voxel of each initial reconstruction model, specifically: The voxel data after voxelization is processed at different scales, and the voxel data is processed by convolution kernels or pooling layers of different sizes to capture spatial features at different levels, and the 3D feature map includes spatial features.

7. The method for constructing a 3D digital model of cultural relics for building an online museum according to claim 1, characterized in that: For the position and size of the high-frequency area in each 3D feature map, RoI pooling operation is performed in the high-frequency area to extract high-frequency and accurate features of the region of interest from the 3D feature map.

8. A system for constructing a 3D digital model of cultural relics for building an online museum, applied to a method for constructing a 3D digital model of cultural relics for building an online museum as claimed in any one of claims 1 to 7, characterized in that: include: The first module is used to divide the scene according to the elements in the scene to be modeled, and obtain multiple target sub-scenes containing a single object or a large-scale scene; The second module is used to obtain image information of each target sub-scene from different angles, and establish an initial reconstruction model corresponding to each target sub-scene according to the image information, wherein the image information includes a picture set and a video; The third module is used for inputting the coordinates of each 3D point in the initial reconstruction model into a preset NeRF model for processing, so as to obtain the radiation field intensity and color of each 3D point; The fourth module is used to voxelize each of the initial reconstructed models, and extract multiple 3D feature maps from each voxel of each initial reconstructed model based on the radiation field intensity and color of each 3D point in each initial reconstructed model; A fifth module is used to obtain the position and size of the high-frequency area in each 3D feature map by calculating the feature change rate of the local voxel based on the multiple 3D feature maps in each initial reconstruction model; The sixth module is used to reconstruct the corresponding initial reconstruction model using the position and size of each high-frequency area, and determine each reconstructed initial reconstruction model as a 3D cultural relic target model of a single object or a large-scale scene.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the method according to any one of claims 1 to 7 is implemented when the processor executes the computer program.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, which enable a computer to execute the method of any one of claims 1-7.

Citation Information

Patent Citations

  • Three-dimensional reconstruction method, device and equipment based on neural radiation field and storage medium

    CN116310076A

  • Three-dimensional reconstruction method of electric power operation and maintenance equipment image and related equipment

    CN117372620A