3D model construction method and system for industrial metaverse

Abstract

The application provides an industrial meta-universe 3D model construction method and system, wherein the industrial meta-universe 3D model construction method comprises the following steps: acquiring multiple target device images by using an image acquisition unit to acquire images of a detected target device from multiple angles in a target scene; selecting image pairs meeting preset conditions from the multiple target device images; performing BA processing on the image pairs in sequence by using SFM to obtain geometric information of the target scene and parameter information of the image acquisition unit; and processing the multiple target device images, the geometric information of the target scene and the parameter information of the image acquisition unit by using a NeRF model to construct a 3D model of an industrial meta-universe corresponding to the target scene. Thus, full-automatic industrial meta-universe 3D modeling can be realized without human participation, and therefore, the modeling efficiency is greatly improved and the cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, specifically to a method and system for constructing a 3D model of an industrial metaverse. Background Technology

[0002] In related technologies, the construction of 3D models of the industrial metaverse usually requires manual intervention, making fully automated modeling impossible. Therefore, the efficiency is low and the cost is high. Summary of the Invention

[0003] To solve the above-mentioned technical problems, this invention provides a method for constructing a 3D model of an industrial metaverse, which can realize fully automatic 3D modeling of the industrial metaverse without human intervention. Therefore, it greatly improves the efficiency of modeling and reduces costs.

[0004] The technical solution adopted in this invention is as follows:

[0005] A method for constructing a 3D model of an industrial metaverse includes the following steps: using an image acquisition unit to acquire images of a detected target device from multiple angles in a target scene to obtain multiple images of the target device; selecting image pairs that meet preset conditions from the multiple images of the target device; and using SFM to sequentially perform BA processing on the image pairs to obtain the geometric information of the target scene and the parameter information of the image acquisition unit.

[0006] The NeRF model is used to process the multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit to construct the 3D model of the industrial metaverse corresponding to the target scene.

[0007] In one embodiment of the present invention, a NeRF model is used to process the multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit to construct a 3D model corresponding to the target scene. This includes: inputting the multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit into a multilayer perceptron to obtain corresponding volume density and RGB color values; using volume rendering technology to synthesize the volume density and RGB color values ​​into a corresponding texture; and constructing the 3D model based on the texture.

[0008] In one embodiment of the present invention, selecting image pairs that meet preset conditions from multiple target device images includes: extracting focal length information from the multiple target device images and extracting corresponding image features using a feature extraction algorithm; calculating the Euclidean distance between image features of every two target device images using a Kd-Tree model, performing feature point matching based on the Euclidean distance, and selecting image pairs from the multiple target device images that have successfully matched feature points.

[0009] In one embodiment of the present invention, selecting image pairs that meet preset conditions from multiple target device images further includes: calculating the corresponding epipolar geometry and F matrix for each image pair, and optimizing the image pair based on the epipolar geometry and the F matrix using the RANSAC algorithm.

[0010] A 3D model construction system for an industrial metaverse includes: a first acquisition module, which uses an image acquisition unit to acquire images of a detected target device from multiple angles in a target scene to obtain multiple images of the target device; an image selection module, which selects image pairs that meet preset conditions from the multiple images of the target device; a second acquisition module, which uses SFM to sequentially perform BA processing on the image pairs to obtain geometric information of the target scene and parameter information of the image acquisition unit; and a model construction module, which uses a NeRF model to process the multiple images of the target device, the geometric information of the target scene, and the parameter information of the image acquisition unit to construct a 3D model corresponding to the target scene.

[0011] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the above-described method for constructing a 3D model of an industrial metaverse.

[0012] A non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for constructing a 3D model of an industrial metaverse.

[0013] The beneficial effects of this invention are:

[0014] This invention enables fully automated 3D modeling of the industrial metaverse without human intervention, thus greatly improving modeling efficiency and reducing costs. Attached Figure Description

[0015] Figure 1 This is a flowchart of a method for constructing a 3D model of an industrial metaverse according to an embodiment of the present invention;

[0016] Figure 2 This is a block diagram of the 3D model construction system for the industrial metaverse according to an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Figure 1 This is a flowchart of a method for constructing a 3D model of an industrial metaverse according to an embodiment of the present invention.

[0019] like Figure 1 As shown, the 3D model construction method for the industrial metaverse of this invention may include the following steps:

[0020] S1, an image acquisition unit is used to acquire images of the detected target device from multiple angles in the target scene to obtain multiple images of the target device.

[0021] The target scene can be a factory area, workshop, or other modeling scene. The image acquisition unit can be a SLAM mobile robot, which integrates multiple sensors such as LiDAR, depth camera, and fisheye camera. It can move autonomously / controlled in the modeling scene in a convenient and quick manner and collect various visual data. After detecting the target device, the SLAM mobile robot will move around the target device and take multiple images of the target device from different positions and angles as image input for subsequent automatic modeling.

[0022] S2, Select image pairs that meet preset conditions from multiple target device images.

[0023] In one embodiment of the present invention, selecting image pairs that meet preset conditions from multiple target device images includes: extracting focal length information from multiple target device images and extracting corresponding image features using a feature extraction algorithm; calculating the Euclidean distance between image features of every two target device images using a Kd-Tree model and performing feature point matching based on the Euclidean distance; and selecting image pairs from multiple target device images that have successfully matched feature points.

[0024] Specifically, firstly, the corresponding focal length information can be extracted from the target device image, and a feature extraction algorithm, such as SIFT (Scale Invariant Feature Transform), can be used to extract the corresponding image features. Then, the Kd-Tree (K-demension tree) model is used to calculate the Euclidean distance between the image features of the two target device images, and feature point matching is performed based on the Euclidean distance. If the Euclidean distance is within a preset range, the feature point matching is successful. The image with successful feature point matching is selected from multiple target device images.

[0025] In other words, as a possible implementation, if the Euclidean distance between the image features of two images is within a preset range, then the two images are an image pair that satisfies the preset conditions.

[0026] In one embodiment of the present invention, selecting image pairs that meet preset conditions from multiple target device images further includes: calculating the corresponding epipolar geometry and F matrix for each image pair, and optimizing the image pair based on the epipolar geometry and F matrix using the ransac algorithm.

[0027] Specifically, after selecting image pairs with successfully matched feature points, the corresponding epipolar geometry and F matrix can be calculated for each image pair, and the image pair can be optimized based on the epipolar geometry and F matrix using the ransac algorithm.

[0028] In other words, as another possible implementation, the image pairs obtained by optimizing image pairs whose Euclidean distance between image features is within a preset range are image pairs that meet the preset conditions.

[0029] S3 uses SFM to sequentially perform BA processing on the image pairs to obtain the geometric information of the target scene and the parameter information of the image acquisition unit.

[0030] Specifically, after selecting image pairs that meet preset conditions, SFM (Structure from Motion) is input. First, the two initially selected images, i.e., an image pair, undergo BA (Bundle Leadjust) processing. Then, new images are added cyclically for new BA processing, until no suitable images can be added, at which point the BA processing ends. This process yields the geometric information of the target scene, i.e., sparse 3D point cloud information, as well as the parameter information of the image acquisition unit. The parameter information may include camera intrinsic parameters, camera position and orientation during image capture, etc. In step S4, a NeRF model is used to process multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit to construct a 3D model of the industrial metaverse corresponding to the target scene.

[0031] S4 uses the NeRF model to process multiple images of the target equipment, the geometric information of the target scene, and the parameter information of the image acquisition unit to construct a 3D model of the industrial metaverse corresponding to the target scene.

[0032] In one embodiment of the present invention, a NeRF model is used to process multiple target device images, geometric information of the target scene, and parameter information of the image acquisition unit to construct a 3D model corresponding to the target scene. This includes: inputting multiple target device images, geometric information of the target scene, and parameter information of the image acquisition unit into a multilayer perceptron to obtain corresponding volume density and RGB color values; using volume rendering technology to synthesize the volume density and RGB color values ​​into a corresponding texture, and constructing a 3D model based on the texture.

[0033] Specifically, the NeRF (Neural Radiance Fields) model can represent a target scene as the volume density and RGB color values ​​of any point in space. The input to the NeRF model can be multiple images of the target device taken from different locations within the same target scene, the geometric information of the target scene, and the parameter information of the image acquisition unit. Specifically, the multiple images of the target device, the geometric information of the target scene, and the parameter information of the image acquisition unit can be input into the multilayer perceptron of the NeRF model to generate corresponding volume density and RGB color values. At different locations, volumetric rendering technology is used to synthesize corresponding textures from the volume density and RGB color values. Finally, a 3D model is constructed based on the textures and the corresponding mesh.

[0034] Therefore, this invention utilizes the NeRF model to simulate the operation of light in the real world through inverse rendering, enabling the reconstruction of a realistic 3D scene from several 2D images taken from different angles. This process maintains speed and compactness while capturing high-frequency, local details. Functions represented by multilayer perceptrons serve as neural graph primitives. This allows the neural network input to be mapped to encodings in a high-dimensional space, extracting high-approximation quality from a compact model.

[0035] It should be noted that the NeRF model can be trained in advance using a training set, the acquisition of which is similar to the acquisition of images from the target device. Specifically, the training set, along with the corresponding geometric and parameter information, can be input into the NeRF model for training. The new images output by the NeRF model are then compared with the training set, and the weights of the multilayer perceptron are adjusted based on the comparison results to optimize the NeRF model. Through continuous training iterations, the NeRF model becomes increasingly accurate in representing 3D objects and scenes.

[0036] Therefore, this invention utilizes technologies such as computer vision, computer graphics, and robot control to automate the process from visual data acquisition to visual data reconstruction to generate 3D models. This eliminates the need for manual intervention, effectively reducing human input, lowering labor costs, and reducing modeling time. Furthermore, it lowers the technical threshold for modeling, making 3D modeling technology more accessible and serving as a fundamental supporting technology to promote the implementation and development of the industrial metaverse.

[0037] In summary, the 3D model construction method for the industrial metaverse according to embodiments of the present invention employs an image acquisition unit to acquire images of the detected target device from multiple angles in the target scene, obtaining multiple images of the target device. Image pairs meeting preset conditions are selected from these multiple target device images, and SFM is used to sequentially perform BA processing on the image pairs to obtain the geometric information of the target scene and the parameter information of the image acquisition unit. Finally, a NeRF model is used to process the multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit to construct a 3D model of the industrial metaverse corresponding to the target scene. Therefore, fully automated 3D modeling of the industrial metaverse can be achieved without manual intervention, thus greatly improving modeling efficiency and reducing costs.

[0038] Corresponding to the above-described method for constructing a 3D model of an industrial metaverse, this invention also proposes a system for constructing a 3D model of an industrial metaverse.

[0039] like Figure 2 As shown, the 3D model building system for the industrial metaverse of this invention may include: a first acquisition module 100, an image selection module 200, a second acquisition module 300, and a model building module 400.

[0040] The first acquisition module 100 is used to acquire images of the detected target device from multiple angles in the target scene using an image acquisition unit, thereby acquiring multiple images of the target device; the image selection module 200 is used to select image pairs that meet preset conditions from the multiple images of the target device; the second acquisition module 300 is used to perform BA processing on the image pairs sequentially using SFM, thereby acquiring the geometric information of the target scene and the parameter information of the image acquisition unit; the model construction module 400 is used to process the multiple images of the target device, the geometric information of the target scene, and the parameter information of the image acquisition unit using a NeRF model, thereby constructing a 3D model corresponding to the target scene.

[0041] In one embodiment of the present invention, the model building module 400 is specifically used to: input multiple target device images, geometric information of the target scene and parameter information of the image acquisition unit into a multilayer perceptron to obtain corresponding volume density and RGB color values; use volume rendering technology to synthesize the volume density and RGB color values ​​into corresponding textures, and build a 3D model based on the textures.

[0042] In one embodiment of the present invention, the second acquisition module 200 is specifically used to: extract focal length information from multiple target device images and extract corresponding image features using a feature extraction algorithm; calculate the Euclidean distance between the image features of every two target device images using a Kd-Tree model, perform feature point matching based on the Euclidean distance, and select image pairs from multiple target device images that have successfully matched feature points.

[0043] In one embodiment of the present invention, the second acquisition module 200 is further configured to:

[0044] It should be noted that a more specific embodiment of the 3D model construction system for the industrial metaverse of the present invention can be found in the embodiment of the 3D model construction method for the industrial metaverse described above. To avoid redundancy, it will not be described in detail here.

[0045] The 3D model building system for the industrial metaverse according to an embodiment of the present invention comprises the following steps: A first acquisition module uses an image acquisition unit to acquire images of a detected target device from multiple angles within a target scene, obtaining multiple images of the target device. An image selection module selects image pairs that meet preset conditions from these multiple target device images. A second acquisition module uses SFM (Sensitive Visualization Model) to sequentially perform BA (Balanced Image Processing) on ​​the image pairs to obtain geometric information of the target scene and parameter information of the image acquisition unit. Finally, a model building module uses a NeRF (Neural Relationship to Rendering Model) to process the multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit to construct a 3D model corresponding to the target scene. This system enables fully automated 3D modeling of the industrial metaverse without manual intervention, thus significantly improving modeling efficiency and reducing costs.

[0046] Corresponding to the above embodiments, the present invention also proposes a computer device.

[0047] The computer device of this invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described method for constructing a 3D model of an industrial metaverse.

[0048] The computer device according to embodiments of the present invention can realize fully automatic industrial metaverse 3D modeling without human intervention, thus greatly improving modeling efficiency and reducing costs.

[0049] Corresponding to the above embodiments, the present invention also proposes a non-transitory computer-readable storage medium.

[0050] The non-transitory computer-readable storage medium of this invention stores a computer program that, when executed by a processor, implements the above-described method for constructing a 3D model of the industrial metaverse.

[0051] The non-transitory computer-readable storage medium according to embodiments of the present invention enables fully automated industrial metaverse 3D modeling without human intervention, thus greatly improving modeling efficiency and reducing costs.

[0052] In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0053] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0054] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0055] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0056] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0057] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0058] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0059] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for constructing a 3D model of an industrial metaverse, characterized in that, Includes the following steps: An image acquisition unit is used to acquire images of the detected target device from multiple angles in the target scene, so as to obtain multiple images of the target device. Selecting image pairs that meet preset conditions from multiple target device images; wherein, selecting image pairs that meet preset conditions from multiple target device images includes: extracting focal length information from multiple target device images, and extracting corresponding image features using a feature extraction algorithm; calculating the Euclidean distance between image features of every two target device images using a Kd-Tree model, and performing feature point matching based on the Euclidean distance; and selecting image pairs from the multiple target device images where feature points are successfully matched. SFM is used to sequentially perform BA processing on the image pairs to obtain the geometric information of the target scene and the parameter information of the image acquisition unit; The NeRF model is used to process the multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit to construct the 3D model of the industrial metaverse corresponding to the target scene. The process includes: inputting the multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit into a multilayer perceptron to obtain corresponding volume density and RGB color values; using volume rendering technology to synthesize the volume density and RGB color values ​​into a corresponding texture; and constructing the 3D model based on the texture.

2. The method for constructing a 3D model of an industrial metaverse according to claim 1, characterized in that, Selecting image pairs that meet preset conditions from multiple images of the target device further includes: For each image pair, the corresponding epipolar geometry and F matrix are calculated, and the image pair is optimized based on the epipolar geometry and the F matrix using the RANSAC algorithm.

3. A 3D model construction system for an industrial metaverse, characterized in that, include: The first acquisition module is used to acquire multiple images of the target device from multiple angles in the target scene using an image acquisition unit. An image selection module is used to select image pairs that meet preset conditions from multiple target device images; specifically, the image selection module is used to: extract focal length information from the multiple target device images, and extract corresponding image features using a feature extraction algorithm; calculate the Euclidean distance between the image features of every two target device images using a Kd-Tree model, and perform feature point matching based on the Euclidean distance; and select image pairs from the multiple target device images where the feature points are successfully matched. The second acquisition module is used to perform BA processing on the image pairs sequentially using SFM to obtain the geometric information of the target scene and the parameter information of the image acquisition unit; The model building module is used to process the multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit using a NeRF model to construct a 3D model corresponding to the target scene. Specifically, the model building module is used to: input the multiple target device images, the geometric information of the target scene, and the parameter information of the image acquisition unit into a multilayer perceptron to obtain corresponding volume density and RGB color values; use volume rendering technology to synthesize the volume density and the RGB color values ​​into a corresponding texture; and construct the 3D model based on the texture.

4. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the 3D model construction method of the industrial metaverse according to claim 1 or 2.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the 3D model construction method for the industrial metaverse as described in claim 1 or 2.

Images