A method and system for generating a high-precision three-dimensional model of a single image of an energy storage device

Abstract

The present invention discloses a method and system for generating a high-precision three-dimensional model of a single image of an energy storage device, and relates to the technical fields of computer vision and graphics. A two-dimensional orthogonal graph and a canonical coordinate graph are generated according to the two-dimensional image of the energy storage device, and are input into a convolutional reconstruction model based on an alternating gated cyclic perceptron for reconstruction; wherein, the three-dimensional image is used to extract the rough value of the SDF value, the rough value of the texture color, and the rough value of the Flexicubes parameter respectively through a multi-layer perceptron; using any two of the rough values, the other rough values ​​are iterated at least once to obtain a refined value; and a three-dimensional reconstructed texture grid of the energy storage device is generated. The present invention is based on a gated cycle method, and the gated cyclic perceptron module designed is used to decode the features of the three-dimensional image. The decoding process is iterated at least once to obtain refined data, optimize redundant information, improve the accuracy of three-dimensional modeling, and realize accurate three-dimensional reconstruction of the energy storage device.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and graphics, and more specifically to a method and system for generating a high-precision three-dimensional model of a single image of an energy storage device. Background Art

[0002] In order to create a 3D-aware method from a single 2D image, it is necessary to have prior knowledge of the properties of the 3D shape itself. This means that one must rely on a large amount of prior knowledge about various 3D shapes, or learn this knowledge from a large amount of data through machine learning algorithms. In 2D deep learning, the convolutional reconstruction model is a very effective method that can learn a compressed representation of the input image, that is, it is able to extract concise and useful features from complex image data. This method has achieved remarkable success in many 2D image processing tasks, such as image compression, denoising, and feature extraction.

[0003] Extending the architecture of convolutional reconstruction models to learn compact 3D shapes is one of the most promising ways to apply deep learning to 3D data. Through this extension, deep learning models can be used to automatically extract and learn the core features of 3D shapes, thereby better inferring and reconstructing the corresponding 3D structure from a single 2D image. However, the information learned by this method has a lot of redundancy, resulting in the accuracy of the generated model not being satisfactory.

[0004] The three-dimensional reconstruction of energy storage equipment is of vital importance for the monitoring, maintenance and management of power systems. With the development of technology, users can obtain three-dimensional objects through acquisition equipment, but the resulting models are usually of poor quality. In addition, when existing methods are applied to the three-dimensional reconstruction of energy storage equipment, the constructed models always have protruding parts at the edges. This is because the features obtained by the network of the existing methods contain redundant information. In the three-dimensional reconstruction process, the network needs to extract key features from the input data to build an accurate model. However, due to the lack of an effective screening mechanism, a large amount of redundant information is extracted and involved in the model construction. This redundant information may come from various factors; when this redundant information is involved in the model construction, it is easy to cause the model to have protruding parts at the edges and other parts, resulting in a large deviation between the reconstructed model and the actual energy storage device.

[0005] Therefore, how to provide a high-precision three-dimensional model generation method and system for a single image of an energy storage device, optimize redundant information, improve the accuracy of three-dimensional modeling, optimize the processing flow of three-dimensional data, and achieve more efficient three-dimensional reconstruction and analysis is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a method and system for generating a high-precision three-dimensional model of a single image of an energy storage device. Based on the gated loop method, a gated loop perceptron module is designed to decode the three-plane features. The decoding process is iterated at least once to form SDF values, texture colors and Flexicubes parameters from coarse values ​​to fine values. This module can well control the propagation of features in the network, and thus construct an accurate three-dimensional model of the energy storage device.

[0007] In order to achieve the above object, the present invention adopts the following technical solution: a method for generating a high-precision three-dimensional model of a single image of an energy storage device, comprising:

[0008] acquiring a two-dimensional image including the energy storage device;

[0009] Processing the two-dimensional image to generate a two-dimensional orthogonal graph and a canonical coordinate graph;

[0010] The two-dimensional orthogonal graph and the regular coordinate graph are input into a convolutional reconstruction model based on an alternating gated recurrent perceptron for reconstruction; wherein, a rough value of the SDF value, a rough value of the texture color, and a rough value of the Flexicubes parameter in the three-dimensional graph are respectively extracted through the alternating gated recurrent perceptron module; using any two of the rough values, the other rough values ​​are iterated at least once to obtain a fine value of the SDF value, a fine value of the texture color, and a fine value of the Flexicubes parameter;

[0011] Generate a 3D reconstructed textured mesh of the energy storage device.

[0012] Preferably, the convolutional reconstruction model based on the alternating gated recurrent perceptron comprises: a three-dimensional image generation module and an alternating gated recurrent perceptron module;

[0013] The three-dimensional graph generation module processes the two-dimensional orthogonal graph and the regular coordinate graph to obtain a three-dimensional graph, and inputs the three-dimensional graph into the alternating gated recurrent perceptron module for refined processing;

[0014] The obtained refined values ​​of the SDF value, the texture color and the Flexicubes parameter are used to perform secondary stepping to generate a three-dimensional reconstructed texture mesh.

[0015] Preferably, the rough value of the SDF value is corrected using the rough value of the texture color and the rough value of the Flexicubes parameter to obtain the fine value of the SDF value. The correction process is as follows:

[0016] ;

[0017] in, x 1tRepresents the data after stacking the rough value of texture color and the rough value of Flexicubes parameter; h 1(t-1) represents a rough value of the SDF value, W 1 , W 1z and W 1r are all trainable parameters, r 1t To reset the gate, z 1t To update the gate, Indicates the hidden state, is the refined value of SDF value; Represents the sigmoid function.

[0018] Preferably, the rough value of the texture color is corrected using the rough value of the SDF value and the rough value of the Flexicubes parameter to obtain the fine value of the texture color. The correction process is as follows:

[0019] ;

[0020] in, x 2t Represents the data after stacking the rough values ​​of SDF values ​​and the rough values ​​of Flexicubes parameters; h 2(t-1) A rough value representing the texture color, W 2 , W 2z and W 2r are all trainable parameters, r 2t To reset the gate, z 2t To update the gate, Indicates the hidden state, is the fine value of the texture color; Represents the sigmoid function.

[0021] Preferably, the rough value of the SDF value and the rough value of the texture color are used to correct the rough value of the Flexicubes parameter to obtain the fine value of the Flexicubes parameter. The correction process is as follows:

[0022] ;

[0023] in, x 3t Represents the data after stacking the rough value of the SDF value and the rough value of the texture color; h3(t-1) represents the rough value of the Flexicubes parameter, W 3 , W 3z and W 3r are all trainable parameters, r 3t To reset the gate, z 3t To update the gate, Indicates the hidden state, is the refined value of the Flexicubes parameter; Represents the sigmoid function.

[0024] Preferably, the three-dimensional graph is spatially aligned with the input two-dimensional orthogonal graph and the canonical coordinate graph, respectively.

[0025] Preferably, the method further comprises: performing texture rendering on the three-dimensional reconstructed texture grid, and applying MSE loss to the texture rendered image. and LPIPS loss Combination of:

[0026] ;

[0027] in, and Represent the 2D image, depth map and mask layer of the reconstructed texture mesh rendering respectively; and It is rendered from a ground truth texture mesh; is the mesh quality regularizer introduced in the Flexicubes parameters; and is the coefficient that balances each loss, Represents the MSE loss and LPIPS loss combination.

[0028] Preferably, the number of iterations is three.

[0029] Preferably, there are 6 two-dimensional orthogonal graphs and 6 canonical coordinate graphs, corresponding to 6 faces of the energy storage device;

[0030] Divide the six 2D orthogonal images and canonical coordinate images into two groups, each with three images;

[0031] Combine these images in each of the four groups into four larger images;

[0032] By concatenating these four sets of images, a 12-channel input is formed, which is processed by the three-dimensional image generation module to produce an output unfolded three-dimensional image.

[0033] Preferably, a high-precision three-dimensional model generation system for a single image of an energy storage device comprises:

[0034] An image acquisition module, used to acquire a two-dimensional image containing an energy storage device;

[0035] An image processing module, used for processing the two-dimensional image to generate a two-dimensional orthogonal graph and a canonical coordinate graph;

[0036] A three-dimensional reconstruction module, used for inputting the two-dimensional orthogonal image and the regular coordinate image into a convolutional reconstruction model based on an alternating gated recurrent perceptron for reconstruction; wherein, the rough value of the SDF value, the rough value of the texture color, and the rough value of the Flexicubes parameter in the three-dimensional image are respectively extracted by the alternating gated recurrent perceptron module; using any two of the rough values, the other rough values ​​are iterated at least once to obtain the fine value of the SDF value, the fine value of the texture color, and the fine value of the Flexicubes parameter respectively;

[0037] The three-dimensional mesh generation module is used to generate a three-dimensional reconstructed texture mesh of the energy storage device.

[0038] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a method and system for generating a high-precision three-dimensional model of a single image of an energy storage device, including: obtaining a two-dimensional image containing an energy storage device; processing the two-dimensional image to generate a two-dimensional orthogonal graph and a regular coordinate graph; inputting the two-dimensional orthogonal graph and the regular coordinate graph into a convolutional reconstruction model based on an alternating gated cyclic perceptron for reconstruction; generating a three-dimensional reconstructed texture grid of the energy storage device. The present invention is based on a gated cycle method, and the gated cyclic perceptron module designed decodes the three-plane features. The decoding process is iterated at least once to form SDF values, texture colors, and Flexicubes parameters from coarse values ​​to fine values. The module can optimize redundant information well, improve the accuracy of three-dimensional modeling, control the propagation of features in the network, and then construct an accurate three-dimensional model of the energy storage device, and realize accurate three-dimensional reconstruction of the energy storage device. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0040] Figure 1 A schematic flow chart of a method for three-dimensional reconstruction of an energy storage device provided in an embodiment of the present invention.

[0041] Figure 2 A schematic diagram of the structure of a three-dimensional reconstruction algorithm for energy storage equipment provided in an embodiment of the present invention.

[0042] Figure 3 A schematic diagram of the convolutional reconstruction model structure provided by an embodiment of the present invention.

[0043] Figure 4 A schematic diagram of the structure of an alternating gated cyclic sensor module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The embodiment of the present invention discloses a method and system for generating a high-precision three-dimensional model of a single image of an energy storage device, including: generating a two-dimensional orthogonal graph and a canonical coordinate graph according to the two-dimensional image of the energy storage device, and inputting them into a convolutional reconstruction model based on an alternating gated recurrent perceptron for reconstruction; wherein, the three-dimensional graph is respectively extracted by a multi-layer perceptron for the rough value of the SDF value, the rough value of the texture color, and the rough value of the Flexicubes parameter; using any two of the rough values, the other rough values ​​are iterated at least once to obtain a fine value; and a three-dimensional reconstructed texture grid of the energy storage device is generated. The alternating gated recurrent perceptron module designed in the embodiment of the present invention optimizes redundant information, which can not only improve the accuracy of three-dimensional modeling, but also significantly optimize the processing flow of three-dimensional data, thereby achieving more efficient three-dimensional reconstruction and analysis in various applications. By combining deep learning and three-dimensional data processing technology, an innovative method that can automatically generate high-quality three-dimensional models is developed based on limited two-dimensional information and rich three-dimensional shape prior knowledge.

[0046] In a specific embodiment of the present invention, a method for generating a high-precision three-dimensional model of a single image of an energy storage device includes:

[0047] acquiring a two-dimensional image including the energy storage device;

[0048] Processing the two-dimensional image to generate a two-dimensional orthogonal graph and a canonical coordinate graph;

[0049] The two-dimensional orthogonal graph and the regular coordinate graph are input into a convolutional reconstruction model based on an alternating gated recurrent perceptron for reconstruction; wherein, a rough value of the SDF value, a rough value of the texture color, and a rough value of the Flexicubes parameter in the three-dimensional graph are respectively extracted through the alternating gated recurrent perceptron module; using any two of the rough values, the other rough values ​​are iterated at least once to obtain a fine value of the SDF value, a fine value of the texture color, and a fine value of the Flexicubes parameter;

[0050] Generate a 3D reconstructed textured mesh of the energy storage device.

[0051] In a specific embodiment of the present invention, Figure 1 and Figure 2 As shown in the figure, a method for accurate 3D modeling of energy storage equipment based on deep learning algorithm is shown in the figure. The specific process is as follows:

[0052] Step 1: Input the captured RGB two-dimensional image of the energy storage device.

[0053] Step 2: Construct a multi-view diffusion model and input the two-dimensional image into the multi-view diffusion model to generate six two-dimensional orthogonal images at predetermined viewpoints from a single input image. A high-performance diffusion model is used here to generate multi-view images from a single image.

[0054] Specifically, the multi-view diffusion model is designed to generate six-directional projection images from a single input image. The embodiment of the present invention initializes the diffusion model based on the ImageDream method. The method proposes a high-performance diffusion model for generating multi-view images from a single image. ImageDream supports the generation of 4-view images, and the embodiment of the present invention removes the last layer of the ImageDream model and changes the generated 4 view angles into 6 view angles.

[0055] Step 3: Construct a canonical coordinate map diffusion model and input the predicted six 2D orthogonal map images into the canonical coordinate map diffusion model to obtain six output canonical coordinate maps containing additional geometric information. This is different from previous work, which usually uses pure RGB images as input to predict 3D objects. It is difficult to predict the correct geometry using pure RGB images, and sometimes the geometry is degraded. Formally, the canonical coordinate map is the coordinate of each point in the canonical space. It contains 3 channels with values ​​in the range [0,1], representing the coordinates in the canonical space.

[0056] Specifically, the design of the canonical coordinate map diffusion model is to generate a six-directional canonical coordinate map image from a six-directional two-dimensional orthogonal map image. The embodiment of the present invention still generates six outputs based on the ImageDream method to initialize the diffusion model. The difference from the generation of the six-directional projection image lies in the forward propagation process of adding noise at the input. What is added is no longer just noise, but six perspective maps generated by the six-directional projection image and noise are stacked and then sent to the ImageDream model. At the same time, the model is fine-tuned using the Objaverse data set. After fine-tuning, six canonical coordinate maps are obtained.

[0057] Diffusion models generate new data instances by iteratively converting random noise into data samples. The core of this type of model lies in its diffusion process, which can be viewed as a series of time steps in which noise is added to the data. As time steps pass, the signal is gradually overwhelmed by noise, generating a set of noisy data. In the generation phase, these models operate in reverse, starting with random noise and gradually removing the added noise. This process is iterated repeatedly until the original data state is reached without any noise at all, and finally a new data instance is obtained. Key parameters include noise level, number of time steps, and network structure. The noise level determines the amount of noise added to the data at each time step; the number of time steps defines the length of the conversion process from pure noise to data instances; and the network structure determines the model's ability to learn conditional probability distributions.

[0058] Step 4: Compare the six two-dimensional orthogonal graphs generated in step 2 with the six regular coordinates generated in step 3. Figure 1 The resulting point cloud is sent to a convolutional reconstruction model based on an alternating gated recurrent perceptron to reconstruct the final point cloud texture mesh.

[0059] Specifically, the convolutional reconstruction model based on the alternating gated recurrent perceptron includes: a three-dimensional image generation module and an alternating gated recurrent perceptron module;

[0060] The three-dimensional graph generation module processes the two-dimensional orthogonal graph and the regular coordinate graph to obtain an unfolded three-dimensional graph, and reshapes the unfolded three-dimensional graph into a three-dimensional three-dimensional graph; in order to avoid unnecessary entanglement between different planes, the unfolded three planes are used.

[0061] Inputting the three-dimensional three-dimensional image into the alternating gated recurrent perceptron module for refined processing;

[0062] The obtained refined values ​​of the SDF value, the texture color and the Flexicubes parameter are used to perform secondary stepping to generate a three-dimensional reconstructed texture mesh.

[0063] Specifically, a rough value of the SDF value, a rough value of the texture color, and a rough value of the Flexicubes parameter are obtained respectively through the SDF branch, the texture color branch, and the Flexicubes parameter branch, and the rough value of the SDF value, the rough value of the texture color, and the rough value of the Flexicubes parameter are refined;

[0064] In a specific embodiment of the present invention, for a convolutional reconstruction model based on an alternating gated recurrent perceptron, such as Figure 3 As shown, according to the input 6 two-dimensional orthogonal images and regular coordinate images, the convolution U-Net is used to convert the input two-dimensional orthogonal image into regular coordinates. Figure 1 The three planes are mapped to the unfolded three planes. The unfolded three planes are then reshaped into a three-dimensional three-plane image. The three-plane features are decoded into SDF values, texture colors, and Flexicubes parameters using the innovative alternating gated recurrent perceptron module of the embodiment of the present invention. Finally, these values ​​are used for secondary stepping to obtain a texture mesh.

[0065] The embodiment of the present invention selects three planes as the three-dimensional representation because it can obtain high-resolution three-dimensional results with two-dimensional computational effort. Each query grid cell is projected onto an axis-aligned orthogonal plane, and then features are aggregated from each plane. In addition, in order to avoid unnecessary entanglement between different planes, an unfolded three plane is used. The three planes are spatially aligned with the six input two-dimensional orthogonal images and the regular coordinate map. In order to match the unfolded three planes, the six two-dimensional orthogonal images and the regular coordinate map are arranged in a similar manner. The embodiment of the present invention renders six images and regular coordinate maps at a resolution of 256×256. They are divided into two groups, each with three images. These images in the four groups are then combined into four larger images, each with a resolution of 256×768, allowing spatial alignment. By connecting these four groups, a 12-channel input is formed, which is processed by the convolutional U-Net to produce the output three planes.

[0066] In a specific embodiment of the present invention, the three-dimensional image is input into the designed alternating gated recurrent perceptron module to estimate the SDF value, texture color and Flexicubes parameter. The alternating gated recurrent perceptron module is as follows: Figure 4As shown. First, a rough SDF value, texture color and Flexicubes parameter are obtained through a multi-layer perceptron, which is a three-layer neural network; then the rough SDF value, texture color and Flexicubes parameter are converted to a refined SDF value, texture color and Flexicubes parameter through a gated recurrent module (GRU module). GRU (Gated Recurrent Unit) is a type of recurrent neural network (RNN) that can solve problems such as the inability to remember long-term memory in RNN and gradients in back propagation. It has a similar function to LSTM, but is simpler than LSTM and easier to train. The formula used by GRU:

[0067] ;

[0068] In the embodiment of the present invention, the alternating gated recurrent perceptron module contains three branches, including an SDF branch, a texture color branch, and a Flexicubes parameter branch, all of which contain a GRU module. The above three branches are iterated multiple times to obtain the final refined SDF value, texture color, and Flexicubes parameter. Finally, in order to train the model, the texture rendering image is subjected to the MSE loss. and LPIPS loss combination.

[0069] Specifically, the number of iterations is three.

[0070] SDF branch: stack texture colors and Flexicubes parameters, and use the stacked data as x 1t , through trainable parameters W 1z and W 1r Calculate the reset gate separately r 1t and update gate z 1t , using the calculated r 1t and x 1t Calculate hidden state ,use z 1t , h 1(t-1) , Calculates and updates the memory expression .

[0071] Specifically, in the SDF branch, the rough value of the texture color and the rough value of the Flexicubes parameter are used to correct the rough value of the SDF value to obtain the fine value of the SDF value. The correction process is as follows:

[0072] ;

[0073] in, x 1t Represents the data after stacking the rough value of texture color and the rough value of Flexicubes parameter; h 1(t-1) represents a rough value of the SDF value, W 1 , W 1z and W 1r are all trainable parameters, r 1t To reset the gate, z 1t To update the gate, Indicates the hidden state, is the refined value of SDF value; Represents the sigmoid function.

[0074] Texture color branch: stack SDF values ​​and Flexicubes parameters, and use the stacked data as x 2t , through trainable parameters W 2z and W 2r Calculate the reset gate separately r 2t and update gate z 2t , using the calculated r 2t and x 2t Calculate hidden state ,use z 2t , h 2(t-1) , Calculates and updates the memory expression .

[0075] Specifically, in the texture color branch, the rough value of the SDF value and the rough value of the Flexicubes parameter are used to correct the rough value of the texture color to obtain the fine value of the texture color. The correction process is as follows:

[0076] ;

[0077] in, x 2t Represents the data after stacking the rough values ​​of SDF values ​​and the rough values ​​of Flexicubes parameters; h 2(t-1) A rough value representing the texture color, W 2 , W 2z and W 2r are all trainable parameters, r 2t To reset the gate, z 2t To update the gate, Indicates the hidden state, is the fine value of the texture color; Represents the sigmoid function.

[0078] Flexicubes parameter branch: stack texture color and SDF value, and use the stacked data as x 3t , through trainable parameters W 3z and W 3r Calculate the reset gate separately r 3t and update gate z 3t , using the calculated r 3t and x 3t Calculate hidden state ,use z 3t , h 3(t-1) , Calculates and updates the memory expression .

[0079] Specifically, the rough value of the SDF value and the rough value of the texture color are used to correct the rough value of the Flexicubes parameter to obtain the fine value of the Flexicubes parameter. The correction process is as follows:

[0080] ;

[0081] in, x 3t Represents the data after stacking the rough value of the SDF value and the rough value of the texture color; h 3(t-1) represents the rough value of the Flexicubes parameter, W 3, W 3z and W 3r are all trainable parameters, r 3t To reset the gate, z 3t To update the gate, Indicates the hidden state, is the refined value of the Flexicubes parameter; Represents the sigmoid function.

[0082] Previous general 3D generation methods mostly use NeRF as a geometric representation, relying on marching cubes to extract isosurfaces, which has topological ambiguity and is difficult to represent high-fidelity geometric details. In an embodiment of the present invention, Flexicubes is used as a geometric representation, which can obtain a mesh from features on the mesh through double-step cubes during training, and the features include SDF values, deformations, and weights. Texture is obtained by querying the color of the surface, and Flexicubes can train the reconstruction model of the embodiment of the present invention in an end-to-end manner with a textured mesh as the final output.

[0083] Specifically, the three-dimensional graph is spatially aligned with the input two-dimensional orthogonal graph and the canonical coordinate graph, respectively.

[0084] Specifically, it also includes: texture rendering of the 3D reconstructed texture mesh, and using MSE loss on the texture rendered image and LPIPS loss Combination of:

[0085] ;

[0086] in, and Represent the 2D image, depth map and mask layer of the reconstructed texture mesh rendering respectively; and It is rendered from a ground truth texture mesh; is the mesh quality regularizer introduced in the Flexicubes parameters; and is the coefficient that balances each loss, Represents the MSE loss and LPIPS loss combination.

[0087] Specifically, there are 6 two-dimensional orthogonal graphs and 6 regular coordinate graphs, corresponding to 6 faces of the energy storage device;

[0088] Divide the six 2D orthogonal images and canonical coordinate images into two groups, each with three images;

[0089] These images in four groups were combined into four larger images; each image had a resolution of 256 × 768, allowing for spatial alignment.

[0090] By concatenating these four sets of images, a 12-channel input is formed, which is processed by the three-dimensional image generation module to produce an output unfolded three-dimensional image.

[0091] The idea and algorithm implemented by the embodiment of the present invention are as follows: Figure 2 The overall structure of the method of the embodiment of the present invention is shown. The input image is sent to the multi-view algorithm model to generate six orthogonal images. Then, the canonical coordinate image diffusion model is used to generate a canonical coordinate image conditioned on the six images. These six images are related to the canonical coordinate image. Figure 1 The points are sent to the convolutional reconstruction model to reconstruct the final texture mesh. On a 3090TI GPU, the entire inference process takes about 20 seconds. The point cloud results generated by different single energy storage devices have good adaptability.

[0092] In a specific embodiment of the present invention, a high-precision three-dimensional model generation system for a single image of an energy storage device includes:

[0093] An image acquisition module, used to acquire a two-dimensional image containing an energy storage device;

[0094] An image processing module, used for processing the two-dimensional image to generate a two-dimensional orthogonal graph and a canonical coordinate graph;

[0095] A three-dimensional reconstruction module, used for inputting the two-dimensional orthogonal image and the regular coordinate image into a convolutional reconstruction model based on an alternating gated recurrent perceptron for reconstruction; wherein, the rough value of the SDF value, the rough value of the texture color, and the rough value of the Flexicubes parameter in the three-dimensional image are respectively extracted by the alternating gated recurrent perceptron module; using any two of the rough values, the other rough values ​​are iterated at least once to obtain the fine value of the SDF value, the fine value of the texture color, and the fine value of the Flexicubes parameter respectively;

[0096] The three-dimensional mesh generation module is used to generate a three-dimensional reconstructed texture mesh of the energy storage device.

[0097] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0098] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a high-precision three-dimensional model of a single image of an energy storage device, characterized in that: include: acquiring a two-dimensional image including the energy storage device; Processing the two-dimensional image to generate a two-dimensional orthogonal graph and a canonical coordinate graph; The two-dimensional orthogonal graph and the regular coordinate graph are input into a convolutional reconstruction model based on an alternating gated cyclic perceptron for reconstruction; the convolutional reconstruction model based on an alternating gated cyclic perceptron comprises: a three-dimensional graph generation module and an alternating gated cyclic perceptron module; the three-dimensional graph generation module processes the two-dimensional orthogonal graph and the regular coordinate graph to obtain a three-dimensional graph, and the three-dimensional graph is input into the alternating gated cyclic perceptron module for refinement processing; wherein, the coarse value of the SDF value, the coarse value of the texture color and the coarse value of the Flexicubes parameter in the three-dimensional graph are respectively extracted by the alternating gated cyclic perceptron module; using any two of the coarse values, the other coarse values ​​are iterated at least once to obtain the fine value of the SDF value, the fine value of the texture color and the fine value of the Flexicubes parameter respectively; Generate a 3D reconstructed textured mesh of the energy storage device.

2. The method for generating a high-precision three-dimensional model of a single image of an energy storage device according to claim 1, characterized in that: The obtained refined values ​​of the SDF value, the texture color and the Flexicubes parameter are used to perform secondary stepping to generate a three-dimensional reconstructed texture mesh.

3. The method for generating a high-precision three-dimensional model of a single image of an energy storage device according to claim 1, characterized in that: Using the rough value of the texture color and the rough value of the Flexicubes parameter, the rough value of the SDF value is corrected to obtain the fine value of the SDF value. The correction process is as follows: Among them, x 1t Represents the data after stacking the rough value of the texture color and the rough value of the Flexicubes parameter; h 1(t-1) Indicates the rough value of SDF value, W1, W 1z and W 1r are all trainable parameters, r 1t is the reset gate, z 1t To update the gate, represents the hidden state, h 1t is the refined value of the SDF value; σ represents the sigmoid function.

4. The method for generating a high-precision three-dimensional model of a single image of an energy storage device according to claim 1, characterized in that: The rough value of the texture color is corrected using the rough value of the SDF value and the rough value of the Flexicubes parameter to obtain the fine value of the texture color. The correction process is as follows: Among them, x 2t Represents the data after stacking the rough values ​​of SDF values ​​and the rough values ​​of Flexicubes parameters; h 2(t-1) Represents the rough value of texture color, W2, W 2z and W 2r are all trainable parameters, r 2t is the reset gate, z 2t To update the gate, represents the hidden state, h 2t is the fine value of the texture color; σ represents the sigmoid function.

5. The method for generating a high-precision three-dimensional model of a single image of an energy storage device according to claim 1, characterized in that: The rough value of the SDF value and the rough value of the texture color are used to correct the rough value of the Flexicubes parameter to obtain the fine value of the Flexicubes parameter. The correction process is as follows: Among them, x 3t Represents the data after stacking the rough value of the SDF value and the rough value of the texture color; h 3(t-1) Represents the rough value of the Flexicubes parameter, W3, W 3z and W 3r are all trainable parameters, r 3t is the reset gate, z 3t To update the gate, represents the hidden state, h 3t is the fine value of the Flexicubes parameter; σ represents the sigmoid function.

6. The method for generating a high-precision three-dimensional model of a single image of an energy storage device according to claim 2, characterized in that: The three-dimensional graph is spatially aligned with the input two-dimensional orthogonal graph and canonical coordinate graph respectively.

7. The method for generating a high-precision three-dimensional model of a single image of an energy storage device according to claim 1, characterized in that: Also includes: Texture rendering is performed on the 3D reconstructed texture mesh, and MSE loss is used on the texture rendered image. and LPIPS loss Combination of: Among them, x, x depth and x mask Represents the 2D image, depth map and mask layer of the reconstructed texture mesh rendering; x GT , and It is rendered from a ground truth texture mesh; is the mesh quality regularizer introduced in the Flexicubes parameters; LPIPS ,λ depth ,λ mask and λ reg is the coefficient that balances each loss, Represents the MSE loss and LPIPS loss combination.

8. The method for generating a high-precision three-dimensional model of a single image of an energy storage device according to claim 1, characterized in that: The number of iterations is three.

9. The method for generating a high-precision three-dimensional model of a single image of an energy storage device according to claim 2, characterized in that: There are six two-dimensional orthogonal graphs and six canonical coordinate graphs, corresponding to six faces of the energy storage device; Divide the six 2D orthogonal images and canonical coordinate images into two groups, each with three images; Combine these images in each of the four groups into four larger images; By concatenating these four sets of images, a 12-channel input is formed, which is processed by the three-dimensional image generation module to produce an output unfolded three-dimensional image.

10. A high-precision three-dimensional model generation system for a single image of an energy storage device, characterized in that: A method for generating a high-precision three-dimensional model of a single image of an energy storage device according to any one of claims 1 to 9, comprising: An image acquisition module, used to acquire a two-dimensional image containing an energy storage device; An image processing module, used for processing the two-dimensional image to generate a two-dimensional orthogonal graph and a canonical coordinate graph; A three-dimensional reconstruction module, used for inputting the two-dimensional orthogonal image and the regular coordinate image into a convolutional reconstruction model based on an alternating gated cyclic perceptron for reconstruction; the convolutional reconstruction model based on an alternating gated cyclic perceptron includes: a three-dimensional image generation module and an alternating gated cyclic perceptron module; the three-dimensional image generation module processes the two-dimensional orthogonal image and the regular coordinate image to obtain a three-dimensional image, and inputs the three-dimensional image into the alternating gated cyclic perceptron module for fine processing; wherein, the rough value of the SDF value, the rough value of the texture color, and the rough value of the Flexicubes parameter in the three-dimensional image are respectively extracted by the alternating gated cyclic perceptron module; using any two of the rough values, the other rough values ​​are iterated at least once to obtain the fine value of the SDF value, the fine value of the texture color, and the fine value of the Flexicubes parameter respectively; The three-dimensional mesh generation module is used to generate a three-dimensional reconstructed texture mesh of the energy storage device.

Images