Three-dimensional reconstruction model generation method and device, computer device, and storage medium

Abstract

This application relates to a method, apparatus, computer device, and storage medium for generating a 3D reconstruction model. The method includes: acquiring a target image set and constructing superpixel maps corresponding to each image in the target image set; calculating features corresponding to each superpixel point in each superpixel map, fusing the features corresponding to each superpixel map to obtain target features corresponding to each superpixel map; performing superpixel point connection operations on each superpixel map to obtain triangular meshes corresponding to each superpixel map; acquiring camera position data corresponding to each triangular mesh; and training an initial network model based on the target features, each triangular mesh, and each camera position data to obtain a target 3D reconstruction model, wherein the target 3D reconstruction model is used to construct a 3D mesh map corresponding to the image set to be processed. This method can improve the accuracy of inferring the icing thickness and distribution of power transmission and transformation equipment.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, and storage medium for generating three-dimensional reconstruction models. Background Technology

[0002] With the rapid development of computer technology, it has also been applied to substation inspection data processing. Due to the wide coverage of the power grid and the varied geographical environments of the areas through which power lines pass, they are easily affected by the natural environment. In some high-altitude areas, icing of power transmission and transformation equipment is more severe, requiring analysis of the icing situation. Therefore, related technologies for analyzing the thickness and distribution of icing on conductors in power transmission and transformation equipment have emerged.

[0003] However, traditional technologies have poor real-time performance, low feasibility for directly measuring operating conductors or insulators, and are relatively dangerous for measurement personnel. Even when using drones to collect relevant signals, it is difficult to avoid situations where complex scenes can obstruct the view, which reduces the accuracy of inferring the thickness and distribution of ice on power transmission and transformation equipment. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, and storage medium for generating three-dimensional models that can infer the thickness and distribution of ice on power transmission and transformation equipment, thereby improving the accuracy of inferring the thickness and distribution of ice on power transmission and transformation equipment, in order to address the above-mentioned technical problems.

[0005] A method for generating a three-dimensional reconstruction model, the method comprising:

[0006] Obtain the target image set and construct the superpixel map corresponding to each image in the target image set;

[0007] Calculate the features corresponding to each superpixel in each superpixel image, and fuse the features corresponding to each superpixel to obtain the target features corresponding to each superpixel image;

[0008] Perform superpixel point connection operations on each superpixel image to obtain the triangular mesh corresponding to each superpixel image;

[0009] The camera position data corresponding to each triangular mesh is obtained. Based on each target feature, each triangular mesh and each camera position data, the initial network model is trained to obtain a target 3D reconstruction model. The target 3D reconstruction model is used to construct a 3D mesh map corresponding to the image set to be processed.

[0010] In one embodiment, obtaining a target image set and constructing a superpixel map corresponding to each image in the target image set includes:

[0011] Obtain the original image set, select a first number of images from the original image set as the first image set, and label each image in the first image set to obtain the first target image set;

[0012] Obtain the target semi-supervised model, take the unfiltered images in the original image set as the second image set, label each image in the second image set based on the target semi-supervised model to obtain the second target image set, and take the combination of the first target image set and the second target image set as the target image set;

[0013] Obtain the target superpixel count, and divide the target superpixel count into superpixel points in each image of the target image set;

[0014] Obtain a first target range, calculate the gradient value of each pixel within the first target range corresponding to each superpixel, and based on the comparison result of the gradient values ​​of each pixel corresponding to each superpixel, select the target pixel corresponding to each superpixel from each pixel, and move the position of each superpixel to the position of the corresponding target pixel to obtain the target superpixel.

[0015] Obtain the second target range, calculate the target distance between each target superpixel and each pixel within the corresponding second target range, and based on the comparison results of each target distance, classify each pixel into the corresponding target superpixel category to obtain the superpixel image corresponding to each image in the target image set.

[0016] In one embodiment, the features corresponding to each superpixel in each superpixel image are calculated, and the features corresponding to each superpixel image are fused to obtain the target features corresponding to each superpixel image, including:

[0017] Obtain the features of each pixel in each superpixel image;

[0018] The features of pixels of the same target superpixel category corresponding to each pixel are fused to obtain the features corresponding to each superpixel in each superpixel image.

[0019] The features corresponding to each superpixel point in each superpixel image are fused to obtain the target features of each superpixel image.

[0020] In one embodiment, an initial network model is trained based on the respective target features, the respective triangular meshes, and the respective camera position data to obtain a target 3D reconstruction model, including:

[0021] Based on the time frame corresponding to each of the triangular grids, the triangular grid corresponding to the current time frame is selected from each of the triangular grids as the current triangular grid;

[0022] Based on the encoding layer of the initial network model, and based on the target features corresponding to the current triangular mesh, the target curvature corresponding to each vertex in the current triangular mesh is calculated;

[0023] Obtain the target dependency matrix and target handle position, and based on the target curvature corresponding to each vertex, the target dependency matrix, and the target handle position, obtain the current deformed mesh corresponding to the current triangle mesh;

[0024] Obtain the target loss function, calculate the current loss value corresponding to the current deformed network based on the target loss function, and take the current deformed mesh as the target deformed mesh corresponding to the current triangular mesh based on the comparison result between the current loss value and the target condition;

[0025] Take the next time frame corresponding to the current time frame as the current time frame, and repeat the operation of selecting the triangle mesh corresponding to the current time frame from each triangle mesh as the current triangle mesh until each triangle mesh has a corresponding target deformable mesh.

[0026] Based on the decoding layer of the initial network model, the target deformable meshes corresponding to each triangular mesh are fused to generate a target 3D mesh map, thus obtaining the target 3D reconstruction model.

[0027] In one embodiment, before obtaining the target loss function, the method further includes:

[0028] Obtain the motion reprojection loss function, the keypoint reprojection loss function, the texture loss function, the local rigidity loss function, and the region similarity loss function.

[0029] The target loss function is obtained by combining the motion reprojection loss function, keypoint reprojection loss function, texture loss function, local rigidity loss function, and region similarity loss function.

[0030] In one embodiment, after training an initial network model based on the respective target features, the respective triangular meshes, and the respective camera position data to obtain a target 3D reconstruction model, the method further includes:

[0031] Obtain the target image set to be processed, and construct the target superpixel map corresponding to each image in the target image set to be processed;

[0032] Calculate the target features corresponding to each target superpixel image;

[0033] Connect the superpixel points of the target superpixel images corresponding to each image to obtain the target triangle mesh corresponding to each target superpixel image;

[0034] Obtain the target camera position data corresponding to each target triangle mesh, obtain the target 3D reconstruction model, input each target feature, each target triangle mesh and each target camera position data into the target 3D reconstruction model, and obtain the target 3D mesh map corresponding to the target image set to be processed.

[0035] A three-dimensional reconstruction model generation device, the device comprising:

[0036] The superpixel image construction module is used to acquire a target image set and construct a superpixel image corresponding to each image in the target image set;

[0037] The feature acquisition module is used to calculate the features corresponding to each superpixel point in each superpixel image, and fuse the features corresponding to each superpixel image to obtain the target features corresponding to each superpixel image.

[0038] The mesh construction module is used to perform superpixel point connection operations on each superpixel image to obtain the triangular mesh corresponding to each superpixel image;

[0039] The target model determination module is used to obtain camera position data corresponding to each triangular grid, and to train an initial network model based on each target feature, each triangular grid and each camera position data to obtain a target 3D reconstruction model. The target 3D reconstruction model is used to construct a 3D mesh map corresponding to the image set to be processed.

[0040] In one embodiment, the three-dimensional reconstruction model generation apparatus further includes:

[0041] The loss design module is used to obtain the motion reprojection loss function, the keypoint reprojection loss function, the texture loss function, the local rigidity loss function, and the region similarity loss function; and to combine the motion reprojection loss function, the keypoint reprojection loss function, the texture loss function, the local rigidity loss function, and the region similarity loss function to obtain the target loss function.

[0042] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0043] Obtain the target image set and construct the superpixel map corresponding to each image in the target image set;

[0044] Calculate the features corresponding to each superpixel point in each superpixel image, and fuse the features corresponding to each superpixel image to obtain the target features corresponding to each superpixel image;

[0045] Perform superpixel point connection operations on each superpixel image to obtain the triangular mesh corresponding to each superpixel image;

[0046] The camera position data corresponding to each triangular mesh is obtained. Based on each target feature, each triangular mesh and each camera position data, the initial network model is trained to obtain a target 3D reconstruction model. The target 3D reconstruction model is used to construct a 3D mesh map corresponding to the image set to be processed.

[0047] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0048] Obtain a target image set and construct a superpixel map corresponding to each image in the target image set;

[0049] Calculate the features corresponding to each superpixel point in each superpixel image, and fuse the features corresponding to each superpixel image to obtain the target features corresponding to each superpixel image;

[0050] Perform superpixel point connection operations on each superpixel image to obtain the triangular mesh corresponding to each superpixel image;

[0051] The camera position data corresponding to each triangular mesh is obtained. Based on each target feature, each triangular mesh and each camera position data, the initial network model is trained to obtain a target 3D reconstruction model. The target 3D reconstruction model is used to construct a 3D mesh map corresponding to the image set to be processed.

[0052] The aforementioned method, apparatus, computer equipment, and storage medium for generating 3D reconstruction models involve: acquiring a target image set; constructing superpixel maps corresponding to each image in the target image set; calculating features corresponding to each superpixel point in each superpixel map; fusing the features corresponding to each superpixel map to obtain target features corresponding to each superpixel map; performing superpixel point connection operations on each superpixel map to obtain triangular meshes corresponding to each superpixel map; acquiring camera position data corresponding to each triangular mesh; and training an initial network model based on the target features, the triangular meshes, and the camera position data to obtain a target 3D reconstruction model. This target 3D reconstruction model is used to construct 3D mesh maps corresponding to the image set to be processed. By constructing superpixel maps corresponding to each image in the target image set, calculating the target features corresponding to each superpixel map, performing superpixel point connection operations on each superpixel map to obtain triangular meshes corresponding to each superpixel map, acquiring camera position data corresponding to each triangular mesh, and training an initial network model based on each target feature, each triangular mesh, and each camera position data to obtain a target 3D reconstruction model, the real-time performance of acquiring the target image set and the feasibility of inferring the icing situation of power transmission and transformation equipment based on the technology of this application are improved. Constructing a 3D mesh map based on a 2D target image set improves the realism of the 3D reconstruction of the surrounding environment corresponding to the 2D target image set, thereby improving the accuracy of inferring the icing thickness and distribution of power transmission and transformation equipment. Attached Figure Description

[0053] Figure 1 This is a diagram illustrating the application environment of a 3D reconstruction model generation method in one embodiment.

[0054] Figure 2 This is a flowchart illustrating a method for generating a 3D reconstruction model in one embodiment;

[0055] Figure 3 This is a schematic diagram of the superpixel map construction process in one embodiment;

[0056] Figure 4 This is a schematic diagram of the target feature acquisition process in one embodiment;

[0057] Figure 5 This is a schematic diagram of the process for determining the target model in one embodiment;

[0058] Figure 6 This is a flowchart illustrating the design process of the loss function in one embodiment;

[0059] Figure 7 This is a flowchart illustrating the process used by the model in one embodiment;

[0060] Figure 8 This is a schematic diagram of the training process for a 3D reconstruction model in one embodiment;

[0061] Figure 9 A structural block diagram of a three-dimensional reconstruction model generation device in one embodiment;

[0062] Figure 10 This is an internal structural diagram of a computer device in one embodiment;

[0063] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] The three-dimensional reconstruction model generation method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. The terminal is used to display a target image set and its corresponding 3D mesh map. Server 104 is used to acquire the target image set, construct superpixel maps corresponding to each image in the target image set, calculate the target features and triangular meshes corresponding to each superpixel map, acquire camera position data corresponding to each triangular mesh, train an initial network model based on the target features, the triangular meshes, and the camera position data to obtain a target 3D reconstruction model, and apply the target 3D reconstruction model to construct the 3D mesh map corresponding to the image set to be processed. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0066] In one embodiment, such as Figure 2 As shown, a method for generating a 3D reconstruction model is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0067] Step S200: Obtain the target image set and construct the superpixel map corresponding to each image in the target image set.

[0068] The target image set refers to the collection of images that have been labeled. A superpixel image refers to a pixel-level image, a region-level image, which is an abstraction of basic information elements. Simply put, it is an image composed of multiple superpixels. A superpixel is a small region composed of a series of adjacent pixels with similar characteristics such as color, brightness, and texture. It refers to an irregular block of pixels with certain meaning composed of adjacent similar pixels with similar texture, color, height, and other characteristics.

[0069] Specifically, in this embodiment, image data from different regions is collected through an inspection system. A small portion of the collected image data is labeled, while the remaining image data is labeled using semi-supervised AI model training technology. Alternatively, the remaining image data can be manually labeled to obtain a target image set for supervised training of the subsequent model. Superpixel segmentation is performed on each image in the target image set to construct a corresponding superpixel map. The superpixel segmentation method can employ SLIC (simple linear iterative clustering), which results in compact and orderly superpixels that effectively represent pixel features. This method also boasts good compatibility, capable of segmenting both color and grayscale images. It requires fewer parameter settings, accelerating the process and improving efficiency, thereby enhancing the realism and accuracy of the constructed 3D mesh map.

[0070] Step S202: Calculate the features corresponding to each superpixel point in each superpixel image, and fuse the features corresponding to each superpixel image to obtain the target features corresponding to each superpixel image.

[0071] In this context, a superpixel refers to a point that can represent the features corresponding to each pixel in a superpixel. Fusion refers to the operation of merging the features corresponding to each superpixel in various superpixel images. The target feature refers to the set of features corresponding to all superpixels in each superpixel image; these features can express the characteristics of each superpixel.

[0072] Specifically, each superpixel can represent the features of a pixel block with similar texture, color, height, and other characteristics. Within this pixel block, there are multiple pixels, each with its own features. The features corresponding to each superpixel are calculated using the features of the pixels corresponding to the same superpixel. That is, the features of each superpixel are obtained by averaging the features of each pixel corresponding to the same superpixel. Then, the features of all superpixels in each superpixel image are merged to obtain the target features corresponding to each superpixel image. The pixel mentioned here is a unit used to calculate digital images; it is the smallest unit in an image represented by a numerical sequence.

[0073] Step S204: Perform superpixel point connection operations on each superpixel image to obtain the triangular mesh corresponding to each superpixel image.

[0074] The superpixel connection operation refers to connecting individual superpixels to obtain a triangular mesh. A triangular mesh is a polygonal mesh composed entirely of triangles, where each vertex retains information about its corresponding superpixel in the original superpixel image.

[0075] Specifically, the superpixel image is composed of multiple irregular polygonal superpixels. In order to simplify and improve the efficiency of subsequent processes, the irregular polygonal pixel blocks are transformed into a more flexible triangular mesh by connecting the corresponding superpixel points of the superpixel image, which improves the execution efficiency of subsequent processes to a certain extent.

[0076] Step S206: Obtain camera position data corresponding to each triangular mesh; train the initial network model based on each target feature, each triangular mesh and each camera position data to obtain a target 3D reconstruction model; the target 3D reconstruction model is used to construct a 3D mesh map corresponding to the image set to be processed.

[0077] Here, camera position data refers to parameters such as the camera's position and angle when capturing images corresponding to the triangular mesh. The initial network model refers to a network model with an Encoder-Decoder framework. The target 3D reconstruction model refers to the model that best represents the image features after multiple training iterations of the initial network model based on target features, the triangular mesh, and camera position data.

[0078] Specifically, the camera position data, target features, and the triangular mesh itself corresponding to each triangular mesh are input into the initial network model for training. During this process, a loss function is designed to measure the predictive quality of the model. Based on the calculation result of the loss function, it is determined whether the currently trained initial network model meets the expected effect. When the expected effect is achieved, the network model that achieves the expected effect is used as the target 3D reconstruction model.

[0079] The aforementioned 3D reconstruction model generation method constructs superpixel maps corresponding to each image in the target image set, calculates the target features corresponding to each superpixel map, performs superpixel point connection operations on each superpixel map to obtain triangular meshes corresponding to each superpixel map, obtains camera position data corresponding to each triangular mesh, and trains an initial network model based on each target feature, each triangular mesh, and each camera position data to obtain a target 3D reconstruction model. The target 3D reconstruction model is used to construct a 3D mesh map corresponding to the image set to be processed, which improves the real-time performance of acquiring the target image set and the feasibility of inferring the icing situation of power transmission and transformation equipment based on the technology of this application. Constructing a 3D mesh map based on a 2D target image set improves the realism of the 3D reconstruction of the surrounding environment corresponding to the 2D target image set, thereby improving the accuracy of inferring the icing thickness and distribution of power transmission and transformation equipment.

[0080] In one embodiment, such as Figure 3 As shown, step S200 includes:

[0081] Step S300: Obtain the original image set, select a first number of images from the original image set as the first image set, and label each image in the first image set to obtain the first target image set.

[0082] Here, the original image set refers to the initial collection of images that have not undergone any processing. The first quantity refers to the number of original images selected by a human. The first image set refers to the set from which the first quantity of original images is selected. The first target image set refers to the set of images after each original image in the first image set has been labeled.

[0083] Specifically, the initial raw images have not yet been labeled. By labeling a small portion of the raw images, the remaining raw images are labeled using a semi-supervised AI model training technique, thus preparing data for subsequent training of the initial network model.

[0084] Step S302: Obtain the target semi-supervised model, take the unfiltered images in the original image set as the second image set, label each image in the second image set based on the target semi-supervised model to obtain the second target image set, and take the set of the first target image set and the second target image set as the target image set.

[0085] In this context, the target semi-supervised model refers to a model trained on both labeled and unlabeled images, which can be used to annotate the original images in the second image set with their corresponding labels. The second image set refers to the set of original images remaining after deleting the first image set from the original image set. The second target image set refers to the set of images in the second image set after labeling each original image in the second image set.

[0086] Specifically, by acquiring a target semi-supervised model trained on unlabeled and labeled images, each original image in the second image set is input into the target semi-supervised model to obtain the corresponding image output results. Based on these output results, each original image in the second image set is labeled to obtain a labeled second target image set. The first and second target image sets are then merged to obtain the target image set. This set prepares the data for subsequent superpixel segmentation of each image in the target image set using the SLIC (simple linear iterative clustering) method to construct the superpixel map corresponding to each image.

[0087] Step S304: Obtain the number of target superpixels and divide the target superpixel points into superpixel points in each image of the target image set.

[0088] The target superpixel count refers to the number of superpixels that are set to be distributed on the image.

[0089] Specifically, each image has a corresponding number of pixels. The total number of pixels in the image is taken as the total number of pixels. The preset number of target superpixels is obtained. The image is divided into the target number of superpixels with the same size. The size of each superpixel is the quotient of the total number of pixels divided by the number of target superpixels. The target number of superpixels are evenly distributed on the image. These superpixels all fall on the corresponding superpixel. At this time, the distance between adjacent superpixels is approximately the square root of the quotient of the total number of pixels divided by the number of target superpixels.

[0090] Step S306: Obtain the first target range, calculate the gradient value of each pixel within the first target range corresponding to each superpixel, and based on the comparison result of the gradient values ​​of each pixel corresponding to each superpixel, select the target pixel corresponding to each superpixel from the pixels, and move the position of each superpixel to the position of the corresponding target pixel to obtain the target superpixel.

[0091] Here, the first target range refers to the region composed of a predetermined number of superpixels, and the superpixels corresponding to the selected superpixels. The gradient value refers to the value obtained by calculating the directional derivative along the direction of the corresponding pixel. The target pixel refers to the pixel with the smallest gradient value selected from all the pixels; this target pixel facilitates the clustering of pixels with similar features. The target superpixel refers to the superpixel after its position has been changed.

[0092] Specifically, initially uniformly distributed superpixels cannot effectively cluster pixels. Therefore, instead of continuously optimizing and iterating, we calculate the gradient value of each pixel within the first target range of each superpixel node, select the pixel with the smallest gradient value within the first target range of each superpixel node as the target superpixel, and then move the position of the superpixel to the corresponding target superpixel. This avoids the superpixel falling on the contour boundary with a large gradient, which would affect the pixel clustering effect.

[0093] Step S308: Obtain the second target range, calculate the target distance between each target superpixel and each pixel within the corresponding second target range, and based on the comparison results of each target distance, classify each pixel into the corresponding target superpixel category to obtain the superpixel image corresponding to each image in the target image set.

[0094] The second target range refers to the area defined based on the distances between adjacent superpixels. For example, if the distance between adjacent superpixels is S, the second target range might be set to 2S*2S. The target distance refers to the combined distances between the target superpixel within the second target range and all pixels within that range; these distances primarily include color distance and spatial distance. The target superpixel category refers to the category of the cluster center corresponding to the pixel, and the corresponding target superpixel is also the cluster center of the corresponding pixel.

[0095] Specifically, the range of the second target is generally larger than the range of the superpixels corresponding to the target superpixels. Therefore, each pixel will be searched by multiple superpixels and the corresponding target distance will be calculated. The smaller the target distance, the closer the pixel is to the corresponding target superpixel. Therefore, the target superpixel with the smallest target distance is selected from the target distances of each pixel as the cluster center of the pixel, that is, the pixel is classified into the target superpixel category. In order to obtain a more ideal effect, the above steps need to be iterated until the error converges, that is, the target superpixel category corresponding to each pixel no longer changes, which is conducive to the subsequent training of the model to better learn the relevant rules of the image.

[0096] In this embodiment, the target image set is obtained by labeling the original image set and constructing superpixel maps corresponding to each image in the target image set. This is beneficial for obtaining more useful information from the images and improves the efficiency of learning important image patterns during subsequent model training to a certain extent. This is conducive to improving the accuracy of inferring the icing thickness and distribution of power transmission and transformation equipment.

[0097] In one embodiment, such as Figure 4 As shown, step S202 includes:

[0098] Step S400: Obtain the features of each pixel in each superpixel image.

[0099] Specifically, each superpixel image contains a corresponding number of pixels. Each pixel has different features such as color, brightness, and texture. It is necessary to obtain the features of each pixel to prepare data for the subsequent calculation of the features of each superpixel in each superpixel image.

[0100] Step S402: The features of pixels with the same target superpixel category corresponding to each pixel are fused to obtain the features corresponding to each superpixel in each superpixel image.

[0101] Fusion refers to averaging the features of each pixel belonging to the same target superpixel category.

[0102] Specifically, each superpixel serves as the cluster center for its corresponding pixels, and can be used to approximate the characteristics of each pixel. The features of each superpixel are obtained by averaging the features of all pixels belonging to the same target superpixel category. This averaging method effectively summarizes the characteristics of each pixel corresponding to the superpixel. The features of each superpixel include, but are not limited to, color, texture, edge characteristics, and image transformation coefficients.

[0103] Step S404: Fuse the features corresponding to each superpixel point in each superpixel image to obtain the target features of each superpixel image.

[0104] Fusion refers to the operation of merging the features corresponding to each superpixel in each superpixel image.

[0105] Specifically, the features corresponding to each superpixel in each extracted superpixel image are still scattered. These features need to be merged to obtain the target features corresponding to each superpixel image, and then input together into the initial network model in the subsequent process.

[0106] In this embodiment, by acquiring the features of each pixel in each superpixel image, calculating the average of the features of each pixel corresponding to each superpixel image, the features corresponding to each superpixel image in each superpixel image are obtained, and the features of each superpixel image are merged to obtain the target features corresponding to each superpixel image. This ensures that each superpixel image contains the feature information of the corresponding pixel, enriching the useful information corresponding to each image, and to a certain extent, improving the generalization ability of the target 3D reconstruction model.

[0107] In one embodiment, such as Figure 5 As shown, step S206 includes:

[0108] Step S500: Based on the time frames corresponding to each of the triangular meshes, select the triangular mesh corresponding to the current time frame from the triangular meshes as the current triangular mesh.

[0109] Here, a time frame is the time it takes for the camera to capture an image. The current time frame refers to the time frame corresponding to the image being processed under the current task. The current triangle grid refers to the triangle grid corresponding to the current image.

[0110] Specifically, image processing must be performed according to the corresponding time frames to ensure continuous changes in the image across consecutive time frames, thereby avoiding significant errors in the final result caused by discontinuous changes. The subsequently formed deformed mesh is obtained by deforming the triangular mesh; therefore, according to the order of the time frames, the triangular mesh corresponding to the image in the current time frame is selected as the current triangular mesh.

[0111] Step S502: Based on the encoding layer of the initial network model and the target features corresponding to the current triangular mesh, calculate the target curvature corresponding to each vertex in the current triangular mesh.

[0112] The encoding layer refers to the encoder layer, which transforms the mathematical sequence corresponding to the image into a fixed-length vector. The target curvature refers to applying the continuous Laplace-Beltrami operator to the image corresponding to the current triangular mesh, achieving cotangent-based discretization, and obtaining the changing curvature corresponding to each vertex of the current triangular mesh.

[0113] Specifically, during the transformation from the superpixel image to the current triangular mesh, the corresponding points remain unchanged; what changes is the connection between the points, i.e., the set of edges. Based on the edge operators extracted from the target features, cotangent-based discretization using the continuous Laplace-Beltrami operator is employed to obtain the curvature corresponding to each vertex of the current triangular mesh, providing a data foundation for the subsequent transformation from the current triangular mesh to the current deformed mesh.

[0114] Step S504: Obtain the target dependency matrix and target handle position. Based on the target curvature corresponding to each vertex, the target dependency matrix, and the target handle position, obtain the current deformed mesh corresponding to the current triangle mesh.

[0115] The target dependency matrix is ​​used to fine-tune the positions of each vertex by weighting the coefficients of the current triangular mesh. The target handle position refers to the position of each vertex from the current triangular mesh to the current deformed mesh. The current deformed mesh refers to the deformed mesh obtained by deforming the current triangular mesh.

[0116] Specifically, to fine-tune each vertex based on the current triangle mesh, a target dependency matrix needs to be randomly initialized. When initializing the target dependency matrix, the target dependency matrix is ​​initialized based on the farthest point sampling (FPS) of the current triangle mesh. A set of vertices is listed, and the vertices in this set are required to be approximately equidistant. Each row of the target dependency matrix is ​​initialized based on the survey distance from each vertex of the current triangle mesh to the vertices in this set, as shown in formula (1). In formula (1), v k For the set of vertices, For vertex v k The distance to a fixed vertex i. Next, using the target dependency matrix, the unit vector (also called the handle) corresponding to each vertex is calculated, as shown in formula (2). In formula (2), H represents the handle matrix, and V is the set of points corresponding to each type of graph, which is also the set of nodes corresponding to the final 3D mesh graph. For the current triangular mesh, the corresponding target handle position can be calculated as shown in formula (3). In formula (3) Let A be the target handle position, A be the target dependency matrix, T be the template handle, and Δ be the target handle position. H This is the node offset. Based on the target handle position. Obtain deformed mesh V * As the minimum value of the second-order loss in formula (4), in formula (4), V * The mesh is a deformable mesh, where L is the target curvature of each vertex, V is the vertex set, T is the template handle, and A is the target dependency matrix. The target handle location, where ||LV-LT|| 2 Used to ensure the generation of V * The distribution satisfies the rationality of curvature. Used to ensure the generation of V * The locations that match the statistically similar data (not necessarily completely overlapping, but as close as possible). The stationary point of formula (4) can be found through formula (5), where L... T Let A be the transpose of the target curvature L at each vertex. T V is the transpose of the target dependency matrix A. * For deformable meshes, T is the template handle, where, given (L T L+A T A) is symmetric, positive semi-definite, and sparse, with solution V * This can be efficiently computed using conjugate gradients or sparse Cholesky decomposition. Furthermore, L and T can be fixed, i.e., by inverting equation (5), we obtain the hypothetical equation (6). In equation (6), C and D are obtained through L... T L+A T The constant matrix obtained by multiplying the inverse matrix of A with the linear system equation (i.e., formula (5)) can effectively solve the ill-posed problem in the three-dimensional reconstruction process and make the operation process simple and fast.

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123] Step S506: Obtain the target loss function; calculate the current loss value corresponding to the current deformed network based on the target loss function; and take the current deformed mesh as the target deformed mesh corresponding to the current triangular mesh based on the comparison result between the current loss value and the target condition.

[0124] The target loss function is a computational function used to measure the difference between the model's test values ​​and the true values. The current loss value refers to the loss value obtained when the current deformed mesh is derived from the current triangular mesh. The target condition refers to the set magnitude of the loss value; a loss value that satisfies this condition indicates that the corresponding current deformed mesh has reached a relatively ideal state. The target deformed mesh refers to the deformed mesh obtained by deforming the current triangular mesh, resulting in a mesh that most closely resembles the corresponding target image.

[0125] Specifically, during model training, it's generally impossible to determine the target deformable mesh corresponding to the current triangular mesh in a single training iteration. Therefore, it's necessary to deform the corresponding triangular mesh multiple times and calculate the corresponding current loss value. When the current loss value meets the target condition, it indicates that the current deformable mesh has reached a relatively ideal state. If not, backpropagation continues, the corresponding parameters are updated, and training continues on the current triangular mesh until the desired target deformable mesh is obtained. When calculating the target loss function value, in addition to using the data corresponding to the current triangular mesh, the camera position data of the triangular mesh corresponding to the next time frame is also needed. The current loss value is obtained by analyzing the changes in camera position and vertex displacement in adjacent time frames.

[0126] Step S508: Take the next time frame corresponding to the current time frame as the current time frame, and repeat the operation of selecting the triangle mesh corresponding to the current time frame from each triangle mesh as the current triangle mesh until each triangle mesh has a corresponding target deformable mesh.

[0127] Specifically, the final 3D reconstruction is achieved based on the target deformation mesh corresponding to each target image. Therefore, it is necessary to know the target deformation mesh corresponding to each target image in order to form the final 3D mesh map based on each target deformation mesh.

[0128] Step S510: Based on the decoding layer of the initial network model, the target deformable meshes corresponding to each triangular mesh are fused to generate a target 3D mesh map, thereby obtaining the target 3D reconstruction model.

[0129] In this context, the decoding layer refers to the decoder layer, which transforms the fixed-length vector obtained by the encoding layer into an output sequence corresponding to the image. Fusion refers to the construction of a 3D model based on the information from the deformed meshes of each target. The target 3D mesh map refers to the 3D image constructed based on the deformed meshes of each target.

[0130] In this embodiment, the initial network model is trained multiple times based on each triangular grid, the target features corresponding to each triangular grid, and the camera position data. The target 3D reconstruction model is determined based on the target loss function, realizing the encoding and decoding process of the target image corresponding to the triangular grid. The training obtains the pattern of transforming the two-dimensional target image set into the target 3D grid map, and reconstructs the 3D map corresponding to the target image set with good realism, which is conducive to improving the accuracy of inferring the icing thickness and distribution of power transmission and transformation equipment.

[0131] In one embodiment, such as Figure 6 As shown, before step S506, the following steps are also included:

[0132] Step S600: Obtain the motion reprojection loss function, the keypoint reprojection loss function, the texture loss function, the local rigidity loss function, and the region similarity loss function.

[0133] The motion reprojection loss function calculates the error in determining the positional changes of corresponding objects in an image across different time frames. The keypoint reprojection loss function calculates the error between predicted surface features and the actual ground conditions of corresponding points. The texture loss function compares the image based on deformed mesh textures and the original image appearance after being masked by a randomly generated mask, using a perceptual similarity metric. The local stiffness loss function calculates the Euclidean distance between vertices within the extended neighborhood of the corresponding vertex in the image at a given time frame. The extended neighborhood refers to the region obtained by drawing a circle with a small radius centered on the corresponding vertex. Euclidean distance, also known as Euclidean distance, is a commonly used distance definition that refers to the true distance between two points in m-dimensional space, or the natural length of a vector (i.e., the distance from the point to the origin). In two-dimensional and three-dimensional space, the Euclidean distance is the actual distance between two points.

[0134] Specifically, in order to determine the quality of the model obtained after training the initial network model, it is necessary to design a corresponding loss function to measure the quality of the trained model. In this embodiment, by combining the application scenario of this application, the original loss function is slightly improved to obtain the corresponding motion reprojection loss function, keypoint reprojection loss function, texture loss function, local rigidity loss function, and region similarity loss function. The target loss function that conforms to this application is designed through these loss functions. The motion reprojection loss function is shown in formula (7), where u i The displacement of adjacent time frames when input into the model (including during the optimization process). γ represents the displacement between adjacent frames. iA visibility mask is an N-dimensional vector containing only 0s and 1s, manually labeled. A 1 indicates that the labeler considers the data point valid, and a 0 indicates invalid. For example, {0,1,1,1,0,…} means that the 2nd, 3rd, and 4th points are valid, while the 1st and 5th points are invalid; where… The calculation process can be shown in formulas (8) to (9). In formula (8), p i,t Let V be the new coordinates of the i-th vertex in the t-th frame under weak perspective projection, where π is the weak perspective projection. i,t Let C be the i-th vertex in frame t. t Let p be the camera parameters corresponding to frame t (camera height and position in a defined coordinate system). In formula (9), p i,t+1 Let V be the new coordinates of the i-th vertex in the (t+1)-th frame under weak perspective projection, where π is the weak perspective projection. i,t+1 C is the i-th vertex in frame t+1. t+1 The camera parameters at frame t+1; the keypoint reprojection loss function is shown in formula (11), where L kp For keypoint reprojection loss, K i π(K) is the scaling constant. i V,C) are the coordinates on the image plane after projection; the texture loss function is as shown in formula (12), where L pixel For texture loss, I represents the superpixel image, and I represents the target image. S is a mask of the same size as I and randomly generated; the local rigidity loss function is shown in Equation (13), where L rigid For local rigidity loss, u is the preservation point, and ||V(u)-V(u′)|| is the distance between the constraint extension neighborhood and each vertex in the current frame. The constraint for the next frame; the region similarity loss function is shown in Equation (14), where L is the region similarity loss function. mask For the region similarity loss, S i For instance segmentation, V i For the vertex, π i These are camera parameters.

[0135]

[0136] p i,t =π(V) i,t C t (8)

[0137] p i,t+1 =π(V) i,t+1 C t+1 (9)

[0138]

[0139]

[0140]

[0141]

[0142]

[0143] Step S602: Combine the motion reprojection loss function, keypoint reprojection loss function, texture loss function, local rigidity loss function, and region similarity loss function to obtain the target loss function.

[0144] The objective loss function is used to measure the quality of models with different parameters obtained by training the initial network model.

[0145] Specifically, in order to obtain a target 3D reconstruction model with better generalization performance, it is necessary to consider the impact of error loss during model training from multiple aspects. The target loss function is designed by adding the motion reprojection loss function, key point reprojection loss function, texture loss function, local rigidity loss function and region similarity loss function. The target loss function is shown in formula (15).

[0146] L total =L motion +L kp +L pixel +L grid +L mask (15)

[0147] In this embodiment, the target loss function is obtained by combining the motion reprojection loss function, keypoint reprojection loss function, texture loss function, local rigidity loss function, and region similarity loss function. This approach helps to reduce the impact of error loss during model training by considering multiple factors, thereby facilitating the training of a target 3D reconstruction model with better generalization performance.

[0148] In one embodiment, such as Figure 7 As shown, after step S206, the following steps are also included:

[0149] Step S700: Obtain the target image set to be processed, and construct the target superpixel map corresponding to each image in the target image set to be processed.

[0150] The target image set refers to the collection of original images acquired to construct the corresponding 3D mesh map. The target superpixel map refers to the superpixel map corresponding to the target image.

[0151] Specifically, when constructing a three-dimensional mesh map corresponding to a two-dimensional target image, it is necessary to obtain the superpixel map corresponding to each image in order to better obtain the features of the pixels on each image.

[0152] Step S702: Calculate the target features corresponding to each target superpixel image.

[0153] Among them, target features refer to the combined features of each superpixel point on the target superpixel image, which can be used to express the characteristics of the corresponding target superpixel image.

[0154] Specifically, before predicting the 3D mesh map corresponding to the target image set to be processed through the target 3D reconstruction model, it is necessary to know the target features corresponding to each image.

[0155] Step S704: Connect the superpixel points of the target superpixel images corresponding to each image to obtain the target triangle mesh corresponding to each target superpixel image.

[0156] The target triangular mesh refers to a mesh image constructed by connecting the various superpixel points in the target superpixel image, where each point is a triangle.

[0157] Specifically, to make the operation more flexible and simple and to improve the efficiency of 3D reconstruction, the superpixel image is further converted into a triangular mesh image.

[0158] Step S706: Obtain the target camera position data corresponding to each target triangle mesh, obtain the target 3D reconstruction model, input the target features, the target triangle mesh and the target camera position data into the target 3D reconstruction model to obtain the target 3D mesh map corresponding to the target image set to be processed.

[0159] Among them, target camera position data refers to the position, height, and other parameters of the camera when capturing the target triangular mesh corresponding to the target image to be processed. The target 3D mesh map refers to the 3D mesh map constructed from the 2D target image to be processed.

[0160] In this embodiment, by acquiring the target features, target triangular mesh, and target camera position data corresponding to the target image to be processed, and inputting the target features, target triangular mesh, and target camera position data into the target 3D mesh map, the corresponding target 3D mesh map is obtained. This realizes the transformation of the 2D target image to be processed into the corresponding target 3D mesh map. Information on the 3D mesh map can be inferred from the target 3D mesh map. To a certain extent, this ensures the authenticity of the 3D mesh map and improves the accuracy of inferring the icing thickness and distribution of power transmission and transformation equipment.

[0161] In one embodiment, image data collected and labeled by a drone using a power grid inspection system is used as the target image set. Superpixel maps corresponding to each image in the target image set are constructed. Target features corresponding to each superpixel map are calculated, and corresponding triangular meshes are constructed. Camera position data corresponding to each triangular mesh is obtained. Based on the target features, triangular meshes, and camera position data, the initial network model is trained multiple times to determine the target 3D reconstruction model. This target 3D reconstruction model is used for... Figure 8 This is one embodiment of the process of encoding and decoding a triangular mesh. Figure 8 In the middle, π represents the camera position data, the Handle offset represents the target handle position, and the texture represents data containing vertex attributes and edge information.

[0162] Based on the same inventive concept, this application also provides a three-dimensional reconstruction model generation apparatus for implementing the three-dimensional reconstruction model generation method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the three-dimensional reconstruction model generation apparatus provided below can be found in the limitations of the three-dimensional reconstruction model generation method described above, and will not be repeated here.

[0163] In one embodiment, such as Figure 9 As shown, a three-dimensional reconstruction model generation device is provided, including: a superpixel map construction module 900, a feature acquisition module 902, a mesh construction module 904, and a target model determination module 906, wherein:

[0164] The superpixel image construction module 900 is used to acquire a target image set and construct superpixel images corresponding to each image in the target image set.

[0165] The feature acquisition module 902 is used to calculate the features corresponding to each superpixel point in each superpixel image, and fuse the features corresponding to each superpixel image to obtain the target features corresponding to each superpixel image.

[0166] The mesh construction module 904 is used to perform superpixel point connection operations on each superpixel image to obtain the triangular mesh corresponding to each superpixel image.

[0167] The target model determination module 906 is used to obtain the camera position data corresponding to each triangular mesh, and to train the initial network model based on each target feature, each triangular mesh and each camera position data to obtain a target 3D reconstruction model. The target 3D reconstruction model is used to construct a 3D mesh map corresponding to the image set to be processed.

[0168] In one embodiment, the superpixel image construction module 900 is further configured to: acquire an original image set; select a first number of images from the original image set as a first image set; label each image in the first image set to obtain a first target image set; acquire a target semi-supervised model; use the unselected images in the original image set as a second image set; label each image in the second image set based on the target semi-supervised model to obtain a second target image set; and use the set of the first target image set and the second target image set as the target image set; acquire the target superpixel count; and divide each image in the target image set into superpixel points of the target superpixel count; acquire the first... The target range is defined by calculating the gradient value of each pixel within a first target range corresponding to each superpixel. Based on the comparison of the gradient values ​​of each pixel within a first target range corresponding to each superpixel, target pixels corresponding to each superpixel are selected from the pixels. The positions of each superpixel are then moved to the positions of the corresponding target pixels to obtain the target superpixels. A second target range is then defined by calculating the target distance between each target superpixel and each pixel within the corresponding second target range. Based on the comparison of the target distances, each pixel is classified into a corresponding target superpixel category to obtain the superpixel images corresponding to each image in the target image set.

[0169] In one embodiment, the feature acquisition module 902 is further configured to acquire the features of each pixel in each superpixel image; fuse the features of pixels of the same target superpixel category corresponding to each pixel to obtain the features corresponding to each superpixel in each superpixel image; and fuse the features corresponding to each superpixel in each superpixel image to obtain the target features of each superpixel image.

[0170] In one embodiment, the target model determination module 906 is further configured to: select the triangle mesh corresponding to the current time frame from the triangle meshes based on the time frames corresponding to the triangle meshes; calculate the target curvature corresponding to each vertex in the current triangle mesh based on the target features corresponding to the current triangle mesh, based on the encoding layer of the initial network model; obtain the target dependency matrix and the target handle position, and obtain the current deformed mesh corresponding to the current triangle mesh based on the target curvature corresponding to each vertex, the target dependency matrix, and the target handle position; obtain the target loss function, calculate the current loss value corresponding to the current deformed mesh based on the target loss function, and use the current deformed mesh as the target deformed mesh corresponding to the current triangle mesh based on the comparison result of the current loss value and the target conditions; take the next time frame corresponding to the current time frame as the current time frame, and repeat the operation of selecting the triangle mesh corresponding to the current time frame from the triangle meshes as the current triangle mesh until each triangle mesh has a corresponding target deformed mesh; and fuse the target deformed meshes corresponding to each triangle mesh based on the decoding layer of the initial network model to generate a target 3D mesh map, thereby obtaining the target 3D reconstruction model.

[0171] In one embodiment, the 3D reconstruction model generation device further includes a loss design module 908, used to obtain a motion reprojection loss function, a keypoint reprojection loss function, a texture loss function, a local rigidity loss function, and a region similarity loss function; and to combine the motion reprojection loss function, the keypoint reprojection loss function, the texture loss function, the local rigidity loss function, and the region similarity loss function to obtain the target loss function.

[0172] In one embodiment, the 3D reconstruction model generation device further includes a model usage module 910, configured to acquire a target image set to be processed, construct a target superpixel map corresponding to each image in the target image set to be processed; calculate the target features corresponding to each target superpixel map; connect the superpixel points of the target superpixel maps corresponding to each image to obtain a target triangle mesh corresponding to each target superpixel map; acquire the target camera position data corresponding to each target triangle mesh; acquire a target 3D reconstruction model; and input the target features, the target triangle mesh, and the target camera position data into the target 3D reconstruction model to obtain a target 3D mesh map corresponding to the target image set to be processed.

[0173] Each module in the aforementioned 3D reconstruction model generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0174] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores image data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When executed by the processor, the computer program implements a method for generating a 3D reconstruction model.

[0175] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for generating a three-dimensional reconstruction model. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0176] Those skilled in the art will understand that Figure 10 and Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0177] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0178] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0179] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0180] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0181] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0182] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0183] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating a three-dimensional reconstruction model, characterized in that, The method includes: Obtain the target image set and construct the superpixel map corresponding to each image in the target image set; Calculate the features corresponding to each superpixel point in each superpixel image, and fuse the features corresponding to each superpixel image to obtain the target features corresponding to each superpixel image. Perform superpixel point connection operations on each superpixel image to obtain the triangular mesh corresponding to each superpixel image; The camera position data corresponding to each triangular grid is obtained. Based on each target feature, each triangular grid and each camera position data, the initial network model is trained to obtain the target 3D reconstruction model. The target 3D reconstruction model is used to construct the 3D grid map corresponding to the image set to be processed. The step of training the initial network model based on each target feature, each triangular mesh, and each camera position data to obtain the target 3D reconstruction model includes: Based on the time frame corresponding to each of the triangular grids, the triangular grid corresponding to the current time frame is selected from each of the triangular grids as the current triangular grid; Based on the encoding layer of the initial network model, and based on the target features corresponding to the current triangular mesh, the target curvature corresponding to each vertex in the current triangular mesh is calculated; Obtain the target dependency matrix and target handle position, and based on the target curvature corresponding to each vertex, the target dependency matrix, and the target handle position, obtain the current deformed mesh corresponding to the current triangle mesh; Obtain the target loss function, calculate the current loss value corresponding to the current deformed mesh based on the target loss function, and take the current deformed mesh as the target deformed mesh corresponding to the current triangular mesh based on the comparison result between the current loss value and the target condition; Take the next time frame corresponding to the current time frame as the current time frame, and repeat the operation of selecting the triangle mesh corresponding to the current time frame from each triangle mesh as the current triangle mesh until each triangle mesh has a corresponding target deformable mesh. Based on the decoding layer of the initial network model, the target deformable meshes corresponding to each triangular mesh are fused to generate a target 3D mesh map, thus obtaining the target 3D reconstruction model.

2. The method according to claim 1, characterized in that, The step of obtaining the target image set and constructing the superpixel map corresponding to each image in the target image set includes: Obtain the original image set, select a first number of images from the original image set as the first image set, and label each image in the first image set to obtain the first target image set; Obtain the target semi-supervised model, take the unfiltered images in the original image set as the second image set, label each image in the second image set based on the target semi-supervised model to obtain the second target image set, and take the combination of the first target image set and the second target image set as the target image set; Obtain the target superpixel count, and divide the target superpixel count into superpixel points in each image of the target image set; Obtain a first target range, calculate the gradient value of each pixel within the first target range corresponding to each superpixel, and based on the comparison result of the gradient values ​​of each pixel corresponding to each superpixel, select the target pixel corresponding to each superpixel from each pixel, and move the position of each superpixel to the position of the corresponding target pixel to obtain the target superpixel. Obtain the second target range, calculate the target distance between each target superpixel and each pixel within the corresponding second target range, and based on the comparison results of each target distance, classify each pixel into the corresponding target superpixel category to obtain the superpixel map corresponding to each image in the target image set.

3. The method according to claim 1, characterized in that, The process of calculating the features corresponding to each superpixel point in each superpixel image and fusing the features corresponding to each superpixel image to obtain the target features corresponding to each superpixel image includes: Obtain the features of each pixel in each superpixel image; The features of pixels of the same target superpixel category corresponding to each pixel are fused to obtain the features corresponding to each superpixel in each superpixel image. The features corresponding to each superpixel point in each superpixel image are fused to obtain the target features of each superpixel image.

4. The method according to claim 1, characterized in that, Before obtaining the target loss function, the process also includes: Obtain the motion reprojection loss function, the keypoint reprojection loss function, the texture loss function, the local rigidity loss function, and the region similarity loss function. The target loss function is obtained by combining the motion reprojection loss function, keypoint reprojection loss function, texture loss function, local rigidity loss function, and region similarity loss function.

5. The method according to claim 1, characterized in that, After training the initial network model based on the target features, the triangular meshes, and the camera position data to obtain the target 3D reconstruction model, the process further includes: Obtain the target image set to be processed, and construct the target superpixel map corresponding to each image in the target image set to be processed; Calculate the target features corresponding to each target superpixel image; Connect the superpixel points of the target superpixel images corresponding to each image to obtain the target triangle mesh corresponding to each target superpixel image; Obtain the target camera position data corresponding to each target triangle mesh, obtain the target 3D reconstruction model, input the target features, the target triangle mesh and the target camera position data into the target 3D reconstruction model, and obtain the target 3D mesh map corresponding to the target image set to be processed.

6. A three-dimensional reconstruction model generation device, characterized in that, The device includes: The superpixel image construction module is used to acquire a target image set and construct a superpixel image corresponding to each image in the target image set; The feature acquisition module is used to calculate the features corresponding to each superpixel point in each superpixel image, and fuse the features corresponding to each superpixel image to obtain the target features corresponding to each superpixel image. The mesh construction module is used to perform superpixel point connection operations on each superpixel image to obtain the triangular mesh corresponding to each superpixel image; The target model determination module is used to acquire camera position data corresponding to each triangular mesh, train an initial network model based on each target feature, each triangular mesh, and each camera position data to obtain a target 3D reconstruction model, which is used to construct a 3D mesh map corresponding to the image set to be processed; it is also used to select the triangular mesh corresponding to the current time frame from the triangular meshes based on the time frames corresponding to each triangular mesh; calculate the target curvature corresponding to each vertex in the current triangular mesh based on the encoding layer of the initial network model and the target features corresponding to the current triangular mesh; obtain the target dependency matrix and the target handle position, and train an initial network model based on the target curvature corresponding to each vertex, the target dependency matrix, and the target handle position. The target handle position is determined, and the current deformed mesh corresponding to the current triangular mesh is obtained. The target loss function is acquired, and based on the target loss function, the current loss value corresponding to the current deformed mesh is calculated. Based on the comparison result between the current loss value and the target conditions, the current deformed mesh is taken as the target deformed mesh corresponding to the current triangular mesh. The next time frame corresponding to the current time frame is taken as the current time frame, and the operation of selecting the triangular mesh corresponding to the current time frame from each triangular mesh is repeated until each triangular mesh has a corresponding target deformed mesh. Based on the decoding layer of the initial network model, the target deformed meshes corresponding to each triangular mesh are fused to generate a target 3D mesh map, thus obtaining the target 3D reconstruction model.

7. The apparatus according to claim 6, characterized in that, The device further includes a loss design module, used to obtain a motion reprojection loss function, a keypoint reprojection loss function, a texture loss function, a local rigidity loss function, and a region similarity loss function; and to combine the motion reprojection loss function, the keypoint reprojection loss function, the texture loss function, the local rigidity loss function, and the region similarity loss function to obtain the target loss function.

8. The apparatus according to claim 6, characterized in that, The feature acquisition module is further configured to: acquire the features of each pixel in each superpixel image; fuse the features of pixels of the same category as the target superpixel corresponding to each pixel to obtain the features corresponding to each superpixel in each superpixel image; and fuse the features corresponding to each superpixel in each superpixel image to obtain the target features of each superpixel image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images