Three-dimensional model construction method, device, computer equipment and storage medium
By filtering and processing mapped points in two-dimensional images and building a three-dimensional model based on vertex similarity, the model gelatinization problem caused by the small number of vertices in the existing technology is solved, and the reconstruction accuracy of the three-dimensional model is improved.
Patent Information
- Application Number
- CN202011009375.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-23
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2040-09-23
Smart Images
Figure CN112150612B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer vision technology, and in particular to a three-dimensional model construction method, device, computer equipment and storage medium. Background Art
[0002] In recent years, with the rapid development of intelligent robots, unmanned driving technology, virtual reality technology, etc., three-dimensional model reconstruction based on two-dimensional images is a very important research direction in the field of computer vision.
[0003] In the prior art, the process of reconstructing a three-dimensional model based on a two-dimensional image is generally: inputting the two-dimensional image into a trained neural network model to obtain the three-dimensional coordinates of the mapping points output by the neural network model, wherein the number of mapping points is pre-set during the training of the neural network model.
[0004] However, in the above method, when reconstructing a three-dimensional model of a target object with a small number of vertices, too many mapping points will appear on the vertices of the reconstructed three-dimensional model of the target object, resulting in model blurring. Therefore, the reconstruction accuracy of the three-dimensional model is poor. Summary of the invention
[0005] Based on this, it is necessary to provide a three-dimensional model construction method, device, computer equipment and storage medium that can improve the accuracy of the three-dimensional model of the target object in order to solve the above technical problems.
[0006] A three-dimensional model construction method, the method comprising:
[0007] Acquire a two-dimensional image, where the two-dimensional image includes a target object for which a three-dimensional model is to be constructed;
[0008] Acquire three-dimensional coordinates of multiple mapping points of the target object and vertex similarity prediction values of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point;
[0009] Based on the vertex similarity prediction value of each mapping point, multiple target mapping points are selected from multiple mapping points;
[0010] A 3D model of the target object is constructed based on the 3D coordinates of multiple target mapping points.
[0011] In one embodiment, based on the vertex similarity prediction value of each mapping point, a plurality of target mapping points are selected from a plurality of mapping points, including:
[0012] Arrange the mapping points in descending order based on the predicted vertex similarity values of the mapping points, and determine the mapping points as candidate mapping points in descending order;
[0013] The distances from other mapping points to the candidate mapping points are calculated, and mapping points whose distances to the candidate mapping points are less than a distance threshold are eliminated to obtain multiple target mapping points.
[0014] In one embodiment, after eliminating the mapping points whose distance to the candidate mapping points is less than the distance threshold, the method further includes:
[0015] Mapping points whose vertex similarity prediction values are less than a similarity threshold are removed from multiple mapping points.
[0016] In one embodiment, before selecting a plurality of target mapping points from a plurality of mapping points based on the vertex similarity prediction values of each mapping point, the method further comprises:
[0017] Decentralizing the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after the decentralized processing;
[0018] Normalizing each first mapping point to obtain a plurality of second mapping points;
[0019] Based on the vertex similarity prediction value of each mapping point, multiple target mapping points are selected from multiple mapping points, including:
[0020] Based on the vertex similarity prediction value of each second mapping point, a plurality of target mapping points are screened out from the plurality of second mapping points.
[0021] In one embodiment, the three-dimensional coordinates of each mapping point are decentralized to obtain a plurality of first mapping points after the decentralized processing, including:
[0022] Obtain the average coordinate value of each coordinate axis in the three-dimensional coordinates of multiple mapping points;
[0023] For each mapping point, the three-dimensional coordinates of the mapping point are respectively subtracted from the average coordinates of the corresponding coordinate axes to obtain the first mapping point after decentralized processing.
[0024] In one embodiment, each first mapping point is normalized to obtain a plurality of second mapping points, including:
[0025] Obtain the maximum distance from each first mapping point to the origin;
[0026] For each first mapping point, the three-dimensional coordinates of the first mapping point are divided by the maximum distance to obtain a second mapping point.
[0027] In one embodiment, obtaining three-dimensional coordinates of a plurality of mapping points of a target object and a vertex similarity prediction value of each mapping point based on a two-dimensional image includes:
[0028] Obtain three-dimensional structural features of two-dimensional images;
[0029] Evenly collect the target number of initial mapping points in a preset reference plane, and construct an initial coordinate matrix according to the two-dimensional coordinates of each initial mapping point, where each row of the initial coordinate matrix corresponds to the two-dimensional coordinate of an initial mapping point;
[0030] After splicing the three-dimensional structural features to each row of the initial coordinate matrix, a spliced matrix is obtained;
[0031] The concatenated matrix is input into the trained decoder, and the three-dimensional coordinates of the target number of mapping points and the vertex similarity prediction value of each mapping point are obtained as output by the decoder.
[0032] A three-dimensional model building device, the device comprising:
[0033] An image acquisition module is used to acquire a two-dimensional image, wherein the two-dimensional image includes a target object to be constructed into a three-dimensional model;
[0034] A mapping point acquisition module, used to acquire the three-dimensional coordinates of multiple mapping points of the target object and the vertex similarity prediction value of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point;
[0035] A screening module, used for screening a plurality of target mapping points from a plurality of mapping points based on a vertex similarity prediction value of each mapping point;
[0036] The model building module is used to build a three-dimensional model of the target object based on the three-dimensional coordinates of multiple target mapping points.
[0037] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0038] Acquire a two-dimensional image, where the two-dimensional image includes a target object for which a three-dimensional model is to be constructed;
[0039] Acquire three-dimensional coordinates of multiple mapping points of the target object and vertex similarity prediction values of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point;
[0040] Based on the vertex similarity prediction value of each mapping point, multiple target mapping points are selected from multiple mapping points;
[0041] A 3D model of the target object is constructed based on the 3D coordinates of multiple target mapping points.
[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:
[0043] Acquire a two-dimensional image, where the two-dimensional image includes a target object for which a three-dimensional model is to be constructed;
[0044] Acquire three-dimensional coordinates of multiple mapping points of the target object and vertex similarity prediction values of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point;
[0045] Based on the vertex similarity prediction value of each mapping point, multiple target mapping points are selected from multiple mapping points;
[0046] A 3D model of the target object is constructed based on the 3D coordinates of multiple target mapping points.
[0047] The above-mentioned three-dimensional model construction method, device, computer equipment and storage medium obtain a two-dimensional image, wherein the two-dimensional image includes a target object to be constructed with a three-dimensional model; obtain the three-dimensional coordinates of multiple mapping points of the target object and the vertex similarity prediction value of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point; based on the vertex similarity prediction value of each mapping point, multiple target mapping points are screened out from multiple mapping points; and a three-dimensional model of the target object is constructed based on the three-dimensional coordinates of the multiple target mapping points. In the embodiment of the present application, screening each mapping point based on the vertex similarity prediction value of each mapping point can filter out some mapping points that do not belong to vertices, thereby better adapting to the reconstruction of three-dimensional models of target objects with different numbers of vertices. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 A schematic diagram of a process of constructing a three-dimensional model in one embodiment;
[0049] Figure 2 A flowchart of the steps of obtaining a mapping point in one embodiment;
[0050] Figure 3 is a schematic diagram of the structure of a multilayer perceptron in one embodiment;
[0051] Figure 4 A flowchart of steps for screening mapping points in one embodiment;
[0052] Figure 5 A flowchart of another step of screening mapping points in one embodiment;
[0053] Figure 6 A structural block diagram of a three-dimensional model building device in one embodiment;
[0054] Figure 7FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0056] In recent years, with the rapid development of intelligent robots, unmanned driving technology, virtual reality technology, etc., 3D model reconstruction based on 2D images is a very important research direction in the field of computer vision. 3D model reconstruction based on 2D images specifically refers to: restoring the 3D model structure of the target object based on a 2D image.
[0057] In the prior art, the process of reconstructing a three-dimensional model based on a two-dimensional image is generally: inputting the two-dimensional image into a trained neural network model to obtain the three-dimensional coordinates of the mapping points output by the neural network model, wherein the number of mapping points is pre-set during the training of the neural network model.
[0058] However, in the above method, when reconstructing a three-dimensional model of a target object with a small number of vertices, too many mapping points will appear on the vertices of the reconstructed three-dimensional model of the target object, resulting in model blurring. Therefore, the reconstruction accuracy of the three-dimensional model is poor.
[0059] In an embodiment of the present application, by obtaining the three-dimensional coordinates and vertex similarity predictions of a large number of mapping points, and then optimizing and screening the mapping points based on the vertex similarity, the mapping points with high similarity to the three-dimensional vertices of the target object are retained, and the mapping points with low similarity to the three-dimensional vertices of the target object are deleted. In this way, the obtained target mapping points are closer to the three-dimensional vertices of the target object. Therefore, the three-dimensional model of the target object constructed based on the target mapping points has higher accuracy.
[0060] In one embodiment, Figure 1 As shown, a three-dimensional model construction method is provided. This embodiment uses the method applied to a computer device as an example. The computer device may be a server, a computer, etc. In this embodiment, the method includes the following steps:
[0061] Step 101: A computer device acquires a two-dimensional image.
[0062] The two-dimensional image includes a target object for which a three-dimensional model is to be constructed. Optionally, the two-dimensional image is a single two-dimensional image.
[0063] Step 102: The computer device obtains the three-dimensional coordinates of multiple mapping points of the target object and the vertex similarity prediction value of each mapping point based on the two-dimensional image.
[0064] The vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point. The three-dimensional vertex of the target object refers to the vertex on the three-dimensional model of the target object.
[0065] In an embodiment of the present application, the computer device can obtain a pre-trained three-dimensional feature extraction model, and then the computer device can input a two-dimensional image into the three-dimensional feature extraction model, and obtain the three-dimensional coordinates of multiple mapping points of the target object output by the three-dimensional feature extraction model and the order similarity prediction value of each mapping point.
[0066] Step 103: The computer device selects a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point.
[0067] In the embodiment of the present application, the larger the vertex similarity prediction value of each mapping point, the greater the possibility that the mapping point is a three-dimensional vertex of the target object, and the smaller the vertex similarity prediction value, the smaller the possibility that the mapping point is a three-dimensional vertex of the target object.
[0068] The computer device can screen multiple mapping points according to the size of the vertex similarity prediction value of each mapping point. For example, the computer device can eliminate mapping points whose vertex similarity prediction value is less than or equal to the similarity threshold. The retained mapping points are target mapping points. The target mapping points are mapping points that are closer to the three-dimensional vertices of the target object. Therefore, the three-dimensional model of the target object constructed based on the target mapping points has higher accuracy.
[0069] Step 104: The computer device constructs a three-dimensional model of the target object based on the three-dimensional coordinates of the plurality of target mapping points.
[0070] Optionally, in an embodiment of the present application, the computer device may input a two-dimensional image and three-dimensional coordinates of multiple target mapping points into a mesh model to obtain a three-dimensional model of the target object output by the mesh model.
[0071] The three-dimensional model construction method provided in the embodiment of the present application obtains the three-dimensional coordinates of a large number of mapping points and vertex similarity predictions, and then optimizes and screens the mapping points based on the vertex similarity, retains the mapping points with high similarity to the three-dimensional vertices of the target object, and deletes the mapping points with low similarity to the three-dimensional vertices of the target object. In this way, the obtained target mapping points are closer to the three-dimensional vertices of the target object. Therefore, the three-dimensional model of the target object constructed based on the target mapping points has higher accuracy.
[0072] In one embodiment, Figure 2 As shown, the process of the computer device acquiring the three-dimensional coordinates of multiple mapping points of the target object and the vertex similarity prediction value of each mapping point based on the two-dimensional image includes:
[0073] Step 201: A computer device obtains three-dimensional structural features of a two-dimensional image.
[0074] In the embodiment of the present application, the computer device can obtain a pre-trained encoder, and optionally, the encoder can use a ResNet50 backbone network. Then the computer device can input the two-dimensional image into the trained encoder, and use the trained encoder to extract features from the two-dimensional image to obtain the three-dimensional structural features of the target object in the target image.
[0075] Optionally, the three-dimensional structural feature of the target object may be a feature vector. For example, the three-dimensional structural feature of the target object may be a 1024-dimensional feature vector.
[0076] In step 202, the computer device uniformly collects a target number of initial mapping points in a preset reference plane, and constructs an initial coordinate matrix according to the two-dimensional coordinates of each initial mapping point.
[0077] Among them, each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of an initial mapping point.
[0078] In the embodiment of the present application, the preset reference plane may be a unit plane with an interval of [0, 1).
[0079] The computer device can perform uniform sampling on the reference plane. Optionally, a uniform grid sampling method can be used, that is, the unit plane is divided into N equal parts in the X-axis and Y-axis directions respectively and evenly. Where N is a preset value. For example, N is 1000. In this way, 1000×1000=1000000 initial mapping points can be obtained on the reference plane.
[0080] It should be noted that, in the embodiment of the present application, the number of initial mapping points is pre-set, and the value of N is determined by reverse calculation based on the number of initial mapping points.
[0081] Optionally, in the embodiment of the present application, the number of initial mapping points (target number) is greater than a preset threshold. In this way, when the number of vertices contained in the three-dimensional model of the target object is large, it can be ensured that the vertices of the three-dimensional model of the target object can be fully covered, avoiding model distortion caused by a small number of initial mapping points.
[0082] In the embodiment of the present application, the computer device can represent the coordinates of the initial mapping points obtained by sampling as an initial coordinate matrix, and the initial coordinate matrix can be a matrix of 1,000,000 rows and 2 columns. The first column represents the X-axis coordinate, and the second column represents the Y-axis coordinate.
[0083] Step 203: The computer device splices the three-dimensional structural features into each row of the initial coordinate matrix to obtain a spliced matrix.
[0084] Continuing with the above example, in an embodiment of the present application, the computer device can splice the three-dimensional structural features to each row of the initial coordinate matrix. For example, the computer device can splice the 1024-dimensional feature vector to each row of a matrix of 1000000 rows and 2 columns to obtain a spliced matrix of 1000000 rows and (2+1024) columns.
[0085] In step 204, the computer device inputs the concatenated matrix into the trained decoder, and obtains the three-dimensional coordinates of the target number of mapping points and the vertex similarity prediction value of each mapping point output by the decoder.
[0086] In the embodiment of the present application, the computer device can obtain a trained decoder, and the optional decoder can use MLP (English: Multi-Layer Perceptron, Chinese: Multi-layer Perceptron), where the structure of MLP can be as follows: Figure 3 The computer device can input the concatenated matrix into the decoder to obtain the decoded matrix output by the decoder. Continuing with the above example, the decoded matrix is a matrix of 1,000,000 rows and 4 columns, where the first three columns are the coordinates in the X-axis, Y-axis and Z-axis directions, and the fourth column is the vertex similarity prediction value of the mapping point.
[0087] In the embodiment of the present application, a target number of initial mapping points are obtained to obtain a target number of mapping points. By obtaining a large number of mapping points, it is ensured that when a three-dimensional model of a target object with complex structures and multiple vertices is reconstructed, all vertices of the target object can be covered, thereby ensuring the accuracy of the three-dimensional model of the target object.
[0088] Furthermore, the embodiment of the present application can determine the similarity between each mapping point and the vertex by obtaining the vertex similarity prediction value of each mapping point, thereby facilitating optimized filtering of the mapping points to avoid model blurring and improve the accuracy of the three-dimensional model of the target object.
[0089] In one embodiment, Figure 4 As shown, the process of selecting multiple target mapping points from multiple mapping points by the computer device based on the vertex similarity prediction value of each mapping point includes:
[0090] Step 401: The computer device arranges the mapping points in descending order based on the vertex similarity prediction values of the mapping points, and determines the mapping points as candidate mapping points in descending order.
[0091] In the embodiment of the present application, a non-maximum value method can be used to remove adjacent repeated mapping points for multiple mapping points. Specifically, the computer device can arrange the mapping points in descending order according to the size of the vertex similarity prediction value of each mapping point, and determine each mapping point as a candidate mapping point in descending order.
[0092] For example, mapping point A, mapping point B, and mapping point C are arranged in descending order according to their respective vertex similarity prediction values in the order BCA. Then the computer device can determine B as a candidate mapping point according to the arrangement order. After processing mapping point B, if mapping point C is not eliminated, then mapping point C can be determined as a candidate mapping point. If mapping point C is eliminated after processing mapping point B, then it can be postponed in sequence and mapping point A can be determined as a candidate mapping point.
[0093] Step 402: The computer device calculates the distances from other mapping points to the candidate mapping points, and removes mapping points whose distances to the candidate mapping points are less than a distance threshold, to obtain a plurality of target mapping points.
[0094] Taking the processing of mapping point B as an example, the computer device can calculate the distance from the candidate mapping point to any other mapping point based on the three-dimensional coordinates of the candidate mapping point and the three-dimensional coordinates of other mapping points, and then eliminate the mapping points whose distance to the candidate mapping point is less than the distance threshold.
[0095] Optionally, the distance threshold may be 0.01.
[0096] In the embodiment of the present application, each mapping point that has not been eliminated is determined as a candidate mapping point in turn, and then the elimination process is performed based on the candidate mapping points through the content disclosed in step 402, until the distance between any two mapping points in the remaining mapping points is greater than or equal to the distance threshold, and the remaining mapping points are the target mapping points.
[0097] Optionally, in the embodiment of the present application, after removing duplicate mapping points, the computer device may further perform the following operations: the computer device may remove mapping points whose vertex similarity prediction values are less than a similarity threshold from the retained multiple mapping points.
[0098] In the embodiment of the present application, the mapping points whose distance to the candidate mapping points is less than the distance threshold correspond to the same three-dimensional vertex as the candidate mapping points. The embodiment of the present application eliminates the mapping points whose distance to the candidate mapping points is less than the distance threshold so that each three-dimensional vertex corresponds to one mapping point, thereby avoiding the aggregation of a large number of mapping points near a three-dimensional vertex and causing the model to become blurred.
[0099] Furthermore, by arranging the vertex similarity prediction values of each mapping point in descending order, the purpose of retaining the mapping point with the highest similarity to the three-dimensional vertex can be achieved, thereby improving the accuracy of the three-dimensional model.
[0100] In one embodiment, Figure 5 As shown, the process of selecting multiple target mapping points from multiple mapping points based on the vertex similarity prediction value of each mapping point by the computer device may also include the following contents:
[0101] Step 501: The computer device performs decentralized processing on the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after the decentralized processing.
[0102] Optionally, in the embodiment of the present application, the process of the computer device performing decentralized processing on each mapping point may include the following:
[0103] In step A1, a computer device obtains the average coordinate value of each coordinate axis in the three-dimensional coordinates of a plurality of mapping points.
[0104] The computer device may calculate the mean values Xmean, Ymean, and Zmean of the first three columns in the matrix after decoding in step 204 .
[0105] That is, the computer device can average 1,000,000 X-axis coordinates to obtain the X-axis coordinate average value Xmean, average 1,000,000 Y-axis coordinates to obtain the Y-axis coordinate average value Ymean, and average 1,000,000 Z-axis coordinates to obtain the Z-axis coordinate average value Zmean.
[0106] Step A2: The computer device subtracts the average coordinate value of the corresponding coordinate axis from the three-dimensional coordinate of each mapping point to obtain a first mapping point after decentralized processing.
[0107] The computer device can subtract the respective coordinate average from the three-dimensional coordinate of each mapping point to obtain the decentralized three-dimensional coordinate of each mapping point. For ease of description, in the embodiment of the present application, the mapping point from which the coordinate average of the corresponding coordinate axis is subtracted is referred to as the first mapping point.
[0108] Step 502: The computer device normalizes each first mapping point to obtain a plurality of second mapping points.
[0109] In the embodiment itself, the computer device may perform normalization processing on the multiple first mapping points after decentralization, specifically:
[0110] Step B1: The computer device obtains the maximum distance from each first mapping point to the origin.
[0111] In the embodiment of the present application, the computer device can calculate the Euclidean distance from each first mapping point to the origin. For example, for the first mapping point i, the Euclidean distance from the origin is Di=sqrt(Xi×Xi+Yi×Yi+Zi×Zi), where sqrt is a square root operation. The computer device can determine the maximum distance Dmax from multiple distances.
[0112] Step B2: For each first mapping point, the computer device divides the three-dimensional coordinates of the first mapping point by the maximum distance to obtain a second mapping point.
[0113] In the embodiment of the present application, the computer device may divide each coordinate value in the three-dimensional coordinates of each first mapping point by the maximum distance to obtain a new three-dimensional coordinate of the first mapping point.
[0114] For the convenience of description, the normalized first mapping point is referred to as the second mapping point.
[0115] Step 503: The computer device selects a plurality of target mapping points from the plurality of second mapping points based on the vertex similarity prediction value of each second mapping point.
[0116] In the embodiment of the present application, the computer device may refer to the contents disclosed in step 401 and step 402 to filter out multiple target mapping points from multiple second mapping points according to the vertex similarity prediction values of each second mapping point.
[0117] In an embodiment of the present application, by performing decentralized and normalized processing on the three-dimensional coordinates of multiple mapping points, and then screening out multiple target mapping points based on the multiple second mapping points after the decentralized and normalized processing, the process of optimizing the mapping points is implemented, thereby improving the accuracy of the three-dimensional model of the target object.
[0118] It should be understood that although Figure 1-Figure 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-Figure 5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.
[0119] In one embodiment, Figure 6As shown, a three-dimensional model construction device is provided, including: an image acquisition module 601, a mapping point acquisition module 602, a screening module 603 and a model construction module 604, wherein:
[0120] An image acquisition module 601 is used to acquire a two-dimensional image, where the two-dimensional image includes a target object to be constructed into a three-dimensional model;
[0121] A mapping point acquisition module 602 is used to acquire three-dimensional coordinates of multiple mapping points of the target object and a vertex similarity prediction value of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point;
[0122] A screening module 603 is used to screen out a plurality of target mapping points from a plurality of mapping points based on the vertex similarity prediction value of each mapping point;
[0123] The model building module 604 is used to build a three-dimensional model of the target object based on the three-dimensional coordinates of multiple target mapping points.
[0124] In one embodiment of the present application, the screening module 603 is also used to arrange each mapping point in descending order based on the vertex similarity prediction value of each mapping point, and determine each mapping point as a candidate mapping point in descending order; calculate the distance from other mapping points to the candidate mapping points, and eliminate mapping points whose distance to the candidate mapping points is less than a distance threshold, to obtain multiple target mapping points.
[0125] In one embodiment of the present application, the screening module 603 is further configured to remove mapping points whose predicted vertex similarity values are less than a similarity threshold from the plurality of mapping points after removing mapping points whose distances to candidate mapping points are less than a distance threshold.
[0126] In one embodiment of the present application, the screening module 603 is also used to perform decentralized processing on the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after decentralized processing; perform normalization processing on each first mapping point to obtain a plurality of second mapping points; and screen out a plurality of target mapping points from the plurality of second mapping points based on the vertex similarity prediction value of each second mapping point.
[0127] In one embodiment of the present application, the screening module 603 is also used to obtain the coordinate average of each coordinate axis in the three-dimensional coordinates of multiple mapping points; for each mapping point, the three-dimensional coordinates of the mapping point are subtracted from the coordinate average of the corresponding coordinate axis to obtain the first mapping point after decentralized processing.
[0128] In one embodiment of the present application, the screening module 603 is further used to obtain the maximum distance from each first mapping point to the origin; for each first mapping point, the three-dimensional coordinates of the first mapping point are divided by the maximum distance to obtain a second mapping point.
[0129] In one embodiment of the present application, the mapping point acquisition module 602 is also used to obtain the three-dimensional structural features of the two-dimensional image; uniformly collect a target number of initial mapping points in a preset reference plane, and construct an initial coordinate matrix based on the two-dimensional coordinates of each initial mapping point, where each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of an initial mapping point; after splicing the three-dimensional structural features to each row of the initial coordinate matrix, a spliced matrix is obtained; the spliced matrix is input into a trained decoder, and the three-dimensional coordinates of the target number of mapping points and the vertex similarity prediction value of each mapping point output by the decoder are obtained.
[0130] For the specific definition of the three-dimensional model construction device, please refer to the definition of the three-dimensional model construction method above, which will not be repeated here. Each module in the above-mentioned three-dimensional model construction device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0131] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pre-trained encoders and decoders. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a three-dimensional model construction method is implemented.
[0132] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0133] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0134] Acquire a two-dimensional image, where the two-dimensional image includes a target object for which a three-dimensional model is to be constructed;
[0135] Acquire three-dimensional coordinates of multiple mapping points of the target object and vertex similarity prediction values of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point;
[0136] Based on the vertex similarity prediction value of each mapping point, multiple target mapping points are selected from multiple mapping points;
[0137] A 3D model of the target object is constructed based on the 3D coordinates of multiple target mapping points.
[0138] In one embodiment, when the processor executes the computer program, the following steps are also implemented: arranging each mapping point in descending order based on the vertex similarity prediction value of each mapping point, and determining each mapping point as a candidate mapping point in descending order; calculating the distance from other mapping points to the candidate mapping point, eliminating the mapping points whose distance to the candidate mapping point is less than the distance threshold, and obtaining multiple target mapping points.
[0139] In one embodiment, when the processor executes the computer program, the following steps are further implemented: mapping points whose vertex similarity prediction values are less than a similarity threshold are eliminated from the plurality of mapping points.
[0140] In one embodiment, when the processor executes the computer program, the following steps are also implemented: decentralizing the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after decentralization; normalizing each first mapping point to obtain a plurality of second mapping points; and selecting a plurality of target mapping points from the plurality of second mapping points based on the vertex similarity prediction value of each second mapping point.
[0141] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the coordinate average value of each coordinate axis in the three-dimensional coordinates of multiple mapping points; for each mapping point, subtracting the coordinate average value of the corresponding coordinate axis from the three-dimensional coordinate of the mapping point to obtain the first mapping point after decentralized processing.
[0142] In one embodiment, when the processor executes the computer program, the following steps are further implemented: obtaining the maximum distance from each first mapping point to the origin; for each first mapping point, dividing the three-dimensional coordinates of the first mapping point by the maximum distance to obtain a second mapping point.
[0143] In one embodiment, the processor further implements the following steps when executing the computer program: obtaining three-dimensional structural features of the two-dimensional image; uniformly collecting a target number of initial mapping points in a preset reference plane, and constructing an initial coordinate matrix based on the two-dimensional coordinates of each initial mapping point, wherein each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of an initial mapping point; after splicing the three-dimensional structural features to each row of the initial coordinate matrix, a spliced matrix is obtained; the spliced matrix is input into a trained decoder, and the three-dimensional coordinates of the target number of mapping points and the vertex similarity prediction value of each mapping point output by the decoder are obtained.
[0144] The computer device provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.
[0145] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0146] Acquire a two-dimensional image, where the two-dimensional image includes a target object for which a three-dimensional model is to be constructed;
[0147] Acquire three-dimensional coordinates of multiple mapping points of the target object and vertex similarity prediction values of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point;
[0148] Based on the vertex similarity prediction value of each mapping point, multiple target mapping points are selected from multiple mapping points;
[0149] A 3D model of the target object is constructed based on the 3D coordinates of multiple target mapping points.
[0150] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: each mapping point is arranged in descending order based on the vertex similarity prediction value of each mapping point, and each mapping point is determined as a candidate mapping point in descending order; the distance from other mapping points to the candidate mapping point is calculated, and the mapping points whose distance to the candidate mapping point is less than the distance threshold are eliminated to obtain multiple target mapping points.
[0151] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: mapping points whose predicted vertex similarity values are less than a similarity threshold are eliminated from the plurality of mapping points.
[0152] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: decentralizing the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after decentralization; normalizing each first mapping point to obtain a plurality of second mapping points; and selecting a plurality of target mapping points from the plurality of second mapping points based on the vertex similarity prediction value of each second mapping point.
[0153] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the coordinate average value of each coordinate axis in the three-dimensional coordinates of multiple mapping points; for each mapping point, subtracting the coordinate average value of the corresponding coordinate axis from the three-dimensional coordinate of the mapping point to obtain the first mapping point after decentralized processing.
[0154] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the maximum distance from each first mapping point to the origin; for each first mapping point, dividing the three-dimensional coordinates of the first mapping point by the maximum distance to obtain a second mapping point.
[0155] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining three-dimensional structural features of a two-dimensional image; uniformly collecting a target number of initial mapping points in a preset reference plane, and constructing an initial coordinate matrix based on the two-dimensional coordinates of each initial mapping point, wherein each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of an initial mapping point; after splicing the three-dimensional structural features to each row of the initial coordinate matrix, a spliced matrix is obtained; the spliced matrix is input into a trained decoder, and the three-dimensional coordinates of the target number of mapping points and the vertex similarity prediction value of each mapping point output by the decoder are obtained.
[0156] The computer-readable storage medium provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.
[0157] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0158] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0159] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A three-dimensional model construction method, characterized in that: The method comprises: Acquire a two-dimensional image, wherein the two-dimensional image includes a target object for which a three-dimensional model is to be constructed; Acquire three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity prediction value of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents a similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point; Filtering out a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each of the mapping points; constructing a three-dimensional model of the target object based on the three-dimensional coordinates of the plurality of target mapping points; The step of selecting a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each of the mapping points comprises: Arrange the mapping points in descending order based on the vertex similarity prediction values of the mapping points, and determine the mapping points as candidate mapping points in descending order; Calculating the distances from other mapping points to the candidate mapping points, and eliminating mapping points whose distances to the candidate mapping points are less than a distance threshold, to obtain the multiple target mapping points; Eliminate mapping points whose vertex similarity prediction values are less than a similarity threshold from the multiple mapping points; The step of acquiring the three-dimensional coordinates of a plurality of mapping points of the target object and the vertex similarity prediction value of each of the mapping points based on the two-dimensional image comprises: Acquiring three-dimensional structural features of the two-dimensional image; Uniformly collecting a target number of initial mapping points in a preset reference plane, and constructing an initial coordinate matrix according to the two-dimensional coordinates of each of the initial mapping points, wherein each row of the initial coordinate matrix corresponds to the two-dimensional coordinate of an initial mapping point; After splicing the three-dimensional structural features to each row of the initial coordinate matrix, a spliced matrix is obtained; The concatenated matrix is input into a trained decoder, and the three-dimensional coordinates of the target number of mapping points and the vertex similarity prediction value of each mapping point output by the decoder are obtained.
2. The method according to claim 1, characterized in that Before selecting a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction values of the mapping points, the method further comprises: Decentralizing the three-dimensional coordinates of each of the mapping points to obtain a plurality of first mapping points after the decentralized processing; Normalizing each of the first mapping points to obtain a plurality of second mapping points; The step of selecting a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each of the mapping points comprises: A plurality of target mapping points are screened out from the plurality of second mapping points based on the vertex similarity prediction value of each of the second mapping points.
3. The method according to claim 2, characterized in that The decentralizing the three-dimensional coordinates of each of the mapping points to obtain a plurality of first mapping points after the decentralization includes: Obtaining the average value of the coordinates of each dimensional coordinate axis in the three-dimensional coordinates of the plurality of mapping points; For each of the mapping points, the average coordinate value of the corresponding coordinate axis is subtracted from the three-dimensional coordinate of the mapping point to obtain the first mapping point after decentralized processing.
4. The method according to claim 2, characterized in that: The normalizing of each of the first mapping points to obtain a plurality of second mapping points includes: Obtaining the maximum distance between each of the first mapping points and the origin; For each of the first mapping points, the three-dimensional coordinates of the first mapping point are divided by the maximum distance to obtain the second mapping point.
5. A three-dimensional model construction device, characterized in that: The device comprises: An image acquisition module, used to acquire a two-dimensional image, wherein the two-dimensional image includes a target object for which a three-dimensional model is to be constructed; A mapping point acquisition module, used for acquiring three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity prediction value of each mapping point based on the two-dimensional image, wherein the vertex similarity prediction value of each mapping point represents a similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point; A screening module, configured to screen out a plurality of target mapping points from the plurality of mapping points based on a vertex similarity prediction value of each of the mapping points; A model building module, used for building a three-dimensional model of the target object based on the three-dimensional coordinates of the multiple target mapping points; The screening module is further used to arrange the mapping points in descending order based on the predicted vertex similarity values of the mapping points, and determine the mapping points as candidate mapping points in descending order; calculate the distances from other mapping points to the candidate mapping points, and eliminate the mapping points whose distances to the candidate mapping points are less than a distance threshold, to obtain multiple target mapping points; and eliminate the mapping points whose predicted vertex similarity values are less than the similarity threshold from the multiple mapping points; The mapping point acquisition module is also used to obtain the three-dimensional structural features of the two-dimensional image; uniformly collect a target number of initial mapping points in a preset reference plane, and construct an initial coordinate matrix according to the two-dimensional coordinates of each initial mapping point, each row of the initial coordinate matrix corresponds to the two-dimensional coordinate of an initial mapping point; after splicing the three-dimensional structural features to each row of the initial coordinate matrix, a spliced matrix is obtained; the spliced matrix is input into a trained decoder, and the three-dimensional coordinates of the target number of mapping points and the vertex similarity prediction value of each mapping point output by the decoder are obtained.
6. The device according to claim 5, characterized in that The screening module is further used to perform decentralized processing on the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after decentralized processing; perform normalization processing on each first mapping point to obtain a plurality of second mapping points; and screen out a plurality of target mapping points from the plurality of second mapping points based on the vertex similarity prediction value of each second mapping point.
7. The device according to claim 6, characterized in that The screening module is further used to obtain the coordinate average value of each coordinate axis in the three-dimensional coordinates of multiple mapping points; for each mapping point, the three-dimensional coordinates of the mapping point are respectively subtracted from the coordinate average value of the corresponding coordinate axis to obtain a first mapping point after decentralized processing.
8. The device according to claim 6, characterized in that The screening module is further used to obtain the maximum distance from each first mapping point to the origin; for each first mapping point, the three-dimensional coordinates of the first mapping point are divided by the maximum distance to obtain a second mapping point.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.