A plant image three-dimensional reconstruction dynamic optimization method and system based on deep learning

Abstract

The present application relates to the technical field of artificial intelligence building design, and particularly relates to a plant image three-dimensional reconstruction dynamic optimization method and system based on deep learning.The method comprises the following steps: (1) data preparation and preprocessing; (2) constructing a random feature learner model; (3) image classification and identification of active progressive learning; (4) plant data consistency feature matching; (5) image classification and feature point matching dynamic optimization; (6) three-dimensional image reconstruction.Compared with the prior art, the present application improves the accuracy and efficiency of three-dimensional reconstruction, and provides more reliable technical support for three-dimensional modeling and visualization of the plant.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence building design, and particularly relates to a plant image three-dimensional reconstruction dynamic optimization method and system based on deep learning. BACKGROUND

[0002] In the construction industry, especially in plant construction and maintenance, three-dimensional modeling technology plays a crucial role. With the rapid development of image recognition technology, plant three-dimensional modeling technology based on image recognition has become a research hotspot in the industry. However, although this technology has made some progress, existing plant three-dimensional modeling technology based on image recognition still has many shortcomings and challenges.

[0003] First, the feature extraction and matching algorithm has shortcomings. In plant three-dimensional modeling, feature extraction and matching are key steps. However, existing feature extraction algorithms mostly rely on fixed templates or rules, making it difficult to adapt to the complexity and diversity of different plant environments. At the same time, the matching algorithm also faces problems such as large amount of calculation and low matching accuracy, resulting in a large number of false matches and misjudgments in the reconstruction results.

[0004] In addition, existing three-dimensional reconstruction methods lack dynamic optimization mechanisms. In the process of plant three-dimensional modeling, due to differences in image data, device performance and other factors, the reconstruction results often have large uncertainties. Existing methods mostly use fixed reconstruction processes and parameter settings, lacking the ability to dynamically adjust and optimize according to actual conditions. This leads to difficulties in ensuring the accuracy and efficiency of the reconstruction results when facing different plant environments.

[0005] In summary, existing plant three-dimensional modeling technology based on image recognition still has many shortcomings and challenges. In order to overcome these shortcomings, the present application proposes a plant image three-dimensional reconstruction dynamic optimization method and system based on deep learning, aiming to improve the accuracy of image data preprocessing, feature extraction and matching through deep learning technology, and introduce a dynamic optimization mechanism to realize accurate, efficient and adaptive plant three-dimensional modeling. SUMMARY

[0006] The purpose of the present application is to provide a plant image three-dimensional reconstruction dynamic optimization method based on deep learning with higher accuracy and better precision.

[0007] The purpose of the present application is achieved as follows:

[0008] A plant image three-dimensional reconstruction dynamic optimization method based on deep learning, comprising the following steps:

[0009] (1) Data preparation and preprocessing: collect multi-angle image data of the factory building, ensure that the images cover all important areas of the factory building; preprocess the collected image data, including removing noise, enhancing image contrast, and correcting image color, to obtain preprocessed factory building images;

[0010] (2) Constructing a random feature learner model: initialize the input weight matrix according to different specific distributions, and impose constraints on the input weight matrix; assign corresponding attention weights according to the importance of each base learner; construct the base learner, including creating input weight matrices following normal and uniform distributions, constructing hidden layer outputs, and obtaining output weight matrices through pseudo-inverse learning, to obtain the base learner output; construct the meta-learner, calculate the similarity between the label matrix and the base learner network output to obtain the meta-learner input weight matrix, and perform single-hidden layer stack pseudo-inverse learning to obtain the meta-learner network output;

[0011] (3) Active progressive learning image classification and recognition: randomly average the preprocessed factory building images into a training set and a candidate set; train the base learner and the meta-learner using the training set by constructing the base learner and the meta-learner; rejoin the factory building image samples in the candidate set to the training set by iterative selection, and perform sample selection and class reassignment according to the meta-learner output; calculate the fuzziness of the candidate samples, construct a fuzziness collection, and perform iterative classification until the iteration ends, classify the images of different factory building structures, and obtain the final factory building image classification result;

[0012] (4) Factory data consistency feature matching: for adjacent images of the same factory building in the factory image classification result, use the ORB algorithm to detect and extract feature points and their descriptors; calculate the total number of matching pairs and the neighborhood value of each feature point; distinguish correct matching pairs and incorrect matching pairs according to the support, remove high-similarity repetitive texture incorrect matching; implement the random sample consensus algorithm on the correct matching set to remove outliers; perform guided matching and motion consistency judgment within the threshold on the matching set, and retain matching pairs that meet the conditions;

[0013] (5) Image classification and feature point matching dynamic optimization: calculate the number of correct and incorrect divisions in image classification, calculate the classification accuracy, classification precision, and feature matching precision; dynamically adjust the parameters according to the classification accuracy, classification precision, and feature matching precision until these indicators are controlled within the preset threshold range, and obtain the optimized matching pairs that meet the conditions;

[0014] (6) 3D image reconstruction: Using the optimized matching pairs that meet the conditions, calculate the position of each feature point in 3D space by triangulation; combine all feature point positions into a point cloud; starting from the point cloud, use a surface reconstruction algorithm to generate a 3D surface model of the factory; map the texture information in the original image onto the 3D surface and extract color information to fill the 3D surface; output the reconstructed 3D image of the factory in an appropriate file format.

[0015] The construction of the random feature learner model specifically includes:

[0016] (2.1) Constructing the base learner: Create different input weight matrices that follow normal and uniform distributions, construct the hidden layer output, obtain the output weight matrix through pseudo-inverse learning, and obtain the base learner output;

[0017] (2.1.1) Create i distinct input weight matrices K that follow a specific distribution. i K i Given any distribution; set i=3, generate 3 different specific distributions of K. i ;

[0018] K1 follows a normal distribution with expected value μ1 and standard deviation δ1:

[0019]

[0020] K2 follows a uniform distribution over the interval [p, q].

[0021] K2~U(p,q);

[0022] K3 follows a normal distribution with an expected value of μ2 and a standard deviation of δ2.

[0023]

[0024] Where U represents a uniform distribution and N represents a normal distribution;

[0025] (2.1.2) Construct the hidden layer output U i :

[0026] U i =ξ(L×K i );

[0027] Where L is the input matrix, which in this invention is the input image data, and ξ is the activation function;

[0028] (2.1.3) For the input matrix L, collect the labels of each image, construct the label matrix G, and obtain the output weight matrix through pseudo-inverse learning:

[0029]

[0030] (2.1.4) Obtain the base learner network output:

[0031] M i = O i × U i ;

[0032] (2.2) Constructing meta-learner: obtaining the meta-learner input weight matrix by calculating the similarity between the label matrix and the base learner network output, performing single hidden layer stack pseudo-inverse learning according to the meta-learner input weight matrix and the base learner network output to obtain the stacked hidden layer output, and obtaining the meta-learner network output through the meta-learner output weight matrix after stacking;

[0033] (2.2.1) Calculate the similarity between the label matrix G and the base learner network output M i :

[0034]

[0035] (2.2.2) Obtain the meta-learner input weight matrix through the logical normalization function:

[0036]

[0037] (2.2.3) Perform single hidden layer stack pseudo-inverse learning to obtain the stacked hidden layer output according to the meta-learner input weight matrix and the base learner network output:

[0038]

[0039] (2.2.4) Obtain the meta-learner output weight matrix after stacking according to the pseudo-inverse learning:

[0040]

[0041] (2.2.5) Obtain the meta-learner network output:

[0042] M f = U f × O f .

[0043] The active progressive learning image classification recognition specifically comprises:

[0044] (3.1) Randomly average the pre-processed factory images as training samples into a training set L tr and a candidate set L ca , train the base learner using (2.1.1)-(2.1.3) through the training set L tr to obtain the initialization weight matrix O i-tr , train Lca get the meta-learner output M f-ca ; select M f-ca ; select L s factory building image samples to the training set L tr , and get the meta-learner output M f-ca again.

[0045] (3.2) The meta-learner output of P classes and T candidate samples Select the index value y corresponding to the maximum value of each column in M , and randomize M :

[0046]

[0047] rand(P, 1) is a random vector with P rows and 1 column;

[0048] (3.3) Reassign the yth class of each factory building selection set to obtain the class edge probability distribution matrix:

[0049]

[0050] i = 1, 2, …, P, P represents the number of factory construction classes, j = 1, 2, …, T, T represents the number of candidate samples, represents the ith row and jth column of M f-ca ;

[0051] (3.4) Calculate the fuzziness of P classes and T candidate samples:

[0052]

[0053] (3.5) Construct the fuzziness collection I:

[0054] I = ]R(M f-ca ), M f-ca , G f-ca ];

[0055] G f-ca is the actual class label of M f-ca ; The actual class label refers to the label based on the actual situation, which is used to classify the graph;

[0056] (3.6) Iterative classification: Sort I in descending order according to R(M f-ca ), if M f-ca ≠ G f-ca , then it is an error classification sample, select the first s error classification samples, and add them to L trIn the middle, re-construct the training set and candidate set, repeat the training z times, until the end of the iteration, get the final factory image classification results.

[0057] The factory data consistency feature matching specifically includes:

[0058] (4.1) For the same three-dimensional scene adjacent image of a factory, the ORB algorithm is used to detect and extract the feature points and their corresponding descriptors in the image, to form a matching set, and each feature point is traversed to find all feature points in the neighborhood radius r and form a matching pair, the neighborhood is the circumscribed circle region of the basic square cell formed by the feature point as the center;

[0059] (4.2) Calculate the total number of matching pairs C j in each neighborhood:

[0060] F mn = |C j |-I;

[0061] -I represents the delete original matching pair operation, (m j ,n j ) represents each matching pair;

[0062] (4.3) The matching pairs are distributed according to the support degree, and the high-similarity repeated texture error matching is removed:

[0063]

[0064] β is the lower threshold parameter of the matching, γ is the upper threshold parameter of the matching, Z is the total number of features in the neighborhood, RI represents that the matching pair is a correct matching pair, E represents that the matching pair is an error matching pair, and RE represents a repeated matching pair;

[0065] The final correct matching set F(RI) is obtained;

[0066] (4.4) Initial matching optimization:

[0067] The random sample consensus algorithm is implemented on the correct matching set F(RI) to remove outliers and retain all matching pairs that meet the conditions:

[0068] DMS=RANSAC[F(RI)];

[0069] RANSAC is a random sample consensus algorithm;

[0070] The basic matrix is fitted for DMS:

[0071] S1=s(DMS);

[0072] S represents the fitting operation;

[0073] (4.5) Guided matching is performed on each feature point for the matching set:

[0074] GuMS = |s(F mn , S1) | < g;

[0075] s() represents distance, and g is distance parameter;

[0076] The feature points satisfying the guided matching condition GuMS are reserved;

[0077] (4.6) Motion consistency within threshold β is determined, and the error matching of point-to-line logic is deleted, and the matching pair after determination is reserved:

[0078]

[0079] ← is calculated for verification.

[0080] The image classification and feature point matching dynamic optimization specifically comprises:

[0081] (5.1) The number of correctly classified positive examples TP, the number of correctly classified negative examples TN, the number of incorrectly classified positive examples FP, and the number of incorrectly classified negative examples FN in image classification are calculated

[0082] (5.2) The classification accuracy Acc is calculated;

[0083]

[0084] (5.3) The classification precision Cr is calculated;

[0085]

[0086] (5.4) The feature matching precision PMS is calculated;

[0087]

[0088] TMS is the number of correct matches, and FMS is the number of error matches;

[0089] (5.5) Parameter dynamic adjustment:

[0090] When Acc < ε or Cr < ∈, re-execute step (3.1), re-perform random average distribution of training samples and execute subsequent steps; ε is the classification accuracy threshold, ∈ is the classification precision threshold; until the classification accuracy and classification precision are controlled within the threshold range;

[0091] When PMS < θ, increase the value of γ, re-execute step (4.3) and execute subsequent steps, until the feature matching precision is controlled within the threshold range.

[0092] The three-dimensional image reconstruction specifically comprises:

[0093] (6.1) Using the matched pairs after discrimination, the position of each feature point in the three-dimensional space is calculated by a triangulation method; the triangulation is based on the matching points in two or more perspectives, the positions of the points in the camera coordinate system are calculated, and then the coordinates of the points in the three-dimensional space are determined;

[0094] (6.2) All the feature point positions obtained by triangulation are combined to form a point cloud; the point cloud is a collection of a series of discrete points in the three-dimensional space, representing the shape and structure of the surface of the factory building and its internal facilities.

[0095] (6.3) Starting from the point cloud, a surface reconstruction algorithm is used to generate a three-dimensional surface model of the factory building;

[0096] (6.4) The texture information in the original image is mapped to the reconstructed three-dimensional surface, and the color information of the point is extracted to fill the three-dimensional surface;

[0097] (6.5) The reconstructed three-dimensional image of the factory building is output in a proper file format.

[0098] The purpose of the present application is also to provide a deep learning-based dynamic optimization system for three-dimensional reconstruction of factory building images, comprising:

[0099] A data preparation and preprocessing module is used to collect multi-angle image data of the factory building and ensure that the images cover all important parts of the factory building; the images are preprocessed, including denoising, contrast enhancement and color correction;

[0100] The random feature learner model construction module includes a base learner construction unit and a meta learner construction unit, wherein the base learner construction unit is used to create an input weight matrix that follows different specific distributions, construct an implicit layer output, and obtain an output weight matrix through pseudo-inverse learning, thereby obtaining the base learner output; the meta learner construction unit obtains the meta learner input weight matrix by calculating the similarity of the label matrix and the base learner network output, performs single hidden layer stack pseudo-inverse learning to obtain the hidden layer output after stacking, and obtains the meta learner network output through the meta learner output weight matrix after stacking;

[0101] An image classification and recognition module: the preprocessed factory building images are randomly and evenly divided into a training set and a candidate set as training samples; the base learner and the meta learner are trained using the training set; sample selection is performed, and the selected factory building image samples are re-added to the training set for iteration until the iteration end condition is met, and the final factory building image classification result is obtained;

[0102] The feature matching module: the ORB algorithm is used to detect and extract feature points and their corresponding descriptors in the adjacent images of the three-dimensional scene of the same factory building; each feature point is traversed, matching pairs are constructed, and the total number of matching pairs in the neighborhood and the neighborhood value are calculated; correct matching pairs and incorrect matching pairs are distinguished according to the support, and high-similarity repeated texture incorrect matching is removed; the random sample consensus algorithm is implemented on the correct matching set to remove outliers, guided matching and motion consistency judgment are performed, and the matching pairs after judgment are reserved;

[0103] The dynamic optimization module: the number of correct classification, negative example and incorrect classification in image classification is calculated; the classification accuracy, classification precision and feature matching precision are calculated; the parameters are dynamically adjusted according to the classification accuracy, classification precision and feature matching precision until the control is within the set threshold range;

[0104] The three-dimensional image reconstruction module: using the matching pairs after judgment, the position of each feature point in the three-dimensional space is calculated by the triangulation method; all the feature point positions obtained by triangulation are combined into a point cloud; the three-dimensional surface model of the factory building is generated by using the surface reconstruction algorithm; the texture information in the original image is mapped to the reconstructed three-dimensional surface, and the color information is extracted to fill the three-dimensional surface; the reconstructed three-dimensional image of the factory building is output in a proper file format.

[0105] The system adopts the above-mentioned factory image three-dimensional reconstruction dynamic optimization method based on deep learning to perform the factory image three-dimensional reconstruction task.

[0106] The beneficial effects of the present application are:

[0107] (1) The prior art often relies on fixed weight distribution when constructing a feature learner, and cannot flexibly adapt to different data characteristics. The importance difference between base learners is ignored, which limits the accuracy and generalization ability of the classification model. The present application introduces a random feature learner model, initializes the input weight matrix through different specific distributions, and applies constraints to the input weight matrix, improving the flexibility and adaptability of the model. According to the importance degree of each base learner, the corresponding attention weight is allocated, and the new distribution formed by stacking processing can more flexibly adapt to the data characteristics, significantly improving the accuracy and generalization ability of the classification model.

[0108] (2) The prior art often uses fixed training set and test set in image classification and recognition, lacking dynamic adjustment and optimization ability. For the misclassified samples, the prior art often lacks effective processing mechanism, resulting in inaccurate classification results. The present application uses active incremental learning method, continuously iterates and optimizes the training set and candidate set, improves the dynamic adjustment and optimization ability of the classification model. The fuzzy degree calculation and iterative classification mechanism are introduced, and the misclassified samples are effectively processed and redistributed, improving the accuracy of the classification results.

[0109] (3) The prior art is often affected by high-similarity repetitive texture and other false matches in feature matching, resulting in inaccurate matching results. Lack of effective initial matching optimization and guided matching mechanism, resulting in limited matching efficiency and accuracy. The present application uses ORB algorithm to detect and extract feature points and their descriptors, effectively removes high-similarity repetitive texture false matches by calculating the total number of matching pairs in the neighborhood and the neighborhood value. The initial matching optimization and guided matching mechanism are introduced, the outliers are removed by the random sample consensus algorithm, the matching pairs that meet the conditions are retained, and the feature points that meet the guided matching conditions are retained, improving the matching efficiency and accuracy.

[0110] (4) The prior art lacks dynamic optimization ability in image classification and feature point matching, and cannot adjust parameters according to classification accuracy and feature matching accuracy. The present application dynamically evaluates and optimizes image classification and feature point matching by calculating classification accuracy, classification precision and feature matching accuracy. According to the evaluation results, the parameters are dynamically adjusted, and the related steps are re-executed until the classification accuracy and classification precision are controlled within the threshold range, and the feature matching accuracy is also effectively improved.

[0111] In summary, the present application proposes a dynamic optimization method for factory image three-dimensional reconstruction based on deep learning, which has achieved significant beneficial effects in data preparation and preprocessing, constructing random feature learner model, active incremental learning image classification and recognition, factory data consistency feature matching, image classification and feature point matching dynamic optimization, and three-dimensional image reconstruction. Compared with the prior art, the present application improves the accuracy and efficiency of three-dimensional reconstruction, and provides more reliable technical support for three-dimensional modeling and visualization of factory. BRIEF DESCRIPTION OF DRAWINGS

[0112] Figure 1 The present application is a schematic diagram of the method.

[0113] Figure 2 The present application is a schematic diagram of the system.

[0114] Figure 3 The present application is a schematic diagram of feature point matching. DETAILED DESCRIPTION

[0115] The application will be further described below with reference to the accompanying drawings.

[0116] As Figure 1 shown, a plant image three-dimensional reconstruction dynamic optimization method based on deep learning comprises the following steps:

[0117] (1) Data preparation and preprocessing: collect multi-angle image data of the plant, ensure that the images cover all important parts of the plant, and preprocess the images, including denoising, contrast enhancement, and color correction, to improve the accuracy of subsequent processing;

[0118] (2) Construct a random feature learner model: initialize the input weight matrix according to different specific distributions to constrain the input weight matrix; and assign corresponding attention weights to the base learners according to their importance. The new distribution formed after stacking processing can more flexibly adapt to the data characteristics, thereby significantly improving the accuracy and generalization ability of the classification model.

[0119] (2.1) Construct a base learner: create input weight matrices that are different from each other and follow normal distribution and uniform distribution, construct hidden layer output, obtain output weight matrix through pseudo-inverse learning, and obtain base learner output;

[0120] (2.1.1) Create i different input weight matrices K i , K i is any distribution; set i=3 to generate three different K i ;

[0121] K1 is a normal distribution with an expected value of μ1 and a standard deviation of δ1:

[0122]

[0123] K2 is a uniform distribution with an interval of [p, q]:

[0124] K2~U(p, q);

[0125] K3 is a normal distribution with an expected value of μ2 and a standard deviation of δ2:

[0126]

[0127] where U is a uniform distribution and N is a normal distribution;

[0128] (2.1.2) Construct the hidden layer output U i :

[0129] U i = ξ(L×K i );

[0130] wherein L is an input matrix, in the present application, the input image data, ξ is an activation function;

[0131] (2.1.3) For the input matrix L, the label of each image is collected to construct the label matrix G, and the output weight matrix is obtained by pseudo-inverse learning:

[0132]

[0133] (2.1.4) Obtain the base learner network output:

[0134] M i =O i ×U i ;

[0135] (2.2) Constructing meta-learner: obtaining the meta-learner input weight matrix by calculating the similarity between the label matrix and the base learner network output, and performing single hidden layer stack pseudo-inverse learning according to the meta-learner input weight matrix and the base learner network output to obtain the stacked hidden layer output, and obtaining the meta-learner network output through the meta-learner output weight matrix after stacking;

[0136] (2.2.1) Calculate the similarity between the label matrix G and the base learner network output M i

[0137]

[0138] (2.2.2) Obtain the meta-learner input weight matrix through the logical normalization function:

[0139]

[0140] (2.2.3) Perform single hidden layer stack pseudo-inverse learning, and obtain the stacked hidden layer output according to the meta-learner input weight matrix and the base learner network output:

[0141]

[0142] (2.2.4) Obtain the meta-learner output weight matrix after stacking according to the pseudo-inverse learning:

[0143]

[0144] (2.2.5) Obtain the meta-learner network output:

[0145] M f =U f ×O f ;

[0146] ​Stacked pseudo-inverse learning, as an ensemble learning technique, relies heavily on its generalization ability for image classification performance. However, most current stacked pseudo-inverse learning methods randomly initialize the input weight matrix in a random subspace without restricting the initial values, leading to instability in training results and a decline in generalization performance. Furthermore, training all samples at once may cause classifier redundancy, further weakening the model's generalization ability. To address these issues, we propose a novel framework aimed at enhancing the generalization ability of pseudo-inverse learning through an active sample selection strategy in a random subspace. Specifically, we first construct a stochastic feature pseudo-inverse learning model, constraining the performance of the input weight matrix in the random subspace through random initialization with a specific distribution, thereby improving generalization performance. Second, we design an active learning progressive model that iteratively selects beneficial samples to optimize classification results, gradually mining and utilizing training sample information to further enhance the model's generalization ability.

[0147] (3) Image classification and recognition using proactive progressive learning:

[0148] (3.1) The preprocessed factory images are randomly and evenly divided into training sets L. tr and candidate set L ca Through training set L tr The base learner is trained using (2.1.1)-(2.1.3) to obtain the initial weight matrix O. i-tr Training L ca Obtain the meta-learner output M f-ca ; For M f-ca Sample selection was performed, and L was selected. s The factory building image samples were re-added to the training set L. tr The process is iterated to obtain the meta-learner output M again. f-ca ;

[0149] (3.2) Meta-learner output for P classes and T candidate samples Select The index y corresponding to the maximum value in each column is... Randomize:

[0150]

[0151] rand(P,1) is a random vector with P rows and 1 column;

[0152] (3.3) Reassign the y-th class of each factory building selection set to obtain the class marginal probability distribution matrix:

[0153]

[0154] i = 1, 2, …, P, P represents the number of factory structure categories, j = 1, 2, …, T, T represents the number of candidate samples, representing M f-ca the i-th row and the j-th column of M

[0155] (3.4) Calculate the ambiguity of P classes and T candidate samples:

[0156]

[0157] (3.5) Construct the ambiguity set I:

[0158] I = [R(M f-ca ), M f-ca , G f-ca ];

[0159] G f-ca is the actual class label of M f-ca ;

[0160] (3.6) Iterative classification: sort I in descending order according to R(M f-ca ) value, if M f-ca ≠ G f-ca , it is an error classification sample, select the first s error classification samples, add them to L tr , re-construct the training set and candidate set, repeat the training z times, until the iteration is completed, and the final factory image classification result is obtained;

[0161] As an experiment, we verified the generalization ability of the present application on three public datasets. The experiment was performed on a computer equipped with an AMD Radeon RX 7900XT graphics card and an Intel Core i7-14700KF processor on three public datasets, COCONut, SUN09 and Pascal Panoptic Parts, respectively. COCONut contains 50 training images and 7546 test images, SUN09 consists of 60000 training images and 80000 test images, and Pascal Panoptic Parts has 800 training images and 530 test images.

[0162] As shown in the following table, when comparing the average accuracy of the traditional method and the method of the present application on the three public datasets, we found the following rules: for dataset 1 and dataset 3, the average training accuracy of the method of the present application remained stable; and for dataset 2, although the training sample size increased with the increase of the number of iterations, resulting in a slight decrease in average training accuracy, the average test accuracy of the present application was significantly improved. This result shows that in all three datasets, the average test accuracy of the present application is better than that of the traditional method, which effectively verifies the effectiveness and reliability of the present algorithm.

[0163] Table 1. Test results of traditional stack pseudo-inverse learning method and the present method on different data sets

[0164]

[0165] The performance of the current network model depends on the specific design of the base learner. Therefore, the latest research focuses on developing diverse network architectures, aiming to create base learners and stack models with different structures. However, these studies lack constraints when initializing the input weight matrix, and the randomization process ignores the potential impact of the randomness of feature block distribution on the performance of neural networks, thereby weakening the generalization ability of the model. In addition, training all ancient building images at once may cause unnecessary repetition of the classifier, damaging the model's generalization effect. The present invention first constrains the initialization of the input weight matrix by a specific distribution, and then assigns different attention weights according to the importance difference of the base learner. This process enables the new distribution after stacking to more flexibly adapt to building image data, significantly improving the accuracy and generalization ability of the factory building classification model. The present invention also divides the factory building training samples into a training set and a candidate set, and uses a sample selection strategy to select useful samples from the candidate set to join the training set, iteratively optimizing the model. This method efficiently utilizes existing training information without increasing additional training samples, reduces bias in factory sample selection, and gradually enhances the generalization performance of the classification model.

[0166] (4) Perform factory data consistency feature matching;

[0167] (4.1) For adjacent images of the same factory building three-dimensional scene, use the ORB algorithm to detect and extract feature points and their corresponding descriptors in the images to form a matching set. Traverse each feature point and find all feature points within a neighborhood radius r centered on each feature point to form a matching pair. The neighborhood is the circumscribed circle region of the basic square cell centered on the feature point;

[0168] (4.2) Calculate the total number of matching pairs C j in each neighborhood, and calculate the neighborhood value:

[0169] F mn =|C j |-I;

[0170] -I represents the operation of deleting the original matching pair, and (m j ,n j ) represents each matching pair;

[0171] (4.3) Distribute the matching pairs according to the support and remove high-similarity repeated texture error matches:

[0172]

[0173] β is the lower threshold parameter of matching, γ is the upper threshold parameter of matching, Z is the total number of neighborhood features, RI represents correct matching pairs, E represents error matching pairs, and RE represents repeated matching pairs; as shown in Figure 3 n1 is the correct matching pair of n, n2 is the repeated matching pair of n, and n3 is the error matching pair of n;

[0174] Finally, the correct matching set F(RI) is obtained;

[0175] (4.4) Initial matching optimization:

[0176] The random sample consensus algorithm is implemented on the correct matching set F(RI) to remove outliers and retain all matching pairs meeting the conditions:

[0177] DMS = RANSAC[F(RI)];

[0178] RANSAC is a random sample consensus algorithm;

[0179] The fundamental matrix is fitted for DMS:

[0180] S1 = s(DMS);

[0181] s represents the fitting operation;

[0182] (4.5) Guided matching is performed on each feature point in the matching set:

[0183] GuMS = |s(F mn ,S1)|<g;

[0184] s() represents the distance, and g is the distance parameter;

[0185] The feature points meeting the guided matching condition GuMS are retained;

[0186] (4.6) Motion consistency judgment within the threshold β is performed, and the error matching of the point-to-line logic is deleted, and the matching pairs after the judgment are retained:

[0187]

[0188] ← is a verification calculation;

[0189] In order to significantly improve the effect of three-dimensional reconstruction, the application introduces an innovative feature matching strategy, which focuses on enhancing the motion consistency through circular neighborhood segmentation. The first step is to implement a circular neighborhood segmentation technique to increase the number of initial feature point matches. Then, in order to improve the accuracy of matching, the application adopts a dual strategy: first, by strengthening the identification criteria of true and false matching points, the error matching rate between similar feature points is reduced; second, outlier rejection is implemented to further refine the accuracy of feature point matching. In addition, the application also broadens the location distribution of feature point matching, thereby enhancing the stability of the three-dimensional model. This series of improvements significantly improves the matching performance and exhibits higher stability in the three-dimensional reconstruction process.

[0190] (5) image classification and feature point matching dynamic optimization is performed;

[0191] (5.1) calculating the number of correctly classified positive examples TP, the number of correctly classified negative examples TN, the number of incorrectly classified positive examples FP, and the number of incorrectly classified negative examples FN in image classification

[0192] (5.2) calculating the classification accuracy Acc;

[0193]

[0194] (5.3) calculating the classification precision Cr;

[0195]

[0196] (5.4) calculating the feature matching precision PMS;

[0197]

[0198] TMS is the number of correct matches, and FMS is the number of incorrect matches;

[0199] (5.5) parameter dynamic adjustment:

[0200] When Acc < ε or Cr < ∈, re-execute step (3.1), re-distribute the training samples randomly and execute the subsequent steps; ε is the classification accuracy threshold, ∈ is the classification precision threshold; until the classification accuracy and classification precision are controlled within the threshold range;

[0201] When PMS < θ, increase the value of γ, re-execute step (4.3) and execute the subsequent steps until the feature matching precision is controlled within the threshold range;

[0202] The application dynamically evaluates and optimizes image classification and feature point matching by calculating indicators such as classification accuracy, classification precision, and feature matching precision. According to the evaluation results, the parameters are dynamically adjusted, and the relevant steps are re-executed until the classification accuracy and classification precision are controlled within the threshold range, and the feature matching precision is effectively improved.

[0203] (6) Three-dimensional image reconstruction:

[0204] (6.1) Using the matched pairs after discrimination, the position of each feature point in the three-dimensional space is calculated by the triangulation method; triangulation is based on two or more matching points in different perspectives, by calculating the positions of these points in the camera coordinate system, and then determining their coordinates in the three-dimensional space;

[0205] (6.2) All feature point positions obtained by triangulation are combined to form a point cloud; the point cloud is a collection of a series of discrete points in three-dimensional space, representing the shape and structure of the factory building surface.

[0206] (6.3) Starting from the point cloud, a surface reconstruction algorithm is used to generate a three-dimensional surface model of the factory building;

[0207] Common surface reconstruction algorithms include Delaunay triangulation, Poisson surface reconstruction, etc. These algorithms can generate continuous and smooth three-dimensional surfaces according to point cloud data;

[0208] (6.4) Map the texture information in the original image to the reconstructed three-dimensional surface, and extract the color information of the point to fill the three-dimensional surface;

[0209] (6.5) The reconstructed three-dimensional image of the factory building is output in a suitable file format (such as OBJ, PLY, etc.).

[0210] For the three-dimensional reconstruction process, it can be realized according to existing technical means such as point cloud data, which will not be repeated here.

[0211] In addition, as Figure 2As shown, the present application provides a kind of plant image three-dimensional reconstruction dynamic optimization system based on deep learning, comprising: data preparation and pre-processing module;Random feature learner model construction module;Image classification identification module;Feature matching module;Dynamic optimization module and three-dimensional image reconstruction module.The system, using the above-mentioned plant image three-dimensional reconstruction dynamic optimization method based on deep learning executes plant image three-dimensional reconstruction task.The system realizes the full automation process from data preparation to three-dimensional image reconstruction by deep learning technology.Deep learning model can automatically extract key features in image, and carry out classification and identification, reduce the need for manual intervention.Random feature learner model can learn the complex features in image, improve the accuracy of image classification and identification.Feature matching module ensures the accuracy and stability of three-dimensional reconstruction by accurate feature matching algorithm.Dynamic optimization module can adjust the reconstruction process in real time according to actual situation, further improve the robustness of system.Deep learning model can learn efficient feature representation in training process, thereby speeding up the speed of image processing and three-dimensional reconstruction.The system adopts modular design, and each module works cooperatively, improves the overall processing efficiency.In terms of real-time, the system can quickly respond to input data, and generate high-quality three-dimensional image, meet the time requirement in practical application.The system has good scalability, can add new functional modules or optimize existing modules according to actual demand.Deep learning model has strong generalization ability, can process plant image three-dimensional reconstruction task in different scenarios.The flexibility of the system is also reflected in the adaptability to input data, can process images under different resolution and different lighting conditions.Compared with traditional three-dimensional reconstruction method, the system reduces the demand for professional equipment and manual operation, thereby reducing cost.Automatic and intelligent characteristics make the system can continue to run efficiently, improve work efficiency and economic benefit.

[0212] The above is only the preferred embodiment of the present application, but the protection scope of the present application is not limited to this, any skilled person in the art can make equivalent replacement or change according to the technical solution and inventive concept of the present application within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A dynamic optimization method for 3D reconstruction of factory building images based on deep learning, characterized in that, Includes the following steps: (1) Data preparation and preprocessing: Collect multi-angle image data of the factory building to ensure that the images cover all important areas of the factory building; The collected image data is preprocessed, including noise removal, image contrast enhancement, and image color correction, to obtain the preprocessed factory image. (2) Constructing a stochastic feature learner model: Initialize the input weight matrix according to different specific distributions and impose constraints on the input weight matrix; Assign corresponding attention weights according to the importance of each base learner; Constructing a base learner involves creating an input weight matrix that follows a normal or uniform distribution, constructing the hidden layer output, and obtaining the output weight matrix through pseudo-inverse learning, thereby obtaining the base learner output. Constructing a meta learner involves calculating the similarity between the label matrix and the output of the base learner network to obtain the meta learner input weight matrix, and performing single-hidden-layer stacked pseudo-inverse learning to obtain the meta learner network output. (3) Active progressive learning image classification and recognition: The preprocessed factory images are randomly and evenly divided into training set and candidate set; the base learner and meta learner are trained using the training set by constructing base learner and meta learner; the factory image samples in the candidate set are re-added to the training set by iterative selection, and the sample selection and class redistribution are performed according to the output of meta learner. The ambiguity of candidate samples is calculated, an ambiguity set is constructed, and iterative classification is performed until the iteration ends. Images with different factory building structure distributions are classified to obtain the final factory building image classification results. (4) Factory building data consistency feature matching: For adjacent images of the same factory building in the factory building image classification results, the ORB algorithm is used to detect and extract feature points and their descriptors; the total number of matching pairs and neighborhood values ​​in the neighborhood of each feature point are calculated; the correct matching pairs and incorrect matching pairs are distinguished according to the support, and the high similarity repeated texture incorrect matching is removed; the random sampling consistency algorithm is implemented on the correct matching set to remove outliers; the matching set is guided matching and motion consistency judgment within the threshold is performed, and the matching pairs that meet the conditions are retained. (5) Dynamic optimization of image classification and feature point matching: Calculate the number of correct and incorrect classifications in image classification, and calculate the classification accuracy, classification precision and feature matching accuracy; dynamically adjust the parameters according to the classification accuracy, classification precision and feature matching accuracy until these indicators are controlled within the preset threshold range, and obtain the optimized matching pairs that meet the conditions. (6) 3D image reconstruction: Using the optimized matching pairs that meet the conditions, calculate the position of each feature point in the 3D space by triangulation; combine all feature point positions into a point cloud; starting from the point cloud, use the surface reconstruction algorithm to generate a 3D surface model of the factory; map the texture information in the original image onto the 3D surface and extract color information to fill the 3D surface; output the reconstructed 3D image of the factory in an appropriate file format.

2. The method for dynamic optimization of 3D reconstruction of factory images based on deep learning according to claim 1, characterized in that, The construction of the random feature learner model specifically includes: (2.1) Constructing the base learner: Create different input weight matrices that follow normal and uniform distributions, construct the hidden layer output, obtain the output weight matrix through pseudo-inverse learning, and obtain the base learner output; (2.1.1) Creation Each input weight matrix is ​​different and follows a specific distribution. , Given any distribution; set i=3, generate 3 different specific distributions. ; To conform to the expected value The standard deviation is Normal distribution: ； To obey the interval is Uniform distribution: ； To conform to the expected value The standard deviation is Normal distribution: ； in To ensure uniform distribution, It follows a normal distribution; (2.1.2) Constructing the hidden layer output : ； Where L is the input matrix, and in this invention, L represents the input image data. For activation functions; (2.1.3) For the input matrix L, collect the labels of each image, construct the label matrix G, and obtain the output weight matrix through pseudo-inverse learning: ； (2.1.4) Obtain the output of the base learner network: ； (2.2) Constructing the meta-learner: Obtain the meta-learner input weight matrix by calculating the similarity between the label matrix and the output of the base learner network. Perform single-hidden-layer stacked pseudo-inverse learning based on the meta-learner input weight matrix and the output of the base learner network to obtain the hidden layer output after stacking. Obtain the meta-learner network output through the meta-learner output weight matrix after stacking. (2.2.1) Calculate the label matrix G and the output of the base learner network. Similarity between them: ； (2.2.2) Obtain the meta-learner input weight matrix through the logistic normalization function: ； (2.2.3) Perform single-hidden-layer stacked pseudo-inverse learning, and obtain the stacked hidden layer output based on the input weight matrix of the meta-learner and the output of the base learner network: ； (2.2.4) Obtain the weight matrix of the stacked meta-learner output based on pseudo-inverse learning: ； (2.2.5) Obtain the output of the meta-learner network: 。 3. The method for dynamic optimization of 3D reconstruction of factory images based on deep learning according to claim 1, characterized in that, The aforementioned active progressive learning-based image classification and recognition specifically includes: (3.1) The preprocessed factory images are randomly and evenly divided into training sets as training samples. and candidate set Through the training set The base learner is trained using (2.1.1)-(2.1.3) to obtain the initial weight matrix. ,train Obtain the output of the meta-learner ;right Perform sample selection, select The factory building image samples were re-added to the training set. The process is iterated to obtain the meta-learner output again. ; (3.2) Meta-learner output for P classes and T candidate samples , Select The index y corresponding to the maximum value in each column is... Randomize: ； It is a random vector with P rows and 1 column; (3.3) Reassign the y-th class of each factory building selection set to obtain the class marginal probability distribution matrix: ； , Indicates the number of factory construction categories. , Indicates the number of candidate samples. express The i-th row and j-th column; (3.4) Calculate the ambiguity of P classes and T candidate samples: ； (3.5) Constructing the ambiguity set I: ； for The actual class label; (3.6) Perform iterative classification: based on value pairs Sort in descending order, if If the first s are misclassified samples, then they are selected and added to the list. In the process, the training set and candidate set are reconstructed, and the training is repeated z times until the end of the iteration to obtain the final factory image classification result.

4. The method for dynamic optimization of 3D reconstruction of factory images based on deep learning according to claim 1, characterized in that, The aforementioned factory data consistency feature matching specifically includes: (4.1) For adjacent images of the same factory building in the factory building image classification results, the ORB algorithm is used to detect and extract feature points and their corresponding descriptors in the image to form a matching set. Each feature point is traversed, and all feature points within the neighborhood radius r are found with each feature point as the center and a matching pair is formed. The neighborhood is the outer circle area of ​​the basic square cell formed with the feature point as the center. (4.2) Calculate the total number of matching pairs in each neighborhood. Calculate the neighborhood value: ； This represents the operation of deleting the original matching pair. Represents each matching pair; (4.3) Distribute the matching pairs according to their support and remove erroneous matches with highly similar and repetitive textures: ； To match the upper limit threshold parameter, The total number of features in the neighborhood. E indicates a correct match, while E indicates an incorrect match. Represents duplicate matching pairs; Finally, the correct matching set is obtained. ; (4.4) Initial matching optimization: For the correct matching set The random sampling consensus algorithm is used to remove outliers and retain all matching pairs that meet the criteria: ； It is a random sampling consensus algorithm; right Fit the fundamental matrix: ； This represents the fitting operation; (4.5) Guided matching for each feature point in the matching set: ； Indicates distance, For distance parameters; Retain those that meet the guiding matching conditions Feature points; (4.6) Thresholding Within the motion consistency check, incorrect point-to-line matching is deleted, and the correctly matched pairs are retained. ； To verify the calculation.

5. The method for dynamic optimization of 3D reconstruction of factory images based on deep learning according to claim 1, characterized in that, The aforementioned image classification and feature point matching dynamic optimization specifically includes: (5.1) Calculate the number of images correctly classified as positive in the image classification. The number of cases correctly classified as negative The number of cases incorrectly classified as positive. and the number of cases incorrectly classified as negative ; (5.2) Calculate the classification accuracy. ; ； (5.3) Calculate classification accuracy ; ； (5.4) Calculate the feature matching accuracy ; ； To ensure the correct number of matches, This represents the number of incorrect matches. (5.5) Dynamic parameter adjustment: when Sometimes If necessary, repeat step (3.1) to redistribute the training samples randomly and evenly and then proceed with the subsequent steps. The threshold for classification accuracy. Set a classification accuracy threshold; continue until both classification accuracy and classification correctness are controlled within the threshold range. when At that time, improve If the value is not found, repeat step (4.3) and perform subsequent steps until the feature matching accuracy is controlled within the threshold range.

6. The method for dynamic optimization of 3D reconstruction of factory images based on deep learning according to claim 1, characterized in that, The aforementioned three-dimensional image reconstruction specifically includes: (6.1) Using the discriminated matching pairs, calculate the position of each feature point in three-dimensional space by triangulation; triangulation is based on matching points in two or more viewpoints, and by calculating the position of these points in the camera coordinate system, their coordinates in three-dimensional space are determined. (6.2) Combine all the feature point positions obtained by triangulation to form a point cloud; the point cloud is a collection of discrete points in three-dimensional space, representing the shape and structure of the surface of the factory building and its internal facilities; (6.3) Starting from the point cloud, a surface reconstruction algorithm is used to generate a three-dimensional surface model of the factory building; (6.4) Map the texture information in the original image onto the reconstructed 3D surface and extract the color information of the point to fill the 3D surface; (6.5) Output the reconstructed 3D image of the factory building in an appropriate file format.

7. A dynamic optimization system for 3D reconstruction of factory building images based on deep learning, characterized in that, include: Data preparation and preprocessing module: used to collect multi-angle image data of the factory and ensure that the images cover all important parts of the factory; The image is preprocessed, including noise reduction, contrast enhancement, and color correction; The stochastic feature learner model building module includes a base learner building unit and a meta learner building unit. The base learner building unit is used to create input weight matrices that follow different specific distributions, build the hidden layer output, and obtain the output weight matrix through pseudo-inverse learning, thereby obtaining the base learner output. The meta-learner building unit obtains the meta-learner input weight matrix by calculating the similarity between the label matrix and the output of the base learner network, performs single-hidden-layer stacked pseudo-inverse learning, obtains the hidden layer output after stacking, and obtains the meta-learner network output through the weight matrix of the meta-learner output after stacking. Image classification and recognition module: The preprocessed factory images are randomly and evenly divided into training set and candidate set as training samples; the base learner and meta learner are trained using the training set; sample selection is performed, and the selected factory image samples are added back to the training set for iteration until the iteration termination condition is met, and the final factory image classification result is obtained. Feature matching module: Utilizes the ORB algorithm to detect and extract feature points and their corresponding descriptors in adjacent 3D scene images of the same factory building; it iterates through each feature point, constructs matching pairs, and calculates the total number of matching pairs and neighborhood values ​​within the neighborhood; it distinguishes between correct and incorrect matching pairs based on support, removing incorrect matches with highly similar repetitive textures; it applies a random sampling consensus algorithm to the correct matching set to remove outliers, and performs guided matching and motion consistency judgment to retain the discriminated matching pairs; Dynamic optimization module: Calculates the number of correctly classified positive examples, negative examples, and incorrectly classified examples in image classification; Calculate classification accuracy, classification precision, and feature matching accuracy; dynamically adjust the parameters based on classification accuracy, classification precision, and feature matching accuracy until they are controlled within the set threshold range; 3D Image Reconstruction Module: Using the discriminated matching pairs, the position of each feature point in 3D space is calculated by triangulation; all feature point positions obtained by triangulation are combined into a point cloud; a 3D surface model of the factory is generated using a surface reconstruction algorithm; texture information in the original image is mapped onto the reconstructed 3D surface, and color information is extracted to fill the 3D surface; the reconstructed 3D image of the factory is output in an appropriate file format.

8. The deep learning-based dynamic optimization system for 3D reconstruction of factory images according to claim 7, characterized in that, The task of three-dimensional reconstruction of factory building images is performed using any one of the methods in claims 1-6.

Images