Workshop image three-dimensional reconstruction dynamic optimization method and system based on deep learning

Abstract

The invention relates to the technical field of artificial intelligence building design, in particular to a plant image three-dimensional reconstruction dynamic optimization method and system based on deep learning. The method comprises the following steps: (1) data preparation and preprocessing; (2) constructing a random feature learner model; (3) image classification and identification based on active progressive learning; (4) matching factory data consistency characteristics; (5) dynamically optimizing image classification and feature point matching; and (6) reconstructing a three-dimensional image. Compared with the prior art, the method has the advantages that the precision and efficiency of three-dimensional reconstruction are improved, and more reliable technical support is provided for three-dimensional modeling and visualization of the factory building.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence architectural design technology, and in particular to a method and system for dynamic optimization of three-dimensional reconstruction of plant images based on deep learning. Background Art

[0002] In the construction industry, especially in plant construction and maintenance, 3D modeling technology plays a vital role. With the rapid development of image recognition technology, plant 3D modeling technology based on image recognition has become a research hotspot in the industry. However, although the technology has made certain progress, the existing plant 3D modeling technology based on image recognition still has many shortcomings and challenges.

[0003] First, there are deficiencies in feature extraction and matching algorithms. Feature extraction and matching are key steps in the 3D modeling of factory buildings. However, most existing feature extraction algorithms rely on fixed templates or rules, which are difficult to adapt to the complexity and diversity of different factory environments. At the same time, matching algorithms also face problems such as large computational complexity and low matching accuracy, resulting in a large number of incorrect matches and misjudgments in the reconstruction results.

[0004] In addition, existing 3D reconstruction methods lack a dynamic optimization mechanism. In the process of 3D modeling of factory buildings, due to differences in image data, equipment performance and other factors, the reconstruction results often have great uncertainty. Most existing methods use fixed reconstruction processes and parameter settings, and lack the ability to dynamically adjust and optimize according to actual conditions. This makes it difficult to ensure the accuracy and efficiency of reconstruction results when facing different factory environments.

[0005] In summary, the existing factory 3D modeling technology based on image recognition still has many shortcomings and challenges. In order to overcome these shortcomings, the present invention proposes a dynamic optimization method and system for 3D reconstruction of factory images based on deep learning, aiming to improve the accuracy of image data preprocessing, feature extraction and matching through deep learning technology, and at the same time introduce a dynamic optimization mechanism to achieve accurate, efficient and adaptive 3D modeling of factory buildings. Summary of the invention

[0006] The purpose of the present invention is to provide a deep learning-based dynamic optimization method for three-dimensional reconstruction of factory images with higher accuracy and better precision.

[0007] The object of the present invention is achieved in that:

[0008] A dynamic optimization method for three-dimensional reconstruction of factory images based on deep learning, comprising the following steps:

[0009] (1) Data preparation and preprocessing: Collect multi-angle image data of the factory to ensure that the images cover all important areas of the factory; preprocess the collected image data, including removing noise, enhancing image contrast, and correcting image color to obtain preprocessed factory images;

[0010] (2) Constructing a random feature learner model: Initializing the input weight matrix according to different specific distributions and applying constraints to the input weight matrix; assigning corresponding attention weights according to the importance of each base learner; constructing a base learner, including creating an input weight matrix that follows a normal distribution and a uniform distribution, constructing a hidden layer output, and obtaining an output weight matrix through pseudo-inverse learning, thereby obtaining the base learner output; constructing a meta-learner, obtaining the meta-learner input weight matrix by calculating the similarity between the label matrix and the base learner network output, and performing single hidden layer stack pseudo-inverse learning to obtain the meta-learner network output;

[0011] (3) Image classification and recognition using active progressive learning: The preprocessed factory images are randomly and evenly divided into a training set and a candidate set; the base learner and the meta learner are constructed and trained using the training set; factory image samples in the candidate set are iteratively selected and added back to the training set, and sample selection and category reallocation are performed based on the output of the meta learner; the fuzziness of the candidate samples is calculated, a fuzziness set is constructed, and iterative classification is performed until the iteration ends, and images with different factory structure distributions are classified to obtain the final factory image classification results;

[0012] (4) Consistent feature matching of factory data: For adjacent images of the same factory building in the factory image classification results, the ORB algorithm is used to detect and extract feature points and their descriptors; the total number of matching pairs and the neighborhood value in the neighborhood of each feature point are calculated; correct matching pairs and incorrect matching pairs are distinguished according to the support, and incorrect matching of highly similar repeated textures is removed; a random sampling consensus algorithm is implemented on the correct matching set to remove outliers; guided matching and motion consistency judgment within the threshold are performed on the matching set, and matching pairs that meet the conditions are retained;

[0013] (5) Dynamic optimization of image classification and feature point matching: Calculate the number of correct and incorrect divisions in image classification, and calculate the classification accuracy, classification precision, and feature matching accuracy; dynamically adjust parameters according to the classification accuracy, classification precision, and feature matching accuracy until these indicators are controlled within the preset threshold range, and obtain the optimized matching pairs that meet the conditions;

[0014] (6) Three-dimensional image reconstruction: Use the optimized matching pairs that meet the conditions to calculate the position of each feature point in the three-dimensional space through triangulation method; combine the positions of all feature points into a point cloud; based on the point cloud, use the surface reconstruction algorithm to generate a three-dimensional surface model of the factory; map the texture information in the original image onto the three-dimensional surface, and extract the color information to fill the three-dimensional surface; output the reconstructed three-dimensional image of the factory to an appropriate file format.

[0015] The construction of the random feature learner model specifically includes:

[0016] (2.1) Constructing base learners: Create different input weight matrices that follow normal distribution and uniform distribution, construct hidden layer outputs, obtain output weight matrices through pseudo-inverse learning, and obtain base learner outputs;

[0017] (2.1.1) Create i different input weight matrices K that follow a specific distribution i , K i is any distribution; set i = 3 to generate 3 K with different specific distributions i ;

[0018] K1 is a normal distribution with an expected value of μ1 and a standard deviation of δ1:

[0019]

[0020] K2 is a uniform distribution with an interval of [p,q]:

[0021] K2~U(p,q);

[0022] K3 is a normal distribution with an expected value of μ2 and a standard deviation of δ2:

[0023]

[0024] Among them, U is uniform distribution and N is normal distribution;

[0025] (2.1.2) Construct the hidden layer output U i :

[0026] U i =ξ(L×K i );

[0027] Wherein L is the input matrix, which is the input image data in the present invention, and ξ is the activation function;

[0028] (2.1.3) For the input matrix L, collect the label of each image, construct the label matrix G, and obtain the output weight matrix through pseudo-inverse learning:

[0029]

[0030] (2.1.4) Get the base learner network output:

[0031] M i =O i ×U i ;

[0032] (2.2) Constructing a meta-learner: Obtain the meta-learner input weight matrix by calculating the similarity between the label matrix and the base learner network output, perform single hidden layer stacking pseudo-inverse learning based on the meta-learner input weight matrix and the base learner network output, obtain the hidden layer output after stacking, and obtain the meta-learner network output through the meta-learner output weight matrix after stacking;

[0033] (2.2.1) Calculate the label matrix G and the base learner network output M i The similarity between:

[0034]

[0035] (2.2.2) The meta-learner input weight matrix is ​​obtained through the logical normalization function:

[0036]

[0037] (2.2.3) Perform pseudo-inverse learning of a single hidden layer stack, and obtain the hidden layer output after stacking according to the meta-learner input weight matrix and the base learner network output:

[0038]

[0039] (2.2.4) According to pseudo-inverse learning, the output weight matrix of the stacked meta-learner is obtained:

[0040]

[0041] (2.2.5) Get the meta-learner network output:

[0042] M f =U f ×O f .

[0043] The active progressive learning of image classification and recognition specifically includes:

[0044] (3.1) The preprocessed factory images are randomly divided into training set L as training samples. tr and the candidate set L ca , through the training set L tr Use (2.1.1)-(2.1.3) to train the base learner and get the initial weight matrix O i-tr , training Lca Get the meta-learner output M f-ca ; for M f-ca For sample selection, select L s Factory image samples are added back to the training set L tr Iterate in and get the meta-learner output M again f-ca ;

[0045] (3.2) The meta-learner output for P categories and T candidate samples is Select The index value y corresponding to the maximum value of each column in To perform randomization:

[0046]

[0047] rand(P,1) is a random vector with P rows and 1 column;

[0048] (3.3) Reassign each factory building set to the yth class and obtain the class marginal probability distribution matrix:

[0049]

[0050] i=1,2,…,P, P represents the number of factory structure categories, j=1,2,…,T, T represents the number of candidate samples, Indicates M f-ca The i-th row and j-th column of ;

[0051] (3.4) Calculate the fuzziness of P categories and T candidate samples:

[0052]

[0053] (3.5) Construct fuzzy set I:

[0054] I=]R(M f-ca ),M f-ca ,G f-ca ];

[0055] G f-ca M f-ca The actual class label is a label that is formulated based on the actual situation and is used to classify the graphics.

[0056] (3.6) Iterate classification: According to R(M f-ca ) value to sort I in descending order, if M f-ca ≠G f-ca , then it is an incorrectly classified sample. Select the first s incorrectly classified samples and add them to L trIn the process, the training set and candidate set are reconstructed, and the training is repeated z times until the iteration ends to obtain the final factory image classification result.

[0057] The plant data consistency feature matching specifically includes:

[0058] (4.1) For adjacent images of the 3D scene of the same factory building, the ORB algorithm is used to detect and extract feature points in the image and their corresponding descriptors to form a matching set. Each feature point is traversed, and all feature points within a neighborhood radius r are searched with each feature point as the center to form a matching pair. The neighborhood is the circumscribed circular area of ​​a basic square cell formed with the feature point as the center;

[0059] (4.2) Calculate the total number of matching pairs C in each neighborhood j , calculate the neighborhood value:

[0060] F mn =|C j |-I;

[0061] -I stands for deleting the original matching pair operation. j ,n j ) represents each matching pair;

[0062] (4.3) The matching pairs are distributed and distinguished according to the support, and the highly similar repeated texture mismatches are removed:

[0063]

[0064] β is the matching lower threshold parameter, γ is the matching upper threshold parameter, Z is the total number of neighborhood features, RI represents the matching pair is a correct matching pair, E represents the matching pair is an incorrect matching pair, and RE represents a repeated matching pair;

[0065] Finally, the correct matching set F(RI) is obtained;

[0066] (4.4) Initial matching optimization:

[0067] A random sampling consensus algorithm is implemented on the correct matching set F(RI) to remove outliers and retain all matching pairs that meet the conditions:

[0068] DMS = RANSAC[F(RI)];

[0069] RANSAC is a random sampling consensus algorithm;

[0070] Fit the basic matrix to DMS:

[0071] S1 = s(DMS);

[0072] s represents the fitting operation;

[0073] (4.5) Guide matching for each feature point in the matching set:

[0074] GuMS=|s(F mn ,S1)| <g;

[0075] s() represents the distance, and g is the distance parameter;

[0076] Retain the feature points that meet the guided matching condition GuMS;

[0077] (4.6) Perform motion consistency judgment within the threshold β, delete the wrong matching of point-to-line logic, and retain the matching pairs after judgment:

[0078]

[0079] ←To verify the calculation.

[0080] The image classification and feature point matching dynamic optimization specifically include:

[0081] (5.1) Calculate the number of correctly classified positive examples TP, the number of correctly classified negative examples TN, the number of incorrectly classified positive examples FP, and the number of incorrectly classified negative examples FN in image classification.

[0082] (5.2) Calculate the classification accuracy Acc;

[0083]

[0084] (5.3) Calculate the classification accuracy Cr;

[0085]

[0086] (5.4) Calculate feature matching accuracy PMS;

[0087]

[0088] TMS is the number of correct matches, and FMS is the number of incorrect matches;

[0089] (5.5) Dynamic adjustment of parameters:

[0090] When Acc<ε or Cr<∈, re-execute step (3.1), re-randomly average the training samples and execute the subsequent steps; ε is the classification accuracy threshold, ∈ is the classification accuracy threshold; until the classification accuracy and classification accuracy are controlled within the threshold range;

[0091] When PMS<θ, increase the value of γ, re-execute step (4.3) and perform subsequent steps until the feature matching accuracy is controlled within the threshold range.

[0092] The three-dimensional image reconstruction specifically includes:

[0093] (6.1) Using the identified matching pairs, the position of each feature point in three-dimensional space is calculated by triangulation. Triangulation is based on matching points in two or more viewpoints, and the positions of these points in the camera coordinate system are calculated to determine their coordinates in three-dimensional space.

[0094] (6.2) All feature point positions obtained through triangulation are combined to form a point cloud; a point cloud is a collection of discrete points in three-dimensional space that represents the surface shape and structure of the factory building and its internal facilities.

[0095] (6.3) Starting from the point cloud, use the surface reconstruction algorithm to generate a 3D surface model of the factory building;

[0096] (6.4) Mapping the texture information in the original image onto the reconstructed three-dimensional surface, and extracting the color information of the point to fill the three-dimensional surface;

[0097] (6.5) Output the reconstructed 3D image of the factory building into an appropriate file format.

[0098] The present invention also aims to provide a deep learning-based dynamic optimization system for three-dimensional reconstruction of plant images, comprising:

[0099] Data preparation and preprocessing module: used to collect multi-angle image data of the factory building and ensure that the images cover all important parts of the factory building; preprocess the images, including denoising, contrast enhancement and color correction;

[0100] A random feature learner model construction module includes a base learner construction unit and a meta learner construction unit, wherein the base learner construction unit is used to create an input weight matrix following different specific distributions, construct a hidden layer output, and obtain an output weight matrix through pseudo-inverse learning, thereby obtaining the base learner output; the meta learner construction unit obtains a meta learner input weight matrix by calculating the similarity between a label matrix and a base learner network output, performs single hidden layer stack pseudo-inverse learning, obtains a hidden layer output after stacking, and obtains a meta learner network output through the meta learner output weight matrix after stacking;

[0101] Image classification and recognition module: The pre-processed plant images are randomly divided into training sets and candidate sets as training samples; the training sets are used to train the base learner and meta-learner; sample selection is performed, and the selected plant image samples are added back to the training set for iteration until the iteration end condition is met to obtain the final plant image classification result;

[0102] Feature matching module: Use the ORB algorithm to detect and extract feature points and their corresponding descriptors in adjacent images of the 3D scene of the same factory building; traverse each feature point, build matching pairs, and calculate the total number of matching pairs and neighborhood values ​​in the neighborhood; distinguish correct matching pairs from incorrect matching pairs based on support, and remove incorrect matching of highly similar repeated textures; implement a random sampling consensus algorithm on the correct matching set to remove outliers, and perform guided matching and motion consistency judgment to retain the identified matching pairs;

[0103] Dynamic optimization module: calculates the number of images correctly classified as positive examples, negative examples, and incorrectly classified images in image classification; calculates classification accuracy, classification precision, and feature matching accuracy; dynamically adjusts parameters according to classification accuracy, classification precision, and feature matching accuracy until they are within the set threshold range;

[0104] 3D image reconstruction module: Use the identified matching pairs to calculate the position of each feature point in 3D space through triangulation method; combine all feature point positions obtained through triangulation into a point cloud; use the surface reconstruction algorithm to generate a 3D surface model of the factory; map the texture information in the original image to the reconstructed 3D surface, and extract color information to fill the 3D surface; output the reconstructed 3D image of the factory to an appropriate file format.

[0105] The system uses the above-mentioned deep learning-based three-dimensional reconstruction dynamic optimization method of factory images to perform the three-dimensional reconstruction task of factory images.

[0106] The beneficial effects of the present invention are:

[0107] (1) When constructing feature learners, the prior art often relies on fixed weight allocation and cannot flexibly adapt to different data characteristics. The importance differences between base learners are ignored, resulting in limited accuracy and generalization ability of the classification model. The present invention introduces a random feature learner model, initializes the input weight matrix through different specific distributions, and imposes constraints on the input weight matrix, thereby improving the flexibility and adaptability of the model. According to the importance of each base learner, the corresponding attention weights are allocated. The new distribution formed by stacking processing can adapt to data characteristics more flexibly, significantly improving the accuracy and generalization ability of the classification model.

[0108] (2) The prior art often uses fixed training sets and test sets for image classification and recognition, and lacks the ability to dynamically adjust and optimize. For samples that are misclassified, the prior art often lacks an effective processing mechanism, resulting in inaccurate classification results. The present invention adopts an active progressive learning method to improve the dynamic adjustment and optimization capabilities of the classification model by continuously iterating and optimizing the training set and candidate set. The fuzzy calculation and iterative classification mechanism are introduced to effectively process and redistribute the misclassified samples, thereby improving the accuracy of the classification results.

[0109] (3) The prior art is often affected by erroneous matches such as highly similar repeated textures during feature matching, resulting in inaccurate matching results. The lack of effective initial matching optimization and guided matching mechanisms limits matching efficiency and accuracy. The present invention uses the ORB algorithm to detect and extract feature points and their descriptors, and effectively removes erroneous matches of highly similar repeated textures by calculating the total number of matching pairs and neighborhood values ​​in the neighborhood. The initial matching optimization and guided matching mechanisms are introduced, and outliers are eliminated through a random sampling consensus algorithm, matching pairs that meet the conditions are retained, and feature points that meet the guided matching conditions are retained, thereby improving matching efficiency and accuracy.

[0110] (4) The prior art lacks the ability to dynamically optimize image classification and feature point matching, and is unable to adjust parameters based on classification accuracy and feature matching accuracy. The present invention dynamically evaluates and optimizes image classification and feature point matching by calculating indicators such as classification accuracy, classification accuracy, and feature matching accuracy. Based on the evaluation results, the parameters are dynamically adjusted and the relevant steps are re-executed until the classification accuracy and classification accuracy are controlled within the threshold range, and the feature matching accuracy is also effectively improved.

[0111] In summary, the present invention proposes a dynamic optimization method for three-dimensional reconstruction of factory images based on deep learning, and has achieved remarkable beneficial effects in data preparation and preprocessing, building a random feature learner model, performing active progressive learning of image classification and recognition, performing consistent feature matching of factory data, performing dynamic optimization of image classification and feature point matching, and three-dimensional image reconstruction. Compared with the prior art, the present invention improves the accuracy and efficiency of three-dimensional reconstruction, and provides more reliable technical support for three-dimensional modeling and visualization of factory buildings. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] Figure 1 Schematic diagram of the method of the present invention.

[0113] Figure 2 Schematic diagram of the system of the present invention.

[0114] Figure 3 This is a schematic diagram of feature point matching. DETAILED DESCRIPTION

[0115] The present invention is further described below in conjunction with the accompanying drawings.

[0116] like Figure 1 As shown, a dynamic optimization method for three-dimensional reconstruction of factory images based on deep learning includes the following steps:

[0117] (1) Data preparation and preprocessing: Collect multi-angle image data of the factory to ensure that the images cover all important parts of the factory. Preprocess the images, including denoising, contrast enhancement, and color correction, to improve the accuracy of subsequent processing;

[0118] (2) Construct a random feature learner model: Initialize the input weight matrix according to different specific distributions to impose constraints on the input weight matrix; assign corresponding attention weights to each base learner according to its importance. The new distribution formed by stacking can adapt to data characteristics more flexibly, thereby significantly improving the accuracy and generalization ability of the classification model.

[0119] (2.1) Constructing base learners: Create different input weight matrices that follow normal distribution and uniform distribution, construct hidden layer outputs, obtain output weight matrices through pseudo-inverse learning, and obtain base learner outputs;

[0120] (2.1.1) Create i different input weight matrices K that follow a specific distribution i , K i is any distribution; set i = 3 to generate 3 K with different specific distributions i ;

[0121] K1 is a normal distribution with an expected value of μ1 and a standard deviation of δ1:

[0122]

[0123] K2 is a uniform distribution with an interval of [p,q]:

[0124] K2~U(p,q);

[0125] K3 is a normal distribution with an expected value of μ2 and a standard deviation of δ2:

[0126]

[0127] Among them, U is uniform distribution and N is normal distribution;

[0128] (2.1.2) Construct the hidden layer output U i :

[0129] U i =ξ(L×K i );

[0130] Wherein L is the input matrix, which is the input image data in the present invention, and ξ is the activation function;

[0131] (2.1.3) For the input matrix L, collect the label of each image, construct the label matrix G, and obtain the output weight matrix through pseudo-inverse learning:

[0132]

[0133] (2.1.4) Get the base learner network output:

[0134] M i =O i ×U i ;

[0135] (2.2) Constructing a meta-learner: Obtain the meta-learner input weight matrix by calculating the similarity between the label matrix and the base learner network output, perform single hidden layer stacking pseudo-inverse learning based on the meta-learner input weight matrix and the base learner network output, obtain the hidden layer output after stacking, and obtain the meta-learner network output through the meta-learner output weight matrix after stacking;

[0136] (2.2.1) Calculate the label matrix G and the base learner network output M i The similarity between:

[0137]

[0138] (2.2.2) The meta-learner input weight matrix is ​​obtained through the logical normalization function:

[0139]

[0140] (2.2.3) Perform pseudo-inverse learning of a single hidden layer stack, and obtain the hidden layer output after stacking according to the meta-learner input weight matrix and the base learner network output:

[0141]

[0142] (2.2.4) According to pseudo-inverse learning, the output weight matrix of the stacked meta-learner is obtained:

[0143]

[0144] (2.2.5) Get the meta-learner network output:

[0145] M f =U f ×O f ;

[0146] As an ensemble learning technique, the effectiveness of stacked pseudo-inverse learning in image classification is highly dependent on its generalization ability. However, most current stacked pseudo-inverse learning methods randomly initialize the input weight matrix in a random subspace without restricting the initial value, which leads to instability in the training results and a decline in generalization performance. In addition, training all samples at once may cause classifier redundancy, further weakening the generalization ability of the model. To address these problems, we propose a novel framework that aims to enhance the generalization ability of pseudo-inverse learning through an active sample selection strategy in a random subspace. Specifically, we first construct a random feature pseudo-inverse learning model, which constrains the performance of the input weight matrix in the random subspace through random initialization with a specific distribution, thereby improving the generalization performance. Secondly, we design an active learning progressive model, which selects beneficial samples in an iterative manner to optimize the classification effect, gradually mines and utilizes the training sample information, and thus improves the generalization ability of the model.

[0147] (3) Image classification and recognition using active progressive learning:

[0148] (3.1) The preprocessed factory images are randomly divided into training set L as training samples. tr and the candidate set L ca , through the training set L tr Use (2.1.1)-(2.1.3) to train the base learner and get the initial weight matrix O i-tr , training L ca Get the meta-learner output M f-ca ; for M f-ca For sample selection, select L s Factory image samples are added back to the training set L tr Iterate in and get the meta-learner output M again f-ca ;

[0149] (3.2) The meta-learner output for P categories and T candidate samples is Select The index value y corresponding to the maximum value of each column in To perform randomization:

[0150]

[0151] rand(P,1) is a random vector with P rows and 1 column;

[0152] (3.3) Reassign each factory building set to the yth class and obtain the class marginal probability distribution matrix:

[0153]

[0154] i=1,2,…,P, P represents the number of factory structure categories, j=1,2,…,T, T represents the number of candidate samples, Indicates M f-ca The i-th row and j-th column of ;

[0155] (3.4) Calculate the fuzziness of P categories and T candidate samples:

[0156]

[0157] (3.5) Construct fuzzy set I:

[0158] I=[R(M f-ca ),M f-ca ,G f-ca ];

[0159] G f-ca M f-ca The actual class label;

[0160] (3.6) Iterate classification: According to R(M f-ca ) value to sort I in descending order, if M f-ca ≠G f-ca , then it is an incorrectly classified sample. Select the first s incorrectly classified samples and add them to L tr In the process, the training set and candidate set are reconstructed, and the training is repeated z times until the iteration ends to obtain the final factory image classification result;

[0161] As an experiment, we verified the generalization ability of the present invention on three public datasets. The experiment was conducted on a computer equipped with an AMD Radeon RX 7900XT graphics card and an Intel Core i7-14700KF processor. The three public datasets are COCONut, SUN09 and Pascal Panoptic Parts. COCONut contains 50 training images and 7546 test images, SUN09 consists of 60,000 training images and 80,000 test images, and Pascal Panoptic Parts has 800 training images and 530 test images.

[0162] As shown in the table below, when comparing the average accuracy of the traditional method and the method of the present invention on three public data sets, we found the following rules: for data sets 1 and 3, the average training accuracy of the method of the present invention remained stable; while for data set 2, although the average training accuracy slightly declined as the number of iterations increased and the training sample size increased, the average test accuracy of the present invention was significantly improved. This result shows that in all three data sets, the average test accuracy of the present invention is better than that of the traditional method, which strongly verifies the effectiveness and reliability of the algorithm.

[0163] Table 1. Comparison of test results between the traditional stack pseudo-inverse learning method and this method on different data sets

[0164]

[0165] The performance of current network models depends on the specific design of base learners. Therefore, the latest research focuses on developing diverse network architectures, aiming to create base learners and stack models with different structures. However, these studies lack constraints when initializing the input weight matrix, and the randomization process ignores the potential impact of the randomness of the feature block distribution on the performance of the neural network, thereby weakening the generalization ability of the model. In addition, training all ancient building images at one time may cause unnecessary repetitions in the classifier, which will damage the generalization effect of the model. The present invention first constrains the initialization of the input weight matrix through a specific distribution, and then assigns different attention weights according to the importance differences of the base learners. This process enables the new distribution after stacking to adapt to the building image data more flexibly, significantly improving the accuracy and generalization ability of the factory building classification model. The present invention also divides the factory building training samples into a training set and an alternative set, uses a sample selection strategy to select useful samples from the alternative set to add to the training set, and iteratively optimizes the model. This method efficiently utilizes the existing training information without adding additional training samples, reduces the bias in the selection of factory building samples, and gradually enhances the generalization performance of the classification model.

[0166] (4) Perform consistency feature matching of plant data;

[0167] (4.1) For adjacent images of the 3D scene of the same factory building, the ORB algorithm is used to detect and extract feature points in the image and their corresponding descriptors to form a matching set. Each feature point is traversed, and all feature points within a neighborhood radius r are searched with each feature point as the center to form a matching pair. The neighborhood is the circumscribed circular area of ​​a basic square cell formed with the feature point as the center;

[0168] (4.2) Calculate the total number of matching pairs C in each neighborhood j , calculate the neighborhood value:

[0169] F mn =|C j |-I;

[0170] -I stands for deleting the original matching pair operation. j ,n j ) represents each matching pair;

[0171] (4.3) The matching pairs are distributed and distinguished according to the support, and the highly similar repeated texture mismatches are removed:

[0172]

[0173] β is the matching lower threshold parameter, γ is the matching upper threshold parameter, Z is the total number of neighborhood features, RI represents the correct matching pair, E represents the wrong matching pair, and RE represents the repeated matching pair; Figure 3 As shown, n1 is a correct matching pair of n, n2 is a repeated matching pair of n, and n3 is an incorrect matching pair of n;

[0174] Finally, the correct matching set F(RI) is obtained;

[0175] (4.4) Initial matching optimization:

[0176] A random sampling consensus algorithm is implemented on the correct matching set F(RI) to remove outliers and retain all matching pairs that meet the conditions:

[0177] DMS = RANSAC[F(RI)];

[0178] RANSAC is a random sampling consensus algorithm;

[0179] Fit the basic matrix to DMS:

[0180] S1 = s(DMS);

[0181] s represents the fitting operation;

[0182] (4.5) Guide matching for each feature point in the matching set:

[0183] GuMS=|s(F mn ,S1)| <g;

[0184] s() represents the distance, and g is the distance parameter;

[0185] Retain the feature points that meet the guided matching condition GuMS;

[0186] (4.6) Perform motion consistency judgment within the threshold β, delete the wrong matching of point-to-line logic, and retain the matching pairs after judgment:

[0187]

[0188] ←To verify the calculation;

[0189] In order to significantly improve the effectiveness of three-dimensional reconstruction, the present invention introduces an innovative feature matching strategy that focuses on enhancing motion consistency through circular neighborhood segmentation. The first step is to implement a circular neighborhood segmentation technique to increase the number of preliminary feature point matches. Next, in order to improve the accuracy of matching, the present invention adopts a dual strategy: first, by strengthening the identification criteria for true and false matching points, the false matching rate between highly similar feature points is reduced; second, outlier removal is implemented to further refine the accuracy of feature point matching. In addition, the present invention also broadens the position distribution of feature point matching, thereby enhancing the stability of the three-dimensional model. This series of improvements has significantly improved the matching performance and demonstrated higher stability during the three-dimensional reconstruction process.

[0190] (5) Perform dynamic optimization of image classification and feature point matching;

[0191] (5.1) Calculate the number of correctly classified positive examples TP, the number of correctly classified negative examples TN, the number of incorrectly classified positive examples FP, and the number of incorrectly classified negative examples FN in image classification.

[0192] (5.2) Calculate the classification accuracy Acc;

[0193]

[0194] (5.3) Calculate the classification accuracy Cr;

[0195]

[0196] (5.4) Calculate feature matching accuracy PMS;

[0197]

[0198] TMS is the number of correct matches, and FMS is the number of incorrect matches;

[0199] (5.5) Dynamic adjustment of parameters:

[0200] When Acc<ε or Cr<∈, re-execute step (3.1), re-randomly average the training samples and execute the subsequent steps; ε is the classification accuracy threshold, ∈ is the classification accuracy threshold; until the classification accuracy and classification accuracy are controlled within the threshold range;

[0201] When PMS<θ, increase the γ value, re-execute step (4.3) and perform subsequent steps until the feature matching accuracy is controlled within the threshold range;

[0202] The present invention dynamically evaluates and optimizes image classification and feature point matching by calculating indicators such as classification accuracy, classification precision and feature matching accuracy. According to the evaluation results, the parameters are dynamically adjusted and the relevant steps are re-executed until the classification accuracy and classification precision are controlled within the threshold range and the feature matching accuracy is effectively improved.

[0203] (6) 3D image reconstruction:

[0204] (6.1) Using the identified matching pairs, the position of each feature point in three-dimensional space is calculated by triangulation. Triangulation is based on matching points in two or more viewpoints, and the positions of these points in the camera coordinate system are calculated to determine their coordinates in three-dimensional space.

[0205] (6.2) All feature point positions obtained through triangulation are combined to form a point cloud; a point cloud is a collection of discrete points in three-dimensional space that represents the shape and structure of the factory surface.

[0206] (6.3) Starting from the point cloud, use the surface reconstruction algorithm to generate a 3D surface model of the factory building;

[0207] Commonly used surface reconstruction algorithms include Delaunay triangulation, Poisson surface reconstruction, etc. These algorithms can generate continuous and smooth three-dimensional surfaces based on point cloud data;

[0208] (6.4) Mapping the texture information in the original image onto the reconstructed three-dimensional surface, and extracting the color information of the point to fill the three-dimensional surface;

[0209] (6.5) Output the reconstructed 3D image of the factory building into an appropriate file format (such as OBJ, PLY, etc.).

[0210] The three-dimensional reconstruction process can be achieved based on existing technical means such as point cloud data, and will not be elaborated here.

[0211] In addition, if Figure 2As shown, the present invention provides a dynamic optimization system for three-dimensional reconstruction of plant images based on deep learning, including: a data preparation and preprocessing module; a random feature learner model construction module; an image classification and recognition module; a feature matching module; a dynamic optimization module and a three-dimensional image reconstruction module. The system uses the above-mentioned dynamic optimization method for three-dimensional reconstruction of plant images based on deep learning to perform the three-dimensional reconstruction task of plant images. The system realizes a fully automated process from data preparation to three-dimensional image reconstruction through deep learning technology. The deep learning model can automatically extract key features in the image, classify and identify them, reducing the need for manual intervention. The random feature learner model can learn complex features in the image, improving the accuracy of image classification and recognition. The feature matching module ensures the accuracy and stability of three-dimensional reconstruction through an accurate feature matching algorithm. The dynamic optimization module can adjust the reconstruction process in real time according to actual conditions, further improving the robustness of the system. The deep learning model can learn efficient feature representation during the training process, thereby accelerating the speed of image processing and three-dimensional reconstruction. The system adopts a modular design, and the modules work together to improve the overall processing efficiency. In terms of real-time performance, the system can respond quickly to input data and generate high-quality three-dimensional images to meet the time requirements in practical applications. The system has good scalability and can add new functional modules or optimize existing modules according to actual needs. The deep learning model has strong generalization ability and can handle the three-dimensional reconstruction tasks of factory images in different scenarios. The flexibility of the system is also reflected in its adaptability to input data, and it can process images with different resolutions and different lighting conditions. Compared with traditional three-dimensional reconstruction methods, the system reduces the need for professional equipment and manual operation, thereby reducing costs. The characteristics of automation and intelligence enable the system to operate continuously and efficiently, improving work efficiency and economic benefits.

[0212] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A dynamic optimization method for three-dimensional reconstruction of factory images based on deep learning, characterized in that: The following steps are involved: (1) Data preparation and preprocessing: Collect multi-angle image data of the factory to ensure that the images cover all important areas of the factory; Preprocess the collected image data, including removing noise, enhancing image contrast, and correcting image color, to obtain a preprocessed plant image; (2) Construct a random feature learner model: initialize the input weight matrix according to different specific distributions and impose constraints on the input weight matrix; assign corresponding attention weights according to the importance of each base learner; Constructing a base learner, including creating an input weight matrix that follows normal distribution and uniform distribution, constructing hidden layer output, and obtaining an output weight matrix through pseudo-inverse learning, thereby obtaining the base learner output; constructing a meta learner, obtaining the meta learner input weight matrix by calculating the similarity between the label matrix and the base learner network output, and performing single hidden layer stack pseudo-inverse learning to obtain the meta learner network output; (3) Image classification and recognition using active progressive learning: The preprocessed plant images are randomly and evenly divided into a training set and a candidate set; the base learner and the meta learner are constructed and trained using the training set; plant image samples in the candidate set are iteratively selected and re-added to the training set, and sample selection and category reallocation are performed based on the output of the meta learner; Calculate the fuzziness of the candidate samples, build a fuzziness collection, and perform iterative classification until the iteration ends. Classify the images with different plant structure distributions to obtain the final plant image classification results; (4) Consistent feature matching of factory data: For adjacent images of the same factory building in the factory image classification results, the ORB algorithm is used to detect and extract feature points and their descriptors; the total number of matching pairs and the neighborhood value in the neighborhood of each feature point are calculated; correct matching pairs and incorrect matching pairs are distinguished according to the support, and incorrect matching of highly similar repeated textures is removed; a random sampling consensus algorithm is implemented on the correct matching set to remove outliers; guided matching and motion consistency judgment within the threshold are performed on the matching set, and matching pairs that meet the conditions are retained; (5) Dynamic optimization of image classification and feature point matching: Calculate the number of correct and incorrect divisions in image classification, and calculate the classification accuracy, classification precision, and feature matching accuracy; dynamically adjust parameters according to the classification accuracy, classification precision, and feature matching accuracy until these indicators are controlled within the preset threshold range, and obtain the optimized matching pairs that meet the conditions; (6) Three-dimensional image reconstruction: Use the optimized matching pairs that meet the conditions to calculate the position of each feature point in the three-dimensional space through triangulation method; combine the positions of all feature points into a point cloud; based on the point cloud, use the surface reconstruction algorithm to generate a three-dimensional surface model of the factory; map the texture information in the original image onto the three-dimensional surface, and extract the color information to fill the three-dimensional surface; output the reconstructed three-dimensional image of the factory to an appropriate file format.

2. According to the method of dynamic optimization of three-dimensional reconstruction of factory building images based on deep learning in claim 1, it is characterized in that: The construction of the random feature learner model specifically includes: (2.1) Constructing base learners: Create different input weight matrices that follow normal distribution and uniform distribution, construct hidden layer outputs, obtain output weight matrices through pseudo-inverse learning, and obtain base learner outputs; (2.1.1) Create i different input weight matrices K that follow a specific distribution i , K i is any distribution; set i = 3 to generate 3 K with different specific distributions i ; K1 is a normal distribution with an expected value of μ1 and a standard deviation of δ1: K2 is a uniform distribution with an interval of [p,q]: K2~U(p,q); K3 is a normal distribution with an expected value of μ2 and a standard deviation of δ2: Among them, U is uniform distribution and N is normal distribution; (2.1.2) Construct the hidden layer output U i : U i =ξ(L×K i ); Wherein L is the input matrix, which is the input image data in the present invention, and ξ is the activation function; (2.1.3) For the input matrix L, collect the label of each image, construct the label matrix G, and obtain the output weight matrix through pseudo-inverse learning: (2.1.4) Get the base learner network output: M i =O i ×U i ; (2.2) Constructing a meta-learner: Obtain the meta-learner input weight matrix by calculating the similarity between the label matrix and the base learner network output, perform single hidden layer stacking pseudo-inverse learning based on the meta-learner input weight matrix and the base learner network output, obtain the hidden layer output after stacking, and obtain the meta-learner network output through the meta-learner output weight matrix after stacking; (2.2.1) Calculate the label matrix G and the base learner network output M i The similarity between: (2.2.2) The meta-learner input weight matrix is ​​obtained through the logical normalization function: (2.2.3) Perform pseudo-inverse learning of a single hidden layer stack, and obtain the hidden layer output after stacking according to the meta-learner input weight matrix and the base learner network output: (2.2.4) According to pseudo-inverse learning, the output weight matrix of the stacked meta-learner is obtained: (2.2.5) Get the meta-learner network output: M f =U f ×O f 。 3. The method for dynamic optimization of three-dimensional reconstruction of factory images based on deep learning according to claim 1, characterized in that: The active progressive learning of image classification and recognition specifically includes: (3.1) The preprocessed factory images are randomly divided into training set L as training samples. tr and the candidate set L ca , through the training set L tr Use (2.1.1)-(2.1.3) to train the base learner and get the initial weight matrix O i-tr , training L ca Get the meta-learner output M f-ca ; for M f-ca For sample selection, select L s Factory image samples are added back to the training set L tr Iterate in and get the meta-learner output M again f-ca ; (3.2) The meta-learner output for P categories and T candidate samples is Select The index value y corresponding to the maximum value of each column in To perform randomization: rand(P,1) is a random vector with P rows and 1 column; (3.3) Reassign each factory building set to the yth class and obtain the class marginal probability distribution matrix: i=1,2,…,P, P represents the number of factory structure categories, j=1,2,…,T, T represents the number of candidate samples, Indicates M f-ca The i-th row and j-th column of ; (3.4) Calculate the fuzziness of P categories and T candidate samples: (3.5) Construct fuzzy set I: I=[R(M f-ca ),M f-ca ,G f-ca ]; G f-ca M f-ca The actual class label; (3.6) Iterate classification: According to R(M f-ca ) value to sort I in descending order, if M f-ca ≠G f-ca , then it is an incorrectly classified sample. Select the first s incorrectly classified samples and add them to L tr In the process, the training set and candidate set are reconstructed, and the training is repeated z times until the iteration ends to obtain the final factory image classification result.

4. The method for dynamic optimization of three-dimensional reconstruction of factory images based on deep learning according to claim 1, characterized in that: The plant data consistency feature matching specifically includes: (4.1) For adjacent images of the same factory building in the factory building image classification results, the ORB algorithm is used to detect and extract feature points in the image and their corresponding descriptors to form a matching set, and each feature point is traversed to find all feature points within a neighborhood radius r with each feature point as the center and form a matching pair. The neighborhood is the circumscribed circular area of ​​a basic square cell formed with the feature point as the center; (4.2) Calculate the total number of matching pairs C in each neighborhood j , calculate the neighborhood value: F mn =|C j |-I; -I stands for deleting the original matching pair operation. j ,n j ) represents each matching pair; (4.3) The matching pairs are distributed and distinguished according to the support, and the highly similar repeated texture mismatches are removed: β is the matching lower threshold parameter, γ is the matching upper threshold parameter, Z is the total number of neighborhood features, RI represents the matching pair is a correct matching pair, E represents the matching pair is an incorrect matching pair, and RE represents a repeated matching pair; Finally, the correct matching set F(RI) is obtained; (4.4) Initial matching optimization: A random sampling consensus algorithm is implemented on the correct matching set F(RI) to remove outliers and retain all matching pairs that meet the conditions: DMS = RANSAC[F(RI)]; RANSAC is a random sampling consensus algorithm; Fit the basic matrix to DMS: S1 = s(DMS); s represents the fitting operation; (4.5) Guide matching for each feature point in the matching set: GuMS=|s(F mn ,S1)|<g; s() represents the distance, and g is the distance parameter; Retain the feature points that meet the guided matching condition GuMS; (4.6) Perform motion consistency judgment within the threshold β, delete the wrong matching of point-to-line logic, and retain the matching pairs after judgment: ←To verify the calculation.

5. The method for dynamic optimization of three-dimensional reconstruction of factory images based on deep learning according to claim 1, characterized in that: The image classification and feature point matching dynamic optimization specifically include: (5.1) Calculate the number of images correctly classified as positive examples TP, the number of images correctly classified as negative examples TN, the number of images incorrectly classified as positive examples FP, and the number of images incorrectly classified as negative examples FN in image classification; (5.2) Calculate the classification accuracy Acc; (5.3) Calculate the classification accuracy Cr; (5.4) Calculate feature matching accuracy PMS; TMS is the number of correct matches, and FMS is the number of incorrect matches; (5.5) Dynamic adjustment of parameters: When Acc<ε or Cr<∈, re-execute step (3.1), re-randomly average the training samples and execute the subsequent steps; ε is the classification accuracy threshold, is the classification accuracy threshold; until the classification accuracy and classification accuracy are controlled within the threshold range; When PMS<θ, increase the value of γ, re-execute step (4.3) and perform subsequent steps until the feature matching accuracy is controlled within the threshold range.

6. The method for dynamic optimization of three-dimensional reconstruction of factory images based on deep learning according to claim 1, characterized in that: The three-dimensional image reconstruction specifically includes: (6.1) Using the identified matching pairs, the position of each feature point in three-dimensional space is calculated by triangulation. Triangulation is based on matching points in two or more viewpoints, and the positions of these points in the camera coordinate system are calculated to determine their coordinates in three-dimensional space. (6.2) All feature point positions obtained through triangulation are combined to form a point cloud; a point cloud is a collection of discrete points in three-dimensional space that represents the surface shape and structure of the factory building and its internal facilities. (6.3) Starting from the point cloud, use the surface reconstruction algorithm to generate a 3D surface model of the factory building; (6.4) Mapping the texture information in the original image onto the reconstructed three-dimensional surface, and extracting the color information of the point to fill the three-dimensional surface; (6.5) Output the reconstructed 3D image of the factory building into an appropriate file format.

7. A dynamic optimization system for three-dimensional reconstruction of factory images based on deep learning, characterized in that: include: Data preparation and preprocessing module: used to collect multi-angle image data of the factory and ensure that the images cover all important parts of the factory; Preprocessing the image, including denoising, contrast enhancement and color correction; A random feature learner model building module, including a base learner building unit and a meta learner building unit, wherein the base learner building unit is used to create an input weight matrix following different specific distributions, build a hidden layer output, and obtain an output weight matrix through pseudo-inverse learning, thereby obtaining a base learner output; The meta-learner construction unit obtains the meta-learner input weight matrix by calculating the similarity between the label matrix and the base learner network output, performs pseudo-inverse learning of a single hidden layer stack, obtains the hidden layer output after stacking, and obtains the meta-learner network output through the meta-learner output weight matrix after stacking; Image classification and recognition module: The pre-processed plant images are randomly divided into training sets and candidate sets as training samples; the training sets are used to train the base learner and meta-learner; sample selection is performed, and the selected plant image samples are added back to the training set for iteration until the iteration end condition is met to obtain the final plant image classification result; Feature matching module: Use the ORB algorithm to detect and extract feature points and their corresponding descriptors in adjacent images of the 3D scene of the same factory building; traverse each feature point, build matching pairs, and calculate the total number of matching pairs and neighborhood values ​​in the neighborhood; distinguish correct matching pairs from incorrect matching pairs based on support, and remove incorrect matching of highly similar repeated textures; implement a random sampling consensus algorithm on the correct matching set to remove outliers, and perform guided matching and motion consistency judgment to retain the identified matching pairs; Dynamic optimization module: calculates the number of images correctly classified as positive examples, negative examples, and incorrect classifications in image classification; Calculate the classification accuracy, classification precision and feature matching accuracy; dynamically adjust the parameters according to the classification accuracy, classification precision and feature matching accuracy until they are within the set threshold range; 3D image reconstruction module: Use the identified matching pairs to calculate the position of each feature point in 3D space through triangulation method; combine all feature point positions obtained through triangulation into a point cloud; use the surface reconstruction algorithm to generate a 3D surface model of the factory; map the texture information in the original image to the reconstructed 3D surface, and extract color information to fill the 3D surface; output the reconstructed 3D image of the factory to an appropriate file format.

8. The deep learning-based dynamic optimization system for three-dimensional reconstruction of factory images according to claim 7, characterized in that: Any one of the methods in claims 1-6 is used to perform the three-dimensional reconstruction task of the factory building image.

Images