Modeling optimization processing method and system based on digital image

Abstract

The invention discloses a modeling optimization processing method and system based on a digital image, and relates to the technical field of three-dimensional modeling, and the method comprises the steps: using a lightweight Vision Transform to extract global features, using a Harris corner point to detect and recognize feature points, generating a multi-scale feature map through Laplace decomposition, generating a semantic segmentation map in combination with DeepLabv3 +, constructing a hybrid Poisson equation, and carrying out solution optimization. A smooth three-dimensional surface model is generated through a Marching-Cubes algorithm, an initial contour is adjusted, optimization is carried out in combination with a dynamic change vector, and a final optimization model is generated through completion, denoising, simplification, quantification and compression processing. According to the method, the feature extraction quality is enhanced through standardization processing and Vision Transform, the initial model precision is improved through multi-scale fusion and semantic segmentation, efficient and refined modeling is realized through mixing Poisson optimization and multi-scale adjustment, and a high-quality and low-complexity three-dimensional model is generated.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and in particular to a modeling optimization processing method and system based on digital images. Background Technology

[0002] With the rapid development of computer vision and deep learning technologies, multimodal data processing methods based on RGB images, depth maps, point cloud data, etc., are widely used in 3D reconstruction and object recognition. Traditional methods usually extract and classify image features through feature point detection, edge detection, and classic machine learning models, and combine stereographic mesh or triangulation techniques to achieve 3D modeling. Deep learning models have shown powerful capabilities in feature extraction and semantic segmentation. The application of DeepLabv3+ models in semantic segmentation has significantly improved segmentation accuracy, while optimization methods based on optical flow algorithms and Poisson equations have been widely used in dynamic scene modeling.

[0003] However, existing technologies still have shortcomings. When dealing with complex topological structures, existing Marching-Cubes surface reconstruction algorithms fail to optimize mesh generation from the perspective of global geometric consistency, which can easily lead to mesh distortion or redundant triangular faces, increasing computational complexity and storage costs, and making it difficult to meet the needs of real-time rendering or resource-constrained scenarios. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a modeling optimization processing method and system based on digital images, which solves the problem that the existing Marching-Cubes surface reconstruction algorithm fails to optimize mesh generation from the perspective of global geometric consistency when dealing with complex topological structures, which easily leads to mesh distortion or redundant triangular faces, increases computational complexity and storage costs, and makes it difficult to meet the needs of real-time rendering or resource-constrained scenarios.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a modeling and optimization processing method based on digital images, comprising,

[0008] Raw digital image data and point cloud data are collected and preprocessed to obtain an enhanced data map. A lightweight Vision Transformer is used to extract global features, and a context feature map is generated through an attention mechanism. Harris corner detection is used to identify feature points, a multi-radius sphere is constructed, gradients are calculated using the Sobel operator, a dual dynamic threshold is generated, an enhanced gradient set is generated through filtering, the dimensional gradient histogram feature vector is calculated, the nearest neighbor is selected based on Euclidean distance to obtain line constraints, a preliminary point feature set is generated, an edge map is generated using the Canny algorithm, the dual dynamic threshold is optimized to generate enhanced line constraints, the significance score of the feature vector is calculated, and a refined feature set is generated through filtering.

[0009] Multi-scale feature maps are generated through Laplacian decomposition, fusion weights are calculated, fusion feature maps are generated, and enhanced line constraints are used for correction. Feature point sets are extracted based on gray-level invariant matrices, clustering is performed using K-means, semantic segmentation maps are generated by combining DeepLabv3+, category labels are mapped, local grids are aligned based on global coordinate system, and overlapping regions are merged to generate a corrected initial model.

[0010] Based on the calibrated fusion feature map, the gradient distribution features and the dynamic change vector of the Horn-Schunck optical flow are calculated. A hybrid Poisson equation is constructed and solved for optimization. A smooth 3D surface model is generated by the Marching-Cubes algorithm. The initial contour is adjusted based on the multi-scale feature map and optimized by combining the dynamic change vector to generate a refined 3D model. The final optimized model is generated through completion, denoising, simplification, quantization and compression processing.

[0011] As a preferred embodiment of the modeling and optimization processing method based on digital images described in this invention, the generation of the refined feature set includes:

[0012] Based on the enhanced data graph, a lightweight Vision Transformer model is used for global feature extraction, which is then concatenated into a feature vector set. Combined with a local saliency weight map, a context feature map is generated through an attention mechanism. The region is then segmented using a threshold segmentation method to output highly saliency regions. Harris corner detection is used for identification to obtain feature points, which are defined as center points, and a multi-radius sphere is constructed.

[0013] The Sobel operator is used to calculate the gradient of standardized image data in a multi-radius sphere, calculate the dual dynamic threshold, generate an enhanced gradient set by horizontal arrangement, calculate the dimensional gradient histogram feature vectors, concatenate the gradient histogram feature vectors within the dimension, generate feature vectors for feature points, calculate the distance between feature points using the Euclidean distance formula, set the nearest neighbor, use the enhanced gradient set for validation, filter to generate line constraints that meet the validation conditions, concatenate with feature vectors to generate a preliminary point-line feature set, use line constraints to calculate the initial edge mask, combine with the edge map to optimize the dual dynamic threshold, generate optimized line constraints, fuse the edge map to generate enhanced line constraints, obtain the enhanced feature set, calculate the significance score of the feature vectors, and generate a refined feature set through filtering.

[0014] As a preferred embodiment of the modeling and optimization processing method based on digital images described in this invention, the step of merging overlapping regions to generate a corrected initial model includes:

[0015] Based on the refined feature set, feature vectors are extracted, scale classification is performed, and multi-scale feature maps are generated through Laplacian decomposition. These feature maps are then stitched together with enhanced line constraints to generate a multi-scale feature map set.

[0016] Based on the context feature map, the fusion weight is calculated, and the multi-scale feature map set is combined for weighted fusion to generate a fusion feature map. The enhanced line constraint is used for correction to generate a corrected fusion feature map.

[0017] Based on the gray-level invariant matrix, feature point sets are extracted from the calibration fusion feature map. K-means clustering is used to cluster the feature point sets. The minimum bounding matrix is ​​calculated using the rotating caliper algorithm. The calibration fusion feature map is cropped to generate image patches. These patches are then combined with their corresponding clustered feature point sets to generate object feature pairs. Finally, the DeepLabv3+ model is used to generate a semantic segmentation map.

[0018] The clustered feature point set is mapped to the semantic segmentation map using the direct coordinate mapping method to obtain the category label of the clustered feature point on the semantic segmentation map. The local point cloud and depth value are calculated, and the local point cloud is converted into a local mesh using Delaunay triangulation.

[0019] Based on the clustered feature point set, the local grid and category labels are spatially aligned using a global coordinate system, and overlapping regions are merged to generate an initial model. Then, affine transformation is used for correction to generate a corrected initial model.

[0020] As a preferred embodiment of the modeling and optimization processing method based on digital images described in this invention, the step of constructing a hybrid Poisson equation and solving it for optimization includes:

[0021] Based on the calibrated fusion feature map, the gradient distribution features are calculated. Using the Horn-Schunck optical flow algorithm, the dynamic change vector is calculated through the calibrated fusion feature map, and a hybrid Poisson equation is constructed. After discretization, a linear equation system is obtained. Sparse decomposition is performed using Cholesky decomposition, and the linear equation system is solved using preconditional conjugate gradients. The solution stops when the convergence condition is met, and the optimized discrete surface index function vector is output.

[0022] As a preferred embodiment of the modeling and optimization processing method based on digital images described in this invention, the step of combining dynamic change vectors for optimization to generate a refined 3D model includes:

[0023] The optimized discrete surface index function vector is converted into a mesh using the Marching-Cubes algorithm, and then mapped to the calibration initial model through affine transformation to obtain a smooth 3D surface model. Boundary point sets are extracted through surface normal vectors and curvature analysis, projected onto the calibration fusion feature map to generate an initial contour, and the local saliency weights at each scale are calculated to adjust the initial contour.

[0024] Based on the dynamically changing vector, the error prediction direction is calculated by combining the source point cloud and the target point cloud. The objective function is defined, and the registration point cloud is generated by ICP iteration. The Delaunay triangulation is used to reconstruct the 3D mesh, generating a refined 3D model.

[0025] As a preferred embodiment of the modeling and optimization processing method based on digital images described in this invention, the step of generating the final optimized model through completion, denoising, simplification, quantization, and compression processing includes:

[0026] Based on the refined 3D model, a 3D edge detection algorithm is used for boundary completion, and Laplacian smoothing filtering is used for noise filtering. A simplified model is generated through a pruning algorithm, quantization is performed through 8-bit uniform quantization, and compression is performed through PCA to generate the final optimized model.

[0027] As a preferred embodiment of the modeling and optimization processing method based on digital images described in this invention, the step of collecting original digital image data and point cloud data and performing preprocessing to obtain an enhanced data map includes:

[0028] Raw digital image data and point cloud data are collected through API interfaces to generate standardized image data. A two-dimensional entropy detection algorithm is used to calculate the local saliency weight map of the standardized data, and enhanced data maps are generated through filtering and processing.

[0029] Secondly, the present invention provides a modeling and optimization processing system based on digital images, comprising,

[0030] The preprocessing and feature extraction module collects and preprocesses raw digital image data and point cloud data to obtain an enhanced data map. It uses a lightweight Vision Transformer to extract global features, generates a context feature map through an attention mechanism, identifies feature points using Harris corner detection, constructs a multi-radius sphere, calculates gradients using the Sobel operator, generates dual dynamic thresholds, generates an enhanced gradient set through filtering, calculates the dimensional gradient histogram feature vector, filters nearest neighbor points based on Euclidean distance to obtain line constraints, generates a preliminary point feature set, generates an edge map using the Canny algorithm, optimizes the dual dynamic thresholds, generates enhanced line constraints, calculates the saliency score of the feature vectors, and generates a refined feature set through filtering.

[0031] The multi-scale fusion and modeling module is used to generate multi-scale feature maps through Laplacian decomposition, calculate fusion weights, generate fused feature maps, perform correction using enhanced line constraints, extract feature point sets based on gray-level invariant matrices, perform clustering through K-means, generate semantic segmentation maps in conjunction with DeepLabv3+, map category labels, align local grids based on global coordinate systems, and merge overlapping regions to generate a corrected initial model.

[0032] The model optimization and refinement module is used to calculate gradient distribution features and Horn-Schunck optical flow dynamic change vectors based on the correction fusion feature map, construct a hybrid Poisson equation, solve and optimize it, generate a smooth 3D surface model through the Marching-Cubes algorithm, adjust the initial contour based on the multi-scale feature map, and optimize it in combination with the dynamic change vector to generate a refined 3D model. Through completion, denoising, simplification, quantization and compression processing, the final optimized model is generated.

[0033] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the modeling and optimization processing method based on digital images as described in the first aspect of the present invention.

[0034] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the modeling and optimization processing method based on digital images as described in the first aspect of the present invention.

[0035] The beneficial effects of this invention are as follows: This invention uses a lightweight Vision Transformer to extract global features, uses Harris corner detection to identify feature points, generates multi-scale feature maps through Laplacian decomposition, combines DeepLabv3+ to generate semantic segmentation maps, constructs a hybrid Poisson equation, performs optimization by solving it, generates a smooth 3D surface model through the Marching-Cubes algorithm, adjusts the initial contour, optimizes it by combining dynamic change vectors, and generates the final optimized model through completion, denoising, simplification, quantization, and compression processing; it enhances the quality of feature extraction, improves the accuracy of the initial model, achieves efficient and refined modeling, and generates a high-quality and low-complexity 3D model. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of the modeling and optimization processing method based on digital images in Example 1.

[0038] Figure 2 This is a schematic diagram of the modeling and optimization processing system based on digital images in Example 1. Detailed Implementation

[0039] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0040] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0041] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0042] Example 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a modeling and optimization processing method based on digital images, including the following steps:

[0043] S1. Collect raw digital image data and point cloud data and preprocess them to obtain an enhanced data map. Use a lightweight Vision Transformer to extract global features, generate a context feature map through an attention mechanism, use Harris corner detection to identify feature points, construct a multi-radius sphere, use the Sobel operator to calculate gradients, generate dual dynamic thresholds, generate an enhanced gradient set through filtering, calculate the dimensional gradient histogram feature vector, filter nearest neighbor points based on Euclidean distance to obtain line constraints, generate a preliminary point feature set, use the Canny algorithm to generate an edge map, optimize the dual dynamic thresholds, generate enhanced line constraints, calculate the significance score of the feature vectors, and generate a refined feature set through filtering.

[0044] Specifically, raw digitized image data and point cloud data are collected and preprocessed to obtain enhanced data maps, including:

[0045] Raw digital image data and point cloud data are collected through API interfaces;

[0046] The raw digital image data includes RGB images, depth maps, and remote sensing image data;

[0047] Based on the original digital image data, bilinear interpolation is used to normalize the resolution, and the RGB image data is converted into HSV images through the RGB-to-HSV algorithm. The depth map data is subjected to grayscale normalization, and the remote sensing image data is subjected to channel normalization to generate a standardized two-dimensional image.

[0048] Based on point cloud data, standardized point cloud data is generated by projecting it onto a stereo pixel grid through trilinear interpolation. After normalization, the data is converted into pseudo-image data through a coordinate system. Combined with a standardized two-dimensional image, the data is stitched together to form standardized image data. A three-dimensional global coordinate system is then constructed using the SGBM algorithm.

[0049] The local saliency weight map of standardized data is calculated using a two-dimensional entropy detection algorithm.

[0050] For maps with local saliency weights greater than those with high saliency weights, Gaussian difference filtering is used to enhance edge details. For maps with local saliency weights less than those with low saliency weights, median filtering is used to reduce noise, generating an enhanced data map (both high and low saliency weight maps are set using statistical quantiles).

[0051] Through standardized API interfaces, the compatibility and consistency of data acquisition are guaranteed, providing a high-quality input foundation for subsequent processing. Customized processing for data characteristics significantly improves the quality of data standardization, preserves the spatial details of point clouds, and the innovative design of pseudo-image generation and 2D image stitching enables subsequent feature extraction algorithms to process multimodal data simultaneously, significantly improving the accuracy of feature fusion. Through saliency weight partitioning, the accuracy of edge detection is improved, the noise level is reduced, the details and textures of high saliency areas are enhanced, and the data quality is ensured.

[0052] Furthermore, the generation of the refined feature set includes:

[0053] Based on the enhanced data graph, a lightweight Vision Transformer model is used to extract global features and concatenate them into a feature vector set.

[0054] Based on the feature vector set and combined with the local saliency weight map, a context feature map is generated through an attention mechanism;

[0055] Based on the context feature map, a saliency threshold is set using empirical rules, and the threshold segmentation method is used to segment and output highly saliency regions.

[0056] Based on highly salient regions, Harris corner detection is used for identification to obtain feature points, such as surface texture points of cultural relics and corner points of building boundaries;

[0057] Define the feature points as center points, set the radius based on historical experience, and construct a multi-radius sphere.

[0058] The gradient of the normalized image data in a multi-radius sphere is calculated using the Sobel operator, and the dual dynamic threshold is calculated using the following formula:

[0059]

[0060] T low =c·T high ,

[0061] Among them, T high This is the higher threshold in the dynamic dual threshold method. To normalize the gradient magnitude of image data I, T is calculated based on the Sobel operator. low is the lower threshold in the dynamic dual threshold, c is a fixed proportional coefficient set based on experience, and med is the median extraction operation;

[0062] Select gradients that are greater than or equal to the lower threshold of the dual dynamic thresholds and arrange them horizontally to generate an enhanced gradient set;

[0063] Based on the enhanced gradient set, the gradient histogram feature vector of the dimension is calculated, and the gradient histogram feature vectors in the dimension are concatenated to generate the feature vector of the feature point.

[0064] Based on highly salient regions, the distance between feature points is calculated using the Euclidean distance formula. These points are then sorted in ascending order, and the feature points with the smallest distance are selected as nearest neighbors. An enhanced gradient set is used for validation, and generated line constraints that meet the validation criteria are selected. The formula is as follows:

[0065] l j ={(p i ,p m ,p n )∣d(p i ,p m )≤d(p i ,p n )≤θ,(p m ,p n )∈G},

[0066] Among them, l j Let be the constraint of the j-th line, representing the connection relationship between the i-th feature point p, the m-th feature point p, and the n-th feature point p, where d is the distance between feature points, θ is the distance threshold set based on the maximum radius value of the multi-radius sphere, and G is the set of enhanced gradients.

[0067] The line constraints and feature vectors are concatenated to generate a preliminary point-line feature set;

[0068] Based on the initial point and line feature set, the initial edge mask is calculated using line constraints, as follows:

[0069]

[0070] Where M(x,y) is the initial edge mask for the horizontal coordinate x and the vertical coordinate y, Path is the linear interpolation path calculated based on the equation of a straight line, and oth is the other way around.

[0071] Based on standardized image data, the Canny algorithm is used to generate an edge map, and the initial edge mask is used to optimize the dual dynamic thresholds to generate optimized line constraints. The formula is as follows:

[0072]

[0073] Among them, T 1 high To optimize the high threshold in dual dynamic thresholding;

[0074] The optimized line constraints and edge map are fused to generate enhanced line constraints, as shown in the formula:

[0075]

[0076] Where L represents the enhanced line constraint, E represents the edge map, and L 1 To optimize line constraints;

[0077] Replace the line constraints in the initial point and line feature set with enhanced line constraints to generate an enhanced feature set;

[0078] Based on the enhanced feature set, the significance score of the feature vector is calculated using the following formula:

[0079]

[0080] Among them, S δ v is the significance score for the δ-th eigenvector. k Let be the absolute value of the k-th component of the δ-th eigenvector V, and c be the number of components of the eigenvector.

[0081] Feature vectors with significance scores greater than a significance score threshold (based on statistical distribution) and their corresponding enhanced line constraints are selected to generate a refined feature set.

[0082] By extracting global features using a lightweight Vision Transformer and combining it with an attention mechanism to highlight contextual relevance, the robustness of the features is significantly improved. The introduction of dual dynamic thresholds addresses the poor adaptability of traditional fixed threshold methods to complex scenes by dynamically adjusting high and low thresholds. Enhanced line constraints are achieved by fusing the Canny algorithm and the initial edge mask, further optimizing edge detection accuracy. The generated highly saliency feature set exhibits higher accuracy and stability in recognition. By setting thresholds using a saliency scoring formula combined with statistical distribution, objective feature selection is achieved, providing a quantitative saliency evaluation standard. This significantly reduces the impact of redundant features while retaining key feature points, improving the refinement and application value of the feature set.

[0083] S2. Generate multi-scale feature maps through Laplacian decomposition, calculate fusion weights, generate fusion feature maps, use enhanced line constraints for correction, extract feature point sets based on gray-level invariant matrix, perform clustering through K-means, combine with DeepLabv3+ to generate semantic segmentation maps, map category labels, align local grids based on global coordinate system, and merge overlapping regions to generate a corrected initial model.

[0084] Specifically, merging overlapping regions to generate a corrected initial model includes:

[0085] Based on the refined feature set, feature vectors are extracted. High-resolution and low-resolution thresholds are set based on scene complexity. Resolutions greater than or equal to the high-resolution threshold are selected, and a high-scale range is set. A medium-scale range is set. A medium-scale range is set. A low-scale range is set. A resolution greater than the low-resolution threshold is selected.

[0086] Based on scale classification, Laplacian decomposition is applied to the feature vectors in the refined feature set to generate multi-scale feature maps.

[0087] Multi-scale feature maps and enhanced line constraints are concatenated to generate a multi-scale feature map set;

[0088] Based on the context feature map, the fusion weight is calculated using the following formula:

[0089]

[0090] in, For the first Fusion weights at each scale For the context feature map in the 1st The projections at each scale are obtained through downsampling, where ∈ is the global contrast factor, and high, mid, and low represent high, mid, and low scales, respectively.

[0091] Based on the fusion weight, a fusion feature map is generated by weighted fusion of multi-scale feature maps.

[0092] Based on the fused feature map, correction is performed using enhanced line constraints to generate a corrected fused feature map, as shown in the formula:

[0093] F′=F fused +β·PPHT(F,L),

[0094] Where F′ is the corrected fusion feature map, β is the corrected weight, PPHT is the asymptotic probabilistic Hough transform, and F is the fusion feature map;

[0095] Based on the gray-level invariant matrix, the feature point set in the correction fusion feature map is extracted. The number of clusters K is set based on the elbow rule, and K-means clustering is used to cluster the feature point set to generate K clusters. Each cluster corresponds to a type of item. For example, K-1 clusters correspond to cultural relics.

[0096] The minimum bounding matrix is ​​calculated by the rotating caliper algorithm, the corrected fusion feature map is cropped to generate image patches, and then combined with the corresponding clustered feature point set to generate object feature pairs.

[0097] Based on image patches, a semantic segmentation map is generated using the DeepLabv3+ model;

[0098] The direct coordinate mapping method is used to map the cluster feature point set to the semantic segmentation map, thereby obtaining the category labels of the cluster feature points on the semantic segmentation map;

[0099] Based on category labels, local point cloud and depth values ​​are calculated. Delaunay triangulation is then used to convert the local point cloud into a local mesh, as shown in the formula:

[0100]

[0101] in, For the first Local point cloud of clustered feature points For the first The depth values ​​of the cluster feature points are calculated based on the SGBM algorithm. for The category labels of the cluster feature points;

[0102] Based on the clustered feature point set, the local grid and category labels are spatially aligned using a global coordinate system, and overlapping regions are merged to generate an initial model. Then, affine transformation is used for correction to generate a corrected initial model.

[0103] Scale classification allows for targeted allocation of computational resources in subsequent processing, effectively balancing computational efficiency and information preservation. This avoids feature extraction bias caused by scale mismatch, improving the robustness and specificity of feature extraction. Multi-scale feature maps generated through Laplacian decomposition better preserve contour and detail information, providing richer underlying features for subsequent accurate segmentation and localization. This addresses the problem of unreasonable feature fusion weight allocation in existing technologies, improving the quality of the fused feature maps and providing more accurate and robust input for subsequent line constraint correction and feature point extraction. It also reduces noise introduced by erroneous fusion. Calculating and cropping the minimum bounding rectangle improves computational efficiency. Delaunay triangulation using local point clouds with class labels improves the spatial consistency and integrity of the final initial model. Affine transformation is applied for correction, enhancing the model's quality and usability.

[0104] S3. Based on the corrected fusion feature map, calculate the gradient distribution features and the dynamic change vector of the Horn-Schunck optical flow, construct the hybrid Poisson equation, solve and optimize it, generate a smooth three-dimensional surface model through the Marching-Cubes algorithm, adjust the initial contour based on the multi-scale feature map, optimize it in combination with the dynamic change vector, generate a refined three-dimensional model, and generate the final optimized model through completion, denoising, simplification, quantization and compression processing.

[0105] Specifically, further, a hybrid Poisson equation is constructed and optimized for solution, including:

[0106] Based on the calibrated fusion feature map, the gradient distribution feature is calculated using the following formula:

[0107]

[0108] Where f represents the gradient distribution characteristic. To correct the fusion feature map F ′ The gradient vector is calculated based on the Sobel operator;

[0109] Using the Horn-Schunck optical flow algorithm, the dynamic change vector is calculated by correcting and fusing the feature map, as shown in the formula:

[0110] V = HS(F ′ (t),F ′ (t-1)),

[0111] Where V is the dynamic change vector, HS is the Horn-Schunck algorithm, and F ′ (t-1) is the corrected fusion map at time t-1;

[0112] Based on the category labels of the corrected initial model, the Poisson optimization parameters are initialized. For example, the optical flow weight is increased for cultural relic scenes and decreased for architectural scenes.

[0113] Based on the gradient feature distribution and dynamically changing vector, and combined with the initialization of Poisson optimization parameters, a hybrid Poisson equation is constructed, as follows:

[0114]

[0115] Where φ is the surface exponential function, div(f 1 ) represents the gradient field f 1 The divergence, calculated based on the Laplace operator, is β. 1 As optical flow constraint weights, The gradient of the dynamically changing vector V is calculated based on the Sobel operator;

[0116] Based on the hybrid Poisson equations, discretization yields a system of linear equations, as shown in the following formula:

[0117] Aφ 1 =b,

[0118] Where A is the Laplacian operator matrix, obtained by discretizing the corrected fused feature map through a stereocell grid, and φ 1 Let be the discrete surface index function vector, b be the source term vector, and be the mixed Poisson equation;

[0119] The Laplacian operator matrix is ​​sparsely decomposed using Cholesky decomposition, and the linear equations are solved using preconditional conjugate gradients. The solution stops when the convergence condition is met. The output is an optimized discrete surface index function vector based on the residual norm threshold method.

[0120] By accurately quantifying gradient distribution characteristics, the subsequent hybrid Poisson equations are better able to adapt to changes in geometric details in complex scenes, preserving more details in 3D reconstruction and significantly improving the fidelity of the surface model. The Horn-Schunck optical flow algorithm is used to calculate dynamic change vectors, which significantly improves the reconstruction accuracy of the 3D model in dynamic scenes, accurately capturing motion trajectories and reducing reconstruction errors caused by motion blur. This provides reliable dynamic constraints for subsequent Poisson optimization and model refinement. Initializing Poisson optimization parameters based on category labels improves the algorithm's adaptability to different scenes, and the optimized parameters can better balance dynamic changes and geometric constraints.

[0121] Furthermore, by combining dynamic change vectors for optimization, a refined 3D model is generated, including:

[0122] The optimized discrete surface index function vector is converted into a mesh using the Marching-Cubes algorithm, and then mapped to the corrected initial model through affine transformation to obtain a smooth three-dimensional surface model.

[0123] Based on a smooth 3D surface model, boundary point sets are extracted using the Shi-Tomasi corner detection method, projected onto a corrected and fused feature map, and an initial contour is generated. The formula is as follows:

[0124] v0 = Proj(P 1 )∩M,

[0125] Where v0 is the initial contour, representing the starting point of the contour, and P 1 To smooth the boundary point set of a 3D surface model, Proj is the operation that projects the boundary point set onto the correction fusion map;

[0126] Based on the multi-scale feature map, the local saliency weight of each scale is calculated using the following formula:

[0127]

[0128] in, Let be the local significance weight for the s-th scale. Let be the significance parameter at the s-th scale, ∈ be the global contrast factor, g and a be the row index and column index respectively, and n be the value of n. 1 x n represents the number of horizontal pixels along the x-axis of the multi-scale feature map. 2 y This represents the number of vertical pixels along the y-axis of the multi-scale feature map.

[0129] The initial contour is adjusted based on local saliency weights and a smooth 3D surface model. The process stops when the maximum number of iterations is reached (based on an early stopping method). The adjusted contour is then output and set as the edge constraint of the model. The formula is as follows:

[0130]

[0131] Among them, v t For the adjusted profile at time t, To correct the gradient magnitude of the fused feature map, it is obtained based on the gradient distribution characteristics. N is the normal vector field, representing the surface normal vector of the smoothed 3D model at coordinates (x, y), obtained through surface geometric analysis (vertex normal vector estimation and principal curvature estimation). is the smoothing weight function, h is the edge weight function, and η is the Laplace operator, which represents the second derivative operation with respect to v;

[0132] Based on the dynamically changing vector, the error prediction term is calculated by combining the source point cloud (based on the mesh point setting of the smooth 3D model) and the target point cloud (based on the feature point setting in the enhanced line constraint). The formula is as follows:

[0133] EW = ∑ i′ ||V(X i′ )·(T 3 t (X i′ )-Y i″ )||,

[0134] Where EW is the prediction error term, V(X) i′ ) is the dynamic change vector of the i′-th source point cloud X, T 3 t Let Y be the spatial position change function at time t, representing the mapping of the source point cloud to the i″-th target point cloud Y through rigid and non-rigid transformations;

[0135] Based on the prediction error term, the objective function is defined as follows:

[0136] EQ t =∑ i′ ||T 3 t (X i′ )-Y i″ || 2 +λ·EW,

[0137] Among them, EQ t The objective function value at time t, where λ is the error prediction weight;

[0138] The target value is minimized using the ICP algorithm to obtain the optimized change function, which is then applied to the source point cloud through multiplication to generate the registration point cloud.

[0139] Based on the registered point cloud, Delaunay triangulation is used to reconstruct a 3D mesh, generating a refined 3D model. The formula is as follows:

[0140] QS = (GY, LK),

[0141] In this context, QS represents the refined 3D model, LK represents the category label, and GY represents the 3D mesh.

[0142] The Marching-Cubes algorithm solves the problem of accurately extracting continuous surfaces from discrete data fields, generating meshes with correct topology, and resolving inconsistencies or positional deviations between the optimization results and existing model coordinate systems, ensuring the continuity and accuracy of subsequent processing steps. The Shi-Tomasi corner detection algorithm exhibits good robustness to corners, addressing the issue of effectively identifying and utilizing key geometric features from the 3D model surface to guide contour generation. It calculates local saliency weights, solving the problem of distinguishing between important and secondary regions during contour adjustment, and mitigating the potential for contour distortion or under-adjustment due to simple global adjustments. It calculates error prediction vectors, addressing the problem of quantifying the differences between the source model and target constraints and predicting how to adjust the model to reduce these differences, achieving precise model positioning and attitude adjustment in space. Delaunay triangulation solves the technical problem of transforming geometric information after multiple rounds of optimization into a high-quality 3D mesh model, generating the final refined 3D model.

[0143] Furthermore, through completion, denoising, simplification, quantization, and compression, the final optimized model is generated, including:

[0144] Based on the refined 3D model, a 3D edge detection algorithm is used for boundary completion, and Laplacian smoothing filtering is used for noise filtering. A simplified model is generated through a pruning algorithm, and quantization is performed through 8-bit uniform quantization (8 bits, set based on actual needs). Finally, PCA is used for compression to generate the final optimized model.

[0145] By using boundary completion and noise filtering, and based on a refined model, 3D edge detection and Laplacian smoothing can not only repair model defects and eliminate noise, but also improve the geometric continuity and data purity of the model, thus solving the problems of model incompleteness and noise interference. Through pruning algorithms, the model complexity can be reduced, the model redundancy problem can be solved, and the subsequent processing and real-time applications can be facilitated. 8-bit uniform quantization and PCA compression can significantly reduce the data storage and transmission bandwidth requirements while ensuring model accuracy, thus solving the problem of excessive model data volume and achieving efficient model storage and transmission.

[0146] This embodiment also provides a modeling and optimization processing system based on digital images, including:

[0147] The preprocessing and feature extraction module collects and preprocesses raw digital image data and point cloud data to obtain an enhanced data map. It uses a lightweight Vision Transformer to extract global features, generates a context feature map through an attention mechanism, identifies feature points using Harris corner detection, constructs a multi-radius sphere, calculates gradients using the Sobel operator, generates dual dynamic thresholds, generates an enhanced gradient set through filtering, calculates the dimensional gradient histogram feature vector, filters nearest neighbor points based on Euclidean distance to obtain line constraints, generates a preliminary point feature set, generates an edge map using the Canny algorithm, optimizes the dual dynamic thresholds, generates enhanced line constraints, calculates the saliency score of the feature vectors, and generates a refined feature set through filtering.

[0148] The multi-scale fusion and modeling module is used to generate multi-scale feature maps through Laplacian decomposition, calculate fusion weights, generate fused feature maps, perform correction using enhanced line constraints, extract feature point sets based on gray-level invariant matrices, perform clustering through K-means, generate semantic segmentation maps in conjunction with DeepLabv3+, map category labels, align local grids based on global coordinate systems, and merge overlapping regions to generate a corrected initial model.

[0149] The model optimization and refinement module is used to calculate gradient distribution features and Horn-Schunck optical flow dynamic change vectors based on the correction fusion feature map, construct a hybrid Poisson equation, solve and optimize it, generate a smooth 3D surface model through the Marching-Cubes algorithm, adjust the initial contour based on the multi-scale feature map, and optimize it in combination with the dynamic change vector to generate a refined 3D model. Through completion, denoising, simplification, quantization and compression processing, the final optimized model is generated.

[0150] This embodiment also provides a computer device applicable to the modeling and optimization processing method based on digital images, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the modeling and optimization processing method based on digital images as proposed in the above embodiment.

[0151] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0152] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the modeling optimization processing method based on digital images as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0153] In summary, this invention extracts global features using a lightweight Vision Transformer, identifies feature points using Harris corner detection, generates multi-scale feature maps through Laplacian decomposition, generates semantic segmentation maps using DeepLabv3+, constructs a hybrid Poisson equation for solution optimization, generates a smooth 3D surface model using the Marching-Cubes algorithm, adjusts the initial contour, and optimizes using dynamic change vectors. Through completion, denoising, simplification, quantization, and compression processing, the final optimized model is generated. This enhances feature extraction quality, improves the accuracy of the initial model, achieves efficient and refined modeling, and generates a high-quality, low-complexity 3D model.

[0154] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A modeling and optimization processing method based on digital images, characterized in that: include, Raw digital image data and point cloud data are collected and preprocessed to obtain an enhanced data map. A lightweight Vision Transformer is used to extract global features, and a context feature map is generated through an attention mechanism. Harris corner detection is used to identify feature points, a multi-radius sphere is constructed, gradients are calculated using the Sobel operator, a dual dynamic threshold is generated, an enhanced gradient set is generated through filtering, the dimensional gradient histogram feature vector is calculated, the nearest neighbor is selected based on Euclidean distance to obtain line constraints, a preliminary point feature set is generated, an edge map is generated using the Canny algorithm, the dual dynamic threshold is optimized to generate enhanced line constraints, the significance score of the feature vector is calculated, and a refined feature set is generated through filtering. Multi-scale feature maps are generated through Laplacian decomposition, fusion weights are calculated, fusion feature maps are generated, and enhanced line constraints are used for correction. Feature point sets are extracted based on gray-level invariant matrices, clustering is performed using K-means, semantic segmentation maps are generated by combining DeepLabv3+, category labels are mapped, local grids are aligned based on global coordinate system, and overlapping regions are merged to generate a corrected initial model. Based on the calibrated fusion feature map, the gradient distribution features and the dynamic change vector of the Horn-Schunck optical flow are calculated. A hybrid Poisson equation is constructed and solved for optimization. A smooth 3D surface model is generated by the Marching-Cubes algorithm. The initial contour is adjusted based on the multi-scale feature map and optimized by combining the dynamic change vector to generate a refined 3D model. The final optimized model is generated through completion, denoising, simplification, quantization and compression processing.

2. The modeling and optimization processing method based on digital images as described in claim 1, characterized in that: The generation of the refined feature set includes: Based on the enhanced data graph, a lightweight Vision Transformer model is used for global feature extraction, which is then concatenated into a feature vector set. Combined with a local saliency weight map, a context feature map is generated through an attention mechanism. The region is then segmented using a threshold segmentation method to output highly saliency regions. Harris corner detection is used for identification to obtain feature points, which are defined as center points, and a multi-radius sphere is constructed. The Sobel operator is used to calculate the gradient of standardized image data in a multi-radius sphere, calculate the dual dynamic threshold, generate an enhanced gradient set by horizontal arrangement, calculate the dimensional gradient histogram feature vectors, concatenate the gradient histogram feature vectors within the dimension, generate feature vectors for feature points, calculate the distance between feature points using the Euclidean distance formula, set the nearest neighbor, use the enhanced gradient set for validation, filter to generate line constraints that meet the validation conditions, concatenate with feature vectors to generate a preliminary point-line feature set, use line constraints to calculate the initial edge mask, combine with the edge map to optimize the dual dynamic threshold, generate optimized line constraints, fuse the edge map to generate enhanced line constraints, obtain the enhanced feature set, calculate the significance score of the feature vectors, and generate a refined feature set through filtering.

3. The modeling and optimization processing method based on digital images as described in claim 2, characterized in that: The process of merging overlapping regions to generate a corrected initial model includes: Based on the refined feature set, feature vectors are extracted, scale classification is performed, and multi-scale feature maps are generated through Laplacian decomposition. These feature maps are then stitched together with enhanced line constraints to generate a multi-scale feature map set. Based on the context feature map, the fusion weight is calculated, and the multi-scale feature map set is combined for weighted fusion to generate a fusion feature map. The enhanced line constraint is used for correction to generate a corrected fusion feature map. Based on the gray-level invariant matrix, feature point sets are extracted from the calibration fusion feature map. K-means clustering is used to cluster the feature point sets. The minimum bounding matrix is ​​calculated using the rotating caliper algorithm. The calibration fusion feature map is cropped to generate image patches. These patches are then combined with their corresponding clustered feature point sets to generate object feature pairs. Finally, the DeepLabv3+ model is used to generate a semantic segmentation map. The clustered feature point set is mapped to the semantic segmentation map using the direct coordinate mapping method to obtain the category label of the clustered feature point on the semantic segmentation map. The local point cloud and depth value are calculated, and the local point cloud is converted into a local mesh using Delaunay triangulation. Based on the clustered feature point set, the local grid and category labels are spatially aligned using a global coordinate system, and overlapping regions are merged to generate an initial model. Then, affine transformation is used for correction to generate a corrected initial model.

4. The modeling and optimization processing method based on digital images as described in claim 3, characterized in that: The construction of the hybrid Poisson equation and the optimization of its solution include: Based on the calibrated fusion feature map, the gradient distribution features are calculated. Using the Horn-Schunck optical flow algorithm, the dynamic change vector is calculated through the calibrated fusion feature map, and a hybrid Poisson equation is constructed. After discretization, a linear equation system is obtained. Sparse decomposition is performed using Cholesky decomposition, and the linear equation system is solved using preconditional conjugate gradients. The solution stops when the convergence condition is met, and the optimized discrete surface index function vector is output.

5. The modeling and optimization processing method based on digital images as described in claim 4, characterized in that: The optimization process, which combines dynamically changing vectors to generate a refined 3D model, includes: The optimized discrete surface index function vector is converted into a mesh using the Marching-Cubes algorithm, and then mapped to the calibration initial model through affine transformation to obtain a smooth 3D surface model. Boundary point sets are extracted through surface normal vectors and curvature analysis, projected onto the calibration fusion feature map to generate an initial contour, and the local saliency weights at each scale are calculated to adjust the initial contour. Based on the dynamically changing vector, the error prediction direction is calculated by combining the source point cloud and the target point cloud. The objective function is defined, and the registration point cloud is generated by ICP iteration. The Delaunay triangulation is used to reconstruct the 3D mesh, generating a refined 3D model.

6. The modeling and optimization processing method based on digital images as described in claim 5, characterized in that: The process of generating the final optimized model through completion, denoising, simplification, quantization, and compression includes: Based on the refined 3D model, a 3D edge detection algorithm is used for boundary completion, and Laplacian smoothing filtering is used for noise filtering. A simplified model is generated through a pruning algorithm, quantization is performed through 8-bit uniform quantization, and compression is performed through PCA to generate the final optimized model.

7. The modeling and optimization processing method based on digital images as described in claim 6, characterized in that: The process of collecting raw digitized image data and point cloud data and preprocessing them to obtain enhanced data maps includes: Raw digital image data and point cloud data are collected through API interfaces to generate standardized image data. A two-dimensional entropy detection algorithm is used to calculate the local saliency weight map of the standardized data, and enhanced data maps are generated through filtering and processing.

8. A modeling and optimization processing system based on digital images, based on the modeling and optimization processing method based on digital images according to any one of claims 1 to 7, characterized in that: include, The preprocessing and feature extraction module collects and preprocesses raw digital image data and point cloud data to obtain an enhanced data map. It uses a lightweight Vision Transformer to extract global features, generates a context feature map through an attention mechanism, identifies feature points using Harris corner detection, constructs a multi-radius sphere, calculates gradients using the Sobel operator, generates dual dynamic thresholds, generates an enhanced gradient set through filtering, calculates the dimensional gradient histogram feature vector, filters nearest neighbor points based on Euclidean distance to obtain line constraints, generates a preliminary point feature set, generates an edge map using the Canny algorithm, optimizes the dual dynamic thresholds, generates enhanced line constraints, calculates the saliency score of the feature vectors, and generates a refined feature set through filtering. The multi-scale fusion and modeling module is used to generate multi-scale feature maps through Laplacian decomposition, calculate fusion weights, generate fused feature maps, perform correction using enhanced line constraints, extract feature point sets based on gray-level invariant matrices, perform clustering through K-means, generate semantic segmentation maps in conjunction with DeepLabv3+, map category labels, align local grids based on global coordinate systems, and merge overlapping regions to generate a corrected initial model. The model optimization and refinement module is used to calculate gradient distribution features and Horn-Schunck optical flow dynamic change vectors based on the correction fusion feature map, construct a hybrid Poisson equation, solve and optimize it, generate a smooth 3D surface model through the Marching-Cubes algorithm, adjust the initial contour based on the multi-scale feature map, and optimize it in combination with the dynamic change vector to generate a refined 3D model. Through completion, denoising, simplification, quantization and compression processing, the final optimized model is generated.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the modeling and optimization processing method based on digital images as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the modeling and optimization processing method based on digital images as described in any one of claims 1 to 7.

Images