A modeling optimization processing method and system based on digitized images

Abstract

The application discloses a kind of based on digital image's modeling optimization processing method and system, it is related to three-dimensional modeling technical field, including using lightweight Vision Transformer extraction global feature, using Harris corner point detection and identification feature point, through Laplace decomposition generates multi-scale feature map, combined with DeepLabv3+ Generation semantic segmentation chart, constructs hybrid Poisson equation, carries out solving optimization, through Marching-Cubes algorithm generates smooth three-dimensional surface model, adjusts initial contour, combined with dynamic change vector optimization, by completing, denoising, simplifying, quantizing and compressing processing, generates final optimization model.The application is enhanced feature extraction quality by standardization processing and Vision Transformer, improves initial model precision by multi-scale fusion and semantic segmentation, realizes efficient refined modeling by hybrid Poisson optimization and multi-scale adjustment, generates high-quality and low-complexity three-dimensional model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional modeling, and particularly relates to a modeling optimization processing method and system based on digital images. BACKGROUND

[0002] With the rapid development of computer vision and deep learning technology, multi-modal data processing methods based on RGB images, depth maps, point cloud data, etc. are widely used in three-dimensional reconstruction and object recognition. Traditional methods usually perform image feature extraction and classification through feature point detection, edge detection and classic machine learning models, and realize three-dimensional modeling by combining stereonet or triangulation technology. Deep learning models have shown great ability in feature extraction and semantic segmentation. The application of DeepLabv3+ model in semantic segmentation has significantly improved the segmentation accuracy. Optimization methods based on optical flow algorithm and Poisson equation have been widely used in dynamic scene modeling.

[0003] However, the prior art still has some problems. The existing Marching-Cubes surface reconstruction algorithm cannot optimize grid generation from the perspective of global geometric consistency when processing complex topological structures, which easily leads to grid distortion or redundant triangular faces, increases the computational complexity and storage cost, and is difficult to meet the needs of real-time rendering or resource-limited scenarios. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a modeling optimization processing method and system based on digital images, which solves the problem that the existing Marching-Cubes surface reconstruction algorithm cannot optimize grid generation from the perspective of global geometric consistency when processing complex topological structures, which easily leads to grid distortion or redundant triangular faces, increases the computational complexity and storage cost, and is difficult to meet the needs of real-time rendering or resource-limited scenarios.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a modeling optimization processing method based on digital images, which includes,

[0008] The original digitized image data and point cloud data are collected and preprocessed to obtain an enhanced data graph, global features are extracted using a lightweight Vision Transformer, context feature maps are generated through an attention mechanism, feature points are identified using Harris corner detection, a multi-radius spherical domain is constructed, gradients are calculated using a Sobel operator, a double dynamic threshold is generated, an enhanced gradient set is generated through screening, a dimension gradient histogram feature vector is calculated, the nearest neighbor points are screened based on the Euclidean distance, line constraints are obtained, a preliminary point feature set is generated, an edge graph is generated using the Canny algorithm, the double dynamic threshold is optimized, an enhanced line constraint is generated, the significance score of the feature vector is calculated, and a refined feature set is generated through screening;

[0009] Multi-scale feature maps are generated through Laplace decomposition, fusion weights are calculated, a fusion feature map is generated, the enhanced line constraint is used for correction, a feature point set is extracted based on the gray-scale invariant matrix, clustering is performed through K-means, a semantic segmentation graph is generated in combination with DeepLabv3+, the category label is mapped, the local grid is aligned based on the global coordinate system, and the overlapping area is merged to generate a corrected initial model;

[0010] Based on the corrected fusion feature map, the gradient distribution feature and the Horn-Schunck optical flow dynamic change vector are calculated, a mixed Poisson equation is constructed, and the solution is optimized, a smooth three-dimensional surface model is generated through the Marching-Cubes algorithm, the initial contour is adjusted based on the multi-scale feature map, and the dynamic change vector is combined for optimization to generate a refined three-dimensional model, and through the processing of completion, denoising, simplification, quantization and compression, the final optimized model is generated.

[0011] As a preferred scheme of the modeling optimization processing method based on digitized images, the generation of the refined feature set comprises:

[0012] Based on the enhanced data graph, global feature extraction is performed using a lightweight Vision Transformer model, which is spliced into a feature vector set, combined with a local saliency weight map, context feature maps are generated through an attention mechanism, segmentation is performed through a threshold segmentation method, high saliency regions are output, feature points are identified using Harris corner detection, defined as center points, and a multi-radius spherical domain is constructed.

[0013] The Sobel operator is used to calculate the gradient of the normalized image data in the multi-radius spherical domain, a double dynamic threshold is calculated, an enhanced gradient set is generated by transverse arrangement, a dimension gradient histogram feature vector is calculated, the gradient histogram feature vectors in the dimension are spliced, a feature vector of a feature point is generated, the distance between feature points is calculated using the Euclidean distance formula, the nearest neighbor points are set, verification is performed using the enhanced gradient set, a generated line constraint that meets the verification condition is screened, splicing is performed in combination with the feature vector, a preliminary point-line feature set is generated, an initial edge mask is calculated using the line constraint, the edge map is combined, the double dynamic threshold is optimized, an optimized line constraint is generated, the edge map is fused, an enhanced line constraint is generated, and an enhanced feature set is obtained, the saliency score of the feature vector is calculated, and a refined feature set is generated by screening.

[0014] As a preferred scheme of the modeling optimization processing method based on digital images provided by the application, wherein: the merging of the overlapping area to generate the corrected initial model comprises:

[0015] Based on the refined feature set, the feature vectors therein are extracted, scale classification is performed, a multi-scale feature map is generated through Laplace decomposition, splicing is performed in combination with the enhanced line constraint, and a multi-scale feature map set is generated;

[0016] Based on the context feature map, a fusion weight is calculated, weighted fusion is performed in combination with the multi-scale feature map set, a fusion feature map is generated, and a corrected fusion feature map is generated by using the enhanced line constraint for correction;

[0017] Based on the gray-scale invariant matrix, a feature point set in the corrected fusion feature map is extracted, the feature point set is clustered using K-means clustering, a minimum circumscribed matrix is calculated through a rotation jamming algorithm, the corrected fusion feature map is cropped to generate an image block, and an object feature pair is spliced by combining the corresponding clustered feature point set, and a semantic segmentation map is generated using a DeepLabv3+ model.

[0018] The clustered feature point set is mapped to the semantic segmentation map using a direct coordinate mapping method to obtain the class label of the clustered feature point on the semantic segmentation map, local point clouds and depth values are calculated, and the local point clouds are converted into a local grid using Delaunay triangulation.

[0019] Based on the clustered feature point set, the local grid and the class label are spatially aligned using a global coordinate system, and the overlapping area is merged to generate an initial model, and the initial model is corrected using affine transformation to generate a corrected initial model.

[0020] As a preferred scheme of the modeling optimization processing method based on digital images provided by the application, wherein: the construction of the mixed Poisson equation for solving optimization comprises:

[0021] Based on the corrected fusion feature map, the gradient distribution feature is calculated, the Horn-Schunck optical flow algorithm is used, the dynamic change vector is calculated through the corrected fusion feature map, the mixed Poisson equation is constructed, the discretization is carried out, the linear equation set is obtained, the Cholesky decomposition is used for sparse decomposition, and the linear equation set is solved using the preconditioned conjugate gradient, and the optimization is stopped when the convergence condition is reached, and the optimized discrete surface index function vector is output.

[0022] As a preferred scheme of the modeling optimization processing method based on digital images provided by the application, wherein: the dynamic change vector is combined for optimization to generate a refined three-dimensional model, comprising:

[0023] The optimized discrete surface index function vector is converted into a grid using the Marching-Cubes algorithm, mapped to the corrected initial model through affine transformation, and a smooth three-dimensional surface model is obtained, the boundary point set is extracted through surface normal vector and curvature analysis, projected to the corrected fusion feature map to generate an initial contour, and the local saliency weight of each scale is calculated to adjust the initial contour.

[0024] Based on the dynamic change vector, the source point cloud and the target point cloud are combined to calculate the error prediction direction, the target function is defined, the ICP iteration is used to generate the registration point cloud, the Delaunay triangulation is used to reconstruct the three-dimensional grid, and the refined three-dimensional model is generated.

[0025] As a preferred scheme of the modeling optimization processing method based on digital images provided by the application, wherein: the final optimized model is generated through completion, denoising, simplification, quantization and compression processing, comprising:

[0026] Based on the refined three-dimensional model, the three-dimensional edge detection algorithm is used for boundary completion, and the Laplace smoothing filter is used for noise filtering, the pruning algorithm is used to generate a simplified model, the 8-bit uniform quantization is used for quantization, and the PCA is used for compression to generate the final optimized model.

[0027] As a preferred scheme of the modeling optimization processing method based on digital images provided by the application, wherein: the original digital image data and point cloud data are collected and preprocessed to obtain enhanced data graphs, comprising:

[0028] The original digital image data and point cloud data are collected through the API interface to generate standardized image data, the two-dimensional entropy detection algorithm is used to calculate the local saliency weight map of the standardized data, and the enhanced data graph is generated through screening processing.

[0029] In a second aspect, the application provides a modeling optimization processing system based on digital images, comprising,

[0030] The preprocessing and feature extraction module is used for collecting and preprocessing original digitized image data and point cloud data to obtain enhanced data graphs, extracting global features by using a lightweight Vision Transformer, generating context feature maps through an attention mechanism, identifying feature points by using Harris corner detection, constructing a multi-radius spherical domain, calculating gradients by using a Sobel operator, generating a double dynamic threshold, generating an enhanced gradient set through screening, calculating a dimension gradient histogram feature vector, screening nearest neighbor points based on Euclidean distance to obtain line constraints, generating a preliminary point feature set, generating an edge map by using a Canny algorithm, optimizing the double dynamic threshold, generating an enhanced line constraint, calculating a saliency score of the feature vector, and generating a refined feature set through screening;

[0031] The multi-scale fusion and modeling module is used for generating multi-scale feature maps through Laplace decomposition, calculating fusion weights, generating a fusion feature map, correcting the fusion feature map by using the enhanced line constraint, extracting a feature point set based on a gray-scale invariant matrix, clustering by using K-means, generating a semantic segmentation map in combination with DeepLabv3+, mapping a category label, aligning a local grid based on a global coordinate system, and merging an overlapping area to generate a corrected initial model.

[0032] The model optimization and refinement module is used for calculating gradient distribution features and Horn-Schunck optical flow dynamic change vectors based on the corrected fusion feature map, constructing a hybrid Poisson equation, performing solving optimization, generating a smooth three-dimensional surface model through a Marching-Cubes algorithm, adjusting an initial contour based on the multi-scale feature map, optimizing in combination with the dynamic change vectors, generating a refined three-dimensional model, and generating a final optimized model through processing such as completion, denoising, simplification, quantization, and compression.

[0033] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, any step of the modeling optimization processing method based on digitized images according to the first aspect of the present application is implemented.

[0034] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, any step of the modeling optimization processing method based on digitized images according to the first aspect of the present application is implemented.

[0035] The present application has the beneficial effects that: the present application extracts global features by using a lightweight Vision Transformer, identifies feature points by using Harris corner point detection, generates a multi-scale feature map through Laplace decomposition, generates a semantic segmentation map in combination with DeepLabv3+, constructs a hybrid Poisson equation, performs solving optimization, generates a smooth three-dimensional surface model through a Marching-Cubes algorithm, adjusts an initial contour, optimizes in combination with a dynamic change vector, generates a final optimized model through completion, denoising, simplification, quantization and compression processing, enhances feature extraction quality, improves initial model precision, realizes efficient refined modeling, and generates a high-quality and low-complexity three-dimensional model. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0037] Figure 1 The flowchart of the modeling optimization processing method based on digital images in embodiment 1.

[0038] Figure 2 The schematic diagram of the modeling optimization processing system based on digital images in embodiment 1. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0040] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0041] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0042] Embodiment 1, refer to Figure 1 and Figure 2 , the first embodiment of the present application provides a modeling optimization processing method based on digital images, comprising the following steps:

[0043] S1, collect and preprocess original digitized image data and point cloud data to obtain an enhanced data graph, extract global features using a lightweight Vision Transformer, generate a context feature map through an attention mechanism, identify feature points using Harris corner detection, construct a multi-radius spherical domain, calculate gradients using a Sobel operator, generate a dual dynamic threshold, generate an enhanced gradient set through screening, calculate a dimension gradient histogram feature vector, filter nearest neighbor points based on Euclidean distance to obtain line constraints, generate a preliminary point feature set, generate an edge map using a Canny algorithm, optimize the dual dynamic threshold to generate an enhanced line constraint, calculate a saliency score of the feature vector, and generate a refined feature set through screening;

[0044] Specifically, the original digitized image data and point cloud data are collected and preprocessed to obtain an enhanced data graph, including:

[0045] The original digitized image data and point cloud data are collected through an API interface;

[0046] The original digitized image data includes RGB images, depth maps, and remote sensing image data;

[0047] Based on the original digitized image data, bilinear interpolation is used for resolution normalization, and the RGB image data is converted to an HSV image through an RGB-to-HSV algorithm, the depth map data is normalized to grayscale, and the remote sensing image data is normalized to a channel to generate a standardized two-dimensional image;

[0048] Based on the point cloud data, the data is projected to a volume element grid through trilinear interpolation to generate standardized point cloud data, which is normalized, converted to pseudo-image data through coordinate system conversion, combined with the standardized two-dimensional image to splice into standardized image data, and a three-dimensional global coordinate system is constructed through an SGBM algorithm;

[0049] A two-dimensional entropy detection algorithm is used to calculate the local saliency weight map of the standardized data;

[0050] The local saliency weight map greater than the high saliency weight map is screened, Gaussian difference filtering is used to enhance edge details, the local saliency weight map less than the low saliency weight map is screened, median filtering is used to reduce noise, and an enhanced data graph is generated (the high saliency weight map and the low saliency weight map are both set by statistical quantile method).

[0051] Compatibility and consistency of data acquisition are guaranteed through standardized API interface, providing high-quality input basis for subsequent processing, customized processing for data characteristics significantly improves data standardization quality, preserves the spatial details of point cloud, and innovative design of pseudo-image generation and two-dimensional image stitching enables subsequent feature extraction algorithms to process multi-modal data simultaneously, significantly improving the accuracy of feature fusion, through significant weight partition processing, improving edge detection accuracy, reducing noise level, enhancing the details of high saliency regions, and ensuring data quality;

[0052] Further, the generating the refined feature set comprises:

[0053] Based on the enhanced data graph, a lightweight Vision Transformer model is used for global feature extraction, and is spliced into a feature vector set;

[0054] Based on the feature vector set, a local saliency weight map is combined to generate a context feature map through an attention mechanism;

[0055] Based on the context feature map, an empirical rule is used to set a saliency threshold, and a threshold segmentation method is used for segmentation to output a high saliency region;

[0056] Based on the high saliency region, Harris corner detection is used for identification to obtain feature points, such as artifact surface texture points and building boundary corner points;

[0057] The feature points are defined as center points, a radius is set based on historical experience, and a multi-radius spherical domain is constructed;

[0058] The Sobel operator is used to calculate the gradient of the normalized image data in the multi-radius spherical domain, and a double dynamic threshold is calculated, the formula is:

[0059]

[0060] T low =c·T high ,

[0061] Wherein, T high is the high threshold in the dynamic double threshold, is the gradient amplitude of the normalized image data I, calculated based on the Sobel operator, T low is the low threshold in the dynamic double threshold, c is a fixed proportion coefficient set based on experience, and med is a median operation;

[0062] The gradients greater than or equal to the low threshold in the double dynamic threshold are screened, and an enhanced gradient set is generated by horizontal arrangement;

[0063] Based on the enhanced gradient set, a dimension gradient histogram feature vector is calculated, the gradient histogram feature vectors within the dimension are spliced, and a feature vector of the feature point is generated;

[0064] Based on the high saliency region, the distance between the feature points is calculated using the Euclidean distance formula, sorted in ascending order, the feature points with the smallest distance are selected as the nearest neighbor points, verified using the enhanced gradient set, and the generated line constraints that meet the verification conditions are selected, and the formula is:

[0065] l j ={(p i ,p m ,p n )∣d(p i ,p m )≤d(p i ,p n )≤θ,(p m ,p n )∈G},

[0066] Where l j is the jth line constraint, indicating the connection relationship of the ith feature point p, the mth feature point p and the nth feature point p, d is the distance between the feature points, θ is the distance threshold value, which is set based on the maximum radius value of the multi-radius spherical domain, and G is the enhanced gradient set;

[0067] Splice the line constraint and the feature vector to generate a preliminary point-line feature set;

[0068] Based on the preliminary point-line feature set, the initial edge mask is calculated using the line constraint, and the formula is:

[0069]

[0070] Where M(x,y) is the initial edge mask of the horizontal coordinate x and the vertical coordinate y, Path is the linear interpolation path, which is calculated based on the straight line equation, and oth is otherwise;

[0071] Based on the normalized image data, the edge map is generated using the Canny algorithm, and the initial edge mask is used to optimize the double dynamic threshold to generate an optimized line constraint, and the formula is:

[0072]

[0073] Where T 1 high is the high threshold value in the optimized double dynamic threshold;

[0074] Fuse the optimized line constraint and the edge map to generate an enhanced line constraint, and the formula is:

[0075]

[0076] wherein L is an enhanced line constraint, E is an edge map, L 1 to optimize the line constraint;

[0077] replace the line constraint in the preliminary point-line feature set with the enhanced line constraint to generate an enhanced feature set;

[0078] based on the enhanced feature set, calculate a saliency score of a feature vector, the formula being:

[0079]

[0080] wherein S δ is a saliency score of the δth feature vector, v k is an absolute value of the kth component of the δth feature vector V, and c is a component number of the feature vector;

[0081] screen the feature vectors and the corresponding enhanced line constraints with saliency scores greater than a saliency score threshold (set based on statistical distribution) to generate a refined feature set.

[0082] The global feature is extracted by the lightweight Vision Transformer, the context correlation is highlighted by combining the attention mechanism, the robustness of the feature is significantly improved, the introduction of the double dynamic threshold solves the problem of poor adaptability of the traditional fixed threshold method to complex scenes, the enhanced line constraint further optimizes the edge detection accuracy through the fusion of the Canny algorithm and the initial edge mask, the generated high saliency feature set has higher accuracy and stability in recognition, the objective screening of the feature is realized through the saliency score formula combined with the statistical distribution threshold, a quantitative saliency evaluation standard is provided, the influence of redundant features is significantly reduced, while the key feature points are retained, the refinement degree and application value of the feature set are improved.

[0083] S2, generate a multi-scale feature map through Laplace decomposition, calculate a fusion weight, generate a fusion feature map, correct using the enhanced line constraint, extract a feature point set based on the gray-scale invariant matrix, cluster through K-means, generate a semantic segmentation map combined with DeepLabv3+, map a class label, align a local grid based on a global coordinate system, and merge overlapping areas to generate a corrected initial model;

[0084] Specifically, merging overlapping areas to generate a corrected initial model comprises:

[0085] based on the refined feature set, extract a feature vector therefrom, set a high-resolution threshold and a low-resolution threshold based on scene complexity, screen feature vectors with resolutions greater than or equal to the high-resolution threshold, set a high scale, screen feature vectors with resolutions greater than or equal to the low-resolution threshold and less than the high-resolution threshold, set a medium scale, and screen feature vectors with resolutions greater than the low-resolution threshold, set a low scale.

[0086] Based on scale classification, Laplacian decomposition is applied to the feature vectors in the refined feature set to generate multi-scale feature maps.

[0087] Multi-scale feature maps and enhanced line constraints are concatenated to generate a multi-scale feature map set;

[0088] Based on the context feature map, the fusion weight is calculated using the following formula:

[0089]

[0090] in, For the first Fusion weights at each scale For the context feature map in the 1st The projections at each scale are obtained through downsampling, where ∈ is the global contrast factor, and high, mid, and low represent high, mid, and low scales, respectively.

[0091] Based on the fusion weight, a fusion feature map is generated by weighted fusion of multi-scale feature maps.

[0092] Based on the fused feature map, correction is performed using enhanced line constraints to generate a corrected fused feature map, as shown in the formula:

[0093] F′=F fused +β·PPHT(F,L),

[0094] Where F′ is the corrected fusion feature map, β is the corrected weight, PPHT is the asymptotic probabilistic Hough transform, and F is the fusion feature map;

[0095] Based on the gray-level invariant matrix, the feature point set is extracted from the correction fusion feature map. The number of clusters K is set based on the elbow rule, and K-means clustering is used to cluster the feature point set to generate K clusters. Each cluster corresponds to a type of item, for example, K-1 clusters correspond to cultural relics.

[0096] The minimum bounding matrix is ​​calculated by the rotating caliper algorithm, the corrected fusion feature map is cropped to generate image patches, and then combined with the corresponding clustered feature point set to generate object feature pairs.

[0097] Based on image patches, a semantic segmentation map is generated using the DeepLabv3+ model;

[0098] The direct coordinate mapping method is used to map the cluster feature point set to the semantic segmentation map, thereby obtaining the category labels of the cluster feature points on the semantic segmentation map;

[0099] Based on category labels, local point cloud and depth values ​​are calculated. Delaunay triangulation is then used to convert the local point cloud into a local mesh, as shown in the formula:

[0100]

[0101] wherein, is the local point cloud of the th cluster feature point, is the depth value of the th cluster feature point, calculated based on the SGBM algorithm, is the category label of the th cluster feature point;

[0102] Based on the set of cluster feature points, the local grid and the category label are spatially aligned using a global coordinate system, and the overlapping area is merged to generate an initial model, and an affine transformation is used for correction to generate a corrected initial model.

[0103] Through scale classification, the subsequent processing can be targetedly allocated with computing resources, the computing efficiency and information retention can be effectively balanced, the feature extraction deviation caused by scale mismatch is avoided, the robustness and pertinence of feature extraction are improved, the multi-scale feature map is generated through Laplace decomposition, which can better retain the contour and detail information, providing more rich underlying feature support for subsequent accurate segmentation and positioning, solving the unreasonable feature fusion weight distribution problem in the prior art, improving the quality of the fused feature map, providing more accurate and robust input for subsequent line constraint correction and feature point extraction, reducing the noise introduced by incorrect fusion, improving the computing efficiency by calculating the minimum bounding rectangle and cropping, using the local point cloud with category label for Delaunay triangulation, improving the spatial consistency and integrity of the final initial model, and applying affine transformation for correction, improving the quality and usability of the model.

[0104] S3, based on the corrected fusion feature map, calculating the gradient distribution feature and the Horn-Schunck optical flow dynamic change vector, constructing a mixed Poisson equation, solving and optimizing, generating a smooth three-dimensional surface model through the Marching-Cubes algorithm, adjusting the initial contour based on the multi-scale feature map, combining the dynamic change vector to optimize, generating a refined three-dimensional model, and through the processing of completion, denoising, simplification, quantization and compression, generating a final optimized model;

[0105] Specifically, further, constructing a mixed Poisson equation, solving and optimizing, including:

[0106] Based on the corrected fusion feature map, the gradient distribution feature is calculated, and the formula is:

[0107]

[0108] wherein, f is the gradient distribution feature, To correct the gradient vector of the fused feature map F ′ , it is calculated based on the Sobel operator;

[0109] Using the Horn-Schunck optical flow algorithm, the dynamic change vector is calculated by correcting the fused feature map, and the formula is:

[0110] V = HS (F ′ (t), F ′ (t-1)),

[0111] wherein V is the dynamic change vector, HS is the Horn-Schunck algorithm, F ′ (t-1) is the corrected fusion map at the t-1 time;

[0112] Based on the class label of the corrected initial model, the Poisson optimization parameters are initialized, for example, the weight of optical flow is increased for cultural relic scenes, and the weight of optical flow is reduced for building scenes;

[0113] Based on the gradient feature distribution and the dynamic change vector, combined with the initialized Poisson optimization parameters, the mixed Poisson equation is constructed, and the formula is:

[0114]

[0115] wherein φ is the surface index function, div(f 1 ) is the divergence of the gradient field f 1 , which is calculated based on the Laplace operator, β 1 is the weight of the optical flow constraint, is the gradient of the dynamic change vector V, which is calculated based on the Sobel operator;

[0116] Based on the mixed Poisson equation, discretization is performed to obtain a linear equation system, and the formula is:

[0117] Aφ 1 = b,

[0118] wherein A is the Laplace operator matrix, which is obtained by discretization based on the corrected fused feature map and the stellate grid, φ 1 is the discrete surface index function vector, b is the source term vector, and is the mixed Poisson equation;

[0119] The Laplace operator matrix is sparsely decomposed using Cholesky decomposition, and the linear equation system is solved using preconditioned conjugate gradient, and the process is stopped when the convergence condition is met, which is set based on the residual norm threshold method. The optimized discrete surface index function vector is output.

[0120] By accurately quantifying the gradient distribution characteristics, it ensures that the subsequent mixed Poisson equation can better adapt to the changes in geometric details in complex scenes, retains more details in three-dimensional reconstruction, and significantly improves the fidelity of the surface model. The Horn-Schunck optical flow algorithm is used to calculate the dynamic change vector, which significantly improves the reconstruction accuracy of the three-dimensional model in dynamic scenes, accurately captures the motion trajectory, and reduces the reconstruction error caused by motion blur, providing reliable dynamic constraints for subsequent Poisson optimization and model refinement. Based on the category label, the Poisson optimization parameters are initialized, which improves the adaptability of the algorithm to different scenes, and the optimized parameters can better balance dynamic changes and geometric constraints.

[0121] Further, combined with the dynamic change vector, the refined three-dimensional model is generated, including:

[0122] The optimized discrete surface index function vector is converted into a grid using the Marching-Cubes algorithm, and is mapped to the corrected initial model through affine transformation to obtain a smooth three-dimensional surface model;

[0123] Based on the smooth three-dimensional surface model, the boundary point set is extracted by the Shi-Tomasi corner detection method, and is projected onto the corrected fusion feature map to generate an initial contour, the formula is:

[0124] v0=Proj(P 1 )∩M,

[0125] Where v0 is the initial contour, representing the starting point of the contour, P 1 is the boundary point set of the smooth three-dimensional surface model, and Proj is the operation of projecting the boundary point set onto the corrected fusion map;

[0126] Based on the multi-scale feature map, the local saliency weight of each scale is calculated, the formula is:

[0127]

[0128] Where, is the local saliency weight of the s-th scale, is the saliency parameter of the s-th scale, ∈ is the global contrast factor, g and a are the row index and column index respectively, n 1 x is the horizontal pixel number of the x-axis of the multi-scale feature map, n 2 y is the vertical pixel number of the y-axis of the multi-scale feature map;

[0129] Based on the local saliency weight and the smooth three-dimensional surface model, the initial contour is adjusted, and the iteration is stopped when the maximum iteration number is reached (based on the early stopping method), and the adjusted contour is output, which is set as the edge constraint of the model, the formula is:

[0130]

[0131] where v t is the adjustment profile at the t-th time, is the gradient modulus of the corrected fusion feature map, which is obtained based on the gradient distribution feature, N is the normal vector field, which represents the surface normal vector of the smoothed 3D model at coordinates (x, y), and is obtained through surface geometry analysis (vertex normal vector estimation and principal curvature estimation), is the smoothing weight function, h is the edge weight function, η is the Laplacian operator, which represents the second-order derivative operation on v;

[0132] Based on the dynamic change vector, the error prediction term is calculated by combining the source point cloud (based on the grid points of the smoothed 3D model) and the target point cloud (based on the feature points in the enhanced line constraint), and the formula is:

[0133] EW=∑ i′ ||V(X i′ )·(T 3 t (X i′ )-Y i″ )||,

[0134] where EW is the prediction error term, V(X i′ ) is the dynamic change vector of the i'th source point cloud X, T 3 t is the spatial position change function at the t-th time, which represents the mapping of the source point cloud to the i''th target point cloud Y through rigid transformation and non-rigid transformation;

[0135] Based on the prediction error term, the objective function is defined, and the formula is:

[0136] EQ t =∑ i′ ||T 3 t (X i′ )-Y i″ || 2 +λ·EW,

[0137] where EQ t is the objective function value at the t-th time, and λ is the error prediction weight;

[0138] The ICP algorithm is used to minimize the objective value, and the optimized change function is obtained, which is applied to the source point cloud through multiplication operation to generate the registered point cloud;

[0139] Based on the registered point cloud, Delaunay triangulation is used to reconstruct a three-dimensional grid to generate a refined three-dimensional model, and the formula is:

[0140] QS = (GY, LK),

[0141] wherein QS is the refined three-dimensional model, LK is the category label, and GY is the three-dimensional mesh.

[0142] The Marching-Cubes algorithm can solve the problem of accurately extracting a continuous surface from a discrete data field, generating a mesh with correct topological structure, solving the problem of inconsistency or positional deviation between the optimized result and the existing model coordinate system, ensuring the continuity and accuracy of subsequent processing steps, Shi-Tomasi corner detection has good robustness for corner points, solving the problem of how to effectively identify and use key geometric features from the surface of a three-dimensional model to guide contour generation, calculating local saliency weights, solving the problem of how to distinguish important and secondary areas during contour adjustment, solving the problem of contour distortion or insufficient adjustment caused by simple global adjustment, calculating error prediction vectors, solving the problem of how to quantify the difference between the source model and the target constraint, and predicting how to adjust the model to reduce this difference, achieving accurate positioning and pose adjustment of the model in space, Delaunay triangulation solves the technical problem of converting geometric information after multiple rounds of optimization processing into a high-quality three-dimensional mesh model, and generates the final refined three-dimensional model.

[0143] Furthermore, through completion, denoising, simplification, quantization and compression processing, the final optimized model is generated, including:

[0144] Based on the refined three-dimensional model, boundary completion is performed using a three-dimensional edge detection algorithm, and noise filtering is performed through Laplacian smoothing filtering, a simplified model is generated through a pruning algorithm, quantization is performed through 8-bit uniform quantization (8 is the number of bits, based on actual demand scenarios), compression is performed through PCA, and the final optimized model is generated.

[0145] Through boundary completion and noise filtering, three-dimensional edge detection and Laplacian smoothing are used based on the refined model, which can repair model defects, eliminate noise, improve model geometric continuity and data purity, solve the problem of model defects and noise interference, through the pruning algorithm, the model complexity can be reduced, the model redundancy problem can be solved, which is beneficial to subsequent processing and real-time application, 8-bit uniform quantization and PCA compression can significantly reduce the data storage amount and transmission bandwidth demand on the premise of ensuring model accuracy, solving the problem of excessive model data volume, and realizing efficient storage and transmission of the model.

[0146] The embodiment also provides a modeling optimization processing system based on digital images, comprising:

[0147] The preprocessing and feature extraction module is used for collecting and preprocessing original digitized image data and point cloud data to obtain enhanced data graphs, extracting global features by using a lightweight Vision Transformer, generating context feature maps through an attention mechanism, identifying feature points by using Harris corner detection, constructing a multi-radius spherical domain, calculating gradients by using a Sobel operator, generating a double dynamic threshold, generating an enhanced gradient set through screening, calculating a dimension gradient histogram feature vector, screening the nearest neighbor points based on the Euclidean distance, obtaining line constraints, generating a preliminary point feature set, generating an edge map by using a Canny algorithm, optimizing the double dynamic threshold, generating an enhanced line constraint, calculating the saliency score of the feature vector, and generating a refined feature set through screening.

[0148] The multi-scale fusion and modeling module is used for generating multi-scale feature maps through Laplace decomposition, calculating fusion weights, generating fusion feature maps, correcting by using enhanced line constraints, extracting a feature point set based on a gray-scale invariant matrix, clustering by using K-means, generating a semantic segmentation map in combination with DeepLabv3+, mapping a class label, aligning a local grid based on a global coordinate system, and merging overlapping areas to generate a corrected initial model.

[0149] The model optimization and refinement module is used for calculating gradient distribution features and Horn-Schunck optical flow dynamic change vectors based on the corrected fusion feature maps, constructing a hybrid Poisson equation, performing solving optimization, generating a smooth three-dimensional surface model through a Marching-Cubes algorithm, adjusting the initial contour based on the multi-scale feature maps, optimizing in combination with the dynamic change vectors, generating a refined three-dimensional model, and generating a final optimized model through processing such as completion, denoising, simplification, quantization, and compression.

[0150] The embodiment also provides a computer device suitable for the modeling optimization processing method based on digitized images, which comprises a memory and a processor.

[0151] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0152] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the modeling optimization processing method based on a digital image as described above. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0153] To sum up, the application extracts global features by using a lightweight Vision Transformer, identifies feature points by using Harris corner point detection, generates a multi-scale feature map through Laplace decomposition, generates a semantic segmentation map in combination with DeepLabv3+, constructs a hybrid Poisson equation, performs solving optimization, generates a smooth three-dimensional surface model through a Marching-Cubes algorithm, adjusts an initial contour, optimizes in combination with a dynamic change vector, generates a final optimized model through processing such as completion, denoising, simplification, quantization and compression, enhances feature extraction quality, improves initial model precision, realizes efficient refined modeling, and generates a high-quality and low-complexity three-dimensional model.

[0154] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not to limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A modeling and optimization processing method based on digital images, characterized in that: include, Raw digital image data and point cloud data are collected and preprocessed to obtain an enhanced data map. A lightweight Vision Transformer is used to extract global features, and a context feature map is generated through an attention mechanism. Harris corner detection is used to identify feature points, a multi-radius sphere is constructed, gradients are calculated using the Sobel operator, a dual dynamic threshold is generated, an enhanced gradient set is generated through filtering, the dimensional gradient histogram feature vector is calculated, the nearest neighbor is selected based on Euclidean distance to obtain line constraints, a preliminary point feature set is generated, an edge map is generated using the Canny algorithm, the dual dynamic threshold is optimized to generate enhanced line constraints, the significance score of the feature vector is calculated, and a refined feature set is generated through filtering. Multi-scale feature maps are generated through Laplacian decomposition, fusion weights are calculated, fusion feature maps are generated, and enhanced line constraints are used for correction. Feature point sets are extracted based on gray-level invariant matrices, clustering is performed using K-means, semantic segmentation maps are generated by combining DeepLabv3+, category labels are mapped, local grids are aligned based on global coordinate system, and overlapping regions are merged to generate a corrected initial model. Based on the calibrated fusion feature map, the gradient distribution features and the dynamic change vector of the Horn-Schunck optical flow are calculated. A hybrid Poisson equation is constructed and solved for optimization. A smooth 3D surface model is generated by the Marching-Cubes algorithm. The initial contour is adjusted based on the multi-scale feature map and optimized by combining the dynamic change vector to generate a refined 3D model. The final optimized model is generated through completion, denoising, simplification, quantization and compression processing.

2. The modeling and optimization processing method based on digital images as described in claim 1, characterized in that: The generation of the refined feature set includes: Based on the enhanced data graph, a lightweight Vision Transformer model is used for global feature extraction, which is then concatenated into a feature vector set. Combined with a local saliency weight map, a context feature map is generated through an attention mechanism. The region is then segmented using a threshold segmentation method to output highly saliency regions. Harris corner detection is used for identification to obtain feature points, which are defined as center points, and a multi-radius sphere is constructed. The Sobel operator is used to calculate the gradient of standardized image data in a multi-radius sphere, calculate the dual dynamic threshold, generate an enhanced gradient set by horizontal arrangement, calculate the dimensional gradient histogram feature vectors, concatenate the gradient histogram feature vectors within the dimension, generate feature vectors for feature points, calculate the distance between feature points using the Euclidean distance formula, set the nearest neighbor, use the enhanced gradient set for validation, filter to generate line constraints that meet the validation conditions, concatenate with feature vectors to generate a preliminary point-line feature set, use line constraints to calculate the initial edge mask, combine with the edge map to optimize the dual dynamic threshold, generate optimized line constraints, fuse the edge map to generate enhanced line constraints, obtain the enhanced feature set, calculate the significance score of the feature vectors, and generate a refined feature set through filtering.

3. The modeling and optimization processing method based on digital images as described in claim 2, characterized in that: The process of merging overlapping regions to generate a corrected initial model includes: Based on the refined feature set, feature vectors are extracted, scale classification is performed, and multi-scale feature maps are generated through Laplacian decomposition. These feature maps are then stitched together with enhanced line constraints to generate a multi-scale feature map set. Based on the context feature map, the fusion weight is calculated, and the multi-scale feature map set is combined for weighted fusion to generate a fusion feature map. The enhanced line constraint is used for correction to generate a corrected fusion feature map. Based on the gray-level invariant matrix, feature point sets are extracted from the calibration fusion feature map. K-means clustering is used to cluster the feature point sets. The minimum bounding matrix is ​​calculated using the rotating caliper algorithm. The calibration fusion feature map is cropped to generate image patches. These patches are then combined with their corresponding clustered feature point sets to generate object feature pairs. Finally, the DeepLabv3+ model is used to generate a semantic segmentation map. The clustered feature point set is mapped to the semantic segmentation map using the direct coordinate mapping method to obtain the category label of the clustered feature point on the semantic segmentation map. The local point cloud and depth value are calculated, and the local point cloud is converted into a local mesh using Delaunay triangulation. Based on the clustered feature point set, the local grid and category labels are spatially aligned using a global coordinate system, and overlapping regions are merged to generate an initial model. Then, affine transformation is used for correction to generate a corrected initial model.

4. The modeling and optimization processing method based on digital images as described in claim 3, characterized in that: The construction of the hybrid Poisson equation and the optimization of its solution include: Based on the calibrated fusion feature map, the gradient distribution features are calculated. Using the Horn-Schunck optical flow algorithm, the dynamic change vector is calculated through the calibrated fusion feature map, and a hybrid Poisson equation is constructed. After discretization, a linear equation system is obtained. Sparse decomposition is performed using Cholesky decomposition, and the linear equation system is solved using preconditional conjugate gradients. The solution stops when the convergence condition is met, and the optimized discrete surface index function vector is output.

5. The modeling and optimization processing method based on digital images as described in claim 4, characterized in that: The optimization process, which combines dynamically changing vectors to generate a refined 3D model, includes: The optimized discrete surface index function vector is converted into a mesh using the Marching-Cubes algorithm, and then mapped to the calibration initial model through affine transformation to obtain a smooth 3D surface model. Boundary point sets are extracted through surface normal vectors and curvature analysis, projected onto the calibration fusion feature map to generate an initial contour, and the local saliency weights at each scale are calculated to adjust the initial contour. Based on the dynamically changing vector, the error prediction direction is calculated by combining the source point cloud and the target point cloud. The objective function is defined, and the registration point cloud is generated by ICP iteration. The Delaunay triangulation is used to reconstruct the 3D mesh, generating a refined 3D model.

6. The modeling and optimization processing method based on digital images as described in claim 5, characterized in that: The process of generating the final optimized model through completion, denoising, simplification, quantization, and compression includes: Based on the refined 3D model, a 3D edge detection algorithm is used for boundary completion, and Laplacian smoothing filtering is used for noise filtering. A simplified model is generated through a pruning algorithm, quantization is performed through 8-bit uniform quantization, and compression is performed through PCA to generate the final optimized model.

7. The modeling and optimization processing method based on digital images as described in claim 6, characterized in that: The process of collecting raw digitized image data and point cloud data and preprocessing them to obtain enhanced data maps includes: Raw digital image data and point cloud data are collected through API interfaces to generate standardized image data. A two-dimensional entropy detection algorithm is used to calculate the local saliency weight map of the standardized data, and enhanced data maps are generated through filtering and processing.

8. A modeling and optimization processing system based on digital images, based on the modeling and optimization processing method based on digital images according to any one of claims 1 to 7, characterized in that: include, The preprocessing and feature extraction module collects and preprocesses raw digital image data and point cloud data to obtain an enhanced data map. It uses a lightweight Vision Transformer to extract global features, generates a context feature map through an attention mechanism, identifies feature points using Harris corner detection, constructs a multi-radius sphere, calculates gradients using the Sobel operator, generates dual dynamic thresholds, generates an enhanced gradient set through filtering, calculates the dimensional gradient histogram feature vector, filters nearest neighbor points based on Euclidean distance to obtain line constraints, generates a preliminary point feature set, generates an edge map using the Canny algorithm, optimizes the dual dynamic thresholds, generates enhanced line constraints, calculates the saliency score of the feature vectors, and generates a refined feature set through filtering. The multi-scale fusion and modeling module is used to generate multi-scale feature maps through Laplacian decomposition, calculate fusion weights, generate fused feature maps, perform correction using enhanced line constraints, extract feature point sets based on gray-level invariant matrices, perform clustering through K-means, generate semantic segmentation maps in conjunction with DeepLabv3+, map category labels, align local grids based on global coordinate systems, and merge overlapping regions to generate a corrected initial model. The model optimization and refinement module is used to calculate gradient distribution features and Horn-Schunck optical flow dynamic change vectors based on the correction fusion feature map, construct a hybrid Poisson equation, solve and optimize it, generate a smooth 3D surface model through the Marching-Cubes algorithm, adjust the initial contour based on the multi-scale feature map, and optimize it in combination with the dynamic change vector to generate a refined 3D model. Through completion, denoising, simplification, quantization and compression processing, the final optimized model is generated.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the modeling and optimization processing method based on digital images as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the modeling and optimization processing method based on digital images as described in any one of claims 1 to 7.

Images