Image monomer method based on live-action three dimensions

Abstract

The invention discloses a live-action three-dimensional image monomer method, which comprises the following steps: S1, acquiring oblique photography data, and preprocessing the oblique photography data to obtain preprocessed oblique photography data; s2, performing segmentation training on the preprocessed oblique photography data through an AI image recognition technology to obtain a segmented mask image; s3, geometric adjustment and topological repair are carried out on the segmented target object in the segmentation mask graph, and a single model is established; s4, binding spatial attributes and geometric features of the monomer models; and S5, constructing a tree structure of the monomer model to adapt to multi-scene business requirements, and completing the image monomer method based on live-action three-dimensional. The method is suitable for the requirements of multiple fields such as building design, city planning and city management, has high practical value and economic benefit, solves the problems of insufficient data quality, insufficient segmentation precision, insufficient model optimization and weak attribute organization ability in the prior art, and improves the applicability and practical value of the model.

Description

Technical Field

[0001] The invention belongs to the technical field of real-scene three-dimensional modeling and data processing, and in particular relates to an image monomerization method based on real-scene three-dimensional. Background Art

[0002] With the rapid development of real-life 3D modeling technology, oblique photography technology collected by drones, LiDAR and other equipment can generate high-precision 3D models, which can be widely used in building management, urban planning and other fields. However, especially in terms of individual segmentation and model optimization, it is difficult to meet diverse business needs and has the following major shortcomings: (1) Insufficient data quality: The 3D models generated by oblique photography usually contain noise, blurred details, inconsistent colors, etc. These problems are caused by the limitations of shooting angle, lighting conditions and equipment performance, which lead to the reduction of model clarity and make it difficult to meet the requirements of high-precision segmentation.

[0003] (2) Insufficient segmentation accuracy: Traditional segmentation techniques have limited performance when dealing with complex scenes, especially when extracting and separating multiple target objects, which are prone to shape deviation or target overlap. In addition, existing algorithms have weak segmentation capabilities for details and are difficult to achieve the accuracy required for practical applications.

[0004] (3) Insufficient model optimization: The segmented models often have problems in geometry and topology, including irregular boundaries, non-manifold edges, isolated vertices, and redundant faces, etc. These defects limit the integrity and usability of the model and cannot directly support subsequent analysis and application requirements.

[0005] (4) Weak attribute organization ability: The lack of spatial attribute binding and hierarchical organization functions for single models makes it difficult to support multi-scenario applications. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for image singulation based on real-scene three-dimensionality, which solves the problems of insufficient data quality, insufficient segmentation accuracy, insufficient model optimization and weak attribute organization capability in the prior art.

[0007] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a method for image singulation based on real scene three-dimensional, comprising the following steps: S1, acquiring oblique photography data, and preprocessing the oblique photography data to obtain preprocessed oblique photography data; S2. Perform segmentation training on the pre-processed oblique photography data through AI image recognition technology to obtain a segmentation mask map; S3, performing geometric adjustment and topological repair on the segmented target object in the segmentation mask image to establish a monomer model; S4, spatial attributes and geometric features of the bound monomer model; S5. Build a tree structure of the monomer model to adapt to multi-scenario business needs and complete the image monomerization method based on real-scene three-dimensional.

[0008] Further: In said S1, the method for pre-processing comprises the following sub-steps: S11, using a Gaussian filter algorithm to perform noise reduction processing on the oblique photography data to smooth out noise interference; S12, based on the image after noise reduction, extracting the image edge through the Laplacian operator to enhance the image details; S13. Eliminate color distortion through a linear mapping function to obtain preprocessed oblique photography data.

[0009] The beneficial effect of the above further scheme is: by pre-processing the oblique photography data, the image quality is improved through denoising, enhancement and detail restoration, which is beneficial to the subsequent steps.

[0010] Further: In S11, the expression of the Gaussian filtering algorithm is specifically:

[0011] in, is the pixel value of the original image after noise reduction processing, is the pixel value of the original image, is the filter window, is the Gaussian weight; In S12, the expression of the image after detail enhancement is specifically:

[0012] In the formula, To enhance the image pixel value after detail enhancement, λ is the enhancement coefficient, which is used to control the degree of edge prominence. is the edge gradient of the image, and its specific expression is:

[0013] In the formula, is the second-order derivative of the image in the horizontal direction, reflecting the rate of change of the brightness of the image in this direction. It is the second-order derivative of the image in the vertical direction, reflecting the brightness change rate of the image in this direction; In S13, the pre-processed oblique photography data The specific expression is:

[0014] In the formula, is the gain coefficient, This is the color cast correction value.

[0015] Further: S2 comprises the following sub-steps: S21, inputting the preprocessed oblique photography data and the corresponding label data into the deep learning network, and optimizing the deep learning network through multiple rounds of training; S22, correcting the output of the deep learning network by manual annotation to obtain a trained deep learning network; S23. Output a segmentation mask image through the trained deep learning network, and each pixel in the segmentation mask image is assigned a corresponding category label.

[0016] The beneficial effect of the above further scheme is that the segmentation training of oblique photography data can be carried out through AI image recognition technology, and the segmentation accuracy can be optimized in combination with user annotation.

[0017] Further: In S21, the parameters of the deep learning network are optimized by a cross entropy loss function, and the cross entropy loss function The specific expression is:

[0018] In the formula, is the true label for each pixel, is the model prediction value, N 1 is the total number of samples.

[0019] Further: S3 includes the following sub-steps: S31, obtaining a model of the segmented target object in the segmentation mask image, fitting the geometric shape of the model by the least square method, correcting the segmentation error, and calculating the deviation between the vertex and the fitting surface, and correcting the vertex whose deviation is greater than the deviation threshold; S32, using the Bezier curve interpolation method to repair the model boundary to ensure the continuity of the model boundary; S33. Use the topology optimization algorithm to repair the non-manifold edges in the model, remove isolated vertices, and merge redundant faces to ensure the integrity of the model's topological structure and establish a single model.

[0020] The beneficial effect of the above further solution is: geometric adjustment and topological repair are performed on the segmented target object to ensure the integrity and accuracy of the monomer model.

[0021] Further: In S31, the expression of the geometric shape of the least squares fitting model is specifically:

[0022] In the formula, is the vertex coordinate of the point cloud, A is the first plane equation parameter, B is the second plane equation parameter, C is the third plane equation parameter, D is the fourth plane equation parameter, x i is the horizontal axis coordinate of the point cloud vertex, y i is the vertical coordinate of the point cloud vertex, z i is the vertical axis coordinate of the point cloud vertex, i is the ordinal number of the point cloud vertex, N 2 is the total number of point cloud vertices; In S32, the formula of the Bezier curve interpolation method is Specifically:

[0023] In the formula, P 0 is the first control point, P 1 is the second control point, P 2 is the third control point, t is the interpolation parameter, .

[0024] Further: S4 includes the following sub-steps: S41, calculating the geometric center of the monomer model; S42, calculating geometric properties of the monomer model, including volume and surface area; S43. Add spatial attributes and geometric features to the monomer model. The geometric features include geometric center and geometric attributes. The spatial attributes include name, purpose, hierarchical relationship and geographic coordinates.

[0025] The beneficial effect of the above further solution is: binding spatial attributes to the model to facilitate subsequent storage, management and application.

[0026] Further: In said S41, the geometric center of the monomer model is calculated E The specific expression is:

[0027] In the formula, For the j Vertex coordinates, is the total number of vertices; In S42, the volume V The specific expression is:

[0028] In the formula, For the The area of ​​the triangle patch, For the The vertical height of the triangle patch to the reference plane, M is the total number of triangle patches; Surface Area S The specific expression is: .

[0029] Further: S5 includes the following sub-steps: S51, using a tree structure to organize the monomer model, the tree structure includes a number of root nodes, a number of child nodes under the root nodes, each child node stores the spatial attributes and geometric features of the monomer model; S52. The tree structure supports dynamic expansion and adjustment, allowing the addition, deletion or modification of root nodes and child nodes to meet the needs of different business scenarios; S53. The tree structure is output in a standardized format to meet the business needs of multiple scenarios and complete the image monomerization method based on real-scene three-dimensional.

[0030] The beneficial effect of the above further solution is that the tree structure of the monomer model is constructed, which can meet the needs of the monomer model to be organized in a hierarchical tree structure for dynamic reorganization.

[0031] The beneficial effects of the present invention are: (1) The present invention provides an image monomerization method based on real-scene three-dimensional, which improves image quality through data preprocessing, optimizes segmentation accuracy by using AI image recognition technology combined with user annotation, and performs geometric adjustment and topological repair on the model to ensure model integrity and accuracy; at the same time, it binds spatial attributes to the model, supports the compilation and dynamic reorganization of tree structures, and significantly enhances the organizational ability and application flexibility of the model. It is suitable for the needs of multiple fields such as architectural design, urban planning, and urban management, and has high practical value and economic benefits. It solves the problems of insufficient data quality, insufficient segmentation accuracy, insufficient model optimization, and weak attribute organization ability in the prior art, and improves the applicability and practical value of the model.

[0032] (2) The monomerization method based on real-scene three-dimensional images of the present invention can efficiently and accurately generate and manage monomer models, providing data support for further spatial analysis and applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The present invention is a flowchart of a method for image singulation based on real-scene three-dimensional. DETAILED DESCRIPTION

[0034] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0035] like Figure 1 As shown, in one embodiment of the present invention, a method for image singulation based on real-scene three-dimensional includes the following steps: S1, acquiring oblique photography data, and preprocessing the oblique photography data to obtain preprocessed oblique photography data; S2. Perform segmentation training on the pre-processed oblique photography data through AI image recognition technology to obtain a segmentation mask map; S3, performing geometric adjustment and topological repair on the segmented target object in the segmentation mask image to establish a monomer model; S4, spatial attributes and geometric features of the bound monomer model; S5. Build a tree structure of the monomer model to adapt to multi-scenario business needs and complete the image monomerization method based on real-scene three-dimensional.

[0036] In S1, the method for pre-processing comprises the following steps: S11, using a Gaussian filter algorithm to perform noise reduction processing on the oblique photography data to smooth out noise interference; S12, based on the image after noise reduction, extracting the image edge through the Laplacian operator to enhance the image details; S13. Eliminate color distortion through a linear mapping function to obtain preprocessed oblique photography data.

[0037] In this embodiment, the preprocessing method is specifically to repair and improve the quality of the oblique photography data by means of noise reduction, detail enhancement and color correction in sequence.

[0038] In S11, the expression of the Gaussian filtering algorithm is specifically:

[0039] in, is the pixel value of the original image after noise reduction processing, is the pixel value of the original image, is the filter window, is the Gaussian weight; Noise in an image can affect subsequent image analysis and object recognition, so noise reduction is the basis for improving image quality. The Gaussian filter algorithm removes noise by smoothing the image. In this process, the calculation of each pixel value depends on the weighted average of the pixels in its surrounding neighborhood. Specifically, when using Gaussian filtering, the value of each pixel is weighted averaged with other pixels in its neighborhood, and the weight is determined by the Gaussian function, and the shape of the Gaussian function determines the influence range of each pixel neighborhood.

[0040] In S12, the expression of the image after detail enhancement is specifically:

[0041] In the formula, To enhance the image pixel value after detail enhancement, λ is the enhancement coefficient, which is used to control the degree of edge prominence. is the edge gradient of the image, and its specific expression is:

[0042] In the formula, is the second-order derivative of the image in the horizontal direction, reflecting the rate of change of the brightness of the image in this direction. It is the second-order derivative of the image in the vertical direction, reflecting the brightness change rate of the image in this direction; In this embodiment, detail enhancement aims to make the object contour clearer by strengthening the edge information of the image. The Laplace operator is widely used for image edge extraction. It obtains the degree of image change by calculating the image gradient, thereby enhancing the image details. Through detail enhancement, the edges and subtle changes in the image can be captured more clearly, which is helpful for subsequent object segmentation.

[0043] In S13, the pre-processed oblique photography data The specific expression is:

[0044] In the formula, is the gain factor, used to adjust the brightness, It is the color deviation correction value, which is used to adjust the color tone deviation.

[0045] In this embodiment, due to the inconsistent shooting conditions and lighting environment of oblique photography, color distortion may occur, affecting the realism of the image. The present invention can eliminate or reduce such color difference and restore the real color of the object through color correction. Color correction is usually performed through a linear mapping method, the purpose of which is to adjust the brightness and hue of the image to make it closer to the real value.

[0046] The S2 comprises the following sub-steps: S21, inputting the preprocessed oblique photography data and the corresponding label data into the deep learning network, and optimizing the deep learning network through multiple rounds of training; S22, correcting the output of the deep learning network by manual annotation to obtain a trained deep learning network; S23. Output a segmentation mask image through the trained deep learning network, and each pixel in the segmentation mask image is assigned a corresponding category label.

[0047] In S21, the parameters of the deep learning network are optimized by the cross entropy loss function. The cross entropy loss function The specific expression is:

[0048] In the formula, is the true label for each pixel, is the model prediction value, N 1 is the total number of samples.

[0049] In this embodiment, model training is to learn by inputting labeled data, using a deep learning network to automatically extract features from images, and learn to associate these features with object categories. During the training process, the present invention uses a cross entropy loss function to optimize the parameters of the model to ensure that the predicted output is as close as possible to the true label.

[0050] In S22, the user can optimize through manual annotation. Since the deep learning network may misjudge some boundaries or objects in complex environments, manually corrected annotations can help the model correct errors, especially for difficult scenes or objects with unclear boundaries. Manually optimized annotations will further improve the accuracy of the model.

[0051] After training and optimization, the deep learning network is able to generate a segmentation mask in which each pixel is assigned a class label. The segmentation mask provides a very important input for subsequent steps and can be used to build a 3D model or perform further geometric analysis.

[0052] The S3 comprises the following sub-steps: S31, obtaining a model of the segmented target object in the segmentation mask image, fitting the geometric shape of the model by the least square method, correcting the segmentation error, and calculating the deviation between the vertex and the fitting surface, and correcting the vertex whose deviation is greater than the deviation threshold; S32, using the Bezier curve interpolation method to repair the model boundary to ensure the continuity of the model boundary; S33. Use the topology optimization algorithm to repair the non-manifold edges in the model, remove isolated vertices, and merge redundant faces to ensure the integrity of the model's topological structure and establish a single model.

[0053] In S31, the expression of the geometric shape of the least squares fitting model is specifically:

[0054] In the formula, is the vertex coordinate of the point cloud, A is the first plane equation parameter, B is the second plane equation parameter, C is the third plane equation parameter, D is the fourth plane equation parameter, x i is the horizontal axis coordinate of the point cloud vertex, y i is the vertical coordinate of the point cloud vertex, z i is the vertical axis coordinate of the point cloud vertex, i is the ordinal number of the point cloud vertex, N 2 is the total number of point cloud vertices; In this embodiment, the main task of geometric adjustment is to correct the geometric shape deviation in the model to ensure that it conforms to the shape of the actual object. A commonly used geometric adjustment method is least squares fitting, which adjusts the shape of the object by minimizing the fitting error. For example, when fitting a plane, the least squares method optimizes the fitting result by calculating the distance from each point to the plane and minimizing the sum of the squares of these distances.

[0055] In S32, the formula of the Bezier curve interpolation method is Specifically:

[0056] In the formula, P 0 is the first control point, P 1 is the second control point, P 2 is the third control point, t is the interpolation parameter, .

[0057] In this embodiment, the boundary repair is mainly to solve the jagged edges and discontinuous object contours in the model. The present invention uses the Bezier curve interpolation method to smooth these boundaries, making the edges of the model smoother, and the control points of the Bezier curve can be adjusted according to actual needs.

[0058] The S4 comprises the following sub-steps: S41, calculating the geometric center of the monomer model; S42, calculating geometric properties of the monomer model, including volume and surface area; S43. Add spatial attributes and geometric features to the monomer model. The geometric features include geometric center and geometric attributes. The spatial attributes include name, purpose, hierarchical relationship and geographic coordinates.

[0059] In S41, the geometric center of the monomer model is calculated E The specific expression is:

[0060] In the formula, For the j Vertex coordinates, is the total number of vertices; In this embodiment, the geometric center of each object is obtained by calculating the average value of all vertex coordinates. The geometric center is the "center of gravity" of the object and plays a role in positioning in space.

[0061] In S42, the volume V The specific expression is:

[0062] In the formula, For the The area of ​​the triangle patch, For the The vertical height of the triangle patch to the reference plane, M is the total number of triangle patches; Surface Area S The specific expression is: .

[0063] In this embodiment, the geometric attributes include volume and surface area, which are basic physical features of the model and can reflect the size and surface characteristics of the object. The volume is calculated by the area of ​​each triangular face and its height to the reference plane, and the surface area is the sum of the areas of all faces.

[0064] The S5 comprises the following sub-steps: S51, using a tree structure to organize the monomer model, the tree structure includes a number of root nodes, a number of child nodes under the root nodes, each child node stores the spatial attributes and geometric features of the monomer model; In this embodiment, the tree structure T The specific expression is:

[0065] In the formula, is the root node, is the first child node set, For the kA collection of child nodes establishes a tree structure so that the model can be managed hierarchically, which is convenient for storage and access.

[0066] S52. The tree structure supports dynamic expansion and adjustment, allowing the addition, deletion or modification of root nodes and child nodes to meet the needs of different business scenarios; S53. The tree structure is output in a standardized format to meet the business needs of multiple scenarios and complete the image monomerization method based on real-scene three-dimensional.

[0067] The beneficial effects of the present invention are as follows: the present invention provides an image singulation method based on real-scene three-dimensional, improves image quality through data preprocessing, optimizes segmentation accuracy by using AI image recognition technology combined with user annotation, and performs geometric adjustment and topological repair on the model to ensure model integrity and accuracy; at the same time, it binds spatial attributes to the model, supports the compilation and dynamic reorganization of tree structures, significantly enhances the organizational ability and application flexibility of the model, is suitable for the needs of multiple fields such as architectural design, urban planning, and urban management, has high practical value and economic benefits, solves the problems of insufficient data quality, insufficient segmentation accuracy, insufficient model optimization, and weak attribute organization ability in the prior art, and improves the applicability and practical value of the model.

[0068] The monomerization method based on real-scene three-dimensional images of the present invention can efficiently and accurately generate and manage monomer models, providing data support for further spatial analysis and application.

[0069] In the description of the present invention, it is necessary to understand that the orientation or positional relationship indicated by the terms "center", "thickness", "upper", "lower", "horizontal", "top", "bottom", "inner", "outer", "radial", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", and "third" are used only for descriptive purposes, and cannot be understood as indicating or implying the relative importance or the number of implicitly specified technical features. Therefore, the features defined by "first", "second", and "third" may explicitly or implicitly include one or more of the features.

Claims

1. A method for image singulation based on real scene three-dimensional, characterized in that: The following steps are involved: S1, acquiring oblique photography data, and preprocessing the oblique photography data to obtain preprocessed oblique photography data; S2. Perform segmentation training on the pre-processed oblique photography data through AI image recognition technology to obtain a segmentation mask map; S3, performing geometric adjustment and topological repair on the segmented target object in the segmentation mask image to establish a monomer model; S4, spatial attributes and geometric features of the bound monomer model; S5. Build a tree structure of the monomer model to adapt to multi-scenario business needs and complete the image monomerization method based on real-scene three-dimensional.

2. The image singulation method based on real scene three-dimensional according to claim 1, characterized in that: In S1, the method for pre-processing comprises the following steps: S11, using a Gaussian filter algorithm to perform noise reduction processing on the oblique photography data to smooth out noise interference; S12, based on the image after noise reduction, extracting the image edge through the Laplacian operator to enhance the image details; S13, eliminating color distortion through a linear mapping function to obtain pre-processed oblique photography data.

3. The image singulation method based on real scene three-dimensional according to claim 2, characterized in that: In S11, the expression of the Gaussian filtering algorithm is specifically: in, is the pixel value of the original image after noise reduction processing, is the pixel value of the original image, is the filter window, is the Gaussian weight; In S12, the expression of the image after detail enhancement is specifically: In the formula, To enhance the image pixel value after detail enhancement, λ is the enhancement coefficient, which is used to control the degree of edge prominence. is the edge gradient of the image, and its specific expression is: In the formula, is the second-order derivative of the image in the horizontal direction, reflecting the rate of change of the brightness of the image in this direction. It is the second-order derivative of the image in the vertical direction, reflecting the brightness change rate of the image in this direction; In S13, the pre-processed oblique photography data The specific expression is: In the formula, is the gain coefficient, This is the color cast correction value.

4. The image singulation method based on real scene three-dimensional according to claim 1, characterized in that: The S2 comprises the following sub-steps: S21, inputting the preprocessed oblique photography data and the corresponding label data into the deep learning network, and optimizing the deep learning network through multiple rounds of training; S22, correcting the output of the deep learning network by manual annotation to obtain a trained deep learning network; S23. Output a segmentation mask image through the trained deep learning network, and each pixel in the segmentation mask image is assigned a corresponding category label.

5. The image singulation method based on real scene three-dimensional according to claim 4, characterized in that: In S21, the parameters of the deep learning network are optimized by the cross entropy loss function. The cross entropy loss function The specific expression is: In the formula, is the true label for each pixel, is the model prediction value, N 1 is the total number of samples.

6. The image singulation method based on real scene three-dimensional according to claim 1, characterized in that: The S3 comprises the following sub-steps: S31, obtaining a model of the segmented target object in the segmentation mask image, fitting the geometric shape of the model by the least square method, correcting the segmentation error, and calculating the deviation between the vertex and the fitting surface, and correcting the vertex whose deviation is greater than the deviation threshold; S32, using the Bezier curve interpolation method to repair the model boundary to ensure the continuity of the model boundary; S33. Use the topology optimization algorithm to repair the non-manifold edges in the model, remove isolated vertices, and merge redundant faces to ensure the integrity of the model's topological structure and establish a single model.

7. The image singulation method based on real scene three-dimensional according to claim 6, characterized in that: In S31, the expression of the geometric shape of the least squares fitting model is specifically: In the formula, is the vertex coordinate of the point cloud, A is the first plane equation parameter, B is the second plane equation parameter, C is the third plane equation parameter, D is the fourth plane equation parameter, x i is the horizontal axis coordinate of the point cloud vertex, y i is the vertical coordinate of the point cloud vertex, z i is the vertical axis coordinate of the point cloud vertex, i is the ordinal number of the point cloud vertex, N 2 is the total number of point cloud vertices; In S32, the formula of the Bezier curve interpolation method is Specifically: In the formula, P 0 is the first control point, P 1 is the second control point, P 2 is the third control point, t is the interpolation parameter, .

8. The image singulation method based on real scene three-dimensional according to claim 1, characterized in that: The S4 comprises the following sub-steps: S41, calculating the geometric center of the monomer model; S42, calculating geometric properties of the monomer model, including volume and surface area; S43. Add spatial attributes and geometric features to the monomer model. The geometric features include geometric center and geometric attributes. The spatial attributes include name, purpose, hierarchical relationship and geographic coordinates.

9. The image singulation method based on real scene three-dimensional according to claim 8, characterized in that: In S41, the geometric center of the monomer model is calculated E The specific expression is: In the formula, For the j Vertex coordinates, is the total number of vertices; In S42, the volume V The specific expression is: In the formula, For the The area of ​​the triangle patch, For the The vertical height of the triangle patch to the reference plane, M is the total number of triangle patches; Surface Area S The specific expression is: 。 10. The image singulation method based on real scene 3D according to claim 1, characterized in that: The S5 comprises the following sub-steps: S51, using a tree structure to organize the monomer model, the tree structure includes a number of root nodes, a number of child nodes under the root nodes, each child node stores the spatial attributes and geometric features of the monomer model; S52. The tree structure supports dynamic expansion and adjustment, allowing the addition, deletion or modification of root nodes and child nodes to meet the needs of different business scenarios; S53. The tree structure is output in a standardized format to meet the business needs of multiple scenarios and complete the image monomerization method based on real-scene three-dimensional.

Images