Three-dimensional modeling method based on photogrammetry

Through manual modeling and projective geometric calculation, a three-dimensional realistic streamlined model is constructed, which solves the problem of large amount of three-dimensional modeling data in the existing technology, and realizes rapid and streamlined modeling of regular surface entities, and generates three-dimensional consistent with reality. Model.

CN118918268APending Publication Date: 2024-11-08西安应用光学研究所
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411208962.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The data generated by the existing photogrammetry three-dimensional modeling technology is huge and difficult to concise, and it is impossible to quickly generate streamlined models. Especially for physical entities with regular surfaces, the existing technology cannot be effectively applied.

Method used

Manual modeling is used to generate the initial embryo type, and the main body space topology is constructed through the combination of basic geometric bodies, the three-dimensional coordinates of feature points are obtained, the feature points are corrected, the three-dimensional simplified model is constructed, the detailed structural features are ignored, and the vertex map parameters are calculated using projective geometry. Generate a three-dimensional model of streamlined data.

Benefits of technology

It realizes concise three-dimensional modeling of regular surface entities such as equipment, vehicles, buildings, etc., generates a three-dimensional model of streamlined data, quickly obtains the outer contour, and is compatible with modeling requirements at different levels, reducing the amount of data and improving modeling efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918268A_ABST
    Figure CN118918268A_ABST
Patent Text Reader

Abstract

The invention provides a three-dimensional modeling method based on photogrammetry, which solves the problem that the three-dimensional modeling data volume is huge and the modeling cannot be concise, and comprises the following steps: step 1, generating an initial embryo of an artificial physical entity by adopting artificial modeling, the artificial physical entity being various entities created by human beings, including equipment, vehicles or buildings; step 2, acquiring three-dimensional coordinates of feature points in the artificial physical entity; 3, correcting the feature points of the initial embryonic model by using the three-dimensional coordinates; and step 4, mapping the initial embryonic model and then constructing a three-dimensional simulation simplified model which ignores the detail structure characteristics of the artificial physical entity. For artificial physical entities with regular surfaces, such as equipment, vehicles, buildings and the like, a three-dimensional model with simplified data is generated through a simple three-dimensional modeling method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of three-dimensional modeling, and in particular relates to a three-dimensional modeling method based on photogrammetry. Background Art

[0002] At present, photogrammetry 3D modeling uses drones, ground cameras and other equipment to shoot targets from different angles and heights to obtain image data. These image data are finally generated into 3D models through preprocessing, feature extraction, matching, point cloud generation and other steps, and are widely used in urban planning, architectural design, disaster assessment, cultural relics protection and other fields.

[0003] Specifically, in the existing photogrammetric 3D modeling technology, the photographic acquisition technology uses a multi-lens camera in the air or on the ground to shoot the target from multiple angles to obtain image information. Preprocessing performs operations such as correction, distortion removal, matching, and registration on the collected images to improve the image quality and data consistency. Point cloud technology classifies, filters, resamples, and extracts feature point clouds of targets such as buildings, roads, and trees through operations such as point cloud data obtained through photogrammetry. 3D modeling technology uses computer vision and 3D reconstruction algorithms to convert the processed point cloud data into a 3D model.

[0004] The above-mentioned 3D modeling is based on the method of converting point cloud data into a 3D model based on images and 3D reconstruction algorithms. Although it can generate a 3D model, it has high dependence and requirements on 3D software, and the functions of the 3D software can only be converted into a 3D model under the premise of meeting the modeling requirements, otherwise it will affect the modeling effect of the 3D model.

[0005] The invention patent publication with publication number 116543117A discloses a high-precision large-scene 3D modeling method for drone images. The drone aerial image set is divided according to circular orbital tracks. After feature extraction, matching and geometric verification of the divided image set, the SFM algorithm is used to restore the drone camera's pose, and then the sub-NeRF is trained. Finally, the sub-NeRF around the target view is merged to complete the implicit construction of the large-scene 3D model. Although the patent document completes the construction of the 3D model through feature extraction, it implicitly constructs the 3D model based on the large scene and cannot be applied to physical entities with regular surfaces.

[0006] The invention patent publication document with the publication number of 115775299A discloses a method for optimizing three-dimensional modeling of human faces, wherein a light source is arranged on the opposite side of the human face, and the light generated by the light source is directly irradiated on the soft light plate and then scattered on the human face. The photographic device and the light source arranged opposite to the human face are centered on the center point of the human face, and the focus of the photographic device moves up and down and left and right along the center of the circle to comprehensively photograph the surface of the human face; the human face is divided into four areas by the center point and the displacement axis of the photographic device through the focus shooting of the photographic device, and the human face in the four areas is captured to obtain two-dimensional image data of the human face; image data of the human face in a single area from multiple angles is obtained, and the image data is synthesized from the two-dimensional image data into three-dimensional modeling data by photogrammetry; and a three-dimensional model of the human face is constructed by parametric modeling. Although the patent document effectively avoids the influence of light on facial features, it needs to merge the texture image with the standard texture. The generation and fusion process of the standard texture is relatively complicated, and rapid modeling cannot be achieved.

[0007] In addition, the modeling method based on visual measurement in the existing technology will generate a large number of control points. On the one hand, the amount of data is huge. On the other hand, the depth error of the spatial point cloud obtained based on visual measurement makes the originally smooth surface show concave and convex features, which requires manual refinement to be processed. Summary of the invention

[0008] The purpose of the present invention is to solve the shortcomings of the prior art that the amount of 3D modeling data is huge and cannot be modeled concisely, and to provide a 3D modeling method based on photogrammetry. For artificial physical entities with regular surfaces such as equipment, vehicles, buildings, etc., a concise 3D modeling method is used to generate a 3D model with simplified data.

[0009] To achieve the above purpose, the technical solution provided by the present invention is:

[0010] The present invention provides a three-dimensional modeling method based on photogrammetry, comprising the following steps:

[0011] Step 1: Generate an initial prototype of an artificial physical entity by artificial modeling, wherein the artificial physical entity is various entities created by humans, including equipment, vehicles or buildings;

[0012] Step 2: obtaining the three-dimensional coordinates of the feature points in the artificial physical entity;

[0013] Step 3: Correcting the feature points of the initial embryo using the three-dimensional coordinates;

[0014] Step 4: Mapping the initial embryonic form to construct a three-dimensional realistic simplified model, wherein the three-dimensional realistic simplified model ignores detailed structural features of the artificial physical entity.

[0015] In one embodiment of the present invention, the step 1 comprises:

[0016] Artificial modeling is performed by combining operations of basic geometric bodies to generate an initial embryonic form for expressing the main spatial topological structure of the artificial physical entity; wherein the basic geometric bodies are spatial configurations accurately defined by a small number of parameters, including cuboids, cylinders or spheres; and the combination operations include merging or cutting the basic geometric bodies;

[0017] During the artificial modeling process, there is a definite corresponding relationship between each vertex of the initial embryo and a specific point of the artificial physical entity.

[0018] In one embodiment of the present invention, the step one further comprises:

[0019] In the process of manual modeling, minute details on the appearance of the artificial physical entity are ignored, wherein the minute details include: tiny protrusions or undulations on the surface of the artificial physical entity.

[0020] In one embodiment of the present invention, the step 2 comprises:

[0021] Step 21: Acquire entity feature points of the artificial physical entity and measure pixel coordinates of the entity feature points;

[0022] Step 22: Calculate the three-dimensional coordinates of the entity feature points according to the pixel coordinates.

[0023] In one embodiment of the present invention, step 21 comprises:

[0024] According to the main spatial topological structure of the artificial physical entity, a plurality of viewing angles are selected from a plurality of viewing angles to photograph the artificial physical entity to obtain an image of the entity, wherein the plurality of viewing angles include a front view, a rear view, a left view, a right view, a top view, and a bottom view;

[0025] Processing the entity image to obtain a processed image;

[0026] The horizontal field of view angle and the vertical field of view angle of the processed image are calculated according to the shooting parameters when shooting the artificial physical entity and the processing parameters in the process of processing the entity image, and the pixel coordinates of the entity feature points in the processed image are obtained based on the horizontal field of view angle and the vertical field of view angle.

[0027] In one embodiment of the present invention, the artificial physical entity is photographed by selecting a plurality of viewing angles from a plurality of viewing angles, and the photographing parameters during the photographing include: a three-dimensional vector V of the viewpoint; x 、3D vector of sight V e 、Three-dimensional vector V on the image plane u ;

[0028] The processing of the entity image to obtain a processed image includes: cutting out surrounding objects of the artificial physical entity in the entity image to retain only the artificial physical entity;

[0029] The calculating the horizontal field of view angle and the vertical field of view angle of the processed image according to the shooting parameters when shooting the artificial physical entity and the processing parameters in the process of processing the entity image includes:

[0030] The vertical field angle v, the aspect ratio k, the number of horizontal pixels n0, and the number of vertical pixels n1 of the physical entity image are calculated; wherein the vertical field angle v and the aspect ratio k are calculated by formula (1):

[0031]

[0032] In formula (1), v c Indicates the vertical field of view of the camera device; k c Indicates the aspect ratio of the picture; n c0 Indicates the horizontal resolution; n c1 Indicates vertical resolution;

[0033] Wherein, the number of horizontal pixels n0 and the number of vertical pixels n1 are obtained through the physical entity image;

[0034] The acquiring pixel coordinates of the entity feature points in the processed image based on the horizontal field of view angle and the vertical field of view angle includes: measuring the pixel coordinates of the entity feature points on the physical entity image.

[0035] In one embodiment of the present invention, step 22 comprises:

[0036] Based on the horizontal field of view angle and the vertical field of view angle of the processed image obtained in step 21, the pixel coordinates of the entity feature point obtained in step 21 are converted into normalized coordinates in image space; wherein the normalized coordinates in image space are coordinate values ​​after the scale transformation of the pixel coordinates, so that the vertical coordinate of the upper edge of the processed image is 1.0, and the vertical coordinate of the lower edge of the processed image is -1.0;

[0037] The image space normalized coordinates of the same entity feature point in the two processed images are jointly calculated to obtain the three-dimensional coordinates of the entity feature point.

[0038] In one embodiment of the present invention, in step 22:

[0039] Define the normalized coordinate X in the image space, the range of its longitudinal coordinate X1 is [-1.0, 1.0], and the range of its transverse coordinate X0 is [-k, k], then: n0 represents horizontal pixels, n1 represents vertical pixels, N0 represents the number of pixels from pixel coordinate N to the left edge of the processed image, and N1 represents the number of pixels from pixel coordinate N to the top edge of the processed image;

[0040] For any view taken from several perspectives, a temporary coordinate system x'y'z' is established by replacing the axis direction, so that the three-dimensional vector V of the viewpoint in the shooting parameters during shooting x Pointing to the negative direction of the x' axis, the three-dimensional vector V on the image plane u Pointing to the positive direction of the z' axis, using d as the initial estimated value of the distance of the entity feature point, the following formula (2) and formula (3) are used to calculate the initial value of the y'z' coordinate of the entity feature point;

[0041]

[0042]

[0043] In formula (2) and formula (3), X1 represents the longitudinal coordinate of the normalized coordinate X in the image space, and X0 represents the transverse coordinate of the normalized coordinate X in the image space;

[0044] Obtain two components of the three-dimensional vector of the feature point coordinates according to the initial value of the y'z' coordinate of the entity feature point, find another view in which the entity feature point is visible, and calculate another component of the three-dimensional vector of the feature point coordinates based on the other view using formula (2) or formula (3);

[0045] The distance of the entity feature point is corrected based on the initial value of the y'z' coordinate of the entity feature point, thereby correcting the two coordinate components associated with the view corresponding to the initial value of the coordinate. Specifically, the coordinate components are corrected and calculated by formula (4);

[0046]

[0047] In formula (4), d represents the initial estimated value of the feature point distance;

[0048] After all coordinate points are corrected, the longest side of their bounding boxes is calculated, and all coordinate values ​​of all coordinate points are scaled so that the longest side of the bounding box is 1.0. The scale transformation factor is recorded as the scale feature of the 3D realistic simplified model.

[0049] In one embodiment of the present invention, the step three comprises:

[0050] The three-dimensional coordinates calculated in step 2 are assigned to the corresponding vertices of the initial embryo shape in step 1 to complete the construction of a white mold corresponding to the artificial physical entity; wherein the white mold is a three-dimensional model without surface texture.

[0051] In one embodiment of the present invention, the step 4 comprises:

[0052] The processed image corresponding to the initial embryo shape after cutting is used as a texture map resource of the white mold, and vertex map parameters are calculated according to the projective geometry, and a three-dimensional realistic simplified model is constructed based on the texture map resources and the vertex map parameters; wherein the three-dimensional realistic simplified model is a three-dimensional model that can be displayed in a three-dimensional scene and is consistent with the visual sense of the corresponding artificial physical entity but ignores the detailed structural features;

[0053] The degree of refinement of the artificial modeling is determined according to the application requirements of the three-dimensional simulated simplified model, and the detailed structural features of the artificial physical entity are ignored based on the degree of refinement, thereby greatly reducing the data volume of the three-dimensional simulated simplified model.

[0054] The advantages of the present invention are:

[0055] 1. The present invention generates a three-dimensional model with simplified data for artificial physical entities with regular surfaces such as equipment, vehicles, and buildings through a concise three-dimensional modeling method.

[0056] 2. The present invention aims at man-made physical entities such as vehicles and buildings that are composed of planes and smooth curved surfaces. The initial prototype is constructed through manual editing, and the surface details of the entity can be ignored, thereby greatly simplifying the model data.

[0057] 3. The present invention is based on the measurement of real-object photos under controllable shooting conditions, and can quickly obtain an outer contour consistent with an artificial physical entity.

[0058] 4. The present invention automatically generates vertex mapping parameters based on projective geometry calculations, and can quickly generate realistic simplified models.

[0059] 5. The present invention adopts manual editing to construct the initial embryonic form, which allows the human-in-the-loop production process, and allows the local details of the three-dimensional model to be refined according to needs, which is compatible with different levels of three-dimensional modeling needs.

[0060] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0062] Figure 1 : A flow chart of a three-dimensional modeling method based on photogrammetry provided by the present invention;

[0063] Figure 2 : The present invention provides a layout diagram of stitching photos into texture map resources;

[0064] Figure 3 : The meaning of the texture coordinates provided by the present invention;

[0065] Figure 4 : Modeling object in the example provided by the present invention;

[0066] Figure 5 : The initial embryo shape and characteristic points provided by the present invention;

[0067] Figure 6 : The photo taken at a normal viewing angle provided by the present invention;

[0068] Figure 7 : The photo taken at the left viewing angle provided by the present invention;

[0069] Figure 8 : The photo taken from a bird's-eye view provided by the present invention;

[0070] Fig. 9 : The photo taken at the bottom viewing angle provided by the present invention;

[0071] Fig.10 : The front view cropping photo provided by the present invention;

[0072] Fig.11 : The left-view cropped photo provided by the present invention;

[0073] Fig.12 : The top-view cropped photo provided by the present invention;

[0074] Fig.13 : The bottom view cropped photo provided by the present invention;

[0075] Fig.14 : The distribution of feature points on the front-view cropped photo provided by the present invention;

[0076] Fig.15 : The distribution of feature points on the left-view cropped photo provided by the present invention;

[0077] Fig.16 : The completed modeling effect provided by the present invention;

[0078] Fig.17 : The comparison table of the selected view number is adjusted when the selected view angle photo is not obtained provided by the present invention;

[0079] Fig.18 : The pixel coordinate table of the characteristic points in the train carriage provided by the present invention;

[0080] Fig.19 : The 16 feature points provided by the present invention correspond to the coordinate tables;

[0081] Fig. 20: The present invention provides a table for determining and selecting view numbers. DETAILED DESCRIPTION

[0082] Embodiments of the present invention are described in detail below. The embodiments are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0083] See also Figure 1 The embodiment of the present invention provides a three-dimensional modeling method based on photogrammetry, comprising the following steps 1 to 4:

[0084] Step 1: Generate an initial prototype of an artificial physical entity by artificial modeling, wherein the artificial physical entity is a variety of entities created by humans, including equipment, vehicles or buildings.

[0085] Specifically, step one includes:

[0086] Artificial modeling is performed through the combination operation of basic geometric bodies to generate an initial embryonic form for expressing the main spatial topological structure of an artificial physical entity; wherein the basic geometric body is a spatial configuration accurately defined by a small number of parameters, including a cuboid, a cylinder or a sphere; the combination operation includes merging or cutting the basic geometric bodies;

[0087] In the process of artificial modeling, there is a definite correspondence between the vertices of the initial embryo and the specific points of the artificial physical entity.

[0088] More specifically, in addition to the above contents, step one also includes: ignoring minute details of the appearance of the artificial physical entity during the artificial modeling process; wherein the minute details include: tiny protrusions or undulations on the surface of the artificial physical entity.

[0089] The embodiment of the present invention generates an initial embryonic form that can express the spatial topological structure of the main body of a physical entity through artificial modeling by combining basic geometric bodies. Basic geometric bodies include spatial configurations such as cuboids, cylinders, and spheres that can be accurately defined by a small number of parameters. The combination operation includes merging and cutting geometric bodies. When establishing the initial embryonic form, tiny details that have little effect on the appearance, such as small protrusions and undulations on the surface, can be ignored, thereby streamlining the amount of model data.

[0090] The specific operation of artificial modeling is as follows: according to the application requirements, determine the level of refinement of the model to be obtained, and ignore the structural details of the modeling objects below the specified level. Perform artificial modeling through the combination of basic geometric bodies to generate an initial embryonic form that can express the spatial topological structure of the main body of the physical entity. The vertices p of the initial embryonic form i Should be consistent with a specific point P on the surface of the physical entity i The scale of the initial embryo is controlled so that the longest side of its bounding box is 1.0.

[0091] Step 2: Obtain the three-dimensional coordinates of feature points in artificial physical entities.

[0092] Specifically, step two includes:

[0093] Step 21: Acquire entity feature points of the artificial physical entity and measure the pixel coordinates of the entity feature points.

[0094] The above step 21 includes:

[0095] According to the main spatial topological structure of the artificial physical entity, a plurality of perspectives are selected from a plurality of perspectives to shoot the artificial physical entity to obtain an image of the entity, wherein the plurality of perspectives include front view, rear view, left view, right view, top view, and bottom view;

[0096] Processing the entity image to obtain a processed image;

[0097] The horizontal and vertical angles of view of the processed image are calculated according to the shooting parameters when shooting the artificial physical entity and the processing parameters in the process of processing the entity image, and the pixel coordinates of the entity feature points in the processed image are obtained based on the horizontal and vertical angles of view.

[0098] Preferably, a plurality of viewing angles are selected from a plurality of viewing angles to shoot the artificial physical entity, and the shooting parameters during shooting include: the three-dimensional vector V of the viewpoint x 、3D vector of sight V e 、Three-dimensional vector V on the image plane u .

[0099] Preferably, processing the entity image to obtain a processed image includes: cutting out surrounding objects of the artificial physical entity in the entity image to retain only the artificial physical entity.

[0100] Preferably, the horizontal field of view angle and the vertical field of view angle of the processed image are calculated according to the shooting parameters when shooting the artificial physical entity and the processing parameters in the process of processing the entity image, including:

[0101] Calculate the vertical field angle v, the aspect ratio k, the number of horizontal pixels n0, and the number of vertical pixels n1 of the physical entity image; the vertical field angle v and the aspect ratio k are calculated by formula (1):

[0102]

[0103] In formula (1), v c Indicates the vertical field of view of the camera device; k c Indicates the aspect ratio of the picture; n c0 Indicates the horizontal resolution; n c1 Indicates vertical resolution;

[0104] Among them, the horizontal pixel number n0 and the vertical pixel number n1 are obtained through the physical entity image.

[0105] Preferably, acquiring pixel coordinates of entity feature points in the processed image based on the horizontal field of view angle and the vertical field of view angle includes: measuring the pixel coordinates of the entity feature points on the physical entity image.

[0106] According to the spatial topological structure characteristics of the main body of the physical entity, the embodiment of the present invention selects several perspectives from the front view, rear view, left and right side view, top view, and bottom view, takes a photo of the physical entity in telephoto mode, cuts off the part that does not contain the photographed object from the acquired photo, and calculates the horizontal and vertical field of view angles of the photo according to the cropping parameters and the camera field of view. The pixel coordinates N of the entity feature point are measured on the acquired photo. The pixel coordinates N are a two-dimensional vector, which can be expressed in the form of (N0, N1), where N0 represents the number of pixels from the pixel to the left edge of the image, and N1 represents the number of pixels from the pixel to the upper edge of the image.

[0107] The specific implementation of measuring the pixel coordinates of feature points is as follows: establish a rectangular coordinate system xyz, the origin of the coordinate system is near the geometric center of the bounding box of the artificial physical entity, and the axial selection of the coordinate system should satisfy the requirement that as many large-area planes of the artificial physical entity as possible are parallel or nearly parallel to the xy plane (i.e., the top-view reference plane) or the yz plane (the front-view reference plane) or the xz plane (the left-view reference plane).

[0108] Select feature points based on the symmetry of the artificial physical entity, and then select several groups of items from the six views of front view, rear view, left view, right view, top view, and bottom view to take photos of the artificial physical entity based on the visibility of the selected feature points. If the artificial physical entity is bilaterally symmetrical, select one of the left and right views; if it is front-to-back symmetrical, select one of the front and rear views; if it is top-to-bottom symmetrical, select one of the top and bottom views.

[0109] The shooting conditions for taking photos of artificial physical entities in each view are composed of three three-dimensional vectors V x 、V e 、V u Define the viewpoint, line of sight, and direction on the image plane respectively. The shooting conditions of different views are as follows:

[0110] Front view shooting conditions: viewpoint (d, 0, 0), line of sight: (-1, 0, 0), image plane direction vector (0, 0, 1), where the factor d is selected so that the object is completely within the frame and occupies as large an area as possible, and is at least 10 times greater than the depth of field of the feature point. The selection of d in the shooting conditions of the following other views also follows this principle, and the d values ​​of different views can be the same or different.

[0111] Back view shooting conditions: viewpoint (-d, 0, 0), sight vector: (1, 0, 0), direction vector on the image plane (0, 0, 1).

[0112] Left view shooting conditions: viewpoint (0,d,0), line of sight: (0,-1,0), direction vector on the image plane (0,0,1).

[0113] Right view shooting conditions: viewpoint (0,-d,0), line of sight: (0,1,0), direction vector on the image plane (0,0,1).

[0114] Top view shooting conditions: viewpoint (0,0,d), sight vector: (0,0,-1), direction vector on the image plane (0,1,0).

[0115] Bottom view shooting conditions: viewpoint (0,0,-d), sight vector: (0,0,1), direction vector on the image plane (0,-1,0).

[0116] The captured image is cropped to remove the part that does not contain the captured object. The cropped image has the following four parameters related to coordinate calculation: vertical field angle v, frame aspect ratio k, horizontal pixel number n0, vertical pixel number n1, where parameters v and k are calculated by the following formula:

[0117]

[0118] In the formula, v c , k c 、n c0 、n c1 They are the vertical field of view, aspect ratio, horizontal resolution, and vertical resolution of the camera device.

[0119] Measure the pixel coordinates N of the entity feature points on the cropped photo.

[0120] Step 22: Calculate the three-dimensional coordinates of the entity feature points according to the pixel coordinates.

[0121] Specifically, the above step 22 includes:

[0122] Based on the horizontal field angle and the vertical field angle of the processed image obtained in step 21, the pixel coordinates of the entity feature point obtained in step 21 are converted into normalized image space coordinates; wherein the normalized image space coordinates are coordinate values ​​after the pixel coordinate scale is transformed, so that the vertical coordinate of the upper edge of the processed image is 1.0, and the vertical coordinate of the lower edge of the processed image is -1.0;

[0123] The normalized image space coordinates of the same entity feature point in the two processed images are solved to obtain the three-dimensional coordinates of the entity feature point.

[0124] In step 22 of the embodiment of the present invention:

[0125] Define the normalized coordinate X in image space, the range of its longitudinal coordinate X1 is [-1.0, 1.0], and the range of its transverse coordinate X0 is [-k, k], then: n0 represents horizontal pixels, n1 represents vertical pixels, N0 represents the number of pixels from pixel coordinate N to the left edge of the processed image, and N1 represents the number of pixels from pixel coordinate N to the top edge of the processed image;

[0126] For any view taken from several perspectives, a temporary coordinate system x'y'z' is established by replacing the axis direction, so that the three-dimensional vector V of the viewpoint in the shooting parameters during shooting x Pointing to the negative direction of the x' axis, the three-dimensional vector V on the image plane u Pointing to the positive direction of the z' axis, using d as the initial estimated value of the distance of the entity feature point, the following formula (2) and formula (3) are used to calculate the initial value of the y'z' coordinate of the entity feature point;

[0127]

[0128]

[0129] In formula (2) and formula (3), X1 represents the longitudinal coordinate of the normalized coordinate X in the image space, and X0 represents the transverse coordinate of the normalized coordinate X in the image space;

[0130] Obtain two components of the three-dimensional vector of the feature point coordinates according to the initial value of the y'z' coordinate of the entity feature point, find another view in which the entity feature point is visible, and calculate another component of the three-dimensional vector of the feature point coordinates based on the other view using formula (2) or formula (3);

[0131] The distance of the entity feature point is corrected based on the initial value of the y'z' coordinate of the entity feature point, thereby correcting the two coordinate components associated with the view corresponding to the initial value of the coordinate. Specifically, the coordinate components are corrected and calculated by formula (4);

[0132]

[0133] In formula (4), d represents the initial estimated value of the feature point distance;

[0134] After all coordinate points are corrected, the longest side of their bounding boxes is calculated, and all coordinate values ​​of all coordinate points are scaled so that the longest side of the bounding box is 1.0. The scale transformation factor is recorded as the scale feature of the 3D realistic simplified model.

[0135] The embodiment of the present invention converts the pixel coordinate data of the feature point into the normalized image space coordinates based on the horizontal and vertical field of view of the photo, and obtains its three-dimensional coordinates by combining the normalized image space coordinates of the same feature point on two photos. The normalized image space coordinates refer to the coordinate values ​​after the pixel coordinates are scaled, so that the vertical coordinate of the upper edge of the image is 1.0 and the vertical coordinate of the lower edge is -1.0.

[0136] The specific implementation is as follows: define the normalized coordinate X in image space: the vertical coordinate X1 range is [-1.0, 1.0], and the horizontal coordinate X0 range is [-k, k], then:

[0137]

[0138] To facilitate calculation, for any view, a temporary coordinate system x'y'z' is established by replacing the axis direction, so that the sight vector points to the negative direction of the x' axis and the image plane direction points to the positive direction of the z' axis. With d as the initial estimate of the distance of the feature point, the initial value of the y'z' coordinate of the feature point is calculated using the following formula:

[0139]

[0140] In this way, the two components of the three-dimensional vector of the feature point coordinates are obtained, and another view where the feature point is visible is found. On this view, the other component is calculated using the above formula.

[0141] After all feature point coordinates are obtained, for any view, the feature point distance can be corrected based on the coordinates obtained in the first round of calculation, thereby correcting the two coordinate components associated with the view. The correction calculation method is as follows:

[0142]

[0143] After all coordinate points are corrected, the longest side of the bounding box of all feature points is calculated, and all coordinate values ​​are scaled so that the longest side of the bounding box is 1.0. At the same time, the scale transformation factor is recorded as the scale feature of the 3D model.

[0144] Step 3: Use three-dimensional coordinates to correct the feature points of the initial embryo shape.

[0145] Specifically, the above step three includes: assigning the three-dimensional coordinates calculated in step two to the corresponding vertices of the initial embryo in step one to complete the construction of a white mold corresponding to the artificial physical entity; wherein the white mold is a three-dimensional model without surface texture.

[0146] Step 4: After mapping the initial embryonic form, a three-dimensional realistic simplified model is constructed, wherein the three-dimensional realistic simplified model ignores the detailed structural features of the artificial physical entity.

[0147] Specifically, the above step 4 includes:

[0148] The processed image corresponding to the initial embryo shape after cropping is used as the texture map resource of the white mold, and the vertex map parameters are calculated according to the projective geometry, and a three-dimensional realistic simplified model is constructed based on the texture map resources and the vertex map parameters; wherein the three-dimensional realistic simplified model is a three-dimensional model that can be displayed in a three-dimensional scene and is consistent with the visual sense of the corresponding artificial physical entity but ignores the detailed structural features;

[0149] The degree of refinement of artificial modeling is determined according to the application requirements of the 3D realistic streamlined model. The detailed structural features of the artificial physical entity are ignored based on the degree of refinement, thereby greatly reducing the data volume of the 3D realistic streamlined model.

[0150] The above step 4 of the embodiment of the present invention is specifically implemented as follows: a blank bitmap is created, the size of the blank bitmap should be able to accommodate all the cropped images obtained in step S21, and the images obtained in step S21 are tiled on the blank bitmap according to a certain layout as the texture map resource T of the white model. The rectangular area occupied by each view is defined by the texture coordinates of the lower left and upper right points of the rectangular area on the texture map resource T (the texture coordinate is a two-dimensional vector used to represent the position of a point in the rectangle, and its meaning is as follows Figure 3 shown.)

[0151] The six views of front view, back view, left view, right view, top view and bottom view are numbered 1 to 6 in sequence. Views 1 to 6 occupy rectangular areas R1 to R6 in the texture map resource T respectively. The texture coordinates Si lb , Si rt The texture coordinates of the lower left and upper right points of the area Ri occupied by view i are shown in Figure 1. Traverse all the vertices of the white model and find the maximum and minimum values ​​of the coordinate x. max ,x min , the maximum and minimum values ​​of coordinate y max ,y min , the maximum and minimum values ​​of coordinate z max ,z min . For each vertex, calculate the following value:

[0152]

[0153] For each triangle that makes up the white model, its surface texture is mapped to a triangular area R of the texture map resource T through the texture coordinates of the three vertices. The texture coordinates of the vertices are calculated as follows:

[0154] Find the one with the largest absolute value among the three components of the triangle normal vector. If it is component 0, select the front view (positive value) or the back view (negative value) as the texture mapping; if it is component 1, select the left view (positive value) or the right view (negative value) as the texture mapping; if it is component 2, select the top view (positive value) or the bottom view (negative value) as the texture mapping. If you take symmetry into consideration when shooting and do not get a photo of the selected perspective, press Fig.17 The adjustment table shown adjusts the selected view number. According to the selected view number, select one of the following 6 calculation formulas to calculate the texture coordinate s of R in the selected view area. View No. 1, i.e. front view: s0 = k y ,s1=k z ; View No. 2, rear view: s0 = 1.0-k y ,s1=k z ; View No. 3, right view: s0 = k x ,s1=k z ; View No. 4, left view: s0 = 1.0-k x ,s1=k z ; View No. 5, top view: s0 = k x ,s1=k y ; View 6, bottom view: s0 = k x ,s1=1.0-k z .

[0155] According to the selected view number i, the texture coordinate s of R in the texture map resource T is calculated according to the following formula: t

[0156] s t0 =Si lb0 +s0(Si rt0 -Si lb0 ),s t1 =Si lb1 +s1(Si rt1 -Si lb1 )

[0157] At this point, for all triangles of the model, the texture map resource T is used to map the texture coordinates s corresponding to the vertices. t Vertices describe the model surface mapping parameters, thus completing the construction of a realistic and streamlined model.

[0158] See also Figures 4 to 16 , a three-dimensional modeling method based on photogrammetry according to an embodiment of the present invention is applied to the train carriage shown in the accompanying drawings. Figure 3 As shown, the embodiment of the present invention uses a mobile phone to shoot a train car model. Mobile phone camera parameters k c =1.5649, nc0 =4000, n c1 =2556.

[0159] Since the train carriage model is symmetrical in both left and right and front and back, four viewing angles, namely "front view", "left view", "top view" and "bottom view", were selected for shooting.

[0160] In step 1, the subtle bumps and details of the surface are ignored and the Figure 3 The initial prototype shown in the figure is composed of a rectangular block as the main structure. The top surface of the rectangular block is cut into a cylindrical surface, and four stepped cylinders are symmetrically arranged on the left and right end surfaces; four protruding guard plates are arranged on the bottom surface, and four wheels are arranged behind the guard plates in a disc-shaped geometry. Considering the symmetry, the scale of the structural elements of the initial prototype can be expressed as Figure 5 A total of 16 feature points P1 to P16 are defined.

[0161] In step 2, four viewing angles, namely, “front view”, “left view”, “top view” and “bottom view”, were selected for shooting, and d was 48 cm for each. The images obtained are as follows: Figure 6 As shown, the cropped image is Figure 7 As shown. Fig.14 , 15 As shown in FIG. 1 , for the feature points in the train compartment of this embodiment, all the feature points can be covered by referring to the pair of photos of “front view” and “left view”. The measured pixel coordinates are as follows: Fig.18 shown.

[0162] In step 2, for the "front view" perspective, there is no need to replace the axis direction; for the left view perspective, the axis direction replacement is x'=y, y'=-x, z'=z. Perform the first round of calculation on the feature point coordinates, where the x and z coordinates are obtained from the front view and the y coordinate is obtained from the left view. Then use the x coordinate to recalculate the y coordinate based on formula (3), and then use the corrected y coordinate to recalculate the x and z coordinates based on formula (3). Finally, calculate the scale factor and normalized coordinates. The result is as follows: Fig.19 shown.

[0163] In step three, the basic geometric parameters of the initial embryo structure elements (these parameters include the length, width and height of the cuboid, the center and radius of the cylinder, etc.) are calculated from the feature point coordinate data and edited and entered to complete the modification of the initial embryo, so that the initial embryo is adjusted to a three-dimensional model consistent with the size of the physical entity.

[0164] In step 4, follow Figure 2The layout stitches the cropped front view, left view, top view, and bottom view photos into a single bitmap image. The white model of the artificial physical entity generated by the 3D modeling software is stored as an STL file. The STL file is read to obtain all the triangle parameters that constitute the model surface. Each triangle includes three vertex coordinates and a normal vector. The following processing is performed based on each triangle:

[0165] The initial value of the counter number m is 0. For the normal vector (n0, n1, n2) of the triangle to be processed, create a vector v: (|n0|, |n1|, |n2|), create a scalar a=v0, if v1 is greater than a, update a=v1, and modify m to 1; continue to judge v2, if v2 is greater than a, modify m to 2. The counter symbol b=Sgn(n m ).according to Fig. 20 As shown, determine the selected view number and calculate the texture coordinate s corresponding to the vertex t : Fig.16 This is the rendering of the completed 3D model.

[0166] The embodiment of the present invention generates a three-dimensional model with simplified data for artificial physical entities with regular surfaces such as equipment, vehicles, and buildings through a concise three-dimensional modeling method.

[0167] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A three-dimensional modeling method based on photogrammetry, characterized in that: The following steps are involved: Step 1: Generate an initial prototype of an artificial physical entity by artificial modeling, wherein the artificial physical entity is various entities created by humans, including equipment, vehicles or buildings; Step 2: obtaining the three-dimensional coordinates of the feature points in the artificial physical entity; Step 3: Correcting the feature points of the initial embryo using the three-dimensional coordinates; Step 4: Mapping the initial embryonic form to construct a three-dimensional realistic simplified model, wherein the three-dimensional realistic simplified model ignores detailed structural features of the artificial physical entity.

2. A three-dimensional modeling method based on photogrammetry according to claim 1, characterized in that: The step one comprises: Artificial modeling is performed by combining operations of basic geometric bodies to generate an initial embryonic form for expressing the main spatial topological structure of the artificial physical entity; wherein the basic geometric bodies are spatial configurations accurately defined by a small number of parameters, including cuboids, cylinders or spheres; and the combination operations include merging or cutting the basic geometric bodies; During the artificial modeling process, there is a definite corresponding relationship between each vertex of the initial embryo and a specific point of the artificial physical entity.

3. A three-dimensional modeling method based on photogrammetry according to claim 2, characterized in that: The step one also includes: In the process of manual modeling, minute details on the appearance of the artificial physical entity are ignored, wherein the minute details include: tiny protrusions or undulations on the surface of the artificial physical entity.

4. The three-dimensional modeling method based on photogrammetry according to claim 1, characterized in that: The second step comprises: Step 21: Acquire entity feature points of the artificial physical entity and measure pixel coordinates of the entity feature points; Step 22: Calculate the three-dimensional coordinates of the entity feature points according to the pixel coordinates.

5. The three-dimensional modeling method based on photogrammetry according to claim 4, characterized in that: The step 21 comprises: According to the main spatial topological structure of the artificial physical entity, a plurality of viewing angles are selected from a plurality of viewing angles to photograph the artificial physical entity to obtain an image of the entity, wherein the plurality of viewing angles include a front view, a rear view, a left view, a right view, a top view, and a bottom view; Processing the entity image to obtain a processed image; The horizontal field of view angle and the vertical field of view angle of the processed image are calculated according to the shooting parameters when shooting the artificial physical entity and the processing parameters in the process of processing the entity image, and the pixel coordinates of the entity feature points in the processed image are obtained based on the horizontal field of view angle and the vertical field of view angle.

6. A three-dimensional modeling method based on photogrammetry according to claim 5, characterized in that: The artificial physical entity is photographed by selecting several viewing angles from a plurality of viewing angles, and the photographing parameters during the photographing include: the three-dimensional vector V of the viewpoint x 、3D vector of sight V e 、Three-dimensional vector V on the image plane u ; The processing of the entity image to obtain a processed image includes: cutting out surrounding objects of the artificial physical entity in the entity image to retain only the artificial physical entity; The calculating the horizontal field of view angle and the vertical field of view angle of the processed image according to the shooting parameters when shooting the artificial physical entity and the processing parameters in the process of processing the entity image includes: The vertical field angle v, the aspect ratio k, the number of horizontal pixels n0, and the number of vertical pixels n1 of the physical entity image are calculated; wherein the vertical field angle v and the aspect ratio k are calculated by formula (1): In formula (1), v c Indicates the vertical field of view of the camera device; k c Indicates the aspect ratio of the picture; n c0 Indicates the horizontal resolution; n c1 Indicates vertical resolution; Wherein, the number of horizontal pixels n0 and the number of vertical pixels n1 are obtained through the physical entity image; The acquiring pixel coordinates of the entity feature points in the processed image based on the horizontal field of view angle and the vertical field of view angle includes: measuring the pixel coordinates of the entity feature points on the physical entity image.

7. The three-dimensional modeling method based on photogrammetry according to claim 4, characterized in that: The step 22 comprises: Based on the horizontal field of view angle and the vertical field of view angle of the processed image obtained in step 21, the pixel coordinates of the entity feature point obtained in step 21 are converted into normalized coordinates in image space; wherein the normalized coordinates in image space are coordinate values ​​after the scale transformation of the pixel coordinates, so that the vertical coordinate of the upper edge of the processed image is 1.0, and the vertical coordinate of the lower edge of the processed image is -1.0; The image space normalized coordinates of the same entity feature point in the two processed images are jointly calculated to obtain the three-dimensional coordinates of the entity feature point.

8. The three-dimensional modeling method based on photogrammetry according to claim 7, characterized in that: In step 22: Define the normalized coordinate X in the image space, the range of its longitudinal coordinate X1 is [-1.0, 1.0], and the range of its transverse coordinate X0 is [-k, k], then: n0 represents horizontal pixels, n1 represents vertical pixels, N0 represents the number of pixels from pixel coordinate N to the left edge of the processed image, and N1 represents the number of pixels from pixel coordinate N to the top edge of the processed image; For any view taken from several perspectives, a temporary coordinate system x'y'z' is established by replacing the axis direction, so that the three-dimensional vector V of the viewpoint in the shooting parameters during shooting x Pointing to the negative direction of the x' axis, the three-dimensional vector V on the image plane u Pointing to the positive direction of the z' axis, using d as the initial estimated value of the distance of the entity feature point, the following formula (2) and formula (3) are used to calculate the initial value of the y'z' coordinate of the entity feature point; In formula (2) and formula (3), X1 represents the longitudinal coordinate of the normalized coordinate X in the image space, and X0 represents the transverse coordinate of the normalized coordinate X in the image space; Obtain two components of the three-dimensional vector of the feature point coordinates according to the initial value of the y'z' coordinate of the entity feature point, find another view in which the entity feature point is visible, and calculate another component of the three-dimensional vector of the feature point coordinates based on the other view using formula (2) or formula (3); The distance of the entity feature point is corrected based on the initial value of the y'z' coordinate of the entity feature point, thereby correcting the two coordinate components associated with the view corresponding to the initial value of the coordinate. Specifically, the coordinate components are corrected and calculated by formula (4); In formula (4), d represents the initial estimated value of the feature point distance; After all coordinate points are corrected, the longest side of their bounding boxes is calculated, and all coordinate values ​​of all coordinate points are scaled so that the longest side of the bounding box is 1.

0. The scale transformation factor is recorded as the scale feature of the 3D realistic simplified model.

9. The three-dimensional modeling method based on photogrammetry according to claim 1, characterized in that: The step three comprises: The three-dimensional coordinates calculated in step 2 are assigned to the corresponding vertices of the initial embryo shape in step 1 to complete the construction of a white mold corresponding to the artificial physical entity; wherein the white mold is a three-dimensional model without surface texture.

10. The three-dimensional modeling method based on photogrammetry according to claim 1, characterized in that: The fourth step comprises: The processed image corresponding to the initial embryo shape after cutting is used as a texture map resource of the white mold, and vertex map parameters are calculated according to the projective geometry, and a three-dimensional realistic simplified model is constructed based on the texture map resources and the vertex map parameters; wherein the three-dimensional realistic simplified model is a three-dimensional model that can be displayed in a three-dimensional scene and is consistent with the visual sense of the corresponding artificial physical entity but ignores the detailed structural features; The degree of refinement of the artificial modeling is determined according to the application requirements of the three-dimensional simulated simplified model, and the detailed structural features of the artificial physical entity are ignored based on the degree of refinement, thereby greatly reducing the data volume of the three-dimensional simulated simplified model.