Image-based three-dimensional modeling method, device, electronic equipment and storage medium

Abstract

The application provides a kind of based on image three-dimensional modeling method, device, electronic equipment and storage medium, based on image three-dimensional modeling method, comprising: obtaining the multiple target images of the object to be modeled, and the target image is the image of different regions on the object to be modeled;Based on any one target image, determine a known plane and known plane equation;Based on multiple target images and known plane, the target plane equation of predetermined target plane is calculated, when the corresponding target plane equation is known, the corresponding predetermined target plane is known plane, and the corresponding target plane equation is known plane equation;Based on all known planes and all known plane equations, a three-dimensional space model of the object to be modeled is constructed.The application can quickly three-dimensional model the object to be modeled based on image three-dimensional modeling method, device, electronic equipment and storage medium, so as to improve the efficiency of three-dimensional modeling, and greatly reduce the cost required.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional modeling, in particular to a three-dimensional modeling method and device based on images, an electronic device and a storage medium. BACKGROUND

[0002] At present, three-dimensional modeling has been widely applied in various fields. A common three-dimensional modeling method is a three-dimensional modeling method based on a laser range finder. This three-dimensional modeling method needs to measure the spatial coordinates of at least three points on each plane of a target object to determine a plane equation, so that the precision of the constructed three-dimensional model is high. However, in some application fields, the precision requirement of three-dimensional modeling of the target object is not high, but the restoration requirement of the appearance of the target object is high. For example, buildings. In these application fields, the modeling time of this three-dimensional modeling method is long, which affects the efficiency of three-dimensional modeling and increases the cost. SUMMARY

[0003] The embodiments of the present application aim to provide a three-dimensional modeling method and device based on images, an electronic device and a storage medium, which can quickly model a target object, thereby improving the efficiency of three-dimensional modeling and greatly reducing the cost.

[0004] In a first aspect, the embodiments of the present application provide a three-dimensional modeling method based on images, comprising:

[0005] Obtaining a plurality of target images of a target object, the target images being images of different regions of the target object;

[0006] Determining a known plane and a known plane equation based on any one of the target images;

[0007] Calculating a target plane equation of a predetermined target plane based on the plurality of target images and the known plane, the corresponding predetermined target plane being a known plane when the corresponding target plane equation is known, and the corresponding target plane equation being a known plane equation;

[0008] Constructing a three-dimensional space model of the target object based on all the known planes and all the known plane equations.

[0009] In the implementation process, the image-based three-dimensional modeling method provided in the embodiments of the present application determines a known plane and a known plane equation through any one of the obtained target images, and calculates a target plane equation of a predetermined target plane based on the obtained target images and the known plane, and then constructs a three-dimensional space model of the object to be modeled based on all the known planes and all the known plane equations. Compared with the prior art, this method can quickly calculate the plane equation of the appearance plane of the object to be modeled, so that the three-dimensional modeling of the object to be modeled can be quickly performed, and the appearance of the object to be modeled can be restored well, thereby improving the efficiency of three-dimensional modeling and greatly reducing the cost.

[0010] Further, the obtaining of the plurality of target images of the object to be modeled comprises:

[0011] obtaining a predetermined horizontal field of view and a predetermined vertical field of view of the camera device;

[0012] determining a horizontal rotation interval and a vertical rotation interval of the rotating device according to the predetermined horizontal field of view and the predetermined vertical field of view;

[0013] determining a starting rotation angle and an ending rotation angle of the rotating device;

[0014] controlling the movement of the rotating device according to the starting rotation angle, the horizontal rotation interval, the vertical rotation interval and the ending rotation angle;

[0015] obtaining a plurality of target images of the object to be modeled by the camera device.

[0016] In the implementation process, the method can better obtain a plurality of target images of the object to be modeled by the camera device through the starting rotation angle, the horizontal rotation interval, the vertical rotation interval and the ending rotation angle to control the movement of the rotating device, and thus the effect of three-dimensional modeling of the object to be modeled by the method can be better guaranteed.

[0017] Further, the calculation of the target plane equation of the predetermined target plane based on the plurality of target images and the known plane comprises:

[0018] determining an auxiliary line corresponding to the predetermined target plane according to the geometric relationship between the known plane and the predetermined target plane on the plurality of target images;

[0019] calculating the target plane equation of the predetermined target plane according to the pixel coordinates of the auxiliary line corresponding to the predetermined target plane.

[0020] In the implementation process, the method first determines the auxiliary line corresponding to the predetermined target plane on multiple target images, and then calculates the target plane equation of the predetermined target plane according to the pixel coordinates of the auxiliary line corresponding to the predetermined target plane, so that the three-dimensional modeling of the object to be modeled can be more quickly performed.

[0021] Further, the determining the auxiliary line corresponding to the predetermined target plane on multiple target images according to the geometric relationship between the known plane and the predetermined target plane comprises:

[0022] On multiple target images, a predetermined target plane perpendicular to the known plane is determined as a first target plane, and an intersection line between the known plane and the first target plane is determined as the corresponding auxiliary line.

[0023] On multiple target images, a predetermined target plane parallel to the known plane is determined as a second target plane, and a line perpendicular to the known plane and the second target plane and intersecting the known plane and the second target plane is determined as the corresponding auxiliary line.

[0024] In the implementation process, the method can quickly determine the auxiliary line corresponding to the predetermined target plane, and then facilitate the calculation of the target plane equation of the predetermined target plane.

[0025] Further, the calculating the target plane equation of the predetermined target plane according to the pixel coordinates of the auxiliary line corresponding to the predetermined target plane comprises:

[0026] When the known plane is perpendicular to and intersects the first target plane, any two first pixel points on the corresponding auxiliary line are obtained, and the corresponding pixel coordinates are obtained.

[0027] According to the camera imaging principle and the known plane equation, three-dimensional space coordinates of the two first pixel points in the camera coordinate system are obtained.

[0028] The plane equation of the plane containing the two space points and perpendicular to the known plane is calculated as the corresponding target plane equation.

[0029] When the known plane is parallel to the second target plane, the pixel coordinates of the two end points on the corresponding auxiliary line are obtained.

[0030] According to the camera imaging principle and the known plane equation, three-dimensional space coordinates of the end points on the known plane in the camera coordinate system are obtained, denoted as the foot coordinates.

[0031] According to the camera imaging principle, the spatial point coordinate of the pixel coordinate of the other end point on the imaging plane is calculated, denoted as the far point coordinate;

[0032] The intersection point coordinate of the line passing through the foot point and perpendicular to the known plane and the line connecting the camera imaging center point and the far point is calculated;

[0033] The plane equation of the plane passing through the intersection point and parallel to the known plane is calculated as the corresponding target plane equation.

[0034] In the above implementation process, the method can more quickly calculate the target plane equation of the predetermined target plane.

[0035] Further, the three-dimensional space model of the object to be modeled is constructed based on all the known planes and all the known plane equations, including:

[0036] According to all the known planes and all the known plane equations, the intersection line equation and the intersection point coordinate of all the known planes are calculated;

[0037] According to the intersection line equation and the intersection point coordinate of all the known planes, the three-dimensional space model of the object to be modeled is constructed.

[0038] In the above implementation process, the method can more quickly perform three-dimensional modeling on the object to be modeled by calculating the intersection line equation and the intersection point coordinate of all the known planes based on all the known planes and all the known plane equations, and constructing the three-dimensional space model of the object to be modeled, and can better guarantee a better restoration degree of the appearance of the object to be modeled.

[0039] Further, after the three-dimensional space model of the object to be modeled is constructed based on all the known planes and all the known plane equations, the method further includes:

[0040] Based on the pixel coordinate of any pixel point of any one of the target images, the three-dimensional space coordinates of the spatial point corresponding to the pixel point on the three-dimensional space model are obtained;

[0041] Based on the three-dimensional space coordinates of the plurality of spatial points, the geometric information between the plurality of spatial points is calculated.

[0042] In the above implementation process, the method can calculate the geometric information between the plurality of spatial points, so as to facilitate better understanding and analysis of the three-dimensional space model.

[0043] In a second aspect, the embodiments of the present application provide a three-dimensional modeling device based on images, including:

[0044] An acquisition module is configured to acquire a plurality of target images of the object to be modeled, the target images being images of different regions on the object to be modeled;

[0045] A determination module is configured to determine a known plane and a known plane equation based on any one of the target images;

[0046] A calculation module is configured to calculate a target plane equation of a predetermined target plane based on the plurality of target images and the known plane, the corresponding predetermined target plane being a known plane when the corresponding target plane equation is known, and the corresponding target plane equation being a known plane equation;

[0047] A construction module is configured to construct a three-dimensional space model of the object to be modeled based on all the known planes and all the known plane equations.

[0048] In the implementation process, the image-based three-dimensional modeling device provided by the embodiments of the present application determines a known plane and a known plane equation based on any one of the acquired target images, calculates a target plane equation of a predetermined target plane based on the plurality of acquired target images and the known plane, and then constructs a three-dimensional space model of the object to be modeled based on all the known planes and all the known plane equations. Compared with the prior art, this method can quickly calculate the plane equation of the appearance plane of the object to be modeled, thereby quickly performing three-dimensional modeling on the object to be modeled, ensuring a good restoration degree of the appearance of the object to be modeled, improving the efficiency of three-dimensional modeling, and greatly reducing the cost.

[0049] In a third aspect, the embodiments of the present application provide an electronic device, including a memory and a processor, the memory is configured to store a computer program, and the processor is configured to run the computer program to make the electronic device execute the image-based three-dimensional modeling method described above.

[0050] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image-based three-dimensional modeling method described above. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0052] Figure 1A flowchart of the method for three-dimensional modeling based on images provided in Embodiment One of the present application is shown in FIG. 1.

[0053] Figure 2 A flowchart of step S110 provided in Embodiment One of the present application is shown in FIG. 2.

[0054] Figure 3 A flowchart of step S130 provided in Embodiment One of the present application is shown in FIG. 3.

[0055] Figure 4 A schematic diagram of determining auxiliary lines when two planes are perpendicular and intersected provided in Embodiment One of the present application is shown in FIG. 4.

[0056] Figure 5 A schematic diagram of determining auxiliary lines when two planes are parallel provided in Embodiment One of the present application is shown in FIG. 5.

[0057] Figure 6 A schematic diagram of calculating the target plane equation of a target plane perpendicular to a known plane provided in Embodiment One of the present application is shown in FIG. 6.

[0058] Figure 7 A schematic diagram of calculating the target plane equation of a target plane parallel to a known plane provided in Embodiment One of the present application is shown in FIG. 7.

[0059] Figure 8 A schematic diagram of multiple layers of planes existing in one target image provided in Embodiment One of the present application is shown in FIG. 8.

[0060] Figure 9 A schematic diagram of calculating the target plane equation of a target plane across images provided in Embodiment One of the present application is shown in FIG. 9.

[0061] Figure 10 A flowchart of step S140 provided in Embodiment One of the present application is shown in FIG. 10.

[0062] Figure 11 A structural block diagram of the three-dimensional modeling device based on images provided in Embodiment Two of the present application is shown in FIG. 11. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0064] It should be noted that similar reference numerals and letters refer to like items in the accompanying drawings, and therefore, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", etc. are merely used to distinguish description, and cannot be understood as indicating or implying relative importance.

[0065] A common three-dimensional modeling method is a three-dimensional modeling method based on a laser range finder. This three-dimensional modeling method needs to measure the spatial coordinates of at least three points on each plane of the target object to determine the plane equation, so that the accuracy of the constructed three-dimensional model is relatively high. However, in some application fields, the accuracy requirement of the three-dimensional modeling of the target object is not high, but the restoration requirement of the appearance of the target object is relatively high, for example, a building. In these application fields, the modeling time of this three-dimensional modeling method is relatively long, which affects the efficiency of the three-dimensional modeling, and leads to an increase in the cost to be spent.

[0066] To solve the problems in the prior art, the application provides a three-dimensional modeling method based on images, a device, an electronic device and a storage medium, which can quickly model a target object, thereby improving the efficiency of three-dimensional modeling and greatly reducing the cost to be spent.

[0067] Embodiment one

[0068] Reference Figure 1 , Figure 1 The application provides a flowchart of a three-dimensional modeling method based on images. The three-dimensional modeling method based on images described in the application can be applied to a server.

[0069] The three-dimensional modeling method based on images of the application includes the following steps:

[0070] In step S110, a plurality of target images of a target object are obtained. The target images are images of different regions of the target object.

[0071] In this embodiment, the target object can be a building, or other objects with a relatively high restoration requirement of the appearance and a relatively low accuracy requirement of the three-dimensional modeling.

[0072] The target images of the target object are images of different regions of the appearance of the target object.

[0073] Optionally, the target images of the target object can include at least two planes of the target object; or the plurality of target images of the target object can be images covering all regions of the target object.

[0074] Optionally, the plurality of target images of the target object can be obtained by a video camera of a measuring device. The measuring device can be a video measuring device including a double-axis holder, a laser range finder and a video camera installed on the double-axis holder and arranged in parallel with each other.

[0075] In step S120, a known plane and a known plane equation are determined based on any one target image.

[0076] In the embodiment, the known plane can be a certain plane on the appearance of the object to be modeled.

[0077] Optionally, the spatial coordinates of any three non-collinear spatial points on the known plane can be obtained by the measuring device; after obtaining the spatial coordinates of any three non-collinear spatial points on the known plane, the plane equation of the known plane in the camera coordinate system can be calculated by using the spatial coordinates of the three spatial points, the rotation angle of the two-axis holder of the measuring device when the target image is taken, and the camera focal length parameter of the video camera.

[0078] In step S130, the target plane equation of the predetermined target plane is calculated based on the plurality of target images and the known plane.

[0079] In the embodiment, when the corresponding target plane equation is calculated so that the corresponding target plane equation is known, the corresponding predetermined target plane becomes the known plane, and the corresponding target plane equation becomes the known plane equation.

[0080] The predetermined target plane is a certain plane or a plurality of planes on the appearance of the object to be modeled, and generally, one target plane corresponds to one target plane equation; when the target plane equation of the target plane cannot be calculated by the known plane, the target plane equation of the target plane can be calculated by sampling the target plane by the measuring device.

[0081] It can be understood that when a certain predetermined target plane becomes the known plane and its corresponding target plane equation becomes the known plane equation, the predetermined target plane that becomes the known plane can be used to calculate the target plane equation of other predetermined target planes.

[0082] It can be understood that each target image corresponds to a camera coordinate system, and the transformation relationship between the plurality of camera coordinate systems corresponding to the plurality of target images can be calculated by the rotation angle of the two-axis holder of the measuring device and the camera focal length parameter of the video camera when the plurality of target images are taken, which belongs to the disclosed technology in the field of photogrammetry, and will not be described here. Therefore, as long as the known plane equation of the known plane in the camera coordinate system of any target image is obtained, the known plane equation of the known plane in the camera coordinate system of other target images can be calculated.

[0083] The target plane equation of the predetermined target plane can be calculated relatively quickly by the way of calculating the target plane equation of the predetermined target plane by the known plane.

[0084] In step S140, the three-dimensional space model of the object to be modeled is constructed based on all the known planes and all the known plane equations.

[0085] Compared with the prior art, the image-based three-dimensional modeling method provided in the embodiments of the present application can quickly calculate the plane equation of the appearance plane of the object to be modeled, and thus can quickly perform three-dimensional modeling on the object to be modeled, and can guarantee a good restoration degree of the appearance of the object to be modeled, thereby improving the efficiency of three-dimensional modeling and greatly reducing the cost.

[0086] Referring to Figure 2 , Figure 2 The flowchart of step S110 provided in the embodiments of the present application is shown.

[0087] In some embodiments of the present application, the step S110 of acquiring the plurality of target images of the object to be modeled can include the following steps:

[0088] The step S111 of acquiring the predetermined horizontal field of view and the predetermined vertical field of view of the camera device;

[0089] The step S112 of determining the horizontal rotation interval and the vertical rotation interval of the rotating device according to the predetermined horizontal field of view and the predetermined vertical field of view;

[0090] The step S113 of determining the starting rotation angle and the ending rotation angle of the rotating device;

[0091] The step S114 of controlling the movement of the rotating device according to the starting rotation angle, the horizontal rotation interval, the vertical rotation interval and the ending rotation angle;

[0092] The step S115 of acquiring the plurality of target images of the object to be modeled by the camera device.

[0093] Optionally, the camera device is a video camera of the above-mentioned measuring device, and the rotating device is a two-axis holder of the above-mentioned measuring device.

[0094] The predetermined horizontal field of view of the camera device is a, and the predetermined vertical field of view of the camera device is b; the rotating device has a horizontal rotation angle and a vertical rotation angle, wherein the horizontal rotation angle of the rotating device is ha, and the vertical rotation angle of the rotating device is va.

[0095] Optionally, if the horizontal rotation interval of the rotating device is step_ha and the vertical rotation interval of the rotating device is step_va, then step_ha=a*0.8 and step_va=b*0.8, the horizontal rotation interval and the vertical rotation interval of the rotating device are smaller than the camera field of view angle of the camera device, which can ensure that the target images cover all regions of the object to be modeled and there is an overlapping region between adjacent images.

[0096] Optionally, in order to determine the starting rotation angle and the ending rotation angle of the rotating device, four predetermined rotation angles of the rotating device can be obtained, which are (ha1, va1), (ha2, va2), (ha3, va3), and (ha4, va4), and the four predetermined rotation angles can surround the region to be photographed of the object to be modeled; if the starting rotation angle of the rotating device is (sha, sva) and the ending rotation angle of the rotating device is (eha, eva), then sha=min(ha1, ha2, ha3, ha4), sva=min(va1, va2, va3, va4), eha=max(ha1, ha2, ha3, ha4), and eva=max(va1, va2, va3, va4).

[0097] After the starting rotation angle, the horizontal rotation interval, the vertical rotation interval, and the ending rotation angle of the rotating device are determined, the rotating device can be controlled to move to the starting rotation angle (sha, sva) and a first target image of the object to be modeled can be obtained by the camera device; then the rotating device can be controlled to move step by step with the horizontal rotation interval (step_ha) and a part of the target images of the object to be modeled can be obtained by the camera device until the horizontal rotation angle of the rotating device is greater than eha; then the rotating device can be controlled to move to the starting rotation angle (sha, sva) and move down by one vertical rotation interval (step_va) and a target image of the object to be modeled can be obtained by the camera device; then the rotating device can be controlled to move step by step with the horizontal rotation interval (step_ha) and a part of the target images of the object to be modeled can be obtained by the camera device until the horizontal rotation angle of the rotating device is greater than eha; and so on, the rotating device can be controlled to move from left to right and from top to bottom and a plurality of target images of the object to be modeled can be obtained by the camera device until the vertical rotation angle of the rotating device is greater than eva, so that the target images of all regions of the object to be modeled can be obtained.

[0098] In the above process, the rotating device can be better controlled to move by the starting rotation angle, the horizontal rotation interval, the vertical rotation interval, and the ending rotation angle, and the plurality of target images of the object to be modeled can be better obtained by the camera device, so that the effect of the method on the three-dimensional modeling of the object to be modeled can be better guaranteed.

[0099] Referring toFigure 3 , Figure 3 A flowchart of step S130 provided by an embodiment of the present application is shown.

[0100] In some embodiments of the present application, step S130 of calculating the target plane equation of the predetermined target plane based on the plurality of target images and the known plane can include the following steps:

[0101] Step S131, determining the auxiliary line corresponding to the predetermined target plane according to the geometric relationship between the known plane and the predetermined target plane on the plurality of target images.

[0102] Step S132, calculating the target plane equation of the predetermined target plane according to the pixel coordinates of the auxiliary line corresponding to the predetermined target plane.

[0103] In the above process, the method can more quickly calculate the target plane equation of the predetermined target plane, and then can more quickly perform three-dimensional modeling on the object to be modeled.

[0104] Optionally, when determining the auxiliary line corresponding to the predetermined target plane according to the geometric relationship between the known plane and the predetermined target plane on the plurality of target images, the method can include:

[0105] On the plurality of target images, determining the predetermined target plane perpendicular to the known plane as a first target plane, and determining the intersection line of the known plane and the first target plane as the corresponding auxiliary line.

[0106] On the plurality of target images, determining the predetermined target plane parallel to the known plane as a second target plane, and determining the line perpendicular to the known plane and the second target plane and intersecting the known plane and the second target plane as the corresponding auxiliary line.

[0107] Specifically, when the known plane and a certain predetermined target plane are neither perpendicular nor parallel, the auxiliary line corresponding to the predetermined target plane cannot be determined, i.e., it can not be necessary to determine; when the known plane and a certain predetermined target plane are perpendicular, and the predetermined target plane has no intersection line with the known plane, the auxiliary line corresponding to the predetermined target plane cannot be determined, i.e., it can not be necessary to determine; when the known plane and a certain predetermined target plane are parallel, and there is no line segment perpendicular to the known plane and the predetermined target plane and intersecting the known plane and the predetermined target plane, the auxiliary line corresponding to the predetermined target plane cannot be determined, i.e., it can not be necessary to determine.

[0108] Referring to Figure 4 , Figure 4 A schematic diagram of determining an auxiliary line when two planes are perpendicular and intersecting, provided by an embodiment of the present application, is shown in Figure 4In the given plane, a known plane is perpendicular to and intersects a predetermined target plane. Points C, D, and E are points on the known plane, and these three points determine the equation of the known plane. Line segment AB is the intersection line between the predetermined target plane and the known plane, so line segment AB can be determined as the corresponding auxiliary line.

[0109] See Figure 5 , Figure 5 This is a schematic diagram illustrating the determination of auxiliary lines between two parallel planes provided in an embodiment of this application. Figure 5 In the given plane and Figure 5 The plane is parallel to a predetermined target plane in the upper left corner. Points C, D, and E are points on the known plane. Points C, D, and E determine the equation of the known plane. The line segment AB is perpendicular to both the known plane and the predetermined target plane, so the line segment AB can be determined as the corresponding auxiliary line.

[0110] In the above process, this method can quickly determine the auxiliary lines corresponding to the predetermined target plane, which in turn makes it easier to calculate the target plane equation of the predetermined target plane.

[0111] Optionally, when calculating the target plane equation of the predetermined target plane based on the pixel coordinates of the auxiliary lines corresponding to the predetermined target plane, the following may be included:

[0112] When the known plane is perpendicular to and intersects the first target plane, obtain any two first pixel points on the corresponding auxiliary line and get the corresponding pixel coordinates;

[0113] Based on the camera imaging principle and the known plane equation, the three-dimensional spatial coordinates of the two first pixel points represented by the spatial points in the camera coordinate system are obtained.

[0114] The equation of the plane containing two spatial points and perpendicular to the known plane is the equation of the corresponding target plane.

[0115] When the known plane is parallel to the second target plane, obtain the pixel coordinates of the two endpoints on the corresponding auxiliary line;

[0116] Based on the camera imaging principle and the known plane equation, the three-dimensional spatial coordinates of the endpoint on the known plane in the camera coordinate system are obtained, denoted as the perpendicular foot coordinates.

[0117] Based on the camera imaging principle, calculate the spatial coordinates of the pixel coordinates of the other endpoint on the imaging plane, and denot them as the far point coordinates.

[0118] Calculate the coordinates of the intersection point of the line connecting the foot of the perpendicular to the known plane and the line connecting the center point and the far point of the camera image;

[0119] The intersection point is calculated and the plane equation of the plane parallel to the known plane is the corresponding target plane equation.

[0120] Referring to Figure 6 , Figure 6 The schematic diagram for calculating the target plane equation of the target plane perpendicular to the known plane provided by the embodiment of the present application is shown in Figure 6 , O is the imaging center point of the camera, P is the imaging plane of the camera, F is the known plane, A and B are respectively the imaging coordinates of two pixel points on the intersection line in the camera coordinate system, the straight line OA and the straight line OB intersect the known plane F at points C and D respectively, thus the three-dimensional space coordinates of the points C and D can be calculated, and then the target plane equation of a predetermined target plane (i.e. the first target plane) containing the straight line CD and perpendicular to the known plane F can be calculated.

[0121] Referring to Figure 7 , Figure 7 The schematic diagram for calculating the target plane equation of the target plane parallel to the known plane provided by the embodiment of the present application is shown in Figure 7 , O is the imaging center point of the camera, P is the imaging plane of the camera, F is the known plane, A and B are respectively the imaging coordinates of two end points on the auxiliary line in the camera coordinate system, wherein A is the foot point, the straight line OA intersects the known plane F at point C, the straight line CD is perpendicular to the known plane F and intersects the straight line OB at point D, thus the three-dimensional space coordinates of the points C and D can be calculated, and then the target plane equation of a predetermined target plane (i.e. the second target plane) passing through the point D and parallel to the known plane F can be calculated.

[0122] Optionally, when there are multiple layers of planes on the same target image, referring to Figure 8 , Figure 8 The schematic diagram for multiple layers of planes on the same target image is shown in Figure 8 , C, D and E are points on the known plane, the known plane equation of the known plane is determined by the points C, D and E, Figure 8 there is a multiple layer platform in the upper left corner, wherein the topmost target plane and the known plane cannot directly determine the auxiliary line, then the target plane equation of the middle target plane can be calculated by using the AB line segment as the auxiliary line, and then the target plane equation of the topmost target plane can be calculated by using the middle target plane as the known plane and the EF line segment as the corresponding auxiliary line.

[0123] Optionally, referring to Figure 9 , Figure 9 The schematic diagram for calculating the target plane equation of the target plane across images provided by the embodiment of the present application is shown in Figure 9In the case, the plane A is a known plane, the plane B is perpendicular to the plane A, and the plane C is parallel to the plane A, and the two small squares in the two target images are the plane B, so that in this case, the equation of the plane B in the camera coordinate system of the bottom target image can be calculated by using the plane A (the known plane) and the auxiliary line, the rotation translation matrix can be calculated by rotating the rotation angle of the device, the equation of the plane B in the camera coordinate system of the top target image can be calculated, and finally the equation of the plane C can be calculated by using the plane B and the auxiliary line in the top target image.

[0124] In the above process, the method can more quickly calculate the target plane equation of the predetermined target plane.

[0125] Referring to Figure 10 , Figure 10 A flowchart of step S140 provided by the embodiment of the present application is shown.

[0126] In some embodiments of the present application, step S140, based on all known planes and all known plane equations, constructing a three-dimensional space model of the object to be modeled, can include the following steps:

[0127] Step S141, calculating the intersection line equation and intersection point coordinates of all known planes according to all known planes and all known plane equations;

[0128] Step S142, constructing a three-dimensional space model of the object to be modeled according to the intersection line equation and intersection point coordinates of all known planes.

[0129] Specifically, after the intersection line equation and intersection point coordinates of all known planes are calculated, the intersection line and intersection point of all known planes can be determined according to the intersection line equation and intersection point coordinates of all known planes, and then a three-dimensional space model of the object to be modeled can be constructed according to the intersection line and intersection point of all known planes.

[0130] In the above process, the method can quickly perform three-dimensional modeling on the object to be modeled, and can better guarantee a good restoration degree of the appearance of the object to be modeled.

[0131] In some embodiments of the present application, the image-based three-dimensional modeling method of the embodiment of the present application, after step S140, based on all known planes and all known plane equations, constructing a three-dimensional space model of the object to be modeled, can further include the following steps:

[0132] Based on the pixel coordinates of any pixel point of any target image, obtaining the three-dimensional space coordinates of the corresponding pixel point on the three-dimensional space model;

[0133] Based on three-dimensional space coordinates of the plurality of space points, geometric information between the plurality of space points is calculated.

[0134] Optionally, by using a camera imaging principle, the imaging point coordinates of the space point corresponding to the pixel point on the imaging plane can be calculated according to the pixel coordinates of the pixel point and the focal length parameter of the camera; thus, a straight line connecting the imaging center point and the imaging point can be calculated, and the space coordinates of the intersection point of the straight line and the known plane are the three-dimensional space coordinates of the space point corresponding to the pixel point.

[0135] Optionally, the geometric information can include spatial distance, area, volume and the like; it can be understood that the geometric information between the plurality of space points can include spatial distance between the plurality of space points, area of a face formed by the plurality of space points, volume of a polyhedron formed by the plurality of space points, and the like.

[0136] In the implementation process, the method can calculate the geometric information between the plurality of space points, so that the three-dimensional space model can be better understood and analyzed.

[0137] Embodiment Two

[0138] In order to perform the method corresponding to the above-mentioned embodiment one to achieve the corresponding functions and technical effects, a three-dimensional modeling device based on images is provided below.

[0139] Reference Figure 11 , Figure 11 The structure block diagram of the three-dimensional modeling device based on images provided by the embodiments of the present application.

[0140] The three-dimensional modeling device based on images of the embodiments of the present application comprises:

[0141] The acquisition module 210 is configured to acquire a plurality of target images of the object to be modeled, the target images being images of different regions of the object to be modeled;

[0142] The determination module 220 is configured to determine a known plane and a known plane equation based on any one of the target images;

[0143] The calculation module 230 is configured to calculate a target plane equation of a predetermined target plane based on the plurality of target images and the known plane, the corresponding predetermined target plane being the known plane, and the corresponding target plane equation being the known plane equation;

[0144] The construction module 240 is configured to construct a three-dimensional space model of the object to be modeled based on all the known planes and all the known plane equations.

[0145] The image-based three-dimensional modeling device provided in the embodiments of the present application determines a known plane and a known plane equation through any one acquired target image, and calculates a target plane equation of a predetermined target plane based on the acquired target images and the known plane, and further constructs a three-dimensional space model of the object to be modeled based on all the known planes and all the known plane equations. Compared with the prior art, this method can quickly calculate the plane equation of the appearance plane of the object to be modeled, so that the three-dimensional modeling of the object to be modeled can be quickly performed, and the appearance of the object to be modeled can be restored well, thereby improving the efficiency of three-dimensional modeling and greatly reducing the cost.

[0146] As an optional implementation, the acquisition module 210 can be specifically configured to:

[0147] acquire a predetermined horizontal field of view and a predetermined vertical field of view of the camera device;

[0148] determine a horizontal rotation interval and a vertical rotation interval of the rotating device according to the predetermined horizontal field of view and the predetermined vertical field of view;

[0149] determine a starting rotation angle and an ending rotation angle of the rotating device;

[0150] control the rotating device to move according to the starting rotation angle, the horizontal rotation interval, the vertical rotation interval and the ending rotation angle;

[0151] acquire a plurality of target images of the object to be modeled by the camera device.

[0152] As an optional implementation, the calculation module 230 can be specifically configured to:

[0153] determine an auxiliary line corresponding to the predetermined target plane according to the geometric relationship between the known plane and the predetermined target plane on the plurality of target images;

[0154] calculate the target plane equation of the predetermined target plane according to the pixel coordinates of the auxiliary line corresponding to the predetermined target plane.

[0155] Optionally, when the calculation module 230 determines the auxiliary line corresponding to the predetermined target plane according to the geometric relationship between the known plane and the predetermined target plane on the plurality of target images, the calculation module 230 can:

[0156] determine a first target plane as the predetermined target plane perpendicular to the known plane on the plurality of target images, and determine the intersection line between the known plane and the first target plane as the corresponding auxiliary line;

[0157] On the plurality of target images, a predetermined target plane parallel to the known plane is determined as a second target plane, and a line perpendicular to the known plane and the second target plane and intersecting the known plane and the second target plane is determined as a corresponding auxiliary line.

[0158] Optionally, when the target plane equation of the predetermined target plane is calculated according to the pixel coordinates of the corresponding auxiliary line of the predetermined target plane, the calculation module 230 can:

[0159] When the known plane is perpendicular to and intersects the first target plane, any two first pixel points on the corresponding auxiliary line are obtained, and corresponding pixel coordinates are obtained;

[0160] According to the camera imaging principle and the known plane equation, three-dimensional space coordinates of the space points represented by the two first pixel points in the camera coordinate system are obtained;

[0161] The plane equation of the plane containing the two space points and being perpendicular to the known plane is calculated as the corresponding target plane equation;

[0162] When the known plane is parallel to the second target plane, the pixel coordinates of the two end points on the corresponding auxiliary line are obtained;

[0163] According to the camera imaging principle and the known plane equation, three-dimensional space coordinates of the end points on the known plane in the camera coordinate system are obtained, denoted as the foot coordinates;

[0164] According to the camera imaging principle, the pixel coordinates of the other end point are calculated as the space point coordinates of the imaging plane, denoted as the far point coordinates;

[0165] The intersection coordinates of the straight line passing through the foot and being perpendicular to the known plane and the line connecting the camera imaging center point and the far point are calculated;

[0166] The plane equation of the plane passing through the intersection and being parallel to the known plane is calculated as the corresponding target plane equation.

[0167] As an optional implementation, the construction module 240 can be specifically used for:

[0168] According to all known planes and all known plane equations, the intersection line equation and intersection point coordinates of all known planes are calculated;

[0169] According to the intersection line equation and intersection point coordinates of all known planes, a three-dimensional space model of the object to be modeled is constructed.

[0170] The above-mentioned image-based three-dimensional modeling device can implement the image-based three-dimensional modeling method of the above-mentioned embodiment one. The options in the above-mentioned embodiment one are also applicable to this embodiment, which will not be described in detail here.

[0171] The remaining content of the embodiments of the present application can refer to the content of the above-mentioned embodiment one. In the present embodiment, no longer be described.

[0172] Embodiment three

[0173] The embodiments of the present application provide an electronic device, comprising a memory and a processor, the memory is used for storing a computer program, the processor runs the computer program to make the electronic device execute the above-mentioned three-dimensional modeling method based on image.

[0174] Optionally, the above-mentioned electronic device can be a server.

[0175] In addition, the embodiments of the present application also provide a computer readable storage medium, which stores a computer program, the computer program is executed by a processor to realize the above-mentioned three-dimensional modeling method based on image.

[0176] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented by other ways. The apparatus embodiment described above is only schematic, for example, the flow chart and block diagram in the drawing show the possible implementation architecture, function and operation of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flow chart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementation ways, the functions noted in the block can also occur in different order from that noted in the drawing. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the function involved. It should also be noted that each block in the block diagram and / or flow chart, and the combination of blocks in the block diagram and / or flow chart, can be implemented by a dedicated hardware-based system for implementing the specified function or action, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0177] In addition, each functional module in the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0178] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts of the prior art that make contributions or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0179] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0180] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0181] It should be noted that, in this document, the terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

Claims

1. A method of image-based three-dimensional modeling, characterized by, The method comprises the following steps: acquiring a plurality of target images of an object to be modeled, the target images being images of different regions on the object to be modeled; determining a known plane and a known plane equation based on any one of the target images; calculating a target plane equation of a predetermined target plane based on the plurality of target images and the known plane, the corresponding predetermined target plane being a known plane when the corresponding target plane equation is known, and the corresponding target plane equation being a known plane equation; constructing a three-dimensional space model of the object to be modeled based on all the known planes and all the known plane equations; wherein the calculation of the target plane equation of the predetermined target plane based on the plurality of target images and the known plane comprises: determining an auxiliary line corresponding to the predetermined target plane according to the geometric relationship between the known plane and the predetermined target plane on the plurality of target images; calculating the target plane equation of the predetermined target plane according to the pixel coordinates of the auxiliary line corresponding to the predetermined target plane; wherein the determination of the auxiliary line corresponding to the predetermined target plane according to the geometric relationship between the known plane and the predetermined target plane on the plurality of target images comprises: determining a first target plane as the predetermined target plane perpendicular to the known plane on the plurality of target images, and determining the intersection line between the known plane and the first target plane as the corresponding auxiliary line; determining a second target plane as the predetermined target plane parallel to the known plane on the plurality of target images, and determining a line perpendicular to the known plane and the second target plane and intersecting the known plane and the second target plane as the corresponding auxiliary line; wherein the calculation of the target plane equation of the predetermined target plane according to the pixel coordinates of the auxiliary line corresponding to the predetermined target plane comprises: when the known plane is perpendicular to and intersects the first target plane, acquiring any two first pixel points on the corresponding auxiliary line and obtaining the corresponding pixel coordinates; obtaining the three-dimensional space coordinates of the space points represented by the two first pixel points in the camera coordinate system according to the camera imaging principle and the known plane equation; calculating the plane equation of the plane containing the two space points and perpendicular to the known plane as the corresponding target plane equation; when the known plane is parallel to the second target plane, acquiring the pixel coordinates of the two end points on the corresponding auxiliary line; obtaining the three-dimensional space coordinates of the end points on the known plane in the camera coordinate system according to the camera imaging principle and the known plane equation, denoted as the foot coordinates; calculating the space point coordinates of the pixel coordinates of the other end point on the imaging plane according to the camera imaging principle, denoted as the far point coordinates; calculating the intersection coordinates of the straight line passing through the foot and perpendicular to the known plane and the line connecting the camera imaging center point and the far point; calculating the plane equation of the plane passing through the intersection and parallel to the known plane as the corresponding target plane equation.

2. The image-based three-dimensional modeling method according to claim 1, characterized in that, The acquisition of the plurality of target images of the object to be modeled comprises: acquire a predetermined horizontal field of view and a predetermined vertical field of view of a camera device; determine a horizontal rotation interval and a vertical rotation interval of a rotating device according to the predetermined horizontal field of view and the predetermined vertical field of view; determine a starting rotation angle and an ending rotation angle of the rotating device; control movement of the rotating device according to the starting rotation angle, the horizontal rotation interval, the vertical rotation interval and the ending rotation angle; acquire a plurality of target images of an object to be modeled by the camera device.

3. The image-based three-dimensional modeling method of claim 1, wherein, construct a three-dimensional space model of the object to be modeled based on all the known planes and all the known plane equations, including: calculate intersection line equations and intersection point coordinates of all the known planes according to all the known planes and all the known plane equations; construct the three-dimensional space model of the object to be modeled according to the intersection line equations and the intersection point coordinates of all the known planes.

4. The image-based three-dimensional modeling method of claim 1, wherein, After the three-dimensional space model of the object to be modeled is constructed based on all the known planes and all the known plane equations, the method further includes: obtain three-dimensional space coordinates of a space point corresponding to a pixel point of any target image on the three-dimensional space model based on pixel coordinates of the pixel point of the target image; calculate geometric information between a plurality of space points based on three-dimensional space coordinates of the plurality of space points.

5. An image-based three-dimensional modeling apparatus, characterized by comprising: includes: an acquisition module configured to acquire a plurality of target images of an object to be modeled, the target images being images of different regions of the object to be modeled; a determination module configured to determine a known plane and a known plane equation based on any target image; a calculation module configured to calculate a target plane equation of a predetermined target plane based on a plurality of target images and the known plane, the corresponding predetermined target plane being a known plane when the corresponding target plane equation is known, and the corresponding target plane equation being a known plane equation; a construction module configured to construct a three-dimensional space model of the object to be modeled based on all the known planes and all the known plane equations; The calculation module is specifically configured to: determine an auxiliary line corresponding to the predetermined target plane on a plurality of target images according to a geometric relationship between the known plane and the predetermined target plane; calculate the target plane equation of the predetermined target plane according to pixel coordinates of the auxiliary line corresponding to the predetermined target plane; When the calculation module determines the auxiliary line corresponding to the predetermined target plane on a plurality of target images according to a geometric relationship between the known plane and the predetermined target plane, the calculation module is specifically configured to determine a first target plane that is perpendicular to the known plane as the predetermined target plane on a plurality of target images, and determine an intersection line of the known plane and the first target plane as the corresponding auxiliary line; determine a second target plane that is parallel to the known plane as the predetermined target plane on a plurality of target images, and determine a line that is perpendicular to the known plane and the second target plane and intersects the known plane and the second target plane as the corresponding auxiliary line. The calculation module is specifically configured to, when calculating a target plane equation of a predetermined target plane according to a pixel coordinate of an auxiliary line corresponding to the predetermined target plane, acquire any two first pixel points on the corresponding auxiliary line and obtain the corresponding pixel coordinates when the known plane is perpendicular to and intersects the first target plane. According to the camera imaging principle and the known plane equation, three-dimensional space coordinates of the two first pixel points in the camera coordinate system are obtained. The plane equation of the plane containing the two space points and being perpendicular to the known plane is calculated as the corresponding target plane equation. When the known plane is parallel to the second target plane, the pixel coordinates of the two end points on the corresponding auxiliary line are acquired. According to the camera imaging principle and the known plane equation, three-dimensional space coordinates of the end points on the known plane in the camera coordinate system are obtained, denoted as the foot coordinates. According to the camera imaging principle, the pixel coordinates of the other end point in the imaging plane are calculated as the space point coordinates, denoted as the far point coordinates. The intersection coordinates of the line passing through the foot and being perpendicular to the known plane and the line connecting the camera imaging center point and the far point are calculated. The plane equation of the plane passing through the intersection and being parallel to the known plane is calculated as the corresponding target plane equation.

6. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to perform the image-based three-dimensional modeling method according to any one of claims 1 to 5.

7. A computer readable storage medium characterized by, The computer program is stored in the memory and is executed by the processor to implement the image-based three-dimensional modeling method according to any one of claims 1 to 5.

Images