Urban three-dimensional reconstruction method and system based on smart lamp pole, and storage medium

Abstract

The invention discloses a city three-dimensional reconstruction method and system based on an intelligent lamp post and a storage medium, and the method comprises the steps: carrying out the point cloud transformation of pixel points in a left image based on an obtained disparity map, generating an initial local point cloud under a camera coordinate system, carrying out the coordinate system transformation of the initial local point cloud, and generating a three-dimensional reconstruction image; generating an initial global point cloud under a world coordinate system; respectively calculating a global normal vector and a local normal vector of any point in the initial global point cloud; transforming the local normal vector into a world coordinate system, and calculating the normal vector similarity between the transformed local normal vector and the global normal vector; and denoising the initial global point cloud based on the normal vector similarity to generate a first global point cloud. According to the technical scheme of the invention, the global normal vector and the local normal vector of any point in the initial global point cloud are calculated, and then the normal vector similarity is obtained, so that the global point cloud data obtained by splicing a plurality of binocular cameras are denoised, and the quality of urban three-dimensional reconstruction is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of smart cities, and more specifically, to a method for three-dimensional reconstruction of a city based on smart lamp poles, a system for three-dimensional reconstruction of a city based on smart lamp poles, and a computer storage medium. Background Art

[0002] Urban 3D reconstruction is a technology that reconstructs scenes by obtaining geographic and architectural information from a variety of data sources (such as satellite images, drone photography, lidar scanning, etc.). This information can be converted into a data source for a 3D reconstruction model of the city using computer vision, photogrammetry, and data processing. This process can accurately reproduce the city's buildings, roads, and terrain, providing basic data support for urban planning, architectural design, virtual reality applications, and other fields.

[0003] In the prior art, data is often collected through mobile devices (such as drones, collection vehicles, etc.), for example, the Chinese invention patent application with publication number CN115294270A "A three-dimensional reconstruction method, device and storage medium based on scene sparse features". This urban three-dimensional reconstruction method based on mobile devices to obtain image data requires re-collection every time data needs to be updated, which undoubtedly increases the cost of urban three-dimensional reconstruction.

[0004] Taking into account that various types of cameras will be installed in the city, these cameras can be used as data acquisition devices for urban three-dimensional reconstruction. Especially for binocular cameras, the three-dimensional information of any target within the field of view can be directly obtained based on the principle of parallax calculation, so as to obtain the point cloud data required for urban three-dimensional reconstruction.

[0005] However, due to the influence of factors such as light intensity, object reflection, and parallax calculation errors, the global point cloud data obtained by stitching multiple binocular cameras will contain noise, which seriously affects the quality of urban 3D reconstruction. Summary of the invention

[0006] The purpose of this application is to: How to denoise the global point cloud data obtained by stitching multiple binocular cameras to ensure the quality of urban three-dimensional reconstruction.

[0007] The technical solution of the first aspect of the present application is: to provide a method for urban 3D reconstruction based on smart lamp poles, the method comprising: step 1: based on the obtained disparity map, performing point cloud transformation on the pixel points in the left image to generate an initial local point cloud f in the camera coordinate system i , and then the initial local point cloud f iPerform coordinate system transformation to generate the initial global point cloud F0 in the world coordinate system, where the disparity map is generated by the binocular camera, and the left image is taken by the left camera in the binocular camera, and the binocular camera is set on the smart light pole; Step 2: Calculate any point p in the initial global point cloud F0 respectively i The global normal vector n Gi and the local normal vector Among them, the global normal vector n Gi For point p i Normal vector in the initial global point cloud F0, local normal vector For point p i In the initial local point cloud f i Normal vector in; Step 3: transform the local normal vector to the world coordinate system, and calculate the normal vector similarity between the transformed local normal vector and the global normal vector; Step 4: Based on the normal vector similarity, denoise the initial global point cloud F0 to generate a first global point cloud F1, wherein the first global point cloud is used for urban 3D reconstruction.

[0008] In any of the above technical solutions, further, step 3 specifically includes: step 31: based on the coordinate transformation matrix, transforming the local normal vector to the world coordinate system; step 32: calculating the normal vector difference value between the transformed local normal vector and the global normal vector; step 33: calculating the normal vector similarity according to the normal vector difference value, wherein the calculation formula of the normal vector similarity is:

[0009]

[0010] γ(p i )=arccos(|n Gi ·n Li |)

[0011] In the formula, D(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding normal vector similarity, γ(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding difference value, t2 is the preset difference parameter, t1 is the preset similarity threshold, n Gi For the i-th point p i The corresponding global normal vector, n Li is the transformed local normal vector.

[0012] In any of the above technical solutions, further, the global normal vector n Gi The calculation process is as follows: Based on the k-nearest neighbor algorithm, in the initial global point cloud F0, determine the point p i The corresponding neighborhood point set And calculate the neighborhood point set The covariance matrix C Gi ; For the covariance matrix C Gi Perform eigenvalue decomposition and select the covariance matrix C Gi The eigenvector corresponding to the minimum eigenvalue is used as the initial global normal vector; the initial global normal vector is normalized to obtain the global normal vector n Gi .

[0013] In any of the above technical solutions, the method further includes: Step 5: when it is determined that any position coordinate in the initial global point cloud F0 corresponds to at least two points p i When, based on the shooting angle and shooting distance, calculate any point p i The confidence relative to the stereo camera, where the shooting angle is determined by point p i The sight direction vector with the optical center of the binocular camera is determined, and the shooting distance is determined by point p i The distance between the binocular camera and the optical center is determined; Step 6: Based on the confidence, filter the initial global point cloud F0 and select the point p with the largest confidence i , as the position coordinates of point p i , generate the second global point cloud F2.

[0014] In any of the above technical solutions, further, the confidence calculation formula is:

[0015]

[0016] r i =∥p i -O j ∥

[0017] θ i =arccos(|d i ·n Gi |)

[0018]

[0019] In the formula, θ i is the shooting angle; r i is the optical center of the binocular camera o j To point p i Distance; d i is the unit vector of the sight direction; Q i For point p i confidence level.

[0020] The technical solution of the second aspect of the present application is: to provide a city 3D reconstruction system based on a smart lamp pole, the system comprising: a first point cloud unit, the first point cloud unit is configured to perform point cloud transformation on the pixel points in the left image based on the acquired disparity map, and generate an initial local point cloud f in the camera coordinate system i , and then the initial local point cloud f i The coordinate system is transformed to generate the initial global point cloud F0 in the world coordinate system, wherein the disparity map is generated by the binocular camera, the left image is taken by the left camera in the binocular camera, and the binocular camera is set on the smart lamp pole; the normal vector calculation unit is configured to calculate any point p in the initial global point cloud F0 respectively i The global normal vector n Gi and the local normal vector Among them, the global normal vector n Gi For point p i Normal vector in the initial global point cloud F0, local normal vector For point p i In the initial local point cloud f i a normal vector in; a similarity calculation unit, the similarity calculation unit is configured to transform the local normal vector into the world coordinate system, and calculate the normal vector similarity between the transformed local normal vector and the global normal vector; a second point cloud unit, the second point cloud unit is configured to denoise the initial global point cloud F0 based on the normal vector similarity to generate a first global point cloud F1, wherein the first global point cloud is used for urban three-dimensional reconstruction.

[0021] In any of the above technical solutions, further, the similarity calculation unit is configured to: transform the local normal vector to the world coordinate system based on the coordinate transformation matrix; calculate the normal vector difference value between the transformed local normal vector and the global normal vector; calculate the normal vector similarity according to the normal vector difference value, wherein the calculation formula of the normal vector similarity is:

[0022]

[0023] γ(p i )=arccos(|n Gi ·n Li |)

[0024] In the formula, D(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding normal vector similarity, γ(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding difference value, t2 is the preset difference parameter, t1 is the preset similarity threshold, n Gi For the i-th point p iThe corresponding global normal vector, n Li is the transformed local normal vector.

[0025] In any of the above technical solutions, further, the normal vector calculation unit is configured to: determine the point p in the initial global point cloud F0 based on the k-nearest neighbor algorithm i The corresponding neighborhood point set And calculate the neighborhood point set The covariance matrix C Gi ; For the covariance matrix C Gi Perform eigenvalue decomposition and select the covariance matrix C Gi The eigenvector corresponding to the minimum eigenvalue is used as the initial global normal vector; the initial global normal vector is normalized to obtain the global normal vector n Gi .

[0026] In any of the above technical solutions, further, the system further includes: a confidence calculation unit, the confidence calculation unit is configured to, when determining that any position coordinate in the initial global point cloud F0 corresponds to at least two points p i When, based on the shooting angle and shooting distance, calculate any point p i The confidence relative to the stereo camera, where the shooting angle is determined by point p i The sight direction vector with the optical center of the binocular camera is determined, and the shooting distance is determined by point p i The distance between the optical center of the binocular camera is determined; the filtering unit is configured to filter the initial global point cloud F0 based on the confidence level, and select the point p with the largest confidence level. i , as the position coordinates of point p i , generate the second global point cloud F2.

[0027] The technical solution of the third aspect of the present application is: a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, a method for urban three-dimensional reconstruction based on smart lamp poles as described in any one of the technical solutions of the first aspect of the present application is implemented.

[0028] The beneficial effects of this application are:

[0029] The technical solution in the present application first performs point cloud transformation based on the disparity map acquired by the binocular camera in the smart lamp pole to obtain the initial local point cloud in the camera coordinate system and the initial global point cloud in the world coordinate system, and then calculates the corresponding local normal vector and global normal vector respectively to obtain the similarity between the two normal vectors, so as to denoise the point cloud data acquired / transformed by multiple binocular cameras and ensure the quality of urban three-dimensional reconstruction.

[0030] In order to denoise / filter the point cloud in the overlapping field of view area, this application also uses the shooting distance and shooting angle of the binocular camera optical center relative to the point to calculate the corresponding confidence level, so as to filter out the points with small confidence level as noise, thereby optimizing the initial global point cloud and further improving the quality of the urban 3D reconstruction data source. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The advantages of the above and / or additional aspects of the present application will become apparent and easily understood in the description of the embodiments in conjunction with the following drawings, in which:

[0032] Figure 1 is a schematic flow chart of a method for urban three-dimensional reconstruction based on smart lamp poles according to an embodiment of the present application;

[0033] Figure 2 is a schematic diagram of the angle between the line of sight and the target plane according to an embodiment of the present application;

[0034] Figure 3 is a schematic diagram of shooting angle of view and shooting distance according to an embodiment of the present application;

[0035] Figure 4 It is a schematic block diagram of a city 3D reconstruction system based on smart lamp poles according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0037] In the following description, many specific details are elaborated to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present application is not limited to the specific embodiments disclosed below.

[0038] Embodiment 1:

[0039] like Figure 1 As shown, this embodiment provides a method for urban 3D reconstruction based on smart lamp poles, the method comprising:

[0040] Step 1: Based on the obtained disparity map, the pixels in the left image are transformed into a point cloud to generate the initial local point cloud f in the camera coordinate system. i , and then the initial local point cloud f iPerform coordinate system transformation to generate the initial global point cloud F0 in the world coordinate system, where the disparity map is generated by the binocular camera, the left image is taken by the left camera in the binocular camera, and the right image is taken by the right camera in the binocular camera. The binocular camera is set on the smart lamp pole, and its field of view is tilted downward;

[0041] Specifically, binocular cameras and related supporting devices (such as power supplies and data transmission devices) are installed on smart lamp poles in the city, and the binocular cameras are numbered according to the numbers of the smart lamp poles. The same number can also be used for both, so as to determine the data source for three-dimensional reconstruction of the city.

[0042] For the i-th binocular camera, it is assumed that it can obtain the distortion-corrected left and right images at time t. Then, the binocular disparity calculation method set inside it takes the left image as the reference and calculates the corresponding disparity map through the matching cost algorithm. The disparity value of each pixel in the left image can be obtained in the disparity map, and then the three-dimensional coordinates of each pixel in the left image in the camera coordinate system are calculated to generate the corresponding initial local point cloud f i , where the initial local point cloud f i Each point in the image has a stereo camera number, that is, the ID attribute of the stereo camera.

[0043] Since the installation height and installation position of the binocular camera are known, and the rotation matrix and offset matrix of the binocular camera can be obtained by calibration, the origin of the camera coordinate system of the binocular cameras on multiple smart light poles can be transformed to the same world coordinate system through coordinate transformation to determine their real physical space positions. Then, multiple initial local point clouds f i The point cloud coordinates in are transformed and spliced ​​to obtain the initial global point cloud F0 in the world coordinate system.

[0044] Step 2: Calculate any point p in the initial global point cloud F0 i The global normal vector n Gi and the local normal vector Among them, the global normal vector n Gi For point p i Normal vector in the initial global point cloud F0, local normal vector For point p i In the initial local point cloud f i The normal vector in ;

[0045] Step 3: Transform the local normal vector to the world coordinate system and calculate the normal vector similarity between the transformed local normal vector and the global normal vector;

[0046] Step 4: Based on the normal vector similarity, normal vector consistency detection is performed, and the initial global point cloud F0 is denoised to generate a first global point cloud F1, wherein the first global point cloud is used for urban 3D reconstruction.

[0047] Specifically, considering that the binocular camera will be affected by factors such as abnormal lighting and object reflection when taking images, and there will inevitably be some errors when calculating parallax, these factors will affect the accuracy of the city's 3D reconstruction data source.

[0048] This embodiment introduces the concept of viewing angle and performs denoising on the initial global point cloud F0 obtained by transformation. When the angle between the normal vector of the plane where the target is located in the scene and the line of sight of the binocular camera is large, even if the corresponding image feature can be obtained at this time, it is considered that the credibility of the image feature is lower than the credibility of the image feature when the angle between the line of sight and the plane normal vector is small. Figure 2 As shown, the point p captured by the rightmost binocular camera in the figure i The credibility is lower than that of the point p captured by the binocular camera in the middle of the figure. i The credibility of the point p captured by the binocular camera at the middle position i The information is most accurate.

[0049] Therefore, under the premise of being able to obtain sufficient image features, the above-mentioned image features with less credibility and the corresponding point cloud coordinates can be filtered out to ensure the accuracy of the urban 3D reconstruction data source.

[0050] Therefore, this embodiment uses the method of calculating the similarity of normal vectors to filter out the point cloud with less credibility, and obtains the first global point cloud F1 after noise reduction, wherein the normal vectors to be calculated include two types: one is the local normal vector One is the global normal vector n Gi , the normal vector can be calculated by the cross product of anisotropic vectors on the plane or by PCL surface estimation.

[0051] In addition, the calculated local normal vector Mapped to the global normal vector n Gi The world coordinate system in which it is located, and the local normal vector n in the world coordinate system is obtained. Li , so as to calculate the normal vector similarity based on the angle and direction of the two normal vectors.

[0052] After obtaining the normal vector similarity, the normal vector consistency test can be performed by threshold judgment. That is, when the normal vector similarity is greater than or equal to a certain set threshold, the point p is considered i is not a noise point; otherwise, it is considered that point p iThe noise points are removed from the initial global point cloud F0. The points retained after denoising form the first global point cloud F1, which is used as the data source for the three-dimensional reconstruction of the city. The first global point cloud F1 can be converted into a three-dimensional mesh model through the Poisson surface reconstruction algorithm to perform three-dimensional reconstruction of the city.

[0053] In any of the above embodiments, further, step 3 specifically includes:

[0054] Step 31: Based on the coordinate transformation matrix, transform the local normal vector to the world coordinate system;

[0055] Step 32: Calculate the normal vector difference value between the transformed local normal vector and the global normal vector; in this embodiment, the angle between the normal vectors is used as the normal vector difference value, and the value range of the normal vector difference value is limited to 0 by adding the absolute value of the dot product result of the two normal vectors. The corresponding calculation formula is:

[0056] γ(p i )=arccos(|n Gi ·n Li |)

[0057] In the formula, γ(p i ) is point p i The corresponding normal vector difference value.

[0058] Step 33: Calculate the normal vector similarity according to the normal vector difference value, wherein the calculation formula of the normal vector similarity is:

[0059]

[0060] γ(p i )=arccos(|n Gi ·n Li |)

[0061] In the formula, D(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding normal vector similarity, γ(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding difference value, t2 is the preset difference parameter, t1 is the preset similarity threshold, n Gi is the i-th point p i The corresponding global normal vector, n Li is the transformed local normal vector.

[0062] In any of the above embodiments, further, the global normal vector n Gi The calculation process is:

[0063] Based on the k-nearest neighbor algorithm, in the initial global point cloud F0, determine the point p i The corresponding neighborhood point set And calculate the neighborhood point set The covariance matrix C Gi ;

[0064] The covariance matrix C Gi Perform eigenvalue decomposition and select the covariance matrix C Gi The eigenvector corresponding to the minimum eigenvalue is used as the initial global normal vector;

[0065] Normalize the initial global normal vector to get the global normal vector n Gi .

[0066] Specifically, this embodiment calculates the global normal vector n Gi , local normal vector In the same way, we now calculate the global normal vector n Gi Take this as an example to illustrate.

[0067] For any point p that belongs to the initial global point cloud F0 i , we can use clustering algorithms such as the k-nearest neighbor algorithm to select the distance point p i The nearest k points are taken as point p i The neighborhood points of Afterwards, the neighborhood point set is calculated The geometric center of is calculated as follows:

[0068]

[0069] In the formula, μ Gi is the geometric center, Neighborhood point set The number of midpoints, q j Neighborhood point set The jth point in .

[0070] Afterwards, based on the neighborhood point set The geometric center μ Gi , calculate the neighborhood point set The covariance matrix C Gi , the corresponding calculation formula is:

[0071]

[0072] Then, the covariance matrix C Gi Perform eigenvalue decomposition and set the covariance matrix C Gi The eigenvalue λ Gi and the eigenvector v GiSatisfies the following formula:

[0073] C Gi v Gi =λ Gi v Gi

[0074] |C Gi -λ Gi I|=0

[0075] In the formula, I is the covariance matrix C Gi The same order of identity matrix, for the eigenvalue λ Gi Solve and select the minimum value of the calculated solution as the covariance matrix C Gi The target eigenvalue is obtained, and the eigenvector corresponding to the target eigenvalue is selected as the normal vector, that is, the initial global normal vector.

[0076] Finally, perform the unitization operation to obtain the global normal vector n Gi The specific process will not be described in detail.

[0077] In any of the above embodiments, further, in order to remove noise from the point cloud in the overlapping area of ​​the field of view, the method in this embodiment also includes:

[0078] Step 5: When it is determined that any position coordinate in the initial global point cloud F0 corresponds to at least two points p i When, based on the shooting angle and shooting distance, calculate any point p i Relative to the confidence of the stereo camera,

[0079] The shooting angle is determined by point p i The sight direction vector with the optical center of the binocular camera is determined, and the shooting distance is determined by point p i The distance between the binocular camera and the optical center is determined;

[0080] Step 6: Based on the confidence, filter the initial global point cloud F0 and select the point p with the largest confidence i , as the position coordinates of point p i , generate the second global point cloud F2.

[0081] Specifically, for an object photographed by at least two binocular cameras at the same time, the initial local point clouds f in different camera coordinate systems can be obtained through the above process. i , after coordinate system transformation and splicing, the local point clouds from different initial points may be i The point in is transformed to the same coordinate point in the world coordinate system.

[0082] That is to say, the position coordinates of some points in the initial global point cloud F0 of the object will correspond to multiple points, and since each point pi The ID attribute of the binocular camera is included in the i Therefore, when it is determined that any position coordinate in the initial global point cloud F0 corresponds to at least two points p i When , the point with lower confidence in the two points can be filtered to optimize the global point cloud and generate the second global point cloud F2. The confidence is determined by the shooting angle and shooting distance of the binocular camera, such as a piecewise function, a weighted function, etc.

[0083] It should be noted that the above process can filter the initial global point cloud F0, generate the second global point cloud F2, and then perform normal vector similarity calculation to obtain the first global point cloud F1; it can also filter the denoised first global point cloud F1 to generate the second global point cloud F2, and then use the second global point cloud F2 to perform three-dimensional reconstruction of the city.

[0084] In any of the above embodiments, further, considering that the smaller the shooting angle of the binocular camera, the clearer the object in the image; the farther the shooting distance, the smaller the volume of the object in the image, therefore, the calculation formula of the confidence set in this embodiment is:

[0085]

[0086] r i =∥p i -o j ∥

[0087] θ i =arccos(|d i ·n Gi |)

[0088]

[0089] In the formula, θ i is the shooting angle; r i is the optical center of the binocular camera o j To point p i Distance; d i is the unit vector of the sight direction; Q i For point p i confidence level.

[0090] Specifically, set the origin of the camera coordinate system to the optical center of the binocular camera o j , for point p i For example, the calculated unit vector d of the sight direction i for:

[0091]

[0092] Then, the sight direction vector di With the global normal vector n Gi The angle between them is taken as the shooting angle θ i , and it is agreed that the normal vector with a non-negative component in the direction of sight is taken as the target normal vector for calculation. Therefore, it is only necessary to add the absolute value of the dot product of the two vectors as the cosine value of the viewing angle, the viewing angle θ i The corresponding calculation formula is:

[0093] θ i =arccos(|d i ·n Gi |)

[0094] Select point p i With the binocular camera optical center o j The Euclidean distance between them is the shooting distance r i , the corresponding calculation formula is:

[0095] r i =||p i -o j ||

[0096] In order to achieve the effect of "the farther the shooting distance, the lower the confidence; the larger the shooting angle, the lower the confidence", the confidence calculation formula is constructed as follows:

[0097]

[0098] Finally, the calculated confidence is used to filter the initial global point cloud F0 and select the point p with the largest confidence. i , as the point p at the position coordinates i .

[0099] like Figure 3 As shown, for point p in the overlapping area of ​​vision i , the binocular camera on the left side of the figure is for point p i The “shooting distance is short and the shooting angle is small” of the binocular camera on the left, while the “shooting distance is long and the shooting angle is large” of the binocular camera on the right. Therefore, the binocular camera on the left can capture more accurate information, that is, the confidence of the points photographed by the binocular camera on the left is greater than the confidence of the points photographed by the binocular camera on the right.

[0100] Embodiment 2:

[0101] like Figure 4 As shown, this embodiment provides a city 3D reconstruction system based on smart lamp poles, the system comprising:

[0102] The first point cloud unit 10 is configured to perform point cloud transformation on the pixels in the left image based on the acquired disparity map to generate an initial local point cloud f in the camera coordinate system.i , and then the initial local point cloud f i Perform coordinate system transformation to generate the initial global point cloud F0 in the world coordinate system.

[0103] The disparity map is generated by a binocular camera, and the left image is captured by the left camera in the binocular camera, which is set on the smart light pole;

[0104] The normal vector calculation unit 20 is configured to calculate the normal vector of any point p in the initial global point cloud F0. i The global normal vector n Gi and the local normal vector

[0105] Among them, the global normal vector n Gi For point p i Normal vector in the initial global point cloud F0, local normal vector For point p i In the initial local point cloud f i The normal vector in ;

[0106] A similarity calculation unit 30, the similarity calculation unit 30 is configured to transform the local normal vector into a world coordinate system, and calculate the normal vector similarity between the transformed local normal vector and the global normal vector;

[0107] The second point cloud unit 40 is configured to denoise the initial global point cloud F0 based on normal vector similarity to generate a first global point cloud F1, wherein the first global point cloud is used for three-dimensional reconstruction of the city.

[0108] In any of the above embodiments, further, the similarity calculation unit 30 is configured as follows:

[0109] Based on the coordinate transformation matrix, transform the local normal vector to the world coordinate system;

[0110] Calculate the normal vector difference between the transformed local normal vector and the global normal vector;

[0111] According to the normal vector difference value, the normal vector similarity is calculated, where the calculation formula of the normal vector similarity is:

[0112]

[0113] γ(p i )=arccos(|n Gi ·n Li |)

[0114] In the formula, D(p i ) is the i-th point p in the initial global point cloud F0 iThe corresponding normal vector similarity, γ(p i ) is the i-th point p in the initial global point cloud f0 i The corresponding difference value, t2 is the preset difference parameter, t1 is the preset similarity threshold, n Gi For the i-th point p i The corresponding global normal vector, n Li is the transformed local normal vector.

[0115] In any of the above embodiments, further, the normal vector calculation unit 20 is configured as follows:

[0116] Based on the k-nearest neighbor algorithm, in the initial global point cloud F0, determine the point p i The corresponding neighborhood point set And calculate the neighborhood point set The covariance matrix C Gi ;

[0117] The covariance matrix C Gi Perform eigenvalue decomposition and select the covariance matrix C Gi The eigenvector corresponding to the minimum eigenvalue is used as the initial global normal vector;

[0118] Normalize the initial global normal vector to get the global normal vector n Gi .

[0119] In any of the above embodiments, further, the system further includes:

[0120] The confidence calculation unit 50 is configured to determine that any position coordinate in the initial global point cloud F0 corresponds to at least two points p i When, based on the shooting angle and shooting distance, calculate any point p i Relative to the confidence of the stereo camera,

[0121] The shooting angle is determined by point p i The sight direction vector with the optical center of the binocular camera is determined, and the shooting distance is determined by point p i The distance between the binocular camera and the optical center is determined;

[0122] The filtering unit 60 is configured to filter the initial global point cloud F0 based on the confidence level, and select the point p with the largest confidence level. i , as the position coordinates of point p i .

[0123] In any of the above embodiments, further, considering that the smaller the shooting angle of the binocular camera, the clearer the object in the image; the farther the shooting distance, the smaller the volume of the object in the image, therefore, the calculation formula of the confidence set in this embodiment is:

[0124]

[0125] r i =∥p i -o j ∥

[0126] θ i =arccos(|d i ·d Gi |)

[0127]

[0128] In the formula, θ i is the shooting angle; r i is the optical center of the binocular camera o j To point p i Distance; d i is the unit vector of the sight direction; Q i For point p i confidence level.

[0129] Embodiment 3:

[0130] This embodiment further provides a computer-readable storage medium, in which a computer program is stored. When the stored computer program is executed by a processor, the city three-dimensional reconstruction method based on smart lamp poles of any one of the above-mentioned embodiments 1 is implemented.

[0131] Specifically, the computer program includes computer instructions, which are stored in a computer-readable storage medium; the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method steps in any one of the above-mentioned embodiments 1.

[0132] The above-mentioned computer-readable storage medium may include random access memory (RAM), read-only memory (ROM), compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage or other magnetic storage devices, etc.

[0133] The above-mentioned processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0134] So far, various embodiments of the present application have been described in detail. In order to avoid obscuring the concept of the present application, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0135] Although some specific embodiments of the present application have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration rather than for limiting the scope of the present application.

[0136] The steps in this application can be adjusted in order, combined, and deleted according to actual needs.

[0137] Although the present application is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present application.

Claims

1. A method for urban 3D reconstruction based on smart lamp poles, characterized in that: The method comprises: Step 1: Based on the obtained disparity map, the pixels in the left image are transformed into a point cloud to generate the initial local point cloud f in the camera coordinate system. i , and then the initial local point cloud f i Perform coordinate system transformation to generate the initial global point cloud F0 in the world coordinate system. The disparity map is generated by a binocular camera, the left image is captured by a left camera in the binocular camera, and the binocular camera is arranged on the smart lamp pole; Step 2: Calculate any point p in the initial global point cloud F0 i The global normal vector n Gi and the local normal vector Wherein, the global normal vector n Gi For the point p i The normal vector in the initial global point cloud F0, the local normal vector For the point p i In the initial local point cloud f i The normal vector in ; Step 3: transform the local normal vector to the world coordinate system, and calculate the normal vector similarity between the transformed local normal vector and the global normal vector; Step 4: Based on the normal vector similarity, the initial global point cloud F0 is denoised to generate a first global point cloud F1, wherein the first global point cloud is used for three-dimensional reconstruction of the city.

2. The method for urban three-dimensional reconstruction based on smart lamp poles according to claim 1, characterized in that: The step 3 specifically includes: Step 31: transforming the local normal vector to the world coordinate system based on a coordinate transformation matrix; Step 32: Calculate the normal vector difference value between the transformed local normal vector and the global normal vector; Step 33: Calculate the normal vector similarity according to the normal vector difference value, wherein the calculation formula of the normal vector similarity is: c(p i )=arccos(|n Gi ·n Li |) In the formula, D(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding normal vector similarity, γ(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding difference value, t2 is the difference parameter, t1 is the similarity threshold, n Gi For the i-th point p i The corresponding global normal vector, n Li is the transformed local normal vector.

3. The urban 3D reconstruction method based on smart lamp poles according to claim 1, characterized in that: The global normal vector n Gi The calculation process is: Based on the k-nearest neighbor algorithm, in the initial global point cloud F0, the point p is determined i The corresponding neighborhood point set And calculate the neighborhood point set The covariance matrix C Gi ; For the covariance matrix C Gi Perform eigenvalue decomposition and select the covariance matrix C Gi The eigenvector corresponding to the minimum eigenvalue is used as the initial global normal vector; The initial global normal vector is normalized to obtain the global normal vector n Gi .

4. The method for urban 3D reconstruction based on smart lamp poles according to any one of claims 1 to 3, characterized in that: The method further comprises: Step 5: When it is determined that any position coordinate in the initial global point cloud F0 corresponds to at least two points p i When the shooting angle and shooting distance are used, the above-mentioned arbitrary point p is calculated respectively. i Relative to the confidence of the binocular camera, The shooting angle is defined by the point p i The shooting distance is determined by the sight direction vector of the optical center of the binocular camera. i Determine the distance between the optical center of the binocular camera and the binocular camera; Step 6: Based on the confidence, filter the initial global point cloud F0 and select the point p with the largest confidence i , as the position coordinates of the point p i , generate the second global point cloud F2.

5. The method for urban 3D reconstruction based on smart lamp poles according to claim 4, characterized in that: The confidence calculation formula is: r i =∥p i -o j ∥ i i =arccos(|d i ·n Gi |) In the formula, θ i is the shooting angle; r i is the optical center of the binocular camera o j To point p i Distance; d i is the unit vector of the sight direction; Q i For point p i confidence level.

6. A city 3D reconstruction system based on smart lamp poles, characterized in that: The system comprises: The first point cloud unit is configured to perform point cloud transformation on the pixels in the left image based on the acquired disparity map to generate an initial local point cloud f in the camera coordinate system. i , and then the initial local point cloud f i Perform coordinate system transformation to generate the initial global point cloud F0 in the world coordinate system. The disparity map is generated by a binocular camera, the left image is captured by a left camera in the binocular camera, and the binocular camera is arranged on the smart lamp pole; A normal vector calculation unit is configured to calculate the normal vector of any point p in the initial global point cloud F0. i The global normal vector n Gi and the local normal vector Wherein, the global normal vector n Gi For the point p i The normal vector in the initial global point cloud F0, the local normal vector For the point p i In the initial local point cloud f i The normal vector in ; a similarity calculation unit, wherein the similarity calculation unit is configured to transform the local normal vector into the world coordinate system, and calculate the normal vector similarity between the transformed local normal vector and the global normal vector; The second point cloud unit is configured to denoise the initial global point cloud F0 based on the normal vector similarity to generate a first global point cloud F1, wherein the first global point cloud is used for three-dimensional reconstruction of the city.

7. The urban 3D reconstruction system based on smart lamp poles according to claim 6, characterized in that: The similarity calculation unit is configured as follows: Based on a coordinate transformation matrix, transform the local normal vector to the world coordinate system; Calculating a normal vector difference value between the transformed local normal vector and the global normal vector; The normal vector similarity is calculated according to the normal vector difference value, wherein the calculation formula of the normal vector similarity is: c(p i )=arccos(|n Gi ·n Li |) In the formula, D(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding normal vector similarity, γ(p i ) is the i-th point p in the initial global point cloud F0 i The corresponding difference value, t2 is the difference parameter, t1 is the similarity threshold, n Gi For the i-th point p i The corresponding global normal vector, n Li is the transformed local normal vector.

8. The urban 3D reconstruction system based on smart lamp poles according to claim 6, characterized in that: The normal vector calculation unit is configured as follows: Based on the k-nearest neighbor algorithm, in the initial global point cloud F0, the point p is determined i The corresponding neighborhood point set And calculate the neighborhood point set The covariance matrix C Gi ; For the covariance matrix C Gi Perform eigenvalue decomposition and select the covariance matrix C Gi The eigenvector corresponding to the minimum eigenvalue is used as the initial global normal vector; The initial global normal vector is normalized to obtain the global normal vector n Gi .

9. The urban 3D reconstruction system based on smart lamp poles according to any one of claims 6 to 8, characterized in that: The system further comprises: A confidence calculation unit, wherein the confidence calculation unit is configured to determine that any position coordinate in the initial global point cloud F0 corresponds to at least two points p i When the shooting angle and shooting distance are used, the above-mentioned arbitrary point p is calculated respectively. i Relative to the confidence of the binocular camera, The shooting angle is defined by the point p i The shooting distance is determined by the sight direction vector of the optical center of the binocular camera. i Determine the distance between the optical center of the binocular camera and the binocular camera; A filtering unit, wherein the filtering unit is configured to filter the initial global point cloud F0 based on the confidence level, and select the point p with the largest confidence level. i , as the position coordinates of the point p i , generate the second global point cloud F2.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images