Binocular vision-based street tree three-dimensional modeling and positioning method and system

Abstract

The invention relates to the technical field of urban tree checking, and discloses a street tree three-dimensional modeling and positioning method and system based on binocular vision. According to the method, a vehicle-mounted binocular camera and a GNSS recorder are used for collecting city ground images and coordinates, a target detection algorithm is used for carrying out street tree detection on the collected city ground images, and a binocular stereo matching algorithm is used for obtaining distance information. On the basis, a binocular camera imaging model and collected coordinate points are combined, the geographic position of the street tree is estimated, and meanwhile the height and crown breadth of the street tree are estimated. According to the method, the urban street tree list can be automatically established to a high degree, and the urban street tree list containing geographic coordinates, crown breadth and height can be directly generated after original data are collected. The method makes up for the relative lag of urban tree information caused by the adoption of street view images, and fills up the blank of collecting high-precision tree crown breadth and tree height information while obtaining tree positioning.

Description

Technical Field

[0001] The present invention relates to the technical field of urban tree inventory, and in particular to a three-dimensional modeling and positioning method and system for roadside trees based on binocular vision. Background Art

[0002] Urban street trees play a vital role in urban ecosystems, involving carbon sequestration, improving air quality, temperature regulation, protecting biodiversity, and mitigating extreme weather events such as floods. In addition, street trees provide landscape value to urban streets and improve the physical and mental health of residents. They are important providers of ecosystem services to urban residents and are indispensable to addressing climate change. In this context, the planning and management of urban street trees based on urban tree inventories have become increasingly prominent. In order to adequately manage urban street trees, a better understanding of tree outlines and accurate geographic positioning are needed. Unfortunately, constructing an accurate tree inventory remains a prominent challenge because information on the spatial distribution, composition, and structure of urban street trees is relatively scarce.

[0003] At present, the research methods for constructing a list of street trees include traditional field survey technology, remote sensing technology, and machine vision-based technologies. For example, traditional street tree survey data is usually obtained by experts directly or indirectly measuring trees in situ, where each tree must be surveyed in person by a technician who uses the Global Navigation Satellite System (GNSS) to locate the trees, identify the tree species, and measure the tree structure, including tree diameter, height, crown width, and tree height.

[0004] Traditional field survey methods require certain technical expertise and a lot of human resources, which is time-consuming and expensive. Traditional techniques have also been shown to contain considerable risk of human error in data collection. As a result, there is a large information gap in urban street trees, and only a few cities have an up-to-date tree inventory.

[0005] The image resolution used by remote sensing technology is not high, and there is a problem that it cannot accurately tell key information such as crown width or identify tree species. Ultra-high-resolution satellite or aerial images (<80cm) can help provide the level of detail required for individual urban tree assessments, but are often affected by urban shadows. In addition, LiDAR or data fusion methods often lack processing methods that can be promoted or automated over large areas and with high accuracy. Although remote sensing technology has enabled us to better understand urban trees, most remote sensing images, such as VHR images and LiDAR, require expensive data sets and trained experts to analyze them.

[0006] Based on machine vision-related technologies, relatively cheap and easy-to-obtain street view data are used, such as Google Street View (GSV) and Baidu Street View (BSV). However, these data sets have problems such as inconsistent temporal and spatial distribution and relatively lagging ground image information, which cannot update the urban tree list in a timely manner. Previous studies have mostly used panoramic street view images, and there are errors caused by image distortion in estimating the precise location and outline of trees. In addition, most methods focus on tree species detection and precise positioning, and there is little research on tree crown estimation.

[0007] Advanced data sources, such as remote sensing information, can obtain more and more accurate information about urban trees. Individual tree crown delineation or detection (ITCD) using remote sensing data has gained popularity since the 1980s as an alternative to traditional field survey techniques. Some related examples are locating trees from satellite images by crown recognition, identifying tree species by analyzing hyperspectral images, or delineating and locating tree crowns through laser radar data (LiDAR). For example, in the study of Pu R, Landry S, Yu Q. Assessing the potential of multi-seasonal high resolution Pléiadessatellite imagery for mapping urban tree species. International Journal of Applied Earth Observation and Geoinformation, 2018, 71: 144-158, very high resolution satellite imagery (VHR) and LiDAR were used to successfully map and classify urban forests.

[0008] Ground images collected by vehicle-mounted cameras are a huge data source in cities, which has not been fully developed and utilized in urban forestry. In recent years, with the popularization of deep learning and computer vision technology in ground image research, urban trees can be automatically located and counted from ground images. Some researchers use the R-CNN model for target detection in street view images and combine photogrammetry to geolocate street trees. In addition, the study "Lumnitz S, Devissacher T, Mayaud JR, et al. Mapping trees along urban street networks with deep learning and street-level imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175: 144-157" uses depth estimation distance in street view images and combines triangulation to determine the final coordinates of trees, so that the average accuracy of 70% of the tree coordinate points is 4-6m.

[0009] Urban roadside trees are of great value to urban ecosystems. Building an accurate tree inventory is an important aspect of urban ecological management. However, obtaining accurate tree characteristics and location information remains a prominent challenge. To address the above issues, a three-dimensional modeling and positioning method and system for roadside trees based on binocular vision is needed to estimate the height and crown width of roadside trees in a more accurate and timely manner, further obtain the geographic coordinates of roadside trees, and use this to build an accurate urban tree inventory. Summary of the invention

[0010] In view of the shortcomings of the method for constructing an accurate list of urban street trees, the present invention proposes a method and system for three-dimensional modeling and positioning of street trees based on binocular vision. The method uses a vehicle-mounted binocular camera and a GNSS recorder to collect urban ground images and coordinates, uses a target detection algorithm to detect street trees on the collected urban ground images, and uses a binocular stereo matching algorithm to obtain distance information. On this basis, the binocular camera imaging model and the collected coordinate points are combined to estimate the geographical location of the street trees, and at the same time, the height and crown width of the street trees are estimated. The method can realize the automated establishment of an urban street tree list to a high degree. After collecting the original data, a list of urban street trees containing geographical coordinates, crown width, and height can be directly generated.

[0011] The technical solution of the present invention is as follows: A three-dimensional modeling and positioning method of roadside trees based on binocular vision, comprising the following steps:

[0012] Step 1) Collect basic pictures, disparity images and GNSS data of the study area through vehicle-mounted equipment;

[0013] Step 2) Identify the collected basic image data according to the target detection algorithm and calculate the absolute depth within the detection frame;

[0014] Step 3), combining the detection frame coordinates and the absolute depth to obtain the geographic coordinates of the street tree;

[0015] Step 4) Based on the principle of perspective projection model, the crown width and tree height of the street trees are obtained.

[0016] The vehicle-mounted equipment includes a binocular depth camera and a GNSS recorder; while collecting basic images, the binocular depth camera calculates in real time the disparity image data corresponding to the viewing angles of the left and right depth cameras; the GNSS recorder collects the coordinates and driving heading angle of the vehicle once per second during the vehicle's driving; when the time when the binocular depth camera collects data is equal to the time when the GNSS recorder collects data, the two types of data are matched to obtain the basic image, disparity image and GNSS data of the coordinate point.

[0017] The target detection algorithm is a YOLOv5 detection algorithm, which detects and identifies the street trees in the basic image, extracts the coordinates of the detection frame, and calculates the absolute depth mean in the detection frame; the optimal weight obtained by the street tree detection training is loaded into the YOLOv5 target detection program, and the basic image is input into the target detection program to obtain the pixel coordinates of the street tree target detection frame and the length and width pixel values ​​of the target detection frame, and further calculates the pixel coordinates of the midpoint of the detection frame area; the disparity image data of step 1 is passed through the semi-global block matching algorithm SGBM to obtain the absolute depth of the corresponding image; when the midpoint pixel coordinates of all detection frames in the image and the absolute depth at the same coordinates are known, the absolute depth mean of the pixels around the midpoint is calculated respectively as the distance d between the street tree and the optical center.

[0018] The geographic coordinates of the roadside tree are determined by detecting the distance and heading angle between the roadside tree and the optical center. The imaging principle of the binocular camera is simplified to a perspective projection model. In the perspective projection model, the proportional relationship between the imaging coordinates and the spatial coordinates is expressed by trigonometric functions. The angle of the point relative to the optical center is calculated from the coordinates of the point on the imaging plane and the focal length of the camera by inverse trigonometric functions. When the focal length of the camera is known, the angle α between the coordinates of the point in the detection frame and the optical center is calculated by formula (1). The angle α is superimposed on the driving heading angle of the vehicle to finally obtain the heading angle θ of the roadside tree.

[0019]

[0020] Where f is the focal length of the camera; u x is the horizontal coordinate of the tree on the image u0 is the horizontal coordinate of the main point of the image;

[0021] Given the distance d to the roadside tree, the heading angle θ of the roadside tree, the latitude coordinate lat1 of the vehicle, and the longitude coordinate lon1 of the vehicle; assuming that the radius of the earth is R, Ad is the angular distance, i.e. d / R; the latitude coordinate lat2 and longitude coordinate lon2 of the roadside tree are obtained by the following formula, in radians:

[0022] lat2=arcsin(sin(lat1)cos(Ad)+cos(lat1)sin(Ad)cos(θ)) (2)

[0023] lon2=lon1+arctan(sin(θ)sin(Ad)cos(lat1),cos(Ad)-sin(lat1)sin(lat2)) (3).

[0025] According to the principle of perspective projection model, the crown width and tree height of the street trees are calculated using the following formula:

[0026]

[0027] Among them, h is the actual height of the roadside tree, w is the actual crown width of the roadside tree, v is the image distance, d is the object distance, and h sensor is the height of the object in the image, w sensor is the width of the object in the image; the image distance v is calculated by the camera focal length and sensor size; for digital cameras, the digital camera focal length f replaces the image distance.

[0028] A three-dimensional modeling and positioning system for roadside trees based on binocular vision, comprising:

[0029] Data matching module, used to match different types of data;

[0030] The data matching module matches and integrates the collected different types of basic pictures, disparity images and GNSS data according to a unified timestamp;

[0031] The matching data of the data matching module is used for data processing in subsequent modules;

[0032] The object detection module is used to detect street trees in the base image and return the detection box information;

[0033] The target detection module is equipped with a YOLOv5 detection model, returns the pixel coordinate values ​​of the detection frame, and further calculates the length and width pixel values ​​of the detection frame and the coordinates of the midpoint of the detection frame pixels;

[0034] The target detection module sends the data to the subsequent data processing module, the midpoint coordinates of the detection frame pixels are used in the absolute depth calculation module, and the length and width pixel values ​​of the detection frame are used in the calculation of the tree size estimation module;

[0035] Absolute depth calculation module, used to calculate the absolute depth value of the target;

[0036] The absolute depth calculation module obtains the absolute depth of the image by using the semi-global block matching algorithm SGBM to obtain the disparity data, and further calculates the absolute depth mean of the coordinates near the midpoint of the detection frame pixel;

[0037] The absolute depth calculation module sends the absolute depth data to the coordinate calculation module;

[0038] A coordinate calculation module is used to calculate the longitude and latitude coordinates of street trees;

[0039] Tree size calculation module, used to calculate the crown width and tree height of street trees;

[0040] Data integration module, used to handle data integration;

[0041] The data integration module processes the data and integrates them into a list, which records the latitude, longitude, crown width, and tree height information of each street tree line by line and saves them as a .csv file.

[0042] Beneficial effects of the invention: The invention makes up for the relative lag of urban tree information caused by the use of street view images, and fills the gap in simultaneously collecting high-precision tree crown width and tree height information when obtaining tree positioning. And through verification in actual case studies, the feasibility of this method in constructing an accurate urban tree inventory is determined, which promotes the scientific and refined urban ecological management.

[0043] It has important practical significance in accurately constructing an inventory of urban street trees. The binocular vision-based method usually only requires two cameras and some basic data processing units, and can achieve similar or better performance at a lower cost than other high-cost sensors (such as lidar) or complex sensor arrays. This low-cost feature makes the technology more popular and suitable for applications in a variety of budget ranges.

[0044] In the updating of urban ecosystem data, the system can regularly and quickly capture environmental changes. For the monitoring and maintenance of urban infrastructure such as street trees, it can quickly respond to environmental changes and update data in a timely manner. In addition, due to its automated characteristics, large-scale data collection and processing can be completed quickly without human intervention, thus greatly saving time.

[0045] In terms of data accuracy, the system can accurately calculate the position and shape of objects in three-dimensional space with an accuracy of millimeters. This high-precision data is crucial for the management and planning of urban ecosystems and can be used for accurate tree health assessment, pruning planning, and urban space design. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a technical flow chart of the method of the present invention;

[0047] Figure 2 YOLOv5 street tree detection;

[0048] Figure 3(a) is the camera imaging model

[0049] Figure 3(b) is a schematic diagram of the calculation of the angle of street trees;

[0050] Figure 4 Visualize the street tree detection results;

[0051] Figure 5 Satellite images of the implementation example range;

[0052] Figure 6 Visualize the measurements of street tree dimensions;

[0053] Figure 7 Geo-location visualization for street trees;

[0054] Figure 8 Performance evaluation of the street tree target detection model; (a) is the accuracy change curve of the target detection model; (b) is the recall change curve of the target detection model; (c) is the mAP@0.5 value change curve of the target detection model;

[0055] Fig. 9 To evaluate the performance of tree height prediction for roadside trees;

[0056] Fig.10 To evaluate the performance of roadside tree crown width prediction;

[0057] Fig.11 Evaluating the geolocation performance of street trees. DETAILED DESCRIPTION

[0058] The implementation process of the street tree modeling and positioning method proposed in this invention is: 1) using vehicle-mounted equipment to collect basic data of the study area; 2) using target detection algorithms to identify the collected images; 3) using depth information to estimate the geographic coordinates of street trees; 4) estimating the crown width and tree height of street trees; 5) generating a street tree list and visualizing the measurement results. The technical flow chart is as follows Figure 1 shown.

[0059] The specific process is as follows:

[0060] 1. Collect basic data of the study area using vehicle-mounted equipment

[0061] The basic data of the research area is collected by the binocular depth camera and GNSS recorder installed on the vehicle. While collecting the original ground image, the binocular depth camera calculates the basic image parallax data corresponding to the left and right camera perspectives in real time. The GNSS recorder collects the vehicle's coordinates and heading angle once per second while the vehicle is driving. When the binocular camera collects data at the same time as the GNSS recorder, the two data are matched to obtain the ground image and depth of the coordinate point.

[0062] 2. Use target detection algorithm to identify collected images

[0063] The pre-trained weights for street tree detection are loaded into the YOLOv5 target detection program, and the collected ground images are detected to obtain the pixel coordinates of the street tree target detection box and the length and width pixel values ​​of the target detection box. The pixel coordinates of the midpoint in the detection box are further calculated. At the same time, the disparity data is passed through the semi-global block matching algorithm (Semi-GlobalBlock Matching) to obtain the absolute depth of the corresponding image. When the pixel coordinates of the midpoint of the detection box and the absolute depth at the same coordinates are known, the absolute depth mean of the coordinates around the midpoint is calculated, that is, the distance d between the street tree and the optical center.

[0064] 3. Use depth information to estimate the geographic coordinates of street trees

[0065] To estimate the geographic coordinates of the roadside trees, it is necessary to know the distance and angle between the detected roadside trees and the optical center. The imaging principle of the binocular camera can be simplified to a perspective projection model, as shown in Figure 3(a) and Figure 3(b). In this perspective projection model, the proportional relationship between the imaging coordinates and the spatial coordinates can be expressed by trigonometric functions. Therefore, the inverse trigonometric function can be used to solve the angle of the point relative to the optical center from the coordinates of the point on the imaging plane and the focal length of the camera. Since the focal length of the camera is known, the angle between the coordinates of the point in the detection frame and the optical center can be calculated by formula (1) and superimposed with the heading angle of the vehicle, and finally the heading angle θ of the roadside tree can be obtained.

[0066]

[0067] Where f is the focal length of the camera. x is the horizontal coordinate of the tree on the image, and u0 is the horizontal coordinate of the main point of the image.

[0068] Given the distance d to the roadside tree, the heading angle θ of the roadside tree, and the latitude coordinate lat1 and longitude coordinate lon1 of the vehicle, let the radius of the earth be R, and Ad be the angular distance, i.e. d / R. The latitude coordinate lat2 and longitude coordinate lon2 (in radians) of the roadside tree can be obtained by the following formula:

[0069] lat2=arcsin(sin(lat1)cos(Ad)+cos(lat1)sin(Ad)cos(θ))(2)

[0070] lon2=lon1+arctan(sin(θ)sin(Ad)cos(lat1),cos(Ad)-sin(lat1)sin(lat2(3)

[0071] 4. Estimate the crown width and height of street trees

[0072] In the perspective projection model, all light rays from an object are projected onto the image plane through a single point. This projection method follows the principle of similar triangles to estimate the actual size of the object. Assuming we know the actual height (denoted as h) and crown width (denoted as w) of the street trees, we can estimate the size of the street trees according to the following formula:

[0073]

[0074] Where v is the image distance, d is the object distance, and h sensor and w sensor are the height and width of the object in the image, respectively.

[0075] The image distance v can be calculated from the camera focal length and sensor size. For digital cameras, since the sensor is very close to the lens, the image distance is usually close to the focal length, that is, the camera's focal length f (usually given in the camera specifications) can be used instead of the image distance.

[0076] 5. Generation of street tree inventory and visualization of measurement results

[0077] The collected basic data is loaded into the modeling and positioning program, which will automatically generate a list of street trees in the area. The list records the latitude, longitude, crown width, and tree height of each street tree line by line and saves it as a .csv file, which is convenient for later importing into ArcGIS or Google Earth to view the distribution of trees. The relevant functions of OpenCV image processing are used to visualize the results of street tree detection to verify the detection effect ( Figure 4 ).

[0078] In order to verify the feasibility of this method, taking some sections of roads in Hanzhong City, Hantai District, Shaanxi Province as an example, the specific implementation process of the street tree modeling and positioning method based on binocular vision will be explained in detail.

[0079] (1) Hanzhong City (107°E, 33.04°N) is located in the southern part of Shaanxi Province. It has a northern subtropical climate with an average annual temperature of 14.5°C. The climate and soil of Hanzhong City allow many evergreen plants to grow normally in the area, and some have become the main greening tree species and important agricultural industrial resources in the area. According to the urban road green space planning of Hanzhong City, in addition to traditional deciduous tree species such as sycamore and ginkgo, the street trees planted in the city are mainly evergreen tree species such as camphor, privet, osmanthus, and magnolia. In recent years, with the acceleration of the pace of urban road construction, Elaeocarpus and Lechang Michelia have been introduced as street trees and planted in some sections of the urban area. Example The main evergreen street trees in the suburbs of Hanzhong City were taken as the research object, and the sample research range was determined to be an orthogonal rectangular area of ​​1200m×2300m ( Figure 5 ).

[0080] (2) Use a vehicle-mounted GNSS recorder and a binocular camera to collect ground-measured data in the study area. The vehicle speed is 40 km / h, and the camera is set to collect one picture every 5 seconds to minimize repeated modeling of street trees. The collected data is brought into a python program containing the above technical process, which consists of a data matching module, a target detection module, an absolute depth calculation module, a coordinate calculation module, a tree size calculation module, and a data integration module. The program runs the above modules in sequence, first matching different types of data based on the same timestamp. Then the basic image is identified to obtain the pixel coordinates corresponding to the detection target, and the absolute depth calculation module is used to calculate the absolute depth of the target from the optical center. The depth information is further substituted into the coordinate calculation module and the tree size calculation module to obtain the coordinates, tree height, and crown width of the street trees. Finally, the geographic coordinates and tree sizes of the street trees were integrated into a .csv list (Table 1) through the data integration module. A total of 1,368 street trees were detected in the study area. The average height of the street trees in the statistical area was 3.96m, and the average crown width was 2.75m. The tree size generally met the basic conditions of street trees in the area.

[0081] Table 1 List of street trees in the implementation example

[0082]

[0083] Substitute the .csv file and the ground image acquisition file into the python program, and use OpenCV to visualize some ground images in the area. In the image, W is the tree crown width, H is the tree height, and D is the distance between the street tree and the camera optical center ( Figure 6 ). Use Google Earth to identify the .csv file and display the identified street trees on the map, with dots representing individual trees ( Figure 7 ) and compare the positioning effect of street trees.

[0084] (3) In order to verify the model's detection and classification performance for urban street trees, we first used the mean average precision (mAP) of the trees in the image to evaluate the trained YOLOv5 model. mAP is a comprehensive indicator that takes into account the performance of positioning (accuracy of the detection box) and classification (correct identification of objects within the detection box). Only when the detection box is accurately positioned and classified correctly will the model obtain a high mAP value. The YOLOv5 series of models are divided into five versions: n, s, m, l and x. The YOLOv5 model used in the study is YOLOv5m. After 60 epochs of training, the model's accuracy (Accuracy) on the training set reached 87%, and the recall rate (Recall) was 79%. The overall mean average precision (mAP) reached 85%, showing ideal performance in the street tree detection task ( Figure 8 ).

[0085] (4) Secondly, the results of this system were compared with the measured data from the field survey to verify the performance of the model in estimating the tree structure outline and predicting the geographical location of street trees. Through the field survey, we used a telescope to measure the tree height and crown width of 100 trees in the study area and marked the geographical coordinates of each tree in Google Earth. A regression function was established between the predicted tree height and crown width and the corresponding true values ​​to evaluate the performance of the system. In the estimation of tree height and crown width, the root mean square error (RMSE) of tree height was 0.68m ( Fig. 9 ), the root mean square error (RMSE) of the crown width is 0.64m ( Fig.10 ), showing the high level of accuracy in tree size measurement. At the same time, the distance error d between the predicted tree position and the actual position is calculated, and a histogram is plotted to observe the distribution range of the error d ( Fig.11 ). In the tree position estimation, the mean value of the error d is 4.76m and the standard deviation (SD) is 2.08m, indicating that the geolocation accuracy of the model is high.

Claims

1. A three-dimensional modeling and positioning method for roadside trees based on binocular vision, characterized in that: The steps include the following: Step 1), collecting basic pictures, disparity images and GNSS data of the study area through vehicle-mounted equipment; Step 2) Identify the collected basic image data according to the target detection algorithm and calculate the absolute depth within the detection frame; Step 3), combining the detection frame coordinates and the absolute depth to obtain the geographic coordinates of the street tree; Step 4) Based on the principle of perspective projection model, the crown width and tree height of the street trees are obtained.

2. The method for three-dimensional modeling and positioning of roadside trees based on binocular vision according to claim 1 is characterized in that: The vehicle-mounted equipment includes a binocular depth camera and a GNSS recorder; while collecting basic images, the binocular depth camera calculates in real time the disparity image data corresponding to the viewing angles of the left and right depth cameras; the GNSS recorder collects the coordinates and driving heading angle of the vehicle once per second during the vehicle's driving; when the time when the binocular depth camera collects data is equal to the time when the GNSS recorder collects data, the two types of data are matched to obtain the basic image, disparity image and GNSS data of the coordinate point.

3. The method for three-dimensional modeling and positioning of roadside trees based on binocular vision according to claim 2 is characterized in that: The target detection algorithm is a YOLOv5 detection algorithm, which detects and identifies the street trees in the basic image, extracts the coordinates of the detection frame, and calculates the absolute depth mean in the detection frame; the optimal weight obtained by the street tree detection training is loaded into the YOLOv5 target detection program, and the basic image is input into the target detection program to obtain the pixel coordinates of the street tree target detection frame and the length and width pixel values ​​of the target detection frame, and further calculates the pixel coordinates of the midpoint of the detection frame area; the disparity image data of step 1 is passed through the semi-global block matching algorithm SGBM to obtain the absolute depth of the corresponding image; when the midpoint pixel coordinates of all detection frames in the image and the absolute depth at the same coordinates are known, the absolute depth mean of the pixels around the midpoint is calculated respectively as the distance d between the street tree and the optical center.

4. The method for three-dimensional modeling and positioning of roadside trees based on binocular vision according to claim 3 is characterized in that: The geographic coordinates of the roadside tree are determined by detecting the distance and heading angle between the roadside tree and the optical center. The imaging principle of the binocular camera is simplified to a perspective projection model. In the perspective projection model, the proportional relationship between the imaging coordinates and the spatial coordinates is expressed by trigonometric functions. The angle of the point relative to the optical center is calculated from the coordinates of the point on the imaging plane and the focal length of the camera by inverse trigonometric functions. When the focal length of the camera is known, the angle α between the coordinates of the point in the detection frame and the optical center is calculated by formula (1). The angle α is superimposed on the driving heading angle of the vehicle to finally obtain the heading angle θ of the roadside tree. Where f is the focal length of the camera; u x is the horizontal coordinate of the tree on the image u0 is the horizontal coordinate of the main point of the image; Given the distance d to the roadside tree, the heading angle θ of the roadside tree, the latitude coordinate lat1 of the vehicle, and the longitude coordinate lon1 of the vehicle; assuming that the radius of the earth is R, Ad is the angular distance, i.e. d / R; the latitude coordinate lat2 and longitude coordinate lon2 of the roadside tree are obtained by the following formula, in radians: lat2=arcsin(sin(lat1)cos(Ad)+cos(lat1)sin(Ad)cos(θ)) (2) lon2=lon1+arctan(sin(θ)sin(Ad)cos(lat1),cos(Ad)-sin(lat1)sin(lat2)) (3).

5. The method for three-dimensional modeling and positioning of roadside trees based on binocular vision according to claim 4 is characterized in that: According to the principle of perspective projection model, the crown width and tree height of the street trees are calculated using the following formula: Among them, h is the actual height of the roadside tree, w is the actual crown width of the roadside tree, v is the image distance, d is the object distance, and h sensor is the height of the object in the image, w sensor is the width of the object in the image; the image distance v is calculated by the camera focal length and sensor size; for digital cameras, the digital camera focal length f replaces the image distance.

6. A three-dimensional modeling and positioning system for roadside trees based on binocular vision, characterized in that: include: Data matching module, used to match different types of data; The data matching module matches and integrates the collected different types of basic pictures, disparity images and GNSS data according to a unified timestamp; The matching data of the data matching module is used for data processing in subsequent modules; The object detection module is used to detect street trees in the base image and return the detection box information; The target detection module is equipped with a YOLOv5 detection model, returns the pixel coordinate values ​​of the detection frame, and further calculates the length and width pixel values ​​of the detection frame and the coordinates of the midpoint of the detection frame pixels; The target detection module sends the data to the subsequent data processing module, the midpoint coordinates of the detection frame pixels are used in the absolute depth calculation module, and the length and width pixel values ​​of the detection frame are used in the calculation of the tree size estimation module; Absolute depth calculation module, used to calculate the absolute depth value of the target; The absolute depth calculation module obtains the absolute depth of the image by using the semi-global block matching algorithm SGBM to obtain the disparity data, and further calculates the absolute depth mean of the coordinates near the midpoint of the detection frame pixel; The absolute depth calculation module sends the absolute depth data to the coordinate calculation module; A coordinate calculation module is used to calculate the longitude and latitude coordinates of street trees; Tree size calculation module, used to calculate the crown width and tree height of street trees; Data integration module, used to handle data integration; The data integration module processes the data and integrates them into a list, which records the latitude, longitude, crown width, and tree height information of each street tree line by line and saves them as a .csv file.

Images