Tree structure data identification method, device and equipment and storage medium

Abstract

The invention relates to the field, in particular to a tree structure data identification method and device, equipment and a storage medium. The method comprises the following steps: firstly, identifying pixel point coordinate data of a tree on a streetscape image, then identifying depth data of the tree, and obtaining an actual distance between the tree and acquisition equipment of the streetscape image based on the depth data; and finally, obtaining the structure data of the tree according to the actual distance and the pixel point coordinate data. The structure data of the tree is calculated based on the image processing technology, field measurement is not needed, and therefore the measurement efficiency of the structure data of the tree is improved.

Description

Technical Field

[0001] The present invention relates to a field, and in particular to a tree structure data recognition method, device, equipment and storage medium. Background Art

[0002] Vegetation is closely linked to climate change. As a key element in the ecosystem, trees have a significant mitigating effect on the urban heat island effect. Through their unique ecological regulation function, trees can absorb and store solar energy, reduce the surface temperature of the city, and mitigate the heat island effect. Trees can also release water through transpiration. This process not only promotes water circulation, but also affects the temperature and humidity conditions in urban areas, helping to create a more pleasant urban microclimate. In addition, trees can block solar radiation and reduce heat accumulation on the surface of the city. This physical barrier effect is crucial to reducing urban temperature fluctuations and maintaining a stable and comfortable thermal environment, and plays an important role in improving the health and comfort of residents. The existing technology uses manual measurement to obtain tree structural data, but manual measurement has the characteristic of low measurement efficiency.

[0003] In summary, the measurement method used in the prior art has the technical defect of low measurement efficiency.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a tree structure data recognition method, device, equipment and storage medium, which solves the problem of low measurement efficiency of the measurement method used in the prior art.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for identifying tree structure data, comprising:

[0008] Identify the pixel coordinate data of trees on street view images;

[0009] Identifying the depth data of the tree, and obtaining the actual distance between the tree and the device for collecting the street view image based on the depth data;

[0010] The structural data of the tree is obtained according to the actual distance and the pixel point coordinate data.

[0011] In one implementation, the step of identifying pixel coordinate data of trees on a street view image includes:

[0012] Acquire an original street view image, and delete a bottom image block of the original street view image to obtain a street view image;

[0013] An instance segmentation model is applied to the street view image to segment the trees from the street view image, and the lowest point coordinates, crown edge coordinates, boundary line coordinates and top coordinates of the trees are read, where the boundary line is the boundary line between the crown and the trunk of the tree, and the lowest point coordinates, the crown edge coordinates and the boundary line coordinates are used as pixel point coordinate data of the tree on the street view image.

[0014] In one implementation, identifying the depth data of the tree and obtaining the actual distance between the tree and the device collecting the street view image based on the depth data includes:

[0015] Applying a monocular depth estimation model to the street view image to obtain a depth value of the tree, and using the depth value as depth data of the tree;

[0016] The calibration parameters of the acquisition device are obtained, and the actual distance between the tree and the acquisition device is obtained according to the calibration parameters and the depth data.

[0017] In one implementation, obtaining the structural data of the tree according to the actual distance and the pixel point coordinate data includes:

[0018] Acquire a first total number of pixels of the street view original image in a height direction, acquire a second total number of pixels of the street view image in a height direction, and acquire an angle of the street view image in a vertical direction;

[0019] Determine a vertical offset angle of the top of the tree relative to the height plane where the acquisition device is located according to the first total number of pixels, the second total number of pixels, the angle of the street view image in the vertical direction, and the top coordinates;

[0020] Determining a top conversion factor corresponding to the top of the tree according to the vertical offset angle;

[0021] The height of the acquisition device is obtained, and the height of the tree is determined according to the height of the acquisition device, the top conversion factor, the vertical offset angle, and the actual distance, and the tree height is used as the structural data of the tree.

[0022] In one implementation, obtaining the structural data of the tree according to the actual distance and the pixel point coordinate data includes:

[0023] Acquire a first total number of pixels of the street view original image in a height direction, acquire a second total number of pixels of the street view image in a height direction, and acquire an angle of the street view image in a vertical direction;

[0024] Determine the ordinate of the dividing line in the dividing line coordinates, and determine the angle between the horizontal line where the acquisition device is located and the dividing line according to the first total number of pixels, the second total number of pixels, and the angle of the street view image in the vertical direction;

[0025] Determine the conversion factor corresponding to the dividing line according to the angle;

[0026] The height of the acquisition device is obtained, and the height under the crown of the tree is determined according to the height of the acquisition device, the conversion factor corresponding to the dividing line, the angle, and the actual distance, and the height under the crown is used as the structural data of the tree.

[0027] In one implementation, obtaining the structural data of the tree according to the actual distance and the pixel point coordinate data includes:

[0028] Determine the horizontal coordinates of two edges in the crown edge coordinates;

[0029] Obtaining the total number of pixels of the street view image in the width direction, and determining the horizontal offset angle of the tree crown edge relative to the street view image according to the total number and the horizontal coordinates of the two edges;

[0030] Determining a conversion factor corresponding to the edge according to the horizontal offset angle;

[0031] The crown width of the tree is obtained according to the actual distance, the horizontal offset angle and the conversion factor corresponding to the edge, and the crown width is used as the structural data of the tree.

[0032] In one implementation, the method further includes:

[0033] Determine a first image and remaining images in the street view image, wherein the first image is an image acquired by the acquisition device at a starting position, and the remaining images are images acquired by the acquisition device at sampling positions located in a moving direction;

[0034] The same target tree included in the first image and the remaining images is identified, and the target geographical location of the target tree is located.

[0035] In a second aspect, an embodiment of the present invention further provides a tree structure data identification device, wherein the device comprises the following components:

[0036] A coordinate recognition module is used to identify the pixel coordinate data of trees on the street view image;

[0037] a distance calculation module, used to identify the depth data of the tree, and obtain the actual distance between the tree and the acquisition device of the street view image based on the depth data;

[0038] The data analysis module is used to obtain the structural data of the tree according to the actual distance and the pixel point coordinate data.

[0039] In a third aspect, an embodiment of the present invention further provides a terminal device, wherein the terminal device includes a memory, a processor, and a tree structure data identification program stored in the memory and executable on the processor, and when the processor executes the tree structure data identification program, the steps of the above-mentioned tree structure data identification method are implemented.

[0040] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a tree structure data identification program is stored. When the tree structure data identification program is executed by a processor, the steps of the tree structure data identification method described above are implemented.

[0041] Beneficial effects: The present invention first identifies the pixel coordinate data of the tree on the street view image, then identifies the depth data of the tree, and based on the depth data, obtains the actual distance between the tree and the street view image acquisition device; finally, according to the actual distance and the pixel coordinate data, obtains the structural data of the tree. The present invention calculates the structural data of the tree based on image processing technology, does not require field measurement, and thus improves the measurement efficiency of the tree structural data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is the overall flow chart of the present invention;

[0043] Figure 2 A flowchart of calculating tree three-dimensional structure data and geographic coordinates in an embodiment of the present invention;

[0044] Figure 3 A tree structure diagram in an embodiment of the present invention;

[0045] Figure 4 A side view of a tree shape detection in an embodiment of the present invention;

[0046] Figure 5 A top view of tree shape detection in an embodiment of the present invention;

[0047] Figure 6 is a schematic diagram of the geographical location of a computing tree in an embodiment of the present invention;

[0048] Figure 7 A structural diagram of a tree structure data recognition device provided by the present invention;

[0049] Figure 8 This is a block diagram of the internal structure principle of the terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following is a clear and complete description of the technical solution of the present invention in combination with the embodiments and the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Studies have found that vegetation is closely linked to climate change. As a key element in the ecosystem, trees have a significant mitigating effect on the urban heat island effect. Through their unique ecological regulation function, trees can absorb and store solar energy, reduce the surface temperature of the city, and mitigate the heat island effect. Trees can also release water through transpiration. This process not only promotes water circulation, but also affects the temperature and humidity conditions in urban areas, helping to create a more pleasant urban microclimate. In addition, trees can block solar radiation and reduce heat accumulation on the surface of the city. This physical barrier effect is crucial to reducing urban temperature fluctuations and maintaining a stable and comfortable thermal environment, and plays an important role in improving the health and comfort of residents. The existing technology uses manual measurement to obtain tree structural data, but manual measurement has the characteristic of low measurement efficiency.

[0052] In order to solve the above technical problems, the present invention provides a tree structure data recognition method, device, equipment and storage medium, which solves the problem of low measurement efficiency of the measurement method used in the prior art.

[0053] The tree structure data recognition method of this embodiment can be applied to a terminal device, which can be a terminal product with image processing function, such as a computer. Figure 1 As shown in , the tree structure data recognition method specifically includes the following steps:

[0054] S100, identifying pixel coordinate data of trees on the street view image;

[0055] S200, identifying the depth data of the tree, and obtaining the actual distance between the tree and the device for collecting the street view image based on the depth data;

[0056] S300, obtaining the structural data of the tree according to the actual distance and the pixel point coordinate data.

[0057] Embodiment 1: In this embodiment, step S100 includes step S101 and step S102:

[0058] S101, obtaining an original street view image, and deleting a bottom image block of the original street view image to obtain a street view image.

[0059] Since the bottom image block does not contain the pixel information of the trees, the bottom image block can be deleted to reduce the image size and thus improve the image processing efficiency. Among them, the original street view image is a panoramic street view image downloaded from Baidu Panorama that covers the entire study area.

[0060] S102, applying an instance segmentation model to the street view image to segment the tree from the street view image, and reading the lowest point coordinates, crown edge coordinates, boundary line coordinates and top coordinates of the tree, where the boundary line is the boundary line between the crown and the trunk of the tree, and using the lowest point coordinates, the crown edge coordinates and the boundary line coordinates as pixel point coordinate data of the tree on the street view image.

[0061] like Figure 2 As shown in , the Mask R-CNN instance segmentation model is applied to the street view image to segment trees from the street view image and characterize the tree with a mask. The Mask R-CNN instance segmentation model can also be used to obtain a single tree mask image and a table file with the name of the corresponding panoramic street view image, the mask code, and the coordinates of the lowest point of the mask. The table file records the coordinates of the lowest point and the edge of the tree crown, as well as the coordinates of the dividing line and the top of the tree. Figure 3 ( Figure 3 The image width direction is the horizontal coordinate, and the image height direction is the vertical coordinate), where the coordinates of the lowest point are (x0, y0), the crown edge coordinates include two edge coordinates, and this embodiment uses x1 and x2 to represent the horizontal coordinates of the two edge coordinates, and uses y1 to represent the vertical coordinate of the top coordinate. The dividing line is a horizontal straight line with a height of y3, and the dividing line is the dividing line between the crown and the trunk. x0, x1, and x2 are all the horizontal coordinates of the image pixel points, and y0, y1, and y3 are all the vertical coordinates of the image pixel points.

[0062] The Mask R-CNN instance segmentation model in this embodiment is a trained model. The training process of the Mask R-CNN instance segmentation model is as follows:

[0063] We implemented Mask R-CNN training on Python and TensorFlow. First, we initialized our model with the pre-trained weights of the COCO dataset, and trained it for 50 generations on images containing trees from the COCO Stuff dataset. Then, we trained it for 100 generations on a detailed urban street tree dataset. Finally, we fine-tuned it on 100 manually labeled panoramic street view images of Shenzhen, trained it layer by layer for 150 generations, and stopped training when the loss function of each round of training became stable.

[0064] Embodiment 2, based on embodiment 1, in this embodiment, step S200 includes the following specific steps S201 and S202:

[0065] S201, applying a monocular depth estimation model to the street view image to obtain a depth value of the tree, and using the depth value as depth data of the tree.

[0066] The monocular depth estimation model in this embodiment is a trained model, and the training method includes: selecting the weight of the best performance of the model pre-trained in the Imagenet dataset and using the KITTI dataset trained model.

[0067] The depth value in this embodiment represents the distance from the shooting point to the target, that is, the distance from the acquisition device to the tree.

[0068] S202, obtaining calibration parameters of the acquisition device, and obtaining the actual distance Z between the tree and the acquisition device based on the calibration parameters and the depth data. tree :

[0069]

[0070] In the formula, is the depth value represented by the depth map of the street view image, B is the baseline of the acquisition device, F is the focal length of the camera of the acquisition device, d is the parallax between all objects including trees in the street view image, and C is the calibration parameter of the acquisition device. C is obtained by randomly selecting street view images, and comparing the average value of the ratio of the distance from the camera to the tree predicted by the actual measurement and the Monodepth2 model (Monodepth2 model is a monocular depth estimation model) as the calibration parameter C.

[0071] Embodiment 3, based on embodiment 1 or embodiment 2, in this embodiment, step S300 includes the following specific steps S301a, S302a, S303a, S304a:

[0072] S301a, obtaining a first total number of pixels of the street view original image in a height direction, obtaining a second total number of pixels H of the street view image in a height direction, and obtaining an angle a of the street view image in a vertical direction.

[0073] S302a, determining a vertical offset angle θ1 (θ1 is a vertical angle θ2) of the top of the tree relative to the height plane where the acquisition device is located, based on the first total number of pixels, the second total number of pixels, the angle a of the street view image in the vertical direction, and the ordinate of the top coordinate. Figure 4 ).

[0074]

[0075] In the formula, y1 represents the ordinate of the top coordinate, k is the total number of pixels in the first The ratio of the total number of pixels H to the second total number of pixels H. For example, removing the bottom of the original street view image image block, then a is equal to 1350 (135°=180°-45°, 180° is 45°), k is

[0076] S303a, determining a top conversion factor C1 corresponding to the top of the tree according to the vertical offset angle θ1.

[0077] C1=cosθ1

[0078] S304a, obtaining the height h' of the acquisition device, and calculating the actual distance Z according to the height h' of the acquisition device, the top conversion factor C1, the vertical offset angle θ1, and the actual distance Z tree , determine the tree height h of the tree tree and use the tree height as the structural data of the tree.

[0079] h tree =D×tanθ1×C1+h′

[0080] D=Z tree ×cosθ0

[0081] D is the horizontal distance between the acquisition device (the acquisition device is the camera) and the tree, θ0 is the vertical offset angle from the height plane where the acquisition device is located to the bottom of the tree, y0 is the ordinate of the lowest point of the tree, which is the ordinate of the pixel point of the lowest point in the street view image.

[0082] Embodiment 4, based on embodiment 1 or embodiment 2 or embodiment 3, in this embodiment, step S300 includes the following specific steps S301b, S302b, S303b, S304b:

[0083] S301b, obtaining a first total number of pixels of the street view original image in a height direction, obtaining a second total number of pixels H of the street view image in a height direction, and obtaining an angle a of the street view image in a vertical direction;

[0084] S302b, determining the vertical coordinate y3 of the dividing line in the dividing line coordinates, and determining the angle θ3 between the horizontal line where the acquisition device is located and the dividing line according to the first total number of pixels, the second total number of pixels H, and the angle a of the street view image in the vertical direction (θ3 is as follows: Figure 4 shown).

[0085] The angle θ3 is also the vertical offset angle of the dividing line relative to the height plane of the acquisition equipment.

[0086]

[0087] k is the total number of pixels The ratio of H to the second total number of pixels.

[0088] S303b: Determine a conversion factor C3 corresponding to the boundary line according to the angle.

[0089] C3=cosθ3

[0090] S304b, obtaining the height h' of the acquisition device, and calculating the actual distance Z according to the height h' of the acquisition device, the conversion factor C3 corresponding to the dividing line, the angle θ3, and the actual distance Z tree , determine the tree's crown height h bole and use the crown height as the structural data of the tree.

[0091] h bole =D×tanθ3×C3+h′

[0092] D=Z tree ×cosθ0

[0093] This embodiment can also be based on the crown height h bole And tree height h tree Calculate the crown height h canopy :

[0094] h canopy =h tree -h bole

[0095] Embodiment 5, based on embodiment 1 or embodiment 2 or embodiment 3 or embodiment 4, in this embodiment, step S300 includes the following specific steps:

[0096] S301c, determining the horizontal coordinates x1 and x2 of the two edges in the crown edge coordinates.

[0097] x1 is the horizontal coordinate of one edge of the crown, and x2 is the horizontal coordinate of the other edge of the crown. One edge of the crown is higher than the other edge.

[0098] S302c, obtaining the total number of pixels W of the street view image in the width direction, and determining the horizontal offset angle Δμ of the tree crown edge relative to the street view image according to the total number W and the horizontal coordinates x1 and x2 of the two edges.

[0099] Δμ=0.5×(μ1-μ2);

[0100]

[0101] like Figure 5 As shown, Δμ is the horizontal offset angle of the crown edge relative to the crown center, μ1 is the horizontal offset angle of one edge of the crown relative to the center of the street view image, and μ2 is the horizontal offset angle of the other edge of the crown relative to the center of the street view image.

[0102] S303c, determining the conversion factor C corresponding to the edge according to the horizontal offset angle Δμ Δμ :

[0103] C Δu =cosΔu

[0104] S304c, based on the actual distance Z tree , the horizontal offset angle Δμ and the conversion factor C corresponding to the edge Δμ , get the crown width w of the tree canopy and use the crown width as the structural data of the tree.

[0105] w canopy =2×D×tanΔμ×C Δμ

[0106] D=Z tree ×cosθ0

[0107] Embodiment 6, based on any one of Embodiments 1 to 5, this embodiment calculates the target geographical location of trees on a street view image, including the following specific steps:

[0108] S401: Determine the collection location of each street view image.

[0109] like Figure 6 As shown, after the collection device (collection device, that is, camera) placed on the vehicle collects the street view image at the collection position S1, the vehicle travels along the traveling direction, and after traveling a certain distance, arrives at the collection position S2. The collection device collects the street view image at the collection position S2, and then continues to travel until it reaches the last collection position. Some of the distances between the collection positions are less than 70 meters.

[0110] The direction of travel in real time is determined as follows:

[0111] The shooting points of two adjacent street view images are used to construct vector segments according to the street view image serial numbers. When the distance between the two points is less than 70m, its direction is the vehicle travel direction of the previous point; for points with a relatively long remaining distance (the distance is greater than 70m), the direction of the vector segment from any of its nearest remaining points to the point whose travel direction has been obtained is calculated. If the difference between this direction and the travel direction of the nearest point is less than or equal to 5°, this direction is used as the vehicle travel direction of the remaining points; the above steps are repeated to determine the vehicle travel direction of all street view images, and points that do not meet the above conditions are eliminated.

[0112] S402, combining the street view images in pairs to obtain groups of image combinations, each group of the image combination comprising a first image and a second image, wherein a first acquisition position of the first image and a second acquisition position of the second image are located on the same acquisition horizontal plane.

[0113] like Figure 6 As shown, the acquisition device acquires street view images at three acquisition positions. If the three street view images are A, B, and C, A and B form an image combination, B and C form an image combination, and A and C form an image combination. And the three acquisition positions are all located on the horizontal plane where the height of the acquisition device is located.

[0114] S403: for each group of the image combinations, determine a first predicted geographical location of vegetation on the first image; and determine a second predicted geographical location of vegetation on the second image.

[0115] The initial position is calculated by using the triangulation method to predict the geographic location of vegetation in each street view image through the coordinates of the shooting point, the distance from the camera to the tree, the relative angle of the tree in the street view image to the camera at the shooting point, and the direction of the vehicle (the predicted geographic location is also Figure 6 The initial positions of the six trees tr11, tr12, tr21, tr22, tr31, and tr32, Figure 6 Not all the sampled trees are marked in the figure. For the convenience of explaining this embodiment, only the initial positions of these six trees are marked).

[0116] S404: Determine a horizontal distance D between the second predicted position and the second acquisition position.

[0117] The vegetation in this embodiment is trees, and the trees are roadside trees. The collection horizontal plane is the horizontal plane where the collection device is located. Through the above step S101, the horizontal distance D between the vegetation and the corresponding collection position on the horizontal plane is calculated, that is, Figure 6 The horizontal distance D between the tree tr11 and the collection position S1 is Figure 6 The horizontal distance D between the predicted position tr21 of the tree and the collected position S2.

[0118] S405, determining an average positioning error c0 of the acquisition device and a maximum positioning error c1 of a monocular depth estimation model in the positioning error data, wherein the monocular depth estimation model is used to extract a depth value of the street view image, and the depth value is used to calculate the horizontal distance.

[0119] The error of the acquisition equipment mainly comes from the positioning error of the positioning device on the vehicle supporting the acquisition equipment, so the acquisition equipment will cause certain errors in the calculation of the horizontal distance D. When calculating the horizontal distance D between the vegetation and the acquisition equipment, the depth value of the image is used, and the depth value of the image is calculated by the monocular depth estimation model. Therefore, the monocular depth estimation model will also cause certain errors in the calculation of the horizontal distance D.

[0120] S406: Determine a horizontal expansion distance L according to the maximum positioning error c1, the average positioning error c0, and the horizontal distance D:

[0121] L = D + c0 + c1 × D;

[0122] Wherein, c0+c1×D is the buffer threshold after considering the error, the value of c1 can be 0.66, and the value of c0 can be 3 meters.

[0123] S407, determining a first straight line formed by the first predicted position and the first collected position; determining a second straight line formed by the second predicted geographical position and the second collected position; and determining an intersection point between the first straight line and the second straight line;

[0124] S408: Filter out candidate intersection points from the intersection points according to the horizontal expansion distance and the position difference between the position of the intersection point and the second predicted position.

[0125] The distance between the alternative intersection point and the second collection position is smaller than the horizontal expansion distance. Since the horizontal distance D is the distance between the second predicted position and the second collection position, and the horizontal expansion distance L is a distance extended on the basis of the horizontal distance D, when the distance between the alternative intersection point and the second collection position is smaller than the horizontal expansion distance L, the alternative intersection point is located within the line segment where the horizontal expansion distance starting from the second collection position is located.

[0126] The candidate intersection point must not only be located within the horizontal expansion distance, but also be the intersection point closest to the second predicted position among all the intersection points.

[0127] S409: Obtain vegetation target position data according to the candidate intersection points corresponding to each group of the image combinations.

[0128] A candidate intersection point can be screened out through the two predicted geographic locations on the two street view images in each group of images. Several candidate intersection points can be obtained from all image combinations. By clustering several candidate intersection points, vegetation target location data can be obtained. In other words, the predicted geographic locations obtained before are not the actual geographic locations of the vegetation. By clustering the intersection points, vegetation target location data that is closer to the actual geographic location can be obtained.

[0129] For example, Figure 6 Taking steps S401 to S409 as an example,

[0130] The camera (collection device) is placed on the vehicle. The vehicle passes through the collection position S1, collection position S2, collection position S3 until the Nth collection position SN. The shooting direction of the camera is perpendicular to the direction of the vehicle's travel. Figure 3 The angle of the tree position corresponds to 0° Figure 3 The angle corresponding to the position of the trees in the image.

[0131] A street view image is collected at each collection position, for example, street view image A is collected at collection position S1, street view image B is collected at collection position S2, and street view image C is collected at collection position S3.

[0132] By combining street view image A, street view image B, and street view image C in pairs, we obtain image combination A-B, image combination A-C, and image combination B-C.

[0133] Taking image combination A and B as an example, street view image A in the combination (street view image collected at collection position S1) is defined as the first image, and street view image B in the combination (street view image collected at collection position S2) is defined as the second image.

[0134] Identify street view image A, obtain predicted location tr11 and predicted location tr12 of vegetation (in image combination A and B, tr11 and tr12 are first predicted geographical locations), identify street view image B, obtain predicted location tr21 and predicted location tr22 of vegetation (in image combination A and B, predicted location tr21 and predicted location tr22 are second predicted geographical locations).

[0135] Connect the predicted position tr11 with the collected position S1 to get the straight line l 11 , the predicted position tr12 is connected with S1 to obtain the straight line l 12 , straight line l 11 and straight line l 12 That is, the first straight line corresponding to the image combination A and B; the predicted position tr21 is connected with the acquisition position S2 to obtain the straight line l 21 , connect the predicted position tr22 with the acquisition position S2 to get the straight line l 22, straight line l 21 and straight line l 22 That is the second straight line corresponding to the image combination A and B.

[0136] Straight line 21 With straight line l 11 , straight line l 12 There are intersection points O211 and O212 respectively. Since the intersection point O212 is closest to the predicted position tr21, if the distance between the intersection point O212 and the acquisition position S2 is less than the horizontal expansion distance L (the horizontal expansion distance L is the horizontal expansion distance corresponding to the predicted position tr21 and the acquisition position S2), then the intersection point O212 is the candidate intersection point corresponding to the image combination A and B. 22 With straight line l 11 , straight line l 12 There is also an intersection point, but this intersection point is not the closest intersection point of the line connecting S3 and S2 for the same tree, so this intersection point cannot be used as an alternative intersection point.

[0137] Similarly, for image combination B and C, a line is established between the predicted position tr22 and the acquisition position S2, and a straight line l is obtained. 22 ; Establish a line between the predicted position tr21 and the acquisition position S2 to obtain a straight line l 21 ; Establish a line between the predicted position tr32 and the acquisition position S3 to obtain a straight line l 32 Straight line l 32 With straight line l 22 , straight line l 21 There are intersection points O321 and O322 respectively. Since the intersection point O322 is the closest to the predicted position tr32 (the predicted position tr32 is located on the straight line l) of the two intersection points O321 and O322, 32 ) recently, if the distance between the intersection O322 and the acquisition position S3 is less than the horizontal expansion distance L (the horizontal expansion distance L is the horizontal expansion distance corresponding to the predicted position tr32 and the acquisition position S3), then the intersection O322 is the alternative intersection corresponding to the image combination B and C.

[0138] Using the same sampling method, the intersection point O132 is selected from the intersection point O131 and the intersection point O132 as the candidate intersection point, and the candidate intersection point corresponding to the image combination A and C is obtained.

[0139] By clustering the intersection points O212, O322, and O132 (the clustering distance is 3 meters), the vegetation target location data is obtained, that is, Figure 6The location of the box in the figure can be considered to have vegetation. The tree height, crown height and crown diameter of the identified street trees are removed by using the interquartile range method and the mean is calculated to obtain the final tree structure parameters.

[0140] This embodiment also provides a tree structure data recognition device, such as Figure 7 As shown, the device comprises the following components:

[0141] Coordinate recognition module 01, used to recognize the pixel coordinate data of trees on the street view image;

[0142] A distance calculation module 02, used to identify the depth data of the tree, and obtain the actual distance between the tree and the acquisition device of the street view image based on the depth data;

[0143] The data analysis module 03 is used to obtain the structural data of the tree according to the actual distance and the pixel point coordinate data.

[0144] Based on the above embodiments, the present invention further provides a terminal device, whose principle block diagram can be shown as follows: Figure 8 As shown. The terminal device includes a processor, a memory, a network interface, and a display screen connected through a system bus. Among them, the processor of the terminal device is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a tree structure data recognition method is implemented. The display screen of the terminal device can be a liquid crystal display screen or an electronic ink display screen.

[0145] Those skilled in the art will understand that Figure 8 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal device to which the scheme of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0146] In one embodiment, a terminal device is provided. The terminal device includes a memory, a processor, and a tree structure data recognition program stored in the memory and executable on the processor. When the processor executes the tree structure data recognition program, the following operation instructions are implemented:

[0147] Identify the pixel coordinate data of trees on street view images;

[0148] Identifying the depth data of the tree, and obtaining the actual distance between the tree and the device for collecting the street view image based on the depth data;

[0149] The structural data of the tree is obtained according to the actual distance and the pixel point coordinate data.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tree structure data recognition method, characterized in that: include: Identify the pixel coordinate data of trees on street view images; Identifying the depth data of the tree, and obtaining the actual distance between the tree and the device for collecting the street view image based on the depth data; The structural data of the tree is obtained according to the actual distance and the pixel point coordinate data.

2. The tree structure data recognition method according to claim 1, characterized in that: The step of identifying pixel coordinate data of trees on the street view image includes: Acquire an original street view image, and delete a bottom image block of the original street view image to obtain a street view image; An instance segmentation model is applied to the street view image to segment the trees from the street view image, and the lowest point coordinates, crown edge coordinates, boundary line coordinates and top coordinates of the trees are read, where the boundary line is the boundary line between the crown and the trunk of the tree, and the lowest point coordinates, the crown edge coordinates and the boundary line coordinates are used as pixel point coordinate data of the tree on the street view image.

3. The tree structure data recognition method according to claim 1, characterized in that: Identifying the depth data of the tree, and obtaining the actual distance between the tree and the device collecting the street view image based on the depth data, comprising: Applying a monocular depth estimation model to the street view image to obtain a depth value of the tree, and using the depth value as depth data of the tree; The calibration parameters of the acquisition device are obtained, and the actual distance between the tree and the acquisition device is obtained according to the calibration parameters and the depth data.

4. The tree structure data recognition method according to claim 2, characterized in that: According to the actual distance and the pixel point coordinate data, the structural data of the tree is obtained, including: Acquire a first total number of pixels of the street view original image in a height direction, acquire a second total number of pixels of the street view image in a height direction, and acquire an angle of the street view image in a vertical direction; Determine a vertical offset angle of the top of the tree relative to the height plane where the acquisition device is located according to the first total number of pixels, the second total number of pixels, the angle of the street view image in the vertical direction, and the top coordinates; Determining a top conversion factor corresponding to the top of the tree according to the vertical offset angle; The height of the acquisition device is obtained, and the height of the tree is determined according to the height of the acquisition device, the top conversion factor, the vertical offset angle, and the actual distance, and the tree height is used as the structural data of the tree.

5. The tree structure data recognition method according to claim 2, characterized in that: According to the actual distance and the pixel point coordinate data, the structural data of the tree is obtained, including: Acquire a first total number of pixels of the street view original image in a height direction, acquire a second total number of pixels of the street view image in a height direction, and acquire an angle of the street view image in a vertical direction; Determine the ordinate of the dividing line in the dividing line coordinates, and determine the angle between the horizontal line where the acquisition device is located and the dividing line according to the first total number of pixels, the second total number of pixels, and the angle of the street view image in the vertical direction; Determine the conversion factor corresponding to the dividing line according to the angle; The height of the acquisition device is obtained, and the height under the crown of the tree is determined according to the height of the acquisition device, the conversion factor corresponding to the dividing line, the angle, and the actual distance, and the height under the crown is used as the structural data of the tree.

6. The tree structure data recognition method according to claim 2, characterized in that: According to the actual distance and the pixel point coordinate data, the structural data of the tree is obtained, including: Determine the horizontal coordinates of two edges in the crown edge coordinates; Obtaining the total number of pixels of the street view image in the width direction, and determining the horizontal offset angle of the tree crown edge relative to the street view image according to the total number and the horizontal coordinates of the two edges; Determining a conversion factor corresponding to the edge according to the horizontal offset angle; The crown width of the tree is obtained according to the actual distance, the horizontal offset angle and the conversion factor corresponding to the edge, and the crown width is used as the structural data of the tree.

7. The tree structure data recognition method according to claim 1, characterized in that: Also includes: Determine a first image and remaining images in the street view image, wherein the first image is an image acquired by the acquisition device at a starting position, and the remaining images are images acquired by the acquisition device at sampling positions located in a moving direction; The same target tree included in the first image and the remaining images is identified, and the target geographical location of the target tree is located.

8. A tree structure data recognition device, characterized in that: The device comprises the following components: A coordinate recognition module is used to identify the pixel coordinate data of trees on the street view image; a distance calculation module, used to identify the depth data of the tree, and obtain the actual distance between the tree and the acquisition device of the street view image based on the depth data; The data analysis module is used to obtain the structural data of the tree according to the actual distance and the pixel point coordinate data.

9. A terminal device, characterized in that: The terminal device includes a memory, a processor, and a tree structure data recognition program stored in the memory and executable on the processor. When the processor executes the tree structure data recognition program, the steps of the tree structure data recognition method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a tree structure data recognition program, and when the tree structure data recognition program is executed by the processor, the steps of the tree structure data recognition method according to any one of claims 1 to 7 are implemented.

Images