Method for automatically obtaining city street tree geometry and geographic information based on street view data

Abstract

The application discloses a kind of based on street view data's urban boulevard tree geometry and geographic information automatic acquisition method, belong to artificial intelligence technical field, the method includes: obtaining the street view image data of target area;Wherein, street view image data refers to the street view image containing the boulevard tree in target area, and each boulevard tree corresponds multiple street view images of different perspectives;Tree contour shape image is extracted from the street view image data by preset tree detection model;Based on tree contour shape image, calculate tree pixel size information;Based on the multiple street view images of different perspectives corresponding to each boulevard tree, the tree position information is obtained by calculation;Based on tree pixel size information and tree position information, the real size information of tree is corrected.Compared with traditional artificial investigation and record method, the application greatly reduces the calculation time, improves the efficiency of boulevard tree information acquisition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a method for automatically obtaining city street tree geometry and geographic information based on street view data. BACKGROUND

[0002] With the acceleration of urbanization and the deepening of the concept of green city, the management and acquisition of city street tree information play an increasingly important role in the sustainable development and management of the city. In view of the large number of city street trees, the traditional manual investigation and recording method can no longer meet the efficient and accurate management needs. Therefore, it is particularly urgent to explore the automatic acquisition method of street tree geometry and geographic information.

[0003] In recent years, street view images are rapidly emerging as an important data source for geographic spatial data collection and urban environment analysis. Street view images not only contain city roads and their affiliated elements, but also record the existence of street trees in almost every street view photo, which makes street view images an important medium for obtaining city street tree information. Therefore, studying how to automatically extract street tree information from street view images has significant scientific value and application prospect for improving the scientific nature of city management and planning.

[0004] However, there is currently no solution to automatically acquire street tree geometry and geographic information through street view data. SUMMARY

[0005] The present application provides a method for automatically obtaining city street tree geometry and geographic information based on street view data, to solve the technical problem that the traditional manual investigation and recording method cannot meet the efficient and accurate management needs of city street tree geometry and geographic information.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] On the one hand, the present application provides a method for automatically obtaining city street tree geometry and geographic information based on street view data, which comprises:

[0008] Obtaining street view image data of a target area; wherein the street view image data refers to street view images containing street trees in the target area, and each street tree corresponds to multiple street view images of different perspectives;

[0009] Extracting tree contour shape images from the street view image data through a preset tree detection model;

[0010] Calculating tree pixel size information based on the tree contour shape images;

[0011] Based on multiple street view images from different perspectives corresponding to each row of trees, the location information of the trees is calculated.

[0012] The actual size information of trees is corrected based on tree pixel size information and tree location information.

[0013] Furthermore, acquiring street view image data of the target area includes:

[0014] Based on the principle that roads do not have corners, the roads in the target area are divided into multiple road segment units;

[0015] Sampling points were set up at preset intervals in each road segment unit;

[0016] The system acquires multiple street view images from different perspectives for each sampling point based on a preset street view image system.

[0017] Further, based on the tree outline shape image, the tree pixel size information is calculated, including:

[0018] Based on the tree outline shape image, obtain a grayscale image of the tree instance;

[0019] Based on the grayscale images of tree instances, Gaussian distribution and region growing algorithms are used to obtain the pixel size information of trees.

[0020] Further, based on the tree outline shape image, a grayscale image of a tree instance is obtained, including:

[0021] The tree outline shape image is converted to grayscale to obtain a grayscale image of the tree outline shape.

[0022] Based on the grayscale image, individual tree grayscale images of different trees are selected by comparing grayscale values.

[0023] Based on the grayscale image of a single tree, a two-dimensional moving average is used to obtain the blurred boundaries of the trees in the grayscale image of a single tree, and the blurred boundaries of the trees are corrected by setting a threshold to obtain the grayscale image of the tree instance.

[0024] Furthermore, the step of obtaining tree pixel size information based on the grayscale image of tree instances using Gaussian distribution and region growing algorithms includes:

[0025] The row vector X and column vector Y of the grayscale image of the tree instance are obtained through projection processing;

[0026] Based on the row vector X and column vector Y of the grayscale image of the tree instance, the tree pixel height, trunk pixel width, crown pixel width, and crown pixel height of the corresponding tree are calculated through geometric relationships; among them,

[0027] The formula for calculating the pixel height of a tree is:

[0028] h = max{X i}

[0029] wherein, h represents the tree pixel height; X i represents the sum of the number of tree pixel points in the i-th column;

[0030] The calculation formula of the trunk pixel width is:

[0031] d = Y r-0.22h

[0032] wherein, d represents the trunk pixel width; r represents the Y vector position of the tree root; Y r-0.22h represents the sum of the number of tree pixel points from the element of the Y vector representing the tree root to the row at 0.22h in the negative direction of the y axis, i.e. the crown direction;

[0033] The calculation formula of the crown pixel width is:

[0034] gd = max{Y j}

[0035] wherein, gd represents the crown pixel width; Y j represents the sum of the number of tree pixel points in the j-th row;

[0036] The tree instance gray scale image is converted into a binary image, and the binary image is combined with the street view image data to extract a single tree image corresponding to the corresponding tree from the street view image data;

[0037] Based on the Gaussian distribution algorithm, the probability that each pixel point in the single tree image belongs to the crown and the trunk is obtained; for the discrete points existing in the distribution, a region growing algorithm is adopted, the trunk region pixel points are selected as the starting points, the trunk part pixel set is obtained, the remaining pixels are taken as the crown part pixel set, and the crown pixel height is obtained:

[0038] gh = max{X crown,i}

[0039] wherein, gh represents the crown pixel height; X crown,i represents the sum of the number of crown pixel points in the i-th row.

[0040] Further, the Gaussian distribution algorithm is based on, the probability that each pixel point in the single tree image belongs to the crown and the trunk, comprising:

[0041] A plurality of sample pictures corresponding to each type of tree are collected, and the crown region and the trunk region are labeled for each sample picture, and then the RGB values of the crown region and the trunk region are obtained;

[0042] Gaussian distribution fitting is performed on the RGB values of the crown region and the trunk region respectively to obtain a probability density function corresponding to the RGB values of the crown region and a probability density function corresponding to the RGB values of the trunk region;

[0043] Based on the probability density function corresponding to the RGB values of the crown region and the probability density function corresponding to the RGB values of the trunk region, the probability of each pixel point in the single tree image belonging to the crown and the trunk is obtained.

[0044] Further, the tree position information is calculated based on the multiple street view images of different perspectives corresponding to each street tree, including:

[0045] The same tree matching is performed based on the multiple street view images of different perspectives corresponding to each street tree.

[0046] According to the matching result, the tree position information is calculated by using a triangulation method.

[0047] Further, the same tree matching is performed based on the multiple street view images of different perspectives corresponding to each street tree, including:

[0048] The similarity of the trees in the two different street view images is calculated, and the formula is:

[0049]

[0050] Wherein, S represents the similarity of the trees in the two different street view images; Ah represents the difference of the tree pixel height of the trees in the two different street view images; Ag represents the difference of the crown pixel height of the trees in the two different street view images; Ad represents the difference of the trunk pixel width of the trees in the two different street view images; Agd represents the difference of the crown pixel width of the trees in the two different street view images.

[0051] The trees with the highest similarity in the two street view images are matched, and they are identified as the same tree.

[0052] Further, according to the matching result, the tree position information is calculated by using a triangulation method, including:

[0053] The azimuth angle of the tree and the sampling point is calculated, and the formula is:

[0054]

[0055] Wherein, sita represents the azimuth angle of the tree and the sampling point; Heading represents the orientation angle of the street view image where the tree is located; W represents the width of the street view image, and Δ represents the pixel distance between the trunk position and the midpoint of the street view image.

[0056] Based on the known coordinates of two sampling points and a triangular region formed by a street tree, the tree position information is calculated, and the formula is:

[0057]

[0058] wherein x t , y t are the coordinate values of the tree, x1, y1 are the coordinate values of one of the sampling points in the triangular region, x2, y2 are the coordinate values of the other sampling point in the triangular region; θ1 and θ2 represent the azimuth angles between the tree and the two sampling points in the triangular region, respectively.

[0059] Further, the tree real size information is corrected based on the tree pixel size information and the tree position information, including:

[0060] The distance between the tree in the real world and the sampling point is calculated;

[0061] The field of view angle occupied by the tree is calculated based on the azimuth angle between the tree and the sampling point;

[0062] The tree real size information is corrected by the distance between the tree and the sampling point and the field of view angle occupied by the tree, to obtain the real crown width, the real tree height, the real crown height and the real trunk width; wherein,

[0063] The calculation formula of the real crown width is:

[0064] GD=L×angle

[0065] wherein GD represents the real crown width; L represents the distance between the tree in the real world and the sampling point; angle represents the field of view angle occupied by the tree;

[0066] The calculation formula of the real tree height is:

[0067]

[0068] H=e×h

[0069] wherein gd represents the pixel width of the crown; h represents the pixel height of the tree; H represents the real tree height;

[0070] The calculation formula of the real crown height is:

[0071] GH=e×gh

[0072] wherein GH represents the real crown height; gh represents the pixel height of the crown;

[0073] The calculation formula of the real trunk width is:

[0074] D=e×d

[0075] wherein, D represents the trunk real width; d represents the trunk pixel width.

[0076] In yet another aspect, the present application also provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above method.

[0077] In yet another aspect, the present application also provides a computer readable storage medium, which stores at least one instruction, which is loaded and executed by the processor to implement the above method.

[0078] The technical solution provided by the present application has at least the following beneficial effects:

[0079] The technical solution of the present application makes full use of the low cost and universality of street view images, making the preliminary preparation work of obtaining street tree information more convenient. Compared with the traditional manual investigation and recording method, this method greatly reduces the calculation time and improves the efficiency of obtaining street tree information, providing scientific and reasonable technical support for the management and statistics of street trees, and also providing strong support for urban greening planning, ecological environment monitoring, public safety assessment and other aspects, which is helpful to promote the sustainable development of cities and has great application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0081] Figure 1 is the execution flow diagram of the method for automatically obtaining city street tree geometry and geographic information based on street view data provided by the embodiment of the present application;

[0082] Figure 2 is a street view image and its tree image segmentation result schematic diagram provided by the embodiment of the present application;

[0083] Figure 3 is a tree crown and trunk segmentation result schematic diagram based on Gaussian distribution and region growing algorithm provided by the embodiment of the present application;

[0084] Figure 4 is a system block diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to make the objects, technical solutions and advantages of the present application clearer, the following further describes the embodiments of the present application with reference to the drawings.

[0086] First, it should be noted that in the embodiments of the present application, the words such as "exemplarily", "for example" are used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplary" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "exemplarily" is intended to present the concept in a specific manner. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0087] First embodiment

[0088] The present embodiment provides a method for automatically obtaining city street tree geometry and geographic information based on street view data, which is used to realize automatic acquisition of street tree information of a specified street view. The method can be realized by an electronic device, which can be a terminal or a server. The execution flow of the method is shown in Figure 1 The method comprises the following steps:

[0089] S1, obtaining street view image data of a target area; wherein the street view image data refers to street view images containing street trees in the target area, and each street tree corresponds to multiple street view images of different perspectives;

[0090] Specifically, in the present embodiment, the implementation process of S1 comprises:

[0091] S11, dividing the roads of the target area into a plurality of road segment units according to the principle that there is no corner in the road;

[0092] S12, arranging sampling points at a fixed interval along the center of each road segment unit;

[0093] S13, obtaining multiple street view images of different perspectives for each sampling point based on a preset street view system;

[0094] In the present embodiment, the street view data of the target area is obtained by using Baidu Street View API. The fixed interval of the arrangement of the sampling points used in the present embodiment is 10m. The street view control parameters set in the present embodiment are width=1024, Height=512, Heading=0, 90, 180, 270, and Fov=90.

[0095] S2, extracting a tree contour shape image from the street view image data by using a preset tree detection model;

[0096] Specifically, the tree detection model used in the embodiment is composed of a tree recognition module and a tree segmentation module: the tree recognition module is used to recognize trees in a street view image to obtain a rectangular frame of the trees in the street view image; and the tree segmentation module is used to segment out a contour shape image of all the trees from the rectangular frame of the trees obtained by the tree recognition module. The training process of the tree recognition module includes:

[0097] Step 1: Collect public plant images from the Plant Science Data Center and street views of part of a certain city;

[0098] Step 2: Perform semantic annotation on the collected images to form a training data set;

[0099] Step 3: Train the tree recognition module using the training data set.

[0100] Specifically, in the embodiment, the network architecture of the tree recognition module adopts a SwinL network architecture based on Transformer, and the tree segmentation module adopts a basic large model Segment Anything Model in the field of visual segmentation. Figure 2 A street view image and its tree image segmentation result schematic diagram provided by the embodiment are shown. Figure 2 As can be seen, the trees in the figure are effectively recognized.

[0101] S3, based on the tree contour shape image, calculate the tree pixel size information;

[0102] Specifically, in the embodiment, the implementation process of S3 includes:

[0103] S31, based on the tree contour shape image, obtain a tree instance gray image; the specific implementation is as follows:

[0104] S311, perform gray scale processing on the tree contour shape image to obtain a gray image of the tree contour shape image;

[0105] S312, based on the gray image, filter out single tree gray images of different trees through gray value comparison;

[0106] S313, based on the single tree gray image, obtain a fuzzy boundary of the single tree gray image by two-dimensional moving average, and correct the boundary by setting a threshold to obtain a tree instance gray image;

[0107] It should be noted that due to the gray scale mixing of the original drawing, the extracted single tree gray scale drawing has missing information on the tree instance boundary, which is not conducive to subsequent calculation, and therefore the boundary needs to be supplemented and corrected to obtain a more accurate boundary profile. The threshold correction of the fuzzy boundary of the tree refers to correcting the gray scale value of the tree boundary pixel point. Here, the threshold is set to the background gray scale value, such as 30. If the gray scale value of the pixel point is greater than the background gray scale value, the pixel point is identified as the gray scale of the tree. The tree boundary is corrected by this method.

[0108] S32, based on the tree instance gray scale drawing, the Gaussian distribution and region growing algorithm are used to obtain the tree pixel size information; the specific implementation is as follows:

[0109] S321, the row vector X and the column vector Y of the tree instance gray scale drawing are obtained by projection processing; the specific process is to project the tree gray scale image along the x-axis and the y-axis to obtain the column vector (the sum of the number of tree pixel points in each row) and the row vector (the sum of the number of tree pixel points in each column);

[0110] S322, according to the row vector X and the column vector Y of the tree instance gray scale drawing, the corresponding tree pixel height, trunk pixel width, crown pixel width, and crown pixel height are calculated through geometric relationship; wherein,

[0111] The calculation formula of the tree pixel height is:

[0112] h = max{X i}

[0113] Where h represents the tree pixel height; X i represents the sum of the number of tree pixel points in the i-th column;

[0114] The trunk width is the diameter at breast height, that is, the trunk width at 1.3 meters from the ground. Considering that in this embodiment, the average height of the street trees in the research area is about 6 meters, that is, the position at 22% of the tree height is taken as the trunk width.

[0115] The calculation formula of the trunk pixel width is:

[0116] d = Y r-0.22h

[0117] Where d represents the trunk pixel width; r represents the Y vector position of the tree root; Y r-0.22h represents the sum of the number of tree pixel points in the row from the element representing the tree root of the Y vector to 0.22h in the negative direction of the y-axis, that is, the crown direction;

[0118] The calculation formula of the crown pixel width is:

[0119] gd = max{Y j}

[0120] wherein gd represents the tree crown pixel width; Y j represents the sum of the number of tree pixel points of the jth row;

[0121] S323, the tree instance gray image is converted into a binary image by setting a threshold value, and the binary image is combined with the original image to extract the tree region from the original image, and an image containing only a single tree is obtained; wherein the process of converting the tree instance gray image into a binary image is: the threshold segmentation method is used to classify the gray values, and the threshold value is set as the gray value of the background, such as 30, according to the threshold value, if the pixel gray value is less than or equal to the background gray value, the pixel is classified as black, if the pixel gray value is greater than the background gray value, the pixel is classified as white, thereby converting the tree instance gray image into a binary image.

[0122] S324, based on the Gaussian distribution algorithm, the probability that each pixel point in the single tree image belongs to the tree crown and the tree trunk is obtained; specifically: according to the predetermined tree species classification, 10 different sample pictures of each tree species are collected, and the tree crown and tree trunk regions are labeled for each picture, and then the RGB values of the tree crown and tree trunk regions are obtained; wherein the predetermined tree species classification includes: Sophora japonica, Robinia pseudoacacia, Salix matsudana, Cedrus deodara, Pinus tabuliformis, Populus tomentosa, Populus canadensis, Populus tremuloides, Sabina pingii, Ginkgo biloba. For each type of tree, Gaussian distribution fitting is performed based on the RGB values of the tree crown and tree trunk regions, and the probability density functions of the tree crown and tree trunk regions are obtained respectively, and the probability that each pixel point in the single tree image belongs to the tree crown and the tree trunk is obtained by means of the function; wherein,

[0123] The RGB values of the tree crown and tree trunk regions are respectively subjected to Gaussian distribution fitting, and the sample data of the tree crown and tree trunk regions are all multi-dimensional vectors, so they all follow the following probability density function:

[0124]

[0125] Wherein μ is the data expectation, ∑ is the covariance, and D is the data dimension.

[0126] The probability distribution value of each pixel point in the single tree image is obtained by means of the function, and the tree crown probability value formula is:

[0127]

[0128] Wherein P crown,i is the probability value of the ith pixel point belonging to the tree crown, P c,i is the probability density of the ith pixel point in the tree crown probability density function, and P c,max is the maximum probability density in the tree crown probability density function.

[0129] Similarly, the trunk probability value formula is:

[0130]

[0131] where P trunk,i is the probability value of the i-th pixel point belonging to the trunk. P t,i is the probability density of the i-th pixel point in the trunk probability density function, P t,max is the maximum probability density in the trunk probability density function.

[0132] S325, for the discrete points existing in the distribution, the region growing algorithm is adopted, the trunk region pixel point is selected as the starting point, the trunk part pixel set is obtained, the remaining pixels are taken as the crown part pixel set, as shown in Figure 3 , thus the crown pixel height can be obtained, and the formula is:

[0133] gh=max{X crown,i}

[0134] where gh represents the crown pixel height; X crown,i represents the sum of the number of crown pixel points in the i-th row.

[0135] S4, the tree position information is calculated based on the multiple street view images of different angles corresponding to each row of trees;

[0136] Specifically, in the embodiment, the implementation process of S4 includes:

[0137] S41, the same tree matching is performed based on the multiple street view images of different angles corresponding to each row of trees;

[0138] It should be noted that a tree usually appears in several adjacent street views at the same time, therefore, the similarity priority method can be adopted to match the trees with the highest similarity in two street views, and the trees are identified as the same tree, and the formula for calculating the tree similarity is:

[0139]

[0140] where S represents the similarity of the trees in two different street view images; Δh represents the difference in the tree pixel height of the trees in two different street view images; Δgh represents the difference in the crown pixel height of the trees in two different street view images; Δd represents the difference in the trunk pixel width of the trees in two different street view images; and Δgd represents the difference in the crown pixel width of the trees in two different street view images.

[0141] S42, according to the matching result, the triangulation method is used to calculate the tree position information, and the specific implementation is as follows:

[0142] S421, the azimuth angle of the tree and the sampling point is calculated, and the formula is:

[0143]

[0144] wherein, sita represents the azimuth angle of the tree and the sampling point; Heading represents the orientation angle of the street view image where the tree is located; W represents the width of the street view image, and Δ represents the pixel distance between the trunk position and the midpoint of the street view image;

[0145] S422, based on two sampling points and the street tree with known coordinates, a solvable triangulation area is formed, and the tree position information is calculated, and the formula is:

[0146]

[0147] wherein, (x t ,y t ) is the coordinate of the tree, (x1,y1) is the coordinate of one of the sampling points in the triangulation area, (x2,y2) is the coordinate of the other sampling point in the triangulation area; θ1 and θ2 represent the azimuth angle between the tree and the two sampling points in the triangulation area, respectively.

[0148] S5, based on the tree pixel size information and the tree position information, the real size information of the tree is corrected;

[0149] Specifically, in the embodiment, the implementation process of S5 above includes:

[0150] S51, based on the tree and the sampling point coordinates, the distance between the tree and the sampling point in the real world is calculated;

[0151] S52, the field of view angle occupied by the tree is calculated based on the azimuth angle of the tree and the sampling point, and the formula is:

[0152]

[0153] angle=sita_end-sita_begin

[0154] wherein, Δ1 is the pixel distance between the starting position of the tree and the midpoint of the street view image, and Δ2 is the pixel distance between the ending position of the tree and the midpoint of the street view image.

[0155] S53, the real size information of the tree is corrected by the distance between the tree and the sampling point and the field of view angle occupied by the tree, to obtain the real width of the crown, the real height of the tree, the real height of the crown, and the real width of the trunk; wherein,

[0156] The formula for calculating the real width of the crown is:

[0157] GD=L×angle

[0158] wherein GD represents the real width of the tree crown; L represents the distance between the tree and the sampling point in the real world; and angle represents the angle of field of view occupied by the tree.

[0159] The calculation formula of the real height of the tree is:

[0160]

[0161] H=e×h

[0162] wherein e represents the conversion coefficient of the real size and the pixel size; gd represents the pixel width of the tree crown; h represents the pixel height of the tree; and H represents the real height of the tree.

[0163] The calculation formula of the real height of the tree crown is:

[0164] GH=e×gh

[0165] wherein GH represents the real height of the tree crown; and gh represents the pixel height of the tree crown.

[0166] The calculation formula of the real width of the trunk is:

[0167] D=e×d

[0168] wherein D represents the real width of the trunk; and d represents the pixel width of the trunk.

[0169] In summary, the embodiment provides a method for automatically obtaining the geometric and geographic information of urban street trees based on street view data. The method makes full use of the low cost and universality of street view images, making the preliminary preparation work of street tree information acquisition more convenient. Compared with the traditional manual investigation and recording method, the method greatly reduces the calculation time and improves the efficiency of street tree information acquisition, providing scientific and reasonable technical support for the management and statistics of street trees, and also providing strong support for urban greening planning, ecological environment monitoring, public safety assessment and other aspects, which helps to promote the sustainable development of the city and has great application prospect.

[0170] Second embodiment

[0171] The embodiment provides an electronic device, as shown in the accompanying drawings, the electronic device comprises a processor and a memory; wherein the processor and the memory can be connected through a communication bus; the memory stores at least one instruction, the instruction is loaded and executed by the processor to realize the method of the above-mentioned first embodiment. In addition, the electronic device can also include a transceiver, the processor and the transceiver can be connected through a communication bus, and the transceiver is used for communicating with other devices. Figure 4 In the following, the embodiment will be described in detail in combination with the drawings.

[0172] Figure 4 ​The various components of the electronic device are described in detail as follows:

[0173] The processor is the control center of the electronic device. The electronic device can include multiple processors. Each of the processors can be a single-CPU or a multi-CPU. The processor can be a single processor or a collective term of multiple processing elements. For example, the processor can be one or more central processing units (CPUs), other general purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present application, such as one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general purpose processor can be a microprocessor or any conventional processor, or the like. The processor can perform various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0174] In a specific implementation, as an embodiment, the processor can include one or more CPUs, such as CPU0 and CPU1 shown in FIG. 1, of course, this is only an exemplary description. Figure 4

[0175] The memory is used to store software programs for implementing the scheme of the present application, and is controlled by the processor to perform the implementation. The specific implementation can refer to the above-mentioned method embodiments, which will not be described here.

[0176] ​Optionally, the memory may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may be integrated with the processor or may exist independently, and may be accessed through the interface circuit of the electronic device (…). Figure 4 (Not shown in the image) is coupled to the processor; however, this embodiment of the invention does not impose specific limitations on this.

[0177] The transceiver may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function. The transceiver can be integrated with the processor or exist independently, and is connected through the interface circuit of the electronic device (…). Figure 4 (Not shown in the image) is coupled to the processor, and this embodiment of the invention does not specifically limit this.

[0178] In addition, it should be noted that, Figure 4 The structure of the electronic device shown is not intended to limit the device. Actual devices may include more or fewer components than shown, or combine certain components, or have different component arrangements. Furthermore, the technical effects achieved by this electronic device when performing the method of the first embodiment described above can be referenced to the technical effects described in the first embodiment; therefore, they will not be repeated here.

[0179] Third Embodiment

[0180] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc. The instruction stored therein can be loaded and executed by a processor in a terminal.

[0181] Moreover, it should be noted that the present application can be provided as a method, an apparatus, or a computer program product. Therefore, the embodiments of the present application can take the form of an entirely or partially hardware embodiment, an entirely or partially software embodiment, or an embodiment combining software and hardware aspects. Furthermore, when implemented in software, the embodiments of the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, a computer diskette, an optical storage medium, a magnetic storage medium, and a semiconductor memory device). The computer program product includes one or more computer instructions that when loaded and executed by a computer, cause the computer to carry out the processes or functions described in the embodiments of the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, such as from a website, a computer, a server, or a data center to another website, computer, server, or data center through a wired (for example, infrared, wireless, microwave, or the like) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, data center, or the like, including one or more collections of available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0182] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, an embedded processor, or a processor of another programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate a device that implements the flow Figure 1 The flow or the plurality of flows and / or blocks Figure 1 The device that implements the functions specified in the flow or the plurality of flows and / or blocks.

[0183] These computer program instructions can also be stored in a computer-readable storage medium that can guide the computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable storage medium produce a product including instruction devices that implement the flow Figure 1 The flow or the plurality of flows and / or blocks Figure 1the functions specified in the individual block or blocks. Such computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate a computer-implemented process, thus the instructions executed on the computer or other programmable devices provide a process for implementing the functions specified in the flowchart block(s). Figure 1 the functions specified in the individual block or blocks. Such computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate a computer-implemented process, thus the instructions executed on the computer or other programmable devices provide a process for implementing the functions specified in the flowchart block(s). Figure 1 the functions specified in the individual block or blocks. Such computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate a computer-implemented process, thus the instructions executed on the computer or other programmable devices provide a process for implementing the functions specified in the flowchart block(s).

[0184] It should also be noted that, in the present document, the terms such as first and second, etc. are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or terminal device. Without more limitations, the element defined by the statement "including a…", does not exclude the presence of other identical elements in the process, method, article or terminal device including the element. In addition, the term "and / or" is merely a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In addition, the character " / " in the present document generally represents an "or" relationship between the front and rear associated objects, but it can also represent an "and / or" relationship, which can be understood in the context before and after. "One or more" means one or more, and "multiple" means two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0185] In addition, it can be understood that in various embodiments of the present application, the size of the sequence number of the above processes does not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0186] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0187] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of functional modules / units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms. The units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment. In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present, or two or more units can be integrated in one unit.

[0188] If the method is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0189] Finally, it should be noted that the above description is only the preferred embodiment of the application, it should be pointed out that although the preferred embodiment of the application has been described, for those skilled in the art, once the basic creative concept of the application is known, several improvements and refinements can be made without departing from the principles of the application, and these improvements and refinements should also be considered as the protection scope of the application. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the application.

Claims

1. A method for automatically acquiring geometric and geographic information of urban street trees based on street view data, characterized in that, The method for automatically acquiring the geometric and geographic information of urban street trees based on street view data includes: Obtain street view image data of the target area; wherein, the street view image data refers to street view images containing street trees in the target area, and each street tree corresponds to multiple street view images from different perspectives; Tree outline shapes are extracted from the street view image data using a preset tree detection model; Based on the tree outline shape image, calculate the tree pixel size information; Based on multiple street view images from different perspectives corresponding to each row of trees, the location information of the trees is calculated. Based on tree pixel size information and tree location information, the actual size information of the tree is corrected; Based on the tree outline shape image, the tree pixel size information is calculated, including: Based on the tree outline shape image, obtain a grayscale image of the tree instance; The row vector X and column vector Y of the grayscale image of the tree instance are obtained through projection processing; Based on the row vector X and column vector Y of the grayscale image of the tree instance, the pixel height, trunk pixel width, and crown pixel width of the corresponding tree are calculated through geometric relationships; where, The formula for calculating the pixel height of a tree is: h=max{X i } Where h represents the tree pixel height; X i This represents the sum of the number of tree pixels in the i-th column; The formula for calculating the tree trunk pixel width is: d=Y r-0.22h Where d represents the width of the tree trunk in pixels; r represents the position of the Y vector at the root of the tree; Y r-0.22h This represents the sum of the number of tree pixels in the row starting from the element representing the tree root in the Y vector, along the negative y-axis, i.e., the direction of the tree crown, up to 0.22h. The formula for calculating the width of the tree crown in pixels is: gd=max{Y j } Where gd represents the width of the tree crown in pixels; Y j This represents the sum of the number of tree pixels in the j-th row; The grayscale image of the tree instance is converted into a binary image, and the binary image is combined with the street view image data to extract the single tree image corresponding to the tree from the street view image data. Based on the Gaussian distribution algorithm, the probability of each pixel in a single tree image belonging to the canopy and trunk is obtained. For discrete points in the distribution, a region growing algorithm is used to select the pixels in the trunk region as the starting point to obtain the pixel set of the trunk part, and take the remaining pixels as the pixel set of the canopy part to obtain the canopy pixel height. gh=max{X crown,i } Where gh represents the height of the tree canopy in pixels; X crown,i This represents the sum of the number of tree crown pixels in the i-th row.

2. The method for automatically acquiring the geometric and geographic information of urban street trees based on street view data as described in claim 1, characterized in that, The acquisition of street view image data of the target area includes: Based on the principle that roads do not have corners, the roads in the target area are divided into multiple road segment units; Sampling points were set up at preset intervals in each road segment unit; The system acquires multiple street view images from different perspectives for each sampling point based on a preset street view image system.

3. The method for automatically acquiring the geometric and geographic information of urban street trees based on street view data as described in claim 1, characterized in that, Based on the tree outline shape image, a grayscale image of a tree instance is obtained, including: The tree outline shape image is converted to grayscale to obtain a grayscale image of the tree outline shape. Based on the grayscale image, individual tree grayscale images of different trees are selected by comparing grayscale values. Based on the grayscale image of a single tree, a two-dimensional moving average is used to obtain the blurred boundaries of the trees in the grayscale image of a single tree, and the blurred boundaries of the trees are corrected by setting a threshold to obtain the grayscale image of the tree instance.

4. The method for automatically acquiring the geometric and geographic information of urban street trees based on street view data as described in claim 1, characterized in that, The Gaussian distribution-based algorithm obtains the probability that each pixel in a single tree image belongs to the crown or trunk, including: Collect multiple sample images for each type of tree, and label the canopy area and trunk area for each sample image to obtain the RGB values ​​of the canopy area and trunk area; Gaussian distributions were fitted to the RGB values ​​of the canopy region and the trunk region respectively to obtain the probability density functions corresponding to the RGB values ​​of the canopy region and the trunk region. Based on the probability density functions corresponding to the RGB values ​​of the canopy region and the trunk region, the probability of each pixel in a single tree image belonging to the canopy and trunk is obtained.

5. The method for automatically acquiring the geometric and geographic information of urban street trees based on street view data as described in claim 1, characterized in that, The tree location information is calculated based on multiple street view images from different perspectives corresponding to each row of trees, including: Matching trees based on multiple street view images from different perspectives corresponding to each row of trees; Based on the matching results, the tree location information is calculated using the triangulation method.

6. The method for automatically acquiring the geometric and geographic information of urban street trees based on street view data as described in claim 5, characterized in that, The matching of trees based on multiple street view images from different perspectives corresponding to each row of trees includes: The similarity between trees in two different street view images is calculated using the following formula: Where S represents the similarity of trees in two different street view images; Δh represents the difference in tree pixel height between the two different street view images; Δgh represents the difference in tree crown pixel height between the two different street view images; Δd represents the difference in tree trunk pixel width between the two different street view images; and Δgd represents the difference in tree crown pixel width between the two different street view images. The trees with the highest similarity between two street view images are matched and identified as the same tree.

7. The method for automatically acquiring the geometric and geographic information of urban street trees based on street view data as described in claim 5, characterized in that, The step of calculating tree location information using triangulation based on the matching results includes: The azimuth angle between the tree and the sampling point is calculated using the following formula: Where, sita represents the azimuth angle between the tree and the sampling point; Heading represents the orientation angle of the street view image where the tree is located; W represents the width of the street view image; and Δ represents the pixel distance between the tree trunk position and the midpoint of the street view image. Based on a triangular region formed by two sampling points with known coordinates and the roadside trees, the tree location information is calculated using the following formula: Where, x t y t Let x1 and y1 be the coordinates of the tree, x2 and y2 be the coordinates of one sampling point in the triangulated region, and θ1 and θ2 be the coordinates of the other sampling point in the triangulated region.

8. The method for automatically acquiring the geometric and geographic information of urban street trees based on street view data as described in claim 1, characterized in that, The process of correcting the actual tree size information based on tree pixel size information and tree location information includes: The distance between the real-world trees and the sampling points was calculated; The field of view angle occupied by the tree is calculated based on the azimuth angle between the tree and the sampling point; By adjusting the distance between the tree and the sampling point and the field of view occupied by the tree, the true size information of the tree is corrected to obtain the true crown width, true tree height, true crown height, and true trunk width; among these... The formula for calculating the true width of the tree crown is: GD = L × angle Where GD represents the actual width of the tree canopy; L represents the distance between the tree and the sampling point in the real world; and angle represents the field of view occupied by the tree. The formula for calculating the true height of a tree is: H = e × h Where gd represents the tree canopy pixel width; h represents the tree pixel height; and H represents the tree's actual height. The formula for calculating the true height of a tree crown is: GH = e × gh Where GH represents the actual height of the tree crown; gh represents the pixel height of the tree crown; The formula for calculating the actual width of the tree trunk is: D = e × d Where D represents the actual width of the tree trunk; d represents the pixel width of the tree trunk.

Images