Method for automatically acquiring geometric and geographic information of urban street tree based on street view data

Abstract

The invention discloses an urban street tree geometric and geographic information automatic acquisition method based on street view data, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring street view image data of a target area; wherein the street view image data refers to street view images containing street trees in the target area, and each street tree corresponds to a plurality of street view images of different visual angles; extracting a tree contour shape image from the streetscape image data through a preset tree detection model; calculating tree pixel size information based on the tree contour shape image; based on the plurality of street view images of different visual angles corresponding to each street tree, calculating to obtain tree position information; and based on the tree pixel size information and the tree position information, correcting the tree real size information. Compared with a traditional manual investigation and recording method, the method has the advantages that the calculation time is greatly shortened, and the street tree information acquisition efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data. Background Art

[0002] With the acceleration of urbanization and the popularization of the concept of green cities, the management and acquisition of urban street tree information plays an increasingly important role in urban sustainable development and management. Given the large number of urban street trees, traditional manual survey and recording methods can no longer meet the needs of efficient and accurate management. Therefore, it is particularly urgent to explore the automated acquisition method of street tree geometry and geographic information.

[0003] In recent years, street view images have rapidly become an important data source for geospatial data collection and urban environment analysis. Street view images not only contain urban roads and their ancillary elements, but almost every street view photo also records the existence of street trees, which makes street view images an important medium for obtaining urban street tree information. Therefore, studying how to automatically extract street tree information from street view images has significant scientific value and application prospects for improving the scientific nature of urban management and planning.

[0004] However, there is currently no solution that can automatically obtain the geometric and geographic information of street trees through street view data. Summary of the invention

[0005] The present invention provides a method for automatically acquiring the geometric and geographic information of urban roadside trees based on street view data, so as to solve the technical problem that the traditional manual survey and recording methods cannot meet the requirements of efficient and accurate management of the geometric and geographic information of urban roadside trees.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In one aspect, the present invention provides a method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data, the method comprising:

[0008] Acquire street view image data of a target area; wherein the street view image data refers to a street view image containing roadside trees in the target area, and each roadside tree corresponds to a plurality of street view images of different viewing angles;

[0009] Extracting tree outline shape images from the street view image data using a preset tree detection model;

[0010] Calculating tree pixel size information based on the tree outline shape image;

[0011] Based on multiple street view images from different perspectives corresponding to each roadside tree, the tree location information is calculated;

[0012] Based on the tree pixel size information and tree position information, the real tree size information is corrected.

[0013] Furthermore, the acquiring of street view image data of the target area includes:

[0014] Divide the roads in the target area into multiple road section units according to the principle that there are no corners in the divided roads;

[0015] Sampling points are arranged at preset intervals in each road section unit;

[0016] Based on the preset street view system, multiple street view images from different perspectives are obtained for each sampling point.

[0017] Further, based on the tree outline shape image, calculating the tree pixel size information includes:

[0018] Based on the tree outline shape image, obtaining a tree instance grayscale image;

[0019] Based on the grayscale image of tree instances, Gaussian distribution and region growing algorithm are used to obtain the tree pixel size information.

[0020] Further, based on the tree outline shape image, a tree instance grayscale image is obtained, including:

[0021] Grayscale processing is performed on the tree outline shape image to obtain a grayscale image of the tree outline shape image;

[0022] Based on the grayscale image, single tree grayscale images of different trees are screened out by grayscale value comparison;

[0023] Based on the single tree grayscale image, a two-dimensional moving average is used to obtain the fuzzy boundaries of the trees in the single tree grayscale image, and the fuzzy boundaries of the trees are corrected by setting a threshold to obtain a tree instance grayscale image.

[0024] Furthermore, the method of obtaining the tree pixel size information based on the tree instance grayscale image by using Gaussian distribution and region growing algorithm includes:

[0025] Obtaining the row vector X and column vector Y of the tree instance grayscale image by projection processing;

[0026] According to the row vector X and column vector Y of the tree instance grayscale image, the tree pixel height, trunk pixel width, crown pixel width and crown pixel height of the corresponding tree are calculated through geometric relationships; where:

[0027] The calculation formula for tree pixel height is:

[0028] h=max{X i}

[0029] Where h is the height of the tree pixel; X i Represents the sum of the number of tree pixels in the i-th column;

[0030] The calculation formula for the pixel width of the tree trunk is:

[0031] d=Y r-0.22h

[0032] Where d represents the pixel width of the tree trunk; r represents the Y vector position of the tree root; Y r-0.22h It means the sum of the number of tree pixels in the row at 0.22h, starting from the Y vector element representing the root of the tree, along the negative direction of the y axis, that is, the direction of the crown;

[0033] The calculation formula for the pixel width of the crown is:

[0034] gd=max{Y j}

[0035] Where gd represents the pixel width of the crown; Y j Represents the sum of the number of tree pixels in the jth row;

[0036] Converting the tree instance grayscale image into a binary image, combining the binary image with the street view image data, and extracting a single tree image corresponding to the corresponding tree from the street view image data;

[0037] Based on the Gaussian distribution algorithm, the probability that each pixel in a single tree image belongs to the crown and trunk is obtained; for discrete points in the distribution, the region growing algorithm is used to select the trunk area pixel point as the starting point, obtain the trunk part pixel set, and take the remaining pixels as the crown part pixel set to obtain the crown pixel height:

[0038] gh=max{X crown,i}

[0039] Among them, gh represents the canopy pixel height; X crown,i Represents the sum of the number of tree crown pixels in the i-th row.

[0040] Furthermore, the Gaussian distribution algorithm is used to obtain the probability that each pixel in a single tree image belongs to the crown and the trunk, including:

[0041] Collect multiple sample images corresponding to each type of tree, and mark the crown area and trunk area of ​​each sample image, and then obtain the RGB values ​​of the crown area and trunk area;

[0042] Gaussian distribution fitting is performed on the RGB values ​​of the crown area and the trunk area respectively to obtain the probability density function corresponding to the RGB value of the crown area and the probability density function corresponding to the RGB value of the trunk area;

[0043] Based on the probability density function corresponding to the RGB values ​​of the crown area and the probability density function corresponding to the RGB values ​​of the trunk area, the probability that each pixel in a single tree image belongs to the crown and the trunk is obtained.

[0044] Furthermore, the tree location information is calculated based on a plurality of street view images of different viewing angles corresponding to each roadside tree, including:

[0045] Perform tree matching based on multiple street view images from different perspectives corresponding to each roadside tree;

[0046] According to the matching results, the tree location information is calculated using the triangulation method.

[0047] Furthermore, the same-tree matching based on the multiple street view images of different perspectives corresponding to each roadside tree includes:

[0048] Calculate the similarity of trees in two different street view images using the formula:

[0049]

[0050] Among them, S represents the similarity of trees in two different street view images; Δh represents the difference in tree pixel heights of trees in two different street view images; Δgh represents the difference in tree crown pixel heights of trees in two different street view images; Δd represents the difference in tree trunk pixel widths of trees in two different street view images; Δgd represents the difference in tree crown pixel widths of trees in two different street view images;

[0051] The trees with the highest similarity in the two street view images are matched and identified as the same tree.

[0052] Furthermore, according to the matching results, the tree location information is calculated using the triangulation method, including:

[0053] Calculate the azimuth between the tree and the sampling point using the formula:

[0054]

[0055] Among them, sita represents the azimuth angle between the tree and the sampling point; Heading represents the heading angle of the street view image where the tree is located; W represents the width of the street view image, and Δ represents the pixel distance between the trunk position and the midpoint of the street view image;

[0056] Based on two sampling points with known coordinates and a triangulated area formed by street trees, the tree location information is calculated using the formula:

[0057]

[0058] Among them, x t ,y t is the coordinate value of the tree, x 1 ,y 1 is the coordinate value of one of the sampling points in the triangulated area, x 2 ,y 2 is the coordinate value of another sampling point in the triangulated area; θ 1 and θ 2 Represent the azimuth between the tree and two sampling points in the triangulated area.

[0059] Furthermore, the correcting of the real size information of the tree based on the tree pixel size information and the tree position information includes:

[0060] Calculate the distance between trees and sampling points in the real world;

[0061] The field of view angle occupied by the tree is calculated based on the azimuth angle between the tree and the sampling point;

[0062] The real size information of the tree is corrected by the distance between the tree and the sampling point and the field angle occupied by the tree to obtain the real width of the crown, the real height of the tree, the real height of the crown and the real width of the trunk;

[0063] The formula for calculating the true width of the crown is:

[0064] GD=L×angle

[0065] Among them, GD represents the real width of the tree crown; L represents the distance between the tree and the sampling point in the real world; angle represents the field of view occupied by the tree;

[0066] The calculation formula for the true height of a tree is:

[0067]

[0068] H=e×h

[0069] Among them, gd represents the pixel width of the tree crown; h represents the pixel height of the tree; H represents the actual height of the tree;

[0070] The calculation formula for the true height of the tree crown is:

[0071] GH=e×gh

[0072] Among them, GH represents the real height of the tree crown; gh represents the pixel height of the tree crown;

[0073] The formula for calculating the true width of the trunk is:

[0074] D=e×d

[0075] Where D represents the true width of the trunk; d represents the pixel width of the trunk.

[0076] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0077] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the above method.

[0078] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0079] The technical solution of the present invention makes full use of the low cost and universality of street view images, making the preliminary preparation work for obtaining street tree information easier. Compared with traditional manual survey and recording methods, this method greatly reduces the calculation time and improves the efficiency of obtaining street tree information, providing scientific and reasonable technical support for the management and statistics of street trees. At the same time, it can also provide strong guarantees for urban greening planning, ecological environment monitoring, public safety assessment, etc., which is helpful to promote the sustainable development of cities and has great application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0081] Figure 1 It is a schematic diagram of the execution flow of a method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data provided by an embodiment of the present invention;

[0082] Figure 2 is a schematic diagram of a street view image and tree image segmentation results thereof provided by an embodiment of the present invention;

[0083] Figure 3 is a schematic diagram of the segmentation results of the crown and trunk based on Gaussian distribution and region growing algorithm provided by an embodiment of the present invention;

[0084] Figure 4 It is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0085] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0086] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0087] First embodiment

[0088] This embodiment provides a method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data, which is used to automatically acquire roadside tree information for a specified street view. The method can be implemented by an electronic device, which can be a terminal or a server. The execution flow of the method is as follows: Figure 1 As shown, the following steps are included:

[0089] S1, obtaining street view image data of a target area; wherein the street view image data refers to a street view image containing roadside trees in the target area, and each roadside tree corresponds to a plurality of street view images from different perspectives;

[0090] Specifically, in this embodiment, the implementation process of the above S1 includes:

[0091] S11, dividing the roads in the target area into a plurality of road section units according to the principle that there are no corners in the divided roads;

[0092] S12, arranging sampling points at fixed intervals along the center of each divided road section unit;

[0093] S13, obtaining multiple street view images of different perspectives for each sampling point based on a preset street view system;

[0094] In this embodiment, Baidu Street View API is used to obtain street view data of the target area; the fixed spacing of the sampling points used in this embodiment is 10m. The street view control parameters set in this embodiment are width is 1024, height is 512, Heading is 0, 90, 180, 270 respectively, and Fov is 90.

[0095] S2, extracting tree outline shape images from street view image data through a preset tree detection model;

[0096] Specifically, the tree detection model used in this embodiment is composed of a tree recognition module and a tree segmentation module: wherein the tree recognition module is used to recognize trees in the street view image and obtain a rectangular frame of trees in the street view image; the tree segmentation module is used to segment the outline shape images of all trees from the rectangular frame of trees obtained by the tree recognition module. The training process of the tree recognition module includes:

[0097] Step 1: Collect public plant images from the Plant Science Data Center and street views of parts of a city;

[0098] Step 2: semantically annotate the collected images to form a training data set;

[0099] Step 3: Use the training data set to train the tree recognition module.

[0100] Specifically, in this embodiment, the network architecture of the tree recognition module adopts the Transformer-based SwinL network architecture, and the tree segmentation module adopts the basic large model Segment Anything Model in the field of visual segmentation. Figure 2 A street view image and a schematic diagram of tree image segmentation results provided by this embodiment are shown. Figure 2 It can be seen that the trees in the image have been effectively identified.

[0101] S3, calculating the tree pixel size information based on the tree outline shape image;

[0102] Specifically, in this embodiment, the implementation process of the above S3 includes:

[0103] S31, based on the tree outline shape image, obtain a tree instance grayscale image; the specific implementation is as follows:

[0104] S311, graying the tree outline shape image to obtain a grayscale image of the tree outline shape image;

[0105] S312, based on the grayscale image, screening out single tree grayscale images of different trees by grayscale value comparison;

[0106] S313, based on the grayscale image of a single tree, a fuzzy boundary of the grayscale image of a single tree is obtained by using a two-dimensional moving average, and the boundary is corrected by setting a threshold to obtain a grayscale image of a tree instance;

[0107] Among them, it should be noted that due to the grayscale mixing of the original image, there is missing information on the tree instance boundary in the extracted single tree grayscale image, which is not conducive to subsequent calculations. Therefore, the boundary needs to be supplemented and corrected to obtain a more accurate boundary contour. Correcting the fuzzy boundary of the tree by setting a threshold means: correcting the grayscale value of the pixel point at the boundary of the tree. Here, the threshold is set to the background grayscale value, such as 30. If the grayscale value of the pixel point is greater than the background grayscale value, the pixel point is identified as the grayscale of the tree, and the tree boundary is corrected by this method.

[0108] S32, based on the tree instance grayscale image, uses Gaussian distribution and region growing algorithm to obtain tree pixel size information; the specific implementation is as follows:

[0109] S321, obtaining a row vector X and a column vector Y of the tree instance grayscale image through projection processing; the specific process is to project the tree grayscale image along the x-axis and the y-axis to obtain a column vector (the sum of the number of tree pixels in each row) and a row vector (the sum of the number of tree pixels in each column);

[0110] S322, according to the row vector X and column vector Y of the tree instance grayscale image, calculate the tree pixel height, trunk pixel width, crown pixel width, and crown pixel height of the corresponding tree through geometric relationship; wherein,

[0111] The calculation formula for tree pixel height is:

[0112] h=max{X i}

[0113] Where h is the height of the tree pixel; X i Represents the sum of the number of tree pixels in the i-th column;

[0114] The trunk width adopts the breast diameter, that is, the trunk width at 1.3 meters above the ground. Considering that in this embodiment, the average height of street trees in the study area is about 6 meters, the trunk width is taken as 22% of the tree height.

[0115] The calculation formula for the pixel width of the tree trunk is:

[0116] d=Y r-0.22h

[0117] Where d represents the pixel width of the tree trunk; r represents the Y vector position of the tree root; Y r-0.22h It means the sum of the number of tree pixels in the row at 0.22h, starting from the Y vector element representing the root of the tree, along the negative direction of the y axis, that is, the direction of the crown;

[0118] The calculation formula for the pixel width of the crown is:

[0119] gd=max{Y j}

[0120] Where gd represents the pixel width of the crown; Y j Represents the sum of the number of tree pixels in the jth row;

[0121] S323, converting the tree instance grayscale image into a binary image by setting a threshold, combining the binary image with the original image, extracting the tree area from the original image, and obtaining an image containing only a single tree; wherein the process of converting the tree instance grayscale image into a binary image is: using a threshold segmentation method to classify the grayscale value, where the threshold is set to the grayscale value of the background, such as 30, and judging according to the threshold, if the grayscale value of the pixel point is less than or equal to the grayscale value of the background, the pixel point is classified as black, and if the grayscale value of the pixel point is greater than the grayscale value of the background, the pixel point is classified as white, thereby converting the tree instance grayscale image into a binary image.

[0122] S324, based on the Gaussian distribution algorithm, obtain the probability that each pixel in a single tree image belongs to the crown and trunk; specifically: according to the predetermined tree species classification, collect 10 different sample pictures of each tree species, and mark the crown and trunk areas of each picture, and then obtain the RGB values ​​of the crown and trunk areas; wherein, the predetermined tree species classification includes: Sophora japonica, Robinia pseudoacacia, Salix matsudana, Cedrus deodara, Pinus tabulaeformis, Populus tomentosa, Populus taiwanensis, Populus roxburghii, Populus davidianus, Juniperus chinensis, and Ginkgo biloba. For each type of tree, Gaussian distribution fitting is performed based on the RGB values ​​of the crown and trunk areas to obtain the probability density functions of the crown and trunk areas, and the probability that each pixel in the image containing only a single tree belongs to the crown and trunk is obtained with the help of this function; wherein,

[0123] Gaussian distribution fitting is performed on the RGB values ​​of the crown and trunk areas respectively. The sample data of the crown and trunk areas are multidimensional vectors, so they all follow the following probability density function:

[0124]

[0125] Among them, μ is the data expectation, ∑ is the covariance, and D is the data dimension.

[0126] With the help of this function, the probability distribution value of each pixel in the image containing only a single tree is obtained. The formula for the crown probability value is:

[0127]

[0128] Among them, P crown,i is the probability value of the i-th pixel belonging to the crown, P c,i is the probability density of the ith pixel in the crown probability density function, P c,max It is the maximum probability density in the crown probability density function.

[0129] Similarly, the trunk probability value formula is:

[0130]

[0131] Among them, P trunk,i is the probability value that the i-th pixel belongs to the trunk. t,i is the probability density of the ith pixel in the tree trunk probability density function, P t,max is the maximum probability density in the trunk probability density function.

[0132] S325, for the discrete points in the distribution, a region growing algorithm is used to select the pixel point in the trunk area as the starting point, obtain the pixel set of the trunk part, and take the remaining pixels as the pixel set of the crown part, such as Figure 3 As shown, the crown pixel height can be obtained using the formula:

[0133] gh=max{X crown,i}

[0134] Among them, gh represents the canopy pixel height; X crown,i Represents the sum of the number of tree crown pixels in the i-th row.

[0135] S4, calculating the tree location information based on a plurality of street view images with different perspectives corresponding to each roadside tree;

[0136] Specifically, in this embodiment, the implementation process of the above S4 includes:

[0137] S41, performing same-tree matching based on a plurality of street view images from different perspectives corresponding to each roadside tree;

[0138] It should be noted that a tree usually appears in several adjacent street scenes at the same time. Therefore, the similarity priority method can be used to match the trees with the highest similarity in two street scenes and identify them as the same tree. The formula for calculating the similarity of trees is:

[0139]

[0140] Among them, S represents the similarity of trees in two different street view images; Δh represents the difference in tree pixel heights of trees in two different street view images; Δgh represents the difference in tree crown pixel heights of trees in two different street view images; Δd represents the difference in tree trunk pixel widths of trees in two different street view images; Δgd represents the difference in tree crown pixel widths of trees in two different street view images;

[0141] S42, according to the matching results, the tree location information is calculated using a triangulation method, which is specifically implemented as follows:

[0142] S421, calculate the azimuth between the tree and the sampling point, the formula is:

[0143]

[0144] Among them, sita represents the azimuth angle between the tree and the sampling point; Heading represents the heading angle of the street view image where the tree is located; W represents the width of the street view image, and Δ represents the pixel distance between the trunk position and the midpoint of the street view image;

[0145] S422, based on the two sampling points with known coordinates and the street trees, a triangulated area that can be solved is formed to calculate the tree location information, and the formula is:

[0146]

[0147] Among them, (x t ,y t ) is the coordinate of the tree, (x 1 ,y 1 ) is the coordinate of one of the sampling points in the triangulated area, (x 2 ,y 2 ) is the coordinate of another sampling point in the triangulated area; θ 1 and θ 2 Represent the azimuth between the tree and two sampling points in the triangulated area.

[0148] S5, based on the tree pixel size information and the tree position information, correcting the tree real size information;

[0149] Specifically, in this embodiment, the implementation process of the above S5 includes:

[0150] S51, based on the coordinates of the tree and the sampling point, calculate the distance between the tree and the sampling point in the real world;

[0151] S52, based on the azimuth angle between the tree and the sampling point, the field of view angle occupied by the tree is calculated using the formula:

[0152]

[0153] angle = sita_end - sita_begin

[0154] Among them, Δ 1 is the pixel distance between the tree start position and the midpoint of the street view image, Δ 2 is the distance in pixels between where the tree ends and the midpoint of the Street View image.

[0155] S53, correcting the tree's real size information according to the distance between the tree and the sampling point and the field of view angle occupied by the tree to obtain the real width of the tree crown, the real height of the tree crown, and the real width of the trunk; wherein,

[0156] The formula for calculating the true width of the crown is:

[0157] GD=L×angle

[0158] Among them, GD represents the real width of the tree crown; L represents the distance between the tree and the sampling point in the real world; angle represents the field of view occupied by the tree;

[0159] The calculation formula for the true height of a tree is:

[0160]

[0161] H=e×h

[0162] Among them, e represents the conversion factor between the real size and pixel size; gd represents the pixel width of the tree crown; h represents the pixel height of the tree; H represents the real height of the tree;

[0163] The calculation formula for the true height of the tree crown is:

[0164] GH=e×gh

[0165] Among them, GH represents the real height of the tree crown; gh represents the pixel height of the tree crown;

[0166] The formula for calculating the true width of the trunk is:

[0167] D=e×d

[0168] Where D represents the true width of the trunk; d represents the pixel width of the trunk.

[0169] In summary, this embodiment provides a method for automatically acquiring the geometric and geographic information of urban street trees based on street view data. This method makes full use of the low cost and universality of street view images, making the preliminary preparation work for acquiring street tree information easier. Compared with traditional manual survey and recording methods, this method greatly reduces the calculation time and improves the efficiency of acquiring street tree information, providing scientific and reasonable technical support for the management and statistics of street trees. At the same time, it can also provide strong guarantees for urban greening planning, ecological environment monitoring, public safety assessment, etc., which helps promote the sustainable development of cities and has great application prospects.

[0170] Second embodiment

[0171] This embodiment provides an electronic device, such as Figure 4As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0172] Next, combine Figure 4 The following is a detailed introduction to the various components of the electronic device:

[0173] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0174] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 4 The CPU0 and CPU1 shown in the figure are, of course, only exemplary.

[0175] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0176] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 4 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0177] The transceiver may include a receiver and a transmitter ( Figure 4 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 4 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0178] In addition, it should be noted that Figure 4 The structure of the electronic device shown in the figure does not constitute a limitation on the device, and the actual device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment above can refer to the technical effects described in the first embodiment above, so they are not repeated here.

[0179] Third embodiment

[0180] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.

[0181] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiment of the present invention can be in the form of a full or partial hardware embodiment, a full or partial software embodiment or an embodiment combining software and hardware. Moreover, when implemented using software, the embodiment of the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state hard disk.

[0182] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0183] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0184] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. In addition, the term "and / or" is only an association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc or abc, where a, b, c can be single or plural.

[0185] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0186] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0187] In several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0188] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0189] Finally, it should be noted that the above is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for ordinary technicians in this technical field, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data, characterized in that: The method for automatically acquiring the geometric and geographic information of urban roadside trees based on street view data comprises: Acquire street view image data of a target area; wherein the street view image data refers to a street view image containing roadside trees in the target area, and each roadside tree corresponds to a plurality of street view images of different viewing angles; Extracting tree outline shape images from the street view image data using a preset tree detection model; Calculating tree pixel size information based on the tree outline shape image; Based on multiple street view images from different perspectives corresponding to each roadside tree, the tree location information is calculated; Based on the tree pixel size information and tree position information, the real tree size information is corrected.

2. The method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data according to claim 1, characterized in that: The step of acquiring street view image data of a target area includes: Divide the roads in the target area into multiple road section units according to the principle that there are no corners in the divided roads; Sampling points are arranged at preset intervals in each road section unit; Based on the preset street view system, multiple street view images from different perspectives are obtained for each sampling point.

3. The method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data according to claim 1, characterized in that: Calculating tree pixel size information based on the tree outline shape image includes: Based on the tree outline shape image, obtaining a tree instance grayscale image; Based on the grayscale image of tree instances, Gaussian distribution and region growing algorithm are used to obtain the tree pixel size information.

4. The method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data as claimed in claim 3, characterized in that: Based on the tree outline shape image, a tree instance grayscale image is obtained, including: Grayscale processing is performed on the tree outline shape image to obtain a grayscale image of the tree outline shape image; Based on the grayscale image, single tree grayscale images of different trees are screened out by grayscale value comparison; Based on the single tree grayscale image, a two-dimensional moving average is used to obtain the fuzzy boundaries of the trees in the single tree grayscale image, and the fuzzy boundaries of the trees are corrected by setting a threshold to obtain a tree instance grayscale image.

5. The method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data as claimed in claim 4, characterized in that: The method of obtaining the tree pixel size information based on the tree instance grayscale image by using Gaussian distribution and region growing algorithm includes: Obtaining the row vector X and column vector Y of the tree instance grayscale image by projection processing; According to the row vector X and column vector Y of the tree instance grayscale image, the tree pixel height, trunk pixel width and crown pixel width of the corresponding tree are calculated through geometric relationships; The calculation formula for tree pixel height is: h=max{X i } Where h is the height of the tree pixel; X i Represents the sum of the number of tree pixels in the i-th column; The calculation formula for the pixel width of the tree trunk is: d=Y r-0.22h Where d represents the pixel width of the tree trunk; r represents the Y vector position of the tree root; Y r-0.22h It means the sum of the number of tree pixels in the row at 0.22h, starting from the Y vector element representing the root of the tree, along the negative direction of the y axis, that is, the direction of the crown; The calculation formula for the pixel width of the crown is: gd=max{Y j } Where gd represents the pixel width of the crown; Y j Represents the sum of the number of tree pixels in the jth row; Converting the tree instance grayscale image into a binary image, combining the binary image with the street view image data, and extracting a single tree image corresponding to the corresponding tree from the street view image data; Based on the Gaussian distribution algorithm, the probability that each pixel in a single tree image belongs to the crown and trunk is obtained; for discrete points in the distribution, the region growing algorithm is used to select the trunk area pixel point as the starting point, obtain the trunk part pixel set, and take the remaining pixels as the crown part pixel set to obtain the crown pixel height: gh=max{X crown,i } Among them, gh represents the canopy pixel height; X crown,i Represents the sum of the number of tree crown pixels in the i-th row.

6. The method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data as claimed in claim 5, characterized in that: The Gaussian distribution algorithm is used to obtain the probability that each pixel in a single tree image belongs to the crown and the trunk, including: Collect multiple sample images corresponding to each type of tree, and mark the crown area and trunk area of ​​each sample image, and then obtain the RGB values ​​of the crown area and trunk area; Gaussian distribution fitting is performed on the RGB values ​​of the crown area and the trunk area respectively to obtain the probability density function corresponding to the RGB value of the crown area and the probability density function corresponding to the RGB value of the trunk area; Based on the probability density function corresponding to the RGB values ​​of the crown area and the probability density function corresponding to the RGB values ​​of the trunk area, the probability that each pixel in a single tree image belongs to the crown and the trunk is obtained.

7. The method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data as claimed in claim 1, characterized in that: The tree location information is calculated based on a plurality of street view images from different perspectives corresponding to each roadside tree, including: Perform tree matching based on multiple street view images from different perspectives corresponding to each roadside tree; According to the matching results, the tree location information is calculated using the triangulation method.

8. The method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data according to claim 7, characterized in that: The same-tree matching based on multiple street view images from different perspectives corresponding to each roadside tree includes: Calculate the similarity of trees in two different street view images using the formula: Among them, S represents the similarity of trees in two different street view images; Δh represents the difference in tree pixel heights of trees in two different street view images; Δgh represents the difference in tree crown pixel heights of trees in two different street view images; Δd represents the difference in tree trunk pixel widths of trees in two different street view images; Δgd represents the difference in tree crown pixel widths of trees in two different street view images; The trees with the highest similarity in the two street view images are matched and identified as the same tree.

9. The method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data according to claim 7, characterized in that: The tree location information is calculated using a triangulation method based on the matching results, including: Calculate the azimuth between the tree and the sampling point using the formula: Among them, sita represents the azimuth angle between the tree and the sampling point; Heading represents the heading angle of the street view image where the tree is located; W represents the width of the street view image, and Δ represents the pixel distance between the trunk position and the midpoint of the street view image; Based on two sampling points with known coordinates and a triangulated area formed by street trees, the tree location information is calculated using the formula: Among them, x t ,y t is the coordinate value of the tree, x1 and y1 are the coordinate values ​​of one of the sampling points in the triangulated area, and x2 and y2 are the coordinate values ​​of another sampling point in the triangulated area; θ1 and θ2 represent the azimuths between the tree and the two sampling points in the triangulated area, respectively.

10. The method for automatically acquiring geometric and geographic information of urban roadside trees based on street view data according to claim 1, characterized in that: The correcting of the real size information of the tree based on the tree pixel size information and the tree position information includes: Calculate the distance between trees and sampling points in the real world; The field of view angle occupied by the tree is calculated based on the azimuth angle between the tree and the sampling point; The real size information of the tree is corrected by the distance between the tree and the sampling point and the field of view occupied by the tree to obtain the real width of the crown, the real height of the tree, the real height of the crown and the real width of the trunk; The formula for calculating the true width of the crown is: GD=L×angle Among them, GD represents the real width of the tree crown; L represents the distance between the tree and the sampling point in the real world; angle represents the field of view occupied by the tree; The calculation formula for the true height of a tree is: H=e×h Among them, gd represents the pixel width of the tree crown; h represents the pixel height of the tree; H represents the actual height of the tree; The calculation formula for the true height of the tree crown is: GH=e×gh Among them, GH represents the real height of the tree crown; gh represents the pixel height of the tree crown; The formula for calculating the true width of the trunk is: D=e×d Where D represents the true width of the trunk; d represents the pixel width of the trunk.

Images