A method and device for detecting street tree lodging

Abstract

The application discloses a street tree lodging detection method and device, and belongs to the technical field of image recognition. The method comprises the following steps: acquiring a target detection image sequence corresponding to a target tree, wherein the target detection image sequence comprises a plurality of target detection images of different sampling periods, and the target detection image comprises the target tree; determining a trunk inclination angle of the target tree in the corresponding sampling period based on the target detection image; and determining a lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period. The method does not need to acquire a large number of features of the target tree to train a detection model, can obtain change information of the trunk inclination angle through a small number of target detection images, and can improve the detection efficiency by performing lodging detection according to the change information of the trunk inclination angle.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image recognition, and particularly relates to a street tree lodging detection method and device. BACKGROUND

[0002] Street trees are the backbone of urban road greening and an important part of urban green space systems. They are also the basic unit and carrier of building a smart garden. Street trees bring many positive benefits to the city, such as reducing noise, purifying air, and beautifying the city. However, due to external urban environment and plant growth conditions, street trees may experience lodging and other abnormal conditions, affecting normal urban traffic and even threatening the safety of residents' lives and property. Therefore, it is of great significance to monitor the tilting condition and potential lodging risk of street trees.

[0003] Currently, the monitoring of street tree lodging usually uses a regression analysis method. For example, the height of the tree trunk, the size of the tree crown, and other parameters are used as independent variables, the observation of the tree lodging is used as the dependent variable, and a machine learning method is used to develop fitting to achieve the prediction of potential lodging.

[0004] However, the machine learning method requires a large amount of observation data, and it is difficult to obtain data such as tree trunk height, tree crown size, and tree lodging observation, making it more difficult to train and verify the machine learning model, and the efficiency of lodging detection is low. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides a street tree lodging detection method and device, which can obtain the change information of the trunk inclination angle from a small amount of target detection images, and perform lodging detection according to the change information of the trunk inclination angle, thereby improving the detection efficiency.

[0006] In a first aspect, the present application provides a street tree lodging detection method, which comprises:

[0007] obtaining a target detection image sequence corresponding to a target tree, the target detection image sequence comprising a plurality of target detection images of different sampling periods, and the target detection image comprising the target tree;

[0008] determining the trunk inclination angle of the target tree at the corresponding sampling period based on the target detection image;

[0009] determining the lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period.

[0010] According to the street tree lodging detection method, the trunk inclination angle of the target tree is determined according to the target detection image sequence corresponding to the target tree, the probability of lodging of the target tree is determined according to the change speed of the trunk inclination angle and other characteristics, a large number of feature training detection models of the target tree are not required, the change characteristics of the trunk inclination angle can be obtained through a small number of target detection images, and the change characteristics of the trunk inclination angle are used for lodging detection, so that the detection efficiency can be improved.

[0011] According to an embodiment of the present application, the lodging detection result of the target tree is determined based on the trunk inclination angle corresponding to each sampling period.

[0012] The trunk inclination angle change information between two adjacent sampling periods is determined based on the trunk inclination angles corresponding to the two adjacent sampling periods.

[0013] The lodging detection result is determined based on the trunk inclination angle change information.

[0014] According to an embodiment of the present application, the lodging detection result is determined based on the trunk inclination angle change information, including:

[0015] In a case where the absolute value of the difference between the trunk inclination angles corresponding to all adjacent two sampling periods is less than a target threshold, it is determined that the target tree does not have lodging.

[0016] In a case where the absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling periods is greater than or equal to the target threshold, the lodging detection result is determined based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold.

[0017] According to an embodiment of the present application, the lodging detection result is determined based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold, including:

[0018] In a case where the absolute value of the difference between the trunk inclination angle corresponding to the last sampling period and the trunk inclination angle corresponding to the second last sampling period is greater than or equal to the target threshold, it is determined that the probability of the target tree having lodging is a first probability.

[0019] According to an embodiment of the present application, the lodging detection result is determined based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold, including:

[0020] In a case where the absolute values of the differences between the trunk inclination angles corresponding to all adjacent two of the sampling periods are greater than or equal to the target threshold value, the lodging detection result is determined based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent two of the sampling periods.

[0021] According to an embodiment of the present application, the determination of the lodging detection result based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent two of the sampling periods comprises:

[0022] In a case where the trunk inclination angle change absolute value or the trunk inclination angle change rate monotonously increases in the order corresponding to the sampling periods, it is determined that the target tree has lodging.

[0023] According to an embodiment of the present application, the determination of the lodging detection result based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent two of the sampling periods comprises:

[0024] In a case where the trunk inclination angle change absolute value and the trunk inclination angle change rate monotonously decrease in the order corresponding to the sampling periods, it is determined that the probability of the target tree having lodging is a second probability.

[0025] According to an embodiment of the present application, the obtaining of the target detection image sequence corresponding to the target tree comprises:

[0026] Obtaining a plurality of first detection images of the target tree in sampling periods;

[0027] Based on the plurality of first detection images, performing three-dimensional reconstruction on the target tree to obtain the target detection image corresponding to the sampling period;

[0028] Obtaining the target detection image corresponding to each of the sampling periods to obtain the target detection image sequence.

[0029] According to an embodiment of the present application, the determination of the trunk inclination angle of the target tree in the corresponding sampling period based on the target detection image comprises:

[0030] Performing image segmentation on the target detection image to obtain a trunk region in the target detection image;

[0031] Fitting a target circle corresponding to the trunk region by a circle fitting algorithm and determining a center of the target circle;

[0032] Determining the trunk inclination angle based on the center of the target circle.

[0033] In a second aspect, the present application provides a street tree lodging detection device, which comprises:

[0034] an acquisition module, configured to acquire a target detection image sequence corresponding to a target tree, the target detection image sequence comprising a plurality of target detection images of different sampling time periods, and the target detection images comprising the target tree;

[0035] a first processing module, configured to determine a trunk inclination angle of the target tree in the corresponding sampling time period based on the target detection images;

[0036] a second processing module, configured to determine a lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling time period.

[0037] According to the street tree lodging detection device of the present application, the trunk inclination angle of the target tree is determined based on the target detection image sequence corresponding to the target tree, the probability of lodging of the target tree is determined according to the change speed of the trunk inclination angle, and a large number of feature training detection models of the target tree are not required, the change feature of the trunk inclination angle can be obtained through a small number of target detection images, and the lodging detection is performed according to the change feature of the trunk inclination angle, so that the detection efficiency can be improved.

[0038] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize the street tree lodging detection method of the first aspect.

[0039] In a fourth aspect, the present application provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the street tree lodging detection method of the first aspect.

[0040] In a fifth aspect, the present application provides a computer program product, comprising a computer program, and the computer program is executed by a processor to realize the street tree lodging detection method of the first aspect.

[0041] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter in the description of the application. BRIEF DESCRIPTION OF DRAWINGS

[0042] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the description of the embodiments, given in conjunction with the accompanying drawings, in which:

[0043] Figure 1 is one of the flowcharts of the street tree lodging detection method provided by the embodiments of the present application.

[0044] Figure 2 FIG. 2 is a flowchart of a method for detecting tree lodging according to an embodiment of the present application;

[0045] Figure 3 FIG. 3 is a flowchart of a method for detecting tree lodging according to an embodiment of the present application;

[0046] Figure 4 FIG. 4 is a flowchart of a method for detecting tree lodging according to an embodiment of the present application;

[0047] Figure 5 FIG. 5 is a structural diagram of a device for detecting tree lodging according to an embodiment of the present application;

[0048] Figure 6 FIG. 6 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0050] The terms “first”, “second”, and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be exchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by “first”, “second”, and the like are generally of a kind and are not limited to the number of objects, for example, the first object can be one or more. In addition, “and / or” in the specification and claims indicates at least one of the connected objects, and the character “ / ” generally indicates that the front and rear associated objects are in an “or” relationship.

[0051] The method for detecting tree lodging, the device for detecting tree lodging, the electronic device, and the readable storage medium provided by the embodiments of the present application will be described in detail below with reference to the drawings and specific embodiments and application scenarios.

[0052] The method for detecting tree lodging can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.

[0053] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad). It will also be appreciated that, in some embodiments, the terminal can not be a portable communication device, but rather a desktop computer having a touch-sensitive surface (e.g., a touchscreen display and / or touchpad).

[0054] In the following embodiments, a terminal comprising a display and a touch-sensitive surface is described. However, it will be appreciated that the terminal can comprise one or more other physical user interface devices such as physical keyboards, mice and joysticks.

[0055] The execution subject of the sidewalk tree lodging detection method provided in the embodiments of the present application can be an electronic device or a function module or function entity capable of implementing the sidewalk tree lodging detection method in the electronic device. The electronic device mentioned in the embodiments of the present application includes but is not limited to a mobile phone, a tablet computer, a computer, a camera, a wearable device, and the like. The sidewalk tree lodging detection method provided in the embodiments of the present application will be described below by taking an electronic device as an execution subject.

[0056] As shown in Figure 1 The sidewalk tree lodging detection method comprises steps 110, 120 and 130.

[0057] In step 110, a target detection image sequence corresponding to a target tree is acquired. The target detection image sequence comprises a plurality of target detection images of different sampling periods, and the target detection images comprise the target tree.

[0058] The target tree is a tree to be subjected to lodging detection, and the target tree can be a sidewalk tree on both sides of a road.

[0059] The target detection image sequence comprises a plurality of target detection images arranged in time sequence. A plurality of sampling periods can be set, and one target detection image is acquired in each sampling period. The two adjacent sampling periods can be separated by several days or several weeks, for example, three weeks. That is, the target tree is sampled once every three weeks to obtain one target detection image. The target detection images are arranged in time sequence according to the sampling periods to obtain the target detection image sequence.

[0060] It should be noted that the target detection image can be a three-dimensional image, which can provide three-dimensional spatial information of the target tree. The target detection image can include part or all information of the target tree, and at least includes information of the trunk. The position of the target tree in each target detection image can be the same.

[0061] In this step, the target detection image can be obtained by means of point cloud generation and the like in a sampling period, and the target detection images obtained in each sampling period are arranged in time sequence to obtain a target detection image sequence.

[0062] In step 120, a trunk inclination angle of the target tree in the corresponding sampling period is determined based on the target detection image.

[0063] The trunk inclination angle can be an angle between a direction of the trunk main shaft and a set reference direction, and the reference direction can be a direction parallel to the ground or a direction perpendicular to the ground.

[0064] In this step, the trunk region of the target tree in the target detection image can be extracted by image segmentation or target detection, the topmost end and the bottommost end of the trunk are identified, the main shaft of the trunk is determined by using a straight line fitting algorithm, and the angle between the direction of the main shaft and the reference direction is calculated as the trunk inclination angle.

[0065] In step 130, a lodging detection result of the target tree is determined based on the trunk inclination angle corresponding to each sampling period.

[0066] The lodging detection result is used to represent whether the target tree has occurred lodging or a probability of the target tree occurring lodging.

[0067] In this embodiment, the change trend of the trunk inclination angle with time can be analyzed based on the trunk inclination angle corresponding to each sampling period, whether there is a continuously increasing or decreasing trend is identified, the change speed and change acceleration are identified, the abnormal change and sudden change of the trunk inclination angle are identified, and the lodging detection result is determined.

[0068] In the related art, the monitoring of street tree lodging usually uses a regression analysis method, for example, using the trunk height, crown size and other parameters of the tree as independent variables, using the lodging observation of the tree as dependent variables, and using a machine learning method to perform fitting to realize the prediction of potential lodging.

[0069] However, the fitting based on the machine learning method needs a large amount of observation data, and the trunk height, crown size and other data of the tree are difficult to obtain, which makes the training and verification of the machine learning model more difficult, and the efficiency of the lodging detection is low.

[0070] In the embodiment of the present application, the trunk inclination angle of the target tree changes with time is determined through the target detection image sequence corresponding to the target tree, and the probability of the target tree existing lodging is determined according to the change speed of the trunk inclination angle and other characteristics. The detection model does not need to obtain a large number of characteristics of the target tree, and the change characteristics of the trunk inclination angle can be obtained through a small number of target detection images. The lodging detection is performed according to the change characteristics of the trunk inclination angle, which can improve the detection efficiency.

[0071] In the related art, the artificial measurement method is used to carry out the inclination measurement of the tree with professional instruments. The traditional artificial measurement method uses professional measurement equipment such as a measuring instrument to carry out tree parameter measurement, and then calculates the inclination of the tree. The working efficiency is low, and the professional equipment required is complex.

[0072] In the embodiment of the present application, the measurement work is not carried out manually using a measuring tool. The target detection image sequence is obtained using an image acquisition device such as a mobile phone or a camera. The lodging detection is performed according to the target detection image sequence. Compared with the traditional artificial measurement method, the efficiency is higher and the equipment cost is lower.

[0073] In the related art, the ground penetrating radar is used to detect the root system and the trunk cavity of the tree, and then the possibility of tree lodging is calculated according to the obtained parameters. The efficiency is low. It takes several hours to detect a tree. The ground penetrating radar and other equipment are expensive and difficult to operate. In addition, the hollow condition of the tree obtained by the ground penetrating radar is only the physiological and structural condition of the tree, which is not the actual inclination change data. There is a certain relationship between the tree lodging, but it is not the actual inclination or lodging state of the tree.

[0074] In the embodiment of the present application, the root system, trunk cavity and other information of the tree do not need to be detected. The target detection image sequence is obtained by photographing the target tree at multiple sampling periods. The time sequence change of the trunk inclination angle of the target tree is determined based on the target detection image sequence corresponding to the target tree. The probability of the target tree existing lodging is determined according to the change trend and change speed of the trunk inclination angle. The efficiency is higher, the equipment requirement is low, and the image acquisition device such as a mobile phone or a camera can be used to carry out the work. In addition, the possible lodging condition of the target tree is described in the time dimension. The trunk inclination angle of multiple sampling periods is calculated by using the observation information of multiple sampling periods. Thus, the actual trunk inclination angle change of the target tree can be effectively obtained. The lodging probability is predicted according to the set determination rule. Compared with the ground penetrating radar and other methods, it is closer to the actual change condition of the tree.

[0075] According to the street tree lodging detection method provided in the embodiment of the present application, the trunk inclination angle of the target tree changes with time is determined through the target detection image sequence corresponding to the target tree, and the probability of the target tree existing lodging is determined according to the change speed and other characteristics of the trunk inclination angle, so that the detection model does not need to be trained by a large number of characteristics of the target tree, the change characteristics of the trunk inclination angle can be obtained through a small number of target detection images, and the lodging detection is performed according to the change characteristics of the trunk inclination angle, so that the detection efficiency can be improved.

[0076] In some embodiments, as shown in Figure 2 Step 130, the lodging detection result of the target tree is determined based on the trunk inclination angle corresponding to each sampling period, including:

[0077] Step 131, the trunk inclination angle change information between the adjacent two sampling periods is determined based on the trunk inclination angles corresponding to the adjacent two sampling periods.

[0078] Step 132, the lodging detection result is determined based on the trunk inclination angle change information.

[0079] The trunk inclination angle change information is used to represent the change trend, change degree and change speed of the trunk of the target tree between the adjacent two sampling periods.

[0080] In this embodiment, the trunk inclination angles corresponding to the adjacent two sampling periods can be subtracted or subjected to other mathematical operations, and the trunk inclination angle change information is obtained according to the calculation result.

[0081] In this embodiment, the change degree and change speed of the trunk inclination angle in the overall time sequence can be analyzed based on the trunk inclination angle change information, and compared with the corresponding set threshold value, and the lodging detection result is determined according to the comparison result.

[0082] For example, when the change degree of the trunk inclination angle is greater than the corresponding threshold value, it is determined that the target tree has lodging, and when the change degree of the trunk inclination angle is less than the corresponding threshold value, it is determined that the target tree has no lodging.

[0083] For another example, the change degree of the trunk inclination angle between the corresponding two sampling periods can also be determined according to the trunk inclination angle change information, all the trunk inclination angle change information is analyzed, and the lodging detection result is determined according to the number of times that the change degree of the trunk inclination angle exceeds the set threshold value.

[0084] In some embodiments, as shown in Figure 3 and Figure 4 Step 132, the lodging detection result is determined based on the trunk inclination angle change information, including:

[0085] Step 132a, in the case that the absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling time periods is less than the target threshold value, it is determined that the target tree does not have lodging.

[0086] Step 132b, in the case that the absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling time periods is greater than or equal to the target threshold value, the lodging detection result is determined based on the sampling time period corresponding to the absolute value of the difference greater than or equal to the target threshold value.

[0087] The absolute value of the difference is the absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling time periods, and the target threshold value is a pre-set value.

[0088] In this embodiment, the absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling time periods is less than the target threshold value, indicating that the trunk inclination angle has a small degree of change or almost no change between the two adjacent sampling time periods.

[0089] The absolute value of the difference between the trunk inclination angles corresponding to all adjacent sampling time periods is less than the target threshold value, indicating that the trunk inclination angle has a small degree of change or almost no change in the entire measurement time sequence, and it is determined that the target tree does not have lodging.

[0090] The absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling time periods is greater than or equal to the target threshold value, indicating that the trunk inclination angle has a large degree of change between the two adjacent sampling time periods.

[0091] The absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling time periods is greater than or equal to the target threshold value, indicating that there is a large degree of change in the trunk inclination angle in the entire measurement time sequence, and the target tree has a possibility of lodging. Further judgment is made based on the sampling time period corresponding to the absolute value of the difference greater than or equal to the target threshold value.

[0092] In this embodiment, the lodging detection result can be determined according to the number or corresponding time sequence of the sampling time period corresponding to the absolute value of the difference greater than or equal to the target threshold value.

[0093] In some embodiments, the lodging detection result is determined based on the sampling time period corresponding to the absolute value of the difference greater than or equal to the target threshold value, including:

[0094] In the case that the absolute value of the difference between the trunk inclination angle corresponding to the last sampling time period and the trunk inclination angle corresponding to the second last sampling time period is greater than or equal to the target threshold value, it is determined that the probability of the target tree having lodging is the first probability.

[0095] In this embodiment, the absolute value of the difference between the trunk inclination angle corresponding to the last sampling period and the trunk inclination angle corresponding to the second last sampling period is greater than or equal to the target threshold, indicating that the trunk inclination angle may have a sudden change, and the target tree may have lodging, and the first probability can be 50%.

[0096] In some embodiments, the lodging detection result is determined based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold, including:

[0097] In the case where the absolute value of the difference between the trunk inclination angles corresponding to all adjacent two sampling periods is greater than or equal to the target threshold, the lodging detection result is determined based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent two sampling periods.

[0098] Wherein, the trunk inclination angle change absolute value is the absolute value of the change value between the trunk inclination angles between the adjacent two sampling periods, which can be equal to the absolute value of the difference, and the trunk inclination angle change rate is the change rate between the trunk inclination angles between the adjacent two sampling periods, which can be equal to the ratio of the trunk inclination angle change absolute value to the time interval between the two sampling periods.

[0099] In this embodiment, the absolute value of the difference between the trunk inclination angles corresponding to all adjacent two sampling periods is greater than or equal to the target threshold, indicating that the trunk inclination angle has a large degree of change in the whole measurement time sequence, and the target tree has a possibility of lodging, and the lodging detection result can be determined according to the trunk inclination angle change absolute value and the trunk inclination angle change rate in the time sequence.

[0100] In some embodiments, the lodging detection result is determined based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent two sampling periods, including:

[0101] In the case where the trunk inclination angle change absolute value or the trunk inclination angle change rate monotonically increases in the order of the sampling periods, it is determined that the target tree has lodging.

[0102] The trunk inclination angle change absolute value or the trunk inclination angle change rate monotonically increases in the order of the sampling periods, indicating that the trunk inclination angle has a larger and larger degree of change or a faster and faster change speed in the whole measurement time sequence, and it is determined that the target tree has lodging.

[0103] In some embodiments, the lodging detection result is determined based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent two sampling periods, including:

[0104] In a case where the absolute value of the trunk inclination angle change and the trunk inclination angle change rate monotonically decrease in the order corresponding to the sampling time period, the probability of the target tree being lodged is determined as the second probability.

[0105] The absolute value of the trunk inclination angle change and the trunk inclination angle change rate monotonically decrease in the order corresponding to the sampling time period, indicating that the trunk inclination angle changes less and less and changes slower and slower over the entire measurement time sequence. The probability of the target tree being lodged is determined as the second probability, which can be 75%.

[0106] A specific embodiment of determining the lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling time period is introduced below.

[0107] Three sampling time periods are set, namely sampling time period 1, sampling time period 2, and sampling time period 3. The trunk inclination angles corresponding to the three sampling time periods are a1, a2, and a3, respectively.

[0108] The absolute value of the trunk inclination angle change k1 and the trunk inclination angle change rate v1 between the sampling time period 1 and the sampling time period 2 are calculated, and the absolute value of the trunk inclination angle change k2 and the trunk inclination angle change rate v2 between the sampling time period 2 and the sampling time period 3 are calculated. The trunk inclination angle estimation error is defined as ε, ε ∈ (-3, 3). It can be understood that the trunk inclination angle estimation error can be determined based on a target threshold.

[0109] In a case where a1±ε=a2±ε=a3±ε, it is determined that the target tree is not lodged.

[0110] In a case where a1±ε=a2±ε≠a3±ε, it is considered that further observation is needed, and the probability of the target tree being lodged is 50%.

[0111] In a case where a1±ε≠a2±ε≠a3±ε, when k1

[0112] When k1>k2 and v1>v2, it is determined that the probability of the target tree being lodged is 75%.

[0113] In some embodiments, the target detection image sequence corresponding to the target tree is obtained, including:

[0114] A plurality of first detection images of the target tree in the sampling time period are obtained.

[0115] Based on the plurality of first detection images, the target tree is three-dimensionally reconstructed to obtain the target detection image corresponding to the sampling time period.

[0116] The target detection image corresponding to each sampling time period is obtained to obtain the target detection image sequence.

[0117] The first detection image is a two-dimensional image obtained by a mobile phone or a camera or other image acquisition device, and the first detection image includes a complete target tree.

[0118] In this embodiment, the first detection image can be obtained by a handheld shooting device under windless conditions, and the coordinates of the target tree can be obtained. A drone or a smart car carrying a shooting device is controlled to move to the target tree to collect the first detection image.

[0119] In this embodiment, 6-10 first detection images can be collected in one sampling period, each first detection image has a resolution of not less than 1 cm, and the information overlap rate between two adjacent first detection images is not less than 60%.

[0120] In actual execution, the target tree can be reconstructed in three dimensions based on multiple first detection images by using an open source computer vision library (OpenCV) to obtain an orthographic image, i.e., a target detection image, so as to realize the three-dimensional space of the target tree.

[0121] A specific embodiment of reconstructing the target tree in three dimensions by using OpenCV is introduced below.

[0122] Step 1: Read multiple first detection images by using OpenCV.

[0123] Step 2: Extract feature points of each first detection image. The scale-invariant feature transform (SIFT) algorithm can be used in OpenCV to extract the feature points of the first detection image.

[0124] Step 3: Use a brute-force matcher (BFMatcher) in OpenCV to match homonymous points between two adjacent first detection images based on the extracted feature points. The homonymous points can be feature points representing the same position of the target tree in different first detection images.

[0125] Step 4: Estimate the internal and external parameters of the corresponding camera based on the matched homonymous points by using an open source structure from motion (OpenSFM).

[0126] Step five, based on the determined internal and external parameters of the camera and multiple first detection images, a three-dimensional model of the target tree is reconstructed by an open multi-view stereo reconstruction library (OpenMVS).

[0127] Step six, based on the three-dimensional model of the target tree, a target detection image under a horizontal perspective is generated and exported by an orthographic generation function of an OpenCV design module (ODM).

[0128] In some embodiments, based on the target detection image, a trunk inclination angle of the target tree in a corresponding sampling period is determined, including:

[0129] The target detection image is image segmented to obtain a trunk region in the target detection image.

[0130] A target circle corresponding to the trunk region is fitted by a circle fitting algorithm, and a center of the target circle is determined.

[0131] Based on the center of the target circle, the trunk inclination angle is determined.

[0132] The trunk region is a region in which a trunk of the target tree is located in the target detection image.

[0133] In this embodiment, the target detection image can be image segmented by the trained trunk segmentation model to obtain the trunk region, the initial value of the center and the radius is estimated according to the geometric center point of the trunk contour and the point on the contour, and the fitting is optimized to find the target circle that can surround the trunk contour, the center of the target circle is connected with the root of the target tree and other set reference points, and the angle between the connection direction and the reference direction is taken as the trunk inclination angle.

[0134] A specific embodiment for determining the trunk inclination angle is introduced below.

[0135] Step one, manually plot the trunk on the orthographic image of the tree to form a trunk training sample with a number of trunks not less than 1000.

[0136] Step two, based on the trunk training sample, a feature refinement network (FR-Net) is taken as an artificial intelligence network to train the FR-Net to obtain a trunk segmentation model that can accurately segment the trunk.

[0137] Step three, the trunk segmentation of the target tree is completed by using the trained FR-Net.

[0138] Step four, the segmented trunk is evenly divided into a trunk head, a trunk middle and a trunk tail according to length.

[0139] Step five, a target circle corresponding to the trunk middle is fitted through a circle fitting algorithm.

[0140] Step six, a line is drawn between the center of the target circle and a point corresponding to the root of the target tree, and an included angle between the line and a horizontal plane is calculated as a trunk inclination angle.

[0141] The sidewalk tree lodging detection method provided in the embodiments of the present application can be executed by a sidewalk tree lodging detection device. The sidewalk tree lodging detection method is executed by the sidewalk tree lodging detection device in the embodiments of the present application, and the sidewalk tree lodging detection device provided in the embodiments of the present application is described.

[0142] The embodiments of the present application also provide a sidewalk tree lodging detection device.

[0143] As shown in the figure, the sidewalk tree lodging detection device comprises: Figure 5 The acquisition module 510 is configured to acquire a target detection image sequence corresponding to a target tree, the target detection image sequence comprising a plurality of target detection images of different sampling periods, and the target detection images comprising the target tree;

[0144] The first processing module 520 is configured to determine a trunk inclination angle of the target tree in a corresponding sampling period based on the target detection images.

[0145] The second processing module 530 is configured to determine a lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period.

[0146] According to the sidewalk tree lodging detection device provided in the embodiments of the present application, the trunk inclination angle of the target tree is determined to change with time based on the target detection image sequence corresponding to the target tree, the probability of lodging of the target tree is determined based on the change speed of the trunk inclination angle, a detection model is not required to be trained based on a large number of features of the target tree, the change feature of the trunk inclination angle can be obtained based on a small number of target detection images, and lodging detection is performed based on the change feature of the trunk inclination angle, so that the detection efficiency can be improved.

[0147] In some embodiments, the second processing module 530 is configured to determine trunk inclination angle change information between two adjacent sampling periods based on the trunk inclination angles corresponding to the two adjacent sampling periods.

[0148] The lodging detection result is determined based on the trunk inclination angle change information.

[0149]

[0150] ​In some embodiments, the second processing module 530 is configured to determine that the target tree does not fall in a case where absolute values of differences between trunk inclination angles corresponding to all adjacent sampling periods are less than a target threshold value.

[0151] In a case where the absolute value of the difference is greater than or equal to the target threshold value, the falling detection result is determined based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold value.

[0152] In some embodiments, the second processing module 530 is configured to determine that the target tree falls with a first probability in a case where an absolute value of a difference between a trunk inclination angle corresponding to a last sampling period and a trunk inclination angle corresponding to a second last sampling period is greater than or equal to a target threshold value.

[0153] In some embodiments, the second processing module 530 is configured to determine the falling detection result based on the absolute value of the change in the trunk inclination angle and the change rate of the trunk inclination angle between all adjacent sampling periods in a case where the absolute values of the differences between the trunk inclination angles corresponding to all adjacent sampling periods are all greater than or equal to the target threshold value.

[0154] In some embodiments, the second processing module 530 is configured to determine that the target tree falls in a case where the absolute value of the change in the trunk inclination angle or the change rate of the trunk inclination angle is monotonically increasing in order of the sampling periods.

[0155] In some embodiments, the second processing module 530 is configured to determine that the target tree falls with a second probability in a case where the absolute value of the change in the trunk inclination angle or the change rate of the trunk inclination angle is monotonically decreasing in order of the sampling periods.

[0156] In some embodiments, the acquisition module 510 is configured to acquire a plurality of first detection images of the target tree in the sampling period.

[0157] The target tree is three-dimensionally reconstructed based on the plurality of first detection images to obtain a target detection image corresponding to the sampling period.

[0158] The target detection image corresponding to each sampling period is obtained to obtain a target detection image sequence.

[0159] In some embodiments, the first processing module 520 is configured to perform image segmentation on the target detection image to obtain a trunk region in the target detection image.

[0160] A target circle corresponding to the trunk region is fitted by a circle fitting algorithm, and a center of the target circle is determined.

[0161] The trunk inclination angle is determined based on the center of the target circle.

[0162] The street tree lodging detection device in the embodiments of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and can also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the like, and the embodiments of the present application are not limited in this regard.

[0163] The street tree lodging detection device in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, an IOS operating system, or other possible operating systems, and the embodiments of the present application are not limited in this regard.

[0164] The street tree lodging detection device provided in the embodiments of the present application can implement the method embodiments Figures 1 to 4 The processes implemented by the method embodiments are not repeated here to avoid repetition.

[0165] In some embodiments, as shown in Figure 6 The embodiments of the present application also provide an electronic device 600, which includes a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, the processes of the above street tree lodging detection method embodiments are implemented, and the same technical effects are achieved. The processes are not repeated here to avoid repetition.

[0166] It should be noted that the electronic device in the embodiments of the present application includes the mobile electronic device and the non-mobile electronic device described above.

[0167] The embodiment of the present application further provides a non-transitory computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement each process of the street tree lodging detection method embodiment and achieve the same technical effects. To avoid repetition, details are not described herein.

[0168] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0169] The embodiment of the present application further provides a computer program product, which includes a computer program. The computer program is executed by a processor to implement the street tree lodging detection method.

[0170] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0171] The embodiment of the present application further provides a chip, which includes a processor and a communication interface. The communication interface is coupled with the processor. The processor is used to run a program or an instruction to implement each process of the street tree lodging detection method and achieve the same technical effects. To avoid repetition, details are not described herein.

[0172] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system level chip, a system chip, a chip system or a system on chip, etc.

[0173] It should be noted that, in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or apparatus including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article or apparatus including the element. In addition, it should be pointed out that the scope of the method and apparatus in the present application is not limited to the order of performing the functions as shown or discussed, but can also include performing the functions in a substantially simultaneous manner or in a reverse order, for example, the described method can be performed in an order different from that described, and various steps can be added, omitted or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned example methods can be realized by means of software and necessary general hardware platforms, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk), and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device) to execute the methods described in the various embodiments of the present application.

[0175] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, rather than limiting. Those skilled in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection of the present application.

[0176] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an illustrative embodiment", "an example", "a specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0177] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements, and variations can be made to the embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A method of detecting tree uprooting, characterized by, The method comprises the following steps: obtaining a target detection image sequence corresponding to a target tree, the target detection image sequence comprising a plurality of target detection images of different sampling periods, and the target detection images comprising the target tree; determining a trunk inclination angle of the target tree at the corresponding sampling period based on the target detection image; determining a lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period; the step of determining the lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period comprises the following steps: determining trunk inclination angle change information between two adjacent sampling periods based on the trunk inclination angles corresponding to the two adjacent sampling periods; determining the lodging detection result based on the trunk inclination angle change information; the step of determining the lodging detection result based on the trunk inclination angle change information comprises the following steps: in the case that the absolute value of the difference between the trunk inclination angles corresponding to all adjacent sampling periods is less than a target threshold value, it is determined that the target tree does not lodge; in the case that the absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling periods is greater than or equal to the target threshold value, the lodging detection result is determined based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold value; the step of determining the lodging detection result based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold value comprises the following steps: in the case that the absolute value of the difference between the trunk inclination angles corresponding to all adjacent sampling periods is greater than or equal to the target threshold value, the lodging detection result is determined based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent sampling periods.

2. The street tree leaning detection method of claim 1, wherein, the step of determining the lodging detection result based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold value comprises the following steps: in the case that the absolute value of the difference between the trunk inclination angle corresponding to the last sampling period and the trunk inclination angle corresponding to the second last sampling period is greater than or equal to the target threshold value, it is determined that the probability of the target tree lodging is a first probability.

3. The street tree leaning detection method of claim 1, wherein, the step of determining the lodging detection result based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent sampling periods comprises the following steps: in the case that the trunk inclination angle change absolute value or the trunk inclination angle change rate monotonically increases in the order of the sampling periods, it is determined that the target tree lodges.

4. The street tree leaning detection method of claim 1, wherein, the step of determining the lodging detection result based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent sampling periods comprises the following steps: in the case that the trunk inclination angle change absolute value and the trunk inclination angle change rate monotonically decrease in the order of the sampling periods, it is determined that the probability of the target tree lodging is a second probability.

5. The street tree leaning detection method according to any one of claims 1-4, wherein, the step of obtaining a target detection image sequence corresponding to a target tree comprises the following steps: obtaining a plurality of first detection images of the target tree at a sampling period; based on the plurality of first detection images, three-dimensional reconstruction is performed on the target tree to obtain the target detection image corresponding to the sampling period; obtaining the target detection image corresponding to each sampling period to obtain the target detection image sequence.

6. The street tree leaning detection method according to any one of claims 1-4, wherein, determining the trunk inclination angle of the target tree at the corresponding sampling period based on the target detection image, comprising: performing image segmentation on the target detection image to obtain the trunk region in the target detection image; fitting a target circle corresponding to the trunk region through a circle fitting algorithm and determining the center of the target circle; determining the trunk inclination angle based on the center of the target circle.

7. A street tree windfall detection apparatus, characterised in that, comprising: an acquisition module configured to obtain a target detection image sequence corresponding to a target tree, the target detection image sequence comprising a plurality of target detection images at different sampling periods, and the target detection image comprising the target tree; a first processing module configured to determine the trunk inclination angle of the target tree at the corresponding sampling period based on the target detection image; a second processing module configured to determine the lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period; the second processing module is further configured to determine the trunk inclination angle change information between two adjacent sampling periods based on the trunk inclination angles corresponding to the two adjacent sampling periods; determining the lodging detection result based on each trunk inclination angle change information; the second processing module is further configured to: in the case that the absolute value of the difference between the trunk inclination angles corresponding to all adjacent sampling periods is less than a target threshold, determining that the target tree has not been lodged; in the case that the absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling periods is greater than or equal to the target threshold, determining the lodging detection result based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold; the second processing module is further configured to: in the case that the absolute value of the difference between the trunk inclination angles corresponding to all adjacent sampling periods is greater than or equal to the target threshold, determining the lodging detection result based on the trunk inclination angle change absolute value and the trunk inclination angle change rate between all adjacent sampling periods.

Images