Street tree lodging detection method and device

Abstract

The invention discloses a street tree lodging detection method and device, and belongs to the technical field of image recognition. The method comprises the steps that a target detection image sequence corresponding to a target tree is acquired, the target detection image sequence comprises a plurality of target detection images in different sampling periods, and the target detection images comprise the target tree; based on the target detection image, determining a trunk inclination angle of the target tree in the corresponding sampling period; and determining a lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period. The method does not need to obtain a large number of feature training detection models of the target tree, can obtain the change information of the trunk inclination angle through a small number of target detection images, carries out lodging detection according to the change information of the trunk inclination angle, and can improve the detection efficiency.

Description

Technical Field

[0001] This application belongs to the technical field of image recognition, and particularly relates to a method and device for detecting the lodging of roadside trees. Background Art

[0002] Roadside trees are the backbone of urban road greening, an important part of the urban green space system, and also the basic unit and carrier for building smart gardens. Roadside trees bring many positive benefits to the city, such as reducing noise, purifying the air, and beautifying the city. However, affected by the external urban environment and the growth conditions of the plants themselves, abnormal conditions such as lodging may occur to roadside trees, affecting normal urban traffic and even threatening the personal and property safety of residents. Therefore, it is of great significance to monitor the inclination of roadside trees and potential lodging risks.

[0003] Currently, the monitoring of roadside tree lodging usually uses the method of regression analysis. For example, parameters such as the trunk height and crown size of the tree are used as independent variables, and the observed lodging situation of the tree is used as the dependent variable, and fitting is carried out based on machine learning methods to realize the prediction of potential lodging situations.

[0004] However, fitting based on machine learning methods requires a large amount of observation data, and it is difficult to obtain data such as trunk height, crown size, and the observed lodging situation of the tree, making the training and verification of machine learning models more difficult and the efficiency of lodging detection lower. Summary of the Invention

[0005] This application aims to at least solve one of the technical problems existing in the prior art. For this purpose, this application proposes a method and device for detecting the lodging of roadside trees, which can obtain the change information of the trunk inclination angle from a small number of object detection images, and perform lodging detection based on the change information of the trunk inclination angle, thereby improving the detection efficiency.

[0006] In a first aspect, this application provides a method for detecting the lodging of roadside trees, and the method includes:

[0007] Obtain a sequence of object detection images corresponding to a target tree, where the sequence of object detection images includes multiple object detection images at different sampling time periods, and the object detection images include the target tree;

[0008] Based on the object detection images, determine the trunk inclination angle of the target tree at the corresponding sampling time period;

[0009] Based on the trunk inclination angle corresponding to each sampling time period, determine the lodging detection result of the target tree.

[0010] According to the street tree lodging detection method of the present application, by means of the target detection image sequence corresponding to the target tree, the change of the trunk inclination angle of the target tree over time is determined. According to characteristics such as the change speed of the trunk inclination angle, the probability of the target tree lodging is determined. It is not necessary to obtain a large number of features of the target tree to train a detection model. The change characteristics of the trunk inclination angle can be obtained through a small number of target detection images, and lodging detection can be carried out according to the change characteristics of the trunk inclination angle, which can improve the detection efficiency.

[0011] According to an embodiment of the present application, determining the lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period includes:

[0012] Based on the trunk inclination angles corresponding to two adjacent sampling periods, determine the change information of the trunk inclination angle between the two adjacent sampling periods;

[0013] Based on each piece of the trunk inclination angle change information, determine the lodging detection result.

[0014] According to an embodiment of the present application, determining the lodging detection result based on each piece of the trunk inclination angle change information includes:

[0015] In the case where the absolute value of the difference between the trunk inclination angles corresponding to all adjacent two sampling periods is less than the target threshold, it is determined that the target tree has not lodged;

[0016] In the case where there is an absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling periods that is greater than or equal to the target threshold, based on the sampling period corresponding to the absolute value of the difference that is greater than or equal to the target threshold, determine the lodging detection result.

[0017] According to an embodiment of the present application, determining the lodging detection result based on the sampling period corresponding to the absolute value of the difference that is greater than or equal to the target threshold includes:

[0018] In the case where the absolute value of the difference between the trunk inclination angle corresponding to the last sampling period and the trunk inclination angle corresponding to the penultimate sampling period is greater than or equal to the target threshold, it is determined that the probability of the target tree lodging is the first probability.

[0019] According to an embodiment of the present application, determining the lodging detection result based on the sampling period corresponding to the absolute value of the difference that is greater than or equal to the target threshold includes:

[0020] When the absolute value of the difference between the trunk tilt angles corresponding to all adjacent sampling periods is greater than or equal to the target threshold, determine the lodging detection result based on the absolute value of the change in the trunk tilt angle and the trunk tilt angle change rate between all adjacent sampling periods.

[0021] According to an embodiment of the present application, the determining the lodging detection result based on the absolute value of the change in the trunk tilt angle and the trunk tilt angle change rate between all adjacent sampling periods includes:

[0022] When the absolute value of the change in the trunk tilt angle or the trunk tilt angle change rate monotonically increases in the order corresponding to the sampling periods, determine that the target tree has lodged.

[0023] According to an embodiment of the present application, the determining the lodging detection result based on the absolute value of the change in the trunk tilt angle and the trunk tilt angle change rate between all adjacent sampling periods includes:

[0024] When the absolute value of the change in the trunk tilt angle and the trunk tilt angle change rate monotonically decrease in the order corresponding to the sampling periods, determine that the probability of the target tree lodging is the second probability.

[0025] According to an embodiment of the present application, the obtaining the target detection image sequence corresponding to the target tree includes:

[0026] Obtain multiple first detection images of the target tree in the sampling period;

[0027] Based on the multiple first detection images, perform three-dimensional reconstruction on the target tree to obtain the target detection image corresponding to the sampling period;

[0028] Obtain the target detection image corresponding to each sampling period to obtain the target detection image sequence.

[0029] According to an embodiment of the present application, the determining the trunk tilt angle of the target tree in the corresponding sampling period based on the target detection image includes:

[0030] Perform image segmentation on the target detection image to obtain the trunk area in the target detection image;

[0031] By the circle fitting algorithm, fit the target circle corresponding to the trunk area and determine the center of the target circle;

[0032] Based on the center of the target circle, determine the trunk tilt angle.

[0033] Second aspect, the present application provides a device for detecting the lodging of roadside trees, and the device includes:

[0034] An acquisition module, configured to acquire a target detection image sequence corresponding to a target tree, where the target detection image sequence includes multiple target detection images at different sampling time periods, and the target detection images include the target tree;

[0035] A first processing module, configured to determine the trunk inclination angle of the target tree at the corresponding sampling time period based on the target detection image;

[0036] A second processing module, configured to determine the lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling time period.

[0037] According to the device for detecting the lodging of roadside trees of the present application, by the target detection image sequence corresponding to the target tree, the change of the trunk inclination angle of the target tree over time is determined. According to features such as the change speed of the trunk inclination angle, the probability of the target tree lodging is determined. It is not necessary to obtain a large number of features of the target tree to train a detection model. The change features of the trunk inclination angle can be obtained through a small number of target detection images. The lodging detection is performed according to the change features of the trunk inclination angle, which can improve the detection efficiency.

[0038] Third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for detecting the lodging of roadside trees as described in the first aspect above is implemented.

[0039] Fourth aspect, the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting the lodging of roadside trees as described in the first aspect above is implemented.

[0040] Fifth aspect, the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the method for detecting the lodging of roadside trees as described in the first aspect above is implemented.

[0041] The additional aspects and advantages of the present application will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present application. Description of the Drawings

[0042] The above and / or additional aspects and advantages of the present application will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, where:

[0043] Figure 1 is one of the flow diagrams of the method for detecting the lodging of roadside trees provided by the embodiments of the present application;

[0044] Figure 2 It is the second schematic flow chart of the street tree lodging detection method provided by the embodiment of the present application;

[0045] Figure 3 It is the third schematic flow chart of the street tree lodging detection method provided by the embodiment of the present application;

[0046] Figure 4 It is the fourth schematic flow chart of the street tree lodging detection method provided by the embodiment of the present application;

[0047] Figure 5 It is the schematic structural diagram of the street tree lodging detection device provided by the embodiment of the present application;

[0048] Figure 6 It is the schematic structural diagram of the electronic device provided by the embodiment of the present application. Detailed Embodiments

[0049] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0050] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0051] Next, the street tree lodging detection method, street tree lodging detection device, electronic device, and readable storage medium provided by the embodiments of the present application will be described in detail in conjunction with the accompanying drawings through specific embodiments and their application scenarios.

[0052] Among them, the street tree lodging detection method can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.

[0053] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablet computers having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad).

[0054] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse, and a joystick.

[0055] The method for detecting the lodging of roadside trees provided by the embodiments of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the method for detecting the lodging of roadside trees. The electronic devices mentioned in the embodiments of the present application include, but are not limited to, mobile phones, tablet computers, computers, cameras, and wearable devices, etc. Hereinafter, taking the electronic device as the execution subject, the method for detecting the lodging of roadside trees provided by the embodiments of the present application will be described.

[0056] As Figure 1 shown, the method for detecting the lodging of roadside trees includes: step 110, step 120, and step 130.

[0057] Step 110: Obtain a target detection image sequence corresponding to a target tree. The target detection image sequence includes multiple target detection images at different sampling time periods, and the target detection images include the target tree.

[0058] Wherein, the target tree is a tree to be detected for lodging, and the target tree may be a roadside tree on both sides of a road.

[0059] The target detection image sequence includes multiple target detection images arranged in chronological order. Multiple sampling time periods may be set, and one target detection image is obtained at each sampling time period. The interval between two adjacent sampling time periods may be several days or several weeks. For example, the interval may be three weeks, that is, the target tree is sampled once every three weeks to obtain one target detection image, and the target detection images are arranged in the chronological order corresponding to the sampling time periods to obtain the target detection image sequence.

[0060] It should be noted that the target detection image may be a three-dimensional image, which can provide three-dimensional spatial information of the target tree. The target detection image may include partial information or all information of the target tree, at least including information of the trunk part. The position of the target tree corresponding to each target detection image may be the same.

[0061] In this step, the target detection image can be obtained by means of point cloud generation during the sampling period, and the target detection images obtained in each sampling period are arranged in chronological order to obtain a target detection image sequence.

[0062] Step 120: Based on the target detection image, determine the trunk inclination angle of the target tree during the corresponding sampling period.

[0063] Among them, the trunk inclination angle can be the angle of the direction of the main axis of the trunk relative to the set reference direction. A direction parallel to the ground can be determined as the reference direction, or a direction perpendicular to the ground can be determined as the reference direction.

[0064] In this step, the area corresponding to the trunk of the target tree in the target detection image can be extracted by methods such as image segmentation or target detection, the topmost and bottommost ends of the trunk are identified, the main axis of the trunk is determined using a linear fitting algorithm, and the angle between the direction of the main axis and the reference direction is calculated as the trunk inclination angle.

[0065] Step 130: Based on the trunk inclination angle corresponding to each sampling period, determine the lodging detection result of the target tree.

[0066] Among them, the lodging detection result is used to characterize whether the target tree has fallen, or to characterize the probability that the target tree has fallen.

[0067] In this embodiment, based on the trunk inclination angle corresponding to each sampling period, the change trend of the trunk inclination angle over time can be analyzed, whether there is a continuous increasing or decreasing trend can be identified, and the change speed and acceleration of the change can be identified to identify abnormal and sudden changes in the trunk inclination angle, so as to determine the lodging detection result.

[0068] In the related art, the monitoring of street tree lodging usually uses the method of regression analysis. For example, parameters such as the trunk height and crown size of the tree are used as independent variables, and the lodging observation situation of the tree is used as the dependent variable, and fitting is carried out based on machine learning methods to realize the prediction of potential lodging situations.

[0069] However, carrying out fitting based on machine learning methods requires a large amount of observation data, and it is difficult to obtain data such as trunk height, crown size, and tree lodging observation situation, which makes the training and verification of machine learning models more difficult and the efficiency of lodging detection lower.

[0070] In the embodiments of the present application, by means of the target detection image sequence corresponding to the target tree, the change of the trunk inclination angle of the target tree over time is determined. According to features such as the change speed of the trunk inclination angle, the probability of the target tree falling is determined. There is no need to obtain a large number of features of the target tree to train a detection model. The change features of the trunk inclination angle can be obtained through a small number of target detection images, and the fall detection can be carried out according to the change features of the trunk inclination angle, which can improve the detection efficiency.

[0071] In the related art, the manual measurement method is used, and professional instruments are carried to measure the inclination of trees. The traditional manual measurement method uses professional measurement equipment such as a measuring instrument to measure the tree parameters, and then calculates the inclination of the tree. The work efficiency is low, and the required professional equipment is complex.

[0072] In the embodiments of the present application, there is no need for manual measurement using measurement tools. An image acquisition device such as a mobile phone or a camera is used to obtain the target detection image sequence, and the fall detection is carried out according to the target detection image sequence. Compared with the traditional manual measurement method, the efficiency is higher and the equipment cost is lower.

[0073] In the related art, a ground penetrating radar is used to detect the root system and trunk cavity of the tree, and then the possibility of the tree falling is calculated according to the obtained parameters. The efficiency is low. It takes several hours to detect one tree. The equipment such as the ground penetrating radar is expensive and troublesome to operate. In addition, the hollow situation of the tree obtained by the ground penetrating radar is only the physiological and structural situation of the tree, not the actual inclination change data. There is a certain connection with the tree falling, but it is not the actual inclination or falling state of the tree.

[0074] In the embodiments of the present application, there is no need to detect information such as the root system and trunk cavity of the tree. The target detection image sequence is obtained by photographing the target tree at multiple sampling time periods. Based on the target detection image sequence corresponding to the target tree, the temporal change situation of the trunk inclination angle of the target tree is determined, and according to features such as the change trend and change speed of the trunk inclination angle, the probability of the target tree falling is determined. The efficiency is higher, the equipment requirements are low, and an image acquisition device such as a mobile phone or a camera can be used to carry out the work. In addition, the possible falling situation of the target tree is described in the time dimension. By using the observation information of multiple sampling time periods, the trunk inclination angles of multiple sampling time periods are calculated, so that the actual change of the trunk inclination angle of the target tree can be effectively obtained, and then the prediction of the falling probability is realized according to the set judgment rules. Compared with means such as the ground penetrating radar, it is closer to the actual change situation of the tree.

[0075] According to the street tree lodging detection method provided by the embodiments of the present application, by means of the target detection image sequence corresponding to the target tree, the change of the trunk tilt angle of the target tree over time is determined. According to features such as the change speed of the trunk tilt angle, the probability of the target tree lodging is determined. It is not necessary to obtain a large number of features of the target tree to train a detection model. The change features of the trunk tilt angle can be obtained through a small number of target detection images, and lodging detection is performed according to the change features of the trunk tilt angle, which can improve the detection efficiency.

[0076] In some embodiments, as Figure 2 shown, step 130, determining the lodging detection result of the target tree based on the trunk tilt angle corresponding to each sampling period, includes:

[0077] Step 131, determining the trunk tilt angle change information between two adjacent sampling periods based on the trunk tilt angles corresponding to the two adjacent sampling periods;

[0078] Step 132, determining the lodging detection result based on each trunk tilt angle change information.

[0079] Among them, the trunk tilt angle change information is used to characterize the change trend, change degree, change speed, etc. of the trunk of the target tree between two adjacent sampling periods.

[0080] In this embodiment, mathematical operations such as subtracting the trunk tilt angles corresponding to two adjacent sampling periods can be performed, and the trunk tilt angle change information can be obtained according to the calculation result.

[0081] In this embodiment, based on each trunk tilt angle change information, the change degree and change speed, etc. of the trunk tilt angle in the overall time series can be analyzed, and compared with the corresponding set threshold, and the lodging detection result is determined according to the comparison result.

[0082] For example, when the change degree of the trunk tilt angle is greater than the corresponding threshold, it can be determined that the target tree has lodged, and when the change degree of the trunk tilt angle is less than the corresponding threshold, it can be determined that the target tree has not lodged.

[0083] For another example, it is also possible to determine the change degree of the trunk tilt angle between two corresponding sampling periods according to the trunk tilt angle change information, analyze all the trunk tilt angle change information, and determine the lodging detection result according to the number of times that the change degree of the trunk tilt angle exceeds the set threshold.

[0084] In some embodiments, as Figure 3 and Figure 4 shown, step 132, determining the lodging detection result based on each trunk tilt angle change information, includes:

[0085] Step 132a: When the absolute value of the difference between the trunk tilt angles corresponding to all adjacent two sampling periods is less than the target threshold, it is determined that the target tree has not toppled over;

[0086] Step 132b: When the absolute value of the difference between the trunk tilt angles corresponding to two adjacent sampling periods is greater than or equal to the target threshold, based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold, the toppling detection result is determined.

[0087] Wherein, the absolute value of the difference is the absolute value of the result of subtracting the trunk tilt angles corresponding to two adjacent sampling periods, and the target threshold is a preset value.

[0088] In this embodiment, when the absolute value of the difference between the trunk tilt angles corresponding to two adjacent sampling periods is less than the target threshold, it indicates that the change degree of the trunk tilt angle is small or almost no change between these two adjacent sampling periods.

[0089] When the absolute value of the difference between the trunk tilt angles corresponding to all adjacent two sampling periods is less than the target threshold, it indicates that the change degree of the trunk tilt angle is small or almost no change in the entire measurement time series, and it is determined that the target tree has not toppled over.

[0090] When the absolute value of the difference between the trunk tilt angles corresponding to two adjacent sampling periods is greater than or equal to the target threshold, it indicates that the change degree of the trunk tilt angle is large between these two adjacent sampling periods.

[0091] When there is an absolute value of the difference between the trunk tilt angles corresponding to two adjacent sampling periods that is greater than or equal to the target threshold, it indicates that there is a large change degree in the entire measurement time series, and the target tree may topple over. Based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold, further judgment is made.

[0092] In this embodiment, the toppling detection result can be determined according to the number of sampling periods corresponding to the absolute value of the difference greater than or equal to the target threshold or the corresponding time series, etc.

[0093] In some embodiments, based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold, determining the toppling detection result includes:

[0094] When the absolute value of the difference between the trunk tilt angle corresponding to the last sampling period and the trunk tilt angle corresponding to the penultimate sampling period is greater than or equal to the target threshold, it is determined that the probability of the target tree toppling over is the first probability.

[0095] In this embodiment, the absolute value of the difference between the trunk tilt angle corresponding to the last sampling period and the trunk tilt angle corresponding to the penultimate sampling period is greater than or equal to the target threshold, indicating that the trunk tilt angle may mutate and the target tree may fall. The first probability may be 50%.

[0096] In some embodiments, determining the lodging detection result based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold includes:

[0097] When the absolute value of the difference between the trunk tilt angles corresponding to all adjacent two sampling periods is greater than or equal to the target threshold, determining the lodging detection result based on the absolute value of the change in the trunk tilt angle and the trunk tilt angle change rate between all adjacent two sampling periods.

[0098] Wherein, the absolute value of the change in the trunk tilt angle is the absolute value of the change value between the trunk tilt angles between adjacent two sampling periods. The absolute value of the change in the trunk tilt angle may be equal to the absolute value of the difference. The trunk tilt angle change rate is the change rate between the trunk tilt angles between adjacent two sampling periods. The trunk tilt angle change rate may be equal to the ratio of the absolute value of the change in the trunk tilt angle to the time interval between the two sampling periods.

[0099] In this embodiment, the absolute value of the difference between the trunk tilt angles corresponding to all adjacent two sampling periods is greater than or equal to the target threshold, indicating that the trunk tilt angle changes greatly during the entire measurement time sequence, and the target tree may fall. The lodging detection result can be determined according to the changes in the absolute value of the change in the trunk tilt angle and the trunk tilt angle change rate in the time sequence.

[0100] In some embodiments, determining the lodging detection result based on the absolute value of the change in the trunk tilt angle and the trunk tilt angle change rate between all adjacent two sampling periods includes:

[0101] When the absolute value of the change in the trunk tilt angle or the trunk tilt angle change rate monotonically increases in the order corresponding to the sampling periods, it is determined that the target tree has fallen.

[0102] The absolute value of the change in the trunk tilt angle or the trunk tilt angle change rate monotonically increases in the order corresponding to the sampling periods, indicating that the trunk tilt angle changes more and more greatly or the change speed becomes faster and faster during the entire measurement time sequence, and it is determined that the target tree has fallen.

[0103] In some embodiments, determining the lodging detection result based on the absolute value of the change in the trunk tilt angle and the trunk tilt angle change rate between all adjacent two sampling periods includes:

[0104] When the absolute value of the change in the trunk tilt angle and the rate of change in the trunk tilt angle monotonically decrease in the order corresponding to the sampling period, the probability that the target tree topples is determined to be the second probability.

[0105] The absolute value of the change in the trunk tilt angle and the rate of change in the trunk tilt angle monotonically decrease in the order corresponding to the sampling period, indicating that the degree of change in the trunk tilt angle becomes smaller and the rate of change becomes slower over the entire measurement time series. The probability that the target tree topples is determined to be the second probability, and the second probability can be 75%.

[0106] The following introduces a specific embodiment for determining the toppling detection result of the target tree based on the trunk tilt angle corresponding to each sampling period.

[0107] Set 3 sampling periods, namely sampling period 1, sampling period 2, and sampling period 3. The trunk tilt angles corresponding to the 3 sampling periods are a1, a2, and a3 respectively.

[0108] Calculate the absolute value k1 of the change in the trunk tilt angle and the rate of change v1 between sampling period 1 and sampling period 2, calculate the absolute value k2 of the change in the trunk tilt angle and the rate of change v2 between sampling period 2 and sampling period 3. Define the trunk tilt angle estimation error as ε, ε ∈ (-3, 3). It can be understood that the trunk tilt angle estimation error can be determined based on the target threshold.

[0109] When a1 ± ε = a2 ± ε = a3 ± ε, it is determined that the target tree has not toppled.

[0110] When a1 ± ε = a2 ± ε ≠ a3 ± ε, it is considered that further observation is needed, and the probability that the target tree topples is 50%.

[0111] When a1 ± ε ≠ a2 ± ε ≠ a3 ± ε, when k1 < k2 or v1 < v2, it is determined that the target tree topples.

[0112] When k1 > k2 and v1 > v2, the probability that the target tree topples is determined to be 75%.

[0113] In some embodiments, obtaining the target detection image sequence corresponding to the target tree includes:

[0114] Obtain multiple first detection images of the target tree during the sampling period;

[0115] Based on the multiple first detection images, perform three-dimensional reconstruction on the target tree to obtain the target detection image corresponding to the sampling period;

[0116] Obtain the target detection image corresponding to each sampling period to obtain the target detection image sequence.

[0117] Among them, the first detection image is a two-dimensional image obtained by photographing with an image acquisition device such as a mobile phone or a camera, and the first detection image includes a complete target tree.

[0118] In this embodiment, the first detection image of the target tree can be obtained by hand-holding a photographing device under windless conditions, or the coordinates of the target tree can be obtained, and a drone, an intelligent vehicle, etc. carrying the photographing device can be controlled to move to the target tree to collect the first detection image.

[0119] In this embodiment, 6 to 10 first detection images can be collected within one sampling period. The resolution of each first detection image is not less than 1 cm, and the information overlap rate between two adjacent first detection images taken at adjacent shooting times is not less than 60%.

[0120] In actual implementation, based on multiple first detection images, the target tree can be three-dimensionally reconstructed through the Open Source Computer Vision Library (OpenCV) to obtain an orthographic image, that is, the target detection image, so as to realize the three-dimensional spatialization of the image of the target tree.

[0121] The following introduces a specific embodiment of three-dimensionally reconstructing a target tree through OpenCV.

[0122] Step 1: Read multiple first detection images through OpenCV.

[0123] Step 2: Extract the feature points of each first detection image. The Scale-Invariant Feature Transform (SIFT) algorithm can be used in OpenCV to extract the feature points of the first detection image.

[0124] Step 3: Use the Brute-Force Matcher (BFMatcher) in OpenCV to match the corresponding points between two adjacent first detection images taken at adjacent shooting times based on the extracted feature points. The corresponding points can be feature points representing the same position of the target tree in different first detection images.

[0125] Step 4: Estimate the internal parameters and external parameters of the corresponding cameras based on the matched corresponding points through the Open Source Structure-from-Motion pipeline (OpenSFM).

[0126] Step 5: Through the Open Multi-View Stereo reconstruction library (OpenMVS), based on multiple first detection images with determined internal and external parameters of the camera, perform 3D reconstruction on the target tree to reconstruct the 3D model of the target tree in the real world.

[0127] Step 6: Through the orthophoto generation function of the OpenCV Design Modules (ODM), based on the 3D model of the target tree, generate and export the target detection image from a horizontal perspective.

[0128] In some embodiments, based on the target detection image, determining the trunk inclination angle of the target tree during the corresponding sampling period includes:

[0129] Perform image segmentation on the target detection image to obtain the trunk area in the target detection image;

[0130] Through the circle fitting algorithm, fit the target circle corresponding to the trunk area and determine the center of the target circle;

[0131] Based on the center of the target circle, determine the trunk inclination angle.

[0132] Wherein, the trunk area is the area where the trunk of the target tree is located in the target detection image.

[0133] In this embodiment, the target detection image can be segmented by a trained trunk segmentation model to obtain the trunk area. The initial values of the center and radius are estimated based on the geometric center point and the points on the trunk contour, and the fitting is optimized to find the circle that can minimally enclose the trunk contour as the target circle. The angle between the line connecting the center of the target circle and a set reference point such as the root of the target tree and the reference direction is used as the trunk inclination angle.

[0134] The following introduces a specific embodiment for determining the trunk inclination angle.

[0135] Step 1: Manually plot the trunk on the orthophoto of the tree to form a trunk training sample with no less than 1000 trunks.

[0136] Step 2: Using the Feature Refinement Network (FR-Net) as the artificial intelligence network, train the FR-Net based on the trunk training sample to obtain a trunk segmentation model that can accurately segment the trunk.

[0137] Step 3: Use the trained FR-Net to complete the trunk segmentation of the target tree.

[0138] Step 4: Divide the segmented tree trunk evenly into three parts: the head of the tree trunk, the middle of the tree trunk, and the tail of the tree trunk according to the length.

[0139] Step 5: Fit the target circle corresponding to the middle part of the tree trunk through the circle fitting algorithm.

[0140] Step 6: Connect the center of the target circle with the corresponding point at the root of the target tree, and calculate the angle between the connection direction and the horizontal plane as the tree trunk inclination angle.

[0141] In the method for detecting the lodging of roadside trees provided by the embodiments of the present application, the execution subject can be a roadside tree lodging detection device. In the embodiments of the present application, taking the roadside tree lodging detection device executing the roadside tree lodging detection method as an example, the roadside tree lodging detection device provided by the embodiments of the present application is described.

[0142] The embodiments of the present application also provide a roadside tree lodging detection device.

[0143] As Figure 5 shown, the roadside tree lodging detection device includes:

[0144] An acquisition module 510, configured to acquire a target detection image sequence corresponding to a target tree, where the target detection image sequence includes multiple target detection images at different sampling time periods, and the target detection images include the target tree;

[0145] A first processing module 520, configured to determine the tree trunk inclination angle of the target tree at the corresponding sampling time period based on the target detection image;

[0146] A second processing module 530, configured to determine the lodging detection result of the target tree based on the tree trunk inclination angle corresponding to each sampling time period.

[0147] According to the roadside tree lodging detection device provided by the embodiments of the present application, by the target detection image sequence corresponding to the target tree, the change situation of the tree trunk inclination angle of the target tree over time is determined. According to characteristics such as the change speed of the tree trunk inclination angle, the probability of the target tree being lodged is determined. It is not necessary to obtain a large number of features of the target tree to train a detection model. The change characteristics of the tree trunk inclination angle can be obtained through a small number of target detection images. The lodging detection is performed according to the change characteristics of the tree trunk inclination angle, which can improve the detection efficiency.

[0148] In some embodiments, the second processing module 530 is configured to determine the change information of the tree trunk inclination angle between two adjacent sampling time periods based on the tree trunk inclination angles corresponding to the two adjacent sampling time periods;

[0149] Based on the change information of each tree trunk inclination angle, determine the lodging detection result.

[0150] In some embodiments, the second processing module 530 is used to determine that the target tree has not fallen down when the absolute value of the difference between the trunk inclination angles corresponding to all two adjacent sampling periods is less than the target threshold;

[0151] When the absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling periods is greater than or equal to the target threshold, the lodging detection result is determined based on the sampling period corresponding to the absolute value of the difference greater than or equal to the target threshold.

[0152] In some embodiments, the second processing module 530 is used to determine that the probability of the target tree falling is a first probability when the absolute value of the difference between the trunk inclination angle corresponding to the last sampling period and the trunk inclination angle corresponding to the second to last sampling period is greater than or equal to the target threshold.

[0153] In some embodiments, the second processing module 530 is used to determine the lodging detection result based on the absolute value of the trunk inclination angle change and the trunk inclination angle change rate between all two adjacent sampling time periods when the absolute value of the difference between the trunk inclination angles corresponding to all two adjacent sampling time periods is greater than or equal to the target threshold.

[0154] In some embodiments, the second processing module 530 is used to determine that the target tree has fallen when the absolute value of the trunk inclination angle changes or the rate of change of the trunk inclination angle increases monotonically in the order corresponding to the sampling time periods.

[0155] In some embodiments, the second processing module 530 is used to determine that the probability of the target tree falling is the second probability when the absolute value of the trunk inclination angle change and the trunk inclination angle change rate decrease monotonically in the order corresponding to the sampling time periods.

[0156] In some embodiments, the acquisition module 510 is used to acquire a plurality of first detection images of the target tree during a sampling period;

[0157] Based on the multiple first detection images, three-dimensionally reconstruct the target tree to obtain a target detection image corresponding to the sampling period;

[0158] The target detection image corresponding to each sampling period is obtained to obtain a target detection image sequence.

[0159] In some embodiments, the first processing module 520 is used to perform image segmentation on the target detection image to obtain a tree trunk region in the target detection image;

[0160] Using a circle fitting algorithm, fit the target circle corresponding to the tree trunk area and determine the center of the target circle;

[0161] Based on the center of the target circle, the trunk inclination angle is determined.

[0162] The device for detecting the lodging of roadside trees in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0163] The device for detecting the lodging of roadside trees in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0164] The device for detecting the lodging of roadside trees provided by the embodiments of the present application can implement Figures 1 to 4 each process implemented by the method embodiments. To avoid repetition, it will not be elaborated here.

[0165] In some embodiments, as Figure 6 shown, the embodiments of the present application further provide an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, it implements each process of the above method embodiment for detecting the lodging of roadside trees and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0166] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0167] The embodiments of the present application also provide a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements each process of the above-mentioned embodiment of the street tree lodging detection method and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0168] Wherein, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks or optical discs, etc.

[0169] The embodiments of the present application also provide a computer program product, including a computer program, which implements the above-mentioned street tree lodging detection method when executed by a processor.

[0170] Wherein, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks or optical discs, etc.

[0171] The embodiments of the present application further provide a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement each process of the above-mentioned embodiment of the street tree lodging detection method and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0172] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system or system-on-a-chip, etc.

[0173] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the methods and devices in the embodiments of the present application are not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described method may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0174] Through the description of the above embodiments, those skilled in the art can clearly understand that the above method of the embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to enable a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0175] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

[0176] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0177] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and purposes of the present application. The scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for detecting fallen roadside trees, characterized in that: include: Acquire a target detection image sequence corresponding to the target tree, wherein the target detection image sequence includes a plurality of target detection images at different sampling periods, and the target detection images include the target tree; Based on the target detection image, determining the trunk inclination angle of the target tree in the corresponding sampling period; Determining a lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period; The determining of the lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period includes: Determine the trunk inclination angle change information between the two adjacent sampling time periods based on the trunk inclination angles corresponding to the two adjacent sampling time periods; The lodging detection result is determined based on the information on the change in the inclination angles of each tree trunk.

2. The method for detecting fallen roadside trees according to claim 1, characterized in that: The determining the lodging detection result based on the change information of the inclination angles of each tree trunk includes: When the absolute value of the difference between the trunk inclination angles corresponding to all two adjacent sampling periods is less than the target threshold, it is determined that the target tree has not fallen; When the absolute value of the difference between the trunk inclination angles corresponding to two adjacent sampling periods is greater than or equal to the target threshold, the lodging detection result is determined based on the sampling period corresponding to the absolute value of the difference that is greater than or equal to the target threshold.

3. The method for detecting fallen roadside trees according to claim 2, characterized in that: The determining the lodging detection result based on the sampling period corresponding to the absolute value of the difference that is greater than or equal to the target threshold value comprises: When the absolute value of the difference between the trunk inclination angle corresponding to the last sampling period and the trunk inclination angle corresponding to the second to last sampling period is greater than or equal to the target threshold, the probability of the target tree falling is determined to be the first probability.

4. The method for detecting fallen roadside trees according to claim 2, characterized in that: The determining the lodging detection result based on the sampling period corresponding to the absolute value of the difference that is greater than or equal to the target threshold value comprises: When the absolute values ​​of the differences between the trunk inclination angles corresponding to all two adjacent sampling periods are greater than or equal to the target threshold, the lodging detection result is determined based on the absolute values ​​of the trunk inclination angle changes and the trunk inclination angle change rates between all two adjacent sampling periods.

5. The method for detecting fallen roadside trees according to claim 4, characterized in that: The determining of the lodging detection result based on the absolute value of the change in the trunk inclination angle and the trunk inclination angle change rate between all two adjacent sampling periods includes: When the absolute value of the change in the trunk inclination angle or the rate of change in the trunk inclination angle increases monotonically in the order corresponding to the sampling time periods, it is determined that the target tree has fallen.

6. The method for detecting fallen roadside trees according to claim 4, characterized in that: The determining of the lodging detection result based on the absolute value of the change in the trunk inclination angle and the trunk inclination angle change rate between all two adjacent sampling periods includes: In the case where the absolute value of the trunk inclination angle change and the trunk inclination angle change rate decrease monotonically in the order corresponding to the sampling time periods, the probability of the target tree falling is determined to be the second probability.

7. The method for detecting fallen roadside trees according to any one of claims 1 to 6, characterized in that: The step of obtaining a target detection image sequence corresponding to the target tree comprises: Acquire a plurality of first detection images of the target tree during a sampling period; Based on the plurality of the first detection images, three-dimensionally reconstruct the target tree to obtain the target detection image corresponding to the sampling period; The target detection image corresponding to each of the sampling time periods is acquired to obtain the target detection image sequence.

8. The method for detecting fallen roadside trees according to any one of claims 1 to 6, characterized in that: The determining, based on the target detection image, the trunk inclination angle of the target tree in the corresponding sampling period comprises: Performing image segmentation on the target detection image to obtain a tree trunk area in the target detection image; Fitting a target circle corresponding to the tree trunk area by a circle fitting algorithm, and determining the center of the target circle; The trunk inclination angle is determined based on the center of the target circle.

9. A roadside tree fall detection device, characterized in that: include: An acquisition module, used for acquiring a target detection image sequence corresponding to a target tree, wherein the target detection image sequence includes a plurality of target detection images at different sampling periods, and the target detection images include the target tree; A first processing module, configured to determine the trunk inclination angle of the target tree in the corresponding sampling period based on the target detection image; A second processing module, configured to determine a lodging detection result of the target tree based on the trunk inclination angle corresponding to each sampling period; The second processing module is further used to determine the trunk inclination angle change information between two adjacent sampling time periods based on the trunk inclination angles corresponding to the two adjacent sampling time periods; The lodging detection result is determined based on the information on the change in the inclination angles of each tree trunk.

Images