Tree fall detection method and system combined with reference image analysis

Abstract

The present invention discloses a tree fall detection method and system combined with reference image analysis. The method comprises: based on the initial image taken by a low-altitude camera, using a tree and reference key point detection model to determine the key point position, fitting to form two straight lines and judging the current weather conditions. If the weather is not extreme weather, the state of the tree is evaluated by two fitted straight lines, and the image and position information of the tree tilt or fall are output. If the weather is extreme weather, it is further determined whether the tree is submerged by accumulated water. If not submerged, the change in the angle between the fitted straight lines is monitored to detect whether the tree has fallen; if it has fallen, the position and direction of the fall are output. If the tree is submerged by accumulated water, the height of the flooded part is calculated using a reference object and accumulated water relationship model, and the key points of the tree are updated, and finally the state of the tree is determined by comparing the fitted straight lines and the corresponding evaluation results are output. By implementing the method of the present invention, it is possible to improve the accuracy and reliability of tree fall monitoring.

Description

Technical Field

[0001] The present invention relates to a tree fall detection method, and more specifically to a tree fall detection method and system combined with reference object image analysis. Background Art

[0002] With the rapid development of urbanization, the greening level of cities has significantly improved, and the number and size of roadside trees and other greening trees in residential areas, parks, streets and various parks have continued to grow. However, this trend has also brought new challenges - under extreme weather conditions, tree fall incidents occur frequently, which not only affects the beauty of the city, but also may pose a threat to public safety. In particular, when tree fall occurs on major traffic arteries, it will directly hinder traffic, causing large-scale traffic jams and bringing inconvenience to citizens' lives.

[0003] In the face of the problem of fallen trees, traditional response measures mainly include manual inspections and reliance on citizen reports. These two methods have obvious lags and are difficult to respond to in the first place, thus delaying the timeliness and effectiveness of the cleanup work. In order to improve monitoring efficiency, in recent years, image analysis-based technical means have been used to automatically detect the state of fallen trees. These methods usually involve using models to identify key parts of trees and comparing whether the vertical angle between the tree and the ground level or the road baseline exceeds a set threshold to determine whether the tree is tilted or fallen.

[0004] However, due to the problem of shooting angle, it is very difficult to obtain a clear ground horizontal line when looking down at a large angle based on image analysis technology, which directly affects the accuracy of the vertical angle measurement of trees. Urban shooting scenes are complex and changeable, and the horizon often appears in the form of a vanishing point or is blocked by other objects, which increases the difficulty of accurate identification. Different cameras are installed at different positions and angles, which may cause the horizon to be tilted, further reducing the reliability of the judgment and easily leading to misjudgment.

[0005] Therefore, it is necessary to design a new method to improve the accuracy and reliability of tree fall monitoring, so as to overcome the technical problems of low accuracy and reliability of existing technologies in the face of complex and changing urban environments, and ensure the effectiveness and safety of urban greening management. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the prior art and provide a tree fall detection method and system combined with reference image analysis.

[0007] To achieve the above object, the present invention adopts the following technical solution: a tree fall detection method combined with reference image analysis, comprising:

[0008] Acquire an initial image taken by a low-altitude camera;

[0009] Inputting the initial image into a tree and reference object key point detection model to determine tree key points and reference object key points;

[0010] Fitting and connecting the key points of the tree and the key points of the reference object respectively to form two fitting straight lines;

[0011] Determine whether the current weather is extreme weather;

[0012] If the current weather is not extreme weather, determining the tree status assessment according to the two fitting straight lines to obtain an assessment result;

[0013] The output evaluation result is the image and location information corresponding to the tree leaning or falling;

[0014] If the current weather is extreme weather, determining whether the tree is submerged by water;

[0015] If the tree is not submerged by water, the changing trend of the angle between the two fitting straight lines is continuously monitored to determine whether the tree has fallen, and when the tree has fallen, the falling position and direction of the tree are determined to obtain an evaluation result, and the output evaluation result is an image and position information corresponding to the tilt or fall of the tree;

[0016] If the tree is submerged by water, the height of the submerged part of the tree is calculated by using the reference object and water accumulation relationship segmentation model and the position of the key points of the tree is updated, and a fitting straight line is formed according to the updated key points of the tree;

[0017] The state of the tree is finally confirmed based on the fitting straight line formed by fitting the key points of the reference object and the fitting straight line formed by the updated key points of the tree to obtain an evaluation result, and the output evaluation result is an image and position information corresponding to the tilt or fall of the tree.

[0018] A further technical solution is as follows: the tree and reference object key point detection model is obtained by training a transformer-based network using images containing trees and reference objects with key point labels as sample sets.

[0019] A further technical solution is: the step of inputting the initial image into a tree and reference object key point detection model to determine the tree key points and the reference object key points comprises:

[0020] Inputting the initial image into a tree and reference object key point detection model, extracting multi-scale features in the initial image using a convolutional layer, and generating a feature map;

[0021] The feature map is downsampled using a spatial reduction module and flattened into a series of key and value tokens;

[0022] Evaluating similarity based on the distance between the markers, and determining the importance score of the cluster center using the local density and the minimum high density distance;

[0023] Determine that the highest marker with the importance score is selected as the cluster center, and assign the remaining markers to the nearest center, and predict the importance score, and determine whether to merge the markers of the same type into a new marker according to the importance score;

[0024] Input the new tag into the Transformer block to transfer the feature information from the tag to the new tag;

[0025] Upsampling the new markers to restore them to the resolution of the feature map to obtain a high-resolution heat map;

[0026] The brightest position of the high-resolution heat map is selected as the key point to obtain the tree key point and the reference object key point.

[0027] A further technical solution is: the similarity is evaluated based on the distance between the markers, and the importance score of the cluster center is determined using the local density and the minimum high density distance, including:

[0028] Performing clustering processing on all the markers, evaluating the similarity between all the markers by calculating the distances between the markers, and estimating the local density according to the average distance between the marker and several nearest neighbor markers;

[0029] For each of the markers, determine the distance from the marker to other markers with higher local density, and select the smallest distance as the distance index. For the marker with the highest local density, set the distance index of the marker to the maximum distance between the marker and all other markers.

[0030] An importance score for each of the tags is determined by combining the local density and the distance indicator.

[0031] A further technical solution is: selecting the brightest position of the high-resolution heat map as a key point to obtain a tree key point and a reference object key point, including:

[0032] The point with the largest value in the high-resolution heat map described by argmax is used as the key point to obtain the tree key point and the reference object key point.

[0033] A further technical solution is: determining the tree status evaluation according to the two fitting straight lines to obtain the evaluation result includes:

[0034] The parallel angle of the two fitting straight lines is compared with two set thresholds to determine an evaluation result, wherein the two set thresholds include a first threshold and a second threshold, respectively, and the first threshold is less than the second threshold; when the parallel angle of the two fitting straight lines is less than the first threshold, it is determined that the tree is growing normally; when the parallel angle of the two fitting straight lines is between the first threshold and the second threshold, it is determined that the tree is tilted; when the parallel angle of the two fitting straight lines is greater than the second threshold, it is determined that the tree has fallen.

[0035] A further technical solution is: the continuously monitoring the changing trend of the angle between the two fitting straight lines to determine whether the tree has fallen, and when the tree has fallen, determining the position and direction of the tree falling to obtain an evaluation result, and executing the output evaluation result as an image and position information corresponding to the tree tilt or fall, including:

[0036] By continuously acquiring initial images and inputting them into a tree and reference object key point detection model to determine tree key points and reference object key points, and performing straight line fitting respectively, two fitting straight lines corresponding to multiple frame images are obtained;

[0037] Calculate the angle between the two fitting straight lines corresponding to each frame image;

[0038] The changing trend of the angle is continuously monitored in chronological order to determine whether the tree has fallen; wherein, when the angles corresponding to the fitting straight lines of all frame images are less than the first threshold, or when the angle corresponding to the fitting straight line of a frame image exceeds the first threshold and then returns to less than the first threshold, it is determined that the tree maintains a normal growth state; if the angles corresponding to the fitting straight lines of consecutive frame images are always between the first threshold and the second threshold and have no obvious changes, it is determined that the tree has tilted to obtain an evaluation result; when the angle corresponding to the fitting straight line of a frame image exceeds the first threshold for the first time, and in subsequent frame images, the angle corresponding to the fitting straight line increases rapidly until it exceeds the second threshold and then tends to be fixed, it is determined that the tree has fallen;

[0039] When a tree has fallen, the relative position of the tree in the initial image and the steering angle of the tree are determined to obtain the falling direction of the tree, and the evaluation result is obtained in combination with the position information in the initial image.

[0040] A further technical solution is: when the tree has fallen, the relative position of the tree in the initial image and the guiding angle of the tree are determined to obtain the falling direction of the tree, and after obtaining the evaluation result in combination with the position information in the initial image, the method further includes:

[0041] The time point when the tree begins to fall, the shooting direction and location information of the camera are recorded and added to the evaluation result.

[0042] A further technical solution is: the reference object and water accumulation relationship segmentation model is used to calculate the height of the flooded part of the tree and update the position of the key points of the tree, and a fitting straight line is formed according to the updated key points of the tree, including:

[0043] Using the reference object and water accumulation relationship segmentation model, a water accumulation boundary line is divided according to the relationship between the root, trunk or crown of the tree and the water accumulation;

[0044] Identify and mark reference information above the waterline;

[0045] Get the mask pixel value of the reference object above submerged by the accumulated water;

[0046] Using the actual target object size ratio pre-stored in the reference object and water accumulation relationship segmentation model, the mask pixel value is converted into the corresponding real physical size, and the specific height of the part submerged by the water accumulation is determined to obtain the water accumulation depth;

[0047] According to the water depth and the key point position of the tree in the flooded part, combined with the reference object information above the water accumulation line, new key point coordinates are established to obtain updated tree key points;

[0048] A fitted straight line is formed based on the updated tree key points.

[0049] The present invention also provides a tree fall detection system combined with reference image analysis, comprising:

[0050] An image acquisition unit, used to acquire an initial image taken by a low-altitude camera;

[0051] A key point detection unit, used for inputting the initial image into a tree and reference object key point detection model to determine the tree key points and the reference object key points;

[0052] A fitting unit, used for fitting and connecting the key points of the tree and the key points of the reference object to form two fitting straight lines;

[0053] A judgment unit, used to judge whether the current weather is extreme weather;

[0054] A first evaluation unit, configured to determine a tree state evaluation based on the two fitting straight lines to obtain an evaluation result if the current weather is not extreme weather;

[0055] An output unit, used for outputting an image and position information corresponding to the evaluation result of the tree being tilted or fallen;

[0056] a water accumulation judgment unit, used for judging whether the tree is submerged by water if the current weather is extreme weather;

[0057] The second evaluation unit is used for continuously monitoring the change trend of the angle between the two fitting straight lines to determine whether the tree has fallen if the tree is not submerged by water, and when the tree has fallen, determining the falling position and direction of the tree to obtain an evaluation result, and executing the output of the evaluation result as an image and position information corresponding to the inclination or falling of the tree;

[0058] A key point updating unit, for calculating the height of the flooded part of the tree by using a reference object and water accumulation relationship segmentation model and updating the position of the key points of the tree if the tree is flooded, and forming a fitting straight line according to the updated key points of the tree;

[0059] The third evaluation unit is used to finally confirm the state of the tree based on the fitting straight line formed by fitting the key points of the reference object and the fitting straight line formed by the updated key points of the tree to obtain an evaluation result, and execute the output evaluation result as an image and position information corresponding to the tilt or fall of the tree.

[0060] Compared with the prior art, the present invention has the following beneficial effects: the present invention can provide a perspective closer to the ground by shooting images with a low-altitude camera, accurately capture the details of trees and the surrounding environment, and improve the accuracy of lodging monitoring, especially in complex urban environments, avoiding the occlusion and distortion of high-altitude perspectives. Combined with the key point detection model of trees and reference objects, the spatial relationship between the two can be accurately identified, reducing the misjudgment caused by environmental interference or changes in perspective. By fitting the key points of trees and reference objects to form a straight line, it is possible to accurately assess whether the trees are tilted or lodged, and enhance the system's ability to perceive changes in tree status. In extreme weather, the system can automatically adjust the monitoring strategy, such as considering the impact of water accumulation, to ensure adaptability and accuracy under various weather conditions. Using the reference object and water accumulation relationship segmentation model, the height of the flooded part of the tree can be accurately calculated, and the position of the key points of the tree can be updated, thereby avoiding water accumulation interfering with the judgment of lodging and improving monitoring reliability. Combining the tree lodging detection results and location information, the system can provide real-time feedback on the lodging situation, and dynamically adjust the assessment according to the latest data, ensuring high efficiency and stability in complex environments, and improving the safety of urban greening management.

[0061] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0063] Figure 1 A schematic flow chart of a tree fall detection method combined with reference object image analysis provided by an embodiment of the present invention;

[0064] Figure 2 A schematic diagram of a tree falling down provided by an embodiment of the present invention;

[0065] Figure 3 A schematic diagram of a tree leaning and encroaching on a road provided by an embodiment of the present invention;

[0066] Figure 4 A schematic diagram of a tree partially submerged by water provided by an embodiment of the present invention;

[0067] Figure 5 A schematic block diagram of a tree fall detection system combined with reference object image analysis provided by an embodiment of the present invention;

[0068] Figure 6 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0071] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0072] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0073] See also Figure 1 , Figure 1 A schematic flow chart of a tree fall detection method combined with reference image analysis provided in an embodiment of the present invention. The tree fall detection method combined with reference image analysis is applied to a server, which exchanges data with a terminal and a low-altitude camera, and combines low-altitude camera images with other sensor data (such as meteorological data and ground sensors) to improve the detection capability of tree fall under complex weather conditions. By continuously monitoring the changes in the angles between the fitted straight lines and timely updating the key points of the trees, the dynamic perception of the tilt and fall of the trees is enhanced. For extreme weather conditions (such as heavy rains, floods, etc.), the water accumulation relationship model is used to accurately assess the state of the trees in the flooded part to ensure the accuracy of monitoring in complex environments. A deep learning model is introduced to perform a more detailed analysis of the image, automatically identify changes in the state of the tree, and thus improve the recognition capability of fallen and tilted trees.

[0074] Figure 1 FIG. 1 is a flow chart of a tree fall detection method combined with reference image analysis provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S200.

[0075] S110: Acquire an initial image captured by a low-altitude camera.

[0076] In this embodiment, the initial image refers to an image containing trees and parameter objects, which also includes road tree information, reference object information, and location information.

[0077] Low-altitude cameras can provide a perspective closer to the ground, reducing the impact of the overhead angle on image acquisition, thereby better capturing the details of trees and their surrounding environment; using low-altitude camera equipment acquisition modules can not only overcome the limitations of traditional methods, but also provide more accurate and reliable tree status monitoring services, greatly improving the speed and accuracy of tree fall detection.

[0078] S120, inputting the initial image into a tree and reference object key point detection model to determine tree key points and reference object key points.

[0079] In this embodiment, the tree and reference object key point detection model is obtained by training a transformer-based network using images containing trees and reference objects with key point labels as sample sets. This model can focus on the key parts of trees and reference objects (such as street lights, electric poles, etc.) in their surroundings to assist in determining whether the trees have fallen.

[0080] The tree key points include:

[0081] Crown: refers to the densely packed area of ​​branches and leaves at the top of a tree.

[0082] Trunk: refers to the main supporting part connecting the crown and roots.

[0083] Root (or base): The part of a tree that is in contact with the ground, usually the bottom of the trunk visible above the ground.

[0084] Key points of reference include:

[0085] Top: The highest point of a reference object (such as a street lamp, telephone pole, etc.).

[0086] Bottom: The location where the reference object contacts the ground.

[0087] In one embodiment, the above step S120 may include steps S121 to S127.

[0088] S121, inputting the initial image into a tree and reference object key point detection model, using a convolutional layer to extract multi-scale features in the initial image, and generating a feature map.

[0089] In this embodiment, the feature map refers to a high-resolution feature map, an image carrying the features of the initial image.

[0090] The initial image is input into the tree and reference object key point detection model. First, the convolutional layer is used to extract multi-scale features from the image and generate feature maps. These feature maps contain different levels of visual information and provide a basis for subsequent processing. They capture important structural and texture information in the image in preparation for key point detection.

[0091] S122. Downsample the feature map using a spatial reduction module and flatten it to form a series of key and value tags.

[0092] In this embodiment, the generated feature map is downsampled and flattened into a series of key and value tags, reducing the amount of computation while retaining structural information. Reducing the resolution of the feature map improves computational efficiency while ensuring that each tag carries enough information for the next step of analysis.

[0093] S123, evaluating the similarity based on the distance between the markers, and determining the importance score of the cluster center using the local density and the minimum high density distance.

[0094] In this embodiment, the importance score is a comprehensive indicator calculated by combining the local density (ρ) and the minimum high density distance (δ) to measure the possibility of a marker being a cluster center. Specifically, it reflects the relative importance and uniqueness of the marker in its neighborhood. Markers with higher scores are more likely to become cluster centers, that is, the key points we are looking for.

[0095] Among them, local density refers to the number of markers around a marker, or the closeness between the marker and other markers. It is estimated by calculating the average distance between each marker and its K nearest neighboring markers. A marker with a higher local density means that there are more other markers around it, indicating that it may be the core position of a cluster.

[0096] The minimum high-density distance means that for any marker, all markers with higher local density are found and the shortest distance between these markers and the current marker is calculated. For the marker with the highest local density, the maximum distance between it and all other markers is taken as its δ value. This parameter helps identify markers that not only have high local density but are also far away from other high-density points. Such markers are often ideal candidates for cluster centers.

[0097] In one embodiment, the above-mentioned step S123 may include steps S1231 - S1233 .

[0098] S1231, clustering all the markers, evaluating the similarity between all the markers by calculating the distances between the markers, and estimating the local density according to the average distance between the marker and several nearest neighbor markers.

[0099] In this embodiment, all markers are clustered, that is, they are grouped according to the distance (or similarity) between the markers. The similarity between the markers is evaluated by calculating the distance between the markers. Generally, the shorter the distance, the more likely the two markers belong to the same feature or structure, and therefore the higher their similarity. The local density ρ is estimated based on the average distance between each marker and its several nearest neighbor markers. Here, "several" refers to a predefined fixed number K, which can be adjusted according to the specific application scenario.

[0100] This allows us to identify which markers are in dense areas, thereby preliminarily screening out candidate markers that may serve as key points.

[0101] S1232. For each of the markers, determine the distance from the marker to other markers with higher local density, and select the smallest distance as the distance index. For the marker with the highest local density, set the distance index of the marker to the maximum distance between the marker and all other markers.

[0102] In this embodiment, for each marker, the distance from it to other markers with higher local density is found, and the minimum distance is selected as the distance index δ of the marker. This step aims to find markers that are not only located in high-density areas (i.e., have higher local density themselves), but also surrounded by other high-density markers.

[0103] In particular, for the marker with the highest local density, its distance index is set to the maximum distance from all other markers. This is to ensure that the marker has the largest distance index, thus ensuring that it is eligible to become one of the cluster centers.

[0104] This determines which markers are not only located in high-density areas but also relatively isolated, and such markers are more likely to be key points.

[0105] S1233. Determine the importance score of each of the tags by combining the local density and the distance index.

[0106] In this embodiment, the local density ρ of each marker is multiplied by its distance index δ to obtain the importance score of the marker. This combination can effectively highlight those markers that are both located in high-density areas and relatively isolated, because they are more likely to be key points in the image.

[0107] Each marker is assigned a score that reflects the likelihood of it being a keypoint. A higher score means the marker is more likely to be an important feature point.

[0108] Through the above steps S1231 to S1233, the most representative key points can be effectively selected from a large number of markers, and these key points will play an important role in subsequent analysis, such as for determining the fallen state of trees. This method combines local density and distance indicators to improve the accuracy of key point detection, while ensuring that the model has good adaptability to images under different shooting conditions.

[0109] S124, determine the highest tag with the importance score and select it as the cluster center, assign the remaining tags to the nearest center, predict the importance score, and determine to merge the tags of the same type into a new tag according to the importance score.

[0110] In this embodiment, the cluster center is selected based on the importance score calculated previously, and other tags are assigned and fused. This process is crucial for reducing redundant tags and optimizing feature representation.

[0111] Specifically, the marker with the highest importance score among all markers is determined as the initial cluster center. The importance score is calculated based on the product of the local density ρ and the distance index δ, so the marker with the highest score is usually located in a high-density area and is relatively isolated, making it an ideal candidate for the cluster center. The marker that is most likely to represent the key structure or feature is identified as the cluster center to ensure the accuracy and reliability of subsequent analysis.

[0112] Next, for all markers except the ones that have been selected as the center, assign them to the nearest cluster center. Here, "nearest" refers to the minimum distance calculated by some distance metric (such as Euclidean distance). Each non-center marker will be associated with a specific cluster center to form a preliminary clustering result. This can group similar markers together for further processing and analysis, while simplifying the data structure.

[0113] Finally, the importance of the tags in each cluster is re-evaluated based on the assigned clustering. For tags belonging to the same category (i.e., tags assigned to the same cluster center), their new importance scores can be predicted in some way (e.g., taking the average, weighted average, etc.).

[0114] Based on the new importance score, decide whether to merge similar markers into a new marker. If a group of markers have highly similar features and their combined importance scores indicate that they jointly represent a more significant feature, consider merging these markers into a new single marker. The fusion method can be selected according to application requirements, such as simply replacing the entire cluster with the cluster center, or creating a new marker to comprehensively represent this group of markers. This reduces redundant information, optimizes feature representation, and improves subsequent processing efficiency and accuracy. Through fusion, the data dimension can be reduced while retaining key features, making the model more concise and efficient.

[0115] In this embodiment, this step realizes the conversion from the original tag set to a more concise but more representative new tag set. This method not only helps to enhance the robustness of feature detection, but also effectively copes with changes in different scenarios, ensuring the system's good adaptability to complex environments.

[0116] S125, inputting the new tag into the Transformer block to transfer the feature information from the tag to the new tag;

[0117] S126. Perform feature upsampling on the new markers to restore the resolution to the feature map to obtain a high-resolution heat map.

[0118] In this embodiment, a high-resolution heat map is a visualization tool used to display the key point locations predicted by the model. Each pixel on the heat map corresponds to a location in the original image, and the color intensity (usually brightness) indicates the probability that the location is a key point. The brighter the location, the more likely it is to be a key point. Finally, the point with the highest brightness on the heat map is selected as the estimated key point output through the argmax operation.

[0119] In this embodiment, these new tokens are optimized through a fusion operation to better represent the key features in the original data. Now, these new tokens are passed as input to one or more Transformer blocks. The Transformer block is a powerful deep learning architecture that is particularly good at processing sequence data and capturing long-range dependencies. In this step, each new token is treated as a "token" or feature vector and interacts with other tokens through a self-attention mechanism. This mechanism allows each token to dynamically adjust its own representation based on its relevance to all other tokens, effectively transferring feature information from the original token to the new token.

[0120] In addition to the self-attention mechanism, Transformers typically contain feed-forward neural network layers and other components to further enrich and refine the feature representation of each token. This helps ensure that the new token not only inherits the key features of the original token, but also captures more complex patterns and contextual information.

[0121] S127, selecting the brightest position of the high-resolution heat map as a key point to obtain a tree key point and a reference object key point.

[0122] In this embodiment, the point with the largest value in the high-resolution heat map is taken as the key point by argmax to obtain the tree key point and the reference object key point.

[0123] Based on the new tags processed by the Transformer block, we need to map these features back to the spatial resolution of the original feature map. This is because the new tags may be the result of operations at a lower dimension or resolution, and in order to obtain high-resolution outputs (such as heat maps), we need to upsample these features to a higher resolution. Common upsampling methods include nearest neighbor interpolation, bilinear interpolation, transposed convolution (also known as deconvolution), etc. Choose a suitable upsampling technique to increase the spatial size of the feature map so that it matches the original input image or other reference image. In this process, not only the spatial dimension needs to be expanded, but also the quality of the feature representation needs to be maintained or improved to ensure that important information is not lost. Once the feature map is successfully upsampled to the required resolution, the final high-resolution heat map can be generated. The heat map intuitively shows the importance or activation level of different regions, which is very useful for tasks such as visualizing the model decision process, understanding model behavior, and assisting diagnosis.

[0124] This ensures that the processed features can reflect the spatial structure of the original data and can provide detailed feature distribution at a higher resolution to support downstream applications.

[0125] S130, fitting and connecting the key points of the tree and the key points of the reference object respectively to form two fitting straight lines.

[0126] In this embodiment, an appropriate mathematical method is selected to fit the straight line. Common straight line fitting algorithms include the least squares method (Least Squares Method), RANSAC (Random Sample Consensus), etc. The least squares method is suitable for situations with less noise, while RANSAC is better at processing data sets containing outliers.

[0127] For the tree keypoints, the selected fitting method is applied to find a straight line that best matches the distribution of these points. This straight line should fit all the tree keypoints as closely as possible, representing the main direction or structure of the tree. Similarly, the same fitting process is performed on the reference object keypoints to generate another straight line representing the main direction or structure of the reference object. By fitting a straight line, a clear spatial connection between the tree and the reference object is established, which helps with more complex analysis tasks later, such as measuring the angle between the two, the change in distance, or inferring the state of the tree based on the direction of the straight line.

[0128] It should be noted that if the key points of the tree are larger than the fitting threshold and cannot be fitted into a straight line, it proves that the tree is leaning and encroaching on the road, which directly forms the evaluation result. The fitting threshold is a threshold set by experience.

[0129] S140: Determine whether the current weather is extreme weather.

[0130] In this embodiment, weather information for the current location is obtained in real time from a reliable meteorological service provider. This may include multiple parameters such as temperature, humidity, precipitation, wind speed, wind direction, etc. Based on local environmental characteristics and historical data analysis, a set of standards are pre-set to define which weather conditions are considered "extreme". For example, when the wind speed exceeds a certain threshold, the precipitation increases sharply in a short period of time, or the temperature fluctuates abnormally, it can be considered extreme weather. Specifically, the real-time weather data is compared with the preset extreme weather standards. If any indicator is found to exceed the set safety range, it is determined to be extreme weather.

[0131] S150: If the current weather is not extreme weather, determine the tree status assessment based on the two fitting straight lines to obtain an assessment result.

[0132] In this embodiment, the evaluation results include the status of the trees.

[0133] Specifically, the parallel angle of the two fitting straight lines is compared with two set thresholds to determine the evaluation result, wherein the two set thresholds respectively include a first threshold and a second threshold, and the first threshold is smaller than the second threshold; when the parallel angle of the two fitting straight lines is smaller than the first threshold, it is determined that the tree is growing normally; when the parallel angle of the two fitting straight lines is between the first threshold and the second threshold, it is determined that the tree is tilted; when the parallel angle of the two fitting straight lines is larger than the second threshold, it is determined that the tree has fallen.

[0134] Calculate the angle between the straight line fitted by the key points of the tree and the straight line fitted by the key points of the reference object. This angle reflects the difference in direction of the tree relative to the reference object. Compare the calculated angle with the preset first threshold and second threshold. These two thresholds are key parameters used to distinguish different tree states. They should be obtained based on statistical analysis of a large amount of sample data and can be adjusted appropriately according to different application scenarios. If the parallel angle of the two fitted straight lines is less than the first threshold, it means that the relative position between the tree and the reference object has hardly changed, and the tree is determined to be growing normally. If the parallel angle of the two fitted straight lines is between the first threshold and the second threshold, it indicates that the tree may be tilted to a certain extent, such as Figure 2 If the parallel angle of the two fitting lines is greater than the second threshold, it means that the tree has shifted significantly and is in a fallen state, as shown in FIG. Figure 3 shown.

[0135] Through this method, the health status of trees can be automatically monitored, potential problems can be detected in time and corresponding measures can be taken.

[0136] S160: Output the evaluation result as an image and position information corresponding to the tree leaning or falling.

[0137] In this embodiment, once a tree is detected to be leaning or falling, a detailed assessment report is automatically generated, including but not limited to:

[0138] Original Image: Displays the most recent image or images of the tree in question for easy visualization.

[0139] Annotation information: Mark the fitting line and its angle on the image in graphic or text form to clearly show the status characteristics of the trees.

[0140] Location details: Provide the exact geographical coordinates (latitude and longitude) of the tree and, if possible, attach a map screenshot showing its location.

[0141] Recommended measures: Provide corresponding maintenance suggestions for different types of abnormal situations, such as strengthening supports, pruning branches and leaves, or replanting.

[0142] S170: If the current weather is extreme weather, determine whether the tree is submerged by water.

[0143] In this embodiment, under extreme weather conditions, such as heavy rain or flooding, trees may be submerged by water, which will affect subsequent lodging monitoring. Therefore, before evaluating the tree status, the system first needs to determine whether the current environment belongs to extreme weather and check whether the trees are flooded. This step relies on integrated meteorological data and image analysis algorithms.

[0144] Real-time meteorological information can be obtained (for example, from a public API or local sensor) to determine whether the current weather is extreme. If it is confirmed to be extreme weather, continuous frame image analysis technology is used to determine whether the trees are submerged by water. Specific methods include but are not limited to computer vision technologies such as background difference method and optical flow method, which can effectively identify dynamically changing water levels and the relationship between trees and water surfaces.

[0145] S180. If the tree is not submerged by water, the changing trend of the angle between the two fitting straight lines is continuously monitored to determine whether the tree has fallen. If the tree has fallen, the position and direction of the tree's fall are determined to obtain an evaluation result, and the step S160 is executed.

[0146] In this embodiment, once it is confirmed that the tree is not submerged by water, the system will continue to execute the tree fall monitoring process. Considering that extreme weather may cause trees to sway, a single picture is prone to misjudgment. Therefore, this embodiment uses a continuous frame image analysis method to accurately record the tree fall process and save key data as output results.

[0147] In one embodiment, the above-mentioned step S180 may include steps S181 to S184.

[0148] S181, continuously acquiring initial images and inputting them into a tree and reference object key point detection model to determine tree key points and reference object key points, and performing straight line fitting respectively to obtain two fitting straight lines corresponding to multiple frame images.

[0149] In this embodiment, initial images are captured periodically and input into a pre-trained "tree and reference object key point detection model". The model can automatically identify and mark the key points of the main structural features of each tree (such as the trunk) and the surrounding fixed reference objects (such as the edge of the building, street lamp posts, etc.). For the image obtained at each time point, straight line fitting is performed on the key points of the tree and the reference object, respectively, to obtain two representative fitting lines. This process will generate a set of fitting lines corresponding to multiple frames of images that change over time.

[0150] The detailed implementation steps of this step can be found in steps S110 to S130 and will not be described again here.

[0151] S182, calculating the angle between two fitting straight lines corresponding to each frame of image.

[0152] In this embodiment, the angle between the two fitted straight lines extracted from each frame of image is calculated, and the angle reflects the change of the posture of the tree relative to the fixed reference object.

[0153] S183, continuously monitoring the change trend of the angle in chronological order to determine whether the tree has fallen; wherein, when the angles corresponding to the fitted straight lines of all frame images are less than the first threshold, or when the angle corresponding to the fitted straight line of a frame image exceeds the first threshold and then returns to less than the first threshold, it is determined that the tree maintains a normal growth state; if the angles corresponding to the fitted straight lines of consecutive frame images are always between the first threshold and the second threshold and have no obvious changes, it is determined that the tree has tilted to obtain an evaluation result; when the angle corresponding to the fitted straight line of a frame image exceeds the first threshold for the first time, and in subsequent frame images, the angle corresponding to the fitted straight line increases rapidly until it exceeds the second threshold and then tends to be fixed, it is determined that the tree has fallen;

[0154] S184. When a tree has fallen, determine the relative position of the tree in the initial image and the steering angle of the tree to obtain the falling direction of the tree, and obtain an evaluation result in combination with the position information in the initial image.

[0155] S185, recording the time point when the tree starts to fall, the shooting direction and location information of the camera, and adding them to the evaluation result.

[0156] In this embodiment, once it is confirmed that a tree has fallen, the system will further analyze the relative position of the tree in the initial image and its steering angle, and then determine the specific direction of the tree's fall. For example, when the tree is on the left side of the picture and the steering angle is positive, it means that it has fallen toward the road; conversely, if the steering angle is negative, it means that the tree has fallen toward the sidewalk. Similarly, there are corresponding interpretation rules for the situation on the right side of the picture.

[0157] Through the above steps, not only can the health of trees in extreme weather be monitored in real time, but potential safety hazards can also be discovered in time.

[0158] S190: If the tree is submerged by water, a reference object and water accumulation relationship segmentation model is used to calculate the height of the submerged part of the tree and update the position of the key points of the tree, and a fitting straight line is formed according to the updated key points of the tree.

[0159] In this embodiment, when it is confirmed that a tree is partially or completely submerged by water under extreme weather conditions, in order to ensure the accuracy of the monitoring system, this situation needs to be specially handled. Specifically, the system will use the reference object and water relationship segmentation model to calculate the height of the submerged part of the tree, and update the key point position of the tree based on this information, so as to continue to evaluate the tree status.

[0160] In one embodiment, the above-mentioned step S190 may include steps S191 to S196.

[0161] S191. Use a reference object and water accumulation relationship segmentation model to divide the water accumulation boundary line according to the relationship between the root, trunk or crown of the tree and the water accumulation.

[0162] In this embodiment, the water accumulation range is determined so as to accurately calculate the flooded height of the trees later.

[0163] Specifically, the pre-trained reference object and water accumulation relationship segmentation model is used to analyze the contact between the tree roots, trunks or crowns and the water accumulation in the image, and automatically identify and divide the water accumulation boundary (i.e., the water surface line). This model is trained using a deep learning algorithm and can efficiently distinguish different objects and backgrounds.

[0164] S192. Identify and mark the reference information above the water accumulation line.

[0165] In this embodiment, the system will automatically identify and mark fixed reference objects (such as lamp posts, building edges, etc.) located above the water line in the image. These reference objects will be used for subsequent size ratio conversion.

[0166] S193, obtaining the width and height of the mask pixels of the reference object above submerged by the accumulated water.

[0167] In this embodiment, for each marked reference object, the system generates a corresponding binary mask and measures the pixel width and height within the mask, which represent the visual size of the reference object in the image.

[0168] S194, using the actual target object size ratio pre-stored in the reference object and water accumulation relationship segmentation model, convert the mask pixel value into the corresponding real physical size, and determine the specific height of the part submerged by the water accumulation to obtain the water accumulation depth.

[0169] In this embodiment, based on the actual target object size ratio stored in the model, the system can convert the pixel size obtained in the previous step into the corresponding real physical size. Then, by comparing the real height of the unflooded reference object with the height difference calculated at present, the actual height of the flooded part of the tree, that is, the depth of the water accumulation, is obtained.

[0170] S195. Establish new key point coordinates according to the water depth and the key point positions of the trees in the flooded part, combined with the reference object information above the water accumulation line, to obtain updated tree key points.

[0171] In this embodiment, the key points of the trees are redefined according to the depth of the water accumulation and the position of the original key points of the trees, combined with the information of the reference objects above the water accumulation line. For example, new key points are added based on the original key points, such as Figure 4 The light blue points shown in the figure reflect the true posture of the tree after being flooded. This step ensures that the tree posture changes can be effectively tracked even in the case of water accumulation. Therefore, the tree key points are updated to adapt to the impact of water accumulation. Figure 4 In the figure, the red one is the reference light pole. After fitting the key points with straight lines, the two fitted straight lines are compared. The angles of the fitted straight lines corresponding to the two trees are greater than the set second threshold, proving that the trees have fallen, and the rest are less than the set first threshold, proving that the trees are growing normally.

[0172] S196. Form a fitting straight line according to the updated tree key points.

[0173] Finally, the updated tree key points are used to create new fitting lines. These lines include not only the structural features of the tree itself, but also the additional tilt or deformation caused by water accumulation. After that, the system will continue to judge the state of the tree according to the previously set angle threshold - normal growth, tilt or lodging.

[0174] Through the above steps, even in a complex waterlogging environment, the present application can effectively monitor the state changes of trees, ensure the safety of urban greening facilities and smooth traffic. This method improves the level of intelligence in tree management under extreme weather conditions, helps to timely detect potential risks and take appropriate preventive measures. Among them, it should be noted that the working principle of the reference object and waterlogging relationship segmentation model can refer to the reference object and waterlogging relationship segmentation model mentioned in Chinese patent CN118691838B.

[0175] S200, finally confirming the state of the tree based on the fitting straight line formed by fitting the key points of the reference object and the fitting straight line formed by the updated key points of the tree to obtain an evaluation result, and executing the step S160.

[0176] It is fitted based on the key points of stable reference objects above the water line (such as light poles, buildings, etc.). The tree fitting line is re-fitted based on the updated tree key points.

[0177] Use geometric algorithms or linear algebra methods to calculate the angle between the fitted straight line of the reference object and the fitted straight line of the tree.

[0178] If the angle is less than the first threshold, it is considered that the tree maintains a normal vertical growth state and has no obvious risk of tilting or falling.

[0179] If the angle is between the first and second thresholds, it indicates that the tree may be in a slight to moderate leaning state and requires further observation and consideration of reinforcement or other preventive measures.

[0180] If the angle is greater than the second threshold, it warns that the tree is at high risk of falling, and the emergency plan should be immediately activated, an on-site inspection should be carried out, and necessary intervention measures should be implemented.

[0181] The assessment results will be output in the form of a report, including but not limited to the specific posture information of the tree, the potential risk level and the recommended action plan. At the same time, all relevant data will be recorded to support subsequent data analysis and model optimization.

[0182] For trees marked as leaning or at risk of falling, the system will continue to track their changes and dynamically adjust the assessment conclusions based on the latest monitoring data. In addition, other environmental factors (such as wind speed, soil moisture, etc.) can be combined to comprehensively assess the safety of the trees.

[0183] In addition, in this embodiment, multi-scale image processing technology can be used to improve the adaptability of images under different lighting conditions. By adjusting the brightness and contrast of the image, or using an automatic exposure compensation algorithm, trees and their key points can still be accurately detected under different lighting environments. In complex environments, especially in scenes with severe shadows and reflections, image denoising, lighting normalization and other technologies can be used to reduce the impact of light changes.

[0184] By training deep learning models (such as convolutional neural networks, Transformer models, etc.), we can adapt to the changes in tree morphology under different viewing angles. Through the training of large-scale data sets, the model can extract key features from images of multiple angles, different heights and distances, and improve the model's recognition accuracy of tree status. If the system can obtain data from multiple viewing angles, it can enhance the 3D reconstruction of trees through multi-view image fusion technology to improve the accurate judgment of tilt or fall status.

[0185] In urban environments, trees are often blocked by obstacles such as buildings and vehicles. Through object detection and instance segmentation algorithms (such as Mask R-CNN, YOLO, etc.), the relationship between trees and obstructions is identified and separated to avoid the influence of obstacles on tree monitoring results. Introduce depth cameras or stereo vision technology to provide depth information about the scene. This helps to identify the parts blocked by obstacles, and at the same time, the relative position of trees and obstacles is determined through depth information to reduce misjudgment.

[0186] By analyzing the time series change data of trees (such as the change of the position of key points of trees over time), even if there is noise or occlusion in a single image, it is possible to infer the change trend of the time series and identify whether there is a risk of tilting or falling. Use data enhancement technology (such as image rotation, translation, cropping, etc.) to simulate different environmental conditions and train the stability of the model in various scenarios. At the same time, through multi-frame image fusion technology, the information of multiple images is integrated to improve the reliability of monitoring results.

[0187] The model parameters are adjusted in real time according to the actual environment and sensor data. For example, if the system detects a significant change in weather or lighting conditions, the image preprocessing and the working mode of the model can be dynamically adjusted to adapt to different environmental changes. In actual applications, the prediction rules of the model in a specific environment are adjusted in combination with monitoring results and manual feedback, so that it has stronger fault tolerance in complex urban environments.

[0188] These fault-tolerant mechanisms ensure that the algorithm can maintain high accuracy and reliability in complex and dynamic urban environments, overcoming the challenges brought by changes in light, perspective, and obstacles.

[0189] In another embodiment, due to the complexity of shooting angles and scenes in urban environments, image analysis from a single perspective is difficult to provide comprehensive information, especially when the overhead angle is large, which can easily lead to misjudgment. In addition to using low-altitude cameras, drones or overhead cameras can also be used to shoot the same area from multiple different angles (including side views, top views, etc.); image data from different perspectives are aligned and fused through image registration algorithms to form a more complete and three-dimensional model of trees and surrounding environments; computer vision technology is used to reconstruct the fused multi-perspective images in three dimensions, so as to more accurately calculate the actual posture of the trees, including but not limited to parameters such as tilt angle and degree of curvature. The problem of insufficient accuracy caused by shooting angle limitations in single-perspective image analysis is solved; the overall accuracy of tree status assessment is improved, and the possibility of misjudgment caused by differences in perspective is reduced.

[0190] Specifically, the framework combines the advantages of low-altitude cameras, drones, and overhead camera equipment to ensure high-quality image capture of the target trees and their surroundings from multiple angles (such as side view, top view, etc.).

[0191] Low-altitude cameras are installed in fixed positions and are mainly responsible for obtaining basic image information from the front or side of the tree. Drones are equipped with high-definition cameras and GPS positioning systems, and can move flexibly within a designated area to provide image data taken from different heights and angles. Elevated camera equipment is set up at higher locations, such as street light poles or other buildings, to capture the top structure and overall outline of trees.

[0192] The collected raw images from different perspectives need to go through a series of preprocessing steps before they can be effectively integrated together. Specifically, they include:

[0193] Use algorithms such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded-Up Robust Features) to identify key feature points in each image.

[0194] The corresponding relationship is found through the RANSAC (Random Sample Consensus) algorithm, and accurate alignment between images is achieved based on these matching points.

[0195] The weighted average method or other advanced fusion strategies are used to combine multiple images with large overlapping areas into a seamless and information-rich panorama.

[0196] Using computer vision technology and deep learning models, the fused panoramic images are deeply analyzed to construct a three-dimensional model of the tree and evaluate its posture characteristics. Specifically, the depth value corresponding to each pixel is obtained by using the principle of stereo vision or directly applying the RGB-D camera. Based on the depth information, a spatial distribution containing a large number of discrete points, the so-called "point cloud", is created as the basic material for subsequent modeling. The point cloud is smoothed using the tools provided in the PCL (Point Cloud Library) library, and the main axis representing the direction of the tree trunk is extracted through the PCA (Principal Component Analysis) method. Based on the tree skeleton information obtained above, the angle change of the tree relative to the ground, such as inclination, curvature and other indicators, is accurately measured to determine whether there is a risk of lodging.

[0197] In addition, deploy small servers or smart terminals close to the data source to complete preliminary data processing tasks, reduce the pressure on the cloud and improve the response speed. According to the characteristics of mobile terminals or embedded platforms, choose a suitable miniaturized neural network architecture to ensure that it can work normally in a resource-constrained environment. Considering the error accumulation problem that may be caused by changes in the external environment, regularly perform self-check procedures and adjust system parameters to maintain long-term accuracy.

[0198] In another embodiment, for the first threshold and the second threshold mentioned in the first embodiment, the stability of the detection results under different conditions (such as seasonal changes, changes in lighting conditions, etc.) needs to be further improved. At this time, an integrated environmental sensor (such as a thermometer, a hygrometer, an anemometer, etc.) can be used to obtain information on external factors affecting the stability of trees in real time. Collect and store data records of past tree status assessments, and establish a historical database containing multiple environmental variables and their corresponding tree states. Use machine learning algorithms (such as support vector machines, random forests, etc.) to train the above data and build a model that can predict the optimal threshold range under the current environment. According to the real-time monitored environmental parameters and combined with the output of the trained machine learning model, the threshold standard for judging the tilt or fall of trees is automatically adjusted. The intelligent and adaptive threshold setting is realized, and the system's adaptability to different environmental conditions is improved. The need for manual intervention is reduced, and the automation level and decision-making efficiency of the system are enhanced.

[0199] Specifically, in order to achieve accurate prediction of tree stability, this technical solution introduces an integrated environmental sensor network to obtain real-time information on external factors that affect tree stability. These sensors may include but are not limited to:

[0200] Thermometer: Measures air temperature.

[0201] Hygrometer: detects relative humidity of air.

[0202] Anemometer: records wind speed and direction.

[0203] Soil Moisture Sensor: Understand the moisture content of the soil around the roots.

[0204] Rainfall sensor: monitors precipitation.

[0205] Light Intensity Sensor: Measures the intensity of solar radiation.

[0206] All sensors should have high precision and low power consumption, and transmit the collected data to the central control system or cloud server for storage and processing through wireless communication modules (such as LoRa, ZigBee). When new environmental data arrives, the pre-trained model is immediately called for rapid prediction to obtain the corresponding threshold range. The actual measured tree posture parameters (such as tilt angle, bending degree) are compared with the predicted thresholds. If they exceed the safety limit, an alarm is triggered. If false alarms are found to occur repeatedly, the adaptive adjustment mechanism will be automatically started to recalibrate the model parameters to ensure long-term stable operation.

[0207] The above-mentioned tree-falling detection method combined with reference object image analysis, by using a low-altitude camera to shoot images, can provide a perspective closer to the ground, capture detailed information about trees and the surrounding environment, and improve the accuracy of tree-falling monitoring, especially in complex urban environments, effectively avoiding occlusion and distortion problems caused by high-altitude perspectives; by combining the key point detection model of trees and reference objects, it can accurately identify the spatial relationship between trees and reference objects, making the judgment of the falling state more reliable and reducing misjudgments caused by environmental interference or changes in perspective; by fitting two straight lines formed by the key points of trees and reference objects respectively, it can accurately evaluate whether the trees are tilted or fallen, enhancing the system's perception of changes in tree status in a dynamic environment. It can effectively improve the accuracy of monitoring; by judging whether the weather is extreme in real time, it can intelligently adjust the monitoring strategy, for example, automatically consider the impact of waterlogging in extreme weather conditions, ensuring the adaptability and accuracy of the monitoring method under various weather conditions; under extreme weather conditions, the reference object and waterlogging relationship segmentation model is used to accurately calculate the height of the flooded part of the tree, and update the key point position of the tree, effectively avoiding the interference of waterlogging on the judgment of lodging, and further improving the reliability of monitoring; combined with the output of tree lodging detection results and position information, the system can feedback the lodging situation in real time, and dynamically adjust the evaluation according to the latest key point position, ensuring the efficiency and stability of the tree lodging monitoring system in complex urban environments, and effectively improving the safety of urban greening management.

[0208] That is, the above method can provide a perspective closer to the ground by taking images with a low-altitude camera, accurately capture the details of trees and the surrounding environment, and improve the accuracy of lodging monitoring, especially in complex urban environments, avoiding occlusion and distortion of high-altitude perspectives. Combined with the key point detection model of trees and reference objects, the spatial relationship between the two can be accurately identified, reducing misjudgments caused by environmental interference or changes in perspective. By fitting the key points of trees and reference objects to form a straight line, it is possible to accurately assess whether the trees are tilted or lodged, and enhance the system's ability to perceive changes in tree status. In extreme weather, the system can automatically adjust the monitoring strategy, such as considering the impact of water accumulation, to ensure adaptability and accuracy under various weather conditions. Using the segmentation model of the relationship between reference objects and water accumulation, the height of the flooded part of the tree can be accurately calculated, and the position of the key points of the tree can be updated, thereby avoiding water accumulation interfering with the judgment of lodging and improving monitoring reliability. Combining the tree lodging detection results and location information, the system can provide real-time feedback on the lodging situation, and dynamically adjust the assessment based on the latest data, ensuring efficiency and stability in complex environments, and improving the safety of urban greening management.

[0209] Figure 5 FIG. 3 is a schematic block diagram of a tree fall detection system 300 combined with reference image analysis provided by an embodiment of the present invention. Figure 5 As shown, corresponding to the above tree falling detection method combined with reference image analysis, the present invention also provides a tree falling detection system 300 combined with reference image analysis. The tree falling detection system 300 combined with reference image analysis includes a unit for executing the above tree falling detection method combined with reference image analysis, and the system can be configured in a server. Figure 5 The tree fall detection system 300 combined with reference image analysis includes an image acquisition unit 301, a key point detection unit 302, a fitting unit 303, a judgment unit 304, a first evaluation unit 305, an output unit 306, a water accumulation judgment unit 307, a second evaluation unit 308, a key point update unit 309 and a third evaluation unit 310.

[0210] The image acquisition unit 301 is used to acquire an initial image taken by a low-altitude camera; the key point detection unit 302 is used to input the initial image into a tree and reference object key point detection model to determine the tree key points and the reference object key points; the fitting unit 303 is used to fit and connect the tree key points and the reference object key points to form two fitting straight lines; the judgment unit 304 is used to judge whether the current weather is extreme weather; the first evaluation unit 305 is used to determine the tree status evaluation according to the two fitting straight lines to obtain an evaluation result if the current weather is not extreme weather; the output unit 306 is used to output the evaluation result as an image and position information corresponding to the tree tilt or fall; the water accumulation judgment unit 307 is used to judge whether the tree is submerged by water if the current weather is extreme weather; the second evaluation unit 308 is used to determine whether the tree is submerged by water if the current weather is extreme weather. If the tree is not submerged by water, the changing trend of the angle between the two fitting lines is continuously monitored to determine whether the tree has fallen, and when the tree has fallen, the position and direction of the tree's fall are determined to obtain an evaluation result, and the output evaluation result is an image and position information corresponding to the tree's tilt or fall; a key point updating unit 309 is used to calculate the height of the submerged part of the tree and update the position of the tree's key points using a reference object and water accumulation relationship segmentation model if the tree is submerged by water, and form a fitting line based on the updated tree's key points; a third evaluation unit 310 is used to finally confirm the state of the tree based on the fitting line formed by fitting the reference object's key points and the fitting line formed by the updated tree's key points to obtain an evaluation result, and the output evaluation result is an image and position information corresponding to the tree's tilt or fall.

[0211] In one embodiment, if Figure 6 As shown, the key point detection unit 302 includes:

[0212] A feature extraction subunit is used to input the initial image into the tree and reference object key point detection model, use the convolution layer to extract the multi-scale features in the initial image, and generate a feature map; a downsampling and flattening subunit is used to downsample the feature map using a space reduction module, and flatten it to form a series of key and value tags; a score calculation subunit is used to evaluate the similarity based on the distance between the tags, and determine the importance score of the cluster center using the local density and the minimum high density distance; a new tag formation subunit is used to determine the highest tag of the importance score as the cluster center, and assign the remaining tags to the nearest center, and predict the importance score, and determine to merge the tags of the same type into a new tag according to the importance score; a transfer subunit is used to input the new tag into the Transformer block to transfer the feature information from the tag to the new tag; an upsampling subunit is used to perform feature upsampling on the new tag and restore it to the resolution of the feature map to obtain a high-resolution heat map; a key point selection subunit is used to select the brightest position of the high-resolution heat map as the key point to obtain the tree key point and the reference object key point.

[0213] In one embodiment, the score calculation subunit includes:

[0214] A clustering processing module is used to perform clustering processing on all the tags, evaluate the similarity between all the tags by calculating the distance between the tags, and estimate the local density based on the average distance between the tag and several nearest neighbor tags; a distance index determination module is used to determine, for each tag, the distance from the tag to other tags with higher local density, and select the minimum distance as the distance index. For the tag with the highest local density, the distance index of the tag is set to the maximum distance between the tag and all other tags; a score determination module is used to determine the importance score of each tag by combining the local density and the distance index.

[0215] In one embodiment, the key point selection subunit is used to use argmax to select the point with the largest value in the high-resolution heat map as the key point, so as to obtain the tree key point and the reference object key point.

[0216] In one embodiment, the first evaluation unit 305 is used to compare the parallel angle of the two fitting straight lines with two set thresholds to determine an evaluation result, wherein the two set thresholds include a first threshold and a second threshold, respectively, and the first threshold is less than the second threshold; when the parallel angle of the two fitting straight lines is less than the first threshold, it is determined that the tree is growing normally; when the parallel angle of the two fitting straight lines is between the first threshold and the second threshold, it is determined that the tree is tilted; when the parallel angle of the two fitting straight lines is greater than the second threshold, it is determined that the tree has fallen.

[0217] In one embodiment, the second evaluation unit 308 includes:

[0218] The continuous processing subunit is used to continuously acquire the initial image and input it into the tree and reference object key point detection model to determine the tree key points and the reference object key points, and perform straight line fitting respectively to obtain two fitting straight lines corresponding to multiple frames of images; the angle calculation subunit is used to calculate the angle between the two fitting straight lines corresponding to each frame of image; the continuous monitoring subunit is used to continuously monitor the change trend of the angle in the order of occurrence in time to determine whether the tree has fallen; wherein, when the angles corresponding to the fitting straight lines of all frame images are less than the first threshold value, or when the angles corresponding to the fitting straight lines of a certain frame image exceed the first threshold value and then return to less than the first threshold value, it is determined that the angles have fallen. The tree maintains a normal growth state; if the angle corresponding to the fitting straight line of the continuous frame image is always between the first threshold and the second threshold and has no obvious change, it is determined that the tree has tilted to obtain an evaluation result; when the angle corresponding to the fitting straight line of a frame image exceeds the first threshold for the first time, and in the subsequent frame image, the angle corresponding to the fitting straight line increases rapidly until it exceeds the second threshold, and then tends to be fixed, it is determined that the tree has fallen; the direction subunit is used to determine the relative position of the tree in the initial image and the guiding angle of the tree when the tree has fallen, so as to obtain the falling direction of the tree, and combine the position information in the initial image to obtain the evaluation result. The recording subunit is used to record the time point when the tree begins to fall, the shooting direction of the camera and the position information, and add them to the evaluation result.

[0219] In one embodiment, the key point updating unit 309 includes:

[0220] The demarcation line determination subunit is used to use the reference object and water accumulation relationship segmentation model to divide the water accumulation demarcation line according to the relationship between the root, trunk or crown of the tree and the water accumulation; the identification subunit is used to identify and mark the reference object information above the water accumulation line; the width and height acquisition subunit is used to obtain the width and height of the mask pixel of the reference object above the water accumulation; the conversion subunit is used to use the actual target object size ratio pre-stored in the reference object and water accumulation relationship segmentation model to convert the mask pixel value into the corresponding real physical size, and determine the specific height of the part submerged by the water accumulation to obtain the depth of the water accumulation; the key point establishment subunit is used to establish new key point coordinates according to the key point position of the tree at the water accumulation depth and the flooded part, combined with the reference object information above the water accumulation line, to obtain updated tree key points; the fitting subunit is used to form a fitting straight line based on the updated tree key points.

[0221] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned tree fall detection system 300 combined with reference object image analysis and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0222] The tree fall detection system 300 combined with reference image analysis can be implemented in the form of a computer program. Figure 6 Runs on the computer device shown.

[0223] See also Figure 6 , Figure 6 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0224] See also Figure 6 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0225] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a tree fall detection method combined with reference object image analysis.

[0226] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0227] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a tree fall detection method combined with reference object image analysis.

[0228] The network interface 505 is used to communicate with other devices over the network. Figure 6 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0229] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0230] Acquire an initial image taken by a low-altitude camera; input the initial image into a tree and reference object key point detection model to determine the tree key points and the reference object key points; respectively fit and connect the tree key points and the reference object key points to form two fitting straight lines; determine whether the current weather is extreme weather; if the current weather is not extreme weather, determine the tree status assessment based on the two fitting straight lines to obtain an assessment result; output the assessment result as an image and position information corresponding to the tree tilt or fall; if the current weather is extreme weather, determine whether the tree is submerged by water; if the tree is not submerged by water, continuously monitor the change trend of the angle between the two fitting straight lines to determine whether the tree has fallen, and when the tree has fallen, determine the position and direction of the tree's fall to obtain an evaluation result, and execute the output evaluation result as an image and position information corresponding to the tree's tilt or fall; if the tree is submerged by water, use a reference object and water accumulation relationship segmentation model to calculate the height of the submerged part of the tree and update the position of the tree's key points, and form a fitting line based on the updated tree's key points; finally confirm the state of the tree based on the fitting line formed by fitting the reference object's key points and the fitting line formed by the updated tree's key points to obtain an evaluation result, and execute the output evaluation result as an image and position information corresponding to the tree's tilt or fall.

[0231] The tree and reference object key point detection model is obtained by training a transformer-based network using images containing trees and reference objects with key point labels as sample sets.

[0232] In one embodiment, when the processor 502 implements the step of inputting the initial image into the tree and reference object key point detection model to determine the tree key points and the reference object key points, the processor 502 specifically implements the following steps:

[0233] The initial image is input into a tree and reference object key point detection model, and a convolutional layer is used to extract multi-scale features in the initial image to generate a feature map; the feature map is downsampled using a spatial reduction module and flattened to form a series of key and value tags; similarity is evaluated based on the distance between the tags, and the importance score of the cluster center is determined using local density and minimum high density distance; the tag with the highest importance score is determined to be selected as the cluster center, and the remaining tags are assigned to the nearest center, and the importance score is predicted, and the tags of the same type are determined to be fused into a new tag according to the importance score; the new tag is input into a Transformer block to transfer feature information from the tag to the new tag; the new tag is feature upsampled and restored to the resolution of the feature map to obtain a high-resolution heat map; the brightest position of the high-resolution heat map is selected as the key point to obtain the tree key point and the reference object key point.

[0234] In one embodiment, when the processor 502 implements the step of evaluating the similarity based on the distance between the tags and determining the importance score of the cluster center using the local density and the minimum high density distance, the processor 502 specifically implements the following steps:

[0235] All the markers are clustered, and the similarities between all the markers are evaluated by calculating the distances between the markers, and the local density is estimated based on the average distance between the marker and several nearest neighbor markers; for each marker, the distance from the marker to other markers with higher local density is determined, and the minimum distance is selected as the distance index, and for the marker with the highest local density, the distance index of the marker is set to the maximum distance between the marker and all other markers; the importance score of each marker is determined by combining the local density and the distance index.

[0236] In one embodiment, when the processor 502 implements the step of selecting the brightest position of the high-resolution heat map as the key point to obtain the tree key point and the reference object key point, the processor 502 specifically implements the following steps:

[0237] The point with the largest value in the high-resolution heat map described by argmax is used as the key point to obtain the tree key point and the reference object key point.

[0238] In one embodiment, when the processor 502 implements the step of determining the tree state evaluation according to the two fitting straight lines to obtain the evaluation result, the processor 502 specifically implements the following steps:

[0239] The parallel angle of the two fitting straight lines is compared with two set thresholds to determine an evaluation result, wherein the two set thresholds include a first threshold and a second threshold, respectively, and the first threshold is less than the second threshold; when the parallel angle of the two fitting straight lines is less than the first threshold, it is determined that the tree is growing normally; when the parallel angle of the two fitting straight lines is between the first threshold and the second threshold, it is determined that the tree is tilted; when the parallel angle of the two fitting straight lines is greater than the second threshold, it is determined that the tree has fallen.

[0240] In one embodiment, the processor 502 implements the step of continuously monitoring the change trend of the angle between the two fitting straight lines to determine whether the tree has fallen, and when the tree has fallen, determining the falling position and direction of the tree to obtain an evaluation result, and executing the step of outputting the evaluation result as an image and position information corresponding to the inclination or falling of the tree, specifically implementing the following steps:

[0241] By continuously acquiring initial images and inputting them into a tree and reference object key point detection model to determine tree key points and reference object key points, and performing straight line fitting respectively, two fitting straight lines corresponding to multiple frame images are obtained;

[0242] Calculate the angle between the two fitting lines corresponding to each frame image; continuously monitor the change trend of the angle in chronological order to determine whether the tree has fallen; wherein, when the angles corresponding to the fitting lines of all frame images are less than the first threshold, or when the angle corresponding to the fitting line of a frame image exceeds the first threshold and then returns to less than the first threshold, it is determined that the tree maintains a normal growth state; if the angles corresponding to the fitting lines of consecutive frame images are always between the first threshold and the second threshold without obvious changes, it is determined that the tree has tilted to obtain an evaluation result; when the angle corresponding to the fitting line of a frame image exceeds the first threshold for the first time, and in subsequent frame images, the angle corresponding to the fitting line increases rapidly until it exceeds the second threshold and then tends to be fixed, it is determined that the tree has fallen; when the tree has fallen, determine the relative position of the tree in the initial image and the guide angle of the tree to obtain the falling direction of the tree, and obtain the evaluation result in combination with the position information in the initial image.

[0243] In one embodiment, after the processor 502 implements the step of determining the relative position of the tree in the initial image and the steering angle of the tree when the tree has fallen to obtain the falling direction of the tree and combining the position information in the initial image to obtain the evaluation result, it further implements the following steps:

[0244] The time point when the tree begins to fall, the shooting direction and location information of the camera are recorded and added to the evaluation result.

[0245] In one embodiment, when the processor 502 implements the step of using the reference object and water accumulation relationship segmentation model to calculate the height of the flooded part of the tree and update the position of the key points of the tree, and form a fitting line according to the updated key points of the tree, the processor 502 specifically implements the following steps:

[0246] The reference object and water accumulation relationship segmentation model is used to divide the water accumulation boundary line according to the relationship between the root, trunk or crown of the tree and the water accumulation; the reference object information above the water accumulation line is identified and marked; the width and height of the mask pixel of the reference object above the water accumulation is obtained; the mask pixel value is converted into the corresponding real physical size using the actual target object size ratio pre-stored in the reference object and water accumulation relationship segmentation model, and the specific height of the part submerged by the water accumulation is determined to obtain the depth of the water accumulation; according to the key point position of the tree at the water accumulation depth and the flooded part, combined with the reference object information above the water accumulation line, new key point coordinates are established to obtain updated tree key points; and a fitting straight line is formed based on the updated tree key points.

[0247] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0248] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0249] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0250] Acquire an initial image taken by a low-altitude camera; input the initial image into a tree and reference object key point detection model to determine the tree key points and the reference object key points; respectively fit and connect the tree key points and the reference object key points to form two fitting straight lines; determine whether the current weather is extreme weather; if the current weather is not extreme weather, determine the tree status assessment based on the two fitting straight lines to obtain an assessment result; output the assessment result as an image and position information corresponding to the tree tilt or fall; if the current weather is extreme weather, determine whether the tree is submerged by water; if the tree is not submerged by water, continuously monitor the change trend of the angle between the two fitting straight lines to determine whether the tree has fallen, and when the tree has fallen, determine the position and direction of the tree's fall to obtain an evaluation result, and execute the output evaluation result as an image and position information corresponding to the tree's tilt or fall; if the tree is submerged by water, use a reference object and water accumulation relationship segmentation model to calculate the height of the submerged part of the tree and update the position of the tree's key points, and form a fitting line based on the updated tree's key points; finally confirm the state of the tree based on the fitting line formed by fitting the reference object's key points and the fitting line formed by the updated tree's key points to obtain an evaluation result, and execute the output evaluation result as an image and position information corresponding to the tree's tilt or fall.

[0251] The tree and reference object key point detection model is obtained by training a transformer-based network using images containing trees and reference objects with key point labels as sample sets.

[0252] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0253] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned storage medium can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0254] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0255] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0256] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the system of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0257] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0258] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A tree fall detection method combined with reference image analysis, characterized in that: include: Acquire an initial image taken by a low-altitude camera; Inputting the initial image into a tree and reference object key point detection model to determine tree key points and reference object key points; Fitting and connecting the key points of the tree and the key points of the reference object respectively to form two fitting straight lines; Determine whether the current weather is extreme weather; If the current weather is not extreme weather, determining the tree status assessment according to the two fitting straight lines to obtain an assessment result; The output evaluation result is the image and location information corresponding to the tree leaning or falling; If the current weather is extreme weather, determining whether the tree is submerged by water; If the tree is not submerged by water, the changing trend of the angle between the two fitting straight lines is continuously monitored to determine whether the tree has fallen, and when the tree has fallen, the falling position and direction of the tree are determined to obtain an evaluation result, and the output evaluation result is an image and position information corresponding to the tilt or fall of the tree; If the tree is submerged by water, the height of the submerged part of the tree is calculated by using the reference object and water accumulation relationship segmentation model and the position of the key points of the tree is updated, and a fitting straight line is formed according to the updated key points of the tree; The state of the tree is finally confirmed based on the fitting straight line formed by fitting the key points of the reference object and the fitting straight line formed by the updated key points of the tree to obtain an evaluation result, and the output evaluation result is an image and position information corresponding to the tilt or fall of the tree.

2. The tree fall detection method combined with reference image analysis according to claim 1, characterized in that: The tree and reference object key point detection model is obtained by training a transformer-based network using images containing trees and reference objects with key point labels as sample sets.

3. The tree fall detection method combined with reference image analysis according to claim 2, characterized in that: The step of inputting the initial image into a tree and reference object key point detection model to determine tree key points and reference object key points includes: Inputting the initial image into a tree and reference object key point detection model, extracting multi-scale features in the initial image using a convolutional layer, and generating a feature map; The feature map is downsampled using a spatial reduction module and flattened into a series of key and value tokens; Evaluating similarity based on the distance between the markers, and determining the importance score of the cluster center using the local density and the minimum high density distance; Determine that the highest marker with the importance score is selected as the cluster center, and assign the remaining markers to the nearest center, and predict the importance score, and determine whether to merge the markers of the same type into a new marker according to the importance score; Input the new tag into the Transformer block to transfer the feature information from the tag to the new tag; Upsampling the new markers to restore them to the resolution of the feature map to obtain a high-resolution heat map; The brightest position of the high-resolution heat map is selected as the key point to obtain the tree key point and the reference object key point.

4. The tree fall detection method combined with reference image analysis according to claim 3, characterized in that: The similarity is evaluated based on the distance between the markers, and the importance score of the cluster center is determined by using the local density and the minimum high density distance, including: Performing clustering processing on all the markers, evaluating the similarity between all the markers by calculating the distances between the markers, and estimating the local density according to the average distance between the marker and several nearest neighbor markers; For each of the markers, determine the distance from the marker to other markers with higher local density, and select the smallest distance as the distance index. For the marker with the highest local density, set the distance index of the marker to the maximum distance between the marker and all other markers. An importance score for each of the tags is determined by combining the local density and the distance indicator.

5. The tree fall detection method combined with reference image analysis according to claim 3, characterized in that: The brightest position of the high-resolution heat map is selected as a key point to obtain a tree key point and a reference object key point, including: The point with the largest value in the high-resolution heat map described by argmax is used as the key point to obtain the tree key point and the reference object key point.

6. The tree fall detection method combined with reference image analysis according to claim 1, characterized in that: Determining the tree status evaluation according to the two fitting straight lines to obtain an evaluation result includes: The parallel angle of the two fitting straight lines is compared with two set thresholds to determine an evaluation result, wherein the two set thresholds include a first threshold and a second threshold, respectively, and the first threshold is less than the second threshold; when the parallel angle of the two fitting straight lines is less than the first threshold, it is determined that the tree is growing normally; when the parallel angle of the two fitting straight lines is between the first threshold and the second threshold, it is determined that the tree is tilted; when the parallel angle of the two fitting straight lines is greater than the second threshold, it is determined that the tree has fallen.

7. The tree fall detection method combined with reference image analysis according to claim 6, characterized in that: The method of continuously monitoring the change trend of the angle between the two fitting straight lines to determine whether the tree has fallen, and when the tree has fallen, determining the position and direction of the tree falling to obtain an evaluation result, and executing the method of outputting the evaluation result as an image and position information corresponding to the tree tilting or falling, includes: By continuously acquiring initial images and inputting them into a tree and reference object key point detection model to determine tree key points and reference object key points, and performing straight line fitting respectively, two fitting straight lines corresponding to multiple frame images are obtained; Calculate the angle between the two fitting straight lines corresponding to each frame image; The changing trend of the angle is continuously monitored in chronological order to determine whether the tree has fallen; wherein, when the angles corresponding to the fitting straight lines of all frame images are less than the first threshold, or when the angle corresponding to the fitting straight line of a frame image exceeds the first threshold and then returns to less than the first threshold, it is determined that the tree maintains a normal growth state; if the angles corresponding to the fitting straight lines of consecutive frame images are always between the first threshold and the second threshold and have no obvious changes, it is determined that the tree has tilted to obtain an evaluation result; when the angle corresponding to the fitting straight line of a frame image exceeds the first threshold for the first time, and in subsequent frame images, the angle corresponding to the fitting straight line increases rapidly until it exceeds the second threshold and then tends to be fixed, it is determined that the tree has fallen; When a tree has fallen, the relative position of the tree in the initial image and the steering angle of the tree are determined to obtain the falling direction of the tree, and the evaluation result is obtained in combination with the position information in the initial image.

8. The tree fall detection method combined with reference image analysis according to claim 7, characterized in that: When the tree has fallen, the relative position of the tree in the initial image and the steering angle of the tree are determined to obtain the falling direction of the tree, and after obtaining the evaluation result in combination with the position information in the initial image, the method further includes: The time point when the tree begins to fall, the shooting direction and location information of the camera are recorded and added to the evaluation result.

9. The tree fall detection method combined with reference image analysis according to claim 6, characterized in that: The method of using the reference object and water accumulation relationship segmentation model to calculate the height of the flooded part of the tree and update the position of the key points of the tree, and forming a fitting straight line according to the updated key points of the tree, includes: Using the reference object and water accumulation relationship segmentation model, a water accumulation boundary line is divided according to the relationship between the root, trunk or crown of the tree and the water accumulation; Identify and mark reference information above the waterline; Get the mask pixel value of the reference object above submerged by the accumulated water; Using the actual target object size ratio pre-stored in the reference object and water accumulation relationship segmentation model, the mask pixel value is converted into the corresponding real physical size, and the specific height of the part submerged by the water accumulation is determined to obtain the water accumulation depth; According to the water depth and the key point position of the tree in the flooded part, combined with the reference object information above the water accumulation line, new key point coordinates are established to obtain updated tree key points; A fitted straight line is formed based on the updated tree key points.

10. A tree fall detection system combined with reference image analysis, characterized in that: include: An image acquisition unit, used to acquire an initial image taken by a low-altitude camera; A key point detection unit, used for inputting the initial image into a tree and reference object key point detection model to determine the tree key points and the reference object key points; A fitting unit, used for fitting and connecting the key points of the tree and the key points of the reference object to form two fitting straight lines; A judgment unit, used to judge whether the current weather is extreme weather; A first evaluation unit, configured to determine a tree state evaluation based on the two fitting straight lines to obtain an evaluation result if the current weather is not extreme weather; An output unit, used for outputting an image and position information corresponding to the evaluation result of the tree being tilted or fallen; a water accumulation judgment unit, used for judging whether the tree is submerged by water if the current weather is extreme weather; The second evaluation unit is used for continuously monitoring the change trend of the angle between the two fitting straight lines to determine whether the tree has fallen if the tree is not submerged by water, and when the tree has fallen, determining the falling position and direction of the tree to obtain an evaluation result, and executing the output of the evaluation result as an image and position information corresponding to the inclination or falling of the tree; A key point updating unit, for calculating the height of the flooded part of the tree by using a reference object and water accumulation relationship segmentation model and updating the position of the key points of the tree if the tree is flooded, and forming a fitting straight line according to the updated key points of the tree; The third evaluation unit is used to finally confirm the state of the tree based on the fitting straight line formed by fitting the key points of the reference object and the fitting straight line formed by the updated key points of the tree to obtain an evaluation result, and execute the output evaluation result as an image and position information corresponding to the tilt or fall of the tree.

Images