Tree crown sagging dynamic detection method and system based on vehicle-mounted vision

Abstract

The invention discloses a crown sagging dynamic detection method and system based on vehicle-mounted vision. The method comprises the following steps: acquiring road and vegetation videos collected by a camera in a vehicle driving process; identifying the video and marking a crown of which the projection is positioned above the traffic lane; calculating depth information from a pixel point to the camera for the video, and generating a depth map; a visual odometer is adopted to analyze feature point movement tracks of continuous frames in a video, the height of a camera relative to the ground is dynamically calculated on the basis of a depth map by taking the dynamic displacement of the camera as a measurement baseline, and the height of a potential low vertical target relative to the camera is calculated by using a triangulation method. Calculating the height from the bottom of the potential low vertical target to the ground; if the height is lower than a safety threshold value, early warning is carried out. By implementing the method provided by the invention, not only can the monitoring accuracy be improved, but also the time from discovery to disposal can be greatly shortened, so that the traffic safety risk caused by crown droop can be effectively reduced.

Description

Technical Field

[0001] This invention relates to computer vision, and more specifically to a method and system for dynamic detection of drooping tree canopies based on vehicle-mounted vision. Background Technology

[0002] In modern urban environments, street trees not only beautify the roadside landscape but also provide ecological services such as shade and noise reduction. However, over time, these trees grow, and without timely pruning, their canopies can become excessively dense. Furthermore, due to natural growth or adverse weather conditions such as storms and snowstorms, branches can droop below the prescribed safe height. This phenomenon poses a potential safety sag, especially for vehicles with high clearance, such as double-decker buses and freight trucks, increasing the risk of traffic accidents.

[0003] Traditional maintenance methods rely primarily on manual inspections to monitor the growth and safety of roadside trees. This approach is not only inefficient and unable to cover all road sections, leading to the failure to quickly identify and address potential safety hazards, but also lacks precision, as manual inspections typically cannot provide accurate measurement data to assess the specific height of the tree canopy and its impact on traffic. This inaccuracy and low coverage prevent timely and effective resolution of safety hazards, increasing the risk of traffic accidents caused by drooping tree canopies.

[0004] Therefore, it is necessary to design a new method to detect in real time whether tree canopies encroach on the driving lane area and accurately calculate their height above the ground. This would not only improve the accuracy of monitoring but also significantly shorten the time from detection to action, thereby effectively reducing the traffic safety risks caused by drooping tree canopies. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for dynamic detection of drooping tree canopies based on vehicle vision.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a dynamic detection method for drooping tree canopies based on vehicle-mounted vision, comprising: Acquire video footage of roads and vegetation captured by cameras during vehicle operation; The video is used to identify and mark tree canopies projected above the driveway to obtain potential low-hanging targets; The depth information of the distance between the pixels and the camera is calculated for the video, and a depth map is generated; Visual odometry is used to analyze the movement trajectory of feature points in consecutive frames of the video. The dynamic displacement of the camera is used as the measurement baseline. Based on the depth map, the height of the camera relative to the ground is dynamically calculated. The height of the potential low-hanging target relative to the camera is calculated using triangulation. The height of the bottom of the potential low-hanging target from the ground is obtained by combining the height of the camera relative to the ground and the height of the potential low-hanging target relative to the camera. An early warning is issued when the bottom of the potential low-hanging target is below a safety threshold.

[0007] The further technical solution is as follows: the method of analyzing the movement trajectory of feature points in consecutive frames of the video using visual odometry, with the dynamic displacement of the camera as the measurement baseline, includes: Image recognition technology is used to locate and pinpoint specific road feature points on the road surface in the video. When the camera moves, the positional changes of the designated road feature points are continuously tracked between consecutive frames in the video, and the movement trajectory of the designated road feature points in consecutive frames in the video is analyzed. The relative displacement and rotational attitude of the camera between any two moments are calculated based on the movement trajectory to generate a measurement baseline.

[0008] A further technical solution is as follows: Based on the depth map, dynamically calculating the height of the camera relative to the ground includes: Combining the camera's internal parameters and the measurement baseline, the coordinates of the specified road feature points relative to the camera in three-dimensional space are inferred using triangulation, and then fitted into a three-dimensional point cloud. Randomly select several points from the three-dimensional point cloud, and calculate the plane equation describing the road surface based on the selected points; Set the camera's coordinates as the origin coordinates, and use the point-to-plane distance formula with the plane equation to calculate the camera's height relative to the ground.

[0009] The further technical solution is as follows: The height of the potential low-hanging target relative to the camera is calculated using triangulation, and the height of the camera relative to the ground and the height of the potential low-hanging target relative to the camera are combined to obtain the height of the bottom of the potential low-hanging target from the ground, including: Determine the pixel coordinates of the potential low-hanging target when the camera moves from one position to another, and use the camera's intrinsic parameter matrix in combination with the pixel coordinates to determine two angles of the potential low-hanging target in the physical coordinate system with the camera's optical center as the origin; Based on geometric relationships and the principles of trigonometric functions, a system of equations is established by combining the height of the camera above the ground and the two angles, and then solved to obtain the height of the potential low-hanging target relative to the measurement baseline. The height of the bottom of the potential low-hanging target from the ground is calculated based on the height of the camera above the ground and the height of the potential low-hanging target relative to the measurement baseline.

[0010] A further technical solution is as follows: The calculation of the bottom height of the potential low-hanging target from the ground based on the height of the camera above the ground and the height of the potential low-hanging target relative to the measurement baseline includes: The height of the camera above the ground and the height of the potential low-hanging target relative to the measurement baseline are calculated to obtain the height of the bottom of the potential low-hanging target above the ground.

[0011] The further technical solution is as follows: the designated road feature points include lane lines, ground markings, curbs, road surface markings, and road texture details.

[0012] A further technical solution is as follows: Identifying and marking tree canopies projected above the driving lane from the video to obtain potential low-hanging targets includes: The video is used to locate the driving lane area; A semantic segmentation model was used to identify the regions where all tree canopies were located in the video. An intersection operation is performed on the driveway area and the area containing all tree canopies to determine the tree canopies projected over the driveway, thus obtaining potential low-hanging targets.

[0013] The further technical solution is as follows: The step of calculating the depth information of the video pixels from the camera to generate a depth map includes: Using monocular depth vision estimation technology, the distance of each pixel from the camera is calculated from each selected image in the video to generate a depth map.

[0014] The further technical solution is as follows: When the bottom of the potential low-lying target is lower than a safety threshold, an early warning is issued, including: When the bottom of the potential low-hanging target is lower than the safety threshold, an early warning work order is generated to issue an early warning. The early warning work order includes GPS location, on-site photos, estimated height, and detection time.

[0015] This invention also provides a dynamic detection system for drooping tree canopies based on vehicle vision, comprising: The acquisition unit is used to acquire video of roads and vegetation captured by the camera during the vehicle's movement. A target determination unit is used to identify and mark tree canopies projected above the driving lane in the video to obtain potential low-hanging targets; The depth map determination unit is used to calculate the depth information of the distance between the pixels and the camera in the video and generate a depth map; The height calculation unit is used to analyze the movement trajectory of feature points in consecutive frames of the video using visual odometry, take the dynamic displacement of the camera as the measurement baseline, dynamically calculate the height of the camera relative to the ground based on the depth map, calculate the height of the potential low-hanging target relative to the camera using triangulation, and combine the height of the camera relative to the ground and the height of the potential low-hanging target relative to the camera to obtain the height of the bottom of the potential low-hanging target from the ground. The early warning unit is used to issue an early warning when the bottom of the potential low-hanging target is lower than a safety threshold from the ground.

[0016] The advantages of this invention compared to existing technologies are as follows: By analyzing real-time video of roads and vegetation captured by cameras during vehicle movement, this invention first identifies and marks tree canopies projected above the driving lane as potential low-hanging targets. Then, it calculates the depth information from pixels in the video to the camera to generate a depth map, and uses visual odometry to track the movement trajectory of feature points in consecutive frames to dynamically calculate the camera's height relative to the ground. Combined with triangulation, it calculates the exact height of the bottom of the potential low-hanging target from the ground. Once the bottom height is detected to be below a preset safety threshold, the system immediately triggers an early warning mechanism. This not only significantly improves monitoring accuracy but also greatly shortens the time from detection to response, thereby effectively reducing the traffic safety risks caused by drooping tree canopies.

[0017] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the dynamic detection method for low-hanging tree canopies based on vehicle vision provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a video showing roads and vegetation provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of all canopy regions provided in the embodiments of the present invention; Figure 4 A schematic diagram of a depth map provided in an embodiment of the present invention; Figure 5 A schematic diagram illustrating the calculation process of the bottom height of a potential low-hanging target from the ground, provided in an embodiment of the present invention; Figure 6 A schematic block diagram of a dynamic detection system for drooping tree canopies based on vehicle vision provided in an embodiment of the present invention; Figure 7 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0022] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] Please see Figure 1 , Figure 1This is a flowchart illustrating the dynamic detection method for drooping tree canopies based on vehicle vision provided in this embodiment of the invention. This method is applied in a server. The server interacts with a camera, acquiring road and vegetation videos captured by the camera during vehicle movement. It identifies and marks tree canopies projected above the driving lane as potential drooping targets and calculates the height of these targets relative to the ground. Specific steps include using visual odometry to analyze the movement trajectory of feature points in the video to determine the relative displacement and rotation attitude of the camera, dynamically calculating the height of the camera and potential drooping targets relative to the ground using depth mapping and triangulation, and triggering an alert when the bottom of a target is detected to be below a safe threshold. This method not only achieves real-time monitoring of tree canopy intrusion into the driving lane area but also accurately calculates its height above the ground, significantly improving monitoring accuracy and response speed, and effectively reducing traffic safety risks caused by drooping tree canopies.

[0025] Figure 1 This is a flowchart illustrating the dynamic detection method for drooping tree canopies based on vehicle vision provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S150.

[0026] S110. Acquire road and vegetation videos captured by cameras during vehicle operation.

[0027] In this embodiment, a suitable camera is selected based on the specific application scenario. For tasks involving monitoring road conditions and vegetation, high-definition or ultra-high-definition cameras are typically required to ensure clear capture of details. To maximize coverage while maintaining image quality, the camera's field of view needs to be optimized. For example, a wide-angle lens can help capture a wider field of view, while a telephoto lens is more suitable for observing specific objects at a distance. Considering usage under different weather conditions, a camera with good low-light performance and waterproof / dustproof capabilities should be selected.

[0028] While the vehicle is in motion, the camera continues to operate, capturing real-time images of the road ahead and the vegetation on both sides, such as... Figure 2 As shown, to ensure smoothness and reduce distortion, an appropriate frame rate needs to be set according to actual needs. Generally speaking, 30fps is the standard choice, but for fast-moving objects or rapidly changing scenes, a higher frame rate (such as 60fps) is needed to ensure clarity.

[0029] Because video files are large, an effective local storage solution is necessary, such as using a high-capacity solid-state drive (SSD) or SD card. Simultaneously, data should be backed up to the cloud to prevent data loss.

[0030] S120. Identify and mark tree canopies projected above the driving lane in the video to obtain potential low-hanging targets.

[0031] In this embodiment, potential low-hanging targets refer to tree canopies whose projections cover the driving lane and may pose a threat to driving safety.

[0032] In one embodiment, step S120 described above may include steps S121 to S123.

[0033] S121. Locate the driving lane area in the video.

[0034] In this embodiment, each frame of the input video is first preprocessed as necessary, such as adjusting brightness and contrast, and removing noise, in order to optimize the performance of subsequent algorithms.

[0035] Image recognition technology is used to accurately delineate the boundaries of driving lanes. This process needs to take into account robustness under different lighting conditions and weather conditions.

[0036] In addition to lane markings, the location of the curb also needs to be identified in order to more accurately define the extent of the driving lanes. This can be achieved through feature matching or deep learning models.

[0037] S122. Use a semantic segmentation model to identify the regions where all tree canopies are located in the video.

[0038] In this embodiment, a suitable semantic segmentation model is selected and trained using a labeled dataset containing various types of vegetation to ensure that the model can accurately distinguish trees from other background elements. For each frame of video, the trained semantic segmentation model is applied to generate a probability map of each pixel belonging to the "canopy" category. By setting a threshold, the probability map can be converted into a binary image, clearly marking all canopy regions, such as... Figure 3 As shown.

[0039] S123. Perform an intersection operation on the driving lane area and the area where all tree canopies are located to determine the tree canopies projected over the driving lane, thus obtaining potential low-hanging targets.

[0040] In this embodiment, based on the results of the first two steps, namely the defined driveway area and canopy area, the intersection between the two is calculated. This part represents the canopy whose vertical projection falls on the driveway.

[0041] Based on the needs of actual application scenarios, certain height thresholds can be set to further filter out low-hanging tree canopies that truly pose a threat. For example, a tree canopy can be marked as a potential low-hanging target only when the height of its base from the ground is below a certain value.

[0042] Finally, the image segments containing tree canopies identified as potential low-hanging targets are marked and uploaded to the monocular depth vision estimation module for further analysis.

[0043] In summary, step S120 effectively identified potential low-hanging targets that could affect driving safety by accurately locating and analyzing the intersection of the lane area and the canopy area in the video frame. This process not only relies on advanced image processing techniques and deep learning models, but also requires corresponding adjustments for specific environmental conditions to improve the accuracy and reliability of the system.

[0044] S130. Calculate the depth information of the distance between the pixels and the camera in the video, and generate a depth map.

[0045] In this embodiment, the depth map refers to the image generated after processing video frames by a monocular depth vision estimation module, where each pixel corresponds to a value representing the distance from that point to the camera.

[0046] In this embodiment, monocular depth vision estimation technology is used to calculate the distance of each pixel from the camera in each selected image of the video, and generate a depth map.

[0047] Specifically, this process includes the following steps: First, images are read frame by frame from the video stream as input for processing.

[0048] A monocular depth vision estimation algorithm is applied to analyze each pixel in each frame of the input image and estimate the distance of that pixel to the camera. This estimation is based on a trained model that can identify various features in the image and infer the corresponding depth information based on these features.

[0049] For each pixel, its estimated depth value (i.e., distance to the camera) is converted into a grayscale or color value, and this value replaces the color of the original pixel, ultimately forming a new image—a depth map. In this image, different colors or grayscale levels represent different distances; typically, objects closer to the camera are lighter in color (e.g., white), while objects farther away are darker in color (e.g., black). Alternatively, color mapping can be used to represent different depth ranges, such as... Figure 4 As shown, a depth map corresponding to the original scene is displayed. The image is presented in pseudo-color, where blue and red chromatic aspects represent distances from the camera, with red-marked trees indicating the closest trees to the camera.

[0050] This process achieves the goal of extracting three-dimensional spatial information from a two-dimensional image from a single perspective, greatly enhancing the computer vision system's ability to understand its surrounding environment.

[0051] S140. Visual odometry is used to analyze the movement trajectory of feature points in consecutive frames of the video. The dynamic displacement of the camera is used as the measurement baseline. Based on the depth map, the height of the camera relative to the ground is dynamically calculated. The height of the potential low-hanging target relative to the camera is calculated using triangulation. The height of the bottom of the potential low-hanging target from the ground is obtained by combining the height of the camera relative to the ground and the height of the potential low-hanging target relative to the camera.

[0052] In one embodiment, visual odometry is used to analyze the movement trajectory of feature points in consecutive frames of the video, with the dynamic displacement of the camera as the measurement baseline, including steps S141 to S143.

[0053] S141. Use image recognition technology to find and lock the specified road feature points located on the road surface in the video.

[0054] In this embodiment, the designated road feature points include lane lines, ground markings, curbs, road surface markings, and road texture details.

[0055] In this embodiment, an advanced image recognition algorithm is first applied to process the input video stream. The system automatically searches for and locks onto a series of stable and easily trackable road feature points as references. These designated road feature points can be at least one of lane lines, ground markings, curbs, road surface markings, and road texture details. These feature points are chosen because they typically have high recognizability and are relatively stable under different lighting conditions and weather conditions, thus ensuring the accuracy of subsequent calculations.

[0056] S142. When the camera moves, continuously track the positional changes of the designated road feature points between consecutive frames in the video, and analyze the movement trajectory of the designated road feature points in consecutive frames in the video.

[0057] In this embodiment, once these feature points are identified, the system tracks their positional changes in real time across each frame of the video. By analyzing the movement trajectories of these feature points across consecutive frames, the system can acquire valuable information about the camera's own motion. This process relies on precise computer vision algorithms, such as optical flow or feature matching methods, to evaluate the displacement of the feature points from one frame to the next. This allows for the construction of a detailed record of the camera's motion, including how it moves along a path and any rotational movements.

[0058] S143. Calculate the relative displacement and rotational attitude of the camera between any two moments based on the movement trajectory to generate a measurement baseline.

[0059] In this embodiment, the measurement baseline refers to a high-precision three-dimensional spatial motion reference established by analyzing the movement trajectory of road feature points in consecutive frames of the video, calculating the relative displacement and rotational attitude of the camera between any two moments.

[0060] Based on the feature point movement trajectory analyzed in S142, the relative displacement (i.e., translation) and rotational attitude of the camera between any two time points are further calculated. This measurement baseline is essentially a coordinate transformation matrix that describes the camera's three-dimensional spatial motion state during this time. Specifically, by combining the camera's internal parameters (such as focal length, pixel size, etc.) and the two-dimensional displacement of the tracked feature points between consecutive frames, the true positions of these points in three-dimensional space are deduced using the principle of triangulation. Then, using the changes in these three-dimensional points, the distance and direction of the camera's movement relative to its initial position, as well as its rotation angles around each axis, can be accurately calculated. Ultimately, this data is used to establish a high-precision measurement baseline, which is crucial for subsequent calculations of tree canopy height, vehicle positioning, and environmental perception. This measurement baseline is continuously updated as the camera moves, ensuring the system's real-time performance and accuracy.

[0061] In one embodiment, the above-mentioned calculation of the height of the potential low-hanging target relative to the camera using triangulation, and the combination of the height of the camera relative to the ground and the height of the potential low-hanging target relative to the camera to obtain the height of the bottom of the potential low-hanging target from the ground, includes steps S144 to S146.

[0062] S144. Determine the pixel coordinates of the potential low-hanging target when the camera moves from one position to another, and use the camera's intrinsic parameter matrix in combination with the pixel coordinates to determine two angles of the potential low-hanging target in the physical coordinate system with the camera's optical center as the origin.

[0063] This step primarily involves identifying the location of potential low-hanging targets within video frames. By analyzing the changes in target feature points (such as tree canopies) across consecutive video frames, their pixel coordinates at different locations can be determined. Then, using the camera's intrinsic parameters (i.e., the intrinsic parameter matrix), these pixel coordinates are converted into angular information in a physical coordinate system with the camera's optical center as the origin. This step is fundamental to achieving three-dimensional spatial positioning.

[0064] S145. Based on geometric relationships and trigonometric function principles, a system of equations is established by combining the height of the camera above the ground and the two angles, and then solved to obtain the height of the potential low-hanging target relative to the measurement baseline.

[0065] In this embodiment, the height of the potential low-hanging target relative to the measurement baseline refers to a mathematical model established using geometric relationships and trigonometric function principles based on the height difference between the camera and the ground, and the angle information obtained by S144.

[0066] This model allows us to construct a set of equations describing the height of the target object, and by solving these equations, we can accurately calculate the height of a potential low-hanging target relative to the measurement baseline.

[0067] S146. Calculate the height of the bottom of the potential low-hanging target from the ground based on the height of the camera above the ground and the height of the potential low-hanging target relative to the measurement baseline.

[0068] In this embodiment, the height of the bottom of the potential low-hanging target from the ground refers to the height value of the target relative to the measurement baseline obtained in S145 plus the height value of the camera relative to the ground. Specifically, the height of the camera above the ground and the height of the potential low-hanging target relative to the measurement baseline are calculated to obtain the height of the bottom of the potential low-hanging target above the ground.

[0069] For S140 mentioned above, by tracking stable feature points of the background environment in the video frames in real time (such as ground markings, curbs, building outlines, etc.), the system accurately calculates the continuous three-dimensional motion trajectory of the vehicle (camera) from position A to B, then to C..., thereby obtaining multiple high-precision measurement baselines L. Combining data from multiple observations, the system uses multi-view geometry principles to calculate the precise three-dimensional coordinates of the target tree canopy point relative to the camera. To overcome the limitation of a fixed camera installation height, the system continuously and dynamically calculates the camera's height above the ground (H_cam(t)) by identifying road surface features. Specifically, to overcome the limitation of a fixed camera installation height, the system first uses image recognition technology to find and lock stable feature points (e.g., lane lines, curbs, road markings, road texture details) located on the road surface in the video frame. As the vehicle (i.e., the camera) moves forward, the system continuously tracks the positional changes of these road feature points locked in the first step between video frames. By analyzing the movement trajectory of these feature points in continuous images, the system can accurately calculate the relative displacement and rotational attitude of the camera between two moments. Combining the camera's internal parameters and calculated displacement, the system uses triangulation to calculate the coordinates of these road feature points relative to the camera in 3D space. Finally, in this calculation, the system uses the camera's own position as the origin (0, 0, 0) and simultaneously tracks multiple road feature points, obtaining their 3D spatial coordinates (X, Y, Z). Since these points are all located on the road surface, the system can use an algorithm to fit these 3D points into a 3D point cloud. Finally, three points are randomly selected from all the road surface 3D points (because three points define a plane), and a plane equation (Ax + By + Cz + D = 0) is calculated based on these three points. Given the camera origin coordinates (0, 0, 0) and the plane equation Ax + By + Cz + D = 0, the height H_cam(t) of the camera above the ground can be calculated using the point-to-plane distance formula. This calculation is repeated every frame or every few frames, thus achieving dynamic and continuous updating of H_cam(t).

[0070] Ultimately, the height (H_tree) of the bottom of the potential low-hanging target from the ground is calculated using the dynamic camera height and the relative height of the tree canopy obtained through triangulation (H_tree = H_cam(t) + ΔH_tree_cam, where ΔH_tree_cam is the vertical distance between the canopy point and the camera calculated using triangulation). In this way, by comprehensively considering the changes in camera height and the relative height of the tree canopy to the camera, the system can accurately assess the height of the lowest point of the tree canopy from the ground. This method ensures accurate measurement of the tree canopy height even when the camera height changes.

[0071] Specifically, the process of calculating the vertical distance ΔH_tree_cam between the tree canopy point and the camera using triangulation is as follows: Please see Figure 5 The figure presents a side view of the vehicle during its movement. A and B are two positions of the vehicle camera during its movement; L is the distance between points A and B (a known measurement baseline); X is an auxiliary geometric variable (unknown); and P1 is the projection of the target tree canopy point P onto the horizontal plane of the camera.

[0072] α and β are determined by an imaging model within the camera. The target point P, in three-dimensional space, is projected onto a two-dimensional image sensor through the camera's optical center, forming a pixel. In the image captured at position A, the pixel coordinates of the target tree canopy point P are (uA, vA). In the image captured at position B, the pixel coordinates of the target tree canopy point P are (uB, vB). The system then uses the camera's intrinsic matrix to convert the pixel coordinates into an observation angle with the camera's optical center as the origin. H_cam(t) is the height of the camera above the ground horizon (known), that is, the camera's height relative to the ground.

[0073] Using the above parameters, a system of equations can be constructed to solve for the height ΔH_tree_cam using the law of sine or simple trigonometric functions. For example: tanα = ΔH_tree_cam / (L+x); tanβ = ΔH_tree_cam / x; solving for ΔH_tree_cam yields the height. Finally, the height from the lowest point P of the tree canopy to the ground is calculated as H_tree = ΔH_tree_cam + H_cam(t), where ΔH_tree_cam is the height of the potential low-hanging target relative to the measurement baseline; and H_tree is the height of the bottom of the potential low-hanging target from the ground. S150. When the bottom of the potential low-hanging target is lower than the ground height than a safety threshold, an early warning is issued.

[0074] When the bottom of the potential low-hanging target is lower than the safety threshold, an early warning work order is generated to issue an early warning. The early warning work order includes GPS location, on-site photos, estimated height, and detection time.

[0075] After the system calculates the precise height of the lowest point of the tree canopy (H_tree), it compares this height with a preset safe passage height (e.g., 4.5 meters). This process ensures that any low-hanging branches or other obstacles that may affect the safe passage of vehicles or pedestrians can be detected in a timely manner.

[0076] Once the system detects that the bottom of a potential low-hanging target (such as a tree canopy) is lower than the set safety threshold from the ground, it will automatically activate the early warning mechanism.

[0077] Generate a detailed alert work order. This work order should include at least the following information: GPS location: Provides accurate geographic coordinates, facilitating quick location of problem areas.

[0078] On-site photos: Includes actual images of the current environment to help maintenance personnel intuitively understand the on-site situation.

[0079] Estimated height: The specific measured height of the low-hanging target was recorded, providing a basis for assessing the risk level.

[0080] Testing time: Indicate the time point of data collection to ensure timeliness.

[0081] Once an alert work order is generated, it is immediately pushed to the relevant maintenance department or responsible person. This ensures that the time between discovering a problem and taking action is minimized.

[0082] The receiving party responds quickly based on the information in the work order, performs necessary checks and repairs, and provides feedback to the system upon completion, forming a closed-loop management system to ensure that every warning is properly handled.

[0083] This step aims to improve the safety of public spaces through technological means, reducing safety hazards caused by obstacles such as low-hanging tree branches through precise monitoring and rapid response mechanisms.

[0084] Specifically, the method in this embodiment employs precise height measurement technology that is not limited by the camera's installation height and achieves automated closed-loop management from detection to response. Through intelligent discrimination and data processing at the source using edge computing devices, the system's intelligence and accuracy are significantly improved, while reducing the need for manual intervention. The system, based on the collaborative work of multiple discrimination models, ensures a substantial improvement in measurement accuracy, is suitable for monitoring needs across all scenarios and from multiple angles, and is characterized by high efficiency and low cost. Overall, this solution not only improves the efficiency and accuracy of monitoring and response but also provides flexible and powerful technical support for various application scenarios.

[0085] The aforementioned vehicle-mounted vision-based dynamic detection method for drooping tree canopies analyzes real-time video of roads and vegetation captured by cameras during vehicle movement. First, it identifies and marks tree canopies projected above the driving lane as potential drooping targets. Then, it calculates the depth information from pixels in the video to the camera to generate a depth map. Visual odometry tracks the movement of feature points in consecutive frames to dynamically calculate the camera's height relative to the ground. Combined with triangulation, it calculates the exact height of the bottom of the potential drooping target from the ground. Once the bottom height is detected to be below a preset safety threshold, the system immediately triggers an early warning mechanism. This significantly improves monitoring accuracy and greatly shortens the time from detection to response, effectively reducing traffic safety risks caused by drooping tree canopies.

[0086] Figure 6 This is a schematic block diagram of a dynamic tree canopy drooping detection system 300 based on vehicle vision provided in an embodiment of the present invention. Figure 6 As shown, corresponding to the above-described vehicle-vision-based dynamic detection method for drooping tree canopies, the present invention also provides a vehicle-vision-based dynamic detection system 300 for drooping tree canopies. This vehicle-vision-based dynamic detection system 300 includes a unit for executing the above-described vehicle-vision-based dynamic detection method for drooping tree canopies, and the system can be configured in a server. Specifically, please refer to... Figure 6 The vehicle-mounted vision-based tree canopy drooping dynamic detection system 300 includes an acquisition unit 301, a target determination unit 302, a depth map determination unit 303, a height calculation unit 304, and an early warning unit 305.

[0087] The acquisition unit 301 is used to acquire video of the road and vegetation during vehicle travel captured by the camera; the target determination unit 302 is used to identify and mark tree canopies projected above the driving lane in the video to obtain potential low-hanging targets; the depth map determination unit 303 is used to calculate the depth information of the distance between pixels and the camera in the video and generate a depth map; the height calculation unit 304 is used to analyze the movement trajectory of feature points in consecutive frames of the video using visual odometry, using the dynamic displacement of the camera as the measurement baseline, dynamically calculate the height of the camera relative to the ground based on the depth map, calculate the height of the potential low-hanging target relative to the camera using triangulation, and obtain the height of the bottom of the potential low-hanging target from the ground by combining the height of the camera relative to the ground and the height of the potential low-hanging target relative to the camera; the warning unit 305 is used to issue a warning when the height of the bottom of the potential low-hanging target from the ground is lower than a safety threshold.

[0088] In one embodiment, the height calculation unit 304 includes: The feature point localization subunit is used to find and lock a specified road feature point located on the road surface in the video using image recognition technology; the tracking and analysis subunit is used to continuously track the positional changes of the specified road feature point between consecutive frames in the video when the camera moves, and analyze the movement trajectory of the specified road feature point in consecutive frames in the video; the measurement baseline generation subunit is used to calculate the relative displacement and rotational attitude of the camera between any two moments based on the movement trajectory, so as to generate a measurement baseline.

[0089] In one embodiment, the height calculation unit 304 further includes: The fitting subunit is used to combine the camera's internal parameters and the measurement baseline, and use triangulation to inversely deduce the coordinates of the specified road feature points in three-dimensional space relative to the camera, and fit them into a three-dimensional spatial point cloud; the plane equation calculation subunit is used to randomly select several points from the three-dimensional spatial point cloud, and calculate the plane equation describing the road surface based on the selected points; the relative height calculation subunit is used to set the camera's coordinates as the origin coordinates, and use the point-to-plane distance formula with the plane equation to calculate the height of the camera relative to the ground.

[0090] In one embodiment, the height calculation unit 304 further includes: An angle determination subunit is used to determine the pixel coordinates of the potential low-hanging target when the camera moves from one position to another, and uses the camera's intrinsic parameter matrix in combination with the pixel coordinates to determine two angles of the potential low-hanging target in the physical coordinate system with the camera's optical center as the origin; a solution subunit is used to establish a system of equations based on geometric relationships and trigonometric function principles, combined with the camera's height above the ground and the two angles, and solve them to obtain the height of the potential low-hanging target relative to the measurement baseline; a summation subunit is used to calculate the height of the bottom of the potential low-hanging target above the ground based on the camera's height above the ground and the height of the potential low-hanging target relative to the measurement baseline.

[0091] In one embodiment, the summation subunit is used to calculate the sum of the height of the camera above the ground and the height of the potential low-hanging target relative to the measurement baseline, so as to obtain the height of the bottom of the potential low-hanging target above the ground.

[0092] In one embodiment, the target determination unit 302 includes: The video includes a lane area localization subunit for locating the driving lane area; a tree canopy identification subunit for identifying the areas where all tree canopies are located in the video using a semantic segmentation model; and an intersection subunit for performing an intersection operation on the driving lane area and the areas where all tree canopies are located to determine the tree canopies projected over the driving lane and obtain potential low-hanging targets.

[0093] In one embodiment, the depth map determination unit 303 is used to calculate the distance of each pixel from the camera in each selected image of the video using monocular depth vision estimation technology, and generate a depth map.

[0094] In one embodiment, the early warning unit 305 is used to generate an early warning work order to issue an early warning when the bottom of the potential low-hanging target is lower than a safety threshold. The early warning work order includes GPS location, on-site image, estimated height, and detection time.

[0095] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned vehicle vision-based tree canopy drooping dynamic detection system 300 and its various units can be referred to the corresponding descriptions in the foregoing method embodiments. For the sake of convenience and brevity, these details will not be repeated here.

[0096] The aforementioned vehicle-mounted vision-based dynamic tree canopy droop detection system 300 can be implemented as a computer program, which can be used in, for example... Figure 7 It runs on the computer device shown.

[0097] Please see Figure 7 , Figure 7 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0098] See Figure 7 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0099] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a dynamic detection method for low-hanging tree canopies based on vehicle vision.

[0100] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0101] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a dynamic detection method for low-hanging tree canopies based on vehicle vision.

[0102] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0103] The processor 502 is used to run the computer program 5032 stored in the memory to implement all the steps of the dynamic detection method for low-hanging tree canopies based on vehicle vision.

[0104] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0105] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0106] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein when executed by a processor, the computer program causes the processor to perform all the steps of the described vehicle-vision-based dynamic canopy droop detection method.

[0107] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0108] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0109] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of each unit is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0110] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the system of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0112] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for dynamic detection of drooping tree canopies based on vehicle-mounted vision, characterized in that, include: Acquire video footage of roads and vegetation captured by cameras during vehicle operation; The video is used to identify and mark tree canopies projected above the driveway to obtain potential low-hanging targets; The depth information of the distance between the pixels and the camera is calculated for the video, and a depth map is generated; Visual odometry is used to analyze the movement trajectory of feature points in consecutive frames of the video. The dynamic displacement of the camera is used as the measurement baseline. Based on the depth map, the height of the camera relative to the ground is dynamically calculated. The height of the potential low-hanging target relative to the camera is calculated using triangulation. The height of the bottom of the potential low-hanging target from the ground is obtained by combining the height of the camera relative to the ground and the height of the potential low-hanging target relative to the camera. An early warning is issued when the bottom of the potential low-hanging target is below a safety threshold.

2. The method for dynamic detection of drooping tree canopies based on vehicle vision according to claim 1, characterized in that, The step of analyzing the movement trajectory of feature points in consecutive frames of the video using visual odometry, with the dynamic displacement of the camera as the measurement baseline, includes: Image recognition technology is used to locate and pinpoint specific road feature points on the road surface in the video. When the camera moves, the positional changes of the designated road feature points are continuously tracked between consecutive frames in the video, and the movement trajectory of the designated road feature points in consecutive frames in the video is analyzed. The relative displacement and rotational attitude of the camera between any two moments are calculated based on the movement trajectory to generate a measurement baseline.

3. The method for dynamic detection of drooping tree canopies based on vehicle vision according to claim 1, characterized in that, The step of dynamically calculating the height of the camera relative to the ground based on the depth map includes: Combining the camera's internal parameters and the measurement baseline, the coordinates of the specified road feature points relative to the camera in three-dimensional space are inferred using triangulation, and then fitted into a three-dimensional point cloud. Randomly select several points from the three-dimensional point cloud, and calculate the plane equation describing the road surface based on the selected points; Set the camera's coordinates as the origin coordinates, and use the point-to-plane distance formula with the plane equation to calculate the camera's height relative to the ground.

4. The method for dynamic detection of drooping tree canopies based on vehicle vision according to claim 1, characterized in that, The step of calculating the height of the potential low-hanging target relative to the camera using triangulation, and combining the height of the camera relative to the ground with the height of the potential low-hanging target relative to the camera to obtain the height of the bottom of the potential low-hanging target from the ground, includes: Determine the pixel coordinates of the potential low-hanging target when the camera moves from one position to another, and use the camera's intrinsic parameter matrix in combination with the pixel coordinates to determine two angles of the potential low-hanging target in the physical coordinate system with the camera's optical center as the origin; Based on geometric relationships and the principles of trigonometric functions, a system of equations is established by combining the height of the camera above the ground and the two angles, and then solved to obtain the height of the potential low-hanging target relative to the measurement baseline. The height of the bottom of the potential low-hanging target from the ground is calculated based on the height of the camera above the ground and the height of the potential low-hanging target relative to the measurement baseline.

5. The method for dynamic detection of drooping tree canopies based on vehicle vision according to claim 4, characterized in that, The calculation of the bottom height of the potential low-hanging target from the ground based on the camera's height above the ground and the potential low-hanging target's height relative to the measurement baseline includes: The height of the camera above the ground and the height of the potential low-hanging target relative to the measurement baseline are calculated to obtain the height of the bottom of the potential low-hanging target above the ground.

6. The method for dynamic detection of drooping tree canopies based on vehicle vision according to claim 2, characterized in that, The designated road feature points include lane lines, ground markings, curbs, road surface markings, and road texture details.

7. The method for dynamic detection of drooping tree canopies based on vehicle vision according to claim 1, characterized in that, The step of identifying and marking tree canopies projected above the driveway in the video to obtain potential low-hanging targets includes: The video is used to locate the driving lane area; A semantic segmentation model was used to identify the regions where all tree canopies were located in the video. An intersection operation is performed on the driveway area and the area containing all tree canopies to determine the tree canopies projected over the driveway, thus obtaining potential low-hanging targets.

8. The method for dynamic detection of drooping tree canopies based on vehicle vision according to claim 1, characterized in that, The step of calculating the depth information of the video pixels from the camera and generating a depth map includes: Using monocular depth vision estimation technology, the distance of each pixel from the camera is calculated from each selected image in the video to generate a depth map.

9. The method for dynamic detection of drooping tree canopies based on vehicle vision according to claim 1, characterized in that, The step of issuing an early warning when the bottom of the potential low-lying target is below a safety threshold includes: When the bottom of the potential low-hanging target is lower than the safety threshold, an early warning work order is generated to issue an early warning. The early warning work order includes GPS location, on-site photos, estimated height, and detection time.

10. A dynamic detection system for drooping tree canopies based on vehicle-mounted vision, characterized in that, include: The acquisition unit is used to acquire video of roads and vegetation captured by the camera during the vehicle's movement. A target determination unit is used to identify and mark tree canopies projected above the driving lane in the video to obtain potential low-hanging targets; The depth map determination unit is used to calculate the depth information of the distance between the pixels and the camera in the video and generate a depth map; The height calculation unit is used to analyze the movement trajectory of feature points in consecutive frames of the video using visual odometry, take the dynamic displacement of the camera as the measurement baseline, dynamically calculate the height of the camera relative to the ground based on the depth map, calculate the height of the potential low-hanging target relative to the camera using triangulation, and combine the height of the camera relative to the ground and the height of the potential low-hanging target relative to the camera to obtain the height of the bottom of the potential low-hanging target from the ground. The early warning unit is used to issue an early warning when the bottom of the potential low-hanging target is lower than a safety threshold from the ground.

Images