Lane weed dynamic detection method and system fusing road surface features

Abstract

The application discloses a driving lane weed dynamic detection method and system fusing road surface features. The method comprises the following steps: acquiring a video of a road and a guardrail area; identifying trees, vegetation and the road and guardrail range to obtain an identification result; calculating the three-dimensional space position of the vegetation and screening a candidate set of weeds invading the road surface; tracking and analyzing motion trajectory features and evaluating the growth direction of the vegetation to obtain weeds to be determined; combining edge analysis and high-precision map data to confirm whether the weeds to be determined grow from a crack and the specific position to determine whether the weeds to be determined are driving lane invading weeds; if yes, outputting the specific position of the candidate weeds and performing an alarm. Through the method of the application, intelligent, efficient and full-scene automatic distinguishing and identification of rigid weeds and non-rigid fallen leaves in the driving lane can be realized, the accuracy and efficiency of road maintenance are significantly improved, and manual intervention and cost are reduced.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a method and system for dynamic detection of weeds in driving lanes that integrates road surface features. Background Technology

[0002] With the expansion of urban road and highway networks and the aging of infrastructure, cracks inevitably appear in road surfaces and guardrails. These cracks provide space for plant seeds to grow, and once conditions are suitable, the seeds will germinate and grow. However, these plants growing in the cracks will gradually widen the cracks, damaging the road structure and reducing its lifespan. Therefore, timely identification and removal of these weeds is a crucial step in maintaining road safety and extending the life of infrastructure.

[0003] Currently, weed detection on driving lanes mainly relies on methods based on color features or traditional image processing techniques. However, these methods exhibit low accuracy in distinguishing weeds from other similarly shaped objects such as fallen leaves, shadows from green belt vegetation, and moss. This is because existing solutions are mostly limited to two-dimensional image analysis, lacking an effective assessment of the three-dimensional spatial relationship between vegetation and the road surface, and cannot accurately determine whether the vegetation is actually growing from gaps or merely located on the road surface. Furthermore, when video is captured while the vehicle is moving, traditional static image analysis algorithms struggle to provide stable and reliable detection results due to factors such as camera shake, vehicle vibration, changes in lighting, and the dynamic nature of the target itself.

[0004] Another challenge lies in the fact that existing systems fail to fully integrate with the actual needs of road maintenance. They cannot effectively distinguish between invasive weeds and compliant green belt vegetation within the driving lanes that require treatment, resulting in detection results needing extensive manual verification before practical application. Simultaneously, facing the demands of large-scale road network inspections, how to efficiently process high-definition video stream data with limited hardware resources to achieve system real-time performance and efficiency is also a pressing issue. These problems limit the effectiveness and widespread application of existing technologies in practical deployments.

[0005] Therefore, it is necessary to design a new method to achieve intelligent, efficient, and automatic differentiation and identification of rigid weeds and non-rigid fallen leaves in driving lanes, which will significantly improve the accuracy and efficiency of road maintenance, while reducing manual intervention and costs. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for dynamic detection of weeds in driving lanes that integrates road surface features.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a dynamic detection method for roadway weeds that integrates road surface features, comprising:

[0008] Acquire video of the road and guardrail area;

[0009] The video is used to identify the range of trees, vegetation, roads, and guardrails to obtain the identification results;

[0010] The three-dimensional spatial location of the vegetation is calculated based on the disparity changes in consecutive frames and the recognition results, and a candidate set of weeds invading the road surface is screened.

[0011] The movement trajectory characteristics of the candidate weeds are tracked and analyzed to distinguish the dynamic behavior of fallen leaves and rooted weeds, and the vegetation growth direction is evaluated to identify flat fallen leaves and vertically growing weeds in order to obtain the weeds to be identified.

[0012] By combining edge analysis and high-precision map data, it is confirmed whether the weeds to be identified are growing out of the cracks and their specific locations, so as to determine whether the weeds to be identified are invasive weeds on the roadway.

[0013] If the candidate weed is an invasive weed in the driving lane, output the specific location of the candidate weed and issue an alarm.

[0014] The further technical solution is as follows: the process of identifying trees, vegetation, roads, and guardrails in the video to obtain identification results includes:

[0015] An instance segmentation algorithm is used to locate and classify various plants in the video, and to delineate road areas and guardrail boundaries on the road surface to obtain recognition results.

[0016] The further technical solution is as follows: The calculation of the three-dimensional spatial location of the vegetation based on continuous frame disparity changes and the recognition results, and the screening of a candidate set of weeds invading the road surface, includes:

[0017] Select consecutive frames from the video, extract feature points from each frame, and establish the correspondence between frames using a feature matching algorithm;

[0018] Based on the aforementioned correspondence, the depth value of each feature point is calculated by utilizing the disparity changes between frames and combining camera motion parameters.

[0019] The depth value is used in conjunction with the recognition results to determine the location of the vegetation in three-dimensional space, so as to obtain the three-dimensional spatial location of the vegetation;

[0020] Based on the three-dimensional spatial position of the vegetation, compare the vertical distance between the bottom of the vegetation and the road plane;

[0021] When the vertical distance is consistently less than a set threshold, the vegetation is identified as a candidate set of weeds invading the road surface.

[0022] The further technical solution is as follows: The movement trajectory characteristics of the candidate weed set are tracked and analyzed to distinguish the dynamic behavior of fallen leaves and rooted weeds, and the vegetation growth direction is assessed to identify horizontally spreading fallen leaves and vertically growing weeds, thereby obtaining the weeds to be determined, including:

[0023] By tracking the motion characteristics of the candidate weeds and calculating displacement and direction changes, the first candidate result can be obtained by distinguishing between fallen leaves that move irregularly due to wind and rooted weeds that sway slightly.

[0024] Image processing and principal component analysis are used to determine the main extension direction of vegetation in the candidate weed set. Based on the angular difference between the main extension direction and the vertical direction, flat fallen leaves or moss and vertically growing weeds are identified to obtain the second candidate result.

[0025] The first candidate result and the second candidate result are combined to obtain the weeds to be identified.

[0026] The further technical solution is as follows: by tracking the motion characteristics of the candidate weed set and calculating displacement and direction changes, to distinguish between fallen leaves that move irregularly due to wind and rooted weeds that sway slightly, to obtain a first candidate result, including:

[0027] Key feature points are extracted from the bottom and middle of the vegetation area corresponding to the weed candidate set, and the key feature points in a continuous frame sequence are tracked to record the movement trajectory of the key feature points.

[0028] Calculate the pixel displacement of each key feature point in the preceding and following frames, and use the displacement magnitude formula to calculate the average or median displacement magnitude of the key feature points in the entire region to obtain the calculation result;

[0029] Analyze the standard deviation of the displacement vectors of all the key feature points;

[0030] The vegetation corresponding to the weed candidate set whose calculation results do not exceed a first threshold or whose directional standard deviation is not greater than a second threshold is selected to obtain the first candidate result.

[0031] The further technical solution is as follows: The main extension direction of vegetation is determined using image processing and principal component analysis on the candidate weed set; based on the angular difference between the main extension direction and the vertical direction, flat-lying fallen leaves or moss and vertically growing weeds are identified to obtain a second candidate result, including:

[0032] Analyze the second-order spatial moments of the vegetation mask corresponding to the candidate weeds, determine the angle between the main extension direction and the vertical direction. If the angle value is outside the preset range, it is confirmed as flat fallen leaves or moss. If the angle value is within the preset range, it is confirmed as vertically growing weeds, so as to obtain the second candidate result.

[0033] The further technical solution is as follows: combining edge analysis and high-precision map data to confirm whether the weeds to be identified grow out of the cracks and their specific locations, in order to determine whether the weeds to be identified are invasive weeds on the driving lane, including:

[0034] Cracks and gaps are detected around the base of the weeds to be identified, and the edges of the candidate weeds are morphologically intersected with the cracks to determine whether the candidate weeds grow out of the cracks, so as to obtain the root-invading weeds and their specific locations.

[0035] High-precision maps or lane line detection are used to define the boundaries of the driving lane. Coordinate mapping is used to determine whether the weeds to be identified are located within the driving lane area, so as to obtain the location of the weeds that violate the rules and their specific locations.

[0036] The weeds with invasive roots and their specific locations, as well as the weeds with illegal locations and their specific locations, are identified as weeds that have invaded the driving lane.

[0037] The further technical solution is as follows: Crack and fissure detection is performed around the base of the weed to be identified, and the edges of the candidate weeds are morphologically intersected with the cracks to determine whether the candidate weeds are growing out of the cracks, thereby obtaining the root-invasive weeds and their specific locations, including:

[0038] Define the region of interest at the bottom of the weeds to be identified;

[0039] Within the region of interest, an edge detection algorithm is applied to identify cracks or guardrail gaps to obtain the lines;

[0040] The contour of the bottom edge of the mask of the weed to be identified is morphologically intersected with the line to obtain the number of intersecting pixels. The ratio of the number of intersecting pixels to the total number of pixels at the bottom edge of the vegetation of the weed to be identified is calculated to determine whether the candidate weed grows out of the crack, so as to obtain the root-invading weed and its specific location.

[0041] Alternatively, the average distance between the line and the bottom outline of the weed to be identified can be calculated to determine whether the candidate weed is growing out of the crack, in order to obtain the root-invading weed and its specific location.

[0042] The further technical solution is as follows: The method of using high-precision maps or lane line detection to define the driving lane boundary, and determining whether the weeds to be identified are located within the driving lane area through coordinate mapping, in order to obtain the location of the illegally located weeds and their specific location, includes:

[0043] The boundaries of the lane where the vehicle is located are determined by high-precision map data or real-time lane detection algorithms.

[0044] Based on the boundary, the three-dimensional coordinates of the roots of the weeds to be determined are transformed into the world coordinate system or the lane coordinate system to obtain the coordinate mapping result;

[0045] Based on the coordinate mapping results, it is determined whether the weeds to be identified are located within the driving lane area, so as to obtain the location of the weeds that violate the rules and their specific locations.

[0046] This invention also provides a dynamic detection system for roadway weeds that integrates road surface features, comprising:

[0047] The acquisition unit is used to acquire video of the road and guardrail areas;

[0048] The identification unit is used to identify the range of trees, vegetation, roads, and guardrails in the video to obtain identification results;

[0049] The first screening unit is used to calculate the three-dimensional spatial position of the vegetation based on the disparity change of consecutive frames and the recognition result, and to screen the candidate set of weeds that invade the road surface.

[0050] The second screening unit is used to track and analyze the movement trajectory characteristics of the candidate weeds to distinguish the dynamic behavior of fallen leaves and rooted weeds, and to evaluate the vegetation growth direction, distinguish between flat fallen leaves and vertically growing weeds, so as to obtain the weeds to be identified.

[0051] The third screening unit is used to combine edge analysis and high-precision map data to confirm whether the weeds to be identified grow out of the cracks and their specific locations, so as to determine whether the weeds to be identified are invasive weeds on the driving lane.

[0052] The output unit is used to output the specific location of the candidate weed and issue an alarm when the candidate weed is an intrusive weed in the driveway.

[0053] The advantages of this invention compared to existing technologies are as follows: This invention acquires video of roads and guardrail areas and identifies trees, vegetation, and road boundaries. It then uses continuous frame parallax variations to calculate the three-dimensional position of vegetation, filtering out a candidate set of weeds that may invade the road surface. Next, it analyzes the motion trajectories of these candidate weeds, distinguishing between rigidly rooted weeds and non-rigidly swaying fallen leaves, and assesses their growth direction, differentiating between flat-lying fallen leaves and vertically growing weeds. Finally, it combines edge analysis technology with high-precision map data to further confirm whether weeds are growing from cracks and their specific locations, thus accurately determining whether they are invasive weeds in the driving lane. Once an invasive weed is confirmed, its location is output and an alarm is triggered. This method achieves intelligent, efficient, and automatic differentiation and identification of rigid weeds and non-rigid fallen leaves in driving lanes across all scenarios, significantly improving the accuracy and efficiency of road maintenance work while reducing the need and cost of manual intervention.

[0054] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram illustrating an application scenario of the dynamic detection method for roadway weeds that integrates road surface features provided in an embodiment of the present invention.

[0057] Figure 2 A schematic diagram of instance segmentation provided in an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of the lines representing minute cracks or gaps in a guardrail, provided for an embodiment of the present invention;

[0059] Figure 4 A schematic diagram of invasive weeds on a driving lane provided in an embodiment of the present invention;

[0060] Figure 5 A schematic block diagram of a dynamic detection system for roadway weeds that integrates road surface features, provided in an embodiment of the present invention;

[0061] Figure 6 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0064] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0065] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0066] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating the dynamic detection method for roadway weeds based on road surface features provided in this embodiment of the invention. This method is applied to a server that interacts with devices such as cameras. By acquiring video of the road and guardrail areas, it uses instance segmentation algorithms to identify and classify trees, vegetation, and road boundaries in the video, and calculates the three-dimensional spatial location of the vegetation to filter out a candidate set of weeds that may be invading the road surface. Subsequently, by tracking and analyzing the movement trajectory features of the candidate weeds, and combining image processing and principal component analysis techniques, it distinguishes between fallen leaves and rooted weeds, while simultaneously assessing the vegetation growth direction to differentiate between flat fallen leaves and vertically growing weeds. Further, edge analysis and high-precision map data are used to confirm whether the weeds to be identified are growing from cracks and their specific locations, ultimately accurately determining whether the weeds are invading the roadway and outputting an alarm. This method achieves intelligent, efficient, and automatic differentiation and identification of rigid weeds and non-rigid fallen leaves within the roadway, significantly improving the accuracy and efficiency of road maintenance while reducing the need and cost of manual intervention.

[0067] Figure 1This is a flowchart illustrating the dynamic detection method for roadway weeds that integrates road surface features, provided in an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S160.

[0068] S110, Obtain video of the road and guardrail area.

[0069] In this embodiment, step S110 involves acquiring video of the road and guardrail areas. This process is fundamental to the entire dynamic detection method for weeds on the driving lane, and its main purpose is to capture real-time video data of the road and guardrail areas traversed by the vehicle during its journey using a camera video data acquisition module. This video data includes not only road surface information but also vegetation (such as trees, shrubs, etc.), weeds, and any fallen leaves or other debris that may be present.

[0070] In practice, the first step is to ensure that the camera is installed in a suitable location, covering the entire area to be monitored, including the road surface and the guardrail areas on both sides. The choice of camera is also crucial; it must have sufficient resolution and frame rate to clearly capture fast-moving targets and detailed features, such as the growth of weeds or changes in the state of fallen leaves. Furthermore, to adapt to different weather conditions and lighting environments, camera equipment with good low-light performance and high dynamic range capabilities is typically selected.

[0071] Once the cameras are installed and configured, the data acquisition system begins operating, continuously recording various scenarios encountered during driving. The acquired video frames are transmitted via the data upload module to the subsequent detection and analysis module for processing. During this process, to ensure the stability and efficiency of data transmission, compression algorithms may be used to reduce the data volume, and reliable communication protocols may be employed to ensure the integrity of the video frame data.

[0072] The key to this step is ensuring the quality and continuity of the video data, as this directly affects the accuracy of plant identification and the reliability of weed invasion assessments in subsequent steps. Therefore, in actual deployment, factors such as camera cleaning and maintenance, and regular calibration must be considered to maintain the system's optimal operating condition. Ultimately, through these measures, high-quality video data is obtained from the source, laying a solid foundation for the subsequent complex analysis process.

[0073] S120. Identify the range of trees, vegetation, roads, and guardrails in the video to obtain the identification results.

[0074] In this embodiment, the identification results include, but are not limited to, the following information:

[0075] Vegetation location and type: The specific location and type of each plant in the video frame were recorded in detail, which is crucial for further screening of weeds that may invade the driveway.

[0076] Road and guardrail area: The actual location of the road and guardrail in the video is clearly defined, which helps the system to more accurately define the work area and avoid misjudgment or omission.

[0077] Precise boundary information: Provides precise boundary coordinates for a given object (whether it is vegetation or road / guardrail). This information will be used in subsequent steps for complex operations such as estimating the 3D spatial position, calculating the displacement of feature points and the standard deviation of orientation.

[0078] Specifically, such as Figure 2 As shown, the application instance segmentation algorithm is used to locate and classify various plants in the video, and to delineate the road area and guardrail boundaries on the road surface to obtain the recognition results.

[0079] Step S120 involves processing the acquired video frame data to identify trees, vegetation, and the boundaries of roads and guardrails, thereby obtaining detailed identification results. This process is achieved by applying an advanced instance segmentation algorithm, which can accurately locate and classify different objects in the video, while distinguishing road areas and guardrail boundaries on the road surface.

[0080] Specifically, using an instance segmentation algorithm, each object within a video frame is first detected, and a category label (e.g., tree, shrub, weed) is assigned to each detected object. This method not only determines the location of objects but also clarifies their boundaries, which is crucial for subsequent depth information calculation and motion analysis.

[0081] In addition to vegetation identification, the same instance segmentation technique is needed to pinpoint the specific boundaries of the road surface and guardrails. This step helps the system understand the current environmental structure and is particularly important for determining which areas belong to the driving lane when performing root analysis and semantic location constraints.

[0082] By employing the methods described above, this step ensures the extraction of high-precision recognition results from the raw video data, providing solid data support for subsequent steps. For example... Figure 2 As shown, these images processed by the instance segmentation algorithm clearly display the location and classification of various plants, while also accurately marking the boundaries of roads and guardrails, laying the foundation for subsequent processing. This process not only improves the system's intelligence but also significantly reduces the need for manual intervention, making the entire system more efficient and reliable.

[0083] S130. Calculate the three-dimensional spatial position of the vegetation based on the disparity changes in consecutive frames and the recognition results, and filter the candidate set of weeds invading the road surface.

[0084] In this embodiment, the weed candidate set refers to

[0085] Step S130 aims to calculate the three-dimensional spatial location of vegetation based on the continuous parallax changes of video frames and the previously obtained recognition results, and to filter out a candidate set of weeds that may invade the road surface. This process includes extracting feature points from video frames, establishing inter-frame correspondences, calculating depth values, determining the three-dimensional location of vegetation, and finally filtering out potential weeds based on specific conditions.

[0086] In one embodiment, step S130 described above may include steps S131 to S135.

[0087] S131. Select consecutive frames from the video, extract feature points for each frame, and establish the correspondence between frames through a feature matching algorithm.

[0088] In this embodiment, a series of consecutive frames are selected from the video stream, which contain vegetation and background information that needs to be analyzed.

[0089] Feature points are extracted for each frame; a feature matching algorithm is used to match the feature points in one frame with the corresponding feature points in another frame, thereby establishing the correspondence between frames.

[0090] S132. Based on the correspondence, the depth value of each feature point is calculated by utilizing the disparity changes between frames and combining the camera motion parameters.

[0091] In this embodiment, by analyzing the parallax variation between frames (i.e., the positional difference of the same feature point in different frames) and combining it with known camera motion parameters (such as translation vector and rotation matrix), the depth value of each feature point (i.e., the distance of the point relative to the camera) can be calculated.

[0092] S133. Combine the recognition results with the depth value to determine the position of the vegetation in three-dimensional space, so as to obtain the three-dimensional spatial position of the vegetation.

[0093] In this embodiment, the three-dimensional spatial position of the vegetation refers to the specific coordinates (x, y, z) of the vegetation in the real-world coordinate system calculated based on video frame feature point extraction and matching, parallax changes, and camera motion parameters, which is used to accurately assess whether it encroaches on the driving lane.

[0094] By using the vegetation boundary information obtained from instance segmentation, feature points are matched to specific vegetation objects.

[0095] Based on the calculated depth value, combined with the position information in the two-dimensional image coordinate system, it is converted into the position in the three-dimensional world coordinate system, thereby determining the exact three-dimensional spatial position of each plant.

[0096] In this embodiment, the three-dimensional spatial location of the vegetation refers to its specific coordinates (x, y, z) in the real world, which helps to accurately assess whether the vegetation has encroached on the driving lane.

[0097] S134. Based on the three-dimensional spatial position of the vegetation, compare the vertical distance between the bottom of the vegetation and the road plane.

[0098] For each detected vegetation object, especially its bottom location, calculate its vertical distance from the road surface.

[0099] S135. When the vertical distance is continuously less than a set threshold, the vegetation is identified as a candidate set of weeds invading the road surface.

[0100] Define a vertical distance threshold (e.g., 0.01 meters). If the vertical distance between the bottom of a vegetation and the road surface is consistently less than this threshold, the vegetation is considered to be a weed rooted in a road crack.

[0101] Vegetation that meets the above criteria is identified as a candidate set of weeds that may invade the road surface, providing basic data for further analysis.

[0102] Through steps S131 to S135 described above, the system can effectively filter out a candidate set of weeds that may pose a threat to road safety from video data. This method not only improves the accuracy of identification but also reduces the possibility of false alarms, ensuring that only weeds that truly need to be addressed are marked. This is of great significance for maintaining road safety and cleanliness.

[0103] S140. Track and analyze the movement trajectory characteristics of the candidate weeds to distinguish the dynamic behavior of fallen leaves and rooted weeds, and evaluate the vegetation growth direction to identify flat fallen leaves and vertically growing weeds in order to obtain the weeds to be identified.

[0104] In this embodiment, the weeds to be identified refer to those that, after a series of screening processes, are considered likely to be rooted in road cracks, rather than fallen leaves or other non-weed vegetation. This screening process is based on dynamic behavior analysis (temporal analysis) and structural characteristic identification (morphological examination) of the candidate weed set to distinguish true weeds from environmental disturbances such as fallen leaves.

[0105] In one embodiment, step S140 described above may include steps S141 to S143.

[0106] S141. By tracking the motion characteristics of the candidate weeds and calculating displacement and direction changes, the fallen leaves that move irregularly due to wind influence and the rooted weeds that sway slightly are distinguished to obtain the first candidate result.

[0107] In this embodiment, the first candidate result refers to the result of distinguishing between deciduous and rooted weeds by calculating the movement trajectory of key feature points of vegetation within the weed candidate set and further analyzing their displacement amplitude and directional standard deviation.

[0108] In one embodiment, step S141 described above may include steps S1411 to S1414.

[0109] S1411 Extract key feature points at the bottom and middle of the vegetation area corresponding to the weed candidate set, track the key feature points in a continuous frame sequence, and record the movement trajectory of the key feature points.

[0110] The main objective at this stage is to identify key feature points that represent the dynamic characteristics of vegetation and to track the positional changes of these points across consecutive frames. Specifically:

[0111] The analysis was performed at the bottom and middle of the vegetation region corresponding to the weed candidate set. Bottom feature points help capture the positional stability of the vegetation roots, while middle feature points help observe the overall movement pattern.

[0112] Key feature points are extracted from selected regions. These algorithms are able to find unique local features in images, which are very useful for subsequent tracking.

[0113] The movement of these feature points in a continuous frame sequence is tracked using sparse optical flow methods (such as the Lucas-Kanade method), and the change path of each feature point over time is recorded.

[0114] S1412. Calculate the pixel displacement of each key feature point in the preceding and following frames, and use the displacement amplitude formula to calculate the average or median displacement amplitude of the key feature points in the entire region to obtain the calculation result.

[0115] In this embodiment, this step aims to quantify the motion characteristics of feature points by calculating their displacement to assess their degree of movement.

[0116] Pixel displacement calculation: For each feature point, calculate the change in its coordinates between consecutive frames, i.e., pixel displacement.

[0117] The displacement amplitude of a single feature point is calculated using the formula Δd=sqrt(Δx²+Δy²).

[0118] Based on the displacement amplitudes of all feature points, the average or median value over the entire region is calculated as an indicator of the overall motion intensity of that region. This value is the so-called "calculation result," which is used for the next step of screening.

[0119] S1413. Analyze the standard deviation of the direction of the displacement vectors of all the key feature points.

[0120] In this embodiment, in order to further distinguish the different dynamic behaviors of fallen leaves and rooted weeds, it is necessary to examine the consistency of movement direction.

[0121] By statistically analyzing the directions of the displacement vectors of all feature points, the standard deviation of the direction is calculated. Fallen leaves, due to the influence of wind, move in a relatively random direction, resulting in a larger standard deviation of direction. In contrast, although rooted weeds also sway in the wind, their overall movement direction is relatively consistent due to their fixed roots, leading to a smaller standard deviation of direction.

[0122] S1414. Select the vegetation corresponding to the weed candidate set whose calculation results do not exceed the first threshold or whose directional standard deviation is not greater than the second threshold, so as to obtain the first candidate result.

[0123] The final step is to filter the results from the first two steps to determine which vegetation is more likely to be true weeds.

[0124] First threshold: If the calculated result of a certain region (i.e., the average or median of the displacement amplitude) exceeds the set first threshold, then the region is considered to contain dynamic disturbances (such as fallen leaves) rather than rooted weeds.

[0125] Second threshold: At the same time, if the standard deviation of direction is greater than the second preset threshold, it also indicates that the vegetation in the area exhibits irregular movement, further supporting its classification as a non-weed object.

[0126] First candidate result: After the above screening process, the remaining weed candidate set is regarded as the "first candidate result". These vegetation have low displacement amplitude and directional standard deviation, which are consistent with the movement characteristics of rooted weeds.

[0127] S142. The main extension direction of the vegetation is determined by image processing and principal component analysis of the candidate weeds. Based on the angular difference between the main extension direction and the vertical direction, the flat-lying fallen leaves or moss and the vertically growing weeds are identified to obtain the second candidate result.

[0128] In this embodiment, the second candidate result refers to the result of performing an opening operation on the binary mask of vegetation in the weed candidate set, followed by PCA analysis to identify the main growth direction of the vegetation, thereby distinguishing between flat fallen leaves and vertically growing weeds.

[0129] In one embodiment, step S142 described above may include:

[0130] Analyze the second-order spatial moments of the vegetation mask corresponding to the candidate weeds, determine the angle between the main extension direction and the vertical direction. If the angle value is outside the preset range, it is confirmed as flat fallen leaves or moss. If the angle value is within the preset range, it is confirmed as vertically growing weeds, so as to obtain the second candidate result.

[0131] Specifically, an opening operation is performed on the binary mask corresponding to the vegetation in the weed candidate set to obtain the processed mask.

[0132] In this embodiment, firstly, a binary image (mask) is generated for each patch of vegetation in the weed candidate set, where the foreground pixels represent the vegetation area and the background pixels represent the non-vegetation area.

[0133] In mathematical morphology, opening is a process of erosion followed by dilation. Here, the opening operation is applied to a binary mask to remove small noise points and smooth boundaries, especially to eliminate interference that may be caused by stray objects or irregular shapes, ensuring that subsequent analysis is based on a clear and accurate vegetation outline.

[0134] The coordinates of all foreground pixels in the processed mask are combined into a point set, and PCA analysis is performed to obtain the directions of the first principal component and the second principal component, wherein the direction of the first principal component represents the main extension direction of the vegetation on the two-dimensional image.

[0135] In this embodiment, all pixel positions marked as vegetation (i.e., foreground) are extracted from the mask after the opening operation to form a two-dimensional coordinate point set.

[0136] PCA (Principal Component Analysis) is a statistical method used to determine the principal directions of variation in data. In this step, PCA analysis is performed on the above point set to calculate the first principal component (PC1) and the second principal component (PC2). Wherein:

[0137] First principal component direction: corresponds to the direction in which the point set extends along its maximum variance direction, which is represented in a two-dimensional image as the main growth direction or longest axis of vegetation.

[0138] The direction of the second principal component is perpendicular to the direction of the first principal component, and it usually reflects the trend of change in the width direction of vegetation.

[0139] Calculate the angle between the direction of the first principal component and the vertical direction to obtain the angle value.

[0140] In this embodiment, the angle between the first principal component direction obtained from PCA analysis and the vertical direction defined in the image is calculated. This angle value can quantify the degree of vegetation tilt relative to the ground.

[0141] In an image coordinate system, the vertical direction generally refers to the direction from bottom to top, but in actual implementation, the reference direction needs to be adjusted according to the actual situation (for example, considering the influence of the camera mounting angle).

[0142] If the included angle value is not within the set range, the vegetation is identified as flat fallen leaves or moss; if it is within the set range, it is identified as vertical weeds.

[0143] A reasonable angle range is predefined as a screening criterion. For example, for vertically growing weeds, the angle between the direction of the first principal component and the vertical direction should be close to 0 degrees; while for flat fallen leaves, since they tend to be randomly distributed and mostly horizontal, the corresponding angle value will be larger.

[0144] By comparing the angle value of each vegetation sample with a set range, vegetation with angle values ​​exceeding this range is excluded, and the remaining vegetation becomes the "second candidate result." This retained vegetation is more likely to be vertically growing weeds rather than flat fallen leaves or other types of disturbance. This screening mechanism helps improve the accuracy of weed identification and reduce the false positive rate.

[0145] S143. Combine the first candidate result and the second candidate result to obtain the weeds to be determined.

[0146] Ultimately, by combining dynamic behavior analysis (first candidate result) and structural characteristic identification (second candidate result), weeds that are indeed rooted in cracks in the road surface can be identified more accurately, thus obtaining a final list of weeds to be identified. This integrated method can effectively reduce false positives and improve the accuracy of weed identification.

[0147] In this embodiment, step S140 first quantifies the motion characteristics of the suspected object to distinguish between fallen leaves and rooted weeds.

[0148] Specifically, ORB or SIFT feature points are extracted from the bottom (emphasis) and middle of the suspected vegetation area. These feature points help track the motion of objects in a video frame sequence.

[0149] A sparse optical flow method (such as the Lucas-Kanade algorithm) is used to track the motion trajectories of these feature points in consecutive frames. This step can capture the positional movement of each feature point over time.

[0150] Calculate the pixel displacement (Δx, Δy) of each feature point between the previous and next frames.

[0151] The displacement amplitude formula Δd=sqrt(Δx²+Δy²) is used to calculate the average or median displacement amplitude of all characteristic points in the entire region.

[0152] Calculate the standard deviation of the direction of the displacement vectors of all feature points. Due to the irregular and large-amplitude movement of fallen leaves, their standard deviation of direction is relatively large; while the small-amplitude, high-frequency swaying of weeds results in a smaller standard deviation of direction.

[0153] If the average displacement amplitude is greater than 10 pixels per frame and the standard deviation of the motion direction exceeds a certain threshold, the object is identified as a dynamic disturbance such as a fallen leaf. Otherwise, it passes through this filtering layer and proceeds to the next step of inspection.

[0154] Secondly, from a morphological perspective, weeds that grow vertically and fallen leaves or mosses that are laid out horizontally can be distinguished.

[0155] Image preprocessing: Opening operation (erosion followed by dilation) is performed on the segmented binary vegetation mask to eliminate small noise points and burrs, and smooth the contour without significantly changing its area.

[0156] Principal Component Analysis (PCA):

[0157] The coordinates (x, y) of all foreground pixels in the mask are combined into a point set.

[0158] PCA analysis was performed on this point set to obtain the directions of the first and second principal components. The direction of the first principal component represents the main extension direction of the object in the two-dimensional image.

[0159] Calculate the angle between the direction of the first principal component and the vertical direction (called the "principal direction angle"). Rooted weeds typically grow as vertically as possible to gain sunlight, so their principal direction angle is close to 90 degrees (e.g., in the range of 45° to 135°).

[0160] If the main orientation angle of an object is not within the preset vertical angle range, it is considered to be flat fallen leaves or moss and is excluded. Conversely, if the main orientation angle is within the set range, the object is further confirmed to be vertically growing weeds.

[0161] This dual identification method, which combines dynamic and structural characteristics, can effectively improve the accuracy of identification and reduce the possibility of misjudgment.

[0162] S150. Combining edge analysis and high-precision map data, confirm whether the weeds to be identified grow out of the cracks and their specific locations, so as to determine whether the weeds to be identified are invasive weeds on the driving lane.

[0163] In one embodiment, step S150 may include step S151, which may include steps S151 to S153.

[0164] S151. Detect cracks and gaps around the bottom of the weeds to be identified, and perform morphological intersection operation between the edges of the candidate weeds and the cracks to determine whether the candidate weeds grow out of the cracks, so as to obtain the root-invading weeds and their specific locations.

[0165] In this embodiment, the root-invading weeds and their specific locations refer to the weeds growing from the road surface cracks and their precise geographical coordinates, which are confirmed through morphological intersection operations.

[0166] In one embodiment, step S151 described above may include steps S1511 to S1513.

[0167] S1511. Define the region of interest at the bottom position of the weeds to be identified.

[0168] In this embodiment, a small region of interest is defined at the bottom of the weeds to be identified. This step, based on the previously calculated positional information of the 3D points projected back into the image, precisely delineates the study area, ensuring the accuracy and efficiency of subsequent processing.

[0169] S1512. Within the region of interest, an edge detection algorithm is applied to identify cracks or guardrail gaps to obtain the lines.

[0170] In this embodiment, within the defined ROI, edge detection algorithms such as Canny edge detection are used to identify fine lines in cracks or gaps in the guardrail. This step aims to locate potential cracks or gaps, providing foundational data for the next step.

[0171] S1513. Perform a morphological intersection operation between the outline of the bottom edge of the mask of the weed to be determined and the line to obtain the number of intersecting pixels, and calculate the ratio of the number of intersecting pixels to the total number of pixels at the bottom edge of the vegetation of the weed to be determined, so as to determine whether the candidate weed grows out of the crack, so as to obtain the root-invading weed and its specific location.

[0172] Alternatively, the average distance between the line and the bottom outline of the weed to be identified can be calculated to determine whether the candidate weed is growing out of the crack, in order to obtain the root-invading weed and its specific location.

[0173] A morphological intersection operation is performed between the bottom edge contour of the weed mask to be identified and the identified crack lines. The presence of weeds is determined by calculating the ratio of the number of intersecting pixels to the total number of pixels at the bottom edge of the vegetation, or by directly calculating the average distance between the crack lines and the bottom contour of the vegetation. If the ratio is high or the average distance is very small (less than 2 pixels), the weed is considered to have invaded the vegetation through root growth from the crack, and its specific location is recorded.

[0174] In this embodiment, the root-invasive weeds and their specific locations refer to the weeds that have been confirmed by the above method to have indeed grown from the cracks in the road surface, and whose specific geographical coordinates have also been determined.

[0175] S152. Use high-precision maps or lane line detection to define the driving lane boundary, and use coordinate mapping to determine whether the weeds to be determined are located within the driving lane range, so as to obtain the location of the illegal weeds and their specific locations.

[0176] In this embodiment, the location of the illegal weeds and their specific locations refer to those weeds that are determined to be located within the driving lane area and their specific geographical coordinates, which require special attention because they may affect driving safety.

[0177] In one embodiment, step S152 described above may include steps S1521 to S1523.

[0178] S1521. Determine the boundary of the lane where the vehicle is located using high-precision map data or real-time lane detection algorithms.

[0179] In this embodiment, by integrating high-precision map data or running a real-time lane detection algorithm, the lane boundary where the current vehicle is located is accurately defined. This step ensures the accuracy of subsequent coordinate transformations and helps to precisely locate the position of weeds.

[0180] S1522. Based on the boundary, the three-dimensional coordinates of the roots of the weeds to be determined are transformed into the world coordinate system or the lane coordinate system to obtain the coordinate mapping result.

[0181] In this embodiment, the coordinate mapping result refers to the result of successfully transforming the three-dimensional coordinates of the weeds to a specific coordinate system.

[0182] Based on the lane boundaries determined in the previous step, the three-dimensional coordinates of the roots of the weeds to be identified are transformed into either the world coordinate system or the lane coordinate system. This process involves complex mathematical transformations, but the ultimate goal is to accurately describe the location of the weeds.

[0183] S1523. Based on the coordinate mapping result, determine whether the weed to be determined is located within the driving lane range, so as to obtain the location of the weed that violates the rules and its specific location.

[0184] Based on the coordinate mapping results, determine whether the weeds to be identified are located within the physical boundaries of the driving lane (including the emergency lane). If the weeds are indeed located within the driving lane, mark them as location violations and record their specific locations.

[0185] In this embodiment, the location of the illegal weeds and their specific locations refer to those weeds that have been confirmed to be located within the driving lane area, and these weeds require special attention and treatment because they affect driving safety.

[0186] Through steps S151 and S152 described above, the system can effectively verify whether weeds are growing out of cracks and whether their specific location is within the driving lane area. This method not only improves the monitoring accuracy of weed intrusion on the road surface but also avoids false alarms in compliant areas (such as green belts and flower beds in the central median). The entire process combines advanced image processing technology with the application of Geographic Information Systems (GIS), demonstrating the efficiency and practicality of modern technology in road maintenance and management.

[0187] S153. Determine the specific location of the root-invasive weeds and the location of the weeds that violate the rules. The weeds to be determined are invasive weeds that invade the driving lane.

[0188] In this embodiment, the purpose of the root density analysis in step S150 is to verify whether the vegetation truly grows from the cracks or crevices in the road surface, rather than simply falling onto the road surface. This is a key criterion for distinguishing between a clump of grass growing close to a crack and a fallen leaf that happens to be stuck in a crack.

[0189] First, a small region of interest (ROI) is defined around the calculated location of the vegetation base. This requires projecting the 3D points back to the original image to ensure accuracy.

[0190] Within a defined Region of Interest (ROI), edge detection algorithms are used to identify subtle cracks or gaps. These lines could be cracks between pavement materials or gaps in other structures such as guardrails. Figure 3 As shown.

[0191] A morphological intersection operation is performed between the outline representing the bottom edge of the vegetation mask and the previously detected crack lines. This process is done by comparing the overlapping portions of the two.

[0192] Calculate the ratio of the number of intersecting pixels to the total number of pixels at the bottom edge of the vegetation, or directly measure the average distance between the crack line and the bottom outline of the vegetation. If the ratio is high or the average distance is very small (less than 2 pixels), it indicates that the vegetation did indeed grow from the crack, and its roots are closely connected to the crack.

[0193] The semantic location constraints in step S140 are designed to avoid false alarms regarding vegetation within compliant areas (such as green belts and flower beds in the central median). The system's primary focus should be on weeds within the driving lane area, as they may affect driving safety.

[0194] High-precision map data or real-time lane detection algorithms are used to accurately define the lane boundary where the current vehicle is located. The three-dimensional coordinates of the determined weed roots are transformed into the world coordinate system or lane coordinate system to facilitate subsequent spatial relationship determination.

[0195] Determine if the transformed coordinates point lies within the physical boundaries of the driving lane (including the emergency lane). Only if the point is indeed within the driving lane area will it be ultimately identified as weeds requiring clearing. Figure 4 As shown, this method effectively excludes vegetation growing on land outside green belts or roads, thereby reducing unnecessary maintenance work.

[0196] The above methods can more accurately identify and locate the weed problems that truly need to be addressed, thereby improving road maintenance efficiency and safety.

[0197] S160. When the candidate weed is an intrusive weed on the roadway, output the specific location of the candidate weed and issue an alarm.

[0198] In previous steps, a series of analyses and judgments (such as root density analysis and semantic location constraints) have been used to determine whether certain vegetation types are considered invasive weeds requiring attention on the roadway. This step ensures that only weeds that are indeed located within the roadway area will be further treated.

[0199] For each target identified as an invading weed on the roadway, its two-dimensional location information in the image is first obtained, and then converted into three-dimensional coordinates in the world coordinate system using camera parameters or a previously established model. This step provides the accurate location of the weed in the actual physical space.

[0200] The obtained three-dimensional coordinates are further converted into geocoded information (such as latitude and longitude coordinates) to facilitate subsequent management and maintenance. Geocoding makes the location of weeds easily usable and understandable by mapping software or other geographic information systems (GIS).

[0201] The alert should include the specific location of the weeds (e.g., latitude and longitude), the expected impact area, and an assessment of the urgency level. Additionally, a partial road map marked with the weed locations can be included to help maintenance personnel quickly pinpoint the problem.

[0202] Depending on the system configuration, alerts can be sent in various ways. For example, email or text message notifications can be sent to the team responsible for maintaining the section of road; warnings can be issued to drivers via the in-vehicle system (if applicable); or updates can be provided to a dedicated online platform for relevant departments to view and manage.

[0203] Every alert should be logged by the system, including its processing status (unprocessed, in progress, resolved) and the result. This not only helps track the progress of each problem's resolution but also provides valuable data for future analysis.

[0204] Provide cleanup recommendations or directly dispatch relevant personnel to the site for handling. For areas with frequent weed infestations, consider long-term solutions, such as improving the drainage system or adopting more effective weed control measures.

[0205] Through the detailed steps described above, the S160 can not only effectively identify and report invasive weeds on the driving lane, but also provide comprehensive support for subsequent maintenance and management, thereby significantly improving road safety and service life.

[0206] This embodiment employs a multi-frame 3D depth recovery algorithm, analyzing multiple consecutive video frames to reconstruct the 3D structure of the scene, thereby accurately identifying weeds and fallen leaves on the road surface. To further distinguish the dynamic characteristics of rigid weeds and non-rigid fallen leaves, this invention combines optical flow analysis to assess motion stability and uses principal component analysis (PCA) to calculate the principal orientation angle of vegetation, thus forming an effective dynamic feature identification method. This method can not only effectively distinguish between weeds and fallen leaves but also assess their growth direction, significantly improving identification accuracy.

[0207] Furthermore, to optimize system performance, this embodiment proposes a distributed computing workflow that performs lightweight initial screening and key area extraction at the edge, while executing complex algorithms in the cloud. Edge devices can quickly process and screen potential weed or leaf litter areas, then transmit this crucial information to the cloud, utilizing more powerful computing resources to complete the final analysis task. This distributed architecture not only improves system efficiency but also significantly reduces latency, making the entire system deployment more efficient and cost-effective.

[0208] From an application perspective, the intelligent discrimination process, aided by edge computing devices, achieves a high degree of automation, reducing manual intervention while ensuring high accuracy. The method based on multiple discrimination models ensures a significant improvement in accuracy, adapting to application needs across all scenarios. Regardless of weather conditions or vegetation types, the system maintains high detection performance. Furthermore, due to its cost-effective computing strategy, the overall solution offers a low-cost advantage, providing a reliable and intelligent management tool for road maintenance.

[0209] The aforementioned dynamic detection method for roadway weeds, which integrates road surface features, acquires video of the road and guardrail areas and identifies trees, vegetation, and road boundaries. It then uses continuous frame parallax variations to calculate the 3D position of vegetation, filtering for potential weed candidates that may invade the road surface. Next, it analyzes the motion trajectories of these candidate weeds, distinguishing between rigidly rooted weeds and non-rigidly swaying fallen leaves, and assesses their growth direction, differentiating between flat-lying fallen leaves and vertically growing weeds. Finally, it combines edge analysis technology with high-precision map data to further confirm whether weeds are growing from cracks and their specific locations, thus accurately determining whether they are invasive weeds in the roadway. Once an invasive weed is confirmed, its location is output and an alarm is triggered. This method achieves intelligent, efficient, and automatic differentiation and identification of rigid weeds and non-rigid fallen leaves in the roadway, significantly improving the accuracy and efficiency of road maintenance while reducing the need and cost of manual intervention.

[0210] Figure 5 This is a schematic block diagram of a dynamic weed detection system 300 for driving lanes that integrates road surface features, provided in an embodiment of the present invention. Figure 5 As shown, corresponding to the above-described method for dynamic detection of lane weeds by integrating road surface features, the present invention also provides a lane weed dynamic detection system 300 that integrates road surface features. This lane weed dynamic detection system 300 includes a unit for executing the above-described method for dynamic detection of lane weeds by integrating road surface features, and the system can be configured in a server. Specifically, please refer to... Figure 5 The dynamic detection system 300 for roadway weeds that integrates road surface features includes an acquisition unit 301, an identification unit 302, a first screening unit 303, a second screening unit 304, a third screening unit 305, and an output unit 306.

[0211] The system comprises: an acquisition unit 301 for acquiring video of the road and guardrail area; an identification unit 302 for identifying trees, vegetation, and the road and guardrail area in the video to obtain identification results; a first filtering unit 303 for calculating the three-dimensional spatial position of the vegetation based on continuous frame parallax changes and the identification results, and filtering a candidate set of weeds invading the road surface; a second filtering unit 304 for tracking and analyzing the motion trajectory features of the candidate weeds to distinguish the dynamic behavior of fallen leaves and rooted weeds, and evaluating the vegetation growth direction to identify flat fallen leaves and vertically growing weeds to obtain the weeds to be identified; a third filtering unit 305 for combining edge analysis and high-precision map data to confirm whether the weeds to be identified grow out of cracks and their specific locations to determine whether the weeds to be identified are invading weeds of the driving lane; and an output unit 306 for outputting the specific location of the candidate weeds and issuing an alarm when the candidate weeds are invading weeds of the driving lane.

[0212] In one embodiment, the identification unit 302 is used to apply an instance segmentation algorithm to locate and classify various plants in the video, and to delineate the road area and guardrail boundaries on the road surface to obtain the identification result.

[0213] In one embodiment, the first screening unit 303 includes:

[0214] The first extraction subunit is used to select consecutive frames from the video, extract feature points for each frame, and establish a correspondence between frames using a feature matching algorithm; the depth value calculation subunit is used to calculate the depth value of each feature point based on the correspondence, using the disparity change between frames, and combined with camera motion parameters; the position determination subunit is used to determine the position of the vegetation in three-dimensional space using the depth value in combination with the recognition result, so as to obtain the three-dimensional spatial position of the vegetation; the distance comparison subunit is used to compare the vertical distance between the bottom of the vegetation and the road plane based on the three-dimensional spatial position of the vegetation; the identification subunit is used to identify the vegetation as a candidate set of weeds invading the road surface when the vertical distance is continuously less than a set threshold.

[0215] In one embodiment, the second screening unit 304 includes:

[0216] The first candidate subunit is used to distinguish between fallen leaves that move irregularly due to wind and rooted weeds that sway slightly by tracking the motion characteristics of the weed candidate set and calculating displacement and direction changes, so as to obtain the first candidate result; the second candidate subunit is used to determine the main extension direction of the vegetation by using image processing and principal component analysis on the weed candidate set, and to identify flat fallen leaves or moss and vertically growing weeds according to the angular difference between the main extension direction and the vertical direction, so as to obtain the second candidate result; the combination subunit is used to combine the first candidate result and the second candidate result to obtain the weed to be determined.

[0217] In one embodiment, the first candidate subunit includes:

[0218] The recording module is used to extract key feature points at the bottom and middle of the vegetation area corresponding to the weed candidate set, track the key feature points in a continuous frame sequence, and record the motion trajectory of the key feature points; the calculation module is used to calculate the pixel displacement of each key feature point in the preceding and following frames, and use the displacement amplitude formula to calculate the average or median displacement amplitude of the key feature points in the entire area to obtain the calculation result; the analysis module is used to analyze the directional standard deviation of the displacement vector of all key feature points; the filtering module is used to filter the vegetation corresponding to the weed candidate set whose calculation result does not exceed a first threshold or whose directional standard deviation is not greater than a second threshold to obtain the first candidate result.

[0219] In one embodiment, the second candidate subunit includes:

[0220] An open calculation module is used to analyze the second-order spatial moments of the vegetation mask corresponding to the candidate weeds, determine the angle between the main extension direction and the vertical direction, and if the angle is outside the preset range, it is confirmed as flat fallen leaves or moss, and if the angle is within the preset range, it is confirmed as vertically growing weeds, so as to obtain the second candidate result.

[0221] In one embodiment, the third screening unit 305 includes:

[0222] The first determining subunit is used to detect cracks and gaps around the bottom of the weed to be determined, and to perform a morphological intersection operation between the edge of the candidate weed and the crack to determine whether the candidate weed grows out of the crack, so as to obtain the root-invasive weed and its specific location; the second determining subunit is used to define the driving lane boundary using a high-precision map or lane line detection, and to determine whether the weed to be determined is located within the driving lane range through coordinate mapping, so as to obtain the location-violation weed and its specific location; the final determining subunit is used to determine the weed to be determined as a driving lane invasive weed by combining the root-invasive weed and its specific location, and the location-violation weed and its specific location.

[0223] In one embodiment, the first determining subunit includes:

[0224] The system includes a definition module for defining a region of interest (ROI) at the bottom of the weed to be identified; a line recognition module for applying an edge detection algorithm to identify cracks or gaps in the fence within the ROI to obtain the lines; and an intersection calculation module for performing a morphological intersection operation between the outline of the bottom edge of the mask of the weed to be identified and the lines to obtain the number of intersecting pixels, and calculating the ratio of the number of intersecting pixels to the total number of pixels at the bottom edge of the vegetation of the weed to determine whether the candidate weed is growing out of the crack, thus obtaining the root-invasive weed and its specific location; or, calculating the average distance between the lines and the bottom outline of the weed to be identified to determine whether the candidate weed is growing out of the crack, thus obtaining the root-invasive weed and its specific location.

[0225] In one embodiment, the second determining subunit includes:

[0226] The boundary determination module is used to determine the boundary of the lane where the vehicle is located through high-precision map data or real-time lane line detection algorithm; the mapping module is used to transform the three-dimensional coordinates of the root of the weed to be determined to the world coordinate system or lane coordinate system based on the boundary to obtain the coordinate mapping result; the judgment module is used to determine whether the weed to be determined is located within the driving lane range according to the coordinate mapping result to obtain the location of the weed that violates the rules and its specific location.

[0227] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned dynamic detection system for roadway weeds 300 that integrates road surface features and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0228] The aforementioned dynamic weed detection system 300 for roadways that integrates road surface features can be implemented as a computer program, which can, for example... Figure 6 It runs on the computer device shown.

[0229] Please see Figure 6 , Figure 6 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0230] See Figure 6 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0231] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a dynamic detection method for lane weeds that incorporates road surface features.

[0232] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0233] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a dynamic detection method for roadway weeds that integrates road surface features.

[0234] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 6The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0235] The processor 502 is used to run the computer program 5032 stored in the memory to implement all the steps of the dynamic detection method for roadway weeds that integrates road surface features.

[0236] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0237] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0238] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein when executed by a processor, the computer program causes the processor to perform all steps of the lane weed dynamic detection method incorporating road surface features.

[0239] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0240] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0241] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of each unit is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0242] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the system of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0243] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0244] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method of detecting a dynamic of a weed in a traffic lane fused with a road surface feature, characterized by, The method comprises the following steps: acquiring a video of a road and a guardrail area; identifying trees, vegetation, and a road and guardrail range in the video to obtain an identification result; calculating a three-dimensional spatial position of the vegetation based on a parallax change between continuous frames and the identification result, and screening a weed candidate set that invades a road surface; tracking and analyzing motion trajectory features of the weed candidate set to distinguish dynamic behaviors of fallen leaves and rooted weeds, and to evaluate a growth direction of the vegetation, so as to distinguish flat fallen leaves and vertically growing weeds, and to obtain a to-be-determined weed; combining edge analysis and high-precision map data to confirm whether the to-be-determined weed grows from a crack and a specific position, so as to determine whether the to-be-determined weed is a lane invading weed; when the to-be-determined weed is a lane invading weed, outputting the specific position of the weed and performing an alarm.

2. The method of claim 1, wherein the method further comprises: The step of identifying trees, vegetation, and a road and guardrail range in the video to obtain an identification result comprises the following steps: applying an instance segmentation algorithm to locate and classify identify various plants in the video, and dividing a road area on a road surface and a guardrail boundary to obtain the identification result.

3. The method of claim 1, wherein the method further comprises: The step of calculating a three-dimensional spatial position of the vegetation based on a parallax change between continuous frames and the identification result, and screening a weed candidate set that invades a road surface comprises the following steps: selecting continuous frames from the video, extracting feature points from each frame, and establishing a corresponding relationship between frames by using a feature matching algorithm; calculating depth values of the feature points based on the corresponding relationship and a parallax change between frames, and combining camera motion parameters; determining a position of the vegetation in a three-dimensional space by using the depth values in combination with the identification result, so as to obtain the three-dimensional spatial position of the vegetation; comparing a vertical distance between a bottom of the vegetation and a road plane based on the three-dimensional spatial position of the vegetation; when the vertical distance continuously is less than a set threshold, identifying the vegetation as a weed candidate set that invades a road surface.

4. The method of claim 1, wherein the method further comprises: The step of tracking and analyzing motion trajectory features of the weed candidate set to distinguish dynamic behaviors of fallen leaves and rooted weeds, and to evaluate a growth direction of the vegetation, so as to distinguish flat fallen leaves and vertically growing weeds, and to obtain a to-be-determined weed comprises the following steps: distinguishing fallen leaves that are irregularly moved due to wind force from rooted weeds that slightly swing by tracking motion features of the weed candidate set and calculating displacement and direction changes, so as to obtain a first candidate result; determining a main extension direction of the vegetation by using image processing and principal component analysis on the weed candidate set, and distinguishing flat fallen leaves or mosses from vertically growing weeds according to an angle difference between the main extension direction and a vertical direction, so as to obtain a second candidate result; combining the first candidate result and the second candidate result to obtain the to-be-determined weed.

5. The method of claim 4, wherein the method further comprises: The step of distinguishing fallen leaves that are irregularly moved due to wind force from rooted weeds that slightly swing by tracking motion features of the weed candidate set and calculating displacement and direction changes to obtain a first candidate result comprises the following steps: extracting key feature points at a bottom and a middle part of a vegetation area corresponding to the weed candidate set, and tracking the key feature points in a continuous frame sequence to record motion trajectories of the key feature points; Calculate the pixel displacement of each key feature point in the front and back frames, and calculate the average or median of the displacement amplitude of the key feature points in the entire region using the displacement amplitude formula to obtain the calculation result; Analyze the direction standard deviation of the displacement vector of all key feature points; Screen the corresponding vegetation of the weed candidate set corresponding to the calculation result not exceeding the first threshold or the direction standard deviation not being greater than the second threshold to obtain the first candidate result.

6. The method of claim 4, wherein the method further comprises: The image processing and principal component analysis are used to determine the main extension direction of the vegetation of the weed candidate set, and the angle difference between the main extension direction and the vertical direction is used to identify the flat fallen leaves or moss and the vertically growing weeds to obtain the second candidate result, including: The second-order spatial moment of the vegetation mask corresponding to the weed candidate set is analyzed to determine the angle value between the main extension direction and the vertical direction, and when the angle value is outside the preset range, it is confirmed as flat fallen leaves or moss, and when the angle value is within the preset range, it is confirmed as vertically growing weeds to obtain the second candidate result.

7. The method of claim 1, wherein the method further comprises: The edge analysis and high-precision map data are combined to confirm whether the to-be-determined weed grows from a crack and the specific position to determine whether the to-be-determined weed is a driving lane invading weed, including: Crack gap detection is performed around the bottom of the to-be-determined weed, and morphological intersection operation is performed between the edge of the to-be-determined weed and the crack to determine whether the weed grows from the crack to obtain the root system invading weed and the specific position; The driving lane boundary is defined by high-precision map or lane line detection, and it is judged whether the to-be-determined weed is located within the driving lane range by coordinate mapping to obtain the position violating weed and the specific position; The root system invading weed and the specific position, the position violating weed and the specific position determine that the to-be-determined weed is a driving lane invading weed.

8. The method of claim 7, wherein the method further comprises: The crack gap detection is performed around the bottom of the to-be-determined weed, and morphological intersection operation is performed between the edge of the to-be-determined weed and the crack to determine whether the weed grows from the crack to obtain the root system invading weed and the specific position, including: A region of interest is defined at the bottom position of the to-be-determined weed; In the region of interest, an edge detection algorithm is applied to identify crack or guardrail gap lines to obtain the lines; Morphological intersection operation is performed between the contour of the bottom edge of the mask of the to-be-determined weed and the lines to obtain the number of intersecting pixels, and the proportion of the number of intersecting pixels to the total number of pixels of the bottom edge of the vegetation of the to-be-determined weed is calculated to determine whether the weed grows from the crack to obtain the root system invading weed and the specific position; Alternatively, the average distance between the lines and the bottom contour of the to-be-determined weed is calculated to determine whether the weed grows from the crack to obtain the root system invading weed and the specific position.

9. The method of claim 7, wherein the method further comprises: The driving lane boundary is defined by high-precision map or lane line detection, and it is judged whether the to-be-determined weed is located within the driving lane range by coordinate mapping to obtain the position violating weed and the specific position, including: The boundary of the lane where the vehicle is located is determined by high-precision map data or real-time lane line detection algorithm; Based on the boundary, the root three-dimensional coordinates of the weed to be determined are converted into a world coordinate system or a lane coordinate system to obtain a coordinate mapping result; According to the coordinate mapping result, it is judged whether the weed to be determined is located within the driving lane range to obtain the position violation weed and the specific position.

10. A system for dynamic detection of weeds in a traffic lane fused with a road surface feature, characterized in that Comprise: An acquisition unit is configured to acquire a video of a road and a guardrail area; An identification unit is configured to identify trees, vegetation, and a road and guardrail range from the video to obtain an identification result; A first screening unit is configured to calculate a three-dimensional spatial position of the vegetation based on continuous frame parallax changes and the identification result, and to screen a weed candidate set that invades a road surface; A second screening unit is configured to track and analyze motion trajectory features of the weed candidate set to distinguish dynamic behaviors of fallen leaves and rooted weeds, and to evaluate vegetation growth directions to distinguish fallen leaves that are flat and weeds that grow vertically to obtain weeds to be determined; A third screening unit is configured to combine edge analysis and high-precision map data to confirm whether the weed to be determined grows from a crack and a specific position to determine whether the weed to be determined is a driving lane invading weed; An output unit is configured to output the specific position of the weed to be determined when the weed to be determined is a driving lane invading weed, and to perform an alarm.

Images