Real-time identification method of spilled objects on highway driving lanes under drone inspection vision

Abstract

This disclosure provides a method for real-time identification of debris scattered on highway lanes under UAV inspection vision. The method includes: segmenting the highway exit curve road surface area into layers and extracting lateral structural boundaries; unfolding the highway curve road surface space into a linear strip region, establishing bridge deck spatial attachment constraints based on the unfolding results, forming an effective attachment region for the highway curve road surface; performing debris target detection, conducting cascaded screening and elimination analysis on the detection results, continuously tracking and analyzing the attachment behavior of candidate debris in the candidate set of debris inside the bridge deck, identifying and eliminating sidewall drift targets; performing lane occupancy analysis and hazard level assessment, generating real-time early warning information, inspection verification information, and operation and maintenance scheduling information, solving the problem of sidewall hanging objects being misidentified as debris scattered on highway lanes, and achieving accurate identification and spatial attribution constraints only for debris scattered inside the actual highway lanes.

Description

Technical Field

[0001] This disclosure relates to the fields of drone inspection and intelligent traffic monitoring technology, and in particular to a method for real-time identification of debris on highway driving lanes under drone inspection vision. Background Technology

[0002] With the rapid development of drone technology, low-altitude inspection of highways using drones has become an important means of identifying road debris. However, in practical applications, especially in urban elevated highway exit areas, there is often a structural environment combining high noise barriers and small-radius curves. When drones conduct low-altitude inspections, some lightweight suspended objects, such as plastic bags and packaging films, may adhere to the side walls of noise barriers or the outside of guardrails. Existing debris identification schemes typically focus on the bridge deck area, but because the lateral structures and the bridge deck have a continuous attachment relationship in the top-view projection, lateral attachments will also form a continuous attachment relationship with the bridge deck area in the drone's top-view image projection, causing the system to continuously misidentify these attachments, which are not originally on the driving lane, as bridge debris. Especially in curve areas, changes in the bridge deck orientation further enhance the projection drift of side wall targets onto the bridge deck area, making the false alarm problem more serious. This misidentification not only reduces the accuracy of debris detection but also triggers a large number of invalid warnings, interfering with normal operation and maintenance scheduling.

[0003] Therefore, there is an urgent need for a real-time identification method for debris in scenarios where continuous curves at highway exits are sandwiched between lateral sound barriers and other obstacles. This method aims to solve the problem of objects hanging on the side walls being misidentified as debris in the highway driving lanes, and to achieve accurate identification and spatial allocation constraints for debris only within the actual highway driving lanes. Summary of the Invention

[0004] In view of this, in order to solve the problems caused by the existing technology, this application provides a method for real-time identification of spilled objects on highway driving lanes under the vision of unmanned aerial vehicle (UAV) inspection.

[0005] In a first aspect, this disclosure provides a method for real-time identification of debris spilled on highway driving lanes under the vision of unmanned aerial vehicle (UAV) inspection, the method comprising:

[0006] S1. The road surface area of ​​the highway exit curve is segmented into layers and the lateral structure boundary is extracted. The driving lane is separated from the side wall of the sound barrier and the outer area of ​​the guardrail, generating a set of highway driving lane areas and a set of lateral structure isolation areas.

[0007] S2. Based on the set of high-speed driving lane areas, the high-speed curved road surface space is expanded into a linear strip area. Based on the expansion result, a bridge surface space attachment constraint is established to form an effective attachment area of ​​the high-speed curved road surface.

[0008] S3. Perform target detection of spilled material within the effective attachment area of ​​the high-speed curved road surface. Based on the effective attachment area of ​​the high-speed curved road surface and the set of lateral structure isolation areas, perform cascade screening and elimination analysis on the detection results to obtain a candidate set of spilled material inside the bridge deck.

[0009] S4. Perform continuous frame tracking and attachment behavior analysis on the candidate debris in the candidate set of debris inside the bridge deck, identify and remove sidewall drifting targets, and obtain stable bridge deck debris identification results.

[0010] S5. Based on the identification results of the stable bridge surface debris, perform lane occupancy analysis and hazard level assessment, and generate real-time early warning information, inspection and verification information and operation and maintenance scheduling information.

[0011] Optionally, S1 includes:

[0012] Distortion correction, tilt correction and scale unification processing are performed on the UAV inspection images to obtain a standardized top view image sequence;

[0013] Based on the standardized top-view image sequence, lane lines, road edge lines, top edge of sound barrier, outer contour of guardrail and slope boundary line are extracted to form a set of basic road contour features;

[0014] A deep learning semantic segmentation network is used to segment the high-speed driving lane area using the standardized top-view image sequence as input, thereby obtaining the initial segmentation result of the high-speed driving lane area.

[0015] Based on the lane boundaries in the initial segmentation results of the high-speed lane area, and combined with the road basic contour feature set, a lateral structure isolation score is calculated for the adjacent areas outside the lane boundaries, and a lateral structure candidate region set is generated based on the lateral structure isolation score.

[0016] The initial segmentation results of the high-speed driving lane area are subjected to connectivity checks and traffic continuity scores. Areas that do not meet the traffic continuity requirements are incorporated into the lateral structure candidate area set to form the final high-speed driving lane area set and lateral structure isolation area set.

[0017] Optionally, S2 includes:

[0018] Based on the set of high-speed driving lane areas, the inner boundary, outer boundary and center zone of the high-speed driving lane are extracted, and the curved road surface is unfolded into a straight road strip area based on the center zone of the lane.

[0019] The direction of change of the lane centerline, the direction of change of the bridge deck boundary, and the direction of guardrail contact are continuously tracked after the lane is unfolded, and the results of the extension direction of the bridge deck centerline, the tracking results of the bridge deck boundary, and the results of the guardrail contact direction are obtained.

[0020] Based on the results of the extension direction of the bridge deck center, the bridge deck adhesion constraint strength is calculated;

[0021] Based on the bridge deck boundary tracking results, a set of bridge deck adhesion attenuation zones is established at locations near the sound barrier sidewall, guardrail edge, and slope connection area; based on the bridge deck adhesion constraint strength and guardrail contact direction results, the attenuated bridge deck adhesion reliability is calculated.

[0022] Based on the bridge deck adhesion attenuation zone set, the attenuated bridge deck adhesion reliability, and the bridge deck boundary tracking results, boundary reinforcement constraints are established for the local compressed lane area in the high-speed exit curve, and an effective adhesion area confirmation score is calculated. When the effective adhesion area confirmation score exceeds a preset threshold, the area is confirmed as an effective adhesion area of ​​the high-speed curve road surface.

[0023] Optionally, S3 includes:

[0024] Using the effective attachment area of ​​the high-speed curved road surface as the only effective detection mask, the UAV inspection image is cropped and the detection range is limited to obtain the image area to be detected.

[0025] A lightweight target detection model adapted to highway road scenes is used to identify candidate clutter in the image region to be detected, thereby obtaining a set of candidate clutter targets;

[0026] The candidate debris target set is processed by algorithm concatenation with the effective attachment area of ​​the high-speed curved road surface. The high-speed driving lane concatenation screening targets are obtained according to the geometric correlation conditions, and the concatenation screening score of each target is calculated.

[0027] Based on the cascaded screening score, and combined with the set of lateral structure isolation areas, lateral structure exclusion analysis is performed on the candidate targets near the bridge deck edge in the cascaded screening targets of the highway driving lane, generating a set of bridge deck candidate targets, and calculating the bridge deck candidate score after lateral structure exclusion for each target.

[0028] For targets in the bridge deck candidate target set, the final lane validity is confirmed based on the bridge deck candidate scores after lateral structure exclusion. The final bridge deck internal debris candidate score for each target is calculated, and targets whose final bridge deck internal debris candidate scores meet a preset threshold are retained to generate a bridge deck internal debris candidate set.

[0029] Optionally, the cascaded processing of the algorithms includes:

[0030] The target is retained only if the center point of the candidate debris is located inside the high-speed driving lane, the bottom contact point falls into the effective attachment area of ​​the high-speed curved road surface, and the overlap ratio between the target coverage area and the effective attachment area exceeds a preset threshold; if the target frame crosses the lane boundary, it will not be retained if the bottom contact position is biased towards the outside of the guardrail or the side wall of the sound barrier.

[0031] Optionally, S4 includes:

[0032] Based on the candidate set of debris inside the bridge deck, the center position, bottom contact position and outer contour area of ​​the candidate targets are used as tracking features, and they are correlated in continuous inspection frames to form a continuous frame tracking set of candidate targets.

[0033] The adhesion continuity between the target in the candidate target continuous frame tracking set and the extension direction of the bridge deck center is verified, and the bridge deck continuous adhesion score is calculated.

[0034] A longitudinal reference is established along the direction extending from the center of the bridge deck, and a lateral reference is established along the direction perpendicular to the direction extending from the center of the bridge deck. The lateral drift enhancement score is calculated, and targets with a lateral drift enhancement trend are marked as anomalies based on the lateral drift enhancement score to obtain lateral drift anomaly marking results.

[0035] Based on the lateral drift anomaly marking results, the bridge deck continuous attachment score, and the lateral drift enhancement score, sidewall attachment determination is performed on targets that continuously approach the edge of the sound barrier, the edge of the guardrail, or the slope connection area and maintain a synchronous drift trend. The sidewall attachment determination results are obtained, and the actual bridge deck attachment score is calculated.

[0036] Based on the sidewall adhesion determination result and the actual bridge deck adhesion score, the candidate targets are subjected to final elimination and stable retention processing to obtain the stable bridge deck spill identification result.

[0037] Optionally, the sidewall attachment determination includes:

[0038] When the outer contour of a candidate target is in long-term contact with the boundary of the lateral structure, the long axis of the target is consistent with the extension direction of the guardrail or sound barrier, and the bottom contact position cannot stably fall into the effective attachment area of ​​the high-speed curved road surface, the target is identified as a side wall attachment.

[0039] For lightweight targets, it is also determined whether their outline extends along the surface of the sound barrier rather than being laid flat on the bridge surface.

[0040] Optionally, S5 includes:

[0041] For each debris identified in the stable bridge deck debris identification results, the lane position, occupied area, and traffic direction are analyzed to obtain the debris lane occupancy analysis results, and the lane occupancy score is calculated;

[0042] Based on the lane occupancy analysis results of the spilled material and the lane occupancy score, the hazard level score is calculated by combining the target category, curve radius and exit merging status.

[0043] Based on the real-time inspection location of the drone, the image capture altitude, the gimbal attitude, and the target's position in the image, a spatial positioning result for the highway exit, corrected along the lane centerline, is generated.

[0044] Based on the hazard level score and the spatial positioning results, real-time early warning information, inspection and verification information, and operation and maintenance scheduling information are generated.

[0045] In a second aspect, this disclosure provides an electronic device including a memory and at least one processor, the memory storing a computer program, and the processor executing the computer program to implement the method of the first aspect described above.

[0046] Thirdly, this disclosure provides a computer storage medium storing a computer program that, when executed, implements the method described in the first aspect.

[0047] The beneficial effects of the present invention are as follows: Compared with the prior art, the present invention has the following advantages:

[0048] The road surface of the highway exit curve is segmented into layers and the lateral structural boundaries are extracted to separate the driving lane from areas such as the sidewalls of the noise barrier and the outer side of the guardrail. The curved road surface is spatially unfolded and bridge deck attachment constraints are established to eliminate projection distortion caused by the curvature of the curve. The results of debris detection and road surface segmentation are cascaded and filtered to eliminate candidate targets within the lateral structural areas. The lateral drift trend of the target is identified by continuous frame tracking, and sidewall attachment judgment is performed to remove drifting targets. This solves the problem of lateral suspended objects being misidentified as debris in the driving lane due to the overlap of the top-view projection, and achieves accurate identification of debris only within the actual highway driving lane, significantly reducing the interference of invalid warnings on operation and maintenance scheduling. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0050] Figure 1 A flowchart of a method for real-time identification of spilled objects on highway lanes under UAV inspection vision provided in an embodiment of this disclosure is shown.

[0051] Figure 2 A flowchart illustrating the continuous frame tracking and sidewall adhesion determination of candidate projectiles provided in an embodiment of this disclosure is shown.

[0052] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0053] The present disclosure will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present disclosure and should not be used to limit the scope of protection of the present disclosure.

[0054] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0055] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0056] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0057] Figure 1 This is a flowchart of a method for real-time identification of spilled objects on highway driving lanes under the vision of unmanned aerial vehicle (UAV) inspection, as provided in this embodiment of the disclosure. Figure 1 As shown, the process may include the following steps:

[0058] S1: The road surface area of ​​the highway exit curve is segmented into layers and the lateral structural boundaries are extracted. The driving lane is separated from the side wall of the sound barrier and the outer area of ​​the guardrail, generating a set of highway driving lane areas and a set of lateral structural isolation areas.

[0059] S1.1: Perform distortion correction, tilt correction and scale unification processing on the UAV inspection images to obtain a standardized top view image sequence.

[0060] Drones are used to conduct low-altitude inspections of continuous curved sections of highway exits. The drone flies along the direction of the lane extension, maintaining a gimbal tilt angle to acquire a continuous sequence of overhead images including the highway lanes, sound barriers, guardrail structures, slope areas, and bridge edges. Flight altitude and gimbal tilt angle can be adjusted according to the actual scene; a flight altitude of 50 to 150 meters and a gimbal tilt angle of 80 to 90 degrees are generally recommended to ensure that the images simultaneously include the main road surface and lateral structures.

[0061] For each captured image frame, distortion correction, tilt correction, and scale unification are performed sequentially. Distortion correction employs a radial distortion correction model. This model uses the normalized distance of any pixel in the original image relative to the image center and the first and second-order radial distortion coefficients to nonlinearly correct the coordinates of that pixel, restoring the stretched or compressed points in the lens edge region caused by wide-angle distortion to their true positions. The radial distortion coefficients can be calibrated using a standard checkerboard pattern before inspection. In practical applications, the lens radial distortion parameter is controlled between 0.01 and 0.25.

[0062] Tilt correction is the process of converting non-vertical view images caused by changes in the attitude of the UAV into images that are approximately vertically overhead. This can be achieved by perspective transformation or automatic compensation methods based on inertial measurement unit data.

[0063] Scale unification processing refers to resampling the corrected image to a uniform ground resolution. The resampling resolution is set to correspond to an actual road surface distance of 3 to 10 centimeters per pixel, ensuring that lane lines, guardrail edges, and small debris can be clearly identified. After the above processing, a standardized top-view image sequence of the curved road section is obtained.

[0064] S1.2: Based on the standardized top-view image sequence, extract lane lines, road edge lines, top edge of sound barrier, outer contour of guardrail, and slope boundary line to form a set of basic road contour features.

[0065] Based on a standardized top-view image sequence, spatial contour extraction is performed on each frame. Specifically, edge detection operators are first used to extract pixels with abrupt changes in brightness and color. Discrete edge points are then connected into continuous line segments using Hough transform or line segment fitting methods. For lane lines, the edges of white or yellow markings are extracted; for road edge lines, the boundaries between the road surface and the shoulder or noise barrier base are extracted; for areas with changes in curvature of curves, the points where the lane lines change direction are marked. For the top edge of the noise barrier, the focus is on identifying its long, continuous structure that forms a significant brightness difference or alternating shadow with the road surface. Since the top of the noise barrier often appears as a continuous bright or dark line, features with a length-to-width ratio greater than 10 can be used for filtering. For the outer contour of the guardrail, periodically appearing vertical or horizontal edges are extracted based on the regular strip texture of the metal railing. For slope boundary areas, the distinction is made based on the abrupt changes in texture between the hard paved area and the earthen slope / green area. Typically, the road surface texture is more uniform and has stronger reflectivity, while the slope area has a rougher texture or vegetation texture.

[0066] Through the above processing, a set of basic road contour features is obtained, which includes geometric information such as lane line endpoints and equations, road boundary lines, top edge lines of sound barriers, outer contour lines of guardrails, and slope boundary lines.

[0067] S1.3: Using a deep learning semantic segmentation network, the standardized top-view image sequence is used as input to segment the high-speed driving lane area, and the initial segmentation result of the high-speed driving lane area is obtained.

[0068] After obtaining the basic road contour feature set, a highway pavement segmentation algorithm is used to accurately segment the highway driving lane area. The target of segmentation is the effective lane range actually used by vehicles, which includes at least the main driving lane, the driving part of the exit ramp, and the merging area of ​​the curve. At the same time, the edge strip next to the noise barrier, the outer area of ​​the guardrail, the slope area, the maintenance strip, and the non-traffic hardened area are clearly excluded.

[0069] This embodiment employs a deep learning-based semantic segmentation network. This network takes a standardized top-down view image as input and outputs the probability that each pixel belongs to a highway lane. During network training, a highway image dataset labeled with the actual lane boundaries is used. During actual inference, for each pixel, the segmentation network outputs a lane score. The score is mapped to the probability of belonging to the driving lane using the Sigmoid function. Its value ranges from 0 to 1. The lane determination threshold is set to 0.65 to 0.85, that is, when Pixels exceeding this threshold are classified as lane pixels. If lane lines are severely worn at curves, the threshold can be appropriately lowered to 0.55, but this must be combined with the continuity of road edge lines for judgment; only pixels that are close to continuous road edge lines are retained. After segmentation, the initial segmentation result of the highway lane area is obtained.

[0070] S1.4: Based on the lane boundary in the initial segmentation result of the high-speed lane area, and combined with the road basic contour feature set, calculate the lateral structure isolation score for the adjacent area outside the lane boundary, and generate a lateral structure candidate area set based on the lateral structure isolation score.

[0071] Based on the initial segmentation results of the highway driving lane area, and utilizing the road basic contour feature set obtained in step S1.2, especially the top edge line of the noise barrier, the outer contour line of the guardrail, and the slope boundary line, the adjacent areas outside the lane boundary are structurally classified to isolate the lateral structures from the driving lane area. Specifically, the boundary pixels of the driving lane area are first extracted, and then extended outwards by a certain distance, such as 0.5 meters to 2 meters, to obtain candidate lateral areas. For these candidate areas, it is determined whether they possess vertical structural features or non-traffic texture features. During the determination process, the position of the candidate area can be matched with the top edge line of the noise barrier and the outer contour line of the guardrail in the road basic contour feature set to enhance the reliability of recognition. The sidewall area of ​​the noise barrier typically appears as a long, continuous high boundary, with its height perpendicular to the road surface forming a significant gray-level abrupt change in the image; the outer area of ​​the guardrail has periodic railing texture, whose dominant frequency features can be analyzed using Fourier transform; the slope area has vegetation texture or rough slope texture, and its gray-level variance is usually large.

[0072] To quantify whether a candidate region belongs to the lateral structure, a lateral structure isolation score is calculated. . The degree of continuous fit between this area and the boundary of the driving lane Vertical structural characteristic strength and non-road surface texture intensity Both are positively correlated. Specifically, Determined by the ratio of the boundary overlap length to the perimeter of the region; The proportion of the vertical direction in the histogram of edge directions within the region is given; Calculated from texture roughness or vegetation index. , , After assigning weights to each component, a weighted average is obtained. .when If the value is greater than 0.6, the region is classified as a lateral structure isolation region; otherwise, it is not included in the isolation set. Through the above processing, regions that meet the lateral structure characteristics are isolated from the driving lane region, resulting in a set of lateral structure candidate regions.

[0073] S1.5: Perform connectivity checks and traffic continuity scoring on the initial segmentation results of the high-speed driving lane area, and incorporate areas that do not meet the traffic continuity requirements into the lateral structure candidate area set to form the final high-speed driving lane area set and lateral structure isolation area set.

[0074] Based on the initial segmentation results of the high-speed driving lane area and the set of candidate lateral structure areas, a constraint relationship for continuous passage areas within the high-speed driving lane is further established. Specifically, a connectivity check is performed on the initial segmentation results of the high-speed driving lane area, retaining only those areas that are consistent with the lane extension direction, have a width sufficient for vehicle passage, and can extend continuously along the curve direction. For areas that are close to the road surface but have a width significantly smaller than the vehicle passage width, such as narrow strips tightly attached to sound barriers or guardrails, they are incorporated into the set of lateral structure isolation areas. The effective width of the driving lane is set at 3.2 meters to 3.8 meters per lane, and can be reduced to 2.8 meters in locally narrowed areas of exit ramps.

[0075] To quantify whether a region to be judged is a valid passage area, a passage continuity score is introduced. . It is positively correlated with the actual lateral width W of the area to be determined, and with the offset distance of the centerline of the area relative to the centerline of the previous continuous lane. Negative correlation, and the degree of continuous matching with the region along the curve direction. Positive correlation. The direction of the centerline of the area is determined by the consistency between the centerline direction of the area and the centerline direction of the lane ahead, for example, by calculating and normalizing the Frescher distance between the two curves. A higher value indicates that the area is more suitable as a valid passage zone. Only when... If the value is greater than 0.6, the area is retained as part of the high-speed driving lane; otherwise, it is classified as a lateral structural isolation area. Through the above processing, the final set of high-speed driving lane areas and the set of lateral structural isolation areas are generated.

[0076] In the technical solution of this disclosure embodiment, by performing layered segmentation and lateral structure boundary extraction on the curved road surface area at the highway exit, it is possible to accurately distinguish between the highway driving lane and lateral structures such as the side wall of the sound barrier, the outer side of the guardrail, and the slope area, thereby avoiding the lateral attachments from being mistakenly included in the driving lane range. This provides an accurate spatial constraint basis for subsequent identification of spilled objects and effectively solves the problem of regional confusion caused by top-view projection.

[0077] S2: Based on the set of high-speed driving lane areas, the high-speed curved road surface space is expanded into a linear strip area. Based on the expansion result, bridge surface space attachment constraints are established to form an effective attachment area for the high-speed curved road surface.

[0078] S2.1: Based on the set of high-speed driving lane areas, extract the inner boundary, outer boundary and center zone of the high-speed driving lane, and unfold the curved road surface into a straight road strip area based on the center zone of the lane.

[0079] Based on the set of high-speed driving lane areas and the set of lateral structure isolation areas generated in step S1, the inner boundary, outer boundary and center zone of each driving lane in the curved section of the high-speed exit are extracted. At the same time, the sound barrier sidewall area, the guardrail outer area and the slope connection area in the set of lateral structure isolation areas are used as non-lane reference boundaries.

[0080] Using the center zone of the lane as the deployment reference, the curved road surface is unfolded into a nearly straight strip of road surface along the direction of travel. This transforms the originally curved and compressed lane area in the top-view image into an unfolded area that facilitates the determination of the target's attachment position. During the unfolding process, the actual lateral distance between the target and the lane boundary remains unchanged; only their arrangement along the curve direction is altered. For small-radius curves at highway exits, a sampling section is set at a spacing of one sampling section every 2 to 5 meters of road length; for ordinary gentle curves, a sampling section is set at a spacing of one sampling section every 5 to 10 meters. The sampling interval needs to be dynamically adjusted according to the curvature of the curve. To quantify the curvature of the current road segment, a curvature score is introduced. . It is positively correlated with the change in the lane center direction angle of adjacent sampling sections, and with the length of the road centerline between the two sections. The square of the negative correlation. When When the curve is relatively sharp, the sampling interval should be a smaller value of 2 to 3 meters; when When the sampling interval is small, a larger value of 5 to 10 meters is used. Through the above processing, the spatial unfolding benchmark results of the curved road surface are obtained, providing a basis for subsequent tracking of the bridge deck center direction.

[0081] S2.2: Continuously track the direction of change of the lane centerline, the direction of change of the bridge deck boundary, and the direction of guardrail contact after the lane is unfolded, and obtain the results of the extension direction of the bridge deck centerline, the tracking results of the bridge deck boundary, and the results of the guardrail contact direction.

[0082] Based on the baseline results of the curved road surface spatial unfolding, the changing directions of the lane centerline, bridge surface boundary, and guardrail attachment direction are continuously tracked after unfolding. Specifically, the lane center point is first determined in each sampling section, and then the center points in adjacent sampling sections are connected sequentially to obtain the extension direction of the bridge surface center. Subsequently, the left and right boundaries of the driving lane are simultaneously tracked to obtain the changes in the bridge surface boundary as the road turns. Finally, the directional consistency of the outer contour of the guardrail and the top edge of the sound barrier in the lateral structural isolation area is judged to identify which areas belong to the lateral attachment structure extending along the bridge surface but not belonging to the driving lane. If the guardrail attachment direction is consistently close to the extension direction of the bridge surface center, but its position is always outside the driving lane boundary, then this area should not be used as an area for the attachment of spilled materials.

[0083] To quantify the stability of the bridge deck center extension direction, a stability score for the bridge deck center extension direction is introduced. . Consistency of direction with adjacent centerline segments Positive correlation with the degree of parallelism between the lane boundary and the center line. Positively correlated with the distance of abrupt change at the centerline position. Negative correlation. A larger value indicates a more stable bridge deck extension direction. Through the above tracking, we obtain the results for the bridge deck center extension direction, bridge deck boundary tracking, and guardrail alignment direction.

[0084] S2.3: Calculate the bridge deck attachment constraint strength based on the results of the bridge deck center extension direction.

[0085] Based on the results of the bridge deck's centerline extension, spatial constraints are applied to the internal area of ​​the high-speed driving lane to establish an attachment determination basis for subsequent debris target detection. Specifically, the tendency of any point to be evaluated to be identified as bridge debris is quantified by calculating the bridge deck attachment constraint strength J. For ease of understanding, the internal area of ​​the driving lane can be divided into a central stable attachment zone, an edge cautious attachment zone, and an outer prohibited attachment zone according to different attachment risks, but in actual calculations, the quantification is directly performed using the formula for J. The central stable attachment zone is located near the lane centerline and is used to receive subsequently detected conventional debris; the edge cautious attachment zone is located near the lane edge line and is used to temporarily store debris that may be located at the lane edge; the outer prohibited attachment zone is located within the lateral structural isolation area, where no debris is allowed to be inherited as bridge debris.

[0086] For any candidate location or area on the road surface (hereinafter referred to as "the point to be evaluated"), three conditions must be met simultaneously to allow the formation of a bridge deck attachment relationship: the point to be evaluated is located within the high-speed driving lane area; the point to be evaluated exists continuously along the direction of extension of the bridge deck center; and the point to be evaluated does not cross into the lateral structural isolation zone. Among these, the contact distance from the point to be evaluated to the lateral structural isolation zone... The distance is the Euclidean distance from the outer contour boundary of the point to be evaluated to the boundary of the nearest lateral structural isolation zone. It is positive if the point to be evaluated does not overlap with the isolation zone, and zero if they overlap. To quantify the bridge deck attachment constraint strength of the point to be evaluated, the bridge deck attachment constraint strength J is introduced. J is the lateral distance from the center point of the point to be evaluated to the nearest lane boundary. Positive correlation with the probability that the point to be evaluated belongs to the driving lane. Positive correlation, with stable score in the direction of bridge deck center extension. Positively correlated with the contact distance from the point to be evaluated to the lateral structural isolation area. Positive correlation. The larger the value of J, the more likely the point to be evaluated is identified as debris from the bridge deck. The bridge deck adhesion constraint strength J will be used in subsequent step S3 to determine the spatial assignment of the detected debris candidates.

[0087] S2.4: Based on the bridge deck boundary tracking results, establish a set of bridge deck adhesion attenuation zones for locations near the sound barrier sidewall, guardrail edge, and slope connection area; based on the bridge deck adhesion constraint strength and guardrail contact direction results, calculate the attenuated bridge deck adhesion reliability.

[0088] Based on the bridge deck boundary tracking results obtained in step S2.2, bridge deck adhesion attenuation zones are established near the sidewalls of the noise barrier, the edges of the guardrails, and the slope connection areas. Specifically, using the bridge deck boundary tracking results obtained in step S2.2, the position of the outer boundary of the driving lane is accurately determined, and an edge attenuation band is set from this boundary inwards into the driving lane. This attenuation band does not directly delete the area, but rather reduces the continuous inheritance ability of the target to be judged as real bridge deck debris. For areas with high noise barrier height, obvious continuous guardrail adhesion, and slopes adjacent to the road surface edges, the attenuation band width is 0.3 meters to 1.5 meters; for ordinary shoulders or areas with clear boundaries, the attenuation band width is 0.2 meters to 0.8 meters. Using the outer boundary of the driving lane as a reference, the width of the attenuation band set according to the lateral structure type is continuously taken at each cross-section along the lane direction, and the area covered is the bridge deck adhesion attenuation zone; the total of all attenuation zones is called the bridge deck adhesion attenuation zone set. This set will serve as the spatial constraint range for identifying lateral drift targets in subsequent steps.

[0089] Based on the bridge deck adhesion constraint strength J calculated in step S2.3 and the guardrail contact direction obtained in step S2.2, the bridge deck adhesion confidence level after attenuation is calculated for candidate targets falling within the aforementioned attenuation zone. Specifically, the direction of target position change is determined using the guardrail contact direction result: if the target remains within the attenuation zone for a long period and its position change direction is closer to the guardrail contact direction than the direction of bridge deck center extension, then subsequent processing should prioritize treating it as a lateral drift target. To quantify the bridge deck adhesion confidence level after attenuation, a bridge deck adhesion confidence level after attenuation is introduced. . It is positively correlated with the original bridge deck adhesion constraint strength J and the contact distance from the target to the lateral structural isolation zone. They exhibit an exponentially decaying negative correlation, meaning the closer the distance, the higher the correlation. The stronger the attenuation, and the greater the continuity with the target along the direction of contact with the guardrail. Negative correlation, degree of continuity with the target along the direction extending from the center of the bridge deck Positive correlation. A higher value indicates that the target still has a high confidence level in bridge deck adhesion, even if it is in the attenuation zone.

[0090] S2.5: Based on the bridge deck adhesion attenuation zone set, the attenuated bridge deck adhesion reliability, and the bridge deck boundary tracking results, establish boundary reinforcement constraints for the local compressed lane area in the high-speed exit curve, and calculate the effective adhesion area confirmation score. When the effective adhesion area confirmation score exceeds a preset threshold, the area is confirmed as an effective adhesion area of ​​the high-speed curve road surface.

[0091] Based on the bridge deck adhesion attenuation zone set formed in step S2.4, the calculated bridge deck adhesion confidence U1 after attenuation, and the bridge deck boundary tracking results obtained in step S2.2, boundary reinforcement constraints are established for the locally compressed lane areas in the highway exit curves. Locally compressed lanes typically appear where exit ramps narrow, sound barriers are close together, guardrail corners turn inward, and slope transition points. In these areas, the lane width in the top-view image will narrow, and the lateral structures are more likely to project and fit onto the bridge deck. Therefore, based on the boundary continuity provided by the bridge deck boundary tracking results, the actual lane width, and the degree of lateral structure fit, a secondary contraction of the effective adhesion area is needed: within the area marked by the bridge deck adhesion attenuation zone set, the effective adhesion range is further restricted, retaining only the area that satisfies the vehicle passage width and bridge deck direction continuity.

[0092] Among them, the lateral structural bonding strength This is used to quantify the tightness of the fit between lateral structures, such as sound barriers and guardrails, and the driveway boundary. It is calculated based on the proportion of the overlap length between the lateral structure boundary and the driveway boundary to the total length of the driveway boundary, as well as the average distance between the lateral structure and the driveway boundary. The value ranges from 0 to 1, with a higher value indicating a tighter fit. To quantify whether an area should be considered a valid attachment area, a valid attachment area confirmation score is introduced. . It is positively correlated with the current actual width W of the local lane and with the traffic continuity score. Positively correlated with the reliability of bridge deck adhesion after attenuation. Positive correlation, and bonding strength with the lateral structure Negative correlation. When If the value is greater than 0.65, the area is confirmed as an effective attachment area of ​​the high-speed curved road surface; otherwise, it is discarded. Through the above processing, the effective attachment area of ​​the high-speed curved road surface is finally formed, which serves as the spatial input for subsequent cascaded filtering of debris target detection and high-speed road surface segmentation algorithms.

[0093] In the technical solution of this disclosure embodiment, by unfolding the high-speed curved road surface space into an approximately straight strip region and establishing bridge surface attachment constraint processing, the projection distortion caused by the curvature of the curve and the interference of lateral structure proximity can be eliminated, making the lateral distance judgment between the target and the lane boundary more accurate. At the same time, the effective attachment area is further defined by attenuation zone and boundary reinforcement constraints, thereby improving the reliability of the spatial ownership determination of the spilled object in the curved scene.

[0094] S3: Perform debris target detection within the effective attachment area of ​​the high-speed curved road surface, and perform cascaded screening and exclusion analysis on the detection results based on the effective attachment area of ​​the high-speed curved road surface and the set of lateral structure isolation areas to obtain a candidate set of debris inside the bridge deck.

[0095] S3.1: Using the effective attachment area of ​​the high-speed curved road surface as the only effective detection mask, the UAV inspection image is cropped and the detection range is limited to obtain the image area to be detected.

[0096] Based on the effective attachment area of ​​the high-speed curved road surface generated in step S2, the UAV inspection image is processed by region cropping and detection range limitation. Specifically, the effective attachment area of ​​the high-speed curved road surface is used as the only effective detection mask, retaining only the road surface image content within it; the areas of the sound barrier sidewall, the outer area of ​​the guardrail, the slope connection area, and the hardened ground area outside the lane range are uniformly treated as non-detection areas and masked. The image area obtained after masking only includes the real road surface range inside the high-speed driving lane, which serves as the input area for the debris target detection algorithm. If there are local jagged edges or breaks at the edge of the effective attachment area, a small-scale smooth repair can be performed along the lane direction, with the repair width controlled between 0.1 meters and 0.3 meters, to avoid including debris next to the guardrail in the detection range again.

[0097] Among them, the spatial confidence of the effective attachment area The smoother and more unbroken the boundary, the better, based on the continuity and geometric regularity of the region's boundary. The higher the value, the range is from 0 to 1. To quantify the effectiveness of the detected region, a detection region effectiveness score is introduced. . Spatial confidence that the area belongs to the effective attachment area Positive correlation with the probability of belonging to the lane segmentation result. Positive correlation with traffic continuity score Positive correlation, and bonding strength with the lateral structure Negative correlation. When If the value of A1 is below 0.4, the area is masked from the detection range and not sent to subsequent debris detection algorithms. Through the above processing, areas with A1 not lower than 0.4 are retained as the areas to be detected in the high-speed driving lane image.

[0098] S3.2: A lightweight target detection model adapted to highway road scenes is adopted to identify candidate clutter in the image region to be detected and obtain a set of candidate clutter targets.

[0099] Based on the image region to be detected within the highway driving lane, an algorithm adapted to highway road surface scenarios is used to identify potential targets such as tire fragments, scattered goods, metal components, plastic obstacles, packaging bags, and fallen loose parts.

[0100] The target detection algorithm employs a lightweight target detection model. The model input is a road surface image after region definition, and the model output includes candidate target bounding boxes, target categories, target confidence scores, and target sizes, forming an initial set of candidate debris targets. During model training, the algorithm focuses on covering small targets on highways, dark tire debris, light-colored packaging bags, metallic reflective parts, and irregularly shaped scattered goods. The target confidence threshold is set between 0.45 and 0.75; if the drone's flight altitude is high, resulting in smaller road surface targets, the confidence threshold can be appropriately lowered, but it should not be lower than 0.35 to avoid excessive inclusion of vehicle shadows and damaged road markings.

[0101] Among them, the degree of overlap or interference between the target and lane markings, cracks, or shadow edges. Defined as the ratio of the area overlapping the outer contour of the target with the aforementioned interference region to the total area of ​​the target, the value ranges from 0 to 1. To quantify the reliability of candidate clutter identification, a candidate clutter target identification score is introduced. . Target confidence level output by the detection model Positively correlated with the visible area of ​​the target The score increases exponentially, meaning the larger the area, the higher the score, which correlates with the degree of texture anomaly in the target area. Positively correlated with the degree of overlap or interference between the target and lane markings, cracks, or shadow edges. Negative correlation. This score will serve as input for subsequent cascaded filtering steps, and the calculated... By associating them with the corresponding candidate targets, a set of candidate clutter targets with recognition scores is obtained.

[0102] S3.3: The candidate debris target set is processed by algorithm concatenation with the effective attachment area of ​​the high-speed curved road surface. The high-speed driving lane concatenation screening targets are obtained according to the geometric correlation conditions, and the concatenation screening score of each target is calculated.

[0103] Based on the candidate debris target set with recognition scores obtained in step S3.2, the detection results of the debris target detection algorithm are combined with the effective attachment area of ​​the high-speed curved road surface formed in step S2. Specifically, for each candidate debris target, the overlap relationship between its target center point, target bottom contact point, and target coverage area and the effective attachment area of ​​the high-speed curved road surface is calculated. Only targets that meet the following conditions are retained as high-speed driving lane cascaded screening targets: the target center point is located inside the high-speed driving lane, the target bottom contact point falls into the effective attachment area of ​​the high-speed curved road surface, and the overlap ratio between the target coverage area and the effective attachment area exceeds a preset overlap threshold (this threshold can be set according to the actual scenario, for example, 0.5 to 0.8). If part of the target box extends beyond the lane boundary, it is not directly deleted, but its bottom contact position is further judged to see if it is still inside the actual driving lane; if the bottom contact position is biased towards the outside of the guardrail or the side wall of the sound barrier, it is no longer retained.

[0104] To quantify the reliability of cascade screening, a cascade screening score is introduced. . Scoring of candidate clutter targets Positively correlated with the judgment value of whether the target center point is located inside the driving lane. Positively correlated with the overlapping area X of the target coverage area and the effective attachment area, and positively correlated with the area falling outside the lane range. Negative correlation, and the degree to which the bottom contact point of the target is close to the lateral structural isolation area. negative correlation, degree of proximity to the center of the lane Positive correlation. For each candidate target that satisfies the above geometric conditions, calculate its... The scores are then assigned and associated with the targets. Through this process, a cascaded selection target set for highway lanes with cascaded selection scores is obtained.

[0105] S3.4: Based on the cascaded screening score, and combined with the set of lateral structure isolation areas, perform lateral structure exclusion analysis on the candidate targets near the bridge deck edge in the cascaded screening targets of the high-speed driving lane, generate a set of bridge deck candidate targets, and calculate the bridge deck candidate score after lateral structure exclusion for each target.

[0106] Based on the cascaded screening target set of the high-speed driving lanes, lateral structure exclusion analysis is performed on candidate targets near the edge of the bridge deck. This lateral structure exclusion analysis is based on the set of lateral structure isolation areas generated in step S1. Specifically, candidate targets are continuously connected to the noise barrier sidewall area, the guardrail outer area, and the slope connection area. If there is a continuous boundary between the target's outer contour and the lateral structure isolation area, or if the target's long axis is aligned with the guardrail's contact direction, or if the target's shadow is integrated with the noise barrier sidewall projection, then the target is marked as a suspected lateral structure target. For lightweight targets such as packaging bags and plastic films, it should also be determined whether they are attached; if the target's main body is located at the edge of the lane but its contour extends towards the noise barrier or guardrail, it is preferentially excluded as a lateral attachment. Lateral structure exclusion is not simply deleted based on distance, but rather judged comprehensively based on connection relationships, directional relationships, and contact relationships. According to the above judgment rules, targets not marked as suspected lateral structure targets are retained, forming a preliminary excluded set of candidate bridge deck targets.

[0107] To quantify the credibility of the excluded targets, a bridge deck candidate score is introduced after lateral structure exclusion. . With cascaded screening scoring Positive correlation, the degree of continuous connectivity of the structural isolation region on the same side as the target. Negative correlation, with the lateral distance from the target center point to the nearest lane boundary. Positive correlation with the lateral distance from the target to the lane centerline. Negative correlation, and with the contact characteristic intensity between the target bottom and the road surface. Positive correlation. The calculated H3 is associated with the corresponding target to obtain a set of bridge deck candidate targets with bridge deck candidate scores.

[0108] S3.5: For the targets in the bridge deck candidate target set, the final lane validity is confirmed based on the bridge deck candidate score after the lateral structure is excluded. The final bridge deck internal debris candidate score for each target is calculated. The targets whose final bridge deck internal debris candidate scores meet the preset threshold are retained to generate a bridge deck internal debris candidate set.

[0109] Based on the candidate target set for the bridge deck after lateral structure exclusion, the targets in this set undergo final lane-in-lane validity confirmation, i.e., determining whether the target is completely located within the high-speed driving lane area and meets the bridge deck attachment conditions. Specifically, candidates already marked as noise barrier sidewall targets, guardrail outer targets, slope connection targets, and targets outside the lane range are first deleted; then, the remaining targets are merged and organized according to target category, target area, target location, and lane occupancy degree to obtain a merged and organized candidate set of debris inside the bridge deck. For cases where the same debris is repeatedly detected in consecutive images, it should be merged according to the principles of spatial proximity and category consistency to avoid duplicate output; for large scattered goods or continuous scattered areas composed of multiple fragments, they can be grouped according to the same scattering event. Each candidate debris in this set includes at least the target category, target location, actual occupied area of ​​the target, and lane area where it is located. For each target obtained after merging and organizing, its final candidate score I for debris inside the bridge deck is calculated.

[0110] To quantify the credibility of the final candidate spills, a final bridge deck interior spill candidate score I is introduced. I is compared with the bridge deck candidate scores after lateral structure exclusion. Positively correlated with the degree to which the target occupies the driving lane area. Positive correlation with the probability that the target is identified as debris outside the lane. Negative correlation, and with the repeated detection penalty value Negative correlation. Validity is confirmed based on the calculated I value: when I is greater than a preset threshold, such as 0.5, the target is confirmed as valid bridge deck debris and retained; otherwise, it is discarded. The calculated I is associated with the corresponding target to obtain a candidate set of bridge deck debris with a final score.

[0111] In the technical solution of this disclosure, by cascading the detection results of the spilled object target with the highway road segmentation results and performing exclusion analysis on the lateral structure, it is possible to effectively eliminate candidate targets located outside the lane range, such as the side wall of the sound barrier, the outside of the guardrail, and the slope area. This significantly reduces false detections caused by lightweight hanging objects or edge debris, and ensures that the targets sent to the subsequent verification stage are all real spilled object candidates inside the highway driving lane.

[0112] S4: Perform continuous frame tracking and attachment behavior analysis on the candidate debris in the candidate set of debris inside the bridge deck, identify and remove sidewall drift targets, and obtain stable bridge deck debris identification results.

[0113] S4.1: Based on the candidate set of spilled material inside the bridge deck, the center position, bottom contact position and outer contour area of ​​the candidate target are used as tracking features to associate them in continuous inspection frames to form a continuous frame tracking set of candidate targets.

[0114] Figure 2 A flowchart illustrating the continuous frame tracking and sidewall attachment determination of candidate projectiles provided in an embodiment of this disclosure is shown, as follows: Figure 2 As shown. Based on the candidate set of debris inside the bridge deck output in step S3, the positional change of each candidate target in continuous UAV inspection frames is tracked. Specifically, the center position, bottom contact position, outer contour area, target category, and lane area of ​​the candidate target are used as tracking features. Targets with similar positions, consistent categories, and small area changes in adjacent frames are associated to form continuous frame trajectories of the same candidate debris. For stable targets such as tire fragments and metal components, the area consistency requirement can be increased; for lightweight targets such as packaging bags and plastic films, the outer contour change requirement can be appropriately relaxed, but it must be ensured that the bottom contact position is still within the effective attachment area of ​​the high-speed curved road surface. The association distance between adjacent frames is controlled between 0.2 meters and 1.5 meters, and the number of consecutive confirmation frames is recommended to be no less than 3 frames; if the UAV inspection speed is fast, it can be increased to 5 frames to avoid instantaneous reflections or vehicle shadows being mistakenly inherited as stable targets.

[0115] To quantify whether targets in adjacent frames belong to the same projectile, an adjacent frame target matching score is introduced. . Consistency with candidate target category Positively correlated with the actual displacement distance d of the target bottom contact point between adjacent frames, negatively correlated with the change in outer contour area s, positively correlated with the target texture similarity q, and positively correlated with the final bridge deck interior debris candidate score I in step S3.5. When When the value is greater than 0.6, targets in two frames are associated with the same projectile trajectory. Through the above processing, a set of candidate target continuous frame tracking is obtained.

[0116] S4.2: Verify the adhesion continuity between the target in the candidate target continuous frame tracking set and the extension direction of the bridge deck center, and calculate the bridge deck continuous adhesion score.

[0117] Based on the continuous frame tracking set of candidate targets, the attachment continuity between the target and the extension direction of the bridge deck center is verified. Specifically, the results of the extension direction of the bridge deck center formed in step S2 and the effective attachment area of ​​the high-speed curved road surface are called to determine whether the bottom contact point of the target is always located within the effective attachment area in the continuous frames, and whether the direction of the target's position change is mainly along the extension direction of the bridge deck center. For spilled material actually located on the road surface, it should show a reasonable position change with the change of viewing angle in the continuous inspection image of the UAV, but its bottom contact point should not continuously shift towards the sound barrier, guardrail or slope; if the bottom contact point of the target gradually moves away from the direction of the lane center in multiple continuous frames, it indicates that it may belong to the lateral structure attachment.

[0118] To quantify the degree of continuous adhesion of the target bridge deck, a bridge deck continuous adhesion score is introduced. . Matching score with adjacent frames Positive correlation, the proportion of the target bottom contact point located within the effective adhesion area. It is positively correlated with the angle difference 'a' between the target's direction of motion and the direction of extension of the bridge deck center, and positively correlated with the minimum distance 'h' from the bottom contact point of the target to the boundary of the effective attachment area. This is the result of the bridge deck continuous adhesion verification, and this score will be used as the input for the subsequent step of calculating the true bridge deck adhesion score.

[0119] S4.3: Establish a longitudinal reference along the direction extending from the center of the bridge deck, establish a lateral reference along the direction perpendicular to the direction extending from the center of the bridge deck, calculate the lateral drift enhancement score, and mark the targets with the lateral drift enhancement trend as anomalies based on the lateral drift enhancement score to obtain the lateral drift anomaly marking results.

[0120] Based on the continuous adhesion verification results of the bridge deck, targets exhibiting an increased trend of lateral drift are anomaly-marked. Specifically, a longitudinal reference is established along the direction extending from the center of the bridge deck, and a lateral reference is established perpendicular to the direction extending from the center of the bridge deck. The distance changes from the bottom contact point of the candidate target to the lane centerline, lane boundary, noise barrier edge, guardrail edge, and slope connection area are continuously calculated. Based on these distance changes, the lateral displacement x and longitudinal displacement y of the target are calculated, and the curvature of the curve is corrected to obtain the correction coefficient k.

[0121] To quantify the degree of lateral drift enhancement of the target, a lateral drift enhancement score is introduced. . It is positively correlated with the square of the lateral displacement x of the target during continuous tracking, negatively correlated with the square of the longitudinal displacement y, and positively correlated with the curvature correction coefficient k. When the value is greater than 0.7, the target is marked as a lateral drift-enhanced target. This is based on the calculated... Value, when If the value is greater than 0.7, the target is marked as a lateral drift-enhanced target, and the lateral drift anomaly labeling result is obtained; otherwise, it is not labeled. Meanwhile, This will be used as input for subsequent steps to calculate the true bridge deck adhesion score.

[0122] S4.4: Based on the lateral drift anomaly marking results, the bridge deck continuous attachment score, and the lateral drift enhancement score, perform sidewall attachment determination on targets that continuously approach the edge of the sound barrier wall, the edge of the guardrail, or the slope connection area and maintain a synchronous drift trend, obtain the sidewall attachment determination results, and calculate the actual bridge deck attachment score.

[0123] Based on the obtained lateral drift anomaly marking results, targets marked as having enhanced lateral drift are filtered out. For targets that continuously approach the edge of the noise barrier, guardrail, or slope connection area and maintain a synchronous drift trend with the corresponding lateral structure, sidewall attachment determination is performed. Specifically, the continuous contact relationship between the target outline and the sidewall area of ​​the noise barrier, the outer area of ​​the guardrail, and the slope connection area is calculated. If the target's outer outline is in contact with the boundary of the lateral structure for multiple consecutive frames (e.g., more than 3 frames), the angle between the target's long axis and the extension direction of the guardrail or noise barrier is less than a preset angle threshold (e.g., 15 degrees), and the target's bottom contact position cannot stably fall into the effective attachment area of ​​the high-speed curved road surface for multiple consecutive frames, then the target is determined to be a sidewall attachment or edge non-lane debris. This determination result serves as one of the sidewall attachment determination results. For lightweight suspended objects such as packaging bags and plastic films, the focus should be on whether their outline spreads along the surface of the noise barrier rather than lying flat on the bridge surface; for fragments on the outer side of the guardrail, the focus should be on whether they cross the lane boundary into the passage area.

[0124] To quantify the reliability of the target's true bridge deck adhesion, a true bridge deck adhesion score is introduced. . Continuous adhesion score of bridge deck Positive correlation with lateral drift enhancement score The correlation is negatively correlated with the degree of continuous fit r of the target's lateral structural boundary, negatively correlated with the degree of consistency u between the target's long axis direction and the lateral structural extension direction, and negatively correlated with the area v of the target contour falling into the lateral structural isolation area. As a quantitative score in the sidewall adhesion determination result, a higher value indicates that the target is more likely to be actual bridge deck debris. This score will be used for the elimination and retention process in step S4.5. Through the above processing, the sidewall adhesion determination result is obtained, which includes a marker indicating whether it is sidewall debris and an actual bridge deck adhesion score. .

[0125] S4.5: Based on the sidewall adhesion determination result and the actual bridge deck adhesion score, perform final elimination and stable retention processing on the candidate targets to obtain the stable bridge deck spill identification result.

[0126] For each candidate target obtained in step S4.4, the sidewall adhesion determination result and the actual bridge deck adhesion score are used as the basis for the determination. Make the following decision:

[0127] If the sidewall attachment determination result is "yes", that is, the target is determined to be a sidewall attachment, then the removal process is directly performed and recorded as the sidewall drift removal result.

[0128] If the sidewall adhesion determination result is "no", then further determination is needed. :when When the retention threshold is reached, the debris is retained as stable bridge surface debris; otherwise, it is discarded. The retention threshold is set between 0.55 and 0.8; for road sections with high vehicle speeds, narrow lanes, and small exit curve radii, the threshold can be increased to above 0.7 to reduce false alarms. The final output of stable bridge surface debris identification results includes at least the target category, target location, lane, actual occupied area, number of consecutive frame confirmations, and bridge surface adhesion score.

[0129] To quantify the reliability of stable bridge deck spill confirmation, a stable bridge deck spill confirmation score is introduced. . Adhesion score of real bridge surface Positively correlated with the number of consecutive frames n of target confirmation, and positively correlated with the probability that the target is identified as a sidewall attachment. The score is negatively correlated with the number of interruptions in the continuous attachment relationship between the target and the driving lane (w). This score can be used as a reliability indicator in the output results for operation and maintenance reference, but it does not participate in the decision-making process for removal or retention. Through the above processing, the results of stable bridge deck debris identification and sidewall drift removal are obtained.

[0130] In the technical solution of this disclosure embodiment, by performing continuous frame tracking and bridge deck continuous attachment verification on candidate spills, and identifying the lateral drift enhancement trend to perform sidewall attachment determination, it is possible to distinguish those laterally suspended objects that gradually drift toward the sound barrier or guardrail from the spills that are actually located on the bridge deck. Finally, the sidewall attachment targets are eliminated, and stable bridge deck spills are retained, thereby significantly reducing repeated false alarms in the scenario of continuous curves and side sound barrier clamping.

[0131] S5: Based on the identification results of the stable bridge deck debris, perform lane occupancy analysis and hazard level assessment, and generate real-time early warning information, inspection and verification information and operation and maintenance scheduling information.

[0132] S5.1: For each debris in the stable bridge deck debris identification results, analyze the lane position, occupied area and traffic direction to obtain the debris lane occupancy analysis results and calculate the lane occupancy score.

[0133] Based on the stable bridge deck debris identification results and sidewall drift removal results output in step S4, each debris actually located inside the highway lane is analyzed for its lane location, occupied area, direction of travel, and bridge deck space area. Specifically, the target's center point, bottom contact point, and outer contour range are mapped to the effective attachment area of ​​the highway exit curve to determine whether it is located in the main lane, exit ramp, lane merging transition zone, or curve compression zone. Simultaneously, based on the sidewall drift removal results, it is confirmed that the target does not pose an attachment risk to the sidewall of the noise barrier, the outside of the guardrail, or the slope area. For targets crossing lane lines, the lane with the largest actual occupied area should be identified as the primary affected lane; for targets with multiple fragments continuously distributed, they should be grouped as a single debris event.

[0134] To quantify the extent to which spilled materials obstruct lanes, a lane occupancy score is introduced. . Compared with the actual occupied area of ​​the target Positive correlation with the proportion of lane width occupied by the target Positively correlated with the confirmation value of whether the target is located within the effective attachment area. Positively correlated with the lateral distance of the target from the lane centerline. Negative correlation. The analysis information of lane location, occupied area, and traffic direction, together with the lane occupancy score, constitutes the lane occupancy analysis result for spilled materials. The above processing yields the lane occupancy analysis result for spilled materials.

[0135] S5.2: Based on the lane occupancy analysis results of the spilled material and the lane occupancy score, calculate the hazard level score by combining the target category, curve radius and exit merging status.

[0136] Based on the lane occupancy analysis results of spilled materials, a comprehensive assessment is conducted on the target category, lane location, direction of travel, curve radius, exit merging status, and lane occupancy score to calculate the hazard level score. . Lane occupancy score Positive correlation with the risk factor of the spilled material category Positive correlation with the gain coefficient in curves or lane merging areas Positive correlation, gain coefficient with the target located in the center region of the lane. Positively correlated with the distance of the target from the road shoulder safety boundary. Negative correlation. Tire debris, metal components, and loose cargo are generally classified as higher base hazard levels; lightweight targets such as packaging bags and plastic films have lower base hazard levels, but their hazard level is increased if they are located in curve compression zones, exit merging zones, or the center of the lane. According to The value of can be used to classify the level of danger: when A value greater than or equal to 0.8 indicates an urgent risk. A risk level of 0.6 or higher and less than 0.8 indicates high risk; 0.4 or higher and less than 0.6 indicates medium risk; and less than 0.4 indicates low risk. (This refers to the risk level scoring.) This will be used for the warning priority calculation in subsequent step S5.4. Through the above processing, the spill hazard level assessment result is obtained, including a score. And the corresponding hazard level.

[0137] S5.3: Based on the real-time inspection location of the UAV, the image shooting height, the gimbal attitude, and the position of the target in the image, generate a spatial positioning result of the highway exit corrected along the lane centerline.

[0138] Based on the real-time inspection location of the drone, the image capture altitude, the gimbal attitude, and the target's position in the image, spatial positioning results for the corresponding highway exit are generated. The positioning results should include at least the road name, exit number, direction of travel, lane number, distance to the exit diversion point, bridge surface area, and a screenshot for on-site verification. For curved road sections, the mileage along the lane centerline should be used instead of simply using straight-line distance to prevent target positioning deviation within curves.

[0139] To obtain the corrected positioning mileage along the lane centerline, the target's corrected positioning mileage along the lane centerline is introduced. . Based on the road mileage corresponding to the drone projection point Combined with the distance of the target along the lane direction in the image The product of the lane center direction angle cosine and the corrected distance calculated from the lateral offset of the target in the image. The product of this product and the sine of the lane center direction angle is linearly superimposed to obtain the mileage. This mileage is the core parameter for the spatial positioning result of the highway exit. Through the above processing, the spatial positioning result of the highway exit is obtained.

[0140] S5.4: Based on the hazard level score and the spatial positioning result, generate real-time early warning information, inspection and verification information and operation and maintenance scheduling information.

[0141] Based on the spatial positioning results of highway exits, real-time early warnings are issued for debris spilled inside actual highway driving lanes. Warning information includes target category and hazard level score. (and its corresponding hazard level), lane location, and positioning mileage. And on-site images; inspection and verification information includes drone re-fly points, verification shooting angles, and verification priorities; operation and maintenance dispatch information includes the affected road section, suggested arrival direction, whether lane closures are needed, and whether manual verification is required. When the hazard level score When the value is greater than or equal to 0.8, high-priority operation and maintenance scheduling information is generated directly; when... If the value is less than 0.4 but continuous confirmation is required, a normal review task is generated.

[0142] To quantify the priority of early warnings, an early warning priority score is introduced. . Risk level rating Positive correlation, with the number of consecutive confirmed frames of the target. Positive correlation with estimated arrival time of maintenance Negative correlation, and confirmed score with stable bridge deck spills. Positive correlation. When When the score is greater than 0.7, high-priority operation and maintenance scheduling information is directly generated; otherwise, a normal review task is generated. This score is used for internal priority determination and is not directly output, but its decision result is reflected in the generated warning information. Through the above processing, real-time warning information, inspection and review information, and operation and maintenance scheduling information are obtained, ultimately achieving real-time identification and accurate warning of debris spilled inside the actual highway driving lane in scenarios where continuous curves at highway exits are sandwiched between lateral sound barriers.

[0143] In the technical solution of this disclosure embodiment, based on the identification results of real bridge surface debris, combined with lane occupancy analysis, hazard level assessment and spatial positioning corrected along the lane centerline, hierarchical early warning information and operation and maintenance dispatch instructions are generated. This enables real-time and accurate alarm for debris in continuous curve areas at highway exits, and provides clear spatial location and priority guidance for inspection review and on-site handling, thereby improving the efficiency and pertinence of highway operation and maintenance response.

[0144] According to embodiments of this disclosure, an electronic device is also provided, which may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other via the communication bus. The processor can invoke logical instructions stored in the memory to execute the methods provided in the above embodiments.

[0145] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0146] On the other hand, this disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0147] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0148] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0149] It should be understood that the above embodiments are only used to illustrate the technical solutions of this disclosure, and not to limit them; although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for real-time identification of spilled materials on highway driving lanes under the vision of unmanned aerial vehicle (UAV) inspection, characterized in that, The method includes: S1. The road surface area of ​​the highway exit curve is segmented into layers and the lateral structure boundary is extracted. The driving lane is separated from the side wall of the sound barrier and the outer area of ​​the guardrail, generating a set of highway driving lane areas and a set of lateral structure isolation areas. S2. Based on the set of high-speed driving lane areas, the high-speed curved road surface space is expanded into a linear strip area. Based on the expansion result, a bridge surface space attachment constraint is established to form an effective attachment area of ​​the high-speed curved road surface. S3. Perform target detection of spilled material within the effective attachment area of ​​the high-speed curved road surface. Based on the effective attachment area of ​​the high-speed curved road surface and the set of lateral structure isolation areas, perform cascade screening and elimination analysis on the detection results to obtain a candidate set of spilled material inside the bridge deck. S4. Perform continuous frame tracking and attachment behavior analysis on the candidate debris in the candidate set of debris inside the bridge deck, identify and remove sidewall drifting targets, and obtain stable bridge deck debris identification results. S5. Based on the identification results of the stable bridge surface debris, perform lane occupancy analysis and hazard level assessment, and generate real-time early warning information, inspection and verification information and operation and maintenance scheduling information.

2. The method according to claim 1, characterized in that, S1 includes: Distortion correction, tilt correction and scale unification processing are performed on the UAV inspection images to obtain a standardized top view image sequence; Based on the standardized top-view image sequence, lane lines, road edge lines, top edge of sound barrier, outer contour of guardrail and slope boundary line are extracted to form a set of basic road contour features; A deep learning semantic segmentation network is used to segment the high-speed driving lane area using the standardized top-view image sequence as input, thereby obtaining the initial segmentation result of the high-speed driving lane area. Based on the lane boundaries in the initial segmentation results of the high-speed lane area, and combined with the road basic contour feature set, a lateral structure isolation score is calculated for the adjacent areas outside the lane boundaries, and a lateral structure candidate region set is generated based on the lateral structure isolation score. The initial segmentation results of the high-speed driving lane area are subjected to connectivity checks and traffic continuity scores. Areas that do not meet the traffic continuity requirements are incorporated into the lateral structure candidate area set to form the final high-speed driving lane area set and lateral structure isolation area set.

3. The method according to claim 1, characterized in that, S2 includes: Based on the set of high-speed driving lane areas, the inner boundary, outer boundary and center zone of the high-speed driving lane are extracted, and the curved road surface is unfolded into a straight road strip area based on the center zone of the lane. The direction of change of the lane centerline, the direction of change of the bridge deck boundary, and the direction of guardrail contact are continuously tracked after the lane is unfolded, and the results of the extension direction of the bridge deck centerline, the tracking results of the bridge deck boundary, and the results of the guardrail contact direction are obtained. Based on the results of the extension direction of the bridge deck center, the bridge deck adhesion constraint strength is calculated; Based on the bridge deck boundary tracking results, a set of bridge deck adhesion attenuation zones is established at locations near the sound barrier sidewall, guardrail edge, and slope connection area; based on the bridge deck adhesion constraint strength and guardrail contact direction results, the attenuated bridge deck adhesion reliability is calculated. Based on the bridge deck adhesion attenuation zone set, the attenuated bridge deck adhesion reliability, and the bridge deck boundary tracking results, boundary reinforcement constraints are established for the local compressed lane area in the high-speed exit curve, and an effective adhesion area confirmation score is calculated. When the effective adhesion area confirmation score exceeds a preset threshold, the area is confirmed as an effective adhesion area of ​​the high-speed curve road surface.

4. The method according to claim 1, characterized in that, S3 includes: Using the effective attachment area of ​​the high-speed curved road surface as the only effective detection mask, the UAV inspection image is cropped and the detection range is limited to obtain the image area to be detected. A lightweight target detection model adapted to highway road scenes is used to identify candidate clutter in the image region to be detected, thereby obtaining a set of candidate clutter targets; The candidate debris target set is processed by algorithm concatenation with the effective attachment area of ​​the high-speed curved road surface. The high-speed driving lane concatenation screening targets are obtained according to the geometric correlation conditions, and the concatenation screening score of each target is calculated. Based on the cascaded screening score, and combined with the set of lateral structure isolation areas, lateral structure exclusion analysis is performed on the candidate targets near the bridge deck edge in the cascaded screening targets of the highway driving lane, generating a set of bridge deck candidate targets, and calculating the bridge deck candidate score after lateral structure exclusion for each target. For targets in the bridge deck candidate target set, the final lane validity is confirmed based on the bridge deck candidate scores after lateral structure exclusion. The final bridge deck internal debris candidate score for each target is calculated, and targets whose final bridge deck internal debris candidate scores meet a preset threshold are retained to generate a bridge deck internal debris candidate set.

5. The method according to claim 4, characterized in that, The algorithm cascade processing includes: The target is retained only if the center point of the candidate debris is located inside the high-speed driving lane, the bottom contact point falls into the effective attachment area of ​​the high-speed curved road surface, and the overlap ratio between the target coverage area and the effective attachment area exceeds a preset threshold; if the target frame crosses the lane boundary, it will not be retained if the bottom contact position is biased towards the outside of the guardrail or the side wall of the sound barrier.

6. The method according to claim 1, characterized in that, S4 includes: Based on the candidate set of debris inside the bridge deck, the center position, bottom contact position and outer contour area of ​​the candidate targets are used as tracking features, and they are correlated in continuous inspection frames to form a continuous frame tracking set of candidate targets. The adhesion continuity between the target in the candidate target continuous frame tracking set and the extension direction of the bridge deck center is verified, and the bridge deck continuous adhesion score is calculated. A longitudinal reference is established along the direction extending from the center of the bridge deck, and a lateral reference is established along the direction perpendicular to the direction extending from the center of the bridge deck. The lateral drift enhancement score is calculated, and targets with a lateral drift enhancement trend are marked as anomalies based on the lateral drift enhancement score to obtain lateral drift anomaly marking results. Based on the lateral drift anomaly marking results, the bridge deck continuous attachment score, and the lateral drift enhancement score, sidewall attachment determination is performed on targets that continuously approach the edge of the sound barrier, the edge of the guardrail, or the slope connection area and maintain a synchronous drift trend. The sidewall attachment determination results are obtained, and the actual bridge deck attachment score is calculated. Based on the sidewall adhesion determination result and the actual bridge deck adhesion score, the candidate targets are subjected to final elimination and stable retention processing to obtain the stable bridge deck spill identification result.

7. The method according to claim 6, characterized in that, The sidewall adhesion determination includes: When the outer contour of a candidate target is in long-term contact with the boundary of the lateral structure, the long axis of the target is consistent with the extension direction of the guardrail or sound barrier, and the bottom contact position cannot stably fall into the effective attachment area of ​​the high-speed curved road surface, the target is identified as a side wall attachment. For lightweight targets, it is also determined whether their outline extends along the surface of the sound barrier rather than being laid flat on the bridge surface.

8. The method according to claim 1, characterized in that, S5 includes: For each debris identified in the stable bridge deck debris identification results, the lane position, occupied area, and traffic direction are analyzed to obtain the debris lane occupancy analysis results, and the lane occupancy score is calculated; Based on the lane occupancy analysis results of the spilled material and the lane occupancy score, the hazard level score is calculated by combining the target category, curve radius and exit merging status. Based on the real-time inspection location of the drone, the image capture altitude, the gimbal attitude, and the target's position in the image, a spatial positioning result for the highway exit, corrected along the lane centerline, is generated. Based on the hazard level score and the spatial positioning results, real-time early warning information, inspection and verification information, and operation and maintenance scheduling information are generated.

9. An electronic device, characterized in that, The electronic device includes a memory and at least one processor. The memory stores a computer program, and the processor executes the computer program to implement the method for real-time identification of spilled materials on highway lanes under the vision of unmanned aerial vehicle (UAV) inspection, as described in any one of claims 1-8.

10. A computer storage medium, characterized in that, It stores a computer program, which, when executed, implements the method for real-time identification of spilled materials on highway lanes under the vision of unmanned aerial vehicle (UAV) inspection, as described in any one of claims 1-8.

Images