A system and method for detecting and analyzing road facility damage

Abstract

The present invention relates to the fields of image processing, target detection, visual large-scale models, and multi-terminal joint detection and analysis technologies, and more particularly to a system and method for detecting and analyzing road facility damage. The system comprises: a road facility anomaly identification module for performing preliminary anomaly identification on road facilities and obtaining suspected anomaly locations; a road facility image acquisition module for obtaining target-angle images of road facilities corresponding to suspected anomaly locations; and a cloud-based anomaly identification module for inputting the target-angle images into a visual large-scale model agent for further anomaly identification, outputting the anomaly identification results and spatial information of the anomaly location. The present invention combines roadside cameras and large-scale model technology for joint analysis, enabling real-time identification of road facility damage and its spatial information, enabling timely repair and updates to ensure road safety.

Description

Technical Field

[0001] The present invention relates to the technical fields of image processing, target detection, large visual models, and multi-terminal joint detection and analysis, and in particular to a system and method for detecting and analyzing damage to road facilities. Background Art

[0002] Road infrastructure refers to a series of structures and equipment installed to ensure traffic safety and improve traffic efficiency. These facilities play a vital role in transportation systems such as highways, urban roads, and expressways. Common road infrastructure includes road signs (including warning signs, direction signs, and directional signs), road markings, traffic lights, guardrails, interceptors, traffic monitoring systems, bridges, and tunnels. Road infrastructure plays a vital role in ensuring traffic safety, improving traffic efficiency, and enhancing the urban traffic environment. Appropriate planning and installation of road infrastructure can significantly enhance road efficiency and safety.

[0003] With the acceleration of urbanization and the increase in traffic volume, the number and complexity of road infrastructure are continuously increasing. These facilities, affected by factors such as the natural environment, traffic flow, and heavy transport, are subject to a growing number of damage and failures caused by overuse, wear, and external impacts, leading to increasingly prominent road infrastructure maintenance issues. Existing technologies are not yet able to effectively and timely monitor and assess the condition of road infrastructure. For example, traditional inspection methods require regular manual inspections by professionals, which is not only time-consuming and expensive, but also prone to missing potential problems. Recently, vehicle-mounted cameras, coupled with detection algorithms, are being used to perform real-time damage detection on road infrastructure. However, factors such as obstruction by other vehicles, vehicle vibration, excessive speed, and weather interference (such as rain and strong sunlight) can interfere with the images captured by vehicle-mounted cameras, significantly reducing detection accuracy and resulting in numerous false positives, requiring careful manual analysis.

[0004] Therefore, it is necessary to develop an effective technical solution to timely and accurately monitor and evaluate the damage status of road facilities and provide intelligent maintenance recommendations. Summary of the Invention

[0005] The purpose of the present invention is to provide a system and method for detecting and analyzing damage to road facilities. By combining roadside cameras and large-scale model technology for joint analysis, the system can identify damage to road facilities and their spatial information in real time, and carry out repairs and updates in a timely manner to ensure road driving safety.

[0006] To achieve the above objectives, the present invention provides, in one aspect, a system for detecting and analyzing road facility damage, comprising:

[0007] The road facility anomaly recognition module is used to perform preliminary anomaly recognition on road facilities and obtain the suspected anomaly location;

[0008] A road facility image acquisition module is used to obtain target angle images of road facilities corresponding to suspected abnormal locations;

[0009] The cloud-based anomaly recognition module is used to input the target angle image into the visual large model Agent for re-anomaly recognition, and output the anomaly recognition result and the spatial information of the anomaly.

[0010] Optionally, the road facility anomaly identification module includes:

[0011] Anomaly recognition unit, used to collect road facility images in real time, use target detection algorithms to identify facility damage in road facility images, and obtain suspected anomalies;

[0012] The location determination unit is configured to obtain location information of a suspected anomaly by using the GPS matching stake number of the first image acquisition device corresponding to the road facility image.

[0013] Optionally, the road facility image acquisition module includes:

[0014] A second image acquisition device matching unit is used to calculate and match the closest second image acquisition device based on the GPS corresponding to the suspected anomaly and the matched stake number;

[0015] A second image acquisition device setting unit, configured to set parameters of the second image acquisition device based on the stake number and the relative position of the matched second image acquisition device;

[0016] The second image acquisition device acquisition unit is used to shoot abnormal road facilities under the parameters of the second image acquisition device to obtain the target angle image.

[0017] Optionally, the second image acquisition device acquisition unit includes:

[0018] A first target angle image acquisition subunit is configured to photograph abnormal road facilities under the parameters of the second image acquisition device to acquire a first target angle image;

[0019] a second image acquisition device adjustment subunit, configured to detect the start and end positions of pixels of abnormal road facilities in the first target angle image, calculate an initial preset frame, and adjust parameters of the second image acquisition device based on the initial preset frame;

[0020] The second target angle image acquisition subunit is used to shoot again based on the adjusted second image acquisition device and output a second target angle image.

[0021] Optionally, the cloud-based anomaly recognition module includes a cloud-based anomaly recognition unit, which is used to input the second target angle image into the visual large model Agent, call the VLM for automatic recognition, and output the anomaly recognition result, spatial information of the anomaly and maintenance suggestions if there is a road facility anomaly; if there is no road facility anomaly, identify the starting and ending points of the road facility, calculate the preset frame, return to the second image acquisition device adjustment subunit to adjust the second image acquisition device parameters again, obtain a new image for re-recognition, and output no anomaly if no road facility anomaly is identified until the preset termination recognition condition is reached.

[0022] Optionally, the preset termination recognition condition includes: the difference between the longitudinal coordinates of the preset frames is less than Ht, and the pitch angle of the second image acquisition device reaches a maximum.

[0023] In another aspect, the present invention provides a method for detecting and analyzing road infrastructure damage, comprising:

[0024] Conduct preliminary anomaly identification on road facilities and obtain suspected anomaly locations;

[0025] Obtain target angle images of road facilities corresponding to suspected abnormal locations;

[0026] The target angle image is input into the visual large model agent for re-anomaly recognition, and the anomaly recognition result and the spatial information of the anomaly are output.

[0027] Optionally, preliminary anomaly identification of road facilities and obtaining suspected anomaly locations may include:

[0028] Collect road facility images in real time, use target detection algorithms to identify facility damage and obtain suspected anomalies;

[0029] The GPS of the first image acquisition device corresponding to the road facility image is used to perform stake number matching to obtain location information of the suspected anomaly.

[0030] Optionally, obtaining a target angle image of a road facility corresponding to the suspected abnormal location includes:

[0031] Based on the GPS corresponding to the suspected anomaly and the matching stake number, calculate and match the second image acquisition device that is closest to it;

[0032] Setting camera parameters based on the stake number and the relative position of the matched second image acquisition device;

[0033] photographing abnormal road facilities under the parameters of the second image acquisition device to obtain a first target angle image;

[0034] Detecting the starting and ending positions of the pixels of the abnormal road facilities in the first target angle image, calculating an initial preset frame, and adjusting the parameters of the second image acquisition device based on the initial preset frame;

[0035] The adjusted second image acquisition device is used to shoot again and output a second target angle image.

[0036] Optionally, the target angle image is input into the visual large model agent for further anomaly recognition, and the output of the anomaly recognition result and the spatial information of the anomaly location includes:

[0037] The second target angle image is input into the visual large model Agent, and the VLM is called for automatic identification. If there is a road facility abnormality, the abnormality identification result, the spatial information of the abnormality and the maintenance suggestion are output; if there is no road facility abnormality, the starting and ending points of the road facility are identified, the preset frame is calculated, and the second image acquisition device adjustment subunit is returned to adjust the parameters of the second image acquisition device again, and a new image is obtained for re-identification. If the road facility abnormality is not identified until the preset termination recognition condition is reached, the output is no abnormality, wherein the preset termination recognition condition includes that the longitudinal coordinate difference of the preset frame is less than Ht and the pitch angle of the second image acquisition device reaches the maximum.

[0038] The beneficial effects of the present invention are:

[0039] (1) High precision: Using vehicle-mounted camera target detection combined with multi-terminal cascade automatic analysis of roadside surveillance cameras can effectively improve recognition accuracy and reduce false alarms.

[0040] (2) High degree of automation: After the on-board camera detects and identifies damaged facilities, the system automatically calls the nearby roadside camera to capture the target location. Combining the visual large model agent and recognition technology, it automatically calls the camera SDK to adjust the gimbal, then identifies the damaged facilities in the image and gives the specific damage type, damage location and repair suggestions.

[0041] (3) Real-time monitoring: Through multi-terminal joint detection and analysis of large visual models, real-time monitoring can be achieved, facility damage can be discovered in a timely manner, and safety hazards and traffic accidents can be reduced.

[0042] (4) Reduce maintenance costs: Through automated detection and intelligent analysis, the frequency and cost of manual inspections can be reduced, maintenance efficiency can be improved, and the service life of facilities can be extended. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 This is a layout diagram for implementing an embodiment of the present invention;

[0045] Figure 2 A diagram showing the structure and workflow of a system for detecting and analyzing road facility damage according to an embodiment of the present invention;

[0046] Figure 3 This is a workflow diagram of the visual large model Agent according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] This embodiment is arranged as follows Figure 1 As shown in the figure, a system for detecting and analyzing road facility damage is provided, including patrol vehicles equipped with on-board cameras and edge devices, roadside cameras, and cloud servers. Figure 2 Shown, including:

[0050] The road facility anomaly recognition module is used to perform preliminary anomaly recognition on road facilities and obtain the suspected anomaly location;

[0051] A road facility image acquisition module is used to obtain target angle images of road facilities corresponding to suspected abnormal locations;

[0052] The cloud-based anomaly recognition module is used to input the target angle image into the visual large model agent for re-anomaly recognition, and output the anomaly recognition result and the spatial information of the anomaly.

[0053] Specifically, the function of the road facility abnormality recognition module of this embodiment is to Figure 1 The inspection vehicle equipped with on-board cameras and edge devices is realized, and the function of the road facility image acquisition module is realized through Figure 1The middle road side camera realizes the function of the cloud abnormality recognition module through Figure 1 Implementation of Zhongyun Server.

[0054] This embodiment uses on-board camera target detection, combined with multi-terminal cascade automatic analysis of roadside surveillance cameras, which can effectively improve recognition accuracy and reduce false alarms. After the on-board camera of this embodiment detects and identifies facility damage, the system automatically calls a nearby roadside camera to capture the target location. Combined with the visual large model agent and recognition technology, it automatically calls the camera SDK to adjust the pan / tilt, then identifies the facility damage and the spatial information where the damage is located in the image, and provides specific damage types and repair suggestions. Through multi-terminal joint detection and visual large model analysis, real-time monitoring can be achieved, facility damage can be discovered in a timely manner, and the occurrence of safety hazards and traffic accidents can be reduced. Through automated detection and intelligent analysis, the frequency and cost of manual inspections can be reduced, maintenance efficiency can be improved, and the service life of facilities can be extended.

[0055] Furthermore, the road facility anomaly recognition module includes:

[0056] Anomaly recognition unit, used to collect road facility images in real time, use target detection algorithms to identify facility damage in road facility images, and obtain suspected anomalies;

[0057] The location determination unit is used to use the GPS matching stake number of the first image acquisition device corresponding to the road facility image to obtain location information of the suspected anomaly.

[0058] Specifically, in this embodiment, target detection is used on the edge device to identify in real time whether there are any abnormalities in the road facilities, and the pile number is matched according to the GPS of the image acquisition device. The road facility abnormality identification module specifically works as follows:

[0059] Step 1.1: Identify anomalies in real time;

[0060] The on-board camera of the patrol vehicle collects real-time images Img_C of road facilities. The edge device uses a target detection algorithm for facility damage identification, including but not limited to the YOLO-V8 algorithm model, to perform real-time detection and identify abnormalities or damage on the Img_C image.

[0061] Step 1.2, calculate the position and pile number matching;

[0062] If step 1.1 identifies an abnormality in the current image, the image acquisition device's GPS is used to match the stake number, such as K1+100.

[0063] This module outputs whether there is an anomaly and a description of the anomaly location. For example, on the uphill section of K1+100, the left guardrail of the road is suspected to be damaged.

[0064] Furthermore, the road facility image acquisition module includes:

[0065] A second image acquisition device matching unit is used to calculate and match the closest second image acquisition device based on the GPS corresponding to the suspected anomaly and the matched stake number;

[0066] A second image acquisition device setting unit, configured to set parameters of the second image acquisition device based on the stake number and the relative position of the matched second image acquisition device;

[0067] The second image acquisition device acquisition unit is used to shoot abnormal road facilities under the second image acquisition device parameters to obtain target angle images.

[0068] Wherein, the second image acquisition device acquisition unit includes:

[0069] A first target angle image acquisition subunit is configured to capture abnormal road facilities under the second image acquisition device parameters to obtain a first target angle image;

[0070] a second image acquisition device adjustment subunit, configured to detect the start and end positions of pixels of abnormal road facilities in the first target angle image, calculate an initial preset frame, and adjust parameters of the second image acquisition device based on the initial preset frame;

[0071] The second target angle image acquisition subunit is used to shoot again based on the adjusted second image acquisition device and output a second target angle image.

[0072] Specifically, in this embodiment, the roadside camera has a 3D positioning function. It inputs the location of a pixel area in the roadside image Img_R, and can move this area to the center of the video, then zoom in or out. The camera's pitch and horizontal angles are rotated based on the relative position of the area center and the center of the roadside image Img_R. The specific workflow of the road facility image acquisition module is as follows:

[0073] Step 2.1, matching nearby roadside cameras;

[0074] According to the GPS and stake number calculated by the road facility anomaly recognition module, the nearest roadside monitoring camera is calculated.

[0075] Step 2.2: Set the initial state of the roadside camera;

[0076] After matching the stake information in step 1.2 and the nearby roadside camera in step 2.1, call the camera SDK based on their approximate relative positions to initially adjust the roadside camera angle, specifically by setting the pitch angle to the lowest and the focal length to the smallest, and then capture the current roadside camera image Img_R.

[0077] Calling the facility key point detection algorithm includes but is not limited to YOLOV8-POSE, detecting the starting and ending positions of the pixel points of the facilities in the roadside camera image Img_R, the starting points are [x0, y0] and [x1, y1] below the image, and calculating the initial preset box (represented by the upper left point and the lower right point, [xmin, ymin, xmax, ymax]) as [x1, y1, x0, y0].

[0078] Step 2.3: Initial view image of the roadside camera;

[0079] Input the initial preset frame obtained in step 2.2 into the PTZ control 3D positioning function in the camera SDK. The roadside camera will move the preset frame area to the center of image Img_R and zoom in. The image will now focus on the target facility and output the captured roadside camera image Img_R.

[0080] Furthermore, the cloud-based anomaly recognition module includes a cloud-based anomaly recognition unit, which is used to input the second target angle image into the visual large model Agent, call the VLM for automatic recognition, and output the anomaly recognition result, spatial information of the anomaly, and maintenance suggestions if there is a road facility anomaly; if there is no road facility anomaly, the starting and ending points of the road facility are identified, the preset frame is calculated, and the second image acquisition device adjustment subunit is returned to adjust the parameters of the second image acquisition device again, and a new image is obtained for re-recognition. If the road facility anomaly is not identified until the preset termination recognition condition is reached, the output is no anomaly.

[0081] The preset termination recognition conditions include: the difference between the longitudinal coordinates of the preset frames is less than Ht, and the pitch angle of the second image acquisition device reaches the maximum.

[0082] Specifically, in this embodiment, the cloud-based anomaly recognition module receives the roadside image Img_R captured by the roadside camera at the initial angle obtained in step 2.3, invokes the VLM for automatic recognition, and analyzes and performs subsequent operations. Facilities such as guardrails are generally long, so the camera needs to be controlled to scan along the guardrail. The scan will magnify the target and gradually extend it into the distance, thereby identifying whether an anomaly actually exists and the spatial information of the anomaly. The specific workflow is as follows:

[0083] Step 3.1, Visual Large Model Agent;

[0084] Determine whether the current roadside camera image Img_R really has any abnormalities and the spatial information of the abnormality. If not, get the next preset frame, call the roadside camera SDK, input the preset frame, get the camera adjusted image, repeat this process, and complete the zoom scan of the facility and abnormality identification, such as Figure 3 shown.

[0085] The visual large model agent includes three functions:

[0086] A. Use tool 1 to obtain the facility location. The input is the roadside camera image Img_R. Call the facility key point detection to identify the facility start and end key points in the input image, calculate the start and end points of the facility in the input image, and return the start and end points.

[0087] B. Use tool 2 to calculate the preset frame. The input is the starting and ending points of the facility. Recalculate the next preset frame. The specific calculation method is:

[0088] a) The horizontal xmin is the minimum value of the starting and ending points of the facility.

[0089] b) The horizontal xmax is the maximum horizontal value of the starting and ending points of the facility.

[0090] c) Longitudinal ymin is the longitudinal coordinate of the starting point of the facility -Hs, and Hs defaults to 100.

[0091] d) Longitudinal ymax is the longitudinal coordinate of the end point of the facility.

[0092] Returns the calculated preset frame. The center point of this preset frame is generally higher than the center point of the image, which will guide the camera to raise the pitch angle and extend the shooting facilities.

[0093] C. Use Tool 3 to obtain the adjusted roadside image, enter the preset frame, call the roadside camera SDK, and configure the preset frame to the 3D control in the SDK. The roadside camera will automatically adjust and return the adjusted roadside camera image Img_R.

[0094] Task: Please pay attention to the road facilities in the image and determine whether there is any damage. The judgment is as follows:

[0095] 1) If damage cannot be effectively identified, call A to obtain the starting and ending points of the facility, then call B to obtain the preset frame, and finally call C to obtain the adjusted roadside image, and repeat this step.

[0096] 2) If damage is found, confirm that damage is indeed present, output the damage, and record the name of the currently damaged facility, damage type, damage location (including spatial information, such as above or below the elevated road, above or below the ramp, etc.), and repair suggestions.

[0097] 3) Repeat steps 1) and 2) above, and terminate if any of the following three conditions are met:

[0098] Termination condition 1: Damage is found.

[0099] Termination condition 2: If the difference between the longitudinal coordinates of the preset frame is less than Ht, which is 200 by default, it is determined that this section of the facility has been scanned to the end.

[0100] Termination condition 3: The pitch angle of the roadside camera reaches the maximum.

[0101] If termination conditions 1 and 2 are met and no damage is found at the adjustment end point, the output is "no abnormal damage here".

[0102] Step 3.2, loop call analysis;

[0103] When damage is found in step 3.1, set the interval to call execution module 3 for 5 consecutive times at 20s.

[0104] The five returned results are comprehensively analyzed. If any of them indicate damaged facilities, the current image area is deemed to have damaged facilities. This allows the user to determine whether damaged facilities are present, including the specific name and type of damaged facility, location, and repair recommendations. This information is then promptly fed back to the road administration for confirmation and repair.

[0105] In another aspect, this embodiment provides a method for detecting and analyzing road facility damage, including:

[0106] Conduct preliminary anomaly identification on road facilities and obtain suspected anomaly locations;

[0107] Obtain target angle images of road facilities corresponding to suspected abnormal locations;

[0108] The target angle image is input into the visual large model agent for re-anomaly recognition, and the anomaly recognition result and the spatial information of the anomaly are output.

[0109] Furthermore, preliminary anomaly identification of road facilities and obtaining suspected anomaly locations include:

[0110] Collect road facility images in real time, use target detection algorithms to identify facility damage and obtain suspected anomalies;

[0111] The GPS of the first image acquisition device corresponding to the road facility image is used to perform stake number matching to obtain location information of the suspected anomaly.

[0112] Furthermore, obtaining a target angle image of the road facility corresponding to the suspected abnormal location includes:

[0113] Based on the GPS corresponding to the suspected anomaly and the matching stake number, calculate and match the second image acquisition device that is closest to it;

[0114] Setting camera parameters based on the stake number and the relative position of the matched second image acquisition device;

[0115] photographing abnormal road facilities under the second image acquisition device parameters to obtain a first target angle image;

[0116] Detecting the starting and ending positions of the pixels of the abnormal road facilities in the first target angle image, calculating an initial preset frame, and adjusting the parameters of the second image acquisition device based on the initial preset frame;

[0117] The adjusted second image acquisition device is used to shoot again and output a second target angle image.

[0118] Furthermore, the target angle image is input into the visual large model agent for further anomaly recognition. The output of the anomaly recognition result and the spatial information of the anomaly location includes:

[0119] The second target angle image is input into the visual large model Agent, and the VLM is called for automatic identification. If there is a road facility abnormality, the abnormality identification result, the spatial information of the abnormality and the maintenance suggestion are output; if there is no road facility abnormality, the starting and ending points of the road facility are identified, the preset frame is calculated, and the second image acquisition device adjustment subunit is returned to adjust the parameters of the second image acquisition device again, and a new image is obtained for re-identification. If the road facility abnormality is not identified until the preset termination recognition condition is reached, the output is no abnormality, wherein the preset termination recognition condition includes that the longitudinal coordinate difference of the preset frame is less than Ht and the pitch angle of the second image acquisition device reaches the maximum.

[0120] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A system for detecting and analyzing road facility damage, characterized in that: include: The road facility anomaly recognition module is used to perform preliminary anomaly recognition on road facilities and obtain the suspected anomaly location; A road facility image acquisition module is used to obtain target angle images of road facilities corresponding to suspected abnormal locations; The cloud-based anomaly recognition module is used to input the target angle image into the visual large model agent for re-anomaly recognition, and output the anomaly recognition result and the spatial information of the anomaly; The road facility anomaly recognition module includes: Anomaly recognition unit, used to collect road facility images in real time, use target detection algorithms to identify facility damage in road facility images, and obtain suspected anomalies; A location determination unit, configured to obtain location information of a suspected anomaly using the GPS matching stake number of the first image acquisition device corresponding to the road facility image; The road facility image acquisition module includes: A second image acquisition device matching unit is used to calculate and match the closest second image acquisition device based on the GPS corresponding to the suspected anomaly and the matched stake number; A second image acquisition device setting unit, configured to set parameters of the second image acquisition device based on the stake number and the relative position of the matched second image acquisition device; A second image acquisition device acquisition unit, configured to photograph abnormal road facilities under the second image acquisition device parameters to acquire the target angle image; The second image acquisition device acquisition unit includes: A first target angle image acquisition subunit is configured to photograph abnormal road facilities under the parameters of the second image acquisition device to acquire a first target angle image; a second image acquisition device adjustment subunit, configured to detect the start and end positions of pixels of abnormal road facilities in the first target angle image, calculate an initial preset frame, and adjust parameters of the second image acquisition device based on the initial preset frame; A second target angle image acquisition subunit, configured to capture images again based on the adjusted second image acquisition device and output a second target angle image; The cloud-based anomaly recognition module includes a cloud-based anomaly recognition unit, which is used to input the second target angle image into the visual large model Agent, call the VLM for automatic recognition, and output the anomaly recognition result, spatial information of the anomaly, and maintenance suggestions if there is a road facility anomaly; if there is no road facility anomaly, identify the starting and ending points of the road facility, calculate the preset frame, return to the second image acquisition device adjustment subunit to adjust the parameters of the second image acquisition device again, obtain a new image for re-recognition, and output no anomaly if no road facility anomaly is identified until the preset termination recognition condition is met; The preset termination recognition condition includes: the difference between the longitudinal coordinates of the preset frames is less than Ht or the pitch angle of the second image acquisition device reaches the maximum.

2. A method for detecting and analyzing road facility damage, characterized in that: include: Conduct preliminary anomaly identification on road facilities and obtain suspected anomaly locations; Obtain target angle images of road facilities corresponding to suspected abnormal locations; Input the target angle image into the visual large model agent for re-anomaly recognition, and output the anomaly recognition result and the spatial information of the anomaly; Conduct preliminary anomaly identification on road facilities and obtain suspected anomaly locations, including: Collect road facility images in real time, use target detection algorithms to identify facility damage and obtain suspected anomalies; Perform stake number matching using the GPS of the first image acquisition device corresponding to the road facility image to obtain location information of the suspected anomaly; Obtaining target angle images of road facilities corresponding to suspected abnormal locations includes: Based on the GPS corresponding to the suspected anomaly and the matching stake number, calculate and match the second image acquisition device that is closest to it; Setting camera parameters based on the stake number and the relative position of the matched second image acquisition device; photographing abnormal road facilities under the parameters of the second image acquisition device to obtain a first target angle image; Detecting the starting and ending positions of the pixels of the abnormal road facilities in the first target angle image, calculating an initial preset frame, and adjusting the parameters of the second image acquisition device based on the initial preset frame; Taking pictures again based on the adjusted second image acquisition device to output a second target angle image; The target angle image is input into the visual large model agent for further anomaly recognition, and the output of the anomaly recognition result and the spatial information of the anomaly includes: The second target angle image is input into the visual large model Agent, and the VLM is called for automatic identification. If there is a road facility abnormality, the abnormality identification result, the spatial information of the abnormality and the maintenance suggestion are output; if there is no road facility abnormality, the starting and ending points of the road facility are identified, the preset frame is calculated, and the second image acquisition device adjustment subunit is returned to adjust the parameters of the second image acquisition device again, and a new image is obtained for re-identification. If the road facility abnormality is not identified until the preset termination recognition condition is reached, the output is no abnormality, wherein the preset termination recognition condition includes that the longitudinal coordinate difference of the preset frame is less than Ht or the pitch angle of the second image acquisition device reaches the maximum.

Images