Roadside facility tilt diagnosis method, device, equipment and storage medium

Abstract

The application discloses a roadside facility tilt diagnosis method and device, equipment and a storage medium, relates to the technical field of highway facility detection, and comprises the following steps: processing the collected video frame sequence by using a preset deep learning model to obtain a corresponding target pixel region set and a reference pixel region set; based on the positioning track data, two-dimensional pixel points in the target pixel region set and the reference pixel region set are projected to a three-dimensional space to obtain a target three-dimensional point cloud; after plane fitting of the target three-dimensional point cloud is performed by using a RANSAC algorithm to obtain a first normal vector and a second normal vector of a surrounding preset reference object, the angle characteristic value corresponding to the target roadside facility is determined; the angle health baseline corresponding to the target roadside facility is determined by acquiring all historical angle characteristic observation values from a historical asset database, so that the angle characteristic value is abnormally diagnosed to obtain the tilt abnormal diagnosis result of the target roadside facility. In this way, the facility abnormal diagnosis success rate can be improved.

Description

Technical Field

[0001] This invention relates to the field of highway facility inspection technology, and in particular to methods, devices, equipment and storage media for diagnosing the tilt of roadside facilities. Background Technology

[0002] With the continuous expansion of highway transportation infrastructure, the structural health of roadside facilities such as traffic signs, posts, and crash barriers is directly related to road operational safety. During long-term service, roadside facilities are prone to structural defects such as tilting due to factors such as foundation settlement, vehicle impacts, and wind loads. Therefore, automated inspection and diagnosis of the tilt status of roadside facilities has become an important requirement in the field of highway maintenance.

[0003] In existing technologies, roadside facility tilt detection schemes mainly fall into the following categories: The first category is a 3D reconstruction scheme based on LiDAR point clouds, which involves acquiring 3D point clouds of roadside facilities using vehicle-mounted LiDAR and measuring their spatial attitude. However, LiDAR equipment is expensive and requires a huge amount of data processing, making it difficult to deploy on a large scale in ordinary inspection vehicles and lacking the economic viability for high-frequency, large-area monitoring. The second category is a detection scheme based on 2D images, which determines whether a roadside facility has tilted by examining its apparent features such as edges and symmetry in a single frame image. However, this type of scheme is a 2D surface detection, which cannot quantify the 3D structural deformation of the roadside facility and is highly susceptible to interference from factors such as changes in lighting and vehicle occlusion, resulting in poor detection reliability. The third type is a judgment scheme based on single visual measurement and fixed thresholds. This involves measuring the tilt angle of roadside facilities in a single measurement and judging tilt anomalies based on fixed physical thresholds such as whether it is perpendicular to the horizontal plane. However, on the one hand, changes in vehicle posture caused by load changes, suspension bumps, or road slope during the inspection vehicle's operation will directly couple to the measured tilt angle, causing serious systematic errors. On the other hand, many roadside facilities have initial tilt during installation. Judgment based on absolute angle from a single measurement will misjudge such initial installation deviations as structural anomalies, resulting in a high false alarm rate and wasting maintenance resources.

[0004] Therefore, how to achieve accurate and reliable diagnosis of the tilt status of roadside facilities is a problem that needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for diagnosing the tilt of roadside facilities, which can improve the accuracy and reliability of roadside facility tilt diagnosis. The specific solution is as follows: In a first aspect, this application discloses a method for diagnosing the tilt of roadside facilities, including: The video frame sequence and positioning trajectory data collected by the target inspection vehicle during its driving process are acquired, and the video frame sequence is processed using a preset deep learning model to obtain the target pixel region set corresponding to the target roadside facility and the reference pixel region set corresponding to the preset reference objects around the target roadside facility. Based on the positioning trajectory data, the two-dimensional pixels in the target pixel region set and the reference pixel region set are projected into three-dimensional space to obtain the target three-dimensional point cloud; The RANSAC algorithm is used to perform planar fitting on the target three-dimensional point cloud to obtain the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference objects, and the angular feature value corresponding to the target roadside facility is determined based on the first normal vector and the second normal vector. All historical angle feature observations corresponding to the target roadside facility are determined from the historical asset database, and the angle health baseline corresponding to the target roadside facility is determined based on all historical angle feature observations. Based on the angle health baseline, anomaly diagnosis of the angle feature value is performed to obtain the tilt anomaly diagnosis result of the target roadside facility.

[0006] Optionally, acquiring the video frame sequence and positioning trajectory data collected by the target inspection vehicle during its operation includes: The control target inspection vehicle collects video streams during its journey using a monocular camera to obtain a video frame sequence, and uses the positioning data corresponding to each video frame collected by the positioning receiver to obtain positioning trajectory data.

[0007] Optionally, the step of processing the video frame sequence using a preset deep learning model to obtain a set of target pixel regions corresponding to the target roadside facility and a set of reference pixel regions corresponding to preset reference objects surrounding the target roadside facility includes: A preset deep learning model is used to perform semantic segmentation on each video frame in the video frame sequence to obtain the target pixel region set and category label corresponding to the target roadside facility in each video frame, as well as the reference pixel region set corresponding to the surrounding preset reference objects of the target roadside facility; the surrounding preset reference objects include the surrounding road surface and / or the surrounding guardrail.

[0008] Optionally, the step of projecting the two-dimensional pixels in the target pixel region set and the reference pixel region set onto three-dimensional space based on the positioning trajectory data to obtain a target three-dimensional point cloud includes: Based on the multi-view geometric consistency method, and utilizing the pose constraints of the target inspection vehicle in the positioning trajectory data, the two-dimensional pixels in the target pixel region set and the reference pixel region set are projected into three-dimensional space to obtain the three-dimensional point cloud of the target roadside facility and the reference three-dimensional point cloud of the surrounding preset reference object.

[0009] Optionally, the step of using the RANSAC algorithm to perform planar fitting on the target 3D point cloud to obtain the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference objects, and determining the angular feature value corresponding to the target roadside facility based on the first normal vector and the second normal vector, includes: The RANSAC algorithm is used to randomly sample and iteratively fit the three-dimensional point cloud of the facility to obtain the plane equation corresponding to the three-dimensional point cloud of the facility, and the normal vector corresponding to the plane equation is determined as the first normal vector. The RANSAC algorithm is used to randomly sample and iteratively fit the reference 3D point cloud to obtain the plane equation corresponding to the reference 3D point cloud, and the normal vector corresponding to the plane equation is determined as the second normal vector. The dot product of the first normal vector and the second normal vector, and the product of the magnitude of the first normal vector and the magnitude of the second normal vector are determined. The inverse cosine of the ratio of the dot product to the product is determined as the angle between the first normal vector and the second normal vector, so as to obtain the angular feature value corresponding to the target roadside facility.

[0010] Optionally, determining all historical angular feature observations corresponding to the target roadside facility from the historical asset database, and determining the angular health baseline corresponding to the target roadside facility based on all historical angular feature observations, includes: In the historical asset database, multiple candidate historical physical instances whose distance to the geographic coordinates corresponding to the target roadside facility is less than a preset radius are retrieved; the geographic coordinates are determined based on the spatial offset between the target 3D point cloud corresponding to the target roadside facility and the positioning point in the positioning trajectory data. From the plurality of candidate historical physical instances, select the candidate historical physical instances whose corresponding category labels are the same as those of the target roadside facility; Among the selected candidate historical physical instances, those whose corresponding geographic coordinates are less than the difference between the geographic coordinates of the target roadside facility and the geographic coordinates of the target roadside facility are determined to be the same historical physical instance as the target roadside facility. Extract all historical angular feature observations corresponding to the same historical physical instance from the historical asset database; Determine the historical mean and historical standard deviation of all historical angle feature observations, and determine the angle health baseline corresponding to the target roadside facility based on the historical mean and historical standard deviation.

[0011] Optionally, the step of performing anomaly diagnosis on the angle feature values ​​based on the angle health baseline to obtain the tilt anomaly diagnosis result of the target roadside facility includes: A significance test is performed on the angular feature value and the angular health baseline to obtain the significance level, and the deviation between the angular feature value and the historical mean is calculated; If the significance level meets the preset significance threshold, and the deviation exceeds the preset physical threshold, then the tilt anomaly diagnosis result is determined to be a tilt anomaly warning. If the significance level does not meet the preset significance threshold, or the deviation does not exceed the preset physical threshold, then the tilt anomaly diagnosis result is determined to be normal.

[0012] Secondly, this application discloses a roadside facility tilt diagnostic device, comprising: The data acquisition module is used to acquire video frame sequences and positioning trajectory data collected by the target inspection vehicle during its driving process, and to process the video frame sequences using a preset deep learning model to obtain the target pixel region set corresponding to the target roadside facility and the reference pixel region set corresponding to the preset reference objects around the target roadside facility. The three-dimensional mapping module is used to project two-dimensional pixels in the target pixel region set and the reference pixel region set into three-dimensional space based on the positioning trajectory data to obtain a target three-dimensional point cloud. The feature value determination module is used to perform plane fitting on the target three-dimensional point cloud using the RANSAC algorithm to obtain the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference object, and to determine the angle feature value corresponding to the target roadside facility based on the first normal vector and the second normal vector. The baseline determination module is used to determine all historical angle feature observations corresponding to the target roadside facility from the historical asset database, and to determine the angle health baseline corresponding to the target roadside facility based on all historical angle feature observations. An anomaly diagnosis module is used to perform anomaly diagnosis on the angle feature value based on the angle health baseline to obtain the tilt anomaly diagnosis result of the target roadside facility.

[0013] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned method for diagnosing roadside facility tilt.

[0014] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned method for diagnosing the tilt of roadside facilities.

[0015] As can be seen, in this application, video frame sequences and positioning trajectory data collected by the target inspection vehicle during its operation are obtained, and the video frame sequences are processed using a preset deep learning model to obtain a set of target pixel regions corresponding to the target roadside facility and a set of reference pixel regions corresponding to the surrounding preset reference objects of the target roadside facility; based on the positioning trajectory data, the two-dimensional pixels in the target pixel region set and the reference pixel region set are projected into three-dimensional space to obtain a target three-dimensional point cloud; the RANSAC algorithm is used to perform plane fitting on the target three-dimensional point cloud to obtain a first normal vector corresponding to the target roadside facility and a second normal vector of the surrounding preset reference objects, and the angle feature value corresponding to the target roadside facility is determined based on the first normal vector and the second normal vector; all historical angle feature observation values ​​corresponding to the target roadside facility are determined from the historical asset database, and the angle health baseline corresponding to the target roadside facility is determined based on all historical angle feature observation values; based on the angle health baseline, the angle feature values ​​are subjected to anomaly diagnosis to obtain the tilt anomaly diagnosis result of the target roadside facility. That is, by projecting the two-dimensional pixels in the target pixel region set and the reference pixel region set into three-dimensional space using positioning trajectory data, a target three-dimensional point cloud is obtained. The spatial position constraints provided by the positioning trajectory data compensate for the lack of three-dimensional geometric information in two-dimensional images, achieving quantitative recovery of the three-dimensional spatial attitude of roadside facilities and overcoming the limitation of two-dimensional surface detection in quantifying three-dimensional structural deformation. Using the RANSAC algorithm to perform planar fitting on the target three-dimensional point cloud, noise and outliers in the point cloud can be effectively eliminated through random sampling and iterative fitting. This allows for accurate and robust extraction of the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference objects. Furthermore, since the angular feature value is determined based on the first and second normal vectors, that is, the surrounding preset reference objects in the same scene as the target roadside facility and observed synchronously are used as reference references to measure the facility's attitude, rather than using the camera coordinate system or the absolute horizontal plane as a fixed reference. This approach eliminates the impact of vehicle attitude interference and initial installation deviations on diagnostic results without relying on expensive 3D sensing hardware, significantly reducing false alarm rates and improving the accuracy, robustness, and economy of roadside facility tilt diagnosis. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of a roadside facility tilt diagnosis method disclosed in this application; Figure 2 This is a partial 3D geometric reconstruction effect diagram disclosed in this application; Figure 3 This application discloses a time-series interaction diagram for cross-batch instance association and anomaly diagnosis decision-making. Figure 4 This is a schematic diagram of dynamic feature modeling and historical health baseline of a landmark surface disclosed in this application; Figure 5 This is a schematic diagram of the comparative distribution of abnormal diagnosis based on significance testing disclosed in this application; Figure 6 This application discloses a specific method for diagnosing the tilt of roadside facilities. Figure 7 This is a schematic diagram of the structure of a roadside facility tilt diagnostic device disclosed in this application; Figure 8 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Roadside infrastructure is highly susceptible to tilting defects during service. Existing solutions rely on costly lidar, struggle to quantify three-dimensional deformation using two-dimensional images, and are prone to interference from vehicle attitude during single measurements, often misreporting initial installation deviations as anomalies. Therefore, this application specifically introduces a method for diagnosing roadside infrastructure tilt, which addresses these issues.

[0020] See Figure 1 As shown in the figure, this application discloses a method for diagnosing the tilt of roadside facilities, including: Step S11: Obtain the video frame sequence and positioning trajectory data collected by the target inspection vehicle during its journey, and process the video frame sequence using a preset deep learning model to obtain the target pixel region set corresponding to the target roadside facility and the reference pixel region set corresponding to the preset reference objects surrounding the target roadside facility.

[0021] In this embodiment, acquiring the video frame sequence and positioning trajectory data collected by the target inspection vehicle during its operation includes: controlling the target inspection vehicle to collect video streams during its operation via a monocular camera to obtain a video frame sequence, and using the positioning data corresponding to each video frame collected by a positioning receiver to obtain positioning trajectory data. That is, it can acquire the vehicle-mounted monocular continuous video stream of the road segment to be inspected, as well as synchronized GPS (Global Positioning System) / GNSS (Global Navigation Satellite System) positioning trajectory data. It should be noted that the positioning receiver can be a GPS / GNSS positioning receiver. The target inspection vehicle can transmit the collected video frame sequence and positioning trajectory data to the diagnostic processing system, which then performs subsequent processing steps to achieve multi-source data fusion acquisition. It is understood that the target roadside facilities can be roadside infrastructure such as signs, posts, crash barriers, and guardrails.

[0022] In this embodiment, processing the video frame sequence using a preset deep learning model to obtain a set of target pixel regions corresponding to the target roadside facility and a set of reference pixel regions corresponding to the surrounding preset reference objects of the target roadside facility includes: performing semantic segmentation on each video frame in the video frame sequence using the preset deep learning model to obtain, in each video frame, a set of target pixel regions and category labels corresponding to the target roadside facility, and a set of reference pixel regions corresponding to the surrounding preset reference objects of the target roadside facility; the surrounding preset reference objects include the surrounding road surface and / or, the surrounding guardrail. It should be noted that the preset deep learning model can be, for example, a model combining YOLOv8 (YouOnly Look Once v8, an object detection model) and Grounded-SAM; wherein, Grounded-SAM is a semantically aware segmentation framework combining Grounding DINO (Grounding DINO) and Segment Anything Model (SAM), which can accurately segment specified category regions such as road surface, guardrail, and signs through text prompts, thereby achieving semantic awareness and refined segmentation of each video frame in the video frame sequence.

[0023] It should be noted that, in this embodiment, during semantic segmentation, not only is the set of target pixel regions corresponding to the target roadside facilities (such as signs) segmented, but the road surface region and / or guardrail region in the same frame of video are also extracted simultaneously to obtain the reference pixel region set, providing geometric support for the subsequent establishment of a dynamic local reference system. Road surfaces and guardrails are geometrically and semantically stable scene elements, and can be used as a set of reference points for registration within the local reconstruction coordinate system, i.e., as scene anchor points. Each frame must simultaneously extract both the "sign region" and the "road surface region" (if the road surface is occluded or segmentation fails, the extraction of the "guardrail region" in the same frame is automatically switched as a substitute); neither can be omitted. The road surface / guardrail region is used for the subsequent establishment of the dynamic reference system, and the sign region is used for subsequent tilt angle calculation.

[0024] Step S12: Based on the positioning trajectory data, project the two-dimensional pixels in the target pixel region set and the reference pixel region set into three-dimensional space to obtain the target three-dimensional point cloud.

[0025] In this embodiment, as Figure 2 As shown, the step of projecting two-dimensional pixels in the target pixel region set and the reference pixel region set into three-dimensional space based on the positioning trajectory data to obtain a target three-dimensional point cloud includes: based on a multi-view geometric consistency method and utilizing the pose constraints of the target inspection vehicle in the positioning trajectory data, projecting two-dimensional pixels in the target pixel region set and the reference pixel region set into three-dimensional space to obtain a facility three-dimensional point cloud corresponding to the target roadside facility and a reference three-dimensional point cloud corresponding to the surrounding preset reference object. In specific spatiotemporal constraints for local 3D geometric reconstruction, the visual motion features between multiple frame image sequences can be used, combined with the spatial constraints provided by the positioning trajectory, to recover the three-dimensional point cloud of the target roadside facility and its surrounding road surface using a multi-view geometric consistency method; that is, after processing the vehicle-mounted monocular continuous video stream, the local three-dimensional point cloud distribution of the target roadside facility and its surrounding environment can be recovered. Monocular 3D reconstruction refers to the technique of recovering the geometric coordinates and orientation of a target object in three-dimensional space using only the video sequence from a vehicle-mounted monocular camera, combined with the motion information of the vehicle's movement.

[0026] In other words, through the above processing, the local three-dimensional point cloud distribution of the target roadside facilities and their surrounding environment can be recovered. Furthermore, based on the semantic perception and refined segmentation of the preset deep learning model, it is possible to accurately identify and extract different types of instance point sets such as road surface, guardrail, post, and road sign. Among them, the road surface point set and guardrail point set extracted simultaneously provide core spatial geometric support for the subsequent construction of a dynamic local reference frame and the elimination of vehicle driving vibration interference.

[0027] Step S13: Use the RANSAC algorithm to perform planar fitting on the target 3D point cloud to obtain the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference object, and determine the angle feature value corresponding to the target roadside facility based on the first normal vector and the second normal vector.

[0028] In this embodiment, the step of using the RANSAC algorithm to perform planar fitting on the target 3D point cloud to obtain a first normal vector corresponding to the target roadside facility and a second normal vector of the surrounding preset reference object, and determining the angular feature value corresponding to the target roadside facility based on the first normal vector and the second normal vector, includes: using the RANSAC algorithm to perform random sampling and iterative fitting on the facility's 3D point cloud to obtain a plane equation corresponding to the facility's 3D point cloud, and determining the normal vector corresponding to the plane equation as the first normal vector; using the RANSAC algorithm to perform random sampling and iterative fitting on the reference 3D point cloud to obtain a plane equation corresponding to the reference 3D point cloud, and determining the normal vector corresponding to the plane equation as the second normal vector; determining the dot product of the first normal vector and the second normal vector, and the product of the magnitude of the first normal vector and the magnitude of the second normal vector, and determining the inverse cosine of the ratio of the dot product to the product as the angle between the first normal vector and the second normal vector, so as to obtain the angular feature value corresponding to the target roadside facility. When performing decoupled geometric feature modeling and reference system correction, for marker surface modeling, the RANSAC algorithm can be used to fit the marker surface point set and extract the normal vector n. sign For establishing a dynamic reference frame, the set of road surface points around the sign can be fitted, and the road surface normal vector n can be extracted. road If the road surface area in the current frame is occluded or segmentation fails, the system automatically switches to the guardrail area point set in the same frame and extracts its normal vector as a substitute reference; then, it calculates the marker surface in a position n. road The angle features in the local coordinate system are taken as the reference. That is, after the local 3D point cloud reconstruction is completed, the RANSAC algorithm is used to perform plane fitting on the point set of the sign surface, and the independent angle features of the sign surface in the local coordinate system are calculated by strictly using the road surface (or guardrail) normal vector extracted in the same frame as the dynamic reference.

[0029] Among them, RANSAC (Random Sample Consensus) is a robust method for estimating mathematical model parameters from point cloud data containing outliers through random sampling and iterative fitting, and is often used in scenarios such as plane fitting and point cloud segmentation; Dynamic Local Reference Frame refers to a local coordinate system established based on the road surface normal vector detected in a single acquisition frame, used for software compensation of bumps and tilts during vehicle driving; Scene Anchor refers to a set of reference points in the local reconstruction coordinate system, using geometrically stable and semantically stable scene elements such as road surface and guardrail as registration references.

[0030] Furthermore, in this embodiment, the angle features, along with the 3D point cloud snapshot, GPS coordinates, and timestamp, can be stored in the historical asset database for subsequent cross-time comparison and retrieval.

[0031] In practical operation, taking the target roadside facility as a sign as an example, this embodiment determines the angular feature value through decoupled geometric feature modeling and reference system correction: on the one hand, sign surface modeling is performed, that is, the RANSAC algorithm is used to fit the sign surface point set (i.e., the three-dimensional point cloud of the facility), and the sign surface normal vector n is extracted. sign That is, the first normal vector; on the other hand, a dynamic reference system is established, that is, the set of road surface points around the sign (i.e., the reference three-dimensional point cloud) is fitted, and the road surface normal vector n is extracted. road That is, the second normal vector, and calculate the mark surface with the road surface normal vector n. road The angle features in the local coordinate system are used as the reference to obtain the angle feature values. That is, in this embodiment, a dynamic local reference system is established based on the road surface normal vector detected in a single acquisition frame to compensate for bumps and tilts during vehicle movement in a software manner. It should be noted that if the road surface area in the current frame is occluded or segmentation fails, the system automatically switches to the guardrail area point set in the same frame and extracts the normal vector of the guardrail area point set as a substitute reference. That is, the normal vector corresponding to the guardrail area point set is determined as the second normal vector to ensure the stable establishment of the dynamic local reference system.

[0032] Step S14: Determine all historical angle feature observations corresponding to the target roadside facility from the historical asset database, and determine the angle health baseline corresponding to the target roadside facility based on all historical angle feature observations.

[0033] In this embodiment, as Figure 3As shown, the step of determining all historical angle feature observations corresponding to the target roadside facility from the historical asset database, and determining the angle health baseline corresponding to the target roadside facility based on all historical angle feature observations, includes: retrieving multiple candidate historical physical instances from the historical asset database whose distance to the geographic coordinates corresponding to the target roadside facility is less than a preset radius; the geographic coordinates are determined based on the spatial offset between the target 3D point cloud corresponding to the target roadside facility and the positioning point in the positioning trajectory data; from the multiple candidate historical physical instances, selecting candidate historical physical instances whose corresponding category labels are the same as those corresponding to the target roadside facility; among the selected candidate historical physical instances, determining those whose difference between their corresponding geographic coordinates and the geographic coordinates corresponding to the target roadside facility is less than a preset threshold as the same historical physical instance corresponding to the target roadside facility; extracting all historical angle feature observations corresponding to the same historical physical instance from the historical asset database; determining the historical mean and historical standard deviation of all historical angle feature observations, and determining the angle health baseline corresponding to the target roadside facility based on the historical mean and historical standard deviation. The resulting health baseline diagram is shown below. Figure 4 As shown.

[0034] Before constructing the aforementioned angle health baseline, cross-batch instance association is required. This involves establishing a mapping of the same identifier instance ID across batches to confirm that the target roadside facility detected in the current inspection is the same physical instance as the roadside facility in the historical records. Specifically, based on the GPS coordinates (i.e., the geographic coordinates) of the current target roadside facility, a set of candidate records within a radius R (i.e., the preset radius) in the historical asset database is retrieved. These are the multiple candidate historical physical instances. Candidate records with the same category label as the current target roadside facility are then selected. For candidate records selected by category, if the difference between their GPS coordinates and current coordinates is less than a threshold τ (i.e., the preset threshold), they are determined to be the same physical instance.

[0035] After confirming that they are the same historical physical instance, extract the historical 3D point cloud snapshot and historical angle feature records of the instance, that is, all historical angle feature observations, and enter the subsequent health baseline construction and anomaly detection process; if they are confirmed to be different instances, the current target roadside facility is directly stored as a new asset in the historical asset database.

[0036] Specifically, when constructing the aforementioned angle health baseline, the inspection observations under normal conditions from N (N≥3) times in the historical asset database are aggregated to construct the health feature distribution set {θ1,θ2,…,θ} for this instance. N}, calculate its historical mean θ base(i.e., the historical mean) and historical standard deviation σ base (i.e., the historical standard deviation); wherein, the N observations can come from different time frames of the same inspection batch, or from different inspection batches. That is, this embodiment constructs an angle health baseline based on multi-temporal observation data; the multi-temporal refers to multiple batches of observation data obtained for roadside facilities at the same geographical location at different inspection cycles and different collection time points, used to analyze the tilt angle evolution trend of the facility over time. It should be noted that the angle health baseline mentioned in this embodiment is the historical health baseline, which is based on the statistical distribution (mean and standard deviation) of the geometric feature values ​​of the roadside facility extracted from multiple historical inspections, representing the reference benchmark of the facility under normal service conditions; since the angle health baseline objectively characterizes the true three-dimensional attitude distribution of the facility under normal service conditions (including initial installation deviation), it can transform the threshold determination of the absolute tilt angle into a comparison of historical deviations.

[0037] Step S15: Based on the angle health baseline, perform anomaly diagnosis on the angle feature value to obtain the tilt anomaly diagnosis result of the target roadside facility.

[0038] In this embodiment, the step of obtaining the tilt anomaly diagnosis result of the target roadside facility by performing anomaly diagnosis on the angle feature value based on the angle health baseline includes: performing a significance test on the angle feature value and the angle health baseline to obtain a significance level, and calculating the deviation between the angle feature value and the historical mean; if the significance level meets a preset significance threshold, and the deviation exceeds a preset physical threshold, then the tilt anomaly diagnosis result is determined to be a tilt anomaly warning; if the significance level does not meet the preset significance threshold, or the deviation does not exceed the preset physical threshold, then the tilt anomaly diagnosis result is determined to be a normal state.

[0039] It should be noted that significance testing refers to using statistical inference methods (such as Welch t-test) to determine whether the current measurement deviation belongs to normal random error or a structural anomaly with physical significance. In this application, the set of measured values ​​of the current inspection batch is compared with the historical distribution corresponding to the angle health baseline, and the significance of the deviation is calculated using statistical inference methods, preferably a two-sample significance test, including but not limited to Welch t-test; correspondingly, the preset significance threshold can be, for example, p<0.05, that is, only when the current measurement deviation reaches statistical significance relative to the historical baseline (e.g., p<0.05), and the measured angle deviation (i.e., the deviation amount) exceeds the preset physical threshold, is a tilt anomaly warning output.

[0040] Considering the inherent uncertainty and random fluctuations in depth measurement during monocular vision reconstruction, this embodiment introduces the aforementioned statistical inference methods to offset the random fluctuations in monocular reconstruction using the multi-frame cumulative effect. For example, as... Figure 5 The historical health baseline distribution (T0, healthy state) is compared with the current batch inspection measured characteristic distribution (T1, abnormal state). The Welch t-test is used to calculate the significance of the deviation between the two sets of observations. If the calculated t=46.27 and P<0.001, it indicates a highly statistically significant difference between the two sets of data. Simultaneously, the measured angle deviation exceeds a preset physical threshold. Combining the statistical significance and physical deviation threshold, the interference of random noise can be effectively filtered out, resulting in a highly robust tilt anomaly warning. Correspondingly, if the fluctuation of the current measurement deviation relative to the historical baseline is within the normal range, the tilt anomaly diagnosis result is determined to be either a normal state or an initial installation deviation. Furthermore, after completing this anomaly diagnosis, the characteristic data of this measurement can be updated to the historical asset database for comparison and retrieval in subsequent inspection cycles. Thus, through road surface reference system self-calibration and historical baseline statistical verification, accurate quantification of the deformation evolution process of roadside facilities can be achieved without expensive equipment such as lidar, improving the economy and reliability of smart maintenance.

[0041] As can be seen, in this embodiment, as Figure 6As shown, the process involves acquiring video frame sequences and positioning trajectory data collected by the target inspection vehicle during its operation, and processing the video frame sequences using a preset deep learning model to obtain a set of target pixel regions corresponding to the target roadside facility and a set of reference pixel regions corresponding to preset reference objects surrounding the target roadside facility. Based on the positioning trajectory data, the two-dimensional pixels in the target pixel region set and the reference pixel region set are projected into three-dimensional space to obtain a target three-dimensional point cloud. The RANSAC algorithm is used to perform planar fitting on the target three-dimensional point cloud to obtain a first normal vector corresponding to the target roadside facility and a second normal vector of the preset reference objects surrounding the target roadside facility. Based on the first normal vector and the second normal vector, the angle feature value corresponding to the target roadside facility is determined. All historical angle feature observation values ​​corresponding to the target roadside facility are determined from the historical asset database, and an angle health baseline corresponding to the target roadside facility is determined based on all historical angle feature observation values. Based on the angle health baseline, anomaly diagnosis is performed on the angle feature values ​​to obtain the tilt anomaly diagnosis result of the target roadside facility. That is, by projecting the two-dimensional pixels in the target pixel region set and the reference pixel region set into three-dimensional space using positioning trajectory data, a target three-dimensional point cloud is obtained. The spatial position constraints provided by the positioning trajectory data compensate for the lack of three-dimensional geometric information in two-dimensional images, achieving quantitative recovery of the three-dimensional spatial attitude of roadside facilities and overcoming the limitation of two-dimensional surface detection in quantifying three-dimensional structural deformation. Using the RANSAC algorithm to perform planar fitting on the target three-dimensional point cloud, noise and outliers in the point cloud can be effectively eliminated through random sampling and iterative fitting. This allows for accurate and robust extraction of the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference objects. Furthermore, since the angular feature value is determined based on the first and second normal vectors, that is, the surrounding preset reference objects in the same scene as the target roadside facility and observed synchronously are used as reference references to measure the facility's attitude, rather than using the camera coordinate system or the absolute horizontal plane as a fixed reference. This approach eliminates the impact of vehicle attitude interference and initial installation deviations on diagnostic results without relying on expensive 3D sensing hardware, significantly reducing false alarm rates and improving the accuracy, robustness, and economy of roadside facility tilt diagnosis.

[0042] refer to Figure 7 The present application also discloses a roadside facility tilt diagnostic device, comprising: The data acquisition module 11 is used to acquire video frame sequences and positioning trajectory data collected by the target inspection vehicle during its driving process, and to process the video frame sequences using a preset deep learning model to obtain the target pixel region set corresponding to the target roadside facility and the reference pixel region set corresponding to the preset reference objects around the target roadside facility. The three-dimensional mapping module 12 is used to project two-dimensional pixels in the target pixel region set and the reference pixel region set into three-dimensional space based on the positioning trajectory data to obtain a target three-dimensional point cloud. The feature value determination module 13 is used to perform plane fitting on the target three-dimensional point cloud using the RANSAC algorithm to obtain the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference object, and to determine the angle feature value corresponding to the target roadside facility based on the first normal vector and the second normal vector. The baseline determination module 14 is used to determine all historical angle feature observations corresponding to the target roadside facility from the historical asset database, and to determine the angle health baseline corresponding to the target roadside facility based on all historical angle feature observations. The anomaly diagnosis module 15 is used to perform anomaly diagnosis on the angle feature value based on the angle health baseline to obtain the tilt anomaly diagnosis result of the target roadside facility.

[0043] As can be seen, in this embodiment, the two-dimensional pixels in the target pixel region set and the reference pixel region set are projected into three-dimensional space using positioning trajectory data to obtain the target three-dimensional point cloud. The spatial position constraints provided by the positioning trajectory data compensate for the lack of three-dimensional geometric information in two-dimensional images, achieving quantitative recovery of the three-dimensional spatial posture of roadside facilities and overcoming the defect that two-dimensional surface detection cannot quantify three-dimensional structural deformation. Using the RANSAC algorithm to perform planar fitting on the target three-dimensional point cloud, noise points and outliers in the point cloud can be effectively removed through random sampling and iterative fitting. This allows for accurate and robust extraction of the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference objects. Furthermore, since the angular feature value is determined based on the first and second normal vectors, that is, the surrounding preset reference objects in the same scene as the target roadside facility and observed synchronously are used as reference references to measure the facility posture, rather than using the camera coordinate system or the absolute horizontal plane as a fixed reference. This approach eliminates the impact of vehicle attitude interference and initial installation deviations on diagnostic results without relying on expensive 3D sensing hardware, significantly reducing false alarm rates and improving the accuracy, robustness, and economy of roadside facility tilt diagnosis.

[0044] In some specific embodiments, the data acquisition module 11 may specifically include: The data acquisition unit is used to control the target inspection vehicle to acquire video streams during its journey through a monocular camera to obtain video frame sequences, and to obtain positioning trajectory data by using the positioning data corresponding to each video frame acquired through the positioning receiver.

[0045] In some specific embodiments, the data acquisition module 11 may specifically include: The data processing unit is used to perform semantic segmentation on each video frame in the video frame sequence using a preset deep learning model, so as to obtain the target pixel region set and category label corresponding to the target roadside facility in each video frame, and the reference pixel region set corresponding to the surrounding preset reference objects of the target roadside facility; the surrounding preset reference objects include the surrounding road surface and / or the surrounding guardrail.

[0046] In some specific embodiments, the three-dimensional mapping module 12 can be used to project two-dimensional pixels in the target pixel region set and the reference pixel region set into three-dimensional space based on the multi-view geometric consistency method and by utilizing the pose constraints of the target inspection vehicle in the positioning trajectory data, so as to obtain the facility three-dimensional point cloud corresponding to the target roadside facility and the reference three-dimensional point cloud corresponding to the surrounding preset reference object.

[0047] In some specific embodiments, the feature value determination module 13 may specifically include: The first normal vector determination unit is used to randomly sample and iteratively fit the three-dimensional point cloud of the facility using the RANSAC algorithm to obtain the plane equation corresponding to the three-dimensional point cloud of the facility, and determine the normal vector corresponding to the plane equation as the first normal vector. The second normal vector determination unit is used to perform random sampling and iterative fitting on the reference 3D point cloud using the RANSAC algorithm to obtain the plane equation corresponding to the reference 3D point cloud, and to determine the normal vector corresponding to the plane equation as the second normal vector. The feature value determination unit is used to determine the dot product of the first normal vector and the second normal vector, and the product of the magnitude of the first normal vector and the magnitude of the second normal vector, and to determine the inverse cosine of the ratio of the dot product to the product as the angle between the first normal vector and the second normal vector, so as to obtain the angle feature value corresponding to the target roadside facility.

[0048] In some specific embodiments, the baseline determination module 14 may specifically include: The data retrieval unit is used to retrieve, in the historical asset database, multiple candidate historical physical instances whose distance from the geographic coordinates corresponding to the target roadside facility is less than a preset radius; the geographic coordinates are determined based on the spatial offset between the target 3D point cloud corresponding to the target roadside facility and the positioning point in the positioning trajectory data. The instance selection unit is used to filter out candidate historical physical instances from the plurality of candidate historical physical instances whose corresponding category labels are the same as those of the target roadside facility. The instance determination unit is used to determine, among the selected candidate historical physical instances, the candidate historical physical instances whose difference between their corresponding geographic coordinates and the geographic coordinates corresponding to the target roadside facility is less than a preset threshold, as the same historical physical instance corresponding to the target roadside facility. An observation extraction unit is used to extract all historical angular feature observations corresponding to the same historical physical instance from the historical asset database. The baseline determination unit is used to determine the historical mean and historical standard deviation of all historical angle feature observations, and to determine the angle health baseline corresponding to the target roadside facility based on the historical mean and historical standard deviation.

[0049] In some specific embodiments, the anomaly diagnosis module 15 may specifically include: An abnormality diagnosis unit is used to perform a significance test on the angle feature value and the angle health baseline to obtain the significance level, and to calculate the deviation between the angle feature value and the historical mean. The first diagnostic result early warning unit is used to determine the tilt anomaly diagnostic result as a tilt anomaly early warning if the significance level meets a preset significance threshold and the deviation exceeds a preset physical threshold. The second diagnostic result early warning unit is used to determine that the tilt anomaly diagnostic result is normal if the significance level does not meet the preset significance threshold or the deviation amount does not exceed the preset physical threshold.

[0050] Furthermore, embodiments of this application also disclose an electronic device, Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0051] Figure 8 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the roadside facility tilt diagnosis method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0052] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0053] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0054] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the roadside facility tilt diagnosis method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0055] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned roadside facility tilt diagnosis method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0056] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0057] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0058] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0059] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0060] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for diagnosing the tilt of roadside facilities, characterized in that, include: The video frame sequence and positioning trajectory data collected by the target inspection vehicle during its driving process are acquired, and the video frame sequence is processed using a preset deep learning model to obtain the target pixel region set corresponding to the target roadside facility and the reference pixel region set corresponding to the preset reference objects around the target roadside facility. Based on the positioning trajectory data, the two-dimensional pixels in the target pixel region set and the reference pixel region set are projected into three-dimensional space to obtain the target three-dimensional point cloud; The RANSAC algorithm is used to perform planar fitting on the target three-dimensional point cloud to obtain the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference objects, and the angular feature value corresponding to the target roadside facility is determined based on the first normal vector and the second normal vector. All historical angle feature observations corresponding to the target roadside facility are determined from the historical asset database, and the angle health baseline corresponding to the target roadside facility is determined based on all historical angle feature observations. Based on the angle health baseline, anomaly diagnosis of the angle feature value is performed to obtain the tilt anomaly diagnosis result of the target roadside facility.

2. The method for diagnosing the tilt of roadside facilities according to claim 1, characterized in that, The acquisition of video frame sequences and positioning trajectory data collected by the target inspection vehicle during its operation includes: The control target inspection vehicle collects video streams during its journey using a monocular camera to obtain a video frame sequence, and uses the positioning data corresponding to each video frame collected by the positioning receiver to obtain positioning trajectory data.

3. The method for diagnosing the tilt of roadside facilities according to claim 1, characterized in that, The step of processing the video frame sequence using a preset deep learning model to obtain a set of target pixel regions corresponding to the target roadside facility and a set of reference pixel regions corresponding to preset reference objects surrounding the target roadside facility includes: A preset deep learning model is used to perform semantic segmentation on each video frame in the video frame sequence to obtain the target pixel region set and category label corresponding to the target roadside facility in each video frame, as well as the reference pixel region set corresponding to the surrounding preset reference objects of the target roadside facility; the surrounding preset reference objects include the surrounding road surface and / or the surrounding guardrail.

4. The method for diagnosing the tilt of roadside facilities according to claim 1, characterized in that, The step of projecting two-dimensional pixels from the target pixel region set and the reference pixel region set into three-dimensional space based on the positioning trajectory data to obtain a target three-dimensional point cloud includes: Based on the multi-view geometric consistency method, and utilizing the pose constraints of the target inspection vehicle in the positioning trajectory data, the two-dimensional pixels in the target pixel region set and the reference pixel region set are projected into three-dimensional space to obtain the three-dimensional point cloud of the target roadside facility and the reference three-dimensional point cloud of the surrounding preset reference object.

5. The method for diagnosing the tilt of roadside facilities according to claim 4, characterized in that, The step of using the RANSAC algorithm to perform planar fitting on the target 3D point cloud to obtain the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference objects, and determining the angular feature value corresponding to the target roadside facility based on the first normal vector and the second normal vector, includes: The RANSAC algorithm is used to randomly sample and iteratively fit the three-dimensional point cloud of the facility to obtain the plane equation corresponding to the three-dimensional point cloud of the facility, and the normal vector corresponding to the plane equation is determined as the first normal vector. The RANSAC algorithm is used to randomly sample and iteratively fit the reference 3D point cloud to obtain the plane equation corresponding to the reference 3D point cloud, and the normal vector corresponding to the plane equation is determined as the second normal vector. The dot product of the first normal vector and the second normal vector, and the product of the magnitude of the first normal vector and the magnitude of the second normal vector are determined. The inverse cosine of the ratio of the dot product to the product is determined as the angle between the first normal vector and the second normal vector, so as to obtain the angular feature value corresponding to the target roadside facility.

6. The method for diagnosing the tilt of roadside facilities according to any one of claims 1 to 5, characterized in that, The step of determining all historical angular feature observations corresponding to the target roadside facility from the historical asset database, and determining the angular health baseline corresponding to the target roadside facility based on all historical angular feature observations, includes: In the historical asset database, multiple candidate historical physical instances whose distance to the geographic coordinates corresponding to the target roadside facility is less than a preset radius are retrieved; the geographic coordinates are determined based on the spatial offset between the target 3D point cloud corresponding to the target roadside facility and the positioning point in the positioning trajectory data. From the plurality of candidate historical physical instances, select the candidate historical physical instances whose corresponding category labels are the same as those of the target roadside facility; Among the selected candidate historical physical instances, those whose corresponding geographic coordinates are less than the difference between the geographic coordinates of the target roadside facility and the geographic coordinates of the target roadside facility are determined to be the same historical physical instance as the target roadside facility. Extract all historical angular feature observations corresponding to the same historical physical instance from the historical asset database; Determine the historical mean and historical standard deviation of all historical angle feature observations, and determine the angle health baseline corresponding to the target roadside facility based on the historical mean and historical standard deviation.

7. The method for diagnosing the tilt of roadside facilities according to claim 6, characterized in that, The method of obtaining the tilt anomaly diagnosis result of the target roadside facility by performing anomaly diagnosis on the angle feature value based on the angle health baseline includes: A significance test is performed on the angular feature value and the angular health baseline to obtain the significance level, and the deviation between the angular feature value and the historical mean is calculated; If the significance level meets the preset significance threshold, and the deviation exceeds the preset physical threshold, then the tilt anomaly diagnosis result is determined to be a tilt anomaly warning. If the significance level does not meet the preset significance threshold, or the deviation does not exceed the preset physical threshold, then the tilt anomaly diagnosis result is determined to be normal.

8. A roadside facility tilt diagnostic device, characterized in that, include: The data acquisition module is used to acquire video frame sequences and positioning trajectory data collected by the target inspection vehicle during its driving process, and to process the video frame sequences using a preset deep learning model to obtain the target pixel region set corresponding to the target roadside facility and the reference pixel region set corresponding to the preset reference objects around the target roadside facility. The three-dimensional mapping module is used to project two-dimensional pixels in the target pixel region set and the reference pixel region set into three-dimensional space based on the positioning trajectory data to obtain a target three-dimensional point cloud. The feature value determination module is used to perform plane fitting on the target three-dimensional point cloud using the RANSAC algorithm to obtain the first normal vector corresponding to the target roadside facility and the second normal vector of the surrounding preset reference object, and to determine the angle feature value corresponding to the target roadside facility based on the first normal vector and the second normal vector. The baseline determination module is used to determine all historical angle feature observations corresponding to the target roadside facility from the historical asset database, and to determine the angle health baseline corresponding to the target roadside facility based on all historical angle feature observations. An anomaly diagnosis module is used to perform anomaly diagnosis on the angle feature value based on the angle health baseline to obtain the tilt anomaly diagnosis result of the target roadside facility.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the roadside facility tilt diagnosis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the roadside facility tilt diagnosis method as described in any one of claims 1 to 7.

Images