A highway corrugated plate integrity detection method based on a vehicle-mounted three-dimensional camera

Abstract

The application discloses a kind of highway wave plate integrity detection methods based on vehicle-mounted three-dimensional camera, it is related to traffic infrastructure wisdom management and maintenance technical field, the method includes: using camera internal geometric parameter and multi-line light plane calibration generation system configuration file;Real-time image sequence acquisition, extract light strip center pixel coordinates;Based on system configuration file, pixel coordinates are back projected to solve three-dimensional intersection generation wave plate three-dimensional point cloud data;Three-dimensional point cloud data is extracted to obtain segmented wave plate three-dimensional point cloud data, fitting reference surface calculates the height deviation value of point to plane, according to multilayer multi-interval sampling and form feature string, bit by bit comparison identifies defect type and degree, combined with three-dimensional curvature and image sequence cross validation, output detection result data and trigger alarm, based on historical data analysis wave plate aging trend.The application can efficiently obtain multilevel three-dimensional geometric features of wave plate surface, realize the automatic identification and positioning of defect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent management and maintenance of traffic infrastructure, and in particular to a highway corrugated plate integrity detection method based on a vehicle-mounted three-dimensional camera. BACKGROUND

[0002] With the rapid development of highway construction and transportation, highway corrugated plates, as important road safety protection facilities, are widely used, and their integrity is directly related to vehicle collision prevention and driving safety. At present, the detection of highway corrugated plates mainly relies on manual inspection or two-dimensional image recording. The manual method is low in efficiency and strong in subjectivity, and it is difficult to find potential structural defects in a timely manner. Some automatic detection systems using two-dimensional images or laser ranging devices can only obtain the surface profile or height profile data of the corrugated plate, and cannot fully grasp the three-dimensional shape information of the corrugated plate, making it difficult to accurately quantify complex defects such as local bending, bulging and sagging.

[0003] The existing highway corrugated plate detection methods generally have the problems of low detection data dimension and inability to accurately reflect the three-dimensional geometric characteristics of the corrugated plate, resulting in limited defect recognition capability, especially when detecting the deformation of the corrugated plate along the length direction and the vertical height, it is difficult to achieve accurate integrity analysis. At the same time, the existing technology lacks means to encode and compare the detected deformation with the standard model, cannot identify the defect type and severity of the corrugated plate based on complete feature fingerprints, and also cannot realize the aging trend analysis of the corrugated plate combined with historical data.

[0004] In view of the above problems, the present application provides a highway corrugated plate integrity detection method based on a vehicle-mounted three-dimensional camera, which constructs a vehicle-mounted three-dimensional measurement system by using multi-line laser and a moving camera, combines high-precision cameras and multi-line laser calibration, frame-by-frame three-dimensional point cloud reconstruction, geometric feature encoding and comparison analysis with standard models, realizes efficient acquisition of three-dimensional shape of the corrugated plate surface and automatic recognition of defects, and can accurately determine the integrity state of the corrugated plate and support highway maintenance and maintenance decision-making. SUMMARY

[0005] The present application provides a highway corrugated plate integrity detection method based on a vehicle-mounted three-dimensional camera to solve the problems of insufficient three-dimensional geometric shape acquisition accuracy and weak defect automatic recognition capability in the prior art.

[0006] To solve the above technical problems, the present application provides the following technical scheme: a highway corrugated plate integrity detection method based on a vehicle-mounted three-dimensional camera, comprising the following steps:

[0007] Step S1, for the calibration image sequence and the laser image sequence, the internal and external parameters of the moving camera are solved and the least square plane fitting is adopted to obtain the camera calibration matrix and the multi-light plane equation and write them into a complete system configuration file;

[0008] Step S2, collect the image sequence during the vehicle driving process, and extract the light bar center pixel coordinates in the image sequence through the gray center algorithm. The light bar center pixel coordinates are back projected and the three-dimensional intersection of the light ray and the light plane is solved by using the complete system configuration file to obtain the three-dimensional point cloud data of the corrugated plate surface;

[0009] Step S3, performing coordinate transformation and splicing, denoising and downsampling, preliminary segmentation based on height and position, accurate segmentation and segment division according to geometric characteristics on the three-dimensional point cloud data of the corrugated plate surface to obtain segmented corrugated plate three-dimensional point cloud data;

[0010] Step S4, using least square plane fitting to obtain the reference surface from the segmented corrugated plate three-dimensional point cloud data, calculating the height deviation value of each point to the reference surface, sampling according to multiple layers and multiple intervals and grading coding according to the preset threshold to obtain the corrugated plate elevation feature coding sequence, and establishing a standard reference model and a defect coding library;

[0011] Step S5, comparing the corrugated plate elevation feature coding sequence with the standard reference model bit by bit, identifying defects according to the defect judgment rule, and cross verifying with three-dimensional curvature and image sequence to obtain the corrugated plate defect identification result;

[0012] Step S6, combining the corrugated plate defect identification result with the positioning identification information, outputting the detection result data and storing, triggering the sound and light alarm in real time according to the major defect judgment threshold, comparing and analyzing the historical detection data to obtain the corrugated plate aging degree evaluation data.

[0013] Further, in step S1, the following sub-steps are further included:

[0014] S1-1, taking a planar calibration board with a calibration pattern at different angles by moving the camera to obtain a plurality of image sequences under different poses, the calibration pattern including a checkerboard pattern and a dot array pattern, extracting the calibration pattern corner point coordinates from the obtained plurality of image sequences under different poses using Zhang Zhengyou plane calibration algorithm, and obtaining the intrinsic matrix, distortion coefficient and extrinsic transformation matrix of the moving camera by solving, saving the obtained intrinsic matrix, distortion coefficient and extrinsic transformation matrix of the moving camera as a camera calibration file;

[0015] S1-2, using the camera calibration file to drive the multi-line laser to project multiple laser lines on the calibration whiteboard to form multiple bright light bar images; moving the calibration whiteboard to different spatial distance positions, and collecting multiple light bar images containing the projected multiple laser lines at each different spatial distance position to form an image sequence, and calculating the complete spatial plane equation of the laser;

[0016] S1-3, write the camera calibration file and the complete spatial plane equation of the laser into a complete system configuration file, and fix the mobile camera and the laser on the mounting position on the detection vehicle.

[0017] Further, in S1-2, the complete spatial plane equation of the laser is calculated as follows:

[0018] The light strip center pixel coordinates corresponding to the projected multiple lasers are extracted in the image sequence by the gray center of gravity algorithm, and the extracted light strip center pixel coordinates are back projected as spatial light rays emitted through the optical center of the mobile camera. Then, the spatial light rays and the plane equation of the current calibration whiteboard in the world coordinate system are associated to calculate the three-dimensional intersection coordinates of the light strip center pixel at the position of the current calibration whiteboard. All the three-dimensional intersection coordinates of the same laser at different positions of the calibration whiteboard are collected, and the least square plane fitting method is used to calculate the complete spatial plane equation of the laser.

[0019] Further, in step S2, the following sub-steps are further included:

[0020] S2-1, during the driving of the vehicle along the road, the multi-line laser is driven to project multiple lasers on the surface of the road waveform board to form multiple bright light strip images, and the mobile camera synchronously collects the light strip images containing the waveform board and the projected multiple lasers to obtain an image sequence collected during the driving of the vehicle;

[0021] S2-2, for each image in the image sequence collected during the driving of the vehicle, grayscale and noise filtering processing are performed, the gray center of gravity algorithm is applied to extract the center pixel coordinates of the projected multiple lasers in the limited region of interest to obtain the light strip center pixel coordinates corresponding to each image;

[0022] S2-3, the camera calibration file and the complete spatial plane equation of the laser are used to back project the light strip center pixel coordinates to solve the three-dimensional intersection coordinates of the spatial light rays emitted through the optical center of the mobile camera and the corresponding laser light plane. The image sequence collected is continuously processed to generate the three-dimensional point cloud data of the surface of the waveform board corresponding to the vehicle driving mileage coordinates.

[0023] Further, in step S3, the following sub-steps are further included:

[0024] S3-1, according to the vehicle motion pose information, the three-dimensional point cloud data of the surface of the waveform board in each frame in the mobile camera coordinate system is converted to the unified world coordinate system, and the point cloud registration algorithm is used to register and splice the three-dimensional point cloud data of the adjacent frames of the surface of the waveform board to obtain the complete and aligned three-dimensional point cloud model of the waveform board in the world coordinate system;

[0025] S3-2, for the corrugated plate three-dimensional point cloud model, a statistical filtering method is used to remove isolated outliers in the point cloud, the point cloud represents the data points in the three-dimensional point cloud model, and the point cloud is down-sampled by using voxel grid filtering to reduce the data amount while retaining the main geometric structure features of the corrugated plate, and the denoised and down-sampled corrugated plate three-dimensional point cloud data is obtained;

[0026] S3-3, for the denoised and down-sampled corrugated plate three-dimensional point cloud data, based on the prior height threshold, the points lower than the bottom edge of the corrugated plate and higher than the top of the corrugated plate are removed, and the strip-shaped point cloud close to the road edge and continuously distributed along the highway direction is extracted by horizontal position screening, and the preliminary segmented corrugated plate point cloud region is obtained;

[0027] S3-4, in the corrugated plate point cloud region, the cross-sectional profile features are calculated along the vertical direction, the point cloud with typical cross-sectional shape of "double wave" and "three wave" is identified by using the wave-shaped geometric features of the corrugated plate, and the point cloud with the wave-shaped geometric features of the corrugated plate is clustered and extracted, and the pure corrugated plate surface three-dimensional point cloud data is obtained;

[0028] S3-5, according to the physical segment length of the corrugated plate, the pure corrugated plate surface three-dimensional point cloud data is segmented into several segments along the highway direction, and the segmented corrugated plate three-dimensional point cloud data is obtained.

[0029] Further, in step S4, the following sub-steps are further included:

[0030] S4-1, selecting the flat and complete region point data in each segment of the segmented corrugated plate three-dimensional point cloud data, using the least square plane fitting method to obtain the best fitting plane, and obtaining the plane equation representing the segmented corrugated plate reference surface;

[0031] S4-2, substituting the three-dimensional coordinates of each point in each segment of the point cloud into the distance formula from the point to the plane, calculating the normal distance value of each point relative to the reference surface, and obtaining the height deviation value of all points of the segmented corrugated plate to the reference surface;

[0032] S4-3, based on the height deviation value of all points of the segmented corrugated plate to the reference surface, the segmented corrugated plate three-dimensional point cloud data is divided into multiple height layer subsets according to the number of multi-line laser scanning along the vertical direction, and each height layer subset is divided into several equal intervals along the length direction of the corrugated plate, and the representative deviation value is selected from each interval to form a multi-layer and multi-interval sampling data matrix;

[0033] S4-4, for the multi-layer and multi-interval sampling data matrix, according to the preset four deviation level thresholds, each sampling deviation value is encoded according to the level, and a multi-bit elevation coding sequence corresponding to the segmented corrugated plate is obtained;

[0034] S4-5, for the intact waveform board sample selected before detection, the same fitting, calculation, sampling and encoding process of S4-1 to S4-4 is executed to obtain the waveform board standard elevation encoding sequence, and the same process is executed for the waveform board samples with different typical defects to obtain the encoding mode of each type of defect, and finally the standard reference model and the defect encoding library are established.

[0035] Further, in step S5, the following sub-steps are further included:

[0036] S5-1, the segmented waveform board elevation feature encoding sequence is compared with the corresponding waveform board standard elevation encoding sequence in the established standard reference model bit by bit to obtain the comparison result data of whether each encoding bit is consistent;

[0037] S5-2, according to the comparison result data, the missing defect of fracture is identified by detecting the continuous missing encoding mode according to the preset defect judgment rule, then the bending deformation defect is identified by detecting the proportion of high-level deviation points in the region exceeding the threshold, and the defect category and the occurrence position are further determined according to the specific position section of the abnormal encoding, and the defect type and position analysis result is generated, the abnormal encoding corresponds to the encoding characteristics of the missing defect of fracture and the bending deformation defect;

[0038] S5-3, based on the defect type and position analysis result, combined with three-dimensional curvature and image sequence, whether there is visible texture in the three-dimensional curvature abnormal area of the original image of the moving camera is checked to complete the cross verification of the defect type, and the waveform board defect discrimination result confirmed by three-dimensional-two-dimensional multi-data is obtained, the three-dimensional curvature represents the spatial curvature variation characteristics of each measuring point estimated by the local neighborhood surface fitting of the segmented waveform board three-dimensional point cloud data.

[0039] Further, in S5-3, the acquisition process of three-dimensional curvature is as follows:

[0040] In the segmented waveform board three-dimensional point cloud data, a neighborhood point set with a fixed radius is constructed around each measuring point, and a local least square surface fitting is performed on each neighborhood point set to calculate the principal curvature, average curvature and Gaussian curvature at the measuring point; compare the above curvature values with the preset threshold to determine whether the measuring point position belongs to the curvature variation abnormal area.

[0041] Further, in step S6, the following sub-steps are further included:

[0042] S6-1, the waveform board defect discrimination result is combined with the unique positioning identification information allocated to each segment of the waveform board in the detection process to obtain the defect position, defect type and defect degree evaluation, and the defect position, defect type and defect degree evaluation are output as the detection result data, and finally the detection result data is stored in the database;

[0043] S6-2, based on the detection result data, whether there is a major defect is detected according to a preset major defect judgment threshold, when the major defect is detected, a defect alarm module is triggered immediately, so that the defect alarm module sends an audible and visual alarm prompt signal, and outputs alarm information to prompt the inspection personnel to take emergency measures;

[0044] S6-3, the detection result data and the segmented waveform board elevation feature code sequence are compared with the historical detection data stored in the database, the waveform board deformation trend with time is analyzed, and waveform board aging degree evaluation data is obtained.

[0045] Further, the defect alarm module comprises: an audible alarm unit, which plays a preset audio warning signal through a loudspeaker installed on the detection vehicle; a light alarm unit, which flashes a high-brightness LED lamp to prompt the detected major defect event; and a defect marking unit, which associates the waveform board segment number where the defect is located with the road segment post number, and presents the text and image forms in the user interface to guide artificial inspection positioning.

[0046] Compared with the prior art, the beneficial effects of the present application are:

[0047] The vehicle-mounted three-dimensional measurement system composed of multi-line laser and high-precision mobile camera is adopted, combined with the internal geometric parameters of the mobile camera and multi-line light plane calibration, the multi-level spatial point cloud data of the waveform board can be obtained in real time during the driving of the vehicle, the defects that the existing two-dimensional detection cannot fully reflect the three-dimensional shape of the waveform board are effectively overcome, and the high-precision reconstruction of the outer surface of the waveform board along the length and height directions is realized.

[0048] The elevation coding system based on the normal deviation value of the point to the reference plane is established, and the waveform board point cloud is sampled and coded in multiple layers and multiple intervals, the three-dimensional shape information of the waveform board surface is converted into a standardized multi-dimensional feature string, the digital expression of the waveform board deformation of different degrees is realized, the defect type and severity of the waveform board can be automatically distinguished by comparing with the standard intact waveform board coding sequence bit by bit, and the accuracy and consistency of defect identification are significantly improved.

[0049] The defect information detected is associated with the highway positioning mark and output, the precise positioning of the defect on the specific road section and the waveform board section is realized, and the deformation trend of the waveform board is analyzed by comparing with the historical detection data, which can assist in judging the aging degree and the remaining service life, and provide scientific maintenance decision support for the road management department.

[0050] The present application can automatically complete data acquisition, point cloud reconstruction, feature extraction and defect identification in the driving state of the vehicle, reduce the frequency of artificial inspection and subjective judgment error, discover the seriously damaged waveform board in time and trigger the alarm prompt, and improve the level of highway traffic safety guarantee. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and understand that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0052] Figure 1 is a method flowchart of the present application. DETAILED DESCRIPTION

[0053] In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only for selected embodiments of the present application.

[0054] Please refer to Figure 1 , Figure 1 is a highway wave plate integrity detection method flowchart based on a vehicle-mounted three-dimensional camera provided by the embodiments of the present application, including the following steps:

[0055] Step S1, for the calibration image sequence and the laser image sequence, respectively using mobile camera internal and external parameter solving and least square plane fitting, obtaining camera calibration matrix and multiple light plane equation and writing into the complete system configuration file.

[0056] S1-1, by mobile camera shooting the plane calibration board with calibration pattern at different angles, acquiring multiple groups of image sequences under different postures, the calibration pattern includes the checkerboard pattern and the dot array pattern, using Zhang Zhengyou plane calibration algorithm to extract the calibration pattern corner coordinates from the acquired multiple groups of image sequences under different postures, and obtaining the internal parameter matrix, distortion coefficient and external parameter transformation matrix of the mobile camera by solving; saving the internal parameter matrix, distortion coefficient and external parameter transformation matrix of the mobile camera obtained by solving as a camera calibration file;

[0057] S1-2, using the intrinsic matrix, distortion coefficients and extrinsic transformation matrix of the mobile camera contained in the camera calibration file, driving the multi-line laser to project multiple laser lines on the calibration whiteboard to form multiple bright light bar images; moving the calibration whiteboard to different spatial distance positions, and collecting multiple light bar images containing the projected multiple laser lines at each different spatial distance position respectively to form an image sequence;

[0058] In S1-2, the complete spatial plane equation of the laser is calculated as follows:

[0059] The light bar center pixel coordinates corresponding to the projected multiple laser lines are extracted on each image in the image sequence by the gray center of gravity algorithm, and the extracted light bar center pixel coordinates are back-projected into spatial rays emitted by the optical center of the mobile camera by using the intrinsic matrix, distortion coefficients and extrinsic transformation matrix of the mobile camera contained in the camera calibration file, and then the obtained spatial rays are associated with the plane equation of the current calibration whiteboard in the world coordinate system to calculate the three-dimensional intersection coordinates of the light bar center pixel corresponding to each collected image at the position of the current calibration whiteboard; then all the three-dimensional intersection coordinates obtained by repeated collection of the same laser at different positions of the calibration whiteboard are summarized, and the least square plane fitting method is used to calculate the complete spatial plane equation of the laser;

[0060] S1-3, the intrinsic matrix, distortion coefficients and extrinsic transformation matrix of the mobile camera contained in the camera calibration file are uniformly written into a complete system configuration file together with the complete spatial plane equation of the laser, and the mobile camera and the laser are fixed at the installation position on the detection vehicle to ensure that the system configuration file can be directly called to perform high-precision conversion of the pixel to three-dimensional coordinates on the collected light bar center pixel coordinates in the subsequent detection stage, without the need to perform the calibration process again.

[0061] Step S2, extracting the light bar center pixel coordinates from the image sequence collected during the vehicle driving process by the gray center of gravity algorithm, and using the complete system configuration file to back-project the light bar center pixel coordinates and solve the three-dimensional intersection of the light rays and the light plane to obtain the three-dimensional point cloud data of the road waveform plate surface with the vehicle driving mileage coordinates.

[0062] S2-1, during the driving of the vehicle along the road, driving the multi-line laser to project multiple laser lines on the surface of the road waveform plate to form multiple bright light bar images, and the mobile camera synchronously collects the light bar images containing the waveform plate and the projected multiple laser lines to obtain the image sequence collected during the driving of the vehicle;

[0063] S2-2, performing gray scale and noise filtering processing on each image in the image sequence collected during vehicle driving, applying a gray center algorithm to extract the center pixel coordinates of the projected multiple laser light in the limited region of interest, to obtain the light bar center pixel coordinates corresponding to each image;

[0064] S2-3, using the internal parameter matrix, distortion coefficient and external parameter transformation matrix of the moving camera contained in the camera calibration file and the complete spatial plane equation of the laser, performing back projection on the light bar center pixel coordinates, solving the three-dimensional intersection coordinates of the space light ray emitted through the camera optical center and the corresponding laser light plane, and continuously processing the collected image sequence to generate the three-dimensional point cloud data of the wave plate surface corresponding to the vehicle driving mileage coordinates.

[0065] Step S3, performing coordinate transformation and splicing, denoising and downsampling, preliminary segmentation based on height and position, accurate segmentation according to geometric features and segment division on the three-dimensional point cloud data of the wave plate surface, to obtain pure three-dimensional point cloud data of the wave plate in physical segments.

[0066] S3-1, converting the three-dimensional point cloud data of the wave plate surface in each frame under the camera coordinate system to the unified world coordinate system according to the vehicle motion pose information, using the pose data output by the vehicle positioning device or using point cloud registration algorithm to register and splice the adjacent frame three-dimensional point cloud data of the wave plate surface, to obtain a complete and aligned three-dimensional point cloud model of the wave plate in the world coordinate system;

[0067] S3-2, using statistical filtering method to remove isolated outlier noise points in the point cloud, using voxel grid filtering to downsample the point cloud, reducing the data volume while retaining the main geometric structure features of the wave plate, to obtain the denoised and downsampled three-dimensional point cloud data of the wave plate;

[0068] S3-3, based on the prior height threshold, removing the points below the bottom edge and above the top of the wave plate, extracting the strip-shaped point cloud close to the road edge and continuously distributed along the highway direction through horizontal position screening, to obtain the preliminary segmented wave plate point cloud region;

[0069] S3-4, calculating the cross-sectional profile features in the vertical direction in the wave plate point cloud region, using the wave-shaped geometric features of the wave plate, identifying the point cloud with typical cross-sectional shapes of "double wave" and "three wave" through surface fitting algorithm, and clustering and extracting the point cloud conforming to the wave-shaped geometric features of the wave plate, to obtain the pure three-dimensional point cloud data of the wave plate surface;

[0070] S3-5, according to the physical segment length of the corrugated sheet (such as the distance between two columns), the pure corrugated sheet surface three-dimensional point cloud data is segmented into several segments along the road direction, and segmented corrugated sheet three-dimensional point cloud data is obtained by dividing the corrugated sheet segment as a unit, which provides input data for subsequent feature extraction and defect analysis.

[0071] Step S4, for segmented corrugated sheet three-dimensional point cloud data, the least square plane fitting is used to calculate the reference surface, the height deviation value of each point to the reference surface is calculated, the multi-layer multi-interval sampling is performed and the grade coding is performed according to the preset threshold, the corrugated sheet elevation feature coding sequence is obtained, and the standard reference model and the defect coding library are established.

[0072] S4-1, selecting the flat and complete area point data in each segment of the segmented corrugated sheet three-dimensional point cloud data, the least square plane fitting method is used to obtain the best fitting plane, and the plane equation representing the reference surface of the segmented corrugated sheet is obtained;

[0073] S4-2, the three-dimensional coordinates of each point in each segment of the point cloud are substituted into the distance formula from the point to the plane, and the normal distance value of each point relative to the reference surface is calculated, and the height deviation value of all points of the segmented corrugated sheet to the reference surface is obtained;

[0074] S4-3, based on the height deviation value of all points of the segmented corrugated sheet to the reference surface, the segmented corrugated sheet three-dimensional point cloud data is divided into multiple height layer subsets along the vertical direction according to the number of multi-line laser scanning, and each height layer subset is divided into several equidistant intervals along the length direction of the corrugated sheet, and the representative deviation value is selected from each interval to form a multi-layer multi-interval sampling data matrix;

[0075] S4-4, for the multi-layer multi-interval sampling data matrix, according to the preset four deviation grade thresholds, each sampling deviation value is graded and coded, and is marked as one of A, B, C and D four grades, and a multi-bit elevation coding sequence corresponding to the segmented corrugated sheet is obtained;

[0076] S4-5, for the intact corrugated sheet sample selected before detection, the same fitting, calculation, sampling and coding process of S4-1 to S4-4 is performed, the standard elevation coding sequence of the corrugated sheet is obtained, and the same process is performed for the corrugated sheet samples with different typical defects, and the coding mode of each type of defect is obtained, and finally the standard reference model and the defect coding library are established.

[0077] Step S5, the corrugated sheet elevation feature coding sequence is compared with the standard reference model bit by bit, the defects are identified according to the defect judgment rule, and the cross verification is performed combined with the three-dimensional curvature and the image sequence, and the corrugated sheet defect judgment result is obtained.

[0078] S5-1, compare the elevation feature coding sequence of the segmented waveform plate with the corresponding standard elevation coding sequence of the standard reference model to obtain comparison result data indicating whether each coding bit is consistent;

[0079] S5-2, according to the comparison result data, identify the missing defect by detecting the continuous missing coding mode according to the preset defect determination rule, then identify the bending deformation defect by detecting the proportion of high-level deviation points in the region exceeding the threshold, and further determine the defect category and the occurrence position according to the specific position section of the abnormal coding, generate the defect type and position analysis result, and the coding features of the missing defect and the bending deformation defect corresponding to the abnormal coding;

[0080] S5-3, based on the defect type and position analysis result, combined with three-dimensional curvature and image sequence, check whether there is a visible texture of the waveform plate interruption or severe bending in the three-dimensional curvature abnormal area of the moving camera original image, complete the cross verification of the defect type, obtain the waveform plate defect discrimination result confirmed by three-dimensional-two-dimensional multi-data, and the three-dimensional curvature represents the spatial curvature variation characteristics of each measurement point estimated by the local neighborhood surface fitting of the segmented waveform plate three-dimensional point cloud data.

[0081] In S5-3, the acquisition process of three-dimensional curvature is as follows:

[0082] In the segmented waveform plate three-dimensional point cloud data, a neighborhood point set with a fixed radius is constructed around each measurement point, and a local least square surface fitting is performed on each neighborhood point set to calculate the principal curvature, average curvature and Gaussian curvature at the measurement point; compare the above curvature values with the preset threshold to determine whether the measurement point position belongs to the curvature variation abnormal area.

[0083] Step S6, combine the waveform plate defect discrimination result with the positioning identification information, output the detection result data and store it, real-time trigger sound and light alarm according to the major defect determination threshold, compare and analyze the historical detection data to obtain the waveform plate aging degree evaluation data.

[0084] S6-1, combine the waveform plate defect discrimination result with the unique positioning identification information assigned to each segment of the waveform plate during the detection process to obtain the defect position (including the road section post number and the waveform plate segment number), the defect type and the defect degree evaluation, output the defect position, the defect type and the defect degree evaluation as the detection result data, and finally store the detection result data in the database or file for query and statistical analysis;

[0085] S6-2, based on the detection result data, whether there is a major defect is detected according to the preset major defect judgment threshold, when detecting major defects such as waveform board fracture loss, immediately trigger the defect alarm module, so that the defect alarm module sends an audible and visual alarm prompt signal, and outputs an alarm information to prompt the inspection personnel to take emergency measures;

[0086] The defect alarm module includes: an acoustic alarm unit, which plays a preset audio warning signal through a loudspeaker installed on the detection vehicle; a light alarm unit, which flashes a high-brightness LED light to prompt the detected major defect event; and a defect marking unit, which associates the defect with the waveform board segment number and the road segment post number, and presents them in text and image form in the user interface to guide manual inspection positioning.

[0087] S6-3, the detection result data and the segmented waveform board elevation feature code sequence are compared with the historical detection data stored in the database, the waveform board deformation trend over time is analyzed, and waveform board aging degree evaluation data is obtained to provide input basis for subsequent maintenance decision.

[0088] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application has various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting the integrity of a highway wavelon based on a vehicle-mounted three-dimensional camera, characterized in that, The method comprises the following steps: Step S1, for the calibration image sequence and the laser image sequence, a mobile camera internal and external parameter solving and a least square plane fitting are used to obtain a camera calibration matrix and a multi-light plane equation and write into a complete system configuration file; Step S2, an image sequence in a vehicle driving process is collected, a light strip center pixel coordinate in the image sequence is extracted through a gray gravity center algorithm, the light strip center pixel coordinate is back projected and a three-dimensional intersection of a light ray and a light plane is solved by using the complete system configuration file, and three-dimensional point cloud data of a corrugated plate surface is obtained; Step S3, coordinate transformation and splicing, denoising and down-sampling, preliminary segmentation based on height and position, accurate segmentation and segment division according to geometric characteristics are performed on the three-dimensional point cloud data of the corrugated plate surface, and segmented corrugated plate three-dimensional point cloud data is obtained; Step S4, for the segmented corrugated plate three-dimensional point cloud data, a least square plane fitting is used to obtain a reference surface, a height deviation value of each point to the reference surface is calculated, multi-layer and multi-interval sampling is performed and grade coding is performed according to a preset threshold value, a corrugated plate elevation feature coding sequence is obtained, and a standard reference model and a defect coding library are established; Step S5, the corrugated plate elevation feature coding sequence is compared with the standard reference model bit by bit, defects are identified according to a defect judgment rule, and cross verification is performed in combination with three-dimensional curvature and the image sequence, and a corrugated plate defect judgment result is obtained; Step S6, the corrugated plate defect judgment result is combined with positioning identification information, detection result data is output and stored, an audible and light alarm is triggered in real time according to a major defect judgment threshold value, historical detection data is compared and analyzed, and corrugated plate aging degree evaluation data is obtained; In step S3, the following sub-steps are further included: S3-1, according to vehicle motion pose information, the three-dimensional point cloud data of the corrugated plate surface in each frame in a mobile camera coordinate system is converted to a unified world coordinate system, a point cloud registration algorithm is used to register and splice adjacent frame three-dimensional point cloud data of the corrugated plate surface, and a complete and aligned corrugated plate three-dimensional point cloud model in the world coordinate system is obtained; S3-2, for the corrugated plate three-dimensional point cloud model, a statistical filtering method is used to remove isolated outlier points in the point cloud, the point cloud represents data points in the three-dimensional point cloud model, and a voxel grid filtering is used to down-sample the point cloud, and the three-dimensional point cloud data of the corrugated plate after denoising and down-sampling is obtained; S3-3, for the three-dimensional point cloud data of the corrugated plate after denoising and down-sampling, points lower than a bottom edge of the corrugated plate and higher than a top of the corrugated plate are removed based on a prior height threshold value, strip-shaped point clouds close to a road edge and continuously distributed along a highway direction are extracted through horizontal position screening, and a preliminarily segmented corrugated plate point cloud region is obtained; S3-4, a cross section profile feature is calculated in a vertical direction in the corrugated plate point cloud region, a wave-shaped geometric feature of the corrugated plate is used, a point cloud with a “double wave” and a “three wave” typical cross section shape is identified through a curved surface fitting algorithm, and the point cloud meeting the wave-shaped geometric feature of the corrugated plate is clustered and extracted, and pure three-dimensional point cloud data of the corrugated plate surface is obtained; S3-5, according to the physical segment length of the corrugated plate, the pure corrugated plate surface three-dimensional point cloud data is divided into several segments along the road direction, and segmented corrugated plate three-dimensional point cloud data is obtained by dividing the corrugated plate segment as a unit.

2. The highway corrugated plate integrity detection method based on a vehicle-mounted three-dimensional camera according to claim 1, characterized in that: In step S1, further comprising the following sub-steps: S1-1, by moving the camera to shoot the planar calibration board with the calibration pattern at different angles, a plurality of image sequences at different poses are obtained, the calibration pattern includes a checkerboard pattern and a circular dot array pattern, the Zhang Zhengyou plane calibration algorithm is used to extract the calibration pattern corner point coordinates from the obtained plurality of image sequences at different poses, and the intrinsic matrix, distortion coefficient and extrinsic transformation matrix of the moving camera are solved, and the solved intrinsic matrix, distortion coefficient and extrinsic transformation matrix of the moving camera are saved as a camera calibration file; S1-2, use the camera calibration file to drive the multi-line laser to project multiple laser lights on the calibration whiteboard to form multiple bright light bar images; move the calibration whiteboard to different spatial distance positions, and collect multiple light bar images containing the projected multiple laser lights at each different spatial distance position respectively, to form an image sequence, and calculate the complete spatial plane equation of the laser; S1-3, write the camera calibration file and the complete spatial plane equation of the laser into a complete system configuration file, and fix the moving camera and the laser on the installation position of the detection vehicle.

3. The highway corrugated plate integrity detection method based on a vehicle-mounted three-dimensional camera according to claim 2, characterized in that: In S1-2, the process of calculating the complete spatial plane equation of the laser is as follows: extract the light bar center pixel coordinates corresponding to the projected multiple laser lights in the image sequence by the gray center algorithm, and project the extracted light bar center pixel coordinates back to the space light rays emitted through the optical center of the moving camera, then combine the space light rays with the plane equation of the current calibration whiteboard in the world coordinate system, and calculate the three-dimensional intersection coordinates of the light bar center pixel at the position of the current calibration whiteboard; collect all three-dimensional intersection coordinates of the same laser at different positions of the calibration whiteboard, and calculate the complete spatial plane equation of the laser by the least square plane fitting method.

4. The highway corrugated plate integrity detection method based on a vehicle-mounted three-dimensional camera according to claim 1, characterized in that: In step S2, further comprising the following sub-steps: S2-1, during the driving of the vehicle along the highway, drive the multi-line laser to project multiple laser lights on the surface of the highway corrugated plate to form multiple bright light bar images, and the moving camera synchronously collects the light bar images containing the corrugated plate and the projected multiple laser lights, to obtain the image sequence collected during the driving of the vehicle; S2-2, for each image in the image sequence collected during the driving of the vehicle, perform grayscale and noise filtering processing, and extract the center pixel coordinates of the projected multiple laser lights in the limited region of interest by the gray center algorithm, to obtain the light bar center pixel coordinates corresponding to each image; S2-3, using the camera calibration file and the complete spatial plane equation of the laser, the light strip center pixel coordinates are back-projected to solve the three-dimensional intersection coordinates of the space light ray passing through the mobile camera optical center and the corresponding laser light plane. The image sequence collected is processed continuously to generate the three-dimensional point cloud data of the corrugated board surface corresponding to the vehicle mileage coordinates.

5. The highway corrugated board integrity detection method based on a vehicle-mounted three-dimensional camera according to claim 1, characterized in that: In step S4, the following sub-steps are further included: S4-1, selecting the straight and complete area point data in each segment of the segmented corrugated board three-dimensional point cloud data, using the least square plane fitting method to obtain the best fitting plane, and obtaining the plane equation representing the reference surface of the segmented corrugated board; S4-2, substituting the three-dimensional coordinates of each point in each segment of the point cloud into the distance formula from the point to the plane, calculating the normal distance value of each point relative to the reference surface, and obtaining the height deviation value of all points of the segmented corrugated board to the reference surface; S4-3, based on the height deviation value of all points of the segmented corrugated board to the reference surface, the segmented corrugated board three-dimensional point cloud data is divided into multiple height layer subsets according to the number of multi-line laser scanning in the vertical direction, and each height layer subset is divided into several equal interval regions along the length direction of the corrugated board. Selecting representative deviation values from each interval to form a multi-layer and multi-interval sampling data matrix; S4-4, for the multi-layer and multi-interval sampling data matrix, according to the preset four deviation level thresholds, each sampling deviation value is encoded according to the level to obtain a multi-bit elevation coding sequence corresponding to the segmented corrugated board; S4-5, for the intact corrugated board sample selected before detection, the same fitting, calculation, sampling and encoding processes of S4-1 to S4-4 are performed to obtain the standard elevation coding sequence of the corrugated board. For corrugated board samples with different typical defects, the same process is also performed, and the coding patterns of various defects are obtained. Finally, a standard reference model and a defect coding library are established.

6. The highway corrugated board integrity detection method based on a vehicle-mounted three-dimensional camera according to claim 1, characterized in that: In step S5, the following sub-steps are further included: S5-1, comparing the elevation feature coding sequence of the segmented corrugated board with the corresponding corrugated board standard elevation coding sequence in the established standard reference model bit by bit to obtain comparison result data whether each coding bit is consistent; S5-2, according to the comparison result data, according to the preset defect judgment rule, the continuous missing coding mode is detected to identify the missing defect, then the area with high level deviation point ratio exceeding the threshold is detected to identify the bending deformation defect, and the specific location section where the abnormal coding is located is further judged to determine the defect type and the occurrence position. The analysis result of the defect type and the position is generated. The abnormal coding corresponds to the coding characteristics of the missing defect and the bending deformation defect; S5-3, based on the analysis results of defect type and location, combined with three-dimensional curvature and image sequence, check whether there is visible texture in the three-dimensional curvature abnormal area of the original image of the moving camera, complete the cross verification of the defect type, get the three-dimensional-two-dimensional multi-data confirmed waveform plate defect judgment result, the three-dimensional curvature represents the spatial curvature variation characteristics of each measurement point estimated by local neighborhood surface fitting of segmented waveform plate three-dimensional point cloud data.

7. The highway waveform plate integrity detection method based on vehicle-mounted three-dimensional camera according to claim 6, characterized in that: In S5-3, the acquisition process of three-dimensional curvature is as follows: In the segmented waveform plate three-dimensional point cloud data, a neighborhood point set with a fixed radius is constructed around each measurement point, and local least squares surface fitting is performed on each neighborhood point set to calculate the principal curvature, average curvature and Gaussian curvature at the measurement point; compare the above principal curvature, average curvature and Gaussian curvature with the preset threshold value to determine whether the measurement point position belongs to the curvature variation abnormal area.

8. The highway waveform plate integrity detection method based on vehicle-mounted three-dimensional camera according to claim 1, characterized in that: In step S6, the following substeps are further included: S6-1, combine the waveform plate defect judgment result with the unique location identification information assigned to each segment of the waveform plate during the detection process to obtain the defect location, defect type and defect degree evaluation, and output the defect location, defect type and defect degree evaluation as the detection result data, and finally store the detection result data into the database; S6-2, based on the detection result data, according to the preset major defect judgment threshold, detect whether there is a major defect, when a major defect is detected, immediately trigger the defect alarm module, so that the defect alarm module sends an audible and visual alarm prompt signal, and outputs the alarm information to prompt the inspection personnel to take emergency measures; S6-3, compare the detection result data and the segmented waveform plate elevation feature code sequence with the historical detection data stored in the database to analyze the change trend of the waveform plate deformation over time, and obtain the waveform plate aging degree evaluation data.

9. The highway waveform plate integrity detection method based on vehicle-mounted three-dimensional camera according to claim 8, characterized in that: The defect alarm module includes: an audible alarm unit, which plays a preset audio warning signal through a loudspeaker installed on the detection vehicle; a light alarm unit, which flashes a high-brightness LED light to prompt the detected major defect event; a defect marking unit, which associates the waveform plate segment number where the defect is located with the road segment post number, and presents it in the form of text and image in the user interface to guide manual inspection positioning.

Images