Visual measurement method and system for surface cracks of hydraulic structure based on multiple active targets

Abstract

The invention relates to a hydraulic structure surface crack vision measurement method and system based on multiple active targets. The invention aims to solve the problem of solving the real physical scale of the plane crack when an included angle exists between a shooting plane and a measured plane in the photogrammetry process. According to the method, on the basis of the basic principle of monocular photogrammetry, the solution of an included angle between a shooting plane and a measured plane is realized by using an analytic geometry method, and a conversion model of a single pixel point from a two-dimensional image coordinate to a physical coordinate system is constructed, so that the restoration of the real physical scale of the plane crack is realized. Aiming at the problems of time and labor consumption, high danger level, low measurement precision and the like of a manual detection mode in a traditional building surface crack detection method, an evaluation model suitable for hydraulic structure surface cracks is established on the basis of a monocular photogrammetry principle, the crack detection efficiency and precision are improved, and the detection accuracy is improved. The method has great application and popularization values in the field of building safety monitoring.

Description

Technical Field

[0001] The invention belongs to the application of monocular photogrammetry in the field of engineering diagnosis, and specifically relates to a visual measurement method for surface cracks of hydraulic structures based on multiple active targets, which is a solution method for a crack measurement device based on multiple active targets. Background Art

[0002] Cracks are the most common disease of hydraulic structures. Cracks in hydraulic concrete structures can be divided into three types: surface cracks, through cracks and deep cracks. Among them, surface cracks have little impact, but they are very extensive; and there is a risk of further development into through or deep cracks. Therefore, daily monitoring of surface cracks in hydraulic structures is crucial.

[0003] Traditional methods for detecting cracks on the surface of hydraulic structures mostly use naked-eye evaluation methods, which are inefficient and poorly accurate. At the same time, the degree of informatization is insufficient, making it difficult to track the development of cracks. In addition, hydraulic structures are huge in size, and there are great safety hazards for personnel during high-altitude inspections. It can be seen that the existing artificial crack detection technology has many drawbacks and is difficult to meet the needs of my country's water conservancy project maintenance.

[0004] The rapid development of computer vision and digital image processing technology has provided the technology for intelligent detection of surface cracks. However, digital image processing technology can only analyze the image characteristics of cracks, and the physical scale analysis of images generally requires the support of orthophoto shooting conditions.

[0005] However, it is difficult to maintain an orthophoto shooting angle during image acquisition, and non-orthophoto images will have varying degrees of distortion relative to the original object. That is, the cracks in the image have projection differences due to different depth information. The greater the difference in depth information, the greater the difference between the crack image and its actual physical scale. Due to the existence of distortion, it is difficult to obtain the physical scale information of the cracks from these images.

[0006] In order to solve the problem of solving the real physical scale of surface cracks of hydraulic structures, the present invention uses the principles of spatial analytical geometry and monocular photogrammetry, and based on the self-developed active target projection device, proposes a real physical scale solution model for surface cracks of hydraulic structures based on multiple active targets. It can realize the acquisition of surface crack images from any angle and restore the real physical scale information of cracks. The present invention is of great significance for the observation and monitoring of surface cracks of hydraulic structures. Summary of the invention

[0007] The method of the invention provides a feasible solution for restoring the true form of cracks in the detection of cracks on the surface of hydraulic structures.

[0008] The purpose of the present invention is to provide a visual measurement method for surface cracks of hydraulic structures based on multiple active targets, and the specific implementation steps are as follows:

[0009] Preprocessing the original plane crack images;

[0010] Extract the coordinates of the laser point from the preprocessed image and solve the angle between the shooting plane and the real object plane;

[0011] Extract crack skeleton from the preprocessed image to obtain skeleton pixel coordinates;

[0012] According to the extracted skeleton pixel coordinates, combined with the angle between the shooting plane and the real object plane, after coordinate system transformation, the crack point coordinates on the real object plane are finally obtained;

[0013] The contours of the crack skeleton and the crack point set are extracted respectively, and the area and length information of the contours are calculated and integrated to calculate the average width of the crack.

[0014] Furthermore, the preprocessing includes: performing noise reduction processing on the input image using a Gaussian filtering noise reduction method, performing grayscale processing on the noise-reduced image, and performing image binarization processing on the grayscale image.

[0015] Furthermore, the specific implementation method of solving the angle between the shooting plane and the real object plane is as follows:

[0016] Step S1, using spatial analytic geometry to decompose the angle between the shooting plane and the object plane, that is, to decompose the angle between the object plane and the shooting plane into c Axis positive direction, Y c The angle a between the two positive directions of the axis 1 and a 2 ;

[0017] Step S2, using plane analytic geometry and pinhole imaging principles to solve the decomposed two-dimensional angle;

[0018] Step S3, constructing a position function expression in the active target camera coordinate system using spatial analytic geometry;

[0019] Step S4, based on the basic principle of monocular photogrammetry, combines the function expression in S3 to solve the camera pose.

[0020] Furthermore, the specific method of step S1 is as follows:

[0021] Step E1, when the shooting plane is parallel to the object plane, the image formed by the active target on the object plane can form a quadrilateral A'B'C'D'; when there is an angle between the shooting plane and the object plane, the image formed by the active target on the object plane can form a quadrilateral ABCD;

[0022] Step E2, assuming that the object plane and the shooting plane exist in the X coordinate system of the camera.c Axis positive direction, Y c The angle a between the two positive directions of the axis 1 and a 2 ;

[0023] Step E3, assuming that the shooting plane is the front plane V, then any straight line l in the space is projected into the horizontal plane H to obtain a horizontal line l 1 , find the corresponding projection l on the imaging plane 1 ', horizontal line l 1 With projection 1 The angle formed is the angle around X c Axis positive deflection angle a 1 ; Then similarly, the projection of the straight line l in the lateral plane W gives a lateral plane line l 2 , find the corresponding projection l on the imaging plane 2 ', side plane l 2 With projection 2 'The angle formed is the angle around Y c Axis positive deflection angle a 2 .

[0024] Furthermore, the specific method of step S2 is as follows:

[0025] Step F1, according to the pinhole imaging principle, assume that the angle between the object plane and the imaging plane is angle α, CD is the real physical scale of the object, GE is the physical scale of the object on the ideal orthographic plane, F is the midpoint of GE imaging, AB is the projection size of the object on the imaging plane, O is the optical center of the camera, f is the focal length of the camera, and z is the object depth information, that is, the distance between the object and the optical center of the camera;

[0026] Step F2, according to the principle of plane analytic geometry, we have △ABO~△EGO, and we can get

[0027] Step F3, in △CGF and △FED, according to the sine theorem, we can get and

[0028] Step F4, according to the pinhole imaging principle, we can get So

[0029] Furthermore, the specific method of step S3 is as follows:

[0030] Step G1, taking the laser beam as the active target, expressing the straight line equation of the active target in the camera coordinate system;

[0031] Step G2, the plane equation of the real object plane can be obtained by converting the distance measured by the laser rangefinder to the value obtained at the optical center of the camera and combining it with the angle between the real object plane and the plane of the measuring device;

[0032] Step G3, by solving the equation of the straight line formed by the laser beam and the equation of the real plane simultaneously, the three-dimensional coordinates of the laser point on the image in the camera coordinate system can be obtained, that is, the real physical scale of the quadrilateral ABCD is obtained. In order to facilitate the solution of the intersection coordinates, the straight line equation is expressed in parameter form;

[0033] Step G4, set four laser emitters as active targets, and solve their straight line equations and plane equations to obtain the coordinates of points A, B, C, and D, including α 1、 α 2 Two unknowns.

[0034] Furthermore, the coordinate system transformation specifically includes: transforming the skeleton pixel point coordinates through the secondary transformation relationship between the pixel coordinate system and the image coordinate system, plotting the obtained image coordinate system coordinates, and obtaining a crack skeleton morphology point drawing in the image coordinate system; then, through the perspective projection transformation relationship between the image coordinate system and the camera coordinate system, the three-dimensional coordinates of the points constituting the crack in the ideal physical plane in the camera coordinate system, that is, the plane perpendicular to the camera optical axis, are obtained, and then the three-dimensional point coordinates of the real physical plane in the camera coordinate system are obtained by using the geometric transformation relationship restored by the world coordinates of a single pixel point, and these point coordinates are plotted to obtain the three-dimensional image morphology of the crack vertical orthographic plane and the real angle plane; finally, the crack point coordinates on the physical plane are obtained according to the coordinate rigid body transformation process from the camera coordinate system to the world coordinate system.

[0035] Furthermore, the specific method for obtaining the three-dimensional point coordinates of a single pixel in the camera coordinate system is as follows:

[0036] Step I1, let P c is a point on the real object plane. The line connecting it and the camera optical center Oc intersects the ideal object plane, i.e., the plane parallel to the image plane, at a point P. c ', intersecting the image plane at point P p , respectively through point P c , P c '、P p Draw plane X c O c Z c The perpendicular line intersects the plane at points A, A', and A", then according to the pinhole imaging principle, we can get △O c A”P p ~△O c A'P c '~△O c AP c , then

[0037] Step I2, similar to step I1, pass through point P c , P c '、P p Draw plane Y c O c Z c The perpendicular line intersects the plane at points B, B', and B", and we get △O c B”C”~△O c B'C'~△O c BC, then Among them, B”C” is obtained by extracting pixel coordinates from the picture and converting them into image coordinates;

[0038] Step I3, due to △O c A'B', △O c B'C' is a right triangle, then

[0039] Step I4: Since the four points Oc, C”, C’, and C are collinear, ∠CC’B=π-∠O c C'B';

[0040] Step I5, in △B'C'C, using the law of sines, we have

[0041] Step I6, in △A'B'A, using the law of sines, we have

[0042] Step I7, combining steps I1 to I6, can solve the camera coordinate system coordinates of point Pc

[0043] Step I8, based on the angle between the plane where the device is located and the plane of the real object, that is, the angle between the plane perpendicular to the camera optical axis and the plane of the real object, the coordinates of the pixel point in the real object imaging plane can be solved by using the rigid body transformation relationship between the world coordinate system and the camera coordinate system. The whole process of this transformation solution is the inverse application of the process of transforming the point coordinates from the world coordinate system to the pixel coordinate system based on the principle of monocular photogrammetry.

[0044] Furthermore, the specific length calculation method is to ignore the width information in the skeleton contour, and take half of the sum of the lengths of the crack skeleton contours as the actual length of the crack; sum the areas of the crack contours as the actual area of ​​the crack, and divide it by the area of ​​the picture to calculate the density of the cracks obtained by the shooting; finally, calculate the average width of the crack based on the area and length of the cracks obtained.

[0045] The present invention also provides a hydraulic structure surface crack visual measurement system based on multiple active targets, comprising:

[0046] A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the visual measurement method for surface cracks of hydraulic structures based on multiple active targets as described in the above technical solution.

[0047] Compared with the prior art, the present invention proposes a model for calculating the true morphology of surface cracks of hydraulic structures based on multiple active targets. The model has the following advantages:

[0048] (1) The rotation angle of the camera in three-dimensional space is decomposed into a two-dimensional plane, which simplifies the calculation ideas and solution steps; (2) The relative position relationship between the laser beams emitted by the laser transmitter and the distance measured by the laser rangefinder are used to solve the camera posture, and the designed known parameters are applied to the solution equation, reducing the number of unknown quantities in the solution process; (3) The laser points formed by multiple laser beams are used to adjust the solution of the rotation angle, which improves the calculation accuracy and effectively controls the errors caused by various factors in the solution process. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the examples of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some examples of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 This is the crack image preprocessing of an embodiment of the present invention, (a) the original crack image, (b) the crack image after Gaussian filtering and noise reduction processing, (c) the crack image after grayscale processing, (d) the crack image after binarization processing.

[0051] Figure 2 This is the laser image processing of an embodiment of the present invention, (a) laser bright spot extraction image, (b) laser point marking image of the original image.

[0052] Figure 3 It is a schematic diagram of straight line projection of an embodiment of the present invention.

[0053] Figure 4 It is a schematic diagram of solving the two-dimensional angle according to an embodiment of the present invention.

[0054] Figure 5This is an example of the extraction of crack image skeleton and crack real morphology in an embodiment of the present invention, (a) input binary crack image, (b) crack skeleton extraction image, (c) crack skeleton image coordinate system point drawing, (d) crack vertical orthographic plane and real angle plane three-dimensional image morphology, (e) crack skeleton real morphology diagram, (f) crack real morphology diagram.

[0055] Figure 6 This is a schematic diagram of the principle of restoring the world coordinates of a pixel point according to an embodiment of the present invention.

[0056] Figure 7 These are contour extraction effect diagrams of an embodiment of the present invention, (a) crack skeleton contour extraction effect diagram, (b) crack contour extraction effect.

[0057] Figure 8 The figure is a flowchart of the overall process of solving the angle between the shooting plane and the real object plane in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians of the present invention without making creative work are within the scope of protection of the present invention.

[0059] The following uses the plane crack image capture as an example and specifically describes the method of the present invention in conjunction with the accompanying drawings:

[0060] Step L1, performing noise reduction processing on the original plane crack image;

[0061] This experiment first uses the Gaussian filter denoising (GaussianBlur) method to perform denoising on the input image to remove noise interference. Noise means that the intensity value of the pixel point has a sudden change relative to its true value. Gaussian filter denoising reduces the sudden change of pixel values ​​by filtering out the high-frequency part of the image, that is, the image details, and retains the low-frequency part of the image, that is, the image smooth area, thereby reducing the noise interference of the image. The general steps are: taking each pixel as the center, take the weighted average of the RGB values ​​of all pixels in its 3×3 area as the grayscale value of the center point. Gaussian filter denoising is essentially a convolution operation on the image using the Gaussian kernel generated by the Gaussian function. For example Figure 1 (a) is the original image obtained by photographing the crack in the embodiment of the present invention; Figure 1 (b) is an image of the crack after Gaussian filtering and noise reduction in the embodiment of the present invention;

[0062] Step L2, grayscale processing is performed on the denoised image;

[0063] Image grayscale processing is an image processing method that uses a single color to represent each pixel, usually displayed from the lowest brightness black to the highest brightness white. The so-called single color means that the three components of the color R, G, and B have the same value. For example, the RGB value (30,30,30) represents a grayscale value of 30; the RGB value (150,150,150) represents a grayscale value of 150. From the above definition, it can be seen that the grayscale image is a single-channel image. Image grayscale processing helps to reduce data dimensions, simplify the image processing process, and provide clearer contrast and brightness information, making image features easier to analyze and identify. In this experiment, the OpenCV built-in function cvtColor in Python is used to grayscale the image obtained after Gaussian filtering and denoising, such as Figure 1 (c) is an image obtained after grayscale conversion of the crack in the embodiment of the present invention;

[0064] Step L3, performing image binarization processing on the grayscale image;

[0065] The main implementation path of the image binarization process is to divide the grayscale image into two different areas: background and foreground according to the grayscale value of the pixels. The foreground is the image feature we need to pay attention to, while the background is the surrounding environment outside the image feature. Image binarization can distinguish the target features to be extracted from their background using two obviously different pixel values, making it easier to achieve tasks such as feature target detection, segmentation, and contour extraction. Image binarization is usually implemented using the threshold processing method (Tresholding). The idea is to set a critical grayscale value, and define all pixel values ​​greater than the critical grayscale value as the grayscale maximum value, and all pixel values ​​less than the critical grayscale value as the grayscale minimum value, thereby converting the image into a binary image with only two color expressions. In this experiment, the OpenCV built-in function threshold in Python is used to perform image binarization on the image obtained by grayscale processing, such as Figure 1 (d) is an image obtained after the crack is binarized in the embodiment of the present invention;

[0066] Step L4, extracting the coordinates of the laser point from the binarized image, and solving the angle between the shooting plane and the real object plane;

[0067] The angle between the shooting plane and the real object plane in this experiment is solved based on the relative position relationship between the actively projected laser points in the captured image. Therefore, accurately extracting the coordinates of the laser points from the collected image is a key step in solving the real physical scale of the crack. First, since the laser points are brighter points in the image, after the grayscale image is obtained through image preprocessing, the pixel threshold is adjusted to 200-255 to obtain Figure 2 (a) Laser bright spot extraction image. Figure 2 In (a), it can be seen that there are small spots projected on the picture by the laser rangefinder in addition to the four laser points. In order to eliminate the influence of this small spot on the extraction of the coordinates of the laser emitter bright spot, the experiment applied the principle of morphological operation. First, the experiment uses the built-in function findContours of OpenCV in Python to detect the contours of the image, and then sorts the detected contours from left to right. After the contours are sorted, they can be processed in a separate loop. Among these contours, the method of extracting the minimum circumscribed circle of the contour is used to mark the position of each laser bright spot, and the coordinates of the center of each circumscribed circle are returned to obtain the pixel coordinates of the laser bright spot, and then the angle between the device plane and the real object plane is solved according to the solution method of the present invention. Figure 2 (b) is the original laser dot marking image of the embodiment of the present invention;

[0068] The specific implementation process of the solution method of the present invention is as follows:

[0069] Step S1, such as Figure 3 As shown in the figure, the angle between the shooting plane and the object plane is decomposed by using spatial analytic geometry, that is, the angle between the object plane and the shooting plane is decomposed into c Axis positive direction, Y c The angle a between the two positive directions of the axis 1 and a 2 ;

[0070] The specific method of step S1 is as follows:

[0071] Step E1, when the shooting plane is parallel to the object plane, the image formed by the active target on the object plane can form a quadrilateral A'B'C'D'; when there is an angle between the shooting plane and the object plane, the image formed by the active target on the object plane can form a quadrilateral ABCD;

[0072] Step E2, assuming that the object plane and the shooting plane exist in the X coordinate system of the camera. c Axis positive direction, Y c The angle a between the two positive directions of the axis 1 and a 2 ;

[0073] Step E3, assuming that the shooting plane is the front plane V, then any straight line l in the space is projected into the horizontal plane H to obtain a horizontal line l 1 , find the corresponding projection l on the imaging plane 1 ', horizontal line l 1 With projection 1 The angle formed is the angle around X c Axis positive deflection angle a 1 ; Then similarly, the projection of the straight line l in the lateral plane W gives a lateral plane line l 2 , find the corresponding projection l on the imaging plane 2 ', side plane l 2 With projection 2 'The angle formed is the angle around Y c Axis positive deflection angle a 2 ;

[0074] Step S2, using plane analytic geometry and pinhole imaging principles to solve the decomposed two-dimensional angle;

[0075] The specific method of step S2 is as follows:

[0076] Step F1, based on the pinhole imaging principle, such as Figure 4 As shown, let the angle between the object plane and the imaging plane be α, CD be the real physical scale of the object, GE be the physical scale of the object in the ideal orthographic plane, F be the midpoint of GE imaging, AB be the projection size of the object in the imaging plane, O be the optical center of the camera, f be the focal length of the camera, and z be the depth information of the object, i.e., the distance between the photographed object and the optical center of the camera;

[0077] Step F2, according to the principle of plane analytic geometry, we have △ABO~△EGO, and we can get

[0078] Step F3, in △CGF and △FED, according to the sine theorem, we can get and

[0079] Step F4, according to the pinhole imaging principle, we can get So

[0080] Step S3, constructing a position function expression in the active target camera coordinate system using spatial analytic geometry;

[0081] The specific method of step S3 is as follows:

[0082] Step G1, taking the laser beam as the active target, expressing the straight line equation of the active target in the camera coordinate system;

[0083] Step G2, the plane equation of the real object plane can be obtained by converting the distance measured by the laser rangefinder to the value obtained at the optical center of the camera and combining it with the angle between the real object plane and the plane of the measuring device;

[0084] Step G3, by solving the equation of the straight line formed by the laser beam and the equation of the real plane, the three-dimensional coordinates of the laser point on the image in the camera coordinate system can be obtained, that is, the real physical scale of the quadrilateral ABCD is obtained. In order to facilitate the solution of the intersection coordinates, the straight line equation is expressed in parameter form.

[0085] Step G4, the experiment sets four laser emitters as active targets, and the coordinates of points A, B, C, and D can be obtained by solving their line equations and plane equations, including α 1、 α 2 Two unknowns;

[0086] Step S4, based on the basic principle of monocular photogrammetry, the function expression in S3 is combined to solve the camera pose;

[0087] The specific method of step S4 is as follows:

[0088] Step H1, using the secondary transformation relationship between the image coordinate system and the pixel coordinate system in photogrammetry and the perspective projection relationship between the camera coordinate system and the image coordinate system, the distance around X between the plane where the device is located and the plane where the real object is located in the constructed camera coordinate system can be obtained respectively. c Axis positive direction, Y c The angle a between the two positive directions of the axis 1 and a 2 , thus the camera pose can be solved;

[0089] Step H2, the experimental setting has four active targets, so there are The combination of laser points can obtain the angle between the real plane and the photographing plane. In order to reduce the error of the measured results, the final angle is determined by adjusting the 6 results.

[0090] Step L5, extracting crack skeleton from the binarized image;

[0091] Crack skeleton extraction is to reduce the display of cracks on the image to a single pixel size, which can be used for image feature extraction and the topology of feature objects. Skeleton extraction is implemented according to the image continuity path. In each path, the boundary pixels that do not affect the connectivity of the image will be identified and deleted. This experiment uses the built-in function morphology.skeletonize of the Skimage library in Python to extract crack skeletons. The input is to set the region size threshold to remove tiny noise. Figure 5 (a) is the input binary crack image of the embodiment of the present invention, Figure 5 (b) is a crack skeleton extraction image of an embodiment of the present invention;

[0092] Step L6, based on the skeleton pixel coordinates extracted in step L5 and combined with the angle between the shooting plane and the physical plane in step L4, the skeleton pixel coordinates are transformed through the secondary transformation relationship between the pixel coordinate system and the image coordinate system, the perspective projection transformation relationship between the image coordinate system and the camera coordinate system, and the rigid body transformation transformation relationship between the camera coordinate system and the world coordinate system, to finally obtain the crack point coordinates of the physical plane;

[0093] Since the crack skeleton extraction image is a binary black-and-white image, the RGB values ​​of the pixels in the crack skeleton part and the background part of the image are (0,0,0) and (255,255,255) respectively. Based on this, the pixel extraction conditions are set, and the skeleton pixel coordinates are extracted after traversing the image pixels. After transforming them between the pixel coordinate system and the image coordinate system, the obtained image coordinate system coordinates are plotted, and the following can be obtained: Figure 5 (c) of the present invention in the image coordinate system of the embodiment of the present invention is plotted; then, through the perspective projection transformation relationship between the image coordinate system and the camera coordinate system, the three-dimensional coordinates of the points constituting the crack in the ideal physical plane in the camera coordinate system, that is, the plane perpendicular to the camera optical axis (blue), and then the three-dimensional point coordinates of the real physical plane in the camera coordinate system are obtained by the geometric transformation relationship of the world coordinate restoration of the single pixel point described in steps I1 to I8 (red), and these point coordinates are plotted to obtain the following Figure 5 The three-dimensional image of the crack vertical plane and the real angle plane in (d). Finally, the coordinates of the crack point on the real plane are obtained according to the rigid body transformation process from the camera coordinate system to the world coordinate system, as shown in Figure 5 (e) is a diagram showing the real morphology of the crack skeleton according to an embodiment of the present invention;

[0094] The specific method to obtain the three-dimensional point coordinates of a single pixel in the camera coordinate system is as follows:

[0095] Step I1, such as Figure 6 As shown, let P c is a point on the real object plane. The line connecting it and the camera optical center Oc intersects the ideal object plane, i.e., the plane parallel to the image plane, at a point P. c ', intersecting the image plane at point P p . Pass through point P c , P c '、P p Draw plane X c O c Z c The perpendicular line intersects the plane at points A, A', and A", then according to the pinhole imaging principle, we can get △O cA”P p ~△O c A'P c '~△O c AP c , then

[0096] Step I2, similar to step I1, pass through point P c , P c '、P p Draw plane Y c O c Z c The perpendicular line intersects the plane at points B, B', and B", and we get △O c B”C”~△O c B'C'~△O c BC, then Among them, B”C” is obtained by extracting pixel coordinates from the picture and converting them into image coordinates;

[0097] Step I3, due to △O c A'B', △O c B'C' is a right triangle, then

[0098] Step I4: Since the four points Oc, C”, C’, and C are collinear, ∠CC’B=π-∠O c C'B';

[0099] Step I5, in △B'C'C, using the law of sines, we have

[0100] Step I6, in △A'B'A, using the law of sines, we have

[0101] Step I7, combining steps I1 to I6, can solve the camera coordinate system coordinates of point Pc

[0102]

[0103] Step I8, according to the angle between the plane where the device is located and the plane of the real object obtained in step S4, that is, the angle between the plane perpendicular to the camera optical axis and the plane of the real object, the coordinates of the pixel point in the real object imaging plane can be solved by using the rigid body transformation relationship between the world coordinate system and the camera coordinate system. The whole process of this transformation solution is the inverse application of the coordinate transformation process from the world coordinate system of the point coordinate to the pixel coordinate system based on the principle of monocular photogrammetry.

[0104] Step L7, extract the contours of the crack skeleton and the crack point set respectively, calculate and integrate the area and length information of the contours, and thus calculate the average width of the cracks.

[0105] The real shape of the crack skeleton can be obtained by the transformation process of step L6. Then, the real shape of the crack can be obtained by performing the same operation on the crack obtained by shooting. Figure 5 (f) in the figure is the true morphological diagram of the crack in the embodiment of the present invention. Since the restored crack skeleton and cracks are both sets of discretized point sets, and the traditional method of extracting information such as crack length and width is based on continuous pixel coordinate points, it is impossible to use the traditional method to evaluate the morphological characteristics of the crack. This experiment chose to use the OpenCV built-in function findConters in Python to extract the contours of the obtained crack skeleton and crack point set respectively, and calculate and integrate the area and length information of the contours. The specific length calculation method is to ignore the width information in the skeleton contour, and take half of the sum of the lengths of the obtained crack skeleton contours as the true length of the crack; take the sum of the obtained crack contour areas as the true area of ​​the crack, and divide it by the area of ​​the picture to calculate the density of the cracks obtained by shooting; finally, calculate the average width of the crack based on the area and length of the cracks obtained. Figure 7 (a) in FIG. 7 is a crack skeleton contour extraction effect diagram of an embodiment of the present invention, and (b) in FIG. 7 is a crack contour extraction effect diagram of an embodiment of the present invention. In this embodiment, the crack length obtained by visual measurement evaluation is 3083.09 mm, and the crack area is 3263.54 mm. 2 The crack density is 1.71%, the crack length calculated by orthophoto is 3165.11mm, and the crack area is 3350mm 2 , the crack density is 1.73%, and the average accuracy of crack physical scale measurement is 98.42%.

[0106] The present invention also provides a hydraulic structure surface crack visual measurement system based on multiple active targets, comprising:

[0107] A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the visual measurement method for surface cracks of hydraulic structures based on multiple active targets as described in the above technical solution.

[0108] Other parts not described in detail are all prior art.

[0109] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0110] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A visual measurement method for surface cracks of hydraulic structures based on multiple active targets, characterized in that: include: Preprocessing the original plane crack images; Extract the coordinates of the laser point from the preprocessed image and solve the angle between the shooting plane and the real object plane; Extract crack skeleton from the preprocessed image to obtain skeleton pixel coordinates; According to the extracted skeleton pixel coordinates, combined with the angle between the shooting plane and the real object plane, after coordinate system transformation, the crack point coordinates on the real object plane are finally obtained; The contours of the crack skeleton and the crack point set are extracted respectively, and the area and length information of the contours are calculated and integrated to calculate the average width of the crack.

2. The method for visually measuring surface cracks of hydraulic structures based on multiple active targets as claimed in claim 1, characterized in that: The preprocessing includes: performing noise reduction processing on the input image by using a Gaussian filtering noise reduction method, performing grayscale processing on the noise-reduced image, and performing image binarization processing on the grayscale image.

3. The method for visually measuring surface cracks of hydraulic structures based on multiple active targets as claimed in claim 1, characterized in that: The specific implementation method of solving the angle between the shooting plane and the real plane is as follows: Step S1, using spatial analytic geometry to decompose the angle between the shooting plane and the object plane, that is, to decompose the angle between the object plane and the shooting plane into c Axis positive direction, Y c The angles a1 and a2 between the two positive directions of the axis; Step S2, using plane analytic geometry and pinhole imaging principles to solve the decomposed two-dimensional angle; Step S3, constructing a position function expression in the active target camera coordinate system using spatial analytic geometry; Step S4, based on the basic principle of monocular photogrammetry, combines the function expression in S3 to solve the camera pose.

4. The method for visually measuring surface cracks of hydraulic structures based on multiple active targets as claimed in claim 3 is characterized by: The specific method of step S1 is as follows: Step E1, when the shooting plane is parallel to the object plane, the image formed by the active target on the object plane can form a quadrilateral A'B'C'D'; when there is an angle between the shooting plane and the object plane, the image formed by the active target on the object plane can form a quadrilateral ABCD; Step E2, assuming that the object plane and the shooting plane exist in the X coordinate system of the camera. c Axis positive direction, Y c The angles a1 and a2 between the two positive directions of the axis; Step E3, assuming that the shooting plane is the front plane V, then any straight line l in the space is projected into the horizontal plane H to obtain a horizontal line l1, and the corresponding projection l1' is found on the imaging plane. The angle formed by the horizontal line l1 and the projection l1' is the angle around X. c The positive deflection angle a1 of the axis; similarly, the straight line l is projected into the lateral plane W to obtain a lateral plane line l2, and the corresponding projection l2' is found on the imaging plane. The angle formed by the lateral plane line l2 and the projection l2' is the angle around Y c Positive axis deflection angle a2.

5. The method for visually measuring surface cracks of hydraulic structures based on multiple active targets as claimed in claim 3 is characterized by: The specific method of step S2 is as follows: Step F1, according to the pinhole imaging principle, assume that the angle between the object plane and the imaging plane is angle α, CD is the real physical scale of the object, GE is the physical scale of the object on the ideal orthographic plane, F is the midpoint of GE imaging, AB is the projection size of the object on the imaging plane, O is the optical center of the camera, f is the focal length of the camera, and z is the object depth information, that is, the distance between the object and the optical center of the camera; Step F2, according to the principle of plane analytic geometry, we have △ABO~△EGO, and we can get Step F3, in △CGF and △FED, according to the sine theorem, we can get and Step F4, according to the pinhole imaging principle, we can get So 6. The method for visually measuring surface cracks of hydraulic structures based on multiple active targets as claimed in claim 3, characterized in that: The specific method of step S3 is as follows: Step G1, taking the laser beam as the active target, expressing the straight line equation of the active target in the camera coordinate system; Step G2, the plane equation of the real object plane can be obtained by converting the distance measured by the laser rangefinder to the value obtained at the optical center of the camera and combining it with the angle between the real object plane and the plane of the measuring device; Step G3, by solving the equation of the straight line formed by the laser beam and the equation of the real plane simultaneously, the three-dimensional coordinates of the laser point on the image in the camera coordinate system can be obtained, that is, the real physical scale of the quadrilateral ABCD is obtained. In order to facilitate the solution of the intersection coordinates, the straight line equation is expressed in parameter form; Step G4, set four laser emitters as active targets, and solve their straight line equations and plane equations to obtain the coordinates of points A, B, C, and D, including α 1、 α2Two unknowns.

7. The method for visually measuring surface cracks of hydraulic structures based on multiple active targets as claimed in claim 1, characterized in that: The coordinate system transformation specifically includes: transforming the skeleton pixel point coordinates through the secondary transformation relationship between the pixel coordinate system and the image coordinate system, plotting the obtained image coordinate system coordinates, and obtaining a crack skeleton morphology point drawing in the image coordinate system; then, through the perspective projection transformation relationship between the image coordinate system and the camera coordinate system, the three-dimensional coordinates of the points that make up the crack in the ideal physical plane in the camera coordinate system, that is, the plane perpendicular to the camera optical axis, are obtained, and then the three-dimensional point coordinates of the real physical plane in the camera coordinate system are obtained by using the geometric transformation relationship restored by the world coordinates of a single pixel point. These point coordinates are plotted to obtain the three-dimensional image morphology of the crack vertical orthographic plane and the real angle plane; finally, the crack point coordinates on the physical plane are obtained according to the coordinate rigid body transformation process from the camera coordinate system to the world coordinate system.

8. The method for visually measuring surface cracks of hydraulic structures based on multiple active targets as claimed in claim 1, characterized in that: The specific method to obtain the three-dimensional point coordinates of a single pixel in the camera coordinate system is as follows: Step I1, let P c is a point on the real object plane. The line connecting it and the camera optical center Oc intersects the ideal object plane, i.e., the plane parallel to the image plane, at a point P. c ', intersecting the image plane at point P p , respectively through point P c , P c '、P p Draw plane X c O c Z c The perpendicular line intersects the plane at points A, A', and A", then according to the pinhole imaging principle, we can get △O c A”P p ~△O c A'P c '~△O c AP c , then Step I2, similar to step I1, pass through point P c , P c '、P p Draw plane Y c O c Z c The perpendicular line intersects the plane at points B, B', and B", and we get △O c B”C”~△O c B'C'~△O c BC, then Among them, B”C” is obtained by extracting pixel coordinates from the picture and converting them into image coordinates; Step I3, due to △O c A'B', △O c B'C' is a right triangle, then Step I4: Since the four points Oc, C”, C’, and C are collinear, ∠CC’B=π-∠O c C'B'; Step I5, in △B'C'C, using the law of sines, we have Step I6, in △A'B'A, using the law of sines, we have Step I7, combining steps I1 to I6, can solve the camera coordinate system coordinate AB of point Pc = Step I8, based on the angle between the plane where the device is located and the plane of the real object, that is, the angle between the plane perpendicular to the camera optical axis and the plane of the real object, the coordinates of the pixel point in the real object imaging plane can be solved by using the rigid body transformation relationship between the world coordinate system and the camera coordinate system. The whole process of this transformation solution is the inverse application of the process of transforming the point coordinates from the world coordinate system to the pixel coordinate system based on the principle of monocular photogrammetry.

9. The method for visually measuring surface cracks of hydraulic structures based on multiple active targets as claimed in claim 1, characterized in that: The specific length calculation method is to ignore the width information in the skeleton contour, and take half of the sum of the crack skeleton contour lengths as the actual length of the crack; sum the crack contour areas as the actual area of ​​the crack, and divide it by the area of ​​the picture to calculate the density of the cracks obtained by the shooting; finally, calculate the average width of the crack based on the area and length of the crack.

10. The visual measurement system for surface cracks of hydraulic structures based on multiple active targets is characterized by: include: A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the visual measurement method for surface cracks of hydraulic structures based on multiple active targets as described in any one of claims 1 to 9.

Images