Dual- efficacy background plate and calibration method for rockfill visual inspection

Abstract

This invention discloses a dual-function background board and a frame-by-frame dynamic self-calibration method for visual inspection of riprap, belonging to the field of intelligent inspection technology for water conservancy and hydropower engineering. The background board has dual functions: as a high-contrast optical background, it uses a customized color and matte surface that contrasts sharply with the dark gray of the riprap, significantly reducing the difficulty of image segmentation; as a built-in geometric ruler, it has precisely known physical dimensions and is fixed in each frame of the image. The frame-by-frame dynamic self-calibration method based on this background board includes: automatically extracting the edges of the background board in each frame, calculating the pixel size, dynamically calculating the pixel equivalent, and then converting the pixel size of the stone into its physical size. This invention achieves multiple uses for the background board, simultaneously solving the difficulties of foreground-background separation and size calibration without increasing hardware costs; the frame-by-frame self-calibration eliminates measurement errors introduced by factors such as camera distance fluctuations, lens distortion, temperature drift, and on-site vibration in principle. Engineering verification shows a pixel equivalent of 0.91~0.92 mm / pixel and a relative measurement error of ≤5%, making it widely applicable to visual inspection of stone gradation in water conservancy and hydropower, mining, and building materials industries.

Description

Technical Field

[0001] This invention belongs to the field of intelligent detection technology for water conservancy and hydropower projects, specifically relating to a dual-function background plate and calibration method for visual inspection of rockfill materials. Background Technology

[0002] Machine vision-based gradation detection technology for riprap has developed rapidly in recent years, but it faces two major challenges in practical applications:

[0003] (1) Difficulty in separating foreground and background: Due to factors such as material and moisture content, piled stones are usually dark gray, and the pixel boundaries with common backgrounds are blurred, making it difficult for image segmentation algorithms to accurately extract the stone outlines. Traditional methods use fixed color background boards or natural backgrounds, which have insufficient contrast, and the segmentation effect is unstable, especially under changes in on-site lighting and dust interference.

[0004] (2) Difficulty in accurately converting pixel size to physical size: The stone outline obtained by visual inspection is in pixels and must be converted to physical size before it can be used for gradation analysis. Traditional calibration methods rely on the precise distance between the camera and the object, and require offline calibration or the use of calibration boards. In engineering sites, there are problems such as calibration parameter drift, inability to adapt to dynamic shooting, and the need for manual operation, which cannot be automated for continuous detection. Summary of the Invention

[0005] The present invention aims to solve the problems existing in the prior art by providing a dual-function background plate and calibration method for visual inspection of riprap, which improves the contrast between the foreground and background, reduces the difficulty of image segmentation, realizes real-time and accurate conversion of pixel size to physical size, and eliminates the errors of traditional calibration methods.

[0006] The present invention relates to a dual-function background plate and calibration method for visual inspection of riprap, characterized in that the background plate is applied in a riprap vibrating and leveling device, the device comprising a conveyor belt, a vibrating screen, a background plate, and an image acquisition device. The conveyor belt is connected to the front end of the vibrating screen, the background plate is vertically installed below the rear end of the vibrating screen, a high-speed camera is positioned opposite the background plate, flush with it, the background plate uses a custom color that contrasts sharply with the color of the riprap, the surface is matte, and its size covers the imaging area of ​​the falling riprap; a scale is provided at the edge; the specific calibration method includes the following steps:

[0007] S1 background panel edge recognition:

[0008] In each frame, the Canny edge detection operator is used to extract the image edges, and then the four vertices of the background are located by Hough transform or contour filtering. The specific implementation is as follows:

[0009] First, the Canny operator is used to detect the pixel-level position of the background edge;

[0010] Then, around the aforementioned locations, perform quadratic curve fitting on the gray-level gradient in the edge normal direction; or use the gray-level centroid method, taking the extreme points of the fitted curve as the edge positions with sub-pixel precision.

[0011] Through the above steps, the edge positioning accuracy is improved from ±0.5 pixels to within ±0.1 pixels;

[0012] S2 pixel size calculation:

[0013] The formula for calculating the pixel width w_pix and pixel height h_pix of the background in the image is:

[0014] Horizontal pixel count: w_pix = |x_right - x_left|

[0015] Vertical pixel count: h_pix = |y_bottom - y_top|

[0016] Where (x_left, y_top) are the pixel coordinates of the top left corner of the background, and (x_right, y_bottom) are the pixel coordinates of the bottom right corner of the background;

[0017] S3 pixel equivalent dynamic calculation:

[0018] Calculate the pixel equivalents in the horizontal and vertical directions separately:

[0019] Horizontal pixel equivalent: k_x = W_b / w_pix (unit: mm / pixel)

[0020] Vertical pixel equivalent: k_y = H_b / h_pix (unit: mm / pixel)

[0021] When there is a difference between k_x and k_y, they are used for dimension measurement in the horizontal and vertical directions respectively, or the geometric mean k=√(k_x·k_y) is taken for isotropic measurement.

[0022] S4 stone size calculation:

[0023] Measure the pixel size of the stone, multiply it by the corresponding pixel equivalent to obtain the physical size, and calculate the equivalent diameter using the following formula:

[0024] d = k·d_pix,

[0025] Where d_pix = 2√(A_pix / π).

[0026] The background panel is a custom color in the dark blue family, with an RGB value of 0,0,139~0,0,205, which creates the greatest color difference with the dark gray of the rubble.

[0027] The background panel is coated with an anti-glare coating with a gloss level of ≤10 GU at 60°.

[0028] The background panel measures 1.44m wide and 1.2m high, with a manufacturing precision of ±1mm.

[0029] The background panel is inlaid with reflective markings or colored markers along its edges to assist image recognition algorithms in rapid localization.

[0030] In step S1, a sub-pixel-level edge detection algorithm is used to make the background panel vertex extraction accuracy better than 0.1 pixels.

[0031] In step S3, when the difference between k_x and k_y exceeds a threshold, it is determined that the camera is tilted or distorted, triggering a correction process or an alarm.

[0032] The present invention has the following beneficial effects:

[0033] (1) Dual functions integrated, one item for multiple uses

[0034] This invention uses the background plate as both an "optical background enhancement" and a "geometric calibration reference," solving both image segmentation and size calibration challenges without increasing hardware costs, thus demonstrating integrated innovation thinking.

[0035] (2) Significantly improves image segmentation quality

[0036] Customized color and matte finish significantly increase the grayscale difference between the stone edge and the background. Even in complex environments such as changes in lighting and dust interference, the image segmentation algorithm can stably extract the stone outline, improving the image segmentation accuracy by more than 15 percentage points.

[0037] (3) Frame-by-frame dynamic self-calibration completely eliminates calibration error.

[0038] This invention enables independent calibration of each image frame and automatically adapts to the following changes:

[0039] Camera distance fluctuations may be caused by loose installation or temperature-induced deformation.

[0040] Lens distortion, especially at the edges of the field of view;

[0041] Focus drift caused by temperature changes;

[0042] Mechanical vibration causes camera displacement, resulting in vibration at the site;

[0043] The aforementioned errors, which are difficult to overcome by traditional calibration methods, are completely eliminated in principle by this invention. The background plate of this invention has precisely known physical dimensions and is fixedly installed in the camera's field of view. In each frame of the video, it is parallel to the trajectory of the falling stone and at a very small distance. The pixel size of the background plate and the pixel size of the stone follow the same imaging ratio. Therefore, by simply extracting the pixel size of the background plate in each frame, the pixel equivalent of that frame can be calculated in real time, thereby eliminating calibration errors introduced by factors such as camera distance fluctuations, lens distortion, and temperature drift.

[0044] (4) High measurement accuracy

[0045] The measured pixel equivalent can reach 0.91~0.92mm / pixel, and the relative error between visual measurement and manual caliper measurement is ≤5%, which meets the engineering accuracy requirements.

[0046] (5) High degree of automation

[0047] Using a background board with markings instead of a calibration board, and in conjunction with a high-speed camera and computer, it can operate fully automatically without offline calibration, meeting the needs of continuous on-site testing in engineering projects.

[0048] (6) Strong robustness

[0049] The background panel edge recognition algorithm has been optimized and is well-adapted to interference such as partial occlusion, stains, and uneven lighting. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the background panel structure.

[0051] Figure 2 This is a schematic diagram of the background panel layout.

[0052] Figure 3 This is a flowchart of the calibration method.

[0053] Figure 4 This is a schematic diagram of background panel edge recognition.

[0054] Figure 5 This is a diagram showing the segmentation effect.

[0055] The components include a high-speed camera (1), a background plate (2), a vibrating screen (3), and a conveyor belt (4). Detailed Implementation

[0056] Example 1: A dual-function background plate and calibration method for visual inspection of riprap, applied in a riprap vibrating paving device. The device includes a conveyor belt, a vibrating screen, a background plate, and an image acquisition device. The conveyor belt is connected to the front end of the vibrating screen. The background plate is vertically installed below the rear end of the vibrating screen. A high-speed camera is positioned opposite the background plate, flush with it. The background plate uses a custom color that contrasts sharply with the color of the riprap, has a matte finish, and its size covers the imaging area of ​​the falling riprap. A scale is provided at the edge. The background plate structure is as follows:

[0057] (1) Background parameters:

[0058] Physical dimensions: Width W_b=1440mm, Height H_b=1200mm, Thickness 10mm;

[0059] Substrate: Aluminum alloy sheet, with anodized surface treatment;

[0060] Color coating: Deep red, using automotive-grade baking paint process, color difference ΔE<2;

[0061] Surface treatment: matte coating, 60° gloss level 8GU;

[0062] Edge markings: Red reflective dots, 10mm in diameter, are inlaid at the four corners;

[0063] (2) Installation method: The device frame is fixed by an adjustable bracket, parallel to the stone falling trajectory, with a spacing of ≤50mm;

[0064] (3) Detect hardware configuration:

[0065] Camera: High-speed industrial camera, 1920×1080 pixels resolution, 240fps frame rate;

[0066] Lens: 25mm focal length, 6m working distance;

[0067] Computer: Industrial PC, Intel i7 processor, 32GB RAM, NVIDIA RTX 3060 graphics card.

[0068] (4) Software implementation steps:

[0069] 1) Image Acquisition: A high-speed camera continuously captures the falling stone process, with each frame having a resolution of 1920×1080 pixels. The video of the stone's free fall is continuously recorded, and the video frames are then extracted as images for processing. The distance between the high-speed camera and the background can be adjusted within the range of 4-8m. A closer distance results in a smaller field of view and higher pixel accuracy, suitable for fine-grained material detection; a greater distance results in a larger field of view and higher recording efficiency, suitable for rapid detection of coarse-grained materials. This distance can be flexibly adjusted according to detection requirements.

[0070] 2) Background edge recognition:

[0071] Each frame of the image is preprocessed by calling the toolbox functions in the software and performing grayscale conversion and Gaussian filtering.

[0072] Canny edge detection is used to extract image edges;

[0073] Based on prior knowledge of the background size and color features, four vertices of the background are selected and located with sub-pixel precision. The specific method is as follows:

[0074] First, the Canny operator is used to detect the pixel-level position of the background edge;

[0075] Then, around the aforementioned locations, perform quadratic curve fitting on the gray-level gradient in the edge normal direction; or use the gray-level centroid method, taking the extreme points of the fitted curve as the edge positions with sub-pixel precision.

[0076] Through the above steps, the edge positioning accuracy is improved from ±0.5 pixels to within ±0.1 pixels;

[0077] 3) Pixel size calculation:

[0078] The coordinates of the top left vertex are (x1, y1), and the coordinates of the bottom right vertex are (x2, y2).

[0079] Pixel width w_pix = x2 - x1 = 1680 pixels;

[0080] Pixel height h_pix = y2 - y1 = 1120 pixels;

[0081] 4) Pixel equivalent calculation:

[0082] Horizontal pixel equivalent k_x = 1440mm / 1680 pixels = 0.857mm / pixel;

[0083] Vertical pixel equivalent k_y = 1200mm / 1120 pixels = 1.071mm / pixel;

[0084] The geometric mean k = √(0.857 × 1.071) = 0.958 mm / pixel;

[0085] Using horizontal and vertical measurement schemes, k=k_x and k=k_y are obtained respectively;

[0086] 5) Stone size calculation:

[0087] The area of ​​a certain stone in the image is A_pix = 12500 pixels²;

[0088] The equivalent diameter d_pix = 2√(12500 / π) ≈ 126.3 pixels;

[0089] The physical equivalent diameter d = 0.958 mm / pixel × 126.3 pixels ≈ 121.0 mm.

[0090] This embodiment tests the performance of the method of the present invention under different lighting conditions at a hydropower station site:

[0091] (1) Test conditions:

[0092] Strong sunlight on a sunny day (illuminance > 10000 lux);

[0093] Overcast diffused light (illuminance 2000~3000 lux);

[0094] Low light intensity at dusk (illuminance <500 lux);

[0095] Fill light on / off;

[0096] (2) The test results are as follows:

[0097] Lighting conditions Background board recognition success rate Pixel equivalent standard deviation Remark Strong sunlight on a sunny day 100% 0.012mm / pixel No reflective interference Cloudy day diffused light 100% 0.008mm / pixel Optimal lighting Evening + supplemental lighting 99.80% 0.015mm / pixel Slight noise in a few frames No supplemental lighting in the evening 85% 0.045mm / pixel It is recommended to turn on the fill light.

[0098] The results show that the method of the present invention is stable and reliable under normal lighting conditions, and can still maintain high accuracy in low-light environments when combined with supplementary lighting.

[0099] This embodiment compares the method of the present invention with two traditional calibration methods:

[0100] (1) Comparison of schemes:

[0101] Method A, traditional offline calibration method: checkerboard calibration board, fixed distance calibration, using the same parameters throughout;

[0102] Method B, traditional reference method: Place a standard ball in the shooting area and recalibrate it every 10 minutes;

[0103] Method C, the solution of this invention: frame-by-frame dynamic self-calibration;

[0104] (2) Test data:

[0105] The camera was continuously filmed for one hour, during which it was slightly moved manually to simulate vibration, and measurements were taken on a standard block of known size. The test data are as follows:

[0106] Calibration method average error Maximum error Standard deviation Remark Method A (Offline) 3.2mm 8.7mm 2.1mm The error increases after the camera moves. Method B (Interval Calibration) 1.8mm 4.2mm 1.5mm Drift still exists within the calibration interval Method C (This invention) 0.9mm 2.1mm 0.6mm Frame-by-frame calibration minimizes error.

[0107] The results show that the method of the present invention has significant advantages in dynamic environments.

[0108] In this embodiment, the method of the present invention was used to test 30 sets of rockfill samples at a hydropower station. The results were compared with those obtained by manual caliper measurement. The comparison results are as follows:

[0109] Particle size range Sample size Mean relative error Maximum relative error 5~40mm 120 4.80% 6.20% 40~100mm 150 pieces 3.50% 5.10% 100~200mm 180 2.90% 4.30% 200~400mm 150 pieces 2.40% 3.80% >400mm 50 3.10% 4.50%

[0110] The results show that the average relative error of the method of the present invention is ≤5% across the entire particle size range, which meets the engineering requirements.

[0111] The dual-function background plate and frame-by-frame dynamic self-calibration method provided by this invention solves the difficulties of image segmentation and size calibration simultaneously by using the background plate as both an optical background enhancement and geometric calibration reference, without increasing hardware costs. The method has a clear principle, is simple to implement, and is highly adaptable. It has been successfully verified in hydropower engineering projects and can be widely applied to visual inspection of stone gradation in fields such as water conservancy and hydropower, mining, and building materials, showing broad industrial application prospects.

Claims

1. A dual-function background board and calibration method for visual inspection of riprap, characterized in that, The aforementioned background plate is used in a stone vibratory paving device, which includes a conveyor belt, a vibrating screen, a background plate, and an image acquisition device. The conveyor belt is connected to the front end of the vibrating screen, and the background plate is vertically installed below the rear end of the vibrating screen. A high-speed camera is positioned opposite the background plate, flush with it. The background plate uses a custom color that contrasts sharply with the color of the piled stones, has a matte finish, and its size covers the imaging area of ​​the falling stones. A scale is provided at the edge. The specific calibration method includes the following steps: S1 background panel edge recognition: In each frame of the image, the vertex coordinates or four edge lines of the background are automatically extracted using an edge detection algorithm; S2 pixel size calculation: The formula for calculating the pixel width w_pix and pixel height h_pix of the background in the image is: Horizontal pixel count: w_pix = |x_right - x_left| Vertical pixel count: h_pix = |y_bottom - y_top| Where (x_left, y_top) are the pixel coordinates of the top left corner of the background, and (x_right, y_bottom) are the pixel coordinates of the bottom right corner of the background; S3 pixel equivalent dynamic calculation: Calculate the pixel equivalents in the horizontal and vertical directions separately: Horizontal pixel equivalent: k_x = W_b / w_pix, unit: mm / pixel Vertical pixel equivalent: k_y = H_b / h_pix, unit: mm / pixel W_b and H_b are the actual height and width of the background panel, i.e., the physical dimensions of the background panel; When there is a difference between k_x and k_y, they are used for dimension measurement in the horizontal and vertical directions respectively, or the geometric mean k=√(k_x·k_y) is taken for isotropic measurement. S4 stone size calculation: Measure the pixel size of the stone, multiply it by the corresponding pixel equivalent to obtain the physical size, and calculate the equivalent diameter using the following formula: d = k·d_pix, Where d_pix is ​​the geometric pixel size of the stone.

2. The dual-function background board and calibration method for visual inspection of riprap as described in claim 1, characterized in that, The background panel is custom-colored in red, yellow, or blue, which creates the greatest color difference from the rubble.

3. The dual-function background board and calibration method for visual inspection of riprap as described in claim 1, characterized in that, The background panel is coated with an anti-glare coating with a gloss level of ≤10 GU at 60°.

4. The dual-function background board and calibration method for visual inspection of riprap as described in claim 1, characterized in that, The background panel measures 1.44m wide and 1.2m high, with a manufacturing precision of ±1mm.

5. The dual-function background board and calibration method for visual inspection of riprap as described in claim 1, characterized in that, The background panel is inlaid with reflective markings or colored markers along its edges to assist image recognition algorithms in rapid localization.

6. The dual-function background board and calibration method for visual inspection of riprap as described in claim 1, characterized in that, In step S1, the Canny edge detection operator is used to extract the image edges, and then the four vertices of the background are located by Hough transform or contour filtering. The specific implementation method is as follows: First, the Canny operator is used to detect the pixel-level position of the background edge; Then, around the aforementioned locations, perform quadratic curve fitting on the gray-level gradient in the edge normal direction; or use the gray-level centroid method, taking the extreme points of the fitted curve as the edge positions with sub-pixel precision. Through the above steps, the edge positioning accuracy is improved from ±0.5 pixels to within ±0.1 pixels.

7. The dual-function background board and calibration method for visual inspection of riprap as described in claim 1, characterized in that, In step S3, when the difference between k_x and k_y exceeds a threshold, it is determined that the camera is tilted or distorted, triggering a correction process or an alarm. The threshold is 5%.

Images