Multi-view fusion-based three-dimensional reconstruction method for sedimentation tank sludge

Abstract

The present application relates to a kind of multi-view fusion's sedimentation tank sludge three-dimensional reconstruction method, comprising: around the pool wall of the four around of sedimentation tank, along the bottom of sedimentation tank upwards successively horizontal setting multiple elevation lines;Synchronously collect the side shot of sedimentation tank at different view angles, containing elevation line and carry out pretreatment;The side shot after pretreatment is spliced into the panorama of sedimentation tank;From panorama, the junction line profile of sludge and sedimentation tank pool wall is extracted, with elevation line as reference, deduce the height component of sludge and clear liquid interface point cloud as sludge surface height;Estimate sludge surface displacement, and then according to sludge surface displacement, sludge surface height and extracted image feature, utilize deep learning model to predict sludge surface profile;Based on sludge surface profile, it is three-dimensionally reconstructed to it.The present application can accurately predict the sludge amount of entire sedimentation tank, to accurately control the start time interval of suction sludge machine.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional reconstruction and treatment of sedimentation tank sludge, and particularly relates to a multi-view fusion three-dimensional reconstruction method for sedimentation tank sludge. BACKGROUND

[0002] A mud scraper is usually arranged in the sedimentation tank in the water treatment process of a water plant for cleaning the sludge at the bottom of the sedimentation tank. The opening time of the mud scraper is usually a fixed time interval. If the time interval is too short, the sludge is less accumulated, and there may be an invalid operation condition, which leads to energy waste. If the time interval is too long, the sludge is long-term accumulated, which may cause equipment failure to be aggravated, the scraper blade of the mud scraper to be stuck or damaged, mechanical load to be increased, torque to be too large, and the sludge discharge pipeline to be easily blocked, which needs to be cleaned manually frequently.

[0003] The existing sludge thickness measurement methods can be basically divided into two kinds. The first kind is a manual direct visual method, which is simple to operate but relies on the experience and proficiency of workers and has strong subjectivity in judging the sludge amount. The second kind is to measure by using equipment, such as an ultrasonic wave type measuring instrument. However, this method can only measure the sludge thickness at a fixed point, and cannot accurately calculate the sludge content of the entire sedimentation tank. SUMMARY

[0004] The present application provides a multi-view fusion three-dimensional reconstruction method for sedimentation tank sludge, which can accurately predict the sludge amount of the entire sedimentation tank and improve the automation degree of sludge cleaning of the sedimentation tank.

[0005] The technical solution adopted by the present application to solve the technical problem is to provide a multi-view fusion three-dimensional reconstruction method for sedimentation tank sludge, which comprises the following steps.

[0006] A plurality of elevation lines are arranged horizontally along the bottom of the sedimentation tank in sequence around the pool wall of the sedimentation tank.

[0007] Sidelight pictures of the sedimentation tank containing the elevation lines are synchronously collected under different views and preprocessed.

[0008] The preprocessed sidelight pictures are spliced into a panoramic picture of the sedimentation tank.

[0009] The intersection line contour of the sludge and the pool wall of the sedimentation tank is extracted from the panoramic picture, the height component of the sludge and clear liquid interface point cloud is derived as the sludge surface height based on the elevation line as a reference, the sludge surface displacement is estimated, and the sludge surface contour is predicted by using a deep learning model according to the sludge surface displacement, the sludge surface height and the extracted image features.

[0010] The sludge surface contour is predicted by using a deep learning model according to the sludge surface displacement, the sludge surface height and the extracted image features.

[0011] The sludge surface contour is predicted by using a deep learning model according to the sludge surface displacement, the sludge surface height and the extracted image features.

[0012] Further, the height component of the sludge and clear liquid interface point cloud is derived based on the contour line, including:

[0013] Based on the contour line, the interface line profiles under different viewing angles are matched, and the three-dimensional point cloud coordinates of the interface line are calculated;

[0014] Based on the three-dimensional point cloud coordinates of the interface line, a plurality of grid surfaces are gradually deduced and constructed from the periphery of the pool body to the center of the pool body, until the complete interface of the sludge and clear liquid is obtained.

[0015] The center height of each grid surface is calculated as the height component of the interface point cloud.

[0016] Further, the plurality of grid surfaces are gradually deduced and constructed from the periphery of the pool body to the center of the pool body by using a triangulation algorithm.

[0017] Further, the center height of each grid surface is calculated based on the height of the grid triangle vertex by using bilinear interpolation.

[0018] Further, the extraction of the interface line profile between the sludge and the pool wall of the sedimentation tank includes:

[0019] An edge detection algorithm is used to extract the edge map of the interface between the sludge and the pool wall of the sedimentation tank;

[0020] The obtained edge map is subjected to binary segmentation, and the continuous closed profile is searched in the segmented binary image to obtain the interface line profile between the sludge and the pool wall of the sedimentation tank.

[0021] Further, before the binary segmentation of the obtained edge map, a morphological opening operation method is used to eliminate the noise of the edge map.

[0022] Further, the pre-processed side-view images are spliced into a panoramic image of the sedimentation tank by extracting features from the side-view images under different viewing angles, and aligning and fusing the extracted multi-view image features.

[0023] Further, the image features are the multi-view image features.

[0024] Further, the deep learning model is constructed based on a multi-layer linear perception machine and a Transformer network with a multi-head self-attention mechanism, and a fully connected layer is added at the output end of the Transformer network, combined with a SoftMax activation function to predict the sludge swelling probability.

[0025] Further, the side shooting picture containing the elevation line under different visual angles is realized by arranging industrial cameras at four corners of the sedimentation tank.

[0026] Advantages

[0027] Compared with the prior art, the present application has the following advantages and positive effects: the present application constructs a three-dimensional solution system based on physical reference by fusing the horizontal elevation line array around the pool wall with the multi-view side shooting image, uses the known height coordinates of the elevation line as the reference to deduce the sludge height, realizes the measurement accuracy of sludge height ±0.5cm and the error control of volume calculation ≤±3%, and significantly improves the measurement accuracy compared with the traditional ultrasonic measurement; the present application extracts the junction line profile of the sludge and the pool wall, and then gradually pushes the grid surface from the pool wall to the center, so that the sludge junction surface reconstructed by three-dimensional reconstruction is continuous and smooth, solving the problem of curve integrity in the sparse point cloud scene; the present application automatically identifies the sludge swelling (accuracy 92%) and abnormal accumulation by embedding a fully connected layer in the deep learning model, and makes the decision by visualizing the heat map, so that the sludge discharge energy consumption is reduced by 15%-20%, the non-contact measurement design prolongs the equipment maintenance period to 5 years (the traditional sensor needs to be maintained quarterly), and the IP68 protection level ensures stable operation in harsh environments such as pH2-11 and humidity 100%. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a flowchart of the embodiment of the present application;

[0029] Figure 2 is a schematic diagram of the device of the embodiment of the present application;

[0030] Figure 3 is a schematic diagram of the application scenario of the embodiment of the present application;

[0031] Figure 4 is a flowchart of the sludge three-dimensional reconstruction method of the embodiment of the present application;

[0032] Figure 5 is a sludge profile and swelling probability prediction network structure flowchart of the embodiment of the present application. DETAILED DESCRIPTION

[0033] The present application will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present application and not to limit the scope of the present application. In addition, it should be understood that those skilled in the art can make various modifications or modifications to the present application after reading the content taught by the present application, and these equivalent forms also fall within the scope defined by the appended claims of the present application.

[0034] The embodiment of the present application relates to a multi-view fusion sedimentation tank sludge three-dimensional reconstruction method, mainly including image acquisition and preprocessing, panorama image splicing and fusion, sludge height calculation and sludge three-dimensional reconstruction.

[0035] More specifically, as shown in Figure 1 , comprising the following steps:

[0036] A plurality of elevation lines are arranged horizontally along the bottom of the sedimentation tank upwards in sequence around the pool wall of the sedimentation tank.

[0037] Sidelight pictures containing the elevation lines of the sedimentation tank under different viewing angles are synchronously collected and preprocessed;

[0038] The preprocessed sidelight pictures are spliced into a panorama picture of the sedimentation tank;

[0039] The intersection line contour of the sludge and the pool wall of the sedimentation tank is extracted from the panorama picture, and the height component of the sludge and supernatant interface point cloud is derived as the sludge surface height based on the elevation line;

[0040] The sludge surface displacement is estimated, and then the sludge surface contour is predicted by using a deep learning model according to the sludge surface displacement, the sludge surface height and the extracted image features;

[0041] The sludge surface contour is three-dimensionally reconstructed.

[0042] The intersection line contour of the sludge and the pool wall of the sedimentation tank can be extracted by the following method:

[0043] An edge map of the intersection of the sludge and the pool wall of the sedimentation tank is extracted by using an edge detection algorithm;

[0044] The obtained edge map is subjected to binary segmentation, and the continuous closed contour is searched in the segmented binary image to obtain the intersection line contour of the sludge and the pool wall of the sedimentation tank.

[0045] The edge detection algorithm can adopt the Canny algorithm, and in the scene with high image contrast and less noise, the gradient can also be calculated by the Sobel operator, and the edge is segmented by the adaptive threshold value combined with the Otsu algorithm. For complex environment, a neural network model such as U-Net can also be used to directly output the edge probability map by inputting the RGB image.

[0046] In some preferred embodiments, the obtained edge map can also be denoised before binary segmentation, and the morphological opening operation method can be used to eliminate the noise of the edge map.

[0047] When deriving the height component of the sludge and supernatant interface point cloud, the following method can be used:

[0048] Taking the elevation line as a reference, the intersection line profiles under different visual angles are matched to calculate the three-dimensional point cloud coordinates of the intersection line;

[0049] Based on the three-dimensional point cloud coordinates of the intersection line, a plurality of mesh surfaces are gradually deduced and constructed from the periphery of the pool body to the center of the pool body until the complete interface of the sludge and the clear liquid is obtained.

[0050] The center height of each mesh surface is calculated as the height component of the interface point cloud.

[0051] In the matching process, SIFT or SURF algorithm can be selected. Taking SIFT algorithm (scale invariant feature transform) as an example, the function is to extract the stable feature points of the sludge line and the elevation line in the multi-view image, and realize the accurate matching of the cross-view pixel coordinates. In the construction of the interface, Delaunay triangulation, greedy triangulation (Greedy Triangulation) and Watson algorithm can be used, and appropriate geometric operation methods are used to calculate the sludge height according to the selected three-dimensional reconstruction algorithm.

[0052] The deep learning model used to predict the sludge surface profile can adopt a multi-layer linear perception network and a Transformer network with a multi-head self-attention mechanism. This model can be used to predict the sludge surface profile. In addition, a fully connected layer can be added at the output end of the Transformer network, and a SoftMax activation function can be combined to further predict the sludge swelling probability, thereby helping to better plan the cleaning strategy.

[0053] A preferred embodiment 1 of the present application relates to a multi-view fusion three-dimensional reconstruction device for a sedimentation tank sludge.

[0054] As shown in Figure 2 The device includes a visual reference system, an image acquisition system and an image processing module. The visual reference system is composed of annularly distributed elevation lines 3, which are used to provide elevation reference. The image acquisition system is composed of an industrial camera 2 integrated with an infrared fill light, which is deployed on the four corners of the sedimentation tank 1, and is used to capture real-time images of the sludge interface elevation. The image processing module is composed of an edge controller 4, which is used for local processing of the collected image information, and completes the three-dimensional reconstruction of the sludge through a model algorithm.

[0055] The elevation line is made of SS304 stainless steel laser etching process, the surface is plated with hard chromium layer to enhance wear resistance, and is distributed annularly from the bottom of the sedimentation tank upward, one every 20 cm, and a total of 5. The bottom of the elevation line is flush with the bottom of the tank, and after installation, it needs to be calibrated horizontally to ensure that the error is ≤1 mm / m, and the distribution accuracy is verified by a laser level. Preferably, the width of the elevation line can be set to 5 cm, and after installation, it protrudes 2 cm from the tank wall. The elevation line can be fixed to the tank wall by M12x150 expansion bolts, and the bolt spacing is ≤30 cm.

[0056] The industrial camera is connected to the image processing module by a data transmission line. The industrial camera uses a 1 / 1.7-inch CMOS sensor with a resolution of 12 million pixels, supports a 16:1 wide dynamic range, and integrates an 850nm infrared fill light powered by PoE++. The industrial camera is externally protected by a polycarbonate shield with a hydrophobic coating to reduce water mist adhesion, with an overall protection rating of IP68. The infrared fill light has a power of 15W and a wavelength deviation of ±10nm, and is equipped with a narrowband filter with a bandwidth of ±5nm.

[0057] There are four industrial cameras, which are fixed to the four corners of the sedimentation tank by a three-axis gimbal, and are fixed to the three-axis gimbal by screws. The three-axis gimbal is directly fixed to the top of the tank wall by expansion bolts, or is fixed to the top of the tank wall by a bracket. The three-axis gimbal has a control range of pitch ±15° and horizontal ±30°, and uses IEEE 1588 protocol synchronization.

[0058] The edge controller integrates PLC, PC and motion controller, and has data processing and logic control functions, which can perform image preprocessing, panoramic stitching and fusion, sludge height calculation, dynamic compensation and volume calculation and output on the data collected by the industrial camera.

[0059] A preferred embodiment 2 of the present application relates to a multi-view fusion sedimentation tank sludge three-dimensional reconstruction method, which is applied to a sludge treatment scene as shown in Figure 3 The four walls of the sedimentation tank 1 are annularly distributed with multiple elevation lines for providing elevation reference. The industrial cameras 2 are deployed at the four corners of the sedimentation tank 1 for capturing real-time images of the sludge interface elevation. The edge controller 4 is composed of a local image information processing device, which completes the three-dimensional reconstruction of the sludge through a model algorithm.

[0060] As shown in Figure 4 The algorithm includes image preprocessing, panoramic stitching and fusion, sludge height calculation, dynamic compensation and volume calculation and output.

[0061] Image preprocessing includes distortion correction and color consistency optimization, mainly used to eliminate the original image distortion and color deviation caused by the difference in shooting angle:

[0062] 1) Distortion correction is based on Brown-Conrady model (12 parameters), using OpenCV's undistort() function to correct radial / tangential distortion, the calibration board uses a 7x9 grid checkerboard (grid spacing 10 cm), and the calibration error is ≤0.3 pixels.

[0063] 2) Color consistency optimization is based on LAB color space, using Histogram Matching algorithm, combined with camera color response curve (CRF) for white balance optimization, ΔE < 3 (CIEDE2000 standard).

[0064] Panoramic stitching and fusion includes feature matching and seamless fusion, mainly used for image splicing of four cameras to achieve natural transition in overlapping parts:

[0065] 1) Feature matching uses DINOv2 model based on self-supervised training to extract image features, and combines multi-layer perception network to align the features extracted by the four cameras;

[0066] 2) Seamless fusion uses Multi-Resolution Blending technology to perform weighted fusion on the Laplacian pyramid (5 layers), with a transition zone gradient change of <5%.

[0067] The core step of the embodiment is to accurately predict the sludge surface profile through sludge height calculation and dynamic compensation mechanism, and the whole process is as shown in Figure 5 .

[0068] Among them, sludge height calculation includes edge detection and height deduction, mainly based on the reference of the elevation line to continuously deduce the sludge height at each position from the four sides of the pool body to the center of the pool body:

[0069] 1) Edge detection based on Canny algorithm (Gaussian filter σ=1.2) to extract the sludge boundary, combined with morphological opening operation (3x3 elliptical kernel) to eliminate noise;

[0070] 2) Height deduction based on the sludge boundary obtained in the previous step, after improving the image contrast, the binary image is obtained by threshold segmentation, and the sludge line pixel coordinates composed of continuous contours are obtained by using the findContours() function of OpenCV, and then the SIFT is combined with the elevation line reference to match the pixel coordinates of the sludge line in each view, and finally the camera parameters are used to calculate the three-dimensional space coordinates of the sludge surface edge by using the Linear-Eigen Method; Finally, based on the sludge edge point cloud, a grid surface is constructed by Delaunay triangulation, and the height of the grid triangle vertex is obtained by bilinear interpolation, and the height measurement error is ≤±0.5 cm.

[0071] Dynamic compensation includes motion blur suppression and adaptive exposure, which can make preliminary prediction during continuous operation of the system, thereby reducing the computational load of the sludge height calculation and providing a reference basis for calibrating the results of the sludge height calculation:

[0072] 1) Motion blur suppression uses optical flow method (Lucas-Kanade) to estimate sludge surface displacement, and the image features extracted in the foregoing step are combined with the sludge height to jointly encode through a multi-layer linear perception network, and then a Transformer encoding network based on multi-head self-attention mechanism is used to predict the sludge contour, and a layer of fully connected network is added to combine the SoftMax activation function to predict the sludge swelling probability;

[0073] 2) Adaptive exposure dynamically adjusts the camera exposure based on YUV histogram (step size 1 / 3 EV), to ensure that the brightness of the sludge area is maintained at 120±20 IRE.

[0074] Volume calculation and output includes three-dimensional reconstruction and data output, which can calculate the real-time sludge volume in the sedimentation tank based on the results of the foregoing steps, and output the corresponding results:

[0075] 1) Three-dimensional reconstruction uses a sparse view reconstruction method based on 3D Gaussian sputtering, and after the grid is extracted from the explicit radiation field representation using the Marching Cubes algorithm, it is optimized through Laplacian smoothing and hole filling to form a watertight sludge grid, and then the volume is calculated through integration method:

[0076]

[0077] wherein is the number of triangles in the grid, , and are the coordinates (vector form) of the three vertices of the th triangle.

[0078] 2) Data output outputs the sludge volume (precision 0.01 m 3 ), distribution heat map and abnormal area coordinates (such as local accumulation >1.5 m) in JSON format through Modbus TCP protocol.

[0079] In this embodiment, by the fusion application of multispectral imaging and three-dimensional reconstruction technology, the measurement accuracy of sludge height ±0.5cm and the error control of volume calculation ≤±3% are realized, which has a significant improvement compared with the traditional ultrasonic measurement accuracy. The system uses 850nm infrared fill light and wide dynamic range imaging technology, which can still work stably in turbid water (transmittance <5%) and low light environment (10lux). Combined with dynamic optical flow compensation algorithm, the motion blur effect can be reduced to pixel level error (<0.1mm). Four camera wide-angle array cooperates with edge computing architecture to realize 100% coverage monitoring of the sedimentation tank, and the single scanning time is compressed to 1.8 seconds, which greatly improves the efficiency of traditional manual inspection. The built-in machine learning module can automatically identify sludge swelling (accuracy 92%) and abnormal accumulation, and the heat map visualization can assist decision making, which reduces the sludge energy consumption by 15%-20%. The non-contact measurement design prolongs the equipment maintenance period to 5 years (traditional sensor needs quarterly maintenance), and IP68 protection level ensures stable operation in harsh environments such as pH2-11 and humidity 100%.

Claims

1. A multi-view fusion sedimentation tank sludge three-dimensional reconstruction method, characterized in that, The method comprises the following steps: a pool wall around the sedimentation tank, a plurality of elevation lines arranged horizontally along the bottom of the sedimentation tank in turn upwards; synchronously collecting side-view pictures of the sedimentation tank at different angles and containing the elevation lines and pre-processing the pictures; splicing the pre-processed side-view pictures into a panorama picture of the sedimentation tank; extracting the contour of the interface between the sludge and the pool wall from the panorama picture, taking the elevation lines as the reference, deriving the height component of the point cloud of the interface between the sludge and the clear liquid as the sludge surface height; estimating the sludge surface displacement, and then predicting the sludge surface contour by using a deep learning model according to the sludge surface displacement, the sludge surface height and the multi-angle image features extracted from the side-view pictures at different angles; reconstructing the sludge surface in three dimensions based on the sludge surface contour.

2. The method of claim 1, wherein, The step of taking the elevation lines as the reference and deriving the height component of the point cloud of the interface between the sludge and the clear liquid comprises the following steps: taking the elevation lines as the reference, matching the contours of the interface at different angles, and calculating the three-dimensional point cloud coordinates of the interface; based on the three-dimensional point cloud coordinates of the interface, gradually deducing and constructing a plurality of mesh surfaces from the periphery of the pool body to the center of the pool body until the complete interface between the sludge and the clear liquid is obtained; calculating the center height of each mesh surface as the height component of the point cloud of the interface.

3. The method of claim 2, wherein, The step of gradually deducing and constructing a plurality of mesh surfaces from the periphery of the pool body to the center of the pool body is realized by a triangulation algorithm.

4. The method of claim 3, wherein, The step of calculating the center height of each mesh surface is realized by bilinear interpolation based on the height of the vertices of the mesh triangle.

5. The method of claim 1, wherein, The step of extracting the contour of the interface between the sludge and the pool wall comprises the following steps: extracting the edge map of the interface between the sludge and the pool wall by using an edge detection algorithm; performing binary segmentation on the obtained edge map, finding the continuous closed contour in the segmented binary image, and obtaining the contour of the interface between the sludge and the pool wall.

6. The method of claim 5, wherein, Before the step of performing binary segmentation on the obtained edge map, the method of morphological opening operation is further included to eliminate the noise of the edge map.

7. The method of claim 1, wherein, The step of splicing the pre-processed side-view pictures into a panorama picture of the sedimentation tank is realized by extracting the features of the side-view pictures at different angles and aligning the extracted multi-angle image features.

8. The method of claim 1, wherein, The deep learning model is constructed based on a multi-layer linear perception machine and a Transformer network with a multi-head self-attention mechanism, and a fully connected layer is added at the output end of the Transformer network, combined with a SoftMax activation function to predict the sludge swelling probability.

9. The method of claim 1, wherein, The step of collecting side-view pictures of the sedimentation tank at different angles and containing the elevation lines is realized by deploying industrial cameras at the four corners of the sedimentation tank.

Images