Method and device for three-dimensional reconstruction of tidal bore based on binocular camera

Abstract

The invention discloses a method and a device for three-dimensional reconstruction of an eagre based on a binocular camera. The method comprises the following steps: S1, determining the position of an eagre monitoring high tower according to an eagre three-dimensional reconstruction area, and determining the camera position and lens parameters of the binocular camera; s2, constructing a target range, circularly moving and shooting the target range to obtain a calibration image, associating pixel coordinates of a target point in the calibration image with physical coordinates of a target in the target range, and calculating to obtain camera internal parameters of the binocular camera; s3, deploying two binocular cameras, and calculating by adopting a parallax method to obtain camera external parameters of the two binocular cameras respectively; and S4, the two binocular cameras periodically collect images at the same time until the tidal bore ends, camera internal reference calibration is used for carrying out distortion correction on the collected images, three-dimensional point cloud is densely reconstructed, and after precision evaluation is carried out on a three-dimensional point cloud area, adjustment is carried out and a tidal bore three-dimensional reconstruction model is output. According to the invention, the problem of shooting the tidal bore image at the same time can be solved, and high-precision three-dimensional reconstruction of the tidal bore form is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of tidal bore monitoring, and in particular to a method and device for three-dimensional reconstruction of tidal bore based on a binocular camera. Background Art

[0002] As a spectacular and complex hydrological phenomenon in nature, tidal bore monitoring is extremely necessary. When a tidal bore occurs, the advance of the tide poses a serious threat to infrastructure such as embankments, bridges, and docks on both sides of the river, and also affects the safety of life and property of surrounding residents. Through continuous and accurate monitoring, the dynamic changes of tidal bores can be grasped in a timely manner, providing key data support for the tidal bore warning system and effectively reducing the losses caused by natural disasters. In addition, tidal bore monitoring can also help scientific research, deepen the understanding of tidal laws, and provide a scientific basis for the comprehensive management and development of estuary areas.

[0003] Traditional tidal bore monitoring mainly relies on single-point water level monitoring. Since tidal bore is a large-scale uneven hydrodynamic event, single-point water level monitoring of tidal bore water level cannot determine the morphology of the tidal bore area and cannot meet the requirements for accurate assessment of the destructive power of tidal bore, resulting in a significant reduction in early warning results. At the same time, it is unable to provide more detailed tidal bore morphology data for scientific research, and cannot guarantee the reliability and accuracy of research results.

[0004] In recent years, with the development of radar 3D monitoring technology of various bands, high-precision 3D reconstruction has become popular. However, due to the strong absorption and scattering of radar waves by tidal bores, the signal attenuation is serious, making it difficult to obtain accurate tidal bore morphological information. Therefore, radars of various bands cannot effectively monitor it, resulting in unsatisfactory 3D reconstruction of tidal bores by radars. Binocular vision 3D imaging technology shows significant potential in tidal bore 3D reconstruction monitoring and can effectively address the shortcomings of existing technologies.

[0005] However, the existing binocular camera products on the market have limited baseline distance, which limits the monitoring distance and makes it impossible to guarantee accuracy, making it impossible to achieve long-distance three-dimensional reconstruction of tidal bores. How to monitor tidal bore shapes with high precision remains a technical problem that needs to be solved urgently. Summary of the invention

[0006] In order to solve the problem of insufficient tidal bore monitoring accuracy in the prior art, the present invention proposes a method and device for three-dimensional reconstruction of tidal bore based on a binocular camera.

[0007] The specific technical solution is as follows: A method for three-dimensionally reconstructing tidal bores based on a binocular camera, the steps comprising:

[0008] S1: Determine the location of the tidal bore monitoring tower according to the 3D reconstruction area of ​​the tidal bore, and determine the camera position and lens parameters of the binocular camera according to the 3D reconstruction area of ​​the tidal bore and the location of the tidal bore monitoring tower;

[0009] S2: Build a target range, obtain the physical coordinates of the target, cyclically move and shoot the target range according to the camera position parameters and lens parameters to obtain a calibration image, write the pixel coordinates of the target point in the calibration image into the reference coordinate system of the first mobile shooting, associate the pixel coordinates of the target point in the calibration image with the physical coordinates of the target in the target range, and calculate the camera intrinsic parameters of the binocular camera;

[0010] S3: Deploy two binocular cameras, measure three common view calibration points within the image range of the two binocular cameras through a total station, and use the parallax method to calculate the camera extrinsic parameters of the two binocular cameras respectively;

[0011] S4: Two binocular cameras simultaneously and periodically collect images until the end of the tidal bore. The collected images are calibrated using camera intrinsic parameters to correct distortion. The 3D point cloud is densely reconstructed based on the camera intrinsic parameters and camera extrinsic parameters. The accuracy of the 3D point cloud area is evaluated and the output tidal bore 3D reconstruction model is adjusted.

[0012] Furthermore, the acquisition of camera internal parameters specifically includes:

[0013] Perform cyclic moving shooting according to the camera position parameters and lens parameters, and stitch all the captured images to obtain the calibration image;

[0014] Obtain the physical position of the binocular camera during each mobile shooting and calculate the rotation parameters. Use the transformation relationship to uniformly write the pixel coordinates of the target points in all captured images into the reference coordinate system of the first mobile shooting.

[0015] The pixel coordinates of the target points in the calibration image are associated with the physical coordinates of the target in the target field, and the distortion coefficient and intrinsic parameter matrix of the binocular camera are obtained by calculating the distortion coefficient.

[0016] Furthermore, the calculation of the rotation parameters includes: measuring the coordinates C1 of the binocular camera and the coordinates P1 of the target point corresponding to the center point of the image by a total station;

[0017] Match the center point of the image and find the coordinate P1 of the corresponding target point in the world coordinate system;

[0018] The x-axis direction of the initial coordinate system is defined by the vector from the camera position of the binocular camera to the target point:

[0019] ;

[0020] ;

[0021] in, Represents the vector from the camera position to the target point, express The normalized vector points to the x-axis vector of the reference coordinate system;

[0022] In vertical The vertical direction in the plane is selected as the y-axis direction, which is recorded as , the z-axis direction is calculated by cross product: , get the rotation matrix parameters of the first shot: .

[0023] Furthermore, the calculation of the distortion coefficient includes:

[0024] The position information of the target in the image is extracted through the image processing algorithm, and matched with the coordinates of the target point in the world coordinate system to obtain the homography matrix of the mapping relationship between the calibration image and the target field plane;

[0025] The distortion coefficient and intrinsic parameter matrix of the binocular camera are reversely derived through the coordinates of the calibration points and their projection coordinates in the calibration image.

[0026] Furthermore, the deployment of the binocular camera includes: fixing two binocular cameras at two ends of a camera bracket respectively, and installing the camera bracket on a tidal bore monitoring tower.

[0027] Furthermore, collecting images specifically includes:

[0028] The industrial computer system of the tidal bore monitoring tower uses a precise time synchronization protocol to synchronize the time of the two binocular cameras so that they are always consistent;

[0029] According to the periodic collection requirements, the industrial controller continuously sends the same image collection instructions to the two binocular cameras. After receiving the instructions, the binocular cameras take pictures according to the instruction time.

[0030] Furthermore, dense reconstruction of 3D point cloud includes:

[0031] The PatchMatch algorithm is used to estimate the disparity of each pixel in the two view images based on the camera intrinsic parameters and camera extrinsic parameters. For each pixel, the disparity with the minimum matching cost among the pixels in the domain is used to replace the disparity of the current pixel.

[0032] Measure the similarity between two patches, use triangulation to convert the disparity of pixels into depth information, and obtain a dense 3D point cloud.

[0033] Furthermore, the accuracy assessment includes:

[0034] Conduct boat verification in the tidal bore 3D reconstruction area, place RTK equipment and targets on the boat, set multiple control points in the tidal bore 3D reconstruction area, sail to each control point and stop, record the time interval between the first and last stop points and continuously record RTK values, and use two binocular cameras to simultaneously capture multiple sets of images at equal time intervals;

[0035] Compare the target positions in multiple groups of images, calculate the average pixel deviation of the target position in the image, intercept all RTK values ​​in the time interval, and calculate the RTK average value;

[0036] Determine whether the absolute value of the pixel average deviation and the difference between all RTK values ​​and the RTK average value are both less than the threshold. If all the judgment conditions are met, perform 3D reconstruction on the captured image.

[0037] The RTK average value of all control points that meet the judgment conditions and the 3D reconstructed coordinates of the binocular camera are obtained, and the mean absolute error, mean height error and mean horizontal error are obtained by comparing the RTK average value and the 3D reconstructed coordinates.

[0038] A device for three-dimensional reconstruction of tidal bore based on binocular cameras is applicable to a method for three-dimensional reconstruction of tidal bore based on binocular cameras, comprising: two binocular cameras and a tidal bore monitoring tower, wherein the tidal bore monitoring tower is arranged on a seawall, a camera bracket is arranged on the tidal bore monitoring tower, the two binocular cameras are respectively arranged at two ends of the camera bracket, and the tidal bore three-dimensional reconstruction area is smaller than the overlapping area of ​​the shooting range of the two binocular cameras.

[0039] Furthermore, a distribution box is provided at the bottom of the tidal bore monitoring tower, and a power supply, an industrial computer and a network router are provided in the distribution box. The industrial computer and the network router are electrically connected to the power supply, an image acquisition card is provided on the industrial computer, the industrial computer is communicatively connected to two binocular cameras, and the power supply is electrically connected to the two binocular cameras.

[0040] The above technical solution has the following advantages or technical effects:

[0041] 1. The present invention adopts binocular camera to reconstruct the tidal bore morphology in three dimensions, thus improving the tidal bore monitoring from single point to surface, and filling the gap in tidal bore morphology monitoring.

[0042] 2. The present invention improves the image quality through camera intrinsic calibration and distortion correction. The baseline distance between the two binocular cameras is increased through the self-developed integrated camera bracket design. At the same time, the accuracy of camera extrinsic parameter calculation is guaranteed through field calibration point measurement. The combination of multiple methods improves the accuracy of tidal bore three-dimensional reconstruction.

[0043] 3. The present invention adopts a precise time synchronization protocol to achieve microsecond-level time synchronization, realizes the simultaneous acquisition of tidal bore images by dual cameras, and solves the problem of flow changes at the tidal bore moment.

[0044] 4. The present invention adopts dense reconstruction of three-dimensional point clouds to improve the point cloud density of the three-dimensional reconstructed sparse point clouds of partial images, thereby improving the availability of the tidal bore three-dimensional reconstruction point cloud.

[0045] 5. The present invention adopts the accuracy verification by sailing to ensure the accuracy of the point cloud and ensure that the reconstructed model meets the accurate monitoring standards. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of the method of the present invention;

[0047] Figure 2 is the image taken by the left-view binocular camera of the present invention;

[0048] Figure 3 is the image taken by the right-view binocular camera of the present invention;

[0049] Figure 4 is a distortion correction diagram of an image shot from a left viewing angle of the present invention;

[0050] Figure 5 is a distortion correction diagram of an image taken from the right perspective of the present invention;

[0051] Figure 6 It is a tidal point cloud distribution diagram of the present invention from a bird's-eye view;

[0052] Figure 7 It is a schematic diagram of the position of the pole of the device of the present invention;

[0053] Figure 8 It is a schematic diagram of the device of the present invention.

[0054] Marked in the attached figure: 1-binocular camera, 2-camera bracket, 3-tidal bore monitoring tower, 4-seawall, 5-river surface, 6-tidal bore 3D reconstruction area, 7-camera network cable, 8-camera power cable, 9-distribution box, 10-industrial computer, 11-image acquisition card, 12-power supply, 13-network router. DETAILED DESCRIPTION

[0055] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] Example 1

[0057] like Figure 1 As shown, a method for three-dimensionally reconstructing a tidal bore based on a binocular camera comprises the following steps:

[0058] S1: Determine the location of the tidal bore monitoring tower according to the 3D reconstruction area of ​​the tidal bore, and determine the camera position and lens parameters of the binocular camera according to the 3D reconstruction area of ​​the tidal bore and the location of the tidal bore monitoring tower;

[0059] S2: Build a target range, obtain the physical coordinates of the target, cyclically move and shoot the target range according to the camera position parameters and lens parameters to obtain a calibration image, write the pixel coordinates of the target point in the calibration image into the reference coordinate system of the first mobile shooting, associate the pixel coordinates of the target point in the calibration image with the physical coordinates of the target in the target range, and calculate the camera intrinsic parameters of the binocular camera;

[0060] S3: Deploy two binocular cameras, measure three common view calibration points within the image range of the two binocular cameras through a total station, and use the parallax method to calculate the camera extrinsic parameters of the two binocular cameras respectively;

[0061] S4: Two binocular cameras simultaneously and periodically collect images until the end of the tidal bore. The collected images are calibrated using camera intrinsic parameters to correct distortion. The 3D point cloud is densely reconstructed based on the camera intrinsic parameters and camera extrinsic parameters. After the accuracy of the 3D point cloud area is evaluated, the tidal bore 3D reconstruction model is adjusted and output.

[0062] Step S1 specifically includes:

[0063] S11: according to the selected tidal bore 3D reconstruction area, determine the position of the tidal bore monitoring tower so that when the tidal bore arrives, the tidal bore monitoring tower will not shake and is located at the place closest to the river surface;

[0064] S12: according to the 3D reconstruction area of ​​the tidal bore and the position of the tidal bore monitoring tower, a binocular camera for overlooking shooting is set up to cover the 3D reconstruction area of ​​the tidal bore with its shooting range, and the camera position of the binocular camera is determined. The camera position includes a camera shooting angle and a shooting distance L. The camera shooting angle ensures that the shooting range covers the 3D reconstruction area of ​​the tidal bore while overlooking the river surface. The shooting distance L is the physical distance between the binocular camera and the center point of the captured image.

[0065] S13: adjusting the focal length of the camera lens according to the camera shooting angle and the shooting distance L so that the 3D reconstruction area of ​​the tidal bore accounts for the largest pixel proportion in the captured image, and adjusting the aperture of the camera lens to ensure clear image formation, and determining the lens parameters of the binocular camera, the lens parameters including the focal length f of the camera lens and the aperture of the camera lens, wherein ensuring clear image formation is achieved by adjusting the aperture of the camera lens and observing the image sharpness and contrast change curves for comprehensive judgment.

[0066] Among them, the height of the tidal bore monitoring tower is 25 meters.

[0067] Camera intrinsic parameter calibration: Zhang’s calibration method is used to calibrate the camera’s intrinsic parameters. A target field is built and the physical coordinates of the target in the target field are obtained by measurement. The target field is a uniformly distributed target field (such as a chessboard or dot array). A total station is used to measure the pixel coordinates of each target point.

[0068] Example 2

[0069] Get the camera internal parameters:

[0070] According to the shooting distance L between the binocular camera and the target field and the focal length f of the binocular camera lens, circular moving shooting is performed, the reference coordinate system during the first moving shooting is recorded, and all the captured images are stitched together to obtain a calibration image, wherein the circular moving shooting refers to obtaining an image set by moving the binocular camera to cyclically shoot the target field, and then using the image set to stitch together a calibration image of the target field covering the entire visual field, so as to ensure the accuracy of the distortion coefficient calculation; the reference coordinate system is the position of the target point corresponding to the binocular camera position and the image center during the first moving shooting, and a reference coordinate system is defined, which will be used as the reference for all calculations in subsequent shooting. The coordinate system defines the x-axis direction with the vector from the binocular camera position to the target point, and uses the plumb direction as the y-axis direction. The target point corresponding to the image center is measured by moving the moving target to the image center point;

[0071] Obtain the physical position of the binocular camera at each shooting and calculate the rotation parameters, and use the transformation relationship to uniformly write the pixel coordinates of the target points in all the captured images into the reference coordinate system of the first mobile shooting;

[0072] The pixel coordinates of the target points in the calibration image are associated with the physical coordinates of the target in the target field, and the distortion coefficient and intrinsic parameter matrix of the binocular camera are obtained by calculating the distortion coefficient.

[0073] The calculation of the rotation parameters includes: measuring the coordinates C1 of the binocular camera and the coordinates P1 of the target point corresponding to the center point of the image by the total station,

[0074] in, , Represents the coordinates of coordinate C1 on the x-axis, y-axis and z-axis;

[0075] Match the center point of the image and find the coordinate P1 of the corresponding target point in the world coordinate system.

[0076] in, , Represents the coordinates of coordinate P1 on the x-axis, y-axis and z-axis;

[0077] The x-axis direction of the initial coordinate system is defined by the vector from the camera position of the binocular camera to the target point:

[0078] ;

[0079] ;

[0080] in, Represents the vector from the camera position to the target point, express The normalized vector points to the x-axis vector of the reference coordinate system;

[0081] In vertical The vertical direction in the plane is selected as the y-axis direction, which is recorded as , the z-axis direction is calculated by cross product: , get the rotation matrix parameters of the first shot: .

[0082] The calculation of the distortion coefficient includes:

[0083] The position information of the target in the image is extracted through the image processing algorithm, and matched with the coordinates of the target point in the world coordinate system to obtain the homography matrix of the mapping relationship between the calibration image and the target field plane;

[0084] Using nonlinear least squares optimization, the distortion coefficients and intrinsic parameter matrix of the binocular camera are reversely derived through the coordinates of the calibration points and their projection coordinates in the calibration image. The distortion coefficients mainly include radial distortion coefficients k1, k2, k3, etc. and tangential distortion coefficients p1 and p2.

[0085] Example 3

[0086] Calculate camera extrinsic parameters: Obtain camera extrinsic parameters through camera installation and deployment and field calibration point measurement to obtain the extrinsic parameter matrix.

[0087] Two binocular cameras are fixed at both ends of the bracket through an integrated camera bracket, the camera bracket is installed on the tower, the network cable and power cable are connected to the industrial computer at the bottom of the tower, and the camera shooting and three-dimensional reconstruction are controlled by the system on the industrial computer. At the same time, the camera bracket ensures that the two cameras are fixed and can only rotate up and down and move left and right at the same time. Such hardware design ensures that the relative external parameters of the two cameras do not change, reduces the error caused by measurement, and improves accuracy. In conventional acquisition and shooting, the camera setting usually adopts single-point shooting or places the camera in a similar position. When dealing with tidal bores, such settings cannot accurately monitor the tidal bore dynamics tens of meters away, and are prone to errors. The camera bracket of the present invention is 6 meters long, and the baseline distance is 6 meters. By shooting simultaneously with two binocular cameras in a relatively far baseline range, the accuracy of long-distance three-dimensional reconstruction can be improved, and the baseline length can ensure that the two binocular cameras act synchronously, further reducing monitoring errors.

[0088] Field calibration point measurement is to measure three common view calibration points within the image range of two binocular cameras through a total station. The spatial physical position and rotation direction of the two cameras are calculated separately using the parallax method through the calibration points. Since the hardware design ensures that the relative external parameters of the two cameras do not change, averaging the rotation direction can reduce the measurement error and thus obtain an accurate external parameter matrix.

[0089] Example 4

[0090] Collect images at the same time: When the tidal bore arrives, the system on the industrial computer starts the image acquisition function, and the two binocular cameras collect images simultaneously once a second, and the collection continues until the tidal bore ends.

[0091] The industrial computer system of the tidal bore monitoring tower uses a precise time synchronization protocol to synchronize the time of the two binocular cameras so that they are always consistent;

[0092] Verify the time synchronization of the two binocular cameras, issue a shooting command to make the two binocular cameras shoot at the same time, and run the microsecond timer to determine whether the error of the timing result is accurate to the microsecond level. If the error range exceeds the specified threshold, repeat the synchronization time operation until the timing result reaches the microsecond level.

[0093] The industrial controller continuously sends the same image acquisition instructions to the two binocular cameras according to the periodic acquisition requirements. After receiving the instructions, the left and right binocular cameras take pictures according to the instructions. The results are as follows: Figure 2 , Figure 3 As shown;

[0094] After completing the acquisition requirements, the industrial computer stops issuing image acquisition instructions.

[0095] Image distortion correction: The collected image is corrected using the distortion coefficients obtained by camera intrinsic calibration. The correction results are as follows: Figure 4 , Figure 5 shown.

[0096] Example 5

[0097] Dense reconstruction of 3D point cloud: Dense reconstruction of 3D point cloud is performed on the distortion-corrected image. The PatchMatch algorithm is used to perform dense matching reconstruction on the image using the intrinsic parameters of the stereo camera and the extrinsic parameters between the two stereo cameras. The PatchMatch algorithm estimates the disparity of each pixel in the multi-view image through three steps: random initialization, neighborhood propagation, and random search disparity correction. The Greedy Strategy is used in the neighborhood propagation of the PatchMatch algorithm, that is, the disparity of the current pixel will be replaced by the disparity with the lowest matching cost among its neighboring pixels.

[0098] Assume that the disparity of the current pixel p is , its neighboring pixels and The parallaxes are and In the process of neighborhood propagation, the algorithm compares the cost function of pixel p and and The size of the disparity is selected with the minimum cost as the new disparity value:

[0099] ;

[0100] In the cost calculation, the photometric consistency of the local area is used to measure the similarity between two patches. and are images from two perspectives, p and q represent pixels and their neighborhoods respectively, d is the disparity value, and the definition of photometric consistency is as follows:

[0101] ;

[0102] Through the internal and external parameters of the binocular camera, the triangulation method can be used to convert the disparity of the pixel points into depth information, and then obtain a dense point cloud, such as Figure 6 The tidal point cloud distribution map from a bird's-eye view can accurately monitor the point cloud height and distance from the shore.

[0103] Accuracy assessment of tidal bore 3D reconstruction area: Accuracy assessment is performed on the tidal bore 3D reconstruction area to obtain the average absolute error, average height error, and average horizontal error in the current area.

[0104] The accuracy assessment method is to verify by sailing in the tidal bore 3D reconstruction area, place RTK equipment and targets on the boat, and evenly set 20 control points in the tidal bore 3D reconstruction area when the water surface is calm. Sail to each control point and keep recording RTK values ​​and time. At the same time, use the binocular camera to shoot 10 times at the same time, once per second, and compare the target positions in 10 groups of images. If the average pixel deviation of the target position in the 10 groups of images is less than 2, the RTK value of this time period is intercepted at the same time, and the RTK average value is calculated. If the absolute value of all RTK values ​​minus the RTK average value is less than 2 cm, the data is available, and the captured images are reconstructed in 3D to calculate the physical coordinates of the control points. After 20 cycles, the RTK average value of the 20 control points and the physical coordinates of the 3D reconstruction of the binocular camera are obtained. The average absolute error, average height error, and average horizontal error are calculated by comparing the two coordinates.

[0105] The mean absolute error refers to the average error between the RTK measurement and the 3D reconstructed physical coordinates. The mean height error refers to the average error in height, and the mean horizontal error refers to the average error in the horizontal direction.

[0106] The judgment threshold of the accuracy assessment is 20 cm. If the mean absolute error of the accuracy assessment is greater than 20 cm, the tidal bore 3D reconstruction area is modified, the camera position and lens parameters are adjusted, and the operations of steps S2 to S4 are re-executed until the mean absolute error of the accuracy assessment does not exceed the specified threshold range, and the tidal bore 3D reconstruction model is output.

[0107] Taking the first shooting of the left-view binocular camera at the position of 20 control points as an example, it is recorded as camera 1. The control point information collected by camera 1 is shown in Table 1.

[0108] Table 1: Control point information collected by camera 1 for the first time

[0109]

[0110] The three-dimensional reconstruction is performed on all the collected control point information. The reconstruction results are shown in Table 2, taking the first shot of the 20 control point positions as an example;

[0111] Table 2: Reconstruction results of the first acquisition of information by camera 1

[0112]

[0113] The error calculation of the two sets of data is performed, and the calculation results are shown in Table 3;

[0114] Table 3: Error calculation results

[0115]

[0116] The average absolute error in Table 3 is 8.1 cm, which is less than 20 cm. Therefore, the model accuracy meets the requirements. Since then, the reconstruction of the three-dimensional tidal bore model has been completed.

[0117] Example 6

[0118] like Figure 7 , Figure 8 As shown, a device for three-dimensional reconstruction of tidal bore based on binocular cameras comprises: two binocular cameras 1 and a tidal bore monitoring tower 3, the tidal bore monitoring tower 3 is arranged on the seawall 4, and is located at a position that will not shake when the tidal bore arrives and is closest to the river surface 5, a camera bracket 2 is arranged on the tidal bore monitoring tower 3, two binocular cameras 1 are respectively arranged at both ends of the camera bracket 2, and the tidal bore three-dimensional reconstruction area 6 is smaller than the overlapping area of ​​the shooting range of the two binocular cameras 1. The tidal bore monitoring tower is 25 meters, and the camera bracket is 6 meters.

[0119] A distribution box 9 is provided at the bottom of the tidal bore monitoring tower 3, and a power supply 12, an industrial computer 10 and a network router 13 are provided in the distribution box 9. The industrial computer 10 and the network router 13 are electrically connected to the power supply 12, and the industrial computer 10 is communicatively connected to the network router 13. An image acquisition card 11 is provided on the industrial computer 10. The industrial computer 10 is communicatively connected to the two binocular cameras 1 through a camera network cable 7 for controlling the two binocular cameras 1 to achieve precise synchronization. The power supply 12 is electrically connected to the two binocular cameras 1 through a camera power line 8.

[0120] The present invention adopts binocular vision 3D imaging technology and self-developed devices, solves the image distortion problem through camera internal parameter calibration and image distortion correction, solves the simultaneous shooting problem of tidal bore images through microsecond timing, and solves the sparse point cloud problem of 3D reconstruction of a small number of images through dense 3D point cloud reconstruction. Finally, a high-precision 3D point cloud model is constructed to achieve high-precision 3D reconstruction of the tidal bore morphology.

[0121] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for three-dimensional reconstruction of tidal bore based on binocular camera, characterized in that the steps include: S1: Determine the location of the tidal bore monitoring tower according to the 3D reconstruction area of ​​the tidal bore, and determine the camera position and lens parameters of the binocular camera according to the 3D reconstruction area of ​​the tidal bore and the location of the tidal bore monitoring tower; S2: Build a target range, obtain the physical coordinates of the target, cyclically move and shoot the target range according to the camera position parameters and lens parameters to obtain a calibration image, write the pixel coordinates of the target point in the calibration image into the reference coordinate system of the first mobile shooting, associate the pixel coordinates of the target point in the calibration image with the physical coordinates of the target in the target range, and calculate the camera intrinsic parameters of the binocular camera; S3: Deploy two binocular cameras, measure three common view calibration points within the image range of the two binocular cameras through a total station, and use the parallax method to calculate the camera extrinsic parameters of the two binocular cameras respectively; S4: Two binocular cameras simultaneously and periodically collect images until the end of the tidal bore. The collected images are calibrated using camera intrinsic parameters to correct distortion. The 3D point cloud is densely reconstructed based on the camera intrinsic parameters and camera extrinsic parameters. After the accuracy of the 3D point cloud area is evaluated, the tidal bore 3D reconstruction model is adjusted and output.

2. The method for three-dimensional reconstruction of tidal bore based on binocular camera according to claim 1, characterized in that: The acquisition of the camera internal parameters specifically includes: Perform cyclic moving shooting according to the camera position parameters and lens parameters, and stitch all the captured images to obtain the calibration image; Obtain the physical position of the binocular camera during each mobile shooting and calculate the rotation parameters. Use the transformation relationship to uniformly write the pixel coordinates of the target points in all captured images into the reference coordinate system of the first mobile shooting. The pixel coordinates of the target points in the calibration image are associated with the physical coordinates of the target in the target field, and the distortion coefficient and intrinsic parameter matrix of the binocular camera are obtained by calculating the distortion coefficient.

3. The method for three-dimensional reconstruction of tidal bore based on binocular camera according to claim 2, characterized in that: The calculation of the rotation parameters includes: measuring the coordinates C1 of the binocular camera and the coordinates P1 of the target point corresponding to the center point of the image by a total station; Match the center point of the image and find the coordinate P1 of the corresponding target point in the world coordinate system; The x-axis direction of the initial coordinate system is defined by the vector from the camera position of the binocular camera to the target point: ； ； in, Represents the vector from the camera position to the target point, express The normalized vector points to the x-axis vector of the reference coordinate system; In vertical The vertical direction in the plane is selected as the y-axis direction, which is recorded as , the z-axis direction is calculated by cross product: , get the rotation matrix parameters of the first shot: .

4. The method for three-dimensional reconstruction of tidal bore based on binocular camera according to claim 3, characterized in that: The calculation of the distortion coefficient includes: The position information of the target in the image is extracted through the image processing algorithm, and matched with the coordinates of the target point in the world coordinate system to obtain the homography matrix of the mapping relationship between the calibration image and the target field plane; The distortion coefficient and intrinsic parameter matrix of the binocular camera are reversely derived through the coordinates of the calibration points and their projection coordinates in the calibration image.

5. The method for three-dimensional reconstruction of tidal bore based on binocular camera according to claim 1, characterized in that: The deployment of the binocular cameras includes: fixing two binocular cameras at two ends of a camera bracket respectively, and installing the camera bracket on a tidal bore monitoring tower.

6. The method for three-dimensional reconstruction of tidal bore based on binocular camera according to claim 1, characterized in that: The image acquisition specifically includes: The industrial computer system of the tidal bore monitoring tower uses a precise time synchronization protocol to synchronize the time of the two binocular cameras so that they are always consistent; According to the periodic collection requirements, the industrial controller continuously sends the same image collection instructions to the two binocular cameras. After receiving the instructions, the binocular cameras take pictures according to the instruction time.

7. The method for three-dimensional reconstruction of tidal bore based on binocular camera according to claim 1, characterized in that: The densely reconstructed three-dimensional point cloud comprises: The PatchMatch algorithm is used to estimate the disparity of each pixel in the two view images based on the camera intrinsic parameters and camera extrinsic parameters. For each pixel, the disparity with the minimum matching cost among the pixels in the domain is used to replace the disparity of the current pixel. Measure the similarity between two patches, use triangulation to convert the disparity of pixels into depth information, and obtain a dense 3D point cloud.

8. The method for three-dimensional reconstruction of tidal bore based on binocular camera according to claim 1, characterized in that: The accuracy assessment includes: Conduct boat verification in the tidal bore 3D reconstruction area, place RTK equipment and targets on the boat, set multiple control points in the tidal bore 3D reconstruction area, sail to each control point and stop, record the time interval between the first and last stop points and continuously record RTK values, and use two binocular cameras to simultaneously capture multiple sets of images at equal time intervals; Compare the target positions in multiple groups of images, calculate the average pixel deviation of the target position in the image, intercept all RTK values ​​in the time interval, and calculate the RTK average value; Determine whether the absolute value of the pixel average deviation and the difference between all RTK values ​​and the RTK average value are both less than the threshold. If all the judgment conditions are met, perform 3D reconstruction on the captured image. The RTK average value of all control points that meet the judgment conditions and the 3D reconstructed coordinates of the binocular camera are obtained, and the mean absolute error, mean height error and mean horizontal error are obtained by comparing the RTK average value and the 3D reconstructed coordinates.

9. A device for three-dimensional reconstruction of tidal bore based on binocular camera, applicable to the method for three-dimensional reconstruction of tidal bore based on binocular camera according to any one of claims 1 to 8, characterized in that: include: Two binocular cameras and a tidal bore monitoring tower, wherein the tidal bore monitoring tower is arranged on the seawall, a camera bracket is arranged on the tidal bore monitoring tower, and the two binocular cameras are respectively arranged at two ends of the camera bracket, and the tidal bore three-dimensional reconstruction area is smaller than the overlapping area of ​​the shooting range of the two binocular cameras.

10. The device for three-dimensionally reconstructing tidal bore based on binocular camera according to claim 9, characterized in that: A distribution box is provided at the bottom of the tidal bore monitoring tower, and a power supply, an industrial computer and a network router are provided in the distribution box. The industrial computer and the network router are electrically connected to the power supply, an image acquisition card is provided on the industrial computer, the industrial computer is communicatively connected to two binocular cameras, and the power supply is electrically connected to the two binocular cameras.

Images