Hydrological cableway construction progress intelligent monitoring method and device

Abstract

The present disclosure provides a hydrological cableway construction progress intelligent monitoring method and device. The method comprises: periodically acquiring a plurality of photos shot by a two-dimensional camera arranged around the hydrological cableway project; identifying tower photos and cableway photos in the plurality of photos by using image recognition technology; determining tower point clouds corresponding to at least one tower of the hydrological cableway project based on the tower photos; determining the integrity of the at least one tower of the hydrological cableway project; determining and outputting the construction progress of the hydrological cableway project; determining tower point clouds corresponding to two towers of the hydrological cableway project based on the tower photos, and determining cableway point clouds corresponding to the cableway of the hydrological cableway project based on the cableway photos; determining the integrity of the two towers of the hydrological cableway project, and determining the integrity of the cableway of the hydrological cableway project; determining the reasonableness of the cableway of the hydrological cableway project; and determining and outputting the construction progress according to the integrity of the two towers, the integrity of the cableway, and the reasonableness of the cableway.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and in particular to a method and device for intelligent monitoring of the construction progress of hydrological cableways. Background Technology

[0002] A hydrological cableway is a hydrological facility that can deliver hydrological testing instruments to any designated location on a test section. A hydrological cableway consists of two towers and a cableway spanning between the two towers.

[0003] To ensure the timely commissioning of the hydrological cableway, its construction progress needs to be monitored manually. However, this monitoring method is labor-intensive. Therefore, determining the construction progress of the hydrological cableway accurately, quickly, and with minimal manpower is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This disclosure provides a method and device for intelligent monitoring of the construction progress of hydrological cableways. The technical solution is as follows:

[0005] This disclosure provides at least one embodiment of a method for intelligent monitoring of the construction progress of hydrological cableways, the method comprising:

[0006] Periodically acquire multiple photos taken by two-dimensional cameras deployed around the hydrological cableway project;

[0007] After acquiring the multiple photos in each cycle, image recognition technology is used to identify tower photos and cableway photos among the multiple photos;

[0008] When the tower photo is identified but the cableway photo is not identified, the tower point cloud corresponding to at least one tower of the hydrological cableway project is determined based on the tower photo.

[0009] Based on the tower point cloud corresponding to the at least one tower, the integrity of at least one tower of the hydrological cableway project is determined.

[0010] Based on the integrity of the at least one tower determined by the tower point cloud corresponding to the at least one tower, the construction progress of the at least one tower is determined; based on the construction progress of the at least one tower, the construction progress of the hydrological cableway project is determined and output.

[0011] When the tower photo and the cableway photo are identified, the tower point cloud corresponding to the two towers of the hydrological cableway project is determined based on the tower photo, and the cableway point cloud corresponding to the cableway of the hydrological cableway project is determined based on the cableway photo.

[0012] determine the integrity of the two towers of the hydrological cableway project based on the tower point cloud corresponding to the two towers, determine the integrity of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway;

[0013] determine the rationality of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway;

[0014] determine the construction progress of the two towers according to the integrity of the two towers determined based on the tower point cloud corresponding to the two towers, determine the construction progress of the cableway according to the integrity and rationality of the cableway determined based on the cableway point cloud corresponding to the cableway, and determine and output the construction progress of the hydrological cableway project based on the construction progress of the two towers and the construction progress of the cableway;

[0015] wherein the tower point cloud is obtained in the following manner:

[0016] based on the identified tower photos, determine 1 reference tower photo and M source tower photos, M being a positive integer;

[0017] extract features of the reference tower photo and the M source tower photos using a first MVSNet, map the features of the M source tower photos to a plane in the coordinate system of the reference tower photo through a homography matrix H2, obtain M feature bodies, aggregate the M feature bodies to obtain a cost volume, regularize the cost volume using a 3D convolution layer to obtain a probability volume, and restore a depth map using the probability volume to reconstruct a first three-dimensional point cloud; the formula of the homography matrix H2 is as follows:

[0018] ;

[0019] (K j ,R j ,t j ) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the jth source tower photo in the M source tower photos, respectively, (K O ,R O ,t O ) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the reference tower photo, respectively, n O is the normal vector of the plane where the reference tower photo is located, I is the unit matrix, and d is the depth;

[0020] input the first three-dimensional point cloud output by the first MVSNet to a first PointNet for processing, and segment a tower point cloud from the first three-dimensional point cloud;

[0021] wherein the cableway point cloud is obtained in the following manner:

[0022] determine 1 reference cable photo and N source cable photos based on the identified cable photos, N being a positive integer;

[0023] extract features of the reference cable photo and the N source cable photos using the second MVSNet, map the features of the N source cable photos to a plane of a coordinate system of the reference cable photo through a homography matrix H1, obtain N feature bodies, aggregate the N feature bodies to obtain a cost volume, regularize the cost volume using a 3D convolution layer to obtain a probability volume, restore a depth map using the probability volume, and reconstruct a second three-dimensional point cloud; a formula of the homography matrix H1 is as follows:

[0024] ;

[0025] (K i ,R i ,t i ) are camera intrinsic parameters, rotation matrices and translation vectors of a two-dimensional camera corresponding to the i-th source cable photo among the N source cable photos, respectively, (K0, R0, t0) are camera intrinsic parameters, rotation matrices and translation vectors of a two-dimensional camera corresponding to the reference cable photo, respectively, n0 is a normal vector of a plane where the reference cable photo is located, I is an identity matrix, and d is a depth;

[0026] input the second three-dimensional point cloud output by the second MVSNet to a second PointNet for processing, and segment a cable point cloud from the second three-dimensional point cloud;

[0027] The tower point cloud is obtained in the following manner:

[0028] determine 1 reference tower photo and M source tower photos based on the identified tower photos, M being a positive integer;

[0029] extract features of the reference tower photo and the M source tower photos using the first MVSNet, map the features of the M source tower photos to a plane of a coordinate system of the reference tower photo through a homography matrix H2, obtain M feature bodies, aggregate the M feature bodies to obtain a cost volume, regularize the cost volume using a 3D convolution layer to obtain a probability volume, restore a depth map using the probability volume, and reconstruct a first three-dimensional point cloud; a formula of the homography matrix H2 is as follows:

[0030] ;

[0031] (K j ,R j ,t j) are the camera intrinsic, rotation matrix and translation vector of the two-dimensional camera corresponding to the jth tower photo in the M tower photos, respectively, (K O ,R O ,t O ) are the camera intrinsic, rotation matrix and translation vector of the two-dimensional camera corresponding to the reference tower photo, respectively, n O is the normal vector of the plane where the reference tower photo is located, I is the identity matrix, and d is the depth;

[0032] inputting the first three-dimensional point cloud output by the first MVSNet to a first PointNet for processing, and segmenting a tower point cloud from the first three-dimensional point cloud.

[0033] Optionally, the way of determining the reasonableness of the cableway of the hydrological cableway project is as follows:

[0034] projecting the cableway point cloud to an XZ plane to obtain a projection graph of the cableway;

[0035] modeling the projection graph of the cableway by using a catenary equation as follows to obtain a catenary equation of the cableway;

[0036] ;

[0037] In the formula, x is the X-axis coordinate value, and the variable in the equation ; T is the tension of the cableway, ;

[0038] is the unit length weight of the cableway, L is the span of the cableway, h is the sag of the cableway, ; is the coordinate of the minimum point of the original X-axis of the cableway after projection, is the coordinate of the maximum point of the original X-axis of the cableway after projection, is the Z-axis coordinate of the minimum point of the Z-axis of the cableway after projection;

[0039] determining the unit length weight and the span of the cableway from the catenary equation of the cableway;

[0040] determining the tension of the cableway based on the unit length weight and the span of the cableway and by using the following way:

[0041] in the empty state of the cableway, the tension H of the cableway is updated in real time by using the following formula:

[0042] ;

[0043] H0 represents the tension in the empty state, h0 represents the sag of the cableway in the empty state, an elevation angle representing a height difference between two ends of the cableway;

[0044] In the cableway load state, the cableway load is generated by a load object, and the tension H of the cableway is dynamically updated in real time by using the following formula:

[0045] ;

[0046] P represents the load;

[0047] ;

[0048] ρ is the fluid density of the water body; C D is the drag coefficient of the load object, A is the projected area of the load object, V is the flow velocity of the water body, and θ is the included angle between the plane and the vertical line in the cableway load state;

[0049] wherein the drag coefficient C D of the load object is dynamically calculated by inputting the convolutional neural network model of the real-time collected load object data, and the convolutional neural network model is trained based on the high-fidelity drag coefficient simulated by a high-fidelity drag coefficient model;

[0050] Based on the tension of the cableway, the maximum loading sag of the cableway is determined.

[0051] Based on the ratio of the maximum loading sag of the cableway to the length of the cableway, the rationality of the cableway of the hydrological cableway project is determined.

[0052] Optionally, the drag coefficient C D of the load object is dynamically calculated by inputting the convolutional neural network model of the real-time collected load object data.

[0053] Obtain the three-dimensional point cloud data of the load object and related environmental parameter data;

[0054] Based on the three-dimensional point cloud data of the load object, geometric analysis is performed on the load object, and geometric parameters of the load object are extracted, including complete geometric description of size, boundary shape, curvature change and local details;

[0055] The geometric parameters are input into a convolutional neural network model to calculate the drag coefficient C D of the load object under different working conditions.

[0056] The convolutional neural network model is trained based on the high-fidelity drag coefficient simulated by a high-fidelity drag coefficient model, comprising:

[0057] The high-fidelity drag coefficient model is constructed by using a hybrid turbulence model combined with a data-driven enhancement model based on the geometric parameters of the load object, and high-fidelity drag coefficient simulation is performed on the load object to obtain the high-fidelity drag coefficient.

[0058] The convolutional neural network model is trained based on the geometric parameters of the load object and the high-fidelity drag coefficient.

[0059] Optionally, the hybrid turbulence model is at least one of a scale adaptive simulation (SAS) model, an improved delayed detached eddy simulation (IDDES) model, and a wall-modeled large eddy simulation (WMLES) model.

[0060] The data-driven enhancement model is at least one of a machine learning (ML) wall model, a physics-informed neural network (PINN), and a reduced order model (ROM).

[0061] In the high-fidelity drag coefficient model, the high-fidelity drag coefficient is calculated according to the following formula:

[0062]

[0063] is the surface of the load object, is the area of the microelement, is the surface normal vector, is the fluid action stress tensor.

[0064] wherein,

[0065]

[0066] is the wall shear stress, is the flow direction normal component, is the surface pressure.

[0067] In training the convolutional neural network model, the geometric parameters of the load object and the scalar operating condition characteristics are input, and the drag coefficient is used as a training target to train the drag coefficient.

[0068]

[0069] wherein, is the network mapping function obtained by training, represents the predicted value of the drag coefficient by the convolutional neural network model.

[0070] A joint loss function of data-driven error and physical constraints is used :

[0071]

[0072] to predict the residual of the velocity field and the pressure field in the Navier-Stokes equation, is a predicted flow field velocity distribution, is a predicted pressure field distribution; is a physical constraint weight, which ensures the physical rationality of the output drag coefficient; wherein i and N are sample numbers and total number of training samples, represents the predicted value of the drag coefficient and the difference between the high-fidelity drag coefficient

[0073] boundary control is performed on the drag coefficient output by the convolutional neural network model:

[0074]

[0075] wherein, and are the lower limit and the upper limit of the drag coefficient;

[0076] The convolutional neural network model is iteratively optimized to realize real-time calculation of the drag coefficient of the load;

[0077] wherein, after each load data collection, the predicted value of the drag coefficient is matched with the high-fidelity drag coefficient and the network parameters are iteratively updated :

[0078]

[0079] is a learning rate, and R is a boundary / physical constraint.

[0080] Optionally, the integrity of each tower reaches a standard, the integrity of the tower corresponding to the construction progress a; the integrity of the cableway reaches a standard, the integrity of the cableway corresponding to the construction progress b; the rationality of the cableway reaches a standard, the rationality of the cableway corresponding to the construction progress c;

[0081] wherein, a, b and c are positive numbers, and 2a+b+c=1;

[0082] The construction progress of the hydrological cableway project is determined and output, comprising:

[0083] ​determine whether the integrity of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project and the reasonableness of the cableway of the hydrological cableway project meet the standard;

[0084] transform the cases that meet the standard among the integrity of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project and the reasonableness of the cableway of the hydrological cableway project into corresponding construction progress, and calculate the sum of the construction progress, to obtain the construction progress of the hydrological cableway project.

[0085] Optionally, the reasonableness of the cableway meeting the standard means that when the lead fish is included in the cableway photo, the ratio of the maximum loading sag of the cableway to the length of the cableway is greater than 1 / 30;

[0086] The reasonableness of the cableway not meeting the standard includes the following two cases:

[0087] When the lead fish is not included in the cableway photo, the ratio of the maximum loading sag of the cableway to the length of the cableway is greater than 1 / 30, and the corresponding construction progress of the reasonableness of the cableway is c / 2;

[0088] When the lead fish is included or not included in the cableway photo, the ratio of the maximum loading sag of the cableway to the length of the cableway is not greater than 1 / 30, and the corresponding construction progress of the reasonableness of the cableway is 0.

[0089] Optionally, the integrity of each tower meets the standard, and the corresponding construction progress of the integrity of the tower is a; the integrity of the cableway meets the standard, and the corresponding construction progress of the integrity of the cableway is b; the reasonableness of the cableway meets the standard, and the corresponding construction progress of the reasonableness of the cableway is c; the reasonableness of each tower meets the standard, and the corresponding construction progress of the reasonableness of the tower is d;

[0090] Wherein, a, b, c and d are positive numbers, and 2a+b+c+2d=1;

[0091] The determination and output of the construction progress of the hydrological cableway project comprises:

[0092] determine whether the integrity of the two towers of the hydrological cableway project, the integrity of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project and the reasonableness of the cableway of the hydrological cableway project meet the standard;

[0093] transform the cases that meet the standard among the integrity of the two towers of the hydrological cableway project, the integrity of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project and the reasonableness of the cableway of the hydrological cableway project into corresponding construction progress, and calculate the sum of the construction progress, to obtain the construction progress of the hydrological cableway project.

[0094] The rationality of the tower is determined as follows:

[0095] Based on the tower corresponding to the tower point cloud, the inclination angle of the tower is determined;

[0096] Based on the inclination angle of the tower, the rationality of the tower is determined.

[0097] At least one embodiment of the present disclosure provides a hydrological cableway construction progress intelligent monitoring device, the device comprising:

[0098] The acquisition module is configured to periodically acquire a plurality of photos taken by a two-dimensional camera arranged around the hydrological cableway project;

[0099] The image recognition module is configured to, after acquiring the plurality of photos in each period, identify tower photos and cableway photos in the plurality of photos using image recognition technology;

[0100] The point cloud acquisition module is configured to, when the tower photos are identified but the cableway photos are not identified, determine a tower point cloud corresponding to at least one tower of the hydrological cableway project based on the tower photos;

[0101] The construction progress determination module is configured to determine the integrity of at least one tower of the hydrological cableway project based on the tower point cloud corresponding to the at least one tower, determine the construction progress of the at least one tower according to the integrity of the at least one tower determined based on the tower point cloud corresponding to the at least one tower, and determine and output the construction progress of the hydrological cableway project based on the construction progress of the at least one tower;

[0102] The point cloud acquisition module is further configured to, when the tower photos and the cableway photos are identified, determine a tower point cloud corresponding to two towers of the hydrological cableway project based on the tower photos, and determine a cableway point cloud corresponding to a cableway of the hydrological cableway project based on the cableway photos;

[0103] The construction progress determination module is further configured to determine the integrity of two towers of the hydrological cableway project based on the tower point cloud corresponding to the two towers, determine the integrity of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway, determine the rationality of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway, determine the construction progress of the two towers according to the integrity of the two towers determined based on the tower point cloud corresponding to the two towers, determine the construction progress of the cableway according to the integrity and rationality of the cableway determined based on the cableway point cloud corresponding to the cableway, and determine and output the construction progress of the hydrological cableway project based on the construction progress of the two towers and the construction progress of the cableway;

[0104] The tower point cloud is obtained in the following manner:

[0105] Based on the identified tower photos, 1 reference tower photo and M source tower photos are determined, M being a positive integer;

[0106] Features of the reference tower photo and the M source tower photos are extracted by using a first MVSNet, features of the M source tower photos are mapped to a plane of a coordinate system of the reference tower photo through a homography matrix H2, M feature bodies are obtained, a cost volume is aggregated based on the M feature bodies, the cost volume is regularized by using a 3D convolution layer to obtain a probability volume, a depth map is recovered by using the probability volume, and a first three-dimensional point cloud is reconstructed; a formula of the homography matrix H2 is as follows:

[0107] ;

[0108] (K j , R j , t j ) are camera intrinsic parameters, rotation matrices and translation vectors of a two-dimensional camera corresponding to the jth source tower photo in the M source tower photos, respectively, (K O , R O , t O ) are camera intrinsic parameters, rotation matrices and translation vectors of a two-dimensional camera corresponding to the reference tower photo, respectively, n O is a normal vector of a plane where the reference tower photo is located, I is an identity matrix, and d is a depth;

[0109] The first three-dimensional point cloud output by the first MVSNet is input to a first PointNet for processing, and a tower point cloud is segmented from the first three-dimensional point cloud;

[0110] The cableway point cloud is obtained in the following manner:

[0111] Based on the identified cableway photos, 1 reference cableway photo and N source cableway photos are determined, N being a positive integer;

[0112] Features of the reference cableway photo and the N source cableway photos are extracted by using a second MVSNet, features of the N source cableway photos are mapped to a plane of a coordinate system of the reference cableway photo through a homography matrix H1, N feature bodies are obtained, a cost volume is aggregated based on the N feature bodies, the cost volume is regularized by using a 3D convolution layer to obtain a probability volume, a depth map is recovered by using the probability volume, and a second three-dimensional point cloud is reconstructed; a formula of the homography matrix H1 is as follows:

[0113] ;

[0114] (K i ,R i ,t i ) are camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the i-th source cableway photo in the N source cableway photos, respectively, (K0, R0, t0) are camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the reference cableway photo, respectively, n0 is the normal vector of the plane where the reference cableway photo is located, I is an identity matrix, and d is a depth;

[0115] inputting the second three-dimensional point cloud output by the second MVSNet to a second PointNet for processing, and segmenting a cableway point cloud from the second three-dimensional point cloud.

[0116] Optionally, the construction progress determination module is configured to determine the rationality of the cableway of the hydrological cableway project in the following manner:

[0117] projecting the cableway point cloud to an XZ plane to obtain a projection graph of the cableway;

[0118] modeling the projection graph of the cableway by using a catenary equation to obtain a catenary equation of the cableway;

[0119] ;

[0120] T is the tension of the cableway, ;

[0121] is the unit length weight of the cableway, L is the span of the cableway, h is the sag of the cableway, ; is the coordinate of the minimum point of the original X-axis of the cableway after projection, is the coordinate of the maximum point of the original X-axis of the cableway after projection, is the Z-axis coordinate of the minimum point of the Z-axis of the cableway after projection;

[0122] determining the unit length weight and the span of the cableway from the catenary equation of the cableway;

[0123] determining the tension of the cableway based on the unit length weight and the span of the cableway and in the following manner:

[0124] in the no-load state of the cableway, the tension H of the cableway is updated in real time and dynamically by using the following formula:

[0125] ;

[0126] H0 represents the tension in the no-load state, h0 represents the sag of the cableway in the no-load state, represents the height difference angle of the two ends of the cableway.

[0127] In the cable load state, the cable load is generated by the load object, and the tension H of the cable is dynamically updated in real time by using the following formula:

[0128] ;

[0129] P represents the load;

[0130] ;

[0131] ρ is the fluid density of the water body; C D is the drag coefficient of the load object, A is the projected area of the load object, V is the flow velocity of the water body, and θ is the included angle between the plane and the vertical line in the cable load state;

[0132] The drag coefficient C D of the load object is dynamically calculated by inputting the convolutional neural network model of the real-time collected load object data, and the convolutional neural network model is trained based on the high-fidelity drag coefficient simulated by the high-fidelity drag coefficient model;

[0133] Based on the tension of the cable, the maximum loading sag of the cable is determined;

[0134] Based on the ratio of the maximum loading sag of the cable to the length of the cable, the rationality of the cable of the hydrological cable engineering is determined.

[0135] Optionally, the point cloud acquisition module is also used to acquire three-dimensional point cloud data of the load object and related environmental parameter data;

[0136] The construction progress determination module is used to perform geometric analysis on the load object based on the three-dimensional point cloud data of the load object, extract geometric parameters of the load object, and input the geometric parameters into a convolutional neural network model to calculate the drag coefficient C D of the load object under different working conditions.

[0137] The device also includes:

[0138] A model training module is used to perform high-fidelity drag coefficient simulation on the load object based on the geometric parameters of the load object and the high-fidelity drag coefficient model constructed by combining a hybrid turbulence model with a data-driven enhancement model, to obtain the high-fidelity drag coefficient; and train the convolutional neural network model based on the geometric parameters of the load object and the high-fidelity drag coefficient.

[0139] Optionally, the hybrid turbulence model is at least one of a scale adaptive simulation (SAS) model, an improved delayed detached eddy simulation (IDDES) model, and a wall-modeled large eddy simulation (WMLES) model.

[0140] The data-driven enhanced model is at least one of a machine learning (ML) wall model, a physics informed neural network (PINN), and a reduced order model (ROM).

[0141] In the high-fidelity drag coefficient model, the drag coefficient is calculated according to the following formula:

[0142]

[0143] is a surface of a load, is an infinitesimal area, is a surface normal vector, is a fluid action stress tensor;

[0144] wherein,

[0145]

[0146] is a wall shear stress, is a flow direction normal component, is a surface pressure;

[0147] In training the convolutional neural network model, a joint loss function of a data-driven error and a physical constraint is adopted :

[0148]

[0149] is a residual error of a predicted velocity field and a pressure field in a Navier-Stokes equation, is a predicted flow field velocity distribution, is a predicted pressure field distribution; is a physical constraint weight, ensuring the physical rationality of the output drag coefficient; wherein i and N are a sample number and a total number of training samples, respectively, represents a difference between a predicted value of the drag coefficient and the high-fidelity drag coefficient .

[0150] Boundary control is performed on the drag coefficient output by the convolutional neural network model:

[0151]

[0152] wherein, and These are the lower and upper limits of the drag coefficient;

[0153] The convolutional neural network model is iteratively optimized to achieve real-time calculation of the drag coefficient of the load.

[0154] Specifically, after each load data acquisition, the predicted value of the drag coefficient is... With the aforementioned high-fidelity drag coefficient Perform matching and iteratively update network parameters. :

[0155]

[0156] R is the learning rate, and R is the boundary / physical constraint.

[0157] Optionally, if the integrity of each tower meets the standard, the construction progress corresponding to the integrity of the tower is a; if the integrity of the cableway meets the standard, the construction progress corresponding to the integrity of the cableway is b; if the rationality of the cableway meets the standard, the construction progress corresponding to the rationality of the cableway is c.

[0158] Where a, b, and c are all positive numbers, and 2a + b + c = 1;

[0159] The construction progress determination module is used to determine whether the integrity of the two towers of the hydrological cableway project, the integrity of the cableway project, and the rationality of the cableway project meet the standards; convert the conditions of meeting the standards for the integrity of the two towers of the hydrological cableway project, the integrity of the cableway project, and the rationality of the cableway project into the corresponding construction progress, and calculate the sum of the construction progress to obtain the construction progress of the hydrological cableway project.

[0160] Optionally, the standard for the rationality of the cableway is that when the cableway photograph includes a lead weight, the ratio of the maximum load sag of the cableway to the length of the cableway is greater than 1 / 30.

[0161] The cableway's failure to meet the standards includes the following two situations:

[0162] When the cableway photograph does not include the lead weight, the ratio of the maximum load sag of the cableway to the length of the cableway is greater than 1 / 30, and the construction progress corresponding to the rationality of the cableway is c / 2.

[0163] When the cableway photograph includes or excludes lead weights, the ratio of the maximum load sag of the cableway to the length of the cableway is no greater than 1 / 30, and the construction progress corresponding to the rationality of the cableway is 0.

[0164] Optionally, the integrity of each tower reaches a standard, the integrity of the tower corresponding to a construction progress a; the integrity of the cableway reaches a standard, the integrity of the cableway corresponding to a construction progress b; the rationality of the cableway reaches a standard, the rationality of the cableway corresponding to a construction progress c; the rationality of each tower reaches a standard, the rationality of the tower corresponding to a construction progress d;

[0165] Wherein a, b, c and d are positive numbers, 2a+b+c+2d=1;

[0166] The construction progress determination module is configured to determine whether the integrity of two towers of the hydrological cableway project, the rationality of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project and the rationality of the cableway of the hydrological cableway project reach a standard;

[0167] The cases where the integrity of two towers of the hydrological cableway project, the rationality of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project and the rationality of the cableway of the hydrological cableway project reach a standard are converted into corresponding construction progress, and the construction progress sum is calculated to obtain the construction progress of the hydrological cableway project;

[0168] The rationality of the tower is determined as follows:

[0169] Based on the tower corresponding to the tower point cloud, the inclination angle of the tower is determined;

[0170] Based on the inclination angle of the tower, the rationality of the tower is determined.

[0171] At least one embodiment of the present disclosure provides an electronic device, which comprises a processor and a memory, the memory stores at least one program code, the program code is loaded and executed by the processor to realize the hydrological cableway construction progress intelligent monitoring method as described above.

[0172] At least one embodiment of the present disclosure provides a computer readable storage medium, the computer readable storage medium stores at least one program code, the program code is loaded and executed by the processor to realize the hydrological cableway construction progress intelligent monitoring method as described in any one of the preceding embodiments.

[0173] The technical scheme provided by the embodiment of the present disclosure has the beneficial effects that:

[0174] Using multiple photos taken by a 2D camera as input, the system identifies tower and cableway photos. If only tower photos are available, it determines the tower point cloud, the tower integrity, and the construction progress based on the tower point cloud. Conversely, if both tower and cableway photos are available, it determines the tower point cloud, the cableway point cloud, the tower integrity, and the cableway integrity and rationality based on the cableway point cloud. Finally, it determines the construction progress based on the tower integrity, cableway integrity, and rationality.

[0175] This solution combines deep learning algorithms and image processing technology to accurately, quickly, and labor-savingly determine the construction progress of hydrological cableway projects, achieving a high level of automation. It utilizes 2D cameras for image capture, eliminating the need for depth cameras, LiDAR, and other advanced equipment, thus significantly reducing costs. This solution is applicable to hydrological cableway projects with towers of various shapes, demonstrating high applicability. Attached Figure Description

[0176] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0177] Figure 1 This is a flowchart of an intelligent monitoring method for the construction progress of a hydrological cableway, provided in an embodiment of this disclosure.

[0178] Figure 2 This is a flowchart of an intelligent monitoring method for the construction progress of a hydrological cableway, provided in an embodiment of this disclosure.

[0179] Figure 3 This is a schematic diagram of a hydrological cableway structure provided in an embodiment of this disclosure;

[0180] Figure 4 This is a schematic diagram illustrating the process for determining the rationality of a cableway in a hydrological cableway project, provided in an embodiment of this disclosure.

[0181] Figure 5 This is a schematic diagram of another hydrological cableway structure provided in this embodiment of the present disclosure;

[0182] Figure 6 This is a schematic diagram of a process for dynamically calculating the drag coefficient of a load, provided in an embodiment of this disclosure;

[0183] Figure 7 This is a schematic diagram of the structure of an intelligent monitoring device for the construction progress of a hydrological cableway provided in an embodiment of this disclosure;

[0184] Figure 8 is a structural block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0185] For the purposes of the present disclosure, technical solutions and advantages, the following will be further described in detail with reference to the drawings.

[0186] Unless otherwise defined, technical terms or scientific terms used herein should be understood as having the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. The use of "first", "second", "third" and similar words in the present patent application specification and claims does not indicate any order, quantity or importance, but is only used to distinguish different components. Similarly, "one" or "a" and similar words do not indicate a quantity limitation, but indicate the presence of at least one. "Include" or "contain" and similar words mean that the elements or objects appearing before "include" or "contain" cover the elements or objects listed after "include" or "contain" and their equivalents, and do not exclude other elements or objects.

[0187] Figure 1 is a flowchart of a hydrological cableway construction progress intelligent monitoring method provided by an embodiment of the present disclosure. Referring to Figure 1 , the method comprises:

[0188] 101: periodically acquire a plurality of photos taken by a two-dimensional camera arranged around a hydrological cableway project.

[0189] Here, the surrounding can refer to a certain range around the location of the hydrological cableway project, and the actual value can be set as needed.

[0190] The cycle length in the periodicity can be set as needed, for example, 1 hour or 2 hours, etc.

[0191] 102: After acquiring the plurality of photos in each cycle, identify tower photos and cableway photos in the plurality of photos using image recognition technology.

[0192] In one example, the tower photo refers to a photo containing a complete tower in the photo. The cableway photo refers to a photo containing a complete cableway in the photo.

[0193] When the tower photo is identified from the plurality of photos, but the cableway photo is not identified, steps 103 to 105 are performed.

[0194] When the tower photo and the cableway photo are identified, steps 106 to 109 are performed.

[0195] 103: Determine a tower point cloud corresponding to at least one tower of the hydrological cableway project based on the tower photo.

[0196] In an example, the tower point cloud is obtained in the following way:

[0197] Based on the identified tower photo, determine 1 reference tower photo and M source tower photos;

[0198] Extract features of the reference tower photo and the M source tower photos using a first multi-view stereo vision network (MVSNet), map the features of the M source tower photos to a plane in the coordinate system of the reference tower photo through a homography matrix H2, obtain M feature bodies, aggregate the M feature bodies to obtain a cost volume, regularize the cost volume using a 3D convolution layer to obtain a probability volume, and restore a depth map using the probability volume to reconstruct a first three-dimensional point cloud; the formula of the homography matrix H2 is as follows:

[0199] ;

[0200] (K j ,R j ,t j ) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the jth source tower photo in the M source tower photos, respectively, (K O ,R O ,t O ) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the reference tower photo, respectively, n O is the normal vector of the plane where the reference tower photo is located, I is the unit matrix, and d is the depth.

[0201] Input the first three-dimensional point cloud output by the first MVSNet into a first point set network (Point Sets Network, PointNet) for processing, and segment the tower point cloud from the first three-dimensional point cloud.

[0202] Wherein, the depth d refers to the distance between the 3D point in the scene and the reference camera, that is, the distance from the two-dimensional camera corresponding to the reference photo to the point in the photo, which is usually determined when the camera is set, or can be determined according to the flight trajectory of the unmanned aerial vehicle.

[0203] 104: Determine the integrity of at least one tower of the hydrological cableway project based on the tower point cloud corresponding to the at least one tower.

[0204] 105: determining the construction progress of the at least one tower based on the integrity of the at least one tower determined according to the tower point cloud corresponding to the at least one tower, and determining and outputting the construction progress of the hydrological cableway project based on the construction progress of the at least one tower.

[0205] 106: determining the tower point clouds corresponding to the two towers of the hydrological cableway project based on the tower photos, and determining the cableway point cloud corresponding to the cableway of the hydrological cableway project based on the cableway photos.

[0206] In an example, the tower point cloud is obtained in the manner of step 103.

[0207] In an example, the cableway point cloud is obtained in the following manner:

[0208] Based on the identified cableway photos, 1 reference cableway photo and N source cableway photos are determined;

[0209] Features of the reference cableway photo and the N source cableway photos are extracted by using a second MVSNet, the features of the N source cableway photos are mapped to a plane in the coordinate system of the reference cableway photo through a homography matrix H1, N feature bodies are obtained, a cost volume is aggregated based on the N feature bodies, a probability volume is obtained by using a 3D convolution layer to regularize the cost volume, a depth map is recovered by using the probability volume, and a second three-dimensional point cloud is reconstructed; the formula of the homography matrix H1 is as follows:

[0210] ;

[0211] (K i ,R i ,t i ) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the i-th source cableway photo among the N source cableway photos, respectively, (K0, R0, t0) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the reference cableway photo, respectively, n0 is the normal vector of the plane where the reference cableway photo is located, I is an identity matrix, and d is a depth;

[0212] The second three-dimensional point cloud output by the second MVSNet is input to a second PointNet for processing, and a cableway point cloud is segmented from the second three-dimensional point cloud.

[0213] 107: determining the integrity of the two towers of the hydrological cableway project based on the tower point clouds corresponding to the two towers, and determining the integrity of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway.

[0214] 108: determining the rationality of the cable of the hydrological cable engineering based on the cable point cloud corresponding to the cable.

[0215] In an example, the manner of determining the rationality of the cable of the hydrological cable engineering is as follows:

[0216] projecting the cable point cloud to the XZ plane to obtain a projection figure of the cable;

[0217] modeling the projection figure of the cable by using the following catenary equation to obtain a catenary equation of the cable;

[0218] ;

[0219] T is the tension of the cable, ;

[0220] is the unit length weight of the cable, L is the cable span, h is the sag of the cable, ; is the coordinate of the minimum point of the original X-axis of the cable after projection, is the coordinate of the maximum point of the original X-axis of the cable after projection, is the Z-axis coordinate of the minimum point of the Z-axis of the cable after projection;

[0221] determining the unit length weight and the cable span of the cable from the catenary equation of the cable;

[0222] determining the tension of the cable based on the unit length weight and the cable span of the cable and by using the following manner:

[0223] in the unloaded state of the cable, the tension H of the cable is updated in real time and dynamically by using the following formula:

[0224] ;

[0225] H0 represents the tension in the unloaded state, h0 represents the sag of the cable in the unloaded state, represents the height difference angle of the two ends of the cable;

[0226] in the loaded state of the cable, the load of the cable is generated by the load, and the tension H of the cable is updated in real time and dynamically by using the following formula:

[0227] ;

[0228] P represents the load;

[0229] ;

[0230] ρ is the fluid density of the water body; C Dis a drag coefficient of the load, A is an incident projection area of the load, V is a flow velocity of the water body, and θ is an included angle between a plane where the cableway is located and a vertical line in a load state of the cableway;

[0231] wherein the drag coefficient C D The drag coefficient of the load is dynamically calculated by inputting the real-time collected load data through a convolutional neural network model, and the convolutional neural network model is trained based on a high-fidelity drag coefficient simulated by a high-fidelity drag coefficient model.

[0232] Based on the tension of the cableway, the maximum loading sag of the cableway is determined.

[0233] Based on a ratio of the maximum loading sag of the cableway to the length of the cableway, the rationality of the cableway of the hydrological cableway project is determined.

[0234] Exemplarily, the load can be a lead fish, and the lead fish is located in the water.

[0235] In this implementation, when calculating the tension of the cableway, if the cableway has a load, the calculation of the tension not only considers the load generated by the load itself, but also considers the load component generated by the load affected by the water flow, so as to ensure more accurate calculation and more reasonable construction progress determination. Moreover, the load component generated by the load affected by the water flow is related to the flow velocity variation of the water flow, and the flow velocity of the water flow is measured by the load to realize dynamic updating of the tension, thereby meeting the dynamic updating requirement of the construction progress.

[0236] Moreover, the drag coefficient of the load is dynamically calculated by inputting the real-time collected load data through a convolutional neural network model, and the convolutional neural network model is trained based on a high-fidelity drag coefficient simulated by a high-fidelity drag coefficient model, thereby realizing dynamic load calculation.

[0237] 109: Determine the construction progress of the two towers according to the completeness of the two towers determined according to the tower point clouds corresponding to the two towers, determine the construction progress of the cableway according to the completeness and rationality of the cableway determined according to the cableway point cloud corresponding to the cableway, and determine and output the construction progress of the hydrological cableway project based on the construction progress of the two towers and the construction progress of the cableway.

[0238] In the embodiments of the present disclosure, multiple photos taken by a two-dimensional camera are taken as input, the tower photo and the cableway photo are recognized, then in the case of only the tower photo, the tower point cloud is determined based on the tower photo, the tower integrity is determined based on the tower point cloud, and the construction progress is determined based on the tower integrity; and in the case of both the tower photo and the cableway photo, the tower point cloud is determined based on the tower photo, the cableway point cloud is determined based on the cableway photo, the tower integrity is determined based on the tower point cloud, the cableway integrity and rationality are determined based on the cableway point cloud, and the construction progress is determined based on the tower integrity, the cableway integrity and rationality.

[0239] In addition, in the process of determining the rationality of the cableway of the hydrological cableway project, the tension of the cableway in the unloaded state / load state is dynamically updated in real time, so that the rationality and construction progress of the cableway can be dynamically updated in real time.

[0240] The scheme combines a deep learning algorithm and an image processing technology, can accurately, quickly and labor-savingly determine the construction progress of the hydrological cableway construction, and has high automation. In the scheme, a two-dimensional camera is used for shooting, without the need of a depth camera, a laser radar and the like, and the requirement for the image acquisition device is low, thereby greatly reducing the cost. The scheme is suitable for the hydrological cableway project including tower racks of various shapes, and has high applicability.

[0241] Figure 2 is a flowchart of a hydrological cableway construction progress intelligent monitoring method provided by the present disclosure. Referring to Figure 2 , the method comprises:

[0242] 201: periodically acquire multiple photos taken by a two-dimensional camera arranged around a hydrological cableway project.

[0243] Exemplarily, the two-dimensional camera can be fixedly arranged for shooting, or carried by an unmanned aerial vehicle to shoot around the tower. Multiple two-dimensional cameras are arranged around the hydrological cableway project, and each two-dimensional camera can shoot multiple photos, thereby obtaining the multiple photos in this step.

[0244] In the embodiments of the present disclosure, the multiple photos differ in at least one of a shooting direction, a shooting angle, a shooting distance, and a shooting height.

[0245] The shooting direction refers to being located at different directions of the tower, such as being located at the east of the tower, the west of the tower, etc.; the shooting angle refers to the shooting picture being different due to the up-down and left-right rotation of the lens at the same position, or being different due to the focal length, etc.; the shooting distance refers to the distance between the lens and the tower during shooting, or the depth; and the shooting height refers to the relative height or absolute height of the lens during shooting.

[0246] 202: After obtaining the plurality of pictures in each cycle, pre-process the plurality of pictures.

[0247] For example, pictures that do not meet the requirements are screened out, and the sizes of the remaining pictures are unified to meet the size requirements of subsequent algorithm models.

[0248] 203: Identify the tower picture and the cableway picture in the plurality of pictures using image recognition technology.

[0249] In the embodiments of the present disclosure, the image recognition technology can use a public large model or a model trained specifically for tower and cableway recognition, and the embodiments of the present disclosure do not make too many limitations.

[0250] When the tower picture is identified from the plurality of pictures but the cableway picture is not identified, steps 204 to 206 are performed.

[0251] When the tower picture and the cableway picture are identified, steps 207 to 210 are performed.

[0252] 204: Determine the tower point cloud corresponding to at least one tower of the hydrological cableway project based on the tower picture.

[0253] In one example, the tower point cloud is obtained as follows:

[0254] Based on the identified tower picture, determine 1 reference tower picture and M source tower pictures;

[0255] Extract the features of the reference tower picture and the M source tower pictures using a first MVSNet, map the features of the M source tower pictures to the plane of the coordinate system of the reference tower picture through a homography matrix H2, obtain M feature bodies, aggregate the M feature bodies to obtain a cost volume, regularize the cost volume using a 3D convolution layer to obtain a probability volume, and restore a depth map using the probability volume to reconstruct a first three-dimensional point cloud, that is, generate a three-dimensional point cloud through MVSNet; the formula of the homography matrix H2 is as follows:

[0256] ;

[0257] (K j , R j , t j ) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the jth source tower picture in the M source tower pictures, respectively. O , R O , t O) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the reference tower photo, n O is the normal vector of the plane where the reference tower photo is located, I is an identity matrix, and d is a depth.

[0258] The first three-dimensional point cloud output by the first MVSNet is input to a first PointNet for processing, and a tower point cloud is segmented from the first three-dimensional point cloud.

[0259] The camera intrinsic parameters can include focal length parameters fx and fy, and imaging origin parameters cx and cy. The rotation matrix and the translation vector are camera extrinsic parameters.

[0260] PointNet is a deep learning network that inputs a 3D point cloud and outputs a segmentation result. PointNet learns the local features of each point in the input point cloud, and then obtains a global point cloud feature using the features of all points. The global feature and the previously learned local features of each point cloud are concatenated, and then a classification result of each point is obtained through a multilayer perceptron (MLP). Based on the classification result of each point, a tower point cloud or a cableway point cloud is segmented from the three-dimensional point cloud.

[0261] The MVSNet and the PointNet mentioned in the embodiments of the present disclosure are obtained by pre-training, and the training method of the network is not limited in the embodiments of the present disclosure.

[0262] In the embodiments of the present disclosure, the reference tower photo can be a photo containing the whole tower obtained by photographing a high center and a horizontal main view. The screening process can be completed through user interaction, or can be automatically selected based on the two-dimensional camera position parameters and camera parameters of the photograph, and the remaining photos are source tower photos.

[0263] 205: Determine the integrity of at least one tower of the hydrological cableway project based on the tower point cloud corresponding to the at least one tower.

[0264] For example, the tower integrity standard defines a series of requirements such as tower height, overall shape, shape of each face, etc.

[0265] When the tower meets each requirement defined by the standard through the point cloud, it is considered that the integrity of the tower meets the standard.

[0266] 206: Determine the construction progress of the at least one tower based on the integrity of the at least one tower determined based on the tower point cloud corresponding to the at least one tower, and determine and output the construction progress of the hydrological cableway project based on the construction progress of the at least one tower.

[0267] Generally, if the integrity of each tower meets the standard, the integrity corresponding progress of the tower is a, 2a is less than 1; if the integrity of one tower does not meet the standard, the integrity corresponding progress of the tower is 0.

[0268] In other examples, if the integrity of one tower does not meet the standard, the integrity corresponding progress of the tower can also be determined according to the proportion of the integrity. For example, according to the ratio of the actual height of the tower to the standard height, the integrity corresponding progress of the tower is a multiplied by the ratio.

[0269] 207: Determine the tower point cloud corresponding to the two towers of the hydrological cableway project based on the tower photos, and determine the cable point cloud corresponding to the cable of the hydrological cableway project based on the cable photos.

[0270] In one example, the tower point cloud is obtained in the manner of step 204.

[0271] In one example, the cable point cloud is obtained in the following manner:

[0272] Based on the identified cable photos, determine 1 reference cable photo and N source cable photos;

[0273] Extract the features of the reference cable photo and the N source cable photos by using the second MVSNet, map the features of the N source cable photos to the plane of the coordinate system of the reference cable photo through a homography matrix H1, obtain N feature bodies, aggregate the N feature bodies to obtain a cost volume, regularize the cost volume by using a 3D convolution layer to obtain a probability volume, and restore a depth map by using the probability volume to reconstruct a second three-dimensional point cloud; the formula of the homography matrix H1 is as follows:

[0274] ;

[0275] (K i , R i , t i ) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the i-th source cable photo among the N source cable photos, respectively, (K0, R0, t0) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the reference cable photo, respectively, n0 is the normal vector of the plane where the reference cable photo is located, I is an identity matrix, and d is a depth.

[0276] Input the second three-dimensional point cloud output by the second MVSNet into a second PointNet for processing, and segment the cable point cloud from the second three-dimensional point cloud.

[0277] In the embodiments of the present disclosure, the reference cableway photo can be a photo containing the whole cableway obtained by taking a highly centered and horizontal main view. The screening process can be completed through interaction with the user, or can be automatically selected based on the shooting two-dimensional camera position parameters and camera parameters, and the remaining photos are all source cableway photos.

[0278] 208: Determine the integrity of the two towers of the hydrological cableway project based on the tower point clouds corresponding to the two towers, and determine the integrity of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway.

[0279] In an example, the tower integrity can refer to step 205.

[0280] Similarly, the integrity of the cableway, for example, the cableway integrity standard defines a series of requirements such as cableway span, cableway sag, overall shape, etc.

[0281] When the cableway meets the various requirements defined by the standard through the point cloud, it is considered that the integrity of the cableway meets the standard.

[0282] 209: Determine the rationality of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway.

[0283] In an example, the way to determine the rationality of the cableway of the hydrological cableway project is as follows:

[0284] Figure 4 is a determination process schematic diagram of the rationality of the cableway of the hydrological cableway project provided by the embodiments of the present disclosure. Referring to Figure 4 , the process includes:

[0285] 2911: Project the cableway point cloud onto the XZ plane to obtain the projection graph of the cableway.

[0286] 2912: Model the projection graph of the cableway using the following catenary equation to obtain the catenary equation of the cableway.

[0287] ;

[0288] T is the tension of the cableway, ;

[0289] is the unit length weight of the cableway, L is the span of the cableway, and h is the sag of the cableway, ; is the coordinate of the minimum point of the original X-axis of the cableway after projection, is the coordinate of the maximum point of the original X-axis of the cableway after projection, is the Z-axis coordinate of the minimum point of the Z-axis of the cableway after projection;

[0290] 2913: determining the unit length weight and cable span of the cableway from the catenary equation of the cableway.

[0291] 2914: determining the tension of the cableway based on the unit length weight and cable span of the cableway.

[0292] 2915: determining the maximum loaded sag of the cableway based on the tension of the cableway.

[0293] 2916: determining the reasonableness of the cableway of the hydrological cableway project based on the ratio of the maximum loaded sag of the cableway to the length of the cableway.

[0294] wherein the span refers to the distance between two endpoints, and the sag refers to the distance from the lowest point to the line connecting the two endpoints. The maximum loaded sag refers to the maximum sag generated by the cableway when in operation.

[0295] In the above formula, the ratio of L and h can be determined according to the projected figure, and thus h can be converted into L. At this time, based on the equation, and L can be solved.

[0296] Verifying the reasonableness of the cableway is actually verifying whether the deformation of the cableway under the working load is reasonable.

[0297] wherein the tension (horizontal tension) of the cableway can be calculated according to the specific working condition (such as empty load, load state) and the type of cableway (single span or multiple spans) by selecting the corresponding formula.

[0298] Exemplarily, in the empty load state of the cableway, the following formula can be used to update the tension H of the cableway in real time and dynamically:

[0299] ;

[0300] H0 represents the tension in the empty load state, h0 represents the sag of the cableway in the empty load state, represents the height difference angle (i.e. the angle between the line connecting the two ends and the horizontal plane) of the cableway at the two ends.

[0301] In the load state of the cableway, the following formula can be used to update the tension H of the cableway in real time and dynamically:

[0302] ;

[0303] P represents the load.

[0304] wherein the real-time and dynamic updating of the tension H of the cableway can be achieved by updating one or more parameters in the above formula.

[0305] ; ​

[0306] p is the fluid density of the water body, which can be obtained by prior measurement.

[0307] C D is the drag coefficient of the load, which is related to the shape of the object and the Reynolds number.

[0308] A is the frontal area of the load, which can be determined based on the volume of the load.

[0309] V is the flow velocity of the water body, which is the flow velocity at the point where the load is located, and can be obtained periodically by a sensor carried by the load or the load, and is dynamically updated, so that the load P, the tension H, the reasonableness of the cableway, and the construction progress are dynamically updated in the embodiment of the present disclosure.

[0310] θ is the angle between the plane where the cableway is located and the vertical line in the load state of the cableway, which can be determined based on the three-dimensional point cloud, and θ is also dynamically updated as V.

[0311] In the embodiment of the present disclosure, the XZ plane can be the plane where the center lines of the two towers are located, and at this time, θ is also the angle between the projection of the cableway and the cableway, and this angle can be easily determined by the point cloud.

[0312] In other embodiments, the XZ plane can also be a plane perpendicular to the plane where the center lines of the two towers are located and the horizontal plane.

[0313] Figure 5 is another schematic diagram of the hydrological cableway structure provided by the embodiment of the present disclosure. As shown in Figure 5 , two towers 20 are constructed on both sides of the river 10 (of course, other water areas can also be constructed), and the towers 20 are connected by a cableway, wherein the reference number 30 is the cableway in the unloaded state, and 40 is the cableway in the load state.

[0314] The XZ plane is the plane where the center lines of the two towers 20 are located, and YZ is the horizontal plane. The load 50 pulls the cableway 40 to form an angle θ with the vertical line under the action of the water flow V. At this time, the load of the load in the vertical direction is the aforementioned P, and the load of the load in the water flow direction generated by the water flow is:

[0315] ;

[0316] The component in the vertical direction is:

[0317] .

[0318] In the embodiment of the present disclosure, the drag coefficient C DThe drag coefficient C of the load object is dynamically calculated by inputting the real-time collected load object data through a convolutional neural network model, and the convolutional neural network model is trained based on high-fidelity drag coefficient simulation.

[0319] In the embodiments of the present disclosure, other parameters such as p, A, V, and q are also measurable.

[0320] In an example, the drag coefficient C of the load object is dynamically calculated by inputting the real-time collected load object data through a convolutional neural network model D The method is as follows:

[0321] Figure 6 FIG. 1 is a flowchart of a method for dynamically calculating the drag coefficient of a load object according to an embodiment of the present disclosure. As shown in FIG. 1, the method includes the following steps: Figure 6

[0322] 2921: Obtain three-dimensional point cloud data of the load object and related environmental parameter data.

[0323] The three-dimensional point cloud data of the load object can be obtained in the same way as the tower and the cableway.

[0324] In implementation, an image acquisition device arranged on site generates three-dimensional point cloud of the load object.

[0325] The related environmental parameters can include temperature, air pressure, wind speed, water flow speed, and other factors that affect the load.

[0326] 2922: Perform geometric analysis on the load object based on the three-dimensional point cloud data of the load object, and extract geometric parameters of the load object, including complete geometric description of size, boundary shape, curvature change, and local details.

[0327] The local details can include surface microstructure, such as grooves, protrusions, textures, chamfers, fillets, notches, and scratches.

[0328] For example, by performing geometric analysis on the collected point cloud, the geometric parameters of the load object are obtained .

[0329] 2923: Input the geometric parameters into a convolutional neural network model to calculate the drag coefficient C of the load object under different working conditions D .

[0330] In an example, the convolutional neural network model is trained based on high-fidelity drag coefficient simulation, including:

[0331] ​Based on the geometric parameters of the load, the high-fidelity drag coefficient model constructed by combining a hybrid turbulence model with a data-driven enhancement model is used to simulate the high-fidelity drag coefficient of the load, and the high-fidelity drag coefficient is obtained.

[0332] The convolutional neural network model is trained based on the geometric parameters of the load and the high-fidelity drag coefficient.

[0333] During training, the geometric parameters of the load can be obtained in the same way as steps 321 and 322, but the process occurs before these steps.

[0334] The hybrid turbulence model can be a Scale-Adaptive Simulation (SAS) / Improved Delayed Detached Eddy Simulation (IDDES) / Wall-Modelled Large Eddy Simulation (WMLES) model, etc.

[0335] The data-driven enhancement model can be a Machine Learning (ML) wall model, a Physics-Informed Neural Network (PINN), or a Reduced-Order Model (ROM).

[0336] In the high-fidelity drag coefficient model, the high-fidelity drag coefficient is calculated as follows:

[0337]

[0338] wherein, is the surface (integration region) of the load, is the infinitesimal area, is the surface normal vector, is the fluid action stress tensor.

[0339] wherein, ;

[0340] wherein, is the wall shear stress, is the flow direction normal component, is the surface pressure.

[0341] ① When the flow velocity V ≤ 1 m / s, the SAS model is used;

[0342] ② When the flow velocity is 1 m / s < V ≤ 5 m / s, the SAS + ML wall model combination model is used. The wall shear stress predicted by machine learning;

[0343] ③ When the flow velocity is 5m / s < V, the SAS / IDDES+ROM combined model is adopted.

[0344] In this embodiment of the disclosure, the high-fidelity drag coefficient model is responsible for simulating the actual scenario to obtain the high-fidelity drag coefficient corresponding to the geometric parameters of the load. Then, based on these data, a convolutional neural network is trained. Subsequently, after obtaining the geometric parameters of the load, the corresponding drag coefficient can be obtained by using the convolutional neural network without further simulation.

[0345] For example, the load characteristic parameters and field measured data are used as inputs, and the drag coefficient obtained by simulation calculation is used as output to train a convolutional neural network model. The network structure includes convolutional layers to extract local flow field and geometric features, and fully connected layers to combine scalar parameters to achieve rapid prediction of the load drag coefficient. During the training process, boundary control constraints are used to constrain the dynamic prediction range of the drag coefficient to ensure physical rationality.

[0346] The model monitors and collects load data in real time on site, generates predicted drag coefficient values ​​through a convolutional neural network, and optimizes them by matching them with boundary constraints and prior drag coefficient data. After each calculation, the network parameters are updated cyclically to achieve adaptive training and dynamic iteration of the model, thereby further improving the accuracy of drag coefficient prediction and the general applicability of the model.

[0347] For example, the geometric parameters of the load Scalar operating condition characteristics (For example, temperature, pressure, flow rate, etc.) are used as inputs, and the drag coefficient is used as the training target to train the drag coefficient:

[0348]

[0349] in, The network mapping function obtained through training. This represents the predicted value of the drag coefficient by the convolutional neural network model.

[0350] The loss function and physical constraints are as follows:

[0351] When training the convolutional neural network model, a joint loss function of data-driven error and physical constraints is used. :

[0352]

[0353] to predict the residual of the velocity field and the pressure field in the Navier-Stokes equation, is the predicted flow field velocity distribution, is the predicted pressure field distribution; is the physical constraint weight, which ensures the physical rationality of the output drag coefficient; wherein i and N are the sample number and the total number of training samples, represents the predicted value of the drag coefficient and the high-fidelity drag coefficient difference;

[0354] boundary control is performed on the drag coefficient output by the convolutional neural network model:

[0355]

[0356] wherein, and are the lower and upper limits of the drag coefficient, which can be set according to the actual range of the drag coefficient;

[0357] The convolutional neural network model is iteratively optimized to realize real-time calculation of the drag coefficient of the load object;

[0358] wherein, after each load object data collection, the predicted value of the drag coefficient is matched with the high-fidelity drag coefficient and the network parameters are iteratively updated :

[0359]

[0360] wherein, is the learning rate, which ensures real-time updating of the model under new working conditions, improves the prediction accuracy and applicability, and R is the boundary / physical constraint.

[0361] In addition to the above formula, the tension can also be calculated in other ways:

[0362] For example, the tension of the cableway can be calculated by combining the unit length weight and the cableway span through the vibration frequency and the bending stiffness (applicable to short cableways).

[0363] Or, the tension of the cableway can be calculated by combining the unit length weight and the cableway span through the vibration frequency and the bending stiffness, and corrected by the sag (applicable to long cableways).

[0364] For example, for multi-span cableway tension calculation, the tension of each span needs to be calculated segmentally, and the overall check is performed through the force balance of the ground anchor and the support, and the key parameters include span, height difference angle, support pressure, etc.

[0365] In an example, the maximum load sag formula of the cableway is as follows:

[0366] f v =( + ) / H;

[0367] f v is the maximum load sag, P is the load, and H is the tension.

[0368] 210: determining the construction progress of the two towers according to the integrity of the two towers determined from the tower point clouds corresponding to the two towers, determining the construction progress of the cableway according to the integrity and rationality of the cableway determined from the cableway point cloud corresponding to the cableway, and determining and outputting the construction progress of the hydrological cableway project based on the construction progress of the two towers and the construction progress of the cableway.

[0369] In an example, the integrity of each tower reaches a standard, and the construction progress corresponding to the integrity of the tower is a; the integrity of the cableway reaches a standard, and the construction progress corresponding to the integrity of the cableway is b; the rationality of the cableway reaches a standard, and the construction progress corresponding to the rationality of the cableway is c.

[0370] wherein a, b and c are positive numbers, and 2a+b+c=1, that is, the progress is at most 100% (completed).

[0371] If the integrity of the cableway does not reach the standard, the progress corresponding to the integrity of the cableway is 0.

[0372] For example, a is 0.2-0.3, b is 0.2-0.3, and c is 0.2-0.3.

[0373] For example, a is 0.3, b is 0.2, and c is 0.2. Alternatively, a is 0.2, b is 0.3, and c is 0.3.

[0374] At this time, step 210 can include:

[0375] determining whether the integrity of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project, and the rationality of the cableway of the hydrological cableway project reach a standard;

[0376] transforming the cases where the integrity of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project, and the rationality of the cableway of the hydrological cableway project reach a standard into corresponding construction progress, and calculating the construction progress sum to obtain the construction progress of the hydrological cableway project.

[0377] In this implementation manner, the construction progress is determined in the above-described manner, and the calculation process is simple and easy to implement.

[0378] In another example, the integrity of each tower reaches a standard, the integrity of the tower corresponds to a construction progress a; the integrity of the cableway reaches a standard, the integrity of the cableway corresponds to a construction progress b; the rationality of the cableway reaches a standard, the rationality of the cableway corresponds to a construction progress c; the rationality of each tower reaches a standard, the rationality of the tower corresponds to a construction progress d;

[0379] Wherein a, b, c and d are positive numbers, and 2a+b+c+2d=1.

[0380] For example, a is 0.1-0.2, b is 0.2-0.3, c is 0.2-0.3, and d is 0.1-0.2.

[0381] For example, a is 0.2, b is 0.2, c is 0.2, and d is 0.1. Or, a is 0.1, b is 0.3, c is 0.3, and d is 0.1.

[0382] At this time, step 210 can include:

[0383] Determining whether the integrity of the two towers of the hydrological cableway project, the rationality of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project, and the rationality of the cableway of the hydrological cableway project reach a standard;

[0384] Converting the cases where the integrity of the two towers of the hydrological cableway project, the rationality of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project, and the rationality of the cableway of the hydrological cableway project reach a standard into corresponding construction progress, and calculating the construction progress sum to obtain the construction progress of the hydrological cableway project.

[0385] In this implementation, the construction progress is determined in the above manner, and the construction progress determination result is more accurate.

[0386] In one example, the rationality of the cableway reaching a standard means that when the lead fish is included in the cableway photo, the ratio of the maximum loaded sag of the cableway to the length of the cableway is greater than 1 / 30;

[0387] That is, when the lead fish is included in the cableway photo, if the ratio of the maximum loaded sag of the cableway to the length of the cableway is greater than 1 / 30, the rationality of the cableway corresponds to a construction progress c.

[0388] The rationality of the cableway not reaching a standard includes the following two cases:

[0389] When the cableway photograph does not include the lead weight, the ratio of the maximum load sag of the cableway to the length of the cableway is greater than 1 / 30, and the construction progress corresponding to the rationality of the cableway is c / 2.

[0390] When the cableway photograph includes or excludes lead weights, the ratio of the maximum load sag of the cableway to the length of the cableway is no greater than 1 / 30, and the construction progress corresponding to the rationality of the cableway is 0.

[0391] In this implementation method, different working conditions are distinguished based on whether the cableway meets the standards, making the construction progress more accurate.

[0392] In one example, the rationality of the tower is determined based on the tower's tilt angle. If the tower's tilt angle is within a set range, the rationality of the tower is determined to meet the standard, and the progress corresponding to the rationality of the tower is d.

[0393] If the tower's rationality does not meet the standard, the corresponding progress for the tower's rationality is 0.

[0394] For example, the rationality of a tower can be determined as follows:

[0395] Based on the tower point cloud corresponding to the tower, the tilt angle of the tower is determined;

[0396] The rationality of the tower is determined based on its tilt angle.

[0397] In one example, determining the tilt angle of the tower may include: segmenting the tower point cloud, determining the centroid of each segment, fitting the tower axis based on the centroid of each segment, and determining the tilt angle of the tower based on the axis.

[0398] Figure 3 This is a schematic diagram of the hydrological cableway structure provided in the embodiments of this disclosure, as shown below. Figure 3 As shown, after the tower construction is completed, cableways need to be erected between the towers. In addition to determining the integrity of the towers and cableways, the intelligent monitoring of the hydrological cableway construction progress also needs to monitor its rationality, so as to determine whether the cableway can complete the hydrological-related tasks.

[0399] Figure 7 This is a schematic diagram of the structure of an intelligent monitoring device for the construction progress of a hydrological cableway, provided in an embodiment of this disclosure. See also... Figure 7 The device includes: an acquisition module 401, an image recognition module 402, a point cloud acquisition module 403, and a construction progress determination module 404.

[0400] Among them, the acquisition module 401 is used to periodically acquire multiple photos taken by two-dimensional cameras arranged around the hydrological cableway project.

[0401] The image recognition module 402 is configured to, after obtaining the plurality of photos in each cycle, recognize the tower photo and the cableway photo in the plurality of photos by using an image recognition technology.

[0402] The point cloud acquisition module 403 is configured to, when the tower photo is recognized but the cableway photo is not recognized, determine a tower point cloud corresponding to at least one tower of the hydrological cableway project based on the tower photo.

[0403] The construction progress determination module 404 is configured to determine the integrity of the at least one tower of the hydrological cableway project based on the tower point cloud corresponding to the at least one tower, determine the construction progress of the at least one tower according to the integrity of the at least one tower determined based on the tower point cloud corresponding to the at least one tower, and determine and output the construction progress of the hydrological cableway project based on the construction progress of the at least one tower.

[0404] The point cloud acquisition module 403 is further configured to, when the tower photo and the cableway photo are recognized, determine a tower point cloud corresponding to two towers of the hydrological cableway project based on the tower photo, and determine a cableway point cloud corresponding to a cableway of the hydrological cableway project based on the cableway photo.

[0405] The construction progress determination module 404 is further configured to determine the integrity of the two towers of the hydrological cableway project based on the tower point cloud corresponding to the two towers, determine the integrity of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway, determine the rationality of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway, determine the construction progress of the two towers according to the integrity of the two towers determined based on the tower point cloud corresponding to the two towers, determine the construction progress of the cableway according to the integrity and rationality of the cableway determined based on the cableway point cloud corresponding to the cableway, and determine and output the construction progress of the hydrological cableway project based on the construction progress of the two towers and the construction progress of the cableway.

[0406] The tower point cloud is obtained in the following manner:

[0407] Based on the recognized tower photo, 1 reference tower photo and M source tower photos are determined, where M is a positive integer.

[0408] The first MVSNet is used to extract features of the reference tower photo and the M source tower photos, the features of the M source tower photos are mapped to a plane of a coordinate system of the reference tower photo through a homography matrix H2, M feature bodies are obtained, a cost volume is aggregated based on the M feature bodies, the cost volume is regularized by using a 3D convolution layer to obtain a probability volume, and a depth map is recovered by using the probability volume to reconstruct a first three-dimensional point cloud; a formula of the homography matrix H2 is as follows:

[0409] ;

[0410] (K j , R j , t j ) are camera intrinsic parameters, a rotation matrix and a translation vector of a two-dimensional camera corresponding to the jth source tower photo in the M source tower photos respectively, (K O , R O , t O ) are camera intrinsic parameters, a rotation matrix and a translation vector of a two-dimensional camera corresponding to the reference tower photo respectively, n O is a normal vector of a plane where the reference tower photo is located, I is a unit matrix, and d is a depth;

[0411] The first three-dimensional point cloud output by the first MVSNet is input to a first PointNet for processing, and a tower point cloud is segmented from the first three-dimensional point cloud;

[0412] The cableway point cloud is obtained in the following manner:

[0413] Based on the identified cableway photo, 1 reference cableway photo and N source cableway photos are determined, and N is a positive integer;

[0414] The second MVSNet is used to extract features of the reference cableway photo and the N source cableway photos, the features of the N source cableway photos are mapped to a plane of a coordinate system of the reference cableway photo through a homography matrix H1, N feature bodies are obtained, a cost volume is aggregated based on the N feature bodies, the cost volume is regularized by using a 3D convolution layer to obtain a probability volume, and a depth map is recovered by using the probability volume to reconstruct a second three-dimensional point cloud; a formula of the homography matrix H1 is as follows:

[0415] ;

[0416] (K i , R i , t i) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the i-th source cableway photo in the N source cableway photos, respectively, (K0, R0, t0) are the camera intrinsic parameters, rotation matrix and translation vector of the two-dimensional camera corresponding to the reference cableway photo, respectively, n0 is the normal vector of the plane where the reference cableway photo is located, I is an identity matrix, and d is a depth;

[0417] inputting the second three-dimensional point cloud output by the second MVSNet to a second PointNet for processing, and segmenting a cableway point cloud from the second three-dimensional point cloud.

[0418] Optionally, the construction progress determination module 404 is configured to determine the rationality of the cableway of the hydrological cableway project in the following manner:

[0419] projecting the cableway point cloud to an XZ plane to obtain a projection graph of the cableway;

[0420] modeling the projection graph of the cableway by using a catenary equation to obtain a catenary equation of the cableway;

[0421] ;

[0422] T is the tension of the cableway, ;

[0423] is the unit length weight of the cableway, L is the span of the cableway, and h is the sag of the cableway, ; is the coordinate of the minimum point of the original X-axis of the cableway after projection, is the coordinate of the maximum point of the original X-axis of the cableway after projection, is the Z-axis coordinate of the minimum point of the Z-axis of the cableway after projection;

[0424] determining the unit length weight and the span of the cableway from the catenary equation of the cableway;

[0425] determining the tension of the cableway based on the unit length weight and the span of the cableway and by using the following manner:

[0426] in the empty state of the cableway, the tension H of the cableway is updated in real time and dynamically by using the following formula:

[0427] ;

[0428] H0 represents the tension in the empty state, h0 represents the sag of the cableway in the empty state, represents the height difference angle of the two ends of the cableway;

[0429] In the cable load state, the cable load is generated by the load object, and the tension H of the cable is dynamically updated in real time by using the following formula:

[0430] ;

[0431] P represents the load;

[0432] ;

[0433] ρ is the fluid density of the water body; C D is the drag coefficient of the load object, A is the projected area of the load object, V is the flow velocity of the water body, and θ is the included angle between the plane and the vertical line in the cable load state;

[0434] wherein the drag coefficient C D is dynamically calculated by inputting the convolutional neural network model of the real-time collected load object data, and the convolutional neural network model is trained based on the high-fidelity drag coefficient simulated by the high-fidelity drag coefficient model;

[0435] Based on the tension of the cable, the maximum loading sag of the cable is determined;

[0436] Based on the ratio of the maximum loading sag of the cable to the length of the cable, the rationality of the cable of the hydrological cable engineering is determined.

[0437] Optionally, the point cloud acquisition module 403 is also used to acquire three-dimensional point cloud data of the load object and related environmental parameter data;

[0438] The construction progress determination module 404 is used to perform geometric analysis on the load object based on the three-dimensional point cloud data of the load object, extract geometric parameters of the load object, and the geometric parameters include complete geometric description such as size, boundary shape, curvature change and local details; input the geometric parameters into a convolutional neural network model, and calculate the drag coefficient C D .

[0439] Optionally, the device can further include:

[0440] The model training module is used to simulate the high-fidelity drag coefficient of the load object based on the geometric parameters of the load object and the high-fidelity drag coefficient model constructed by combining the hybrid turbulence model with the data-driven enhancement model, to obtain the high-fidelity drag coefficient; and train the convolutional neural network model based on the geometric parameters of the load object and the high-fidelity drag coefficient.

[0441] Optionally, the hybrid turbulence model is at least one of a scale adaptive simulation (SAS) model, an improved delayed detached eddy simulation (IDDES) model, and a wall-modeled large eddy simulation (WMLES) model.

[0442] The data-driven enhanced model is at least one of a machine learning (ML) wall model, a physics-informed neural network (PINN), and a reduced order model (ROM).

[0443] Optionally, the integrity of each of the towers corresponds to a construction progress of a, the integrity of the cableway corresponds to a construction progress of b, and the rationality of the cableway corresponds to a construction progress of c.

[0444] wherein a, b, and c are positive numbers, and 2a+b+c=1.

[0445] The construction progress determination module 404 is configured to determine whether the integrity of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project, and the rationality of the cableway of the hydrological cableway project meet the standards, convert the cases where the standards are met among the integrity of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project, and the rationality of the cableway of the hydrological cableway project into corresponding construction progress, and calculate the construction progress sum to obtain the construction progress of the hydrological cableway project.

[0446] Optionally, the rationality of the cableway meeting the standard means that when the lead fish is included in the cableway photo, the ratio of the maximum loaded sag of the cableway to the length of the cableway is greater than 1 / 30.

[0447] The rationality of the cableway not meeting the standard includes the following two cases:

[0448] When the lead fish is not included in the cableway photo, the ratio of the maximum loaded sag of the cableway to the length of the cableway is greater than 1 / 30, and the construction progress corresponding to the rationality of the cableway is c / 2.

[0449] When the lead fish is included or not included in the cableway photo, the ratio of the maximum loaded sag of the cableway to the length of the cableway is not greater than 1 / 30, and the construction progress corresponding to the rationality of the cableway is 0.

[0450] Optionally, the integrity of each of the towers corresponds to a construction progress of a, the integrity of the cableway corresponds to a construction progress of b, the rationality of the cableway corresponds to a construction progress of c, and the rationality of each of the towers corresponds to a construction progress of d.

[0451] wherein a, b, c and d are all positive numbers, 2a+b+c+2d=1;

[0452] The construction progress determination module 404 is configured to determine whether the integrity of the two towers of the hydrological cableway project, the rationality of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project and the rationality of the cableway of the hydrological cableway project meet the standards.

[0453] The case where the integrity of the two towers of the hydrological cableway project, the rationality of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project and the rationality of the cableway of the hydrological cableway project meet the standards is converted into the corresponding construction progress, and the construction progress is calculated, to obtain the construction progress of the hydrological cableway project.

[0454] The rationality of the tower is determined in the following manner:

[0455] Based on the tower corresponding to the tower point cloud, the inclination angle of the tower is determined.

[0456] Based on the inclination angle of the tower, the rationality of the tower is determined.

[0457] It should be noted that the hydrological cableway construction progress intelligent monitoring device provided in the above embodiment defines the report only by the above-mentioned division of each functional module, and in actual application, the above-mentioned functions can be distributed by different functional modules to complete all or part of the functions described above, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the hydrological cableway construction progress intelligent monitoring device and the hydrological cableway construction progress intelligent monitoring method provided in the above embodiment belong to the same concept, and the specific implementation process is described in the method embodiment, which will not be repeated here.

[0458] Figure 8 is a structural block diagram of an electronic device provided by an embodiment of the present disclosure. Generally, the electronic device includes a processor 501 and a memory 502.

[0459] The processor 501 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 501 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 501 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also known as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state.

[0460] The memory 502 can include one or more computer-readable storage media, which can be non-transitory. The memory 502 can also include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 502 is used to store at least one instruction for being executed by the processor 501 to implement the hydrological cable construction progress intelligent monitoring method provided by the method embodiment of the present application.

[0461] A person of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, and the like.

[0462] The above description is only optional embodiments of the present disclosure and does not limit the present disclosure. Any modification, equivalent replacement, improvement, and the like made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for intelligent monitoring of the construction progress of hydrological cableways, characterized in that, The method includes: Periodically acquire multiple photos taken by two-dimensional cameras deployed around the hydrological cableway project; After acquiring the multiple photos in each cycle, image recognition technology is used to identify tower photos and cableway photos among the multiple photos; When the tower photo is identified but the cableway photo is not identified, the tower point cloud corresponding to at least one tower of the hydrological cableway project is determined based on the tower photo. Based on the tower point cloud corresponding to the at least one tower, the integrity of at least one tower of the hydrological cableway project is determined. Based on the integrity of the at least one tower determined by the tower point cloud corresponding to the at least one tower, the construction progress of the at least one tower is determined; based on the construction progress of the at least one tower, the construction progress of the hydrological cableway project is determined and output. When the tower photo and the cableway photo are identified, the tower point cloud corresponding to the two towers of the hydrological cableway project is determined based on the tower photo, and the cableway point cloud corresponding to the cableway of the hydrological cableway project is determined based on the cableway photo. Based on the tower point cloud corresponding to the two towers, the integrity of the two towers of the hydrological cableway project is determined, and based on the cableway point cloud corresponding to the cableway, the integrity of the cableway of the hydrological cableway project is determined. Based on the cableway point cloud corresponding to the cableway, the rationality of the cableway in the hydrological cableway project is determined. Based on the integrity of the two towers determined by the tower point cloud corresponding to the two towers, the construction progress of the two towers is determined; based on the integrity and rationality of the cableway determined by the cableway point cloud corresponding to the cableway, the construction progress of the cableway is determined; based on the construction progress of the two towers and the construction progress of the cableway, the construction progress of the hydrological cableway project is determined and output. The rationality of the cableway for the aforementioned hydrological cableway project is determined as follows: The cableway point cloud is projected onto the XZ plane to obtain the projected image of the cableway. The catenary equation of the cableway is obtained by modeling the projected shape of the cableway using the following catenary equation. ； T represents the cableway tension. , Where L is the weight per unit length of the cableway, h is the cableway span, and h is the cableway sag. ; Let be the coordinates of the original minimum point on the X-axis of the cableway after projection. Let be the coordinates of the original maximum X-axis point of the cableway in the projected form. The minimum Z-axis point of the cableway is located on the projected Z-axis coordinates. The weight per unit length and span of the cableway are determined from the catenary equation of the cableway. The tension of the cableway is determined based on its weight per unit length and span, as follows: When the cableway is unloaded, the tension H of the cableway is dynamically updated in real time using the following formula: ； H0 represents the tension under no-load conditions, and h0 represents the cableway sag under no-load conditions. This indicates the elevation difference angle between the two ends of the cableway; Under the cableway load condition, the cableway load is generated by the load-bearing structure, and the cableway tension H is dynamically updated in real time using the following formula: ； P represents the load; ； ρ is the fluid density of the water body; C D is the drag coefficient of the load, A is the projected area of ​​the load facing the current, V is the flow velocity of the water body, and θ is the angle between the plane of the cableway under load and the vertical line. Wherein, the drag coefficient C of the load D The convolutional neural network model is dynamically calculated by inputting real-time collected load data, and the convolutional neural network model is trained based on the high-fidelity drag coefficient simulated by the high-fidelity drag coefficient model. Based on the tension of the cableway, determine the maximum load sag of the cableway; The rationality of the cableway in the hydrological cableway project is determined based on the ratio of the maximum load sag of the cableway to the length of the cableway.

2. The method according to claim 1, characterized in that, The tower point cloud was obtained using the following method: Based on the identified tower photographs, one reference tower photograph and M source tower photographs are determined, where M is a positive integer. The first MVSNet is used to extract features from the reference tower photograph and the M source tower photographs. The features of the M source tower photographs are mapped onto the plane of the coordinate system of the reference tower photograph using a homography matrix H2, resulting in M ​​feature volumes. These M feature volumes are then aggregated to obtain a cost volume. A 3D convolutional layer is used to regularize the cost volume, resulting in a probability volume. The probability volume is then used to reconstruct the depth map, thus reconstructing a first 3D point cloud. The formula for the homography matrix H2 is as follows: ； (K) j ,R j ,t j (K) are the camera intrinsic parameters, rotation matrix, and translation vector of the 2D camera corresponding to the j-th source tower photo in M ​​source tower photos, respectively. O ,R O ,t O These are the camera intrinsic parameters, rotation matrix, and translation vector of the 2D camera corresponding to the reference tower photograph, respectively. O Let I be the normal vector of the plane containing the reference tower photograph, where I is the identity matrix and d is the depth. The first 3D point cloud output by the first MVSNet is input into the first PointNet for processing, and the tower point cloud is segmented from the first 3D point cloud. The cableway point cloud is obtained in the following way: Based on the identified cableway photos, one reference cableway photo and N source cableway photos are determined, where N is a positive integer; The second MVSNet is used to extract features from the reference cableway photograph and the N source cableway photographs. The features of the N source cableway photographs are mapped onto the plane of the coordinate system of the reference cableway photograph using a homography matrix H1, resulting in N feature volumes. These N feature volumes are then aggregated to obtain a cost volume. A 3D convolutional layer is used to regularize the cost volume, yielding a probability volume. The probability volume is then used to reconstruct the depth map, thus reconstructing a second 3D point cloud. The formula for the homography matrix H1 is as follows: ； (K) i ,R i ,t i (k0, r0, t0) are the camera intrinsic parameters, rotation matrix, and translation vector of the two-dimensional camera corresponding to the i-th source cableway photo in N source cableway photos, respectively; (k0, r0, t0) are the camera intrinsic parameters, rotation matrix, and translation vector of the two-dimensional camera corresponding to the reference cableway photo, respectively; n0 is the normal vector of the plane where the reference cableway photo is located; I is the identity matrix; and d is the depth. The second 3D point cloud output by the second MVSNet is input into the second PointNet for processing, and the cableway point cloud is segmented from the second 3D point cloud.

3. The method according to claim 1, characterized in that, The drag coefficient C of the load is dynamically calculated by a convolutional neural network model that inputs real-time collected load data. D ,include: Acquire the three-dimensional point cloud data of the load and related environmental parameter data; Based on the three-dimensional point cloud data of the load, a geometric analysis is performed on the load to extract the geometric parameters of the load. The geometric parameters include a complete geometric description including size, boundary shape, curvature change and local details. The geometric parameters are input into a convolutional neural network model to calculate the drag coefficient C of the load under different operating conditions. D ; The convolutional neural network model is trained based on the high-fidelity drag coefficient obtained from the high-fidelity drag coefficient simulation model, including: Based on the geometric parameters of the load, a high-fidelity drag coefficient model is constructed using a hybrid turbulence model combined with a data-driven enhancement model. This model is then used to simulate the drag coefficient of the load and obtain the high-fidelity drag coefficient. The convolutional neural network model is trained based on the geometric parameters of the load and the high-fidelity drag coefficient.

4. The method according to claim 3, characterized in that, The hybrid turbulence model is at least one of the following: scale-adaptive simulation (SAS) model, improved delayed separated eddy simulation (IDDES) model, and wall-modeled large eddy simulation (WMLES) model. The data-driven enhancement model is at least one of the following: machine learning ML wall model, physical informed neural network PINN, and reduced-order model ROM; In the high-fidelity drag coefficient model, the high-fidelity drag coefficient The calculation formula is as follows: ； For the surface of the load, Let the area be the infinitesimal element. For the surface normal vector, For fluid-induced stress tensor; in, ； For wall shear stress, For the normal component of the flow, Surface pressure; When training the convolutional neural network model, the geometric parameters of the load are... Scalar operating condition characteristics The resistance coefficient is used as input and as a training target to train the resistance coefficient. ； in, The network mapping function obtained through training. This represents the predicted value of the drag coefficient by the convolutional neural network model; A joint loss function using data-driven error and physical constraints is employed. : ； To predict the residuals of the velocity and pressure fields in the Navier-Stokes equations, It is the predicted flow field velocity distribution. It is the predicted pressure field distribution; These are the physical constraint weights; where i and N are the sample ID and the total number of training samples, respectively. This represents the predicted value of the drag coefficient. and the high-fidelity drag coefficient difference; Boundary control is applied to the drag coefficient output by the convolutional neural network model: ； in, and These are the lower and upper limits of the drag coefficient; The convolutional neural network model is iteratively optimized to achieve real-time calculation of the drag coefficient of the load. Specifically, after each load data acquisition, the predicted value of the drag coefficient is... With the aforementioned high-fidelity drag coefficient Perform matching and iteratively update network parameters. : ； R is the learning rate, and R is the boundary / physical constraint.

5. The method according to any one of claims 1 to 4, characterized in that, The integrity of each tower meets the standard, and the construction progress corresponding to the integrity of the tower is a; the integrity of the cableway meets the standard, and the construction progress corresponding to the integrity of the cableway is b; the rationality of the cableway meets the standard, and the construction progress corresponding to the rationality of the cableway is c. Where a, b, and c are all positive numbers, and 2a + b + c = 1; The process of determining and outputting the construction progress of the hydrological cableway project includes: Determine whether the integrity of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project, and the rationality of the cableway of the hydrological cableway project meet the standards. The integrity of the two towers of the hydrological cableway project, the integrity of the cableway, and the rationality of the cableway are converted into the corresponding construction progress when they meet the standards. The construction progress is then calculated to obtain the construction progress of the hydrological cableway project.

6. The method according to claim 5, characterized in that, The cableway is deemed reasonable if, when the cableway photograph includes a lead weight, the ratio of the maximum load sag of the cableway to the length of the cableway is greater than 1 / 30. The cableway's failure to meet the standards includes the following two situations: When the cableway photograph does not include the lead weight, the ratio of the maximum load sag of the cableway to the length of the cableway is greater than 1 / 30, and the construction progress corresponding to the rationality of the cableway is c / 2. When the cableway photograph includes or excludes lead weights, the ratio of the maximum load sag of the cableway to the length of the cableway is no greater than 1 / 30, and the construction progress corresponding to the rationality of the cableway is 0.

7. The method according to any one of claims 1 to 4, characterized in that, The integrity of each tower meets the standard, and the construction progress corresponding to the integrity of the tower is a; the integrity of the cableway meets the standard, and the construction progress corresponding to the integrity of the cableway is b; the rationality of the cableway meets the standard, and the construction progress corresponding to the rationality of the cableway is c; the rationality of each tower meets the standard, and the construction progress corresponding to the rationality of the tower is d. Where a, b, c, and d are all positive numbers, and 2a + b + c + 2d = 1; The process of determining and outputting the construction progress of the hydrological cableway project includes: Determine whether the integrity of the two towers of the hydrological cableway project, the rationality of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project, and the rationality of the cableway of the hydrological cableway project meet the standards. The conditions for meeting the standards in the integrity of the two towers of the hydrological cableway project, the rationality of the two towers of the hydrological cableway project, the integrity of the cableway of the hydrological cableway project, and the rationality of the cableway of the hydrological cableway project are converted into the corresponding construction progress, and the construction progress is calculated to obtain the construction progress of the hydrological cableway project. The rationality of the tower is determined as follows: Based on the tower point cloud corresponding to the tower, the tilt angle of the tower is determined; The rationality of the tower is determined based on its tilt angle.

8. A smart monitoring device for the construction progress of a hydrological cableway, characterized in that, The device includes: The acquisition module is used to periodically acquire multiple photos taken by two-dimensional cameras deployed around the hydrological cableway project; The image recognition module is used to identify tower photos and cableway photos in the multiple photos after acquiring them in each cycle, using image recognition technology. The point cloud acquisition module is used to determine the tower point cloud corresponding to at least one tower of the hydrological cableway project based on the tower photo when the tower photo is identified but the cableway photo is not identified. The construction progress determination module is used to determine the integrity of at least one tower of the hydrological cableway project based on the tower point cloud corresponding to the at least one tower; determine the construction progress of the at least one tower based on the integrity of the at least one tower determined by the tower point cloud corresponding to the at least one tower; and determine and output the construction progress of the hydrological cableway project based on the construction progress of the at least one tower. The point cloud acquisition module is also used to determine the tower point cloud corresponding to the two towers of the hydrological cableway project based on the tower photo when the tower photo and the cableway photo are identified, and to determine the cableway point cloud corresponding to the cableway of the hydrological cableway project based on the cableway photo. The construction progress determination module is further configured to: determine the integrity of the two towers of the hydrological cableway project based on the tower point clouds corresponding to the two towers; determine the integrity of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway; determine the rationality of the cableway of the hydrological cableway project based on the cableway point cloud corresponding to the cableway; determine the construction progress of the two towers based on the integrity of the two towers determined by the tower point clouds corresponding to the two towers; determine the construction progress of the cableway based on the integrity and rationality of the cableway determined by the cableway point cloud corresponding to the cableway; and determine and output the construction progress of the hydrological cableway project based on the construction progress of the two towers and the construction progress of the cableway. The rationality of the cableway for the aforementioned hydrological cableway project is determined as follows: The cableway point cloud is projected onto the XZ plane to obtain the projected image of the cableway. The catenary equation of the cableway is obtained by modeling the projected shape of the cableway using the following catenary equation. ； T represents the cableway tension. , Where L is the weight per unit length of the cableway, h is the cableway span, and h is the cableway sag. ; Let be the coordinates of the original minimum point on the X-axis of the cableway after projection. Let be the coordinates of the original maximum X-axis point of the cableway in the projected form. The minimum Z-axis point of the cableway is located on the projected Z-axis coordinates. The weight per unit length and span of the cableway are determined from the catenary equation of the cableway. The tension of the cableway is determined based on its weight per unit length and span, as follows: When the cableway is unloaded, the tension H of the cableway is dynamically updated in real time using the following formula: ； H0 represents the tension under no-load conditions, and h0 represents the cableway sag under no-load conditions. This indicates the elevation difference angle between the two ends of the cableway; Under the cableway load condition, the cableway load is generated by the load-bearing structure, and the cableway tension H is dynamically updated in real time using the following formula: ； P represents the load; ； ρ is the fluid density of the water body; C D is the drag coefficient of the load, A is the projected area of ​​the load facing the current, V is the flow velocity of the water body, and θ is the angle between the plane of the cableway under load and the vertical line. Wherein, the drag coefficient C of the load D The convolutional neural network model is dynamically calculated by inputting real-time collected load data, and the convolutional neural network model is trained based on the high-fidelity drag coefficient simulated by the high-fidelity drag coefficient model. Based on the tension of the cableway, determine the maximum load sag of the cableway; The rationality of the cableway in the hydrological cableway project is determined based on the ratio of the maximum load sag of the cableway to the length of the cableway.

9. The apparatus according to claim 8, characterized in that, The tower point cloud was obtained using the following method: Based on the identified tower photographs, one reference tower photograph and M source tower photographs are determined, where M is a positive integer. The first MVSNet is used to extract features from the reference tower photograph and the M source tower photographs. The features of the M source tower photographs are mapped onto the plane of the coordinate system of the reference tower photograph using a homography matrix H2, resulting in M ​​feature volumes. These M feature volumes are then aggregated to obtain a cost volume. A 3D convolutional layer is used to regularize the cost volume, resulting in a probability volume. The probability volume is then used to reconstruct the depth map, thus reconstructing a first 3D point cloud. The formula for the homography matrix H2 is as follows: ； (K) j ,R j ,t j (K) are the camera intrinsic parameters, rotation matrix, and translation vector of the 2D camera corresponding to the j-th source tower photo in M ​​source tower photos, respectively. O ,R O ,t O These are the camera intrinsic parameters, rotation matrix, and translation vector of the 2D camera corresponding to the reference tower photograph, respectively. O Let I be the normal vector of the plane containing the reference tower photograph, where I is the identity matrix and d is the depth. The first 3D point cloud output by the first MVSNet is input into the first PointNet for processing, and the tower point cloud is segmented from the first 3D point cloud. The cableway point cloud is obtained in the following way: Based on the identified cableway photos, one reference cableway photo and N source cableway photos are determined, where N is a positive integer; The second MVSNet is used to extract features from the reference cableway photograph and the N source cableway photographs. The features of the N source cableway photographs are mapped onto the plane of the coordinate system of the reference cableway photograph using a homography matrix H1, resulting in N feature volumes. These N feature volumes are then aggregated to obtain a cost volume. A 3D convolutional layer is used to regularize the cost volume, yielding a probability volume. The probability volume is then used to reconstruct the depth map, thus reconstructing a second 3D point cloud. The formula for the homography matrix H1 is as follows: ； (K) i ,R i ,t i (k0, r0, t0) are the camera intrinsic parameters, rotation matrix, and translation vector of the two-dimensional camera corresponding to the i-th source cableway photo in N source cableway photos, respectively; (k0, r0, t0) are the camera intrinsic parameters, rotation matrix, and translation vector of the two-dimensional camera corresponding to the reference cableway photo, respectively; n0 is the normal vector of the plane where the reference cableway photo is located; I is the identity matrix; and d is the depth. The second 3D point cloud output by the second MVSNet is input into the second PointNet for processing, and the cableway point cloud is segmented from the second 3D point cloud.

Images