A substation construction crane and live equipment safety distance measurement method based on charged interval scale calibration

Abstract

The application discloses a kind of based on charged interval scale calibration substation construction crane and charged equipment safety distance measurement method, comprising the following steps: S1: constructing crane construction scene dataset.S2: target frame is input into relative depth estimation network, and the relative depth value of target frame is output.S3: the relative pose of target frame to source frame is obtained using interframe motion estimation network.S4: constructing crane movement credible graph and multiple loss functions.S5: construct multi-loss joint optimization function, and relative depth estimation network and interframe motion estimation network are jointly trained.S6: the absolute scale depth value is converted to whole graph relative depth value.S7: calculate safety distance and output measurement result or early warning result.The application records continuous multiple frames of images using monocular camera, and combines multiple loss functions to constrain cross-frame timing and smooth the profile correction, ensure the accurate selection of final measurement point, and can directly serve the safety monitoring scene of crane near charged equipment operation.

Description

Technical Field

[0001] This invention relates to the field of power construction safety monitoring technology, and in particular to a method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration. Background Technology

[0002] During substation expansion, maintenance, and renovation, crane booms, hooks, wire ropes, or lifted loads may come into close proximity to energized equipment such as busbars, disconnectors, circuit breakers, surge arresters, and bushings. If the actual distance between the construction crane and the energized equipment is less than the safety requirements, it may cause discharge, electric shock, or equipment damage. Traditional safety distance control relies heavily on manual observation, mechanical limits, laser ranging, or fixed sensors, which suffer from limitations such as limited field of view, high costs of temporary deployment, and difficulty in covering the dynamic movement of the boom end.

[0003] Monocular vision devices offer advantages such as flexible installation, low cost, and easy integration into existing video surveillance systems; however, monocular images inherently possess scale uncertainty. Existing self-supervised monocular depth estimation methods suffer from the following drawbacks: (1) It is mostly used for safety distance detection between static objects or objects that move relatively slowly. The collected images and training modes are relatively simple. There is no need to consider the dynamic changes of the images and cross-frame consistency processing. This detection structure is not suitable for the safety distance detection of cranes with large motion amplitude and large shape differences.

[0004] (2) Training signals are often established by reconstructing adjacent frames. When the crane is close to the energized bay of the substation, it is easily affected by the movement of the boom, the shape difference of the energized equipment, the occlusion of the frame, the camera distortion and the complex background.

[0005] (3) The existing technology outputs an instance mask to each independent object and calculates the distance by uniformly and randomly sampling points within the mask. However, it does not accurately measure the minimum distance between objects, resulting in low test accuracy.

[0006] In summary, designing a high-precision visual ranging method for substation construction cranes and energized bays is a key problem that needs to be solved. Summary of the Invention

[0007] To address the technical problem that existing methods for testing the safe distance between live equipment are subject to limitations in viewing angle and the dynamic movement of the boom, resulting in deviations in test results, this invention provides a method for measuring the safe distance between a substation construction crane and live equipment based on calibrated live interval dimensions.

[0008] The technical solution adopted by this invention to solve its technical problem is: a method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration, comprising the following steps: Step S1: Use a monocular camera to continuously capture multiple sets of images, select the target frame and the source frame, and construct a crane construction scene dataset.

[0009] Step S2: Input the target frame into the relative depth estimation network and output the relative depth value of the target frame.

[0010] Step S3: Input the target frame and the source frame into the inter-frame motion estimation network to obtain the relative pose of the target frame to the source frame.

[0011] Step S4: Construct a reliable graph of crane motion and various loss functions based on the relative pose and crane construction scene dataset.

[0012] Step S5: Based on the crane motion credibility map and loss function in Step S4, construct a multi-loss joint optimization function, and jointly train the relative depth estimation network and the inter-frame motion estimation network.

[0013] Step S6: Identify the interval between two specific structural points based on the optimized target frame relative depth value, and convert the full image relative depth value into an absolute scale depth value according to the conversion ratio between the relative depth value and the corresponding preset interval size.

[0014] Step S7: Select measurement points in the absolute scale depth value of the target frame, calculate the safe distance between the construction crane and the live equipment, and output the measurement results or warning results.

[0015] In an optional embodiment of the present invention, the step of constructing the crane construction scene dataset in step S1 is as follows: Step S11: Use a monocular camera to acquire a continuous image sequence of the substation construction scene, including construction cranes, live equipment and live compartments. The construction crane includes at least one visible component among boom, hook, body or outriggers.

[0016] Step S12: Calibrate the monocular camera, obtain the camera intrinsic parameters, radial distortion coefficient, tangential distortion coefficient and image size, and store the camera parameters in association with the corresponding image sequence.

[0017] Step S13: Establish a charged gap size library, wherein the charged gap size library uses the bus phase-to-phase distance of different voltage levels as the preset gap size. .

[0018] Step S14: Generate training samples according to the correspondence between the target frame and adjacent source frames, and set the preset interval size. Associated with the corresponding energized equipment target.

[0019] In an optional embodiment of the present invention, in step S2, the relative depth estimation network performs the following steps: Step S21: Input the target frame into the multi-receptive-domain edge texture extraction module to extract the boom edge, hook outline, live equipment outline, live bay boundary, and substation frame background texture, thus obtaining the first... Scale convergence characteristics.

[0020] Step S22: Place the first The scale convergence feature is input into the structural response reshaping module, which establishes the structural response relationship between the construction crane, live equipment, live interval boundary, and background in the channel dimension, and obtains the structural reshaping feature, as shown in the following formula: , In the formula: Indicates structural reorganization characteristics; Indicates the first Scale convergence characteristics This represents the edge response mapping function between the construction crane and the live equipment; Represents the correlation mapping function of charged spacer structures; Represents the candidate deep feature transformation function; represents the learnable remodeling weights, and i represents the index of the multi-scale feature level.

[0021] Step S23: Extract the image gradient response based on the target frame; based on the... Scale convergence features generate an edge weight map; the edge weight map is used to weight the image gradient response to obtain candidate edge responses for the construction crane outline, the edge of the energized equipment, and the boundary of the energized interval region. These candidate edge responses are then input into an edge mapping convolutional layer to obtain the edge response of the first convolutional layer. Scale-decoded edge hint features with consistent size.

[0022] S24: Input the structural reorganization features and edge cue features into the edge-preserving decoding module to obtain the first... Scale decoding features.

[0023] S25: Map the decoding features of each scale to the target frame resolution and perform multi-scale fusion to obtain fused decoding features.

[0024] S26: Output the relative depth value of the target frame through the depth prediction head.

[0025] In an optional embodiment of the present invention, the relative pose acquisition step from the target frame to the source frame in step S3 is as follows: Step S31: Concatenate the target frame and the source frame along the channel dimension to form an image pair input data. .

[0026] Step S32: Utilize an inter-frame motion estimation network to extract the input data from the image pairs. Extract temporal motion features and output the relative rotation matrix and relative translation vector from the target frame to the source frame.

[0027] Step S33: Construct the relative pose transformation matrix based on the relative rotation matrix and the relative translation vector, as shown in the following equation: In the formula, This represents the relative pose transformation matrix from the target frame to the source frame. The relative rotation matrix from the target frame to the source frame. This represents the relative translation vector from the target frame to the source frame.

[0028] In an optional embodiment of the present invention, in step S4, the loss function includes distortion embedded photometric reconstruction loss, cross-frame depth coordination loss, and edge smoothing loss.

[0029] The distortion-embedded photometric reconstruction loss reprojects the target frame pixels onto the source frame sampled pixels, constructs the loss using the grayscale difference between the reconstructed image and the original image, and embeds the camera distortion parameters into the photometric loss calculation to compensate for lens distortion system errors.

[0030] The cross-frame depth coordination loss transforms all the three-dimensional points of the target frame to the source frame coordinate system and then compares the differences to construct the loss, so that the same spatial object maintains geometric self-consistency in depth between frames.

[0031] The edge smoothing loss is used to optimize the quality of a single-frame depth map and distinguish between object outlines and internal regions.

[0032] The formula for the joint optimization function in step S5 is shown below: ; In the formula: Indicates the joint optimization loss; Indicates the first Scale distortion embedded photometric reconstruction loss; Indicates the first Scale-based cross-frame depth coordination loss; Indicates the first Scale edge smoothing loss; This represents a reliable graph showing the crane's motion. , , This represents the weighting coefficient.

[0033] In an optional embodiment of the present invention, the construction of the distortion-embedded photometric reconstruction loss includes the following steps: Step S41: Based on target frame pixels and target frame relative depth Relative pose Camera internal parameters and distortion parameters To obtain the source frame sampling coordinates through distortion-embedded reprojection, the following formula is used: ; In the formula: This represents the sampled coordinates of a pixel in the target frame after it has been projected onto the source frame. Indicates the pixel coordinates in the target frame; Represents the target frame pixels The relative depth at that location; Represents the camera intrinsic parameter matrix; Indicates camera distortion parameters; This represents the relative rotation matrix from the target frame to the source frame; This represents the relative translation vector from the target frame to the source frame; This represents the back projection function that combines camera intrinsic parameters and distortion parameters, used to recover the target frame pixels and their relative depths into three-dimensional points in the camera coordinate system; This represents a projection function that combines camera intrinsics and distortion parameters to project transformed 3D points onto the source frame image plane.

[0034] Step S42: Input the source frame into a relative depth estimation network that shares parameters with the target frame to obtain the relative depth value of the source frame. .

[0035] Step S43: Reconstruct the target frame from the source frame based on the source frame sampling coordinates obtained in step S41, and calculate the photometric reconstruction loss, as shown in the following formula: , In the formula: Indicates that it is from the source frame The reconstructed target frame pixel values; Indicates in the source frame According to sampling coordinates Pixel values ​​obtained by interpolation sampling; Indicates the photometric reconstruction error; Indicates the target frame at pixel point The actual pixel value at that location; It represents the 1-norm.

[0036] In an optional embodiment of the present invention, the cross-frame depth coordination loss is obtained by transforming the relative depth of the target frame to the coordinate system of the source frame and coordinating and comparing it with the relative depth of the source frame, as shown in the following formula: , In the formula: Indicates cross-frame depth coordination error; Indicates the source frame at the sampling coordinates The relative depth at that location; Represents the target frame pixels The relative depth at that location.

[0037] The formula for calculating the edge smoothing loss is as follows: In the formula: Indicates the first Relative depth value of the target frame at scale; Indicates the first Scale target frame image; and These represent the gradient operators in the horizontal and vertical directions, respectively; and This represents the image edge weight term.

[0038] The formula for generating the crane motion reliability map is: In the formula: Represents the target frame pixels The reliability weight of crane movement at the location; , , These represent the weighting coefficients for the construction crane area, the electrical equipment area, and the background area, respectively. , and These are target masks for the construction crane region, the electrical equipment region, and the background region obtained from the semantic segmentation of the target frame, respectively. The electrical equipment region includes the electrical equipment and the region where the corresponding electrical interval is located.

[0039] In an optional embodiment of the present invention, the step of converting the relative depth value of the entire map to the absolute scale depth value in step S6 is as follows: Step S61: Identify the energized equipment and its corresponding energized interval region in the target frame, and call the corresponding preset interval size according to the voltage level of the energized interval region. .

[0040] Step S62: Extract the preset interval size within the charged interval region. The two corresponding structural points and Among them, the two structural points are image points that can characterize the positions at both ends of a preset interval size, i.e., the bus endpoints.

[0041] Step S63: Based on camera intrinsic parameters relative depth at structural point The coordinate transformation formula is as follows: Two structural points are restored from image coordinates to relative 3D coordinates. , In the formula: Indicates the first The relative three-dimensional coordinates of each structural point; Indicates the first The pixel coordinates of each structural point in the target frame; Representing structural points The relative depth at that location; Represents the camera intrinsic parameter matrix; Represents the homogeneous pixel coordinates of the structural points; This represents two structural points used for scale calibration.

[0042] Step S64: According to the preset interval size The scale factor is calculated based on the relative three-dimensional distance between the two structural points, using the following formula: , In the formula: The scaling factor represents the conversion from a relative scale to an absolute scale; Indicates the preset interval size; This represents the Euclidean distance between two structural points in a relative three-dimensional coordinate system; This represents a stable term that prevents the denominator from being zero.

[0043] Step S65: Utilize the scaling factor The relative depth of the entire target frame is converted to absolute scale depth as shown in the following formula: , In the formula: Represents the target frame pixels The absolute depth at that location; Represents the target frame pixels The relative depth at that location; This represents the scaling factor.

[0044] In an optional embodiment of the present invention, in step S7, the calculation step of the safety distance is as follows: Step S71: Obtain the construction crane area separately and areas with electrical equipment The target mask is used to sample the crane candidate point set from the target mask or the target contour obtained from the target mask. and candidate point set of live equipment The crane candidate point set is mentioned above. The candidate point set for the energized equipment includes at least candidate points at the end of the boom and on the hook. Candidate points are located on the edge of the live equipment.

[0045] Step S72: Depth based on absolute scale and camera internal reference The candidate points are restored to absolute three-dimensional points, as shown in the following formula: , In the formula: Indicate candidate points The corresponding absolute three-dimensional coordinates; This represents the pixel coordinates of the candidate point in the target frame; Indicate candidate points The absolute depth at that location; Represents the camera intrinsic parameter matrix; Represents the homogeneous pixel coordinates of the candidate point.

[0046] Step S73: In the crane candidate point set and candidate point set of live equipment The system searches for the pair of 3D points with the smallest Euclidean distance and uses the distance between these points as the safe distance between the construction crane and the live equipment, as shown in the following formula: , In the formula: This indicates the minimum safe distance between construction cranes and electrical equipment; Represents the set of candidate points for cranes Candidate points in; Represents the candidate point set of live equipment Candidate points in; and Representing candidate points respectively and candidate points The corresponding absolute three-dimensional coordinates; This represents the Euclidean distance.

[0047] In an optional embodiment of the present invention, step S7 further includes step S74: setting the safety distance With preset safety threshold When comparing, When, the output includes warning results containing the construction crane component category, electrical equipment category, nearest point location, safe distance value, safety threshold, and risk level; when At that time, the safe distance measurement result is output.

[0048] The beneficial effects of this invention are: (1) This invention uses a monocular camera to record multiple consecutive frames of images. Through the built-in inter-frame motion estimation network, the relative pose from the target frame to the source frame is obtained. The cross-frame temporal constraints and contour smoothing correction are combined with multiple loss function constraints to ensure the accurate selection of the final measurement point. It is specifically designed for working conditions such as large boom movements, frame occlusion, and complex metal backgrounds. It can directly serve the safety monitoring scenario of cranes operating near live equipment, and improve the matching degree between visual ranging results and on-site safety management requirements.

[0049] (2) In the calculation of relative depth value, the present invention establishes structural response relationship in four channel dimensions: crane channel, live equipment channel, live interval channel and background channel, respectively, to obtain structural reorganization features. The structural reorganization features are fused with edge prompt features to obtain fused decoding features including the contour of each region and information inside the region. Thus, the contour structure points can be accurately extracted as the ranging reference, rather than randomly selecting uniform points in the region.

[0050] (3) The present invention uses the preset size of the bus phase distance in the energized bay of the substation for scale calibration, which can reduce the reliance on artificial markers or additional distance measuring devices and improve the deployment convenience in temporary construction scenarios of substations.

[0051] (4) The present invention embeds camera distortion parameters into the temporal reprojection link and combines the crane motion credibility map to reduce the impact of boom motion, occlusion and strong background interference on training supervision, thereby improving the stability of depth estimation at the edge of the charged equipment, the interval boundary and the edge of the crane.

[0052] (5) The present invention uses the three-dimensional nearest point pair between the construction crane area and the electrical equipment area as the safety distance, which can more accurately reflect the minimum spatial gap between the end of the boom, the hook or the hoisted object and the electrical equipment. Attached Figure Description

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] Figure 1 This is a flowchart of the method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration, as described in this invention. Figure 2 This is a flowchart of the operation of the relative depth estimation network in this invention. Detailed Implementation

[0055] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0056] like Figure 1 As shown, a method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration includes the following steps: Step S1: Use a monocular camera to continuously capture multiple sets of images, select the target frame and source frame, and construct a crane construction scene dataset. The target frame and source frame are adjacent real-world images in the time series.

[0057] The crane construction scene dataset contains information including a continuous image sequence, camera parameters corresponding to the image sequence, and a preset interval size corresponding to the electrical equipment. .

[0058] The steps for constructing the crane construction scenario dataset are as follows: Step S11: Use a monocular camera to acquire a continuous image sequence of the substation construction scene, including the construction crane, live equipment and live bays. That is, the acquired images include the construction crane, live equipment and the area of ​​the reasonable live bay around the live equipment. The construction crane includes at least one visible component of the boom, hook, body or outriggers.

[0059] Image acquisition is carried out during the operation of the construction crane. There are many large structures around the construction crane, which will obstruct the moving boom and hook. Therefore, the movement of the construction crane and the obstruction of the structures will affect the image processing results. It is necessary to use a targeted network model and model training function to process this situation. See the following steps for details.

[0060] Step S12: Calibrate the monocular camera, obtain the camera intrinsic parameters, radial distortion coefficient, tangential distortion coefficient and image size, and store the camera parameters in association with the corresponding image sequence.

[0061] Camera calibration is the entire process of establishing a precise mathematical mapping relationship between the camera's two-dimensional pixel plane and the real-world three-dimensional physical space by taking multiple images of a standard calibration board (chessboard). Its purpose is to solve for a set of fixed parameters that uniquely describe the camera's optical and imaging characteristics, including the camera intrinsic parameters, radial distortion coefficient, tangential distortion coefficient, and image size that this invention needs to obtain.

[0062] Step S13: Establish a charged gap size library, wherein the charged gap size library uses the bus phase-to-phase distance of different voltage levels as the preset gap size. .

[0063] Step S14: Generate training samples according to the correspondence between the target frame and adjacent source frames, and set the preset interval size. Associated with the corresponding energized equipment target.

[0064] Pack the target frame and the source frame into a pair as training samples for subsequent model training to ensure cross-frame consistency and avoid ghosting during reprojection, which would cause inaccurate measurement of safe distance.

[0065] Step S2: Input the target frame into the relative depth estimation network and output the relative depth value of the target frame.

[0066] In this invention, the relative depth value is obtained by collecting and calculating the spatial relationship and edge contour information of various structural regions. Compared with the traditional relative depth value testing method, it can avoid the problem of inaccurate depth value collection caused by boom movement, differences in the shape of electrical equipment, frame obstruction and complex background.

[0067] like Figure 2 As shown, the relative depth estimation network performs the following steps: Step S21: Input the target frame into the multi-receptive-domain edge texture extraction module to extract the boom edge, hook outline, live equipment outline, live bay boundary, and substation frame background texture, thus obtaining the first... The scale convergence feature can be expressed by the following equation (1): (1) In the formula: Indicates the first Scale convergence characteristics; This represents a multi-receptor field edge texture extraction function; Indicates the target frame; Represents the set of sampling intervals; This represents the texture noise reduction parameter for the crane construction scene, where i is a positive integer.

[0068] Step S22: Place the first The scale-converged feature input structural response reshaping module establishes the structural response relationship between the construction crane, live equipment, live interval boundary, and background in the channel dimension. The scale convergence feature is a feature tensor containing multiple feature channels. Different feature channels have different degrees of response to the construction crane structure, the energized equipment structure, the energized interval boundary and the background texture. The structural response reshaping module performs interactive calculations between the channels to obtain the structural reshaping feature, as shown in formula (2): (2) In the formula: Indicates structural reorganization characteristics; Indicates the first Scale convergence characteristics This represents the edge response mapping function between the construction crane and the live equipment, used to enhance the true contours of the boom and hook, and suppress false textures on the steel frame. This represents the correlation mapping function of the energized bay structure, used to screen for standard and complete three-phase bus bays and weaken the obstruction area; This represents the candidate deep feature transformation function, which is used to receive dual semantic fusion weights and complete feature normalization, nonlinear noise reduction, and multi-scale feature optimization. This indicates learnable renormalization weights. This represents the normalized activation function, with a value range of [0,1], used to stabilize the weight distribution interval.

[0069] Step S23: Extract the image gradient response based on the target frame, as shown in formula (3): (3) In the formula: Indicates the target frame is in the 1st month. Gradient response at different scales; Represents the gradient operator; Indicates downsampling to the th scale.

[0070] According to the The scale convergence feature generates an edge weight map, as shown in formula (4): (4) In the formula: Indicates by the first Scale convergence features The generated edge weight map; Represents edge weight convolution mapping; This represents the Sigmoid activation function.

[0071] Image gradient response outputs all edges, such as fine lines on steel frames, equipment outlines, and light and shadow boundaries. To remove unnecessary edge information, the image gradient response is weighted using the edge weight map to obtain candidate edge responses for the construction crane outline, the edges of energized equipment, and the boundaries of energized interval regions. At this point, useless textures and noisy edges are suppressed. The candidate edge responses are then input into the edge mapping convolutional layer to obtain the result. The edge cue features with consistent scale decoding feature size are shown in formula (5): (5) In the formula: This indicates element-wise multiplication; Represents an edge-mapped convolutional layer; Indicates the first Scale edge hint features.

[0072] S24: Input the structural reorganization features and edge cue features obtained in step S23 into the edge-preserving decoding module to obtain the first... The scale decoding features are shown in formula (6): (6) In the formula: Indicates the first Scale-based decoding features; This indicates an upsampling operation.

[0073] S25: Map the decoding features from each scale to the target frame resolution and perform multi-scale fusion to obtain fused decoding features. At this point, a frame that takes into account both global structure and local details, resolution, and original image quality can be obtained. Figure 1 The comprehensive decoding features are shown in formula (7): (7) In the formula: This represents multi-scale fusion decoding features; This represents a multi-scale feature fusion convolutional mapping; Indicates feature concatenation operation; Indicates the first The scale-decoding features are upsampled to the target frame resolution.

[0074] S26: Output the relative depth value of the target frame through the depth prediction head, as shown in formula (8): (8) In the formula: Indicates the depth prediction head; This represents the relative depth value of the target frame.

[0075] Step S3: Input the target frame and the source frame into the inter-frame motion estimation network to obtain the relative pose of the target frame to the source frame, in order to prepare for the subsequent transformation between the coordinate systems of the target frame and the source frame.

[0076] The steps for obtaining the relative pose from the target frame to the source frame are as follows: Step S31: Concatenate the target frame and the source frame along the channel dimension to form an image pair input data. This means that the channels of the target frame and the source frame are superimposed without changing the image pixels, so as to facilitate the simultaneous reading of the image information of the preceding and following frames, as shown in formula (9): (9) In the formula: Indicates the source frame. Indicates the target frame.

[0077] Step S32: Utilize an inter-frame motion estimation network to extract the input data from the image pairs. Extract temporal motion features and output the relative rotation matrix and relative translation vector from the target frame to the source frame, as shown in formula (10): (10) In the formula: This represents an inter-frame motion estimation network; This represents the input data of the image pair obtained by stitching together the target frame and the source frame; This represents the relative rotation matrix from the target frame to the source frame; This represents the relative translation vector from the target frame to the source frame.

[0078] Step S33: Construct the relative pose transformation matrix based on the relative rotation matrix and relative translation vector calculated by formula (10), as shown in formula (11): (11) In the formula, This represents the relative pose transformation matrix from the target frame to the source frame. The relative rotation matrix from the target frame to the source frame. This represents the relative translation vector from the target frame to the source frame.

[0079] Step S4: Construct a reliable graph of crane motion and various loss functions based on the relative pose and crane construction scene dataset.

[0080] Loss functions typically employ photometric reconstruction loss. This invention utilizes three loss functions: distortion-embedded photometric reconstruction loss, cross-frame depth coordination loss, and edge smoothing loss. Distortion-embedded photometric reconstruction loss embeds distortion parameters during the photometric loss calculation to compensate for systematic errors caused by lens distortion. Cross-frame depth coordination loss avoids depth value discrepancies at the same 3D point between consecutive frames, preventing size drift and ensuring the consistency of the dynamic construction crane's timing. Edge smoothing loss maintains depth continuity in low-texture areas and preserves abrupt depth changes at the edges of the construction crane, electrical equipment, and electrical intervals. The crane motion confidence map is a single-channel weighted map obtained by segmenting a binary mask image. It is used to weight the photometric reconstruction loss calculation, specifically filtering out false texture interference such as steel frames, occlusions, and lighting effects during 2D grayscale matching.

[0081] The construction of the distortion-embedded photometric reconstruction loss includes the following steps: Step S41: Based on target frame pixels and target frame relative depth Relative pose Camera internal parameters and distortion parameters Distortion-embedded reprojection is performed to obtain the source frame sampling coordinates. The distortion-embedded reprojection mainly includes four steps: the first step is to convert the two-dimensional pixels of the target frame into three-dimensional spatial points of the target camera (achieved through the camera intrinsic parameter matrix); the second step is to transform the three-dimensional points of the target frame into the source frame camera coordinate system through relative pose; the third step is to convert the three-dimensional points of the source frame into the source frame normalized imaging plane (embedding distortion parameters); and the fourth step is to map back to the two-dimensional pixel coordinates of the source frame (source frame sampling coordinates). The above process can be shown by formula (12): (12) In the formula: This represents the sampled coordinates of a pixel in the target frame after it has been projected onto the source frame. Indicates the pixel coordinates in the target frame; Represents the target frame pixels The relative depth at that location; Represents the camera intrinsic parameter matrix; Indicates camera distortion parameters; This represents the relative rotation matrix from the target frame to the source frame; This represents the relative translation vector from the target frame to the source frame; This represents the back projection function that combines camera intrinsic parameters and distortion parameters, used to recover the target frame pixels and their relative depths into three-dimensional points in the camera coordinate system; This represents a projection function that combines camera intrinsics and distortion parameters to project transformed 3D points onto the source frame image plane.

[0082] Step S42: Input the source frame into a relative depth estimation network that shares parameters with the target frame to obtain the relative depth value of the source frame. .

[0083] Step S43: Reconstruct the target frame from the source frame based on the source frame sampling coordinates obtained in step S41, and calculate the photometric reconstruction loss, as shown in formula (13): (13) In the formula: This represents the pixel value of the target frame reconstructed from the source frame; This indicates that the data in the source frame is based on the sampling coordinates. Pixel values ​​obtained by interpolation sampling; Indicates the photometric reconstruction error; Indicates the target frame at pixel point The actual pixel value at that location; It represents the 1-norm.

[0084] The cross-frame depth coordination loss is obtained by transforming the relative depth of the target frame to the coordinate system of the source frame and coordinating and comparing it with the relative depth of the source frame, as shown in formula (14): (14) In the formula: Indicates cross-frame depth coordination error; Indicates the source frame at the sampling coordinates The relative depth at that location; Represents the target frame pixels The relative depth at that location.

[0085] The edge smoothing loss is based on the first... The spatial gradient of the scale relative to the depth map and the gradient of the target frame image are constructed, and the calculation method is shown in formula (15): (15) In the formula: Indicates the first Relative depth value of the target frame at scale; Indicates the first Scale target frame image; and These represent the gradient operators in the horizontal and vertical directions, respectively; and This represents the image edge weight term.

[0086] The crane motion credibility map is a distribution map of the credibility weights of crane motion. The binary mask in this invention is obtained by semantic segmentation of the target frame, and is divided into three types of target masks according to the construction crane area, the live equipment area, and the background area. , and It does not need to be divided by the quantity of items, but only by category. It is used for weighted calculation of photometric reconstruction loss. The number of masks is small, which can reduce the workload of weighted calculation.

[0087] The calculation method for the crane motion confidence weight is shown in formula (16): (16) In the formula: Represents the target frame pixels The reliability weight of crane movement at the location; , , These represent the weighting coefficients for the construction crane area, the electrical equipment area, and the background area, respectively. , and These are target masks for the construction crane region, the electrical equipment region, and the background region obtained from the semantic segmentation of the target frame, respectively. The electrical equipment region includes the electrical equipment and the region where the corresponding electrical interval is located.

[0088] Step S5: Based on the crane motion credibility map and loss function in Step S4, construct a multi-loss joint optimization function, and jointly train the relative depth estimation network and the inter-frame motion estimation network.

[0089] That is, the distortion embedded photometric reconstruction loss, cross-frame depth coordination loss, edge smoothing loss and crane motion credibility map are fused to obtain a multi-loss joint optimization function, as shown in formula (17): (17) In the formula: Indicates the joint optimization loss; Indicates the first Scale distortion embedded photometric reconstruction loss; Indicates the first Scale-based cross-frame depth coordination loss; Indicates the first Scale edge smoothing loss; This represents a reliable graph showing the crane's motion. , , This represents the weighting coefficient.

[0090] Step S6: Convert the relative depth values ​​of the entire map to absolute scale depth values.

[0091] The general conversion principle is based on identifying the interval between two specific structural points using the optimized relative depth value of the target frame, calculating the conversion ratio between the image size and the actual size according to the corresponding preset interval size, and then using this conversion ratio to convert the relative depth value of the entire image into an absolute scale depth value. In other words, it converts the relative depth map obtained by monocular depth estimation into a depth map with actual physical scale.

[0092] The specific conversion steps are as follows: Step S61: Identify the energized equipment and its corresponding energized interval region in the target frame, and call the corresponding preset interval size according to the voltage level of the energized interval region. The preset interval size This represents the actual phase-to-phase distance between busbars.

[0093] Step S62: Extract the preset interval size within the charged interval region. The two corresponding structural points and Among them, the two structural points are image points that can characterize the positions at both ends of a preset interval size, i.e., the bus endpoints.

[0094] Step S63: Based on camera intrinsic parameters relative depth at structural point The two structural points are restored from image coordinates to relative three-dimensional coordinates, as shown in formula (18): (18) In the formula: Indicates the first The relative three-dimensional coordinates of each structural point; Indicates the first The pixel coordinates of each structural point in the target frame; Representing structural points The relative depth at that location; Represents the camera intrinsic parameter matrix; Represents the homogeneous pixel coordinates of the structural points; This indicates the sequence number of two structural points used for scale calibration.

[0095] Step S64: According to the preset interval size The scale factor is calculated based on the relative three-dimensional distance between the two structural points, as shown in formula (19): (19) In the formula: The scaling factor represents the conversion from a relative scale to an absolute scale; Indicates the preset interval size; This represents the Euclidean distance between two structural points in a relative three-dimensional coordinate system; This represents a stable term that prevents the denominator from being zero.

[0096] Step S65: Utilize the scaling factor Convert the relative depth of the entire target frame to the absolute scale depth, as shown in formula (20): (20) In the formula: Represents the target frame pixels The absolute depth at that location; Represents the target frame pixels The relative depth at that location; This represents the scaling factor.

[0097] Step S7: Calculate the safe distance between the construction crane and the live equipment and output the measurement results or warning results.

[0098] The safe distance calculation involves selecting the two closest measurement points between the construction crane and the live equipment in the absolute scale depth value of the target frame, converting the two-dimensional coordinates of the two measurement points into three-dimensional coordinates, and then using the two-point distance formula to calculate the actual safe distance.

[0099] The specific steps are as follows: Step S71: Obtain the construction crane area separately and areas with electrical equipment The target mask is used to sample the crane candidate point set from the target mask or the target contour obtained from the target mask. and candidate point set of live equipment The crane candidate point set is mentioned above. The candidate point set for the energized equipment includes at least candidate points at the end of the boom and on the hook. Candidate points are located on the edge of the live equipment (the nearest points are all located on the edge).

[0100] Step S72: Depth based on absolute scale and camera internal reference The candidate points are restored to absolute three-dimensional points, as shown in formula (21): (twenty one) In the formula: Indicate candidate points The corresponding absolute three-dimensional coordinates; This represents the pixel coordinates of the candidate point in the target frame; Indicate candidate points The absolute depth at that location; Represents the camera intrinsic parameter matrix; Represents the homogeneous pixel coordinates of the candidate point.

[0101] Step S73: In the crane candidate point set and candidate point set of live equipment The system searches for the pair of three-dimensional points with the smallest Euclidean distance and uses the distance between these two points as the safe distance between the construction crane and the live equipment, as shown in formula (22). (twenty two) In the formula: This indicates the minimum safe distance between construction cranes and electrical equipment; Represents the set of candidate points for cranes Candidate points in; Represents the candidate point set of live equipment Candidate points in; and Representing candidate points respectively and candidate points The corresponding absolute three-dimensional coordinates; This represents the Euclidean distance, which is the closest three-dimensional point-to-point distance between the construction crane area and the electrical equipment area.

[0102] Measurement result output or safety warning: The safety distance With preset safety threshold When comparing, When, the output includes warning results containing the construction crane component category (such as boom, hook), electrical equipment category, nearest point location, safe distance value, safety threshold, and risk level (graded according to the difference between the safe distance and the safety threshold); when At that time, the safe distance measurement result is output.

[0103] In this specification, the illustrative expressions of the terms do not necessarily refer to the same embodiments. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0104] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration, characterized in that, Includes the following steps: Step S1: Use a monocular camera to continuously capture multiple sets of images, select the target frame and source frame, and construct a crane construction scene dataset; Step S2: Input the target frame into the relative depth estimation network and output the relative depth value of the target frame; Step S3: Input the target frame and the source frame into the inter-frame motion estimation network to obtain the relative pose of the target frame to the source frame; Step S4: Construct a reliable graph of crane motion and various loss functions based on the relative pose and crane construction scene dataset; Step S5: Based on the crane motion credibility map and loss function in step S4, construct a multi-loss joint optimization function, and jointly train the relative depth estimation network and the inter-frame motion estimation network; Step S6: Identify the interval between two specific structural points based on the optimized target frame relative depth value, and convert the overall relative depth value into an absolute scale depth value according to the conversion ratio between the relative depth value and the corresponding preset interval size; the preset interval size is the predefined actual phase-to-phase distance between busbars; Step S7: Select measurement points in the absolute scale depth value of the target frame, calculate the safe distance between the construction crane and the live equipment, and output the measurement results or warning results; The loss function includes distortion-embedded photometric reconstruction loss, cross-frame depth coordination loss, and edge smoothing loss; The distortion-embedded photometric reconstruction loss reprojects the target frame pixels onto the source frame sampled pixels, constructs the loss using the grayscale difference between the reconstructed image and the original image, and embeds the camera distortion parameters into the photometric loss calculation to compensate for lens distortion system errors. The cross-frame depth coordination loss transforms all the three-dimensional points of the target frame to the coordinate system of the source frame and then compares the differences to construct the loss, so that the same spatial object maintains geometric self-consistency in depth between frames. The edge smoothing loss is used to optimize the quality of a single-frame depth map and distinguish between object contours and internal regions. The formula for the joint optimization function in step S5 is shown below: ； In the formula: Indicates the joint optimization loss; Indicates the first Scale distortion embedded photometric reconstruction loss; Indicates the first Scale-based cross-frame depth coordination loss; Indicates the first Scale edge smoothing loss; Represents a reliable diagram of crane movement; , , This represents the weighting coefficient.

2. The method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration according to claim 1, characterized in that, In step S1, the steps for constructing the crane construction scenario dataset are as follows: Step S11: Use a monocular camera to acquire a continuous image sequence of the substation construction scene, including construction cranes, live equipment and live compartments. The construction crane includes at least one visible component among boom, hook, body or outriggers. Step S12: Calibrate the monocular camera, obtain the camera intrinsic parameters, radial distortion coefficient, tangential distortion coefficient and image size, and store the camera parameters in association with the corresponding image sequence; Step S13: Establish a energized gap dimension library. The energized gap dimension library defines the corresponding bus phase-to-phase distance for equipment of different voltage levels, which serves as the preset gap size for different voltage levels. ; Step S14: Generate training samples according to the correspondence between the target frame and adjacent source frames, and set the preset interval size. Associated with the corresponding energized equipment target.

3. The method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration according to claim 1, characterized in that, In step S2, the relative depth estimation network performs the following steps: Step S21: Input the target frame into the multi-receptive-domain edge texture extraction module to extract the boom edge, hook outline, live equipment outline, live bay boundary, and substation frame background texture, thus obtaining the first... Scale convergence characteristics; Step S22: Place the first The scale convergence feature is input into the structural response reshaping module, which establishes the structural response relationship between the construction crane, energized equipment, energized interval boundary, and background in the channel dimension, thereby obtaining the structural reshaping feature, as shown in the following formula: ， In the formula: Indicates structural reorganization characteristics; Indicates the first Scale convergence characteristics This represents the edge response mapping function between the construction crane and the live equipment; Represents the correlation mapping function of charged spacer structures; Represents the candidate deep feature transformation function; This represents the learnable remodeling weights, and i represents the index of the multi-scale feature level; Step S23: Extract the image gradient response based on the target frame; based on the... Scale convergence features generate an edge weight map; the edge weight map is used to weight the image gradient response to obtain candidate edge responses for the construction crane outline, the edge of the energized equipment, and the boundary of the energized interval region. These candidate edge responses are then input into an edge mapping convolutional layer to obtain the edge response of the first convolutional layer. Scale-decoded edge cue features with consistent size; S24: Input the structural reorganization features and edge cue features into the edge-preserving decoding module to obtain the first... Scale-based decoding features; S25: Map the decoding features of each scale to the target frame resolution and perform multi-scale fusion to obtain fused decoding features; S26: Output the relative depth value of the target frame through the depth prediction head.

4. The method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration according to claim 1, characterized in that, In step S3, the relative pose acquisition steps from the target frame to the source frame are as follows: Step S31: Concatenate the target frame and the source frame along the channel dimension to form an image pair input data. ; Step S32: Utilize an inter-frame motion estimation network to extract the input data from the image pairs. Extract temporal motion features and output the relative rotation matrix and relative translation vector from the target frame to the source frame; Step S33: Construct the relative pose transformation matrix based on the relative rotation matrix and the relative translation vector, as shown in the following equation: In the formula, This represents the relative pose transformation matrix from the target frame to the source frame. The relative rotation matrix from the target frame to the source frame. This represents the relative translation vector from the target frame to the source frame.

5. The method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration according to claim 1, characterized in that, The construction of the distortion-embedded photometric reconstruction loss includes the following steps: Step S41: Based on target frame pixels and target frame relative depth Relative pose Camera internal parameters and distortion parameters To obtain the source frame sampling coordinates through distortion-embedded reprojection, the following formula is used: ； In the formula: This represents the sampled coordinates of a pixel in the target frame after it has been projected onto the source frame. Indicates the pixel coordinates in the target frame; Represents the target frame pixels The relative depth at that location; Represents the camera intrinsic parameter matrix; Indicates camera distortion parameters; This represents the relative rotation matrix from the target frame to the source frame; This represents the relative translation vector from the target frame to the source frame; This represents the back projection function that combines camera intrinsic parameters and distortion parameters, used to recover the target frame pixels and their relative depths into three-dimensional points in the camera coordinate system; This represents a projection function that combines camera intrinsics and distortion parameters to project the transformed 3D points onto the source frame image plane. Step S42: Input the source frame into a relative depth estimation network that shares parameters with the target frame to obtain the relative depth value of the source frame. ; Step S43: Reconstruct the target frame from the source frame based on the source frame sampling coordinates obtained in step S41, and calculate the photometric reconstruction loss, as shown in the following formula: ， In the formula: Indicates that the source frame The reconstructed target frame pixel values; Indicates in the source frame According to sampling coordinates Pixel values ​​obtained by interpolation sampling; Indicates the photometric reconstruction error; Indicates the target frame at pixel point The actual pixel value at that location; It represents the 1-norm.

6. The method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration according to claim 1, characterized in that: The cross-frame depth coordination loss is obtained by transforming the relative depth of the target frame to the coordinate system of the source frame and coordinating it with the relative depth of the source frame, as shown in the following formula: ， In the formula: Indicates cross-frame depth coordination error; Indicates the source frame at the sampling coordinates The relative depth at that location; Represents the target frame pixels The relative depth at that location; The formula for calculating the edge smoothing loss is as follows: In the formula: Indicates the first Relative depth value of the target frame at scale; Indicates the first Scale target frame image; and These represent the gradient operators in the horizontal and vertical directions, respectively; and Represents the image edge weight term; The formula for generating the crane motion reliability map is: In the formula: Represents the target frame pixels The reliability weight of crane movement at the location; , , These represent the weighting coefficients for the construction crane area, the electrical equipment area, and the background area, respectively. , and These are target masks for the construction crane region, the electrical equipment region, and the background region obtained from the semantic segmentation of the target frame, respectively. The electrical equipment region includes the electrical equipment and the region where the corresponding electrical interval is located.

7. The method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration according to claim 1, characterized in that, The step in step S6, converting the relative depth values ​​of the entire map to absolute scale depth values, is as follows: Step S61: Identify the energized equipment and its corresponding energized interval region in the target frame, and call the corresponding preset interval size according to the voltage level of the energized interval region. ; Step S62: Extract the preset interval size within the charged interval region. The two corresponding structural points and Among them, the two structural points are image points that can characterize the positions at both ends of the preset interval size, i.e., the endpoints of the busbar; Step S63: Based on camera intrinsic parameters relative depth at structural points The coordinate transformation formula is as follows: Two structural points are restored from image coordinates to relative 3D coordinates. ， In the formula: Indicates the first The relative three-dimensional coordinates of each structural point; Indicates the first The pixel coordinates of each structural point in the target frame; Representing structural points The relative depth at that location; Represents the camera intrinsic parameter matrix; Represents the homogeneous pixel coordinates of the structural points; This represents two structural points used for scale calibration; Step S64: According to the preset interval size The scale factor is calculated based on the relative three-dimensional distance between the two structural points, using the following formula: ， In the formula: The scale coefficient represents the conversion from a relative scale to an absolute scale; Indicates the preset interval size; This represents the Euclidean distance between two structural points in a relative three-dimensional coordinate system; This represents a stable term to prevent the denominator from being zero. Step S65: Utilize the scaling factor The relative depth of the entire target frame is converted to absolute scale depth as shown in the following formula: ， In the formula: Represents the target frame pixels The absolute depth at that location; Represents the target frame pixels The relative depth at that location; This represents the scaling factor.

8. The method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration according to claim 1, characterized in that, In step S7, the calculation steps for the safe distance are as follows: Step S71: Obtain the construction crane area separately and areas with electrical equipment The target mask is used to sample the crane candidate point set from the target mask or the target contour obtained from the target mask. and candidate point set of live equipment The crane candidate point set is mentioned above. The candidate point set for the energized equipment includes at least candidate points at the end of the boom and on the hook. Candidate points on the edge of live equipment; Step S72: Depth based on absolute scale and camera internal reference The candidate points are restored to absolute three-dimensional points, as shown in the following formula: ， In the formula: Indicate candidate points The corresponding absolute three-dimensional coordinates; This represents the pixel coordinates of the candidate point in the target frame; Indicate candidate points The absolute depth at that location; Represents the camera intrinsic parameter matrix; Represents the homogeneous pixel coordinates of the candidate point; Step S73: In the crane candidate point set and candidate point set of live equipment The system searches for the pair of 3D points with the smallest Euclidean distance and uses the distance between these points as the safe distance between the construction crane and the live equipment, as shown in the following formula: ， In the formula: This indicates the minimum safe distance between construction cranes and electrical equipment; Represents the set of candidate points for cranes Candidate points in; Represents the candidate point set of live equipment Candidate points in; and Representing candidate points respectively and candidate points The corresponding absolute three-dimensional coordinates; This represents the Euclidean distance.

9. The method for measuring the safe distance between a substation construction crane and energized equipment based on energized interval scale calibration according to claim 8, characterized in that, Step S7 further includes step S74: setting the safety distance With the preset safety threshold When comparing, When, the output includes warning results containing the construction crane component category, electrical equipment category, nearest point location, safe distance value, safety threshold, and risk level; when At that time, the safe distance measurement result is output.

Images