A method and system for inspecting cable channels using unmanned aerial vehicles (UAVs).

Abstract

This invention discloses a method and system for inspecting cable ducts using unmanned aerial vehicles (UAVs). The method includes: during the duct construction phase, using a UAV to inspect the cable duct under construction, and calculating and modeling the cable duct's geographical location based on a language prompt word encoding module and closing operations; planning the cable duct inspection route based on the established cable duct model; during the inspection phase, the UAV autonomously flies along the planned route and captures real-time images, performing hazard target identification and effective alarm analysis on each frame of the image based on its location and direction of movement. This invention offers high route planning efficiency, improving the inspection effect and timeliness.

Description

Technical Field

[0001] This invention belongs to the field of cable channel inspection technology, and relates to a method and system for inspecting cable channels using drones. Background Technology

[0002] Cable channels are one of the important targets of power system inspection. Timely detection of hidden dangers in the channels is crucial to ensuring power safety. With the maturity of drone technology and products, they are increasingly being used in cable channel inspection to replace manual on-site inspections, which can improve inspection efficiency and the timeliness of hidden danger detection.

[0003] However, drone inspections of cable tunnels still present some problems, mainly in two aspects. First, they cannot automatically generate flight paths; instead, they require the cooperation of professionals familiar with the cable tunnel locations and drone pilots to manually delineate the inspection routes. Second, drones are only used for data collection during flight, with hazard analysis conducted only after the drone's flight has ended. The former leads to a large workload for flight path planning, which is prone to errors and has low efficiency; the latter results in a disconnect between flight and analysis, leading to poor inspection results and a certain degree of delay. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and system for inspecting cable channels using unmanned aerial vehicles (UAVs). During the channel construction phase, the system automatically calculates and models the geographical location of the cable channel to automatically plan flight routes. During UAV inspections, it uses computer vision algorithms to identify potential hazard targets in real time and provides effective warnings through location and movement analysis.

[0005] The present invention adopts the following technical solution.

[0006] The first aspect of this invention provides a method for inspecting cable channels using a drone, comprising:

[0007] During the construction phase of the cable channel, drones are used to inspect the cable channel under construction, and the geographical location of the cable channel is calculated and modeled based on the language prompt word encoding module and closed operation.

[0008] Based on the established cable channel model, plan the cable channel inspection route;

[0009] During the inspection phase, the drones fly autonomously along the planned route and capture real-time images. Each frame of the image is used to identify potential hazards and analyze their location and direction of movement for alarm purposes.

[0010] Preferably, during the channel construction phase, drones are used to inspect the cable channel under construction, and the geographical location of the cable channel is calculated and modeled based on a language prompt word encoding module and closing operations, specifically including:

[0011] The drone inspects the cable tunnel under construction, and its onboard gimbal camera captures real-time footage of the excavated cable tunnel, producing flight video.

[0012] Each frame of the flight video is extracted and image segmentation is performed based on the language prompt word encoding module to obtain the image mask of the corresponding frame segmentation;

[0013] Based on the closing operation, connectivity analysis is performed on the mask of each frame, and the largest connected component is fitted into a quadrilateral to obtain the quadrilateral of the channel captured in each frame of the image.

[0014] Based on the UAV's GPS location and camera parameters, calculate the longitude and latitude of the quadrilateral of the channel captured in each frame of the image;

[0015] The channel positions of adjacent frame images are smoothly connected based on the longitude and latitude to obtain the channel edges.

[0016] Preferably, the step of extracting each frame of the flight video and performing image segmentation based on the language prompt word encoding module to obtain the corresponding frame segmented image mask includes:

[0017] The SAM model takes an image and the cue word "excavated tunnel" as input and outputs a segmented image mask, where pixels belonging to the "excavated tunnel" category have a value of 1, otherwise 0. The SAM model uses a pre-trained language cue word encoding module to recognize the "excavated tunnel" cue word. Specifically:

[0018] The language prompt word encoding module tokenizes the text and converts it into word vectors through an embedding matrix. At the same time, it uses a cosine function to generate position encoding vectors. The position encoding vectors and word vectors are added together and then input into a self-attention encoding layer, which learns the word relationships of the input vectors through multi-head self-attention.

[0019] Preferably, the step of performing connectivity analysis on the mask of each frame based on the closing operation and fitting the largest connected component into a quadrilateral to obtain the quadrilateral of the channels captured in each frame of the image includes:

[0020] Perform image morphology closing operations on the mask of the current frame to segment out more complete tunnels; perform connectivity analysis on the mask of the current frame to find the largest connected component, fit the connected component into a quadrilateral, and retain it if the parameters of the fitted quadrilateral are within the set range; otherwise, skip the current frame and continue processing the next frame.

[0021] Preferably, the kernel initialization method used in the closing operation is as follows:

[0022] Randomly select more than 10% of video frames to form a missed detection calibration dataset, use the SAM model for segmentation, manually review and mark the missed detection areas based on the segmentation results, mark the missed detection areas as 1, and mark other areas as 0;

[0023] The missed detection calibration dataset was labeled and the missed detection statistics were performed. The diameter distribution of all missed detection areas was statistically analyzed, and the corresponding histogram was obtained.

[0024] After Gaussian smoothing of the histogram, the diameter of the peak position is used as the kernel size. The kernel is initialized with this size to perform a closing operation on the mask.

[0025] Preferably, the planning of cable channel inspection routes based on the established cable channel model specifically includes:

[0026] Select one side of the cable channel model as route 1 and the other side as route 2. Take waypoints on route 1 at set distance intervals. Assuming any waypoint is M, calculate the point on route 2 that is closest to M and set it as N. Then calculate the center point of M and N. Perform curve fitting on all center points to obtain a smooth inspection route. Set the minimum inspection altitude of the UAV for the obtained inspection route.

[0027] Preferably, during the inspection phase, the drone autonomously flies along a planned route and captures real-time images. For each frame of image, it performs hazard target identification and alarm analysis based on the target's location and direction of movement, including:

[0028] For each frame of the image, a target detection algorithm is used to identify potential hazards in the image, including various types of construction machinery and vehicles. Once a potential hazard is identified, an alarm analysis based on its location and direction of movement is initiated.

[0029] Preferably, the alarm analysis method for the location is as follows:

[0030] For each frame of the image, obtain the GPS coordinate information of the drone when the frame was captured, and calculate the GPS coordinate information of each pixel in the frame based on parameters such as the gimbal attitude angle and the focal length of the camera when it was captured.

[0031] If the GPS of a pixel is within the set channel alarm range, it is marked as 1; otherwise, it is marked as 0, and the alarm binarization image of the frame is obtained.

[0032] For each identified potential hazard target, a rectangle is used to represent its position in the image. The pixel value inside the rectangle is marked as 1, and otherwise as 0, thus obtaining the target binarized image of the image.

[0033] Perform a union operation on the alarm binarized image and the target binarized image, and output the intersection mask image;

[0034] If there is a point with a value of 1 in the intersecting mask diagram, then there is a potential target within the alarm range, and an alarm will be triggered.

[0035] Preferably, after location analysis, for potential hazards outside the alarm range, a movement direction alarm analysis is performed. The specific analysis steps are as follows:

[0036] For any potential target, track the position change of the potential target in consecutive image frames to obtain the pixel distance that the potential target moves in the image. Based on the time interval between images and the pixel distance moved, the speed of the potential target relative to the UAV can be calculated.

[0037] The flight control system obtains the movement speed of the drone itself, and calculates the movement speed of the hazard relative to the ground based on the movement speed of the drone itself and the movement speed of the hazard relative to the drone. When the movement speed of the hazard relative to the ground is 0, the hazard target is determined to be stationary; otherwise, it is considered to be moving.

[0038] For a moving potential hazard, continue to calculate its component in the direction orthogonal to the channel along the drone. If the component is negative, it is determined that the movement direction is away from the channel. If the component is positive, it is determined that the movement direction is approaching the channel and an alarm is triggered. If the component is zero, it is determined that the hazard is stationary and not within the alarm area.

[0039] A second aspect of the present invention provides a UAV-based cable channel inspection system, comprising:

[0040] The channel modeling module is used during the channel construction phase to inspect the cable channel under construction using drones, and to perform geographical location calculation and modeling of the cable channel based on the language prompt word encoding module and closing operation;

[0041] The route planning module is used to plan cable channel inspection routes based on the established cable channel model.

[0042] The identification and analysis module is used for the drone to fly autonomously along the planned route and capture real-time images. It performs hazard target identification and alarm analysis on each frame of the image, as well as the location and direction of movement.

[0043] A terminal includes a processor and a storage medium, the storage medium being used to store instructions; the processor being used to perform operations according to the instructions to execute the steps of the method.

[0044] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method.

[0045] The beneficial effects of this invention are compared with those of the prior art:

[0046] This invention automatically calculates and models the geographical location of cable channels during the channel construction phase. By modeling the geographical location of cable channels in advance, flight routes can be automatically generated, resulting in high flight route planning efficiency. This effectively solves the problem that after the channel is completed, it is buried underground and its location is not visible, making it impossible to accurately determine whether various targets that need to be identified are within the protected area of ​​the channel during subsequent drone inspections.

[0047] This invention can identify potential hazards in real time during drone inspections and perform position and movement analysis on moving targets to identify whether the target is approaching the cable channel, thereby providing early warning of impending risks, preventing further harm from hazards, and improving the effectiveness and timeliness of hazard identification during inspections.

[0048] This invention improves the prompt word encoding module of the SAM model and proposes a language prompt word encoding module that can segment the "excavated tunnel" region in the image through language prompt words.

[0049] This invention proposes an adaptive kernel initialization method for closing operations, which solves the problem of poor closing operation performance caused by large differences in the morphology of missed objects, thereby improving the completeness of the next step of extracting the maximum connected component tunnel. Attached Figure Description

[0050] Figure 1 This is a flowchart of the method of the present invention;

[0051] Figure 2 A schematic diagram showing the calculation of the cable channel location;

[0052] Figure 3 A schematic diagram of the drone inspection route is generated. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, other embodiments obtained by those skilled in the art without creative effort are all within the protection scope of this invention.

[0054] Embodiment 1 of this invention provides a method for inspecting cable channels using drones. The method utilizes an algorithm to generate the channel's location during the construction phase and automatically plans the drone's inspection route. During the inspection phase, the algorithm automatically analyzes potential hazards and generates effective alarms. Figure 1 As shown, the method specifically includes:

[0055] S1: During the construction phase of the cable channel, drones are used to inspect the cable channel under construction and to calculate and model the geographical location of the cable channel.

[0056] More preferably, during the cable channel construction phase, drones are used to inspect the cable channel under construction, automatically creating a geographic location model of the cable channel to obtain its location. The cable channel modeling method is as follows:

[0057] S1.1: The drone inspects and films the excavated cable channel along the construction site, obtaining flight video.

[0058] During the construction phase of the cable tunnel, the tunnel is excavated to form a cable trench. The drone pilot controls the drone to fly along the cable tunnel under construction. During flight, the drone flies longitudinally along the top of the trench with its nose facing the trench. The drone's GPS positioning is activated, and at the same time, the gimbal camera mounted on the drone is used to film the trench, that is, the cable tunnel that has been excavated on the ground. The filmed images are stored as video, resulting in flight video.

[0059] After the flight, Method 1 was used to analyze the video, segmenting the tunnel in each frame. Combining GPS location, gimbal attitude, camera parameters, and channel width information, edge curves on both sides of the tunnel with accurate GPS information were generated. The specific steps are as follows:

[0060] S1.2: Extract each frame from the flight video and perform image segmentation to obtain the segmented image mask for the corresponding frame; specifically:

[0061] Extract footage from the flight video. For each extracted image frame, use a segmentation model such as SAM. Input the image and the prompt "dug-out tunnel". The model outputs an image mask. Pixels belonging to the "dug-out tunnel" category have a value of 1, otherwise it has a value of 0.

[0062] Because the original SAM model can only accept three forms of prompt words as input: points, rectangles, and binary segmented regions, and does not support category prompt words described in natural language, this invention improves the prompt word encoding module of the SAM model as follows:

[0063] First, the text is tokenized and then converted into vectors through an embedding matrix, thereby mapping each word to a vector space of fixed dimension, which is specified here as 512.

[0064] Since Transformer does not use recursive or convolutional structures, it uses cosine functions to generate positional codes to capture word positional information.

[0065] Add the positional encoding vector and the word vector and input them into the self-attention encoding layer, which consists of multi-head self-attention with 4 heads, allowing the model to learn the relationships among the 4 phrases such as "by", "dug out", "of", and "tunnel" in the prompt word. Designing the number of heads of the encoder based on such prior information can effectively accelerate the convergence speed during model fine-tuning.

[0066] After replacing the original prompt word encoding module of SAM with this prompt word encoding module to form a new model, the new model needs to be fine-tuned. Specifically:

[0067] Manually annotate no less than one thousand segmented images of the tunnel area using an annotation tool as the fine-tuning dataset. Freeze the parameters of the image encoder and Mask decoder of the SAM model, input the fine-tuning dataset images and annotations, and train the newly replaced prompt word encoder. Stop training when the model converges to obtain a new model that can segment the "dug out tunnel" area in the image through a language prompt word.

[0068] S1.3: Perform connectivity analysis on the mask of each frame and fit the largest connected component to a quadrilateral to obtain the quadrilateral of the channel captured in each frame of the picture; specifically:

[0069] Perform a closing operation of image morphology on the mask to make the tunnel pixels in the mask more coherent and improve the smoothness of the segmentation edge, and segment a more complete tunnel. When performing the closing operation of image morphology, a major problem is how to give an appropriate size of the closing operation kernel. To determine the appropriate kernel size, it is necessary to uniformly sample image samples from the dataset according to time, and perform omission detection statistics on the masks segmented by SAM. Statistically analyze the diameter size distribution of all omitted detection areas to obtain a histogram. Perform Gaussian smoothing on the histogram, and use the diameter at the peak position as the kernel size. Initialize the kernel with this size, and then perform a closing operation on the mask. Specifically as follows:

[0070] Randomly select more than 10% of the video frames to form an omission detection calibration dataset, segment it using the SAM model, manually review and mark the omitted detection areas according to the segmentation results. When marking, mark the omitted detection areas as 1 and other areas as 0;

[0071] Perform omission detection statistics on the marking results of the omission detection calibration dataset, statistically analyze the diameter size distribution of all omitted detection areas to obtain the corresponding histogram;

[0072] After Gaussian smoothing of the histogram, the diameter of the peak position is used as the kernel size. The kernel is initialized with this size to perform closing operations on the mask. This initialization method is a domain-adaptive closing operation kernel initialization method, which solves the problem of poor closing operation effect caused by large differences in the shape of missed objects, thereby improving the completeness of the next step of extracting the maximum connected component.

[0073] Continue performing connectivity analysis on the processed mask to find the largest connected component, and fit the connected component into a quadrilateral. Set reasonable ranges for the area, long side, and short side. If the parameters of the fitted quadrilateral are within a reasonable range, retain it; otherwise, skip the current frame and continue processing the next frame.

[0074] S1.4: Based on the UAV's GPS location and camera parameters, calculate the longitude and latitude of the quadrilateral of the channel captured in each frame;

[0075] 1) Calculate the channel design width and the pixel value ratio of the fitted short side to obtain the scale ratio s of the frame image and the actual physical world, i.e., pixels / meter.

[0076] 2) Based on the camera focal length f, scale ratio s, and triangle similarity, the straight-line distance d = f / s between the camera and the channel being photographed can be calculated.

[0077] 3) Based on the known angle α between the gimbal and the ground when the gimbal takes the image, the height h of the gimbal and the ground is calculated using trigonometric functions: h = d * cos(α).

[0078] 4) Continue processing the fitted quadrilateral. Based on the UAV's GPS position, the angle α between the gimbal and the ground, and the straight-line distance d, the position corresponding to the image center is calculated using trigonometric functions. The approximate straight-line distance between the position and the UAV is d*sin(α). Combining the UAV's pose and navigation angle, the GPS position of the image center point can be approximately calculated. Then, based on the scale s and the distance and angle between the quadrilateral's vertices and the center, the latitude and longitude of the quadrilateral can be calculated, i.e., the longitude and latitude of the cable tunnel. Figure 2 For example, when the drone captures the first frame, the position is P, and the positions of the channels are calculated as positions A, B, C, and D.

[0079] S1.5: Smoothly connect the channel positions of adjacent frame images according to the longitude and latitude to obtain the channel edges.

[0080] After calculating the longitude and latitude of the quadrilateral of the channel captured in each frame, the channel positions of adjacent frames are smoothly connected. For example... Figure 2As shown, the drone is located at position P when it takes the first frame and at position Q when it takes the next frame. At position Q, the GPS positions of the four vertices of the channel are obtained using the above method, which are E, F, G and H in sequence.

[0081] A, D, E, and H represent one side of the channel, while B, C, F, and G represent the other side.

[0082] Interpolate the position points on each side, for example, using cubic spline interpolation, to obtain a smooth curve on one side, representing the edge of the channel.

[0083] S2: Based on the cable channel model, plan the cable channel inspection route;

[0084] More preferably, such as Figure 3 As shown, after calculating the edge curves on both sides of the tunnel, a usable UAV inspection route that meets the height requirements is automatically planned using the route generation method. If the tunnel is discontinuous, i.e., there are cable channels spanning multiple locations, multiple inspection routes are generated, as follows:

[0085] S2.1: Based on S1, obtain the GPS location information of the cable channel. In this step, based on the location information, automatically generate a drone inspection route along the channel.

[0086] Select one side of the channel generated in S1 as route 1 and the other side as route 2. Take waypoints on route 1 at certain intervals. Assuming any waypoint is M, use Newton's method or other numerical optimization algorithms to calculate the point on route 2 closest to M, denoted as N. Then calculate the center position O between M and N. Perform curve fitting on all center points to obtain a smooth inspection route.

[0087] S2.2: Set the minimum inspection altitude h for the UAV for the inspection route obtained in S2.1.

[0088] In addition to GPS location, the drone's inspection route should also include altitude information. The inspection range of the passage should be set manually to be within d meters on both sides of the passage. Based on the camera's focal length and sensor imaging size during drone inspection, and using triangle similarity, the minimum altitude h that should be maintained to capture the complete inspection range of the passage can be calculated.

[0089] S3: During the inspection phase, the drone flies autonomously along the route planned in S2 and captures real-time images. It then performs hazard target identification and effective alarm analysis on each frame of the image.

[0090] More preferably, once the inspection route is generated, the drone can autonomously fly along the route for inspection. While the drone is inspecting along the route, real-time analysis algorithms are used to analyze the video footage captured by the camera, identify potential hazards, and issue effective alarms based on the target's location, direction of movement, and other information. If a target meeting the requirements is found, an alarm message is generated and sent to the monitoring system. The alarm message includes the drone's location information, target information, and image footage, as detailed below:

[0091] S3.1: During the inspection phase, the drone flies autonomously along the route planned in S2, with the gimbal camera capturing real-time footage. For each frame, a target detection algorithm is used to identify potential hazards in the image, including construction machinery and vehicles that may damage the cable channel, such as excavators, bulldozers, pile drivers, and cranes.

[0092] S3.2: Once the target is identified, begin effective alarm analysis.

[0093] a) Position analysis. For each frame of image, obtain the coordinate information (lat, lon, alt) when the frame was captured. Based on parameters such as the gimbal attitude angle and the camera focal length when capturing the image, calculate the GPS coordinate information (latpixel, lonpixel, alt) of each pixel (xp, yp) in the frame (the height change is small).

[0094] Pixel GPS information calculation process:

[0095] The drone's GPS location: (lat,lon,alt);

[0096] Gimbal attitude angles: yaw (ψ), pitch (θ), roll (φ);

[0097] Camera parameters: focal length f, sensor size (width w and height h), image resolution (pixel width W and height H);

[0098] Earth's radius: R earth

[0099] The calculation steps are as follows:

[0100] 1) Convert pixel coordinates to image coordinates

[0101] Let the coordinates of a certain pixel be (xp, yp).

[0102] • Transform to the center of the image coordinate system:

[0103]

[0104]

[0105] 2) Convert camera coordinates to world coordinates:

[0106] Project the image coordinates onto the camera coordinate system:

[0107] X = x c

[0108] Y = y c

[0109] Z = f

[0110] • Consider gimbal attitude:

[0111]

[0112] The rotation matrix R is formed by the yaw, pitch, and roll angles:

[0113] R = R yaw (ψ)·R pitch (θ)·R roll (φ)

[0114] 3) Converting world coordinates to geographic coordinates

[0115] • Convert camera coordinates to GPS coordinates, assuming the drone is on the ground:

[0116]

[0117] Final GPS coordinates:

[0118] lat pixel =lat+Δlat

[0119] lon pixel =lon + Δlon

[0120] If a pixel's GPS is within the effective alarm range of the channel, it is marked as 1; otherwise, it is marked as 0, resulting in the effective alarm binarized image of that frame. For each identified target, its position in the image is represented by a rectangle, with the pixel value within the rectangle marked as 1; otherwise, it is marked as 0, resulting in the target binarized image of that frame. The effective alarm binarized image and the target binarized image are then combined to output an intersection mask. If there is a point with a value of 1 in the intersection mask, then there is a target within the effective alarm range, and an alarm should be triggered.

[0121] b) Motion Analysis. For targets outside the effective alarm range, motion analysis is performed. If the movement direction is towards the passage, it is identified as a risk target. The specific analysis steps are as follows: For any target, an optical flow algorithm is used to track the positional changes of the potential hazard target in consecutive image frames. The pixel distance the potential hazard target moves in the image can be obtained. Based on the time interval between images and the pixel distance moved, the movement speed of the potential hazard relative to the UAV can be calculated. Within the bounding box of each frame, the optical flow algorithm is used to calculate the movement speed within the bounding box.

[0122] The flight control system obtains the movement speed of the drone itself (relative to the ground). Based on the movement speed of the drone itself and the movement speed of the hazard relative to the drone, the movement speed of the hazard relative to the ground is calculated. When the movement speed of the hazard relative to the ground is 0, the hazard target is determined to be stationary; otherwise, it is considered to be moving.

[0123] For a moving target, continue to calculate its component in the direction orthogonal to the channel along the drone. That is, perform a dot product operation between the speed of the hazard and the unit vector in the direction orthogonal to the channel. If the result is negative, it is determined that the target is moving away from the channel. If the result is positive, it is determined that the target is approaching the channel and an alarm should be triggered. If the component is zero, it is determined that the hazard is stationary and not within the effective alarm area.

[0124] Embodiment 2 of the present invention provides a UAV inspection system for cable channels, the system comprising:

[0125] The channel modeling module is used during the channel construction phase to inspect the cable channel under construction using drones, and to perform geographical location calculation and modeling of the cable channel based on the language prompt word encoding module and closing operation;

[0126] The route planning module is used to plan cable channel inspection routes based on the established cable channel model.

[0127] The identification and analysis module is used for the UAV to fly autonomously along the planned route and capture real-time images. It performs hazard target identification and effective alarm analysis on the location and direction of movement of each frame of the image.

[0128] Embodiment 3 of the present invention provides a terminal, including a processor and a storage medium; the storage medium is used to store instructions;

[0129] The processor is configured to operate according to the instructions to execute the steps of the method.

[0130] Embodiment 4 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0131] The beneficial effects of this invention are compared with those of the prior art:

[0132] This invention automatically calculates and models the geographical location of cable channels during the channel construction phase. By modeling the geographical location of cable channels in advance, flight routes can be automatically generated, resulting in high flight route planning efficiency. This effectively solves the problem that after the channel is completed, it is buried underground and its location is not visible, making it impossible to accurately determine whether various targets that need to be identified are within the protected area of ​​the channel during subsequent drone inspections.

[0133] This invention can identify potential hazards in real time during drone inspections and perform position and movement analysis on moving targets to identify whether the target is approaching the cable channel, thereby providing early warning of impending risks, preventing further harm from hazards, and improving the effectiveness and timeliness of hazard identification during inspections.

[0134] This invention improves the prompt word encoding module of the SAM model and proposes a language prompt word encoding module that can segment the "excavated tunnel" region in the image through language prompt words.

[0135] This invention proposes an adaptive kernel initialization method for closing operations, which solves the problem of poor closing operation performance caused by large differences in the morphology of missed objects, thereby improving the completeness of the next step of extracting the maximum connected component tunnel.

[0136] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0137] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0138] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0139] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for inspecting cable channels using unmanned aerial vehicles (UAVs), characterized in that, The method includes: During the cable channel construction phase, drones are used to inspect the cable channels under construction. Based on a language prompt word encoding module and a closing operation, the geographical location of the cable channels is calculated and modeled. The kernel initialization method used in the closing operation is as follows: more than 10% of video frames are randomly selected to form a missed detection calibration dataset. The dataset is segmented using the SAM model. Based on the segmentation results, missed detection areas are manually reviewed and marked. During marking, missed detection areas are marked as 1, and other areas are marked as 0. Missed detection statistics are performed on the marked results of the missed detection calibration dataset. The diameter distribution of all missed detection areas is statistically analyzed to obtain the corresponding histogram. After Gaussian smoothing of the histogram, the diameter at the peak position is used as the kernel size. The kernel is initialized with this size to perform the closing operation on the mask. Based on the established cable channel model, plan the cable channel inspection route; During the inspection phase, the drones fly autonomously along the planned route and capture real-time images. Each frame of the image is used to identify potential hazards and analyze their location and direction of movement for alarm purposes.

2. The method for inspecting cable channels by unmanned aerial vehicles according to claim 1, characterized in that: During the cable tunnel construction phase, drones are used to inspect the cable tunnel under construction, and the geographical location of the cable tunnel is calculated and modeled based on a language prompt word encoding module and closing operations. Specifically, this includes: The drone inspects the cable channel under construction, and its onboard gimbal camera captures real-time footage of the excavated cable channel, producing flight video. Each frame of the flight video is extracted and image segmentation is performed based on the language prompt word encoding module to obtain the image mask of the corresponding frame segmentation; Based on the closing operation, connectivity analysis is performed on the mask of each frame, and the largest connected component is fitted into a quadrilateral to obtain the quadrilateral of the channel captured in each frame of the image. Based on the UAV's GPS location and camera parameters, calculate the longitude and latitude of the quadrilateral of the channel captured in each frame of the image; The channel positions of adjacent frames are smoothly connected based on the longitude and latitude to obtain the channel edges.

3. The method for inspecting cable channels by unmanned aerial vehicles according to claim 2, characterized in that: The process of extracting each frame of the flight video and performing image segmentation based on the language prompt word encoding module to obtain the corresponding frame segmented image mask includes: The SAM model takes an image and the cue phrase "excavated tunnel" as input and outputs a segmented image mask, where pixels belonging to the "excavated tunnel" category have a value of 1, otherwise 0. The SAM model uses a pre-trained language cue word encoding module to recognize the "excavated tunnel" cue phrase. Specifically: The language prompt word encoding module tokenizes the text and converts it into word vectors through an embedding matrix. At the same time, it uses a cosine function to generate positional encoding vectors. The positional encoding vectors and word vectors are added together and then input into a self-attention encoding layer, which learns the word relationships of the input vectors through multi-head self-attention.

4. The method for inspecting cable channels by unmanned aerial vehicles according to claim 2, characterized in that: The process of performing connectivity analysis on the mask of each frame based on closing operations and fitting the largest connected component into a quadrilateral to obtain the quadrilateral of the channels captured in each frame image includes: Perform image morphology closing operations on the mask of the current frame to segment out more complete tunnels; perform connectivity analysis on the mask of the current frame to find the largest connected component, fit the connected component into a quadrilateral, and retain it if the parameters of the fitted quadrilateral are within the set range; otherwise, skip the current frame and continue processing the next frame.

5. The method for inspecting cable channels by unmanned aerial vehicles according to claim 1, characterized in that: The planning of cable tunnel inspection routes based on the established cable tunnel model specifically includes: Select one side of the cable channel model as route 1 and the other side as route 2. Take waypoints on route 1 at set distance intervals. Assuming any waypoint is M, calculate the point on route 2 that is closest to M and set it as N. Then calculate the center point of M and N. Perform curve fitting on all center points to obtain a smooth inspection route. Set the minimum inspection altitude of the UAV for the obtained inspection route.

6. The method for inspecting cable channels by unmanned aerial vehicles according to claim 1, characterized in that: During the inspection phase, the drone autonomously flies along a planned route and captures real-time images. Each frame of the image is used for hazard target identification and alarm analysis based on its location and direction of movement, including: For each frame of the image, a target detection algorithm is used to identify potential hazards in the image, including various types of construction machinery and vehicles. Once a potential hazard is identified, an alarm analysis based on its location and direction of movement is initiated.

7. A method for inspecting cable channels by unmanned aerial vehicles according to claim 6, characterized in that: The alarm analysis method for the specified location is as follows: For each frame of the image, obtain the GPS coordinate information of the drone when the frame was captured, and calculate the GPS coordinate information of each pixel in the frame based on parameters such as the gimbal attitude angle and the focal length of the camera when it was captured. If the GPS of a pixel is within the set channel alarm range, it is marked as 1; otherwise, it is marked as 0, and the alarm binarization image of the frame is obtained. For each identified potential hazard target, a rectangle is used to represent its position in the image. The pixel value inside the rectangle is marked as 1, and otherwise as 0, thus obtaining the target binarized image of the image. Perform a union operation on the alarm binarized image and the target binarized image, and output the intersection mask image; If there is a point with a value of 1 in the intersecting mask diagram, then there is a potential target within the alarm range, and an alarm will be triggered.

8. A method for inspecting cable channels by unmanned aerial vehicles according to claim 6, characterized in that: After location analysis, alarm analysis is performed on the movement direction of potential hazards that are not within the alarm range. The specific analysis steps are as follows: For any potential target, track the position change of the potential target in consecutive image frames to obtain the pixel distance that the potential target moves in the image. Based on the time interval between images and the pixel distance moved, the speed of the potential target relative to the UAV can be calculated. The speed of the drone body is obtained through the flight control system, and the speed of the hazard relative to the ground is calculated based on the speed of the drone body and the speed of the hazard relative to the drone. When the speed of movement of the hazard relative to the ground is 0, the hazard target is judged to be stationary; otherwise, it is considered to be moving. For a moving potential hazard, continue to calculate its component in the direction orthogonal to the channel along the drone. If the component is negative, it is determined that the movement direction is away from the channel. If the component is positive, it is determined that the movement direction is approaching the channel and an alarm is triggered. If the component is zero, it is determined that the hazard is stationary and not within the alarm area.

9. A UAV-based cable channel inspection system, utilizing the method described in any one of claims 1-8, characterized in that, The system includes: The channel modeling module is used during the channel construction phase to inspect the cable channel under construction using drones, and to perform geographical location calculation and modeling of the cable channel based on the language prompt word encoding module and closing operation; The route planning module is used to plan cable channel inspection routes based on the established cable channel model. The identification and analysis module is used for the drone to fly autonomously along the planned route and capture real-time images. It performs hazard target identification and alarm analysis on each frame of the image, as well as the location and direction of movement.

10. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-8.

Images