Method, device and equipment for segmenting icing area of power transmission line and storage medium

Abstract

The embodiment of the invention provides a segmentation method and device for an icing area of a power transmission line, equipment and a storage medium. The method comprises the following steps: acquiring a target image of an unmanned aerial vehicle at a target acquisition moment; wherein the target image comprises a power transmission line; based on a preset visual large model, extracting an initial mask image of an icing area in the target image; determining a mask image of an icing area in the target image according to the initial mask image and a preset Kalman filtering algorithm; and according to the mask image of the icing area, segmenting the icing area in the target image. The method is used for achieving the effect of reducing the training cost.

Description

Technical Field

[0001] The present application relates to the technical field of ice detection, and in particular to a method, device, equipment and storage medium for segmenting ice-covered areas of power transmission lines. Background Art

[0002] In cold climates, ice easily forms on the surface of power transmission lines. When ice accumulates and increases the load on conductors, it can damage them, collapse towers, and even cause regional power outages. Efficiently and accurately detecting ice on transmission lines in complex and changing environments has become a key research topic in power line inspection.

[0003] Related technologies often train deep learning models based on a large number of sample ice-covered images, enabling them to segment ice-covered areas on transmission lines. However, this approach requires a large number of ice-covered images to train the deep learning model to ensure segmentation accuracy, resulting in high training costs. Summary of the Invention

[0004] The embodiments of the present application provide a method, apparatus, device, and storage medium for segmenting ice-covered areas of transmission lines, so as to reduce training costs.

[0005] In a first aspect, an embodiment of the present application provides a method for segmenting an ice-covered area of ​​a transmission line, comprising:

[0006] Acquire a target image of the UAV at the target acquisition moment; wherein the target image includes a power transmission line;

[0007] Extracting an initial mask image of the ice-covered area in the target image based on a preset visual macro model;

[0008] According to the initial mask image and a preset Kalman filter algorithm, a mask image of the ice-covered area in the target image is determined; and according to the mask image of the ice-covered area, the ice-covered area in the target image is segmented.

[0009] In a possible implementation, determining the mask image of the ice-covered area in the target image based on the initial mask image and a preset Kalman filter algorithm includes:

[0010] Determining whether the initial mask image is normal according to the initial mask image and a preset Kalman filter algorithm;

[0011] If yes, performing corrosion and expansion processing on the initial mask image to obtain the mask image;

[0012] If not, the initial mask image is corrected to obtain the mask image.

[0013] In a possible implementation, determining whether the initial mask image is normal based on the initial mask image and a preset Kalman filter algorithm includes:

[0014] Extracting mask features according to the initial mask image; wherein the mask features include centroid coordinates, circumscribed rectangle parameters, and area of ​​the initial mask image;

[0015] Determining a current observation state and a predicted state according to the mask feature and the Kalman filter algorithm;

[0016] Whether the initial mask image is normal is determined according to the Mahalanobis distance between the current observed state and the predicted state.

[0017] In a possible implementation, the correcting the initial mask image to obtain the mask image includes:

[0018] Obtaining a historical mask image before the target acquisition moment;

[0019] Determining a predicted mask image according to the predicted state;

[0020] The historical mask image, the initial mask image, and the predicted mask image are fused to obtain the mask image.

[0021] In a possible implementation, extracting an initial mask image of the ice-covered area in the target image based on a preset large visual model includes:

[0022] extracting image features from the target image based on an image encoder of the visual macro model;

[0023] Obtaining prompt information, and extracting prompt features from the prompt information based on a prompt encoder of the visual macro model;

[0024] A mask decoder based on the visual large model determines the initial mask image according to the image features and the hint features.

[0025] In one possible implementation, the method further includes:

[0026] determining an area ratio of the ice-covered area to the target image;

[0027] determining a risk value at the target acquisition moment according to the area ratio;

[0028] A test report is generated based on the risk value at each collection moment.

[0029] In a possible implementation, obtaining a target image of the drone at the target acquisition time includes:

[0030] Obtain the original image of the UAV at the target acquisition moment;

[0031] The original image is subjected to denoising, contrast enhancement and dedistortion processing in sequence to obtain the target image.

[0032] In a second aspect, an embodiment of the present application provides a device for segmenting an ice-covered area of ​​a power transmission line, comprising:

[0033] An acquisition module, configured to acquire a target image of the UAV at the target acquisition moment; wherein the target image includes a power transmission line;

[0034] An extraction module, configured to extract an initial mask image of the ice-covered area in the target image based on a preset visual macro model;

[0035] A segmentation module is used to determine the mask image of the ice-covered area in the target image based on the initial mask image and a preset Kalman filter algorithm; and to segment the ice-covered area in the target image based on the mask image of the ice-covered area.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor;

[0037] The memory stores computer-executable instructions;

[0038] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0041] The methods, devices, equipment, and storage media for segmenting ice-covered areas on power transmission lines provided in the embodiments of the present application enable electronic devices to quickly segment ice-covered areas from a target image based on a preset large visual model, generate an initial mask image, and dynamically predict and correct the state of the ice-covered area using a Kalman filter algorithm to obtain a more accurate mask image of the ice-covered area, thereby enabling more accurate segmentation of the ice-covered area from the target image. This approach allows accurate segmentation of ice-covered areas based on a pre-trained large visual model, eliminating the need for additional model training and reducing training costs. Furthermore, combined with the Kalman filter algorithm, the accuracy of the segmented ice-covered area can be guaranteed based on the spatiotemporal continuity of the ice-covered area. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] Figure 1 A schematic diagram of an application scenario provided by this application;

[0044] Figure 2 Schematic diagram of the process of segmenting the ice-covered area of ​​the transmission line provided in this application Figure 1 ;

[0045] Figure 3 Schematic diagram of the process of segmenting the ice-covered area of ​​the transmission line provided in this application Figure 2 ;

[0046] Figure 4 A schematic diagram of an application scenario provided by this application;

[0047] Figure 5 A schematic diagram of the structure of the device for dividing the ice-covered area of ​​a transmission line provided in this application;

[0048] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application.

[0049] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0050] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0051] In related technologies, deep learning models are often trained based on a large number of sample ice-covered images, so that the deep learning model can segment the ice-covered areas on the transmission lines. However, this method requires a large number of ice-covered images to train the deep learning model to ensure segmentation accuracy, and the training cost is relatively high. Therefore, the present application provides a method for segmenting ice-covered areas of transmission lines, which uses a pre-trained large visual model to segment the ice-covered areas in the image. Since the large visual model is trained based on a large amount of sample data, there is no need for user retraining, which saves training costs. In addition, in order to ensure the accuracy of the ice-covered area, the segmented mask image is judged to be spatiotemporally continuous, which further improves the accuracy of the segmentation of the ice-covered area.

[0052] Figure 1 This is a schematic diagram of an application scenario provided by this application. Figure 1 As shown, the specific application scenario of this application is: the drone can inspect the transmission line according to the preset flight trajectory, and take images of the transmission line, and return them to the electronic equipment. The electronic equipment processes the images taken by the drone and segments the ice-covered area on the transmission line.

[0053] The above-mentioned drone is an unmanned aerial vehicle used for power inspection. It is equipped with image acquisition equipment, such as a camera, and can automatically fly along a preset path and collect images of transmission lines.

[0054] Optionally, the UAV's flight control system can automatically correct and execute predetermined waypoint tasks point by point based on the Global Navigation Satellite System (GNSS), so as to maintain a basically parallel or relatively fixed lateral and pitch angle with the transmission line in space, so as to achieve continuous observation and shooting of the line. For example, a Kalman filter algorithm can be used for attitude solution and position correction. This method can not only improve the stability of the UAV in the air, but also ensure accurate path following of the target line during flight. In addition, the flight control system will have a built-in fault response strategy: when the GNSS signal quality seriously degrades or the signal is lost, the UAV can automatically switch to attitude mode for short-term manual control, or it can perform fault protection (Fail-safe) measures such as "automatic hovering" or "automatic return".

[0055] The electronic device is used to segment the received image. The execution subject of the embodiment of the present application can be an electronic device with processing capabilities, such as a computer, a server, etc., and the embodiment of the present application does not limit this.

[0056] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0057] Figure 2 Schematic diagram of the process of segmenting the ice-covered area of ​​the transmission line provided in this application Figure 1 ,like Figure 2 As shown, the method includes:

[0058] S101: Acquire a target image of the UAV at the target acquisition moment.

[0059] Exemplarily, the target acquisition moment represents the specific time point at which the drone performs the image capture operation. The target image represents the image data to be segmented corresponding to the target acquisition moment, and may include a power transmission line; the target image may be the original image acquired by the drone, or an image after image preprocessing. The electronic device may communicate with the drone in real time to obtain the target image of the drone at the target acquisition moment, or the drone may store the acquired image in a storage device during the inspection process. After the inspection is completed, the electronic device may extract the target image at the target acquisition moment from the storage device. The storage device may be, for example, a Secure Digital (SD) card, a solid-state drive, or the like.

[0060] S102: extracting an initial mask image of the ice-covered area in the target image based on a preset large visual model.

[0061] Exemplarily, the aforementioned large visual model refers to a deep learning model that has been pre-trained on a large number of image datasets, such as a segmented arbitrary model (SAM). The large visual model has powerful image semantic understanding and target segmentation capabilities, and can identify various target objects in the image. The initial mask image of the ice-covered area represents the binary image output by the large visual model after processing the target image, wherein the pixel value corresponding to the ice-covered area is marked as 1, and the pixel value of the background area is marked as 0, which can reflect the position and range of the ice-covered area in the target image.

[0062] In one example, the electronic device may input the target image into the visual large model and output an initial mask image of the ice-covered area in the target image.

[0063] S103 : Determine a mask image of the ice-covered area in the target image according to the initial mask image and a preset Kalman filter algorithm.

[0064] For example, the Kalman filter algorithm is an optimal recursive estimation algorithm based on a linear system state-space model. By fusing the system's dynamic model with observational data, it predicts and corrects the system state, effectively addressing noise and uncertainty in the data. The mask image of the ice-covered area is a binary image obtained by optimizing the initial mask image using the Kalman filter algorithm. Compared to the initial mask image, the mask image of the ice-covered area more accurately reflects the actual boundaries and shape of the ice-covered area.

[0065] It is understandable that the ice-covered areas at different acquisition times should have spatiotemporal continuity. Although the locations of the drones shooting at different times may be different, if their flight path is around a specific transmission line and the environmental factors of the same transmission line are generally relatively stable within a certain time and space range, this makes the growth conditions of the ice relatively stable, resulting in the spatiotemporal continuity of the ice-covered features. In other words, even if the drones shoot at different locations, under the same environmental conditions, the ice-covered features will show similar change trends. Therefore, combined with the Kalman filter algorithm, the spatiotemporal continuity of the ice-covered areas can be used to further modify the initial mask image, overcome the influence of factors such as misjudgment of the large visual model and image noise, and improve the accuracy of the segmented ice-covered areas.

[0066] In one example, the electronic device can determine whether the initial mask image is normal based on the initial mask image and a preset Kalman filter algorithm, using the spatiotemporal continuity of the mask features. If so, the initial mask image can be used as the mask image of the ice-covered area; if not, the initial mask image can be corrected based on the preset Kalman filter algorithm to obtain the mask image of the ice-covered area. For example, the geometric features of the ice-covered area can be extracted, and the state transfer equation and observation equation of the Kalman filter algorithm can be constructed based on the geometric features, the predicted state can be calculated, and the state estimation value can be obtained. The state estimation value represents the geometric features of the predicted ice-covered area, such as the center of mass coordinates and shape parameters of the ice-covered area, and then the initial mask image is corrected based on the state estimation value to obtain the mask image of the ice-covered area.

[0067] S104: Segment the ice-covered area in the target image according to the mask image of the ice-covered area.

[0068] For example, an ice-covered area represents an area on the surface of a power transmission line covered in ice. The electronic device can use a mask image of the ice-covered area as a mask to perform a pixel-by-pixel operation on the target image: for locations where the pixel value in the mask image is 1, the pixel at the corresponding location in the target image is retained; for locations where the pixel value in the mask image is 0, the pixel at the corresponding location in the target image is set to a specific value, such as black or transparent, thereby separating the ice-covered area from the target image.

[0069] The method for segmenting ice-covered areas on power transmission lines provided in an embodiment of the present application allows an electronic device to quickly segment the ice-covered area from a target image based on a pre-set large visual model, generate an initial mask image, and dynamically predict and correct the state of the ice-covered area using a Kalman filter algorithm, thereby obtaining a more accurate mask image of the ice-covered area. This allows for more accurate segmentation of the ice-covered area from the target image. This method allows for accurate segmentation of the ice-covered area based on a pre-trained large visual model, eliminating the need for additional model training and reducing training costs. Furthermore, combined with the Kalman filter algorithm, the accuracy of the segmented ice-covered area can be guaranteed based on the spatiotemporal continuity of the ice-covered area.

[0070] Figure 3 Schematic diagram of the process of segmenting the ice-covered area of ​​the transmission line provided in this application Figure 2 , Figure 4 A schematic diagram of an application scenario provided for this application, such as Figure 3 and Figure 4 As shown, this embodiment Figure 2 Based on the embodiment, taking the visual large model SAM as an example, a method for segmenting the ice-covered area of ​​the transmission line is described in detail. The method includes:

[0071] S201: Obtain the original image of the UAV at the target acquisition moment.

[0072] For example, the raw image represents image data captured by a drone photographing a power transmission line at a target acquisition time, and may include the power transmission line. The electronic device may communicate with the drone in real time to obtain the raw image captured by the drone at the target acquisition time. Alternatively, the drone may store the captured image in a storage device during an inspection. After the inspection is complete, the electronic device may retrieve the raw image captured at the target acquisition time from the storage device, such as a Secure Digital (SD) card or a solid-state drive.

[0073] S202 , performing denoising, contrast enhancement, and dedistortion processing on the original image in sequence to obtain a target image.

[0074] For example, the electronic device can use a mean filter or Gaussian filter algorithm to calculate the neighborhood pixel mean or weighted mean through a sliding window, replace the center pixel value, and eliminate salt and pepper noise, Gaussian noise, etc. Specifically, taking mean filtering as an example, the formula of mean filtering is shown in the following formula (1). Based on the formula of mean filtering, each pixel in the original image can be subjected to mean filtering to obtain the original image after denoising. For any point (x, y) in the original image, by calculating its neighborhood window S xy The average value of the pixel values ​​g(s,t) of all pixels in the neighborhood is used to obtain the pixel value f(x,y) after mean filtering. In other words, the average value of the pixels in the neighborhood is used to replace the original pixel value, thereby achieving the effect of smoothing the image and removing noise.

[0075]

[0076] Among them, f(x,y) represents the pixel value after mean filtering at point (x,y); g(s,t) represents the pixel value at coordinates (s,t) in the original image; S xy Represents a neighborhood window centered at point (x, y), usually a rectangular or square area; m and n represent the size of the neighborhood window in the horizontal and vertical directions (i.e., the number of pixel rows and columns in the window), respectively.

[0077] Contrast enhancement refers to increasing the difference in brightness between different areas of an image, making image details clearer and facilitating subsequent feature extraction and analysis. Electronic devices can use histogram equalization to calculate the grayscale distribution of image pixels, map the original grayscale to a new distribution range, and enhance the overall contrast. Specifically, the following formulas (2) to (5) can be used to perform histogram equalization on the original image after denoising to obtain the enhanced original image.

[0078]

[0079]

[0080] cdf y (y′)=cdf y (T(k))=cdf x (k), Formula (4)

[0081] y′=y·(max{x}-min{x})+min{x}, formula (5)

[0082] Among them, p x (i) is the probability mass function, which represents the probability of the pixel with gray value i appearing in the image; n irepresents the number of pixels with grayscale value i in the image; n is the total number of pixels in the image; L is the total number of possible grayscale levels in the image; cdf x (i) is the cumulative distribution function of the image grayscale value, which represents the sum of the probabilities of occurrence of all pixels with grayscale values ​​less than or equal to i; T(k) is the grayscale transformation function; max{x} and min{x} represent the maximum and minimum grayscale values ​​in the original image after denoising, respectively.

[0083] Referring to the above formula, based on formula (2), the probability of each gray level appearing can be obtained by counting the proportion of the number of gray level pixels in the original image after denoising to the total number of pixels; based on formula (3), the probability mass function p can be obtained by x (j), accumulate from j = 0 to j = i to obtain the cumulative distribution function cdf x (i) is used to reflect the cumulative distribution of grayscale values ​​in the image; then, based on formula (4), the original grayscale value k is mapped to the new grayscale value y′, and the goal of histogram equalization is achieved by making the cumulative distribution functions before and after the transformation equal, that is, redistributing the grayscale values ​​of the image to make the grayscale distribution of the image more uniform; finally, based on formula (5), the transformed grayscale value y is linearly mapped to the value range of the grayscale value of the original image after denoising, and the final output grayscale value y′ is obtained, ensuring that the grayscale value of the equalized image is within a reasonable and expected range, and the original image after enhancement processing is obtained.

[0084] Dedistortion processing refers to restoring the true shape of an image through a calibration algorithm, targeting geometric distortions (such as barrel distortion and pincushion distortion) caused by the camera lens. Electronic equipment can calibrate the camera installed on the drone in advance to obtain the camera's intrinsic and extrinsic parameters. These parameters are then used to dedistort the enhanced original image to obtain the target image. The intrinsic parameters include focal length, principal point coordinates, and distortion coefficients, while the extrinsic parameters include rotation and translation matrices. Specifically, dedistortion processing can be performed using the following formulas (6) to (9).

[0085]

[0086]

[0087] r 2 =x′ 2 +y′ 2 , formula (8)

[0088]

[0089] Where (u, v) represents the final pixel coordinates of the image, that is, the pixel position of the point on the image plane after dedistortion and transformation; (x″, y″) represents the normalized coordinates after distortion correction; f x and f y Represents the focal length of the camera in the x and y directions respectively; c x and c y They represent the coordinates of the principal point of the image, that is, the position of the intersection of the camera optical axis and the image plane in the pixel coordinate system; (x′, y′) represents the original normalized coordinates; k1, k2, k3, k4, k5 and k6 are radial distortion coefficients, which are used to correct radial distortion caused by factors such as lens shape; p1 and p2 are tangential distortion coefficients, which are used to correct tangential distortion caused by factors such as lens installation; s1, s2, s3 and s4 are thin lens distortion coefficients, which are used to further correct complex distortion caused by the lens; r represents the distance from a point to the origin on the camera normalized plane; (X c ,Y c ,Z c ) is the coordinate of the point in the camera coordinate system.

[0090] Referring to the above formula, based on formula (9), the X in the camera coordinate system is c and Y c Divide by Z c , obtain the normalized coordinates (x′, y′) to project the point in the three-dimensional camera coordinate system onto the two-dimensional plane; then, based on formula (7) and formula (8), the original normalized coordinates (x′, y′) are calculated through a series of coefficients to obtain the corrected normalized coordinates (x″, y″), thereby eliminating or reducing the distortion of the image; finally, based on formula (6), the image coordinates are converted into actual pixel coordinates by combining the corrected normalized coordinates with the focal length and principal point coordinates of the camera to obtain the target image.

[0091] S203: An image encoder based on a large visual model extracts image features from the target image.

[0092] Exemplarily, SAM includes an image encoder (Image Encoder), a prompt encoder (Prompt Encoder) and a mask decoder (Mask Decoder). The image encoder usually adopts a large-scale ViT (Vision Transformer) to learn high-dimensional feature representations of the input image. The prompt encoder processes the "user input" required for the segmentation task, which often appears in the form of text prompts or user click prompts in the ice-covered line segmentation task. Finally, the mask decoder combines the image features with the prompt information, infers the segmentation results of the specific target, and outputs one or more potential segmentation masks (Mask). In this step, the electronic device can extract features of the target image I1 through the image encoder Encoder1 based on F1=Encoder1(I1) to obtain the image feature F1 in the target image.

[0093] S204: Obtain prompt information, and extract prompt features from the prompt information based on the prompt encoder of the visual large model.

[0094] Exemplarily, the prompt information represents the input provided to assist the model in locating the ice-covered area, and may be, for example, a text description ("ice-covered area of ​​the transmission line"), an image annotation point, or the like. The electronic device may receive a text description input by the user, or may receive a point manually marked or an area selected by the user on the image to obtain the prompt information I2. Furthermore, based on F2=Encoder2(I2), the prompt information may be input to the prompt encoder Encoder2 of the SAM, which converts the prompt information into an embedding vector, encodes the text semantics or the spatial location information of the image annotation, and forms a prompt feature F2.

[0095] S205 . The mask decoder based on the visual large model determines an initial mask image according to the image features and the prompt features.

[0096] Exemplarily, the mask decoder Decoder may match the image feature F1 with the prompt information F2 based on R=Decoder(F1, F2), and generate an initial mask image R for the ice-covered area.

[0097] S206 : Determine whether the initial mask image is normal based on the initial mask image and a preset Kalman filter algorithm.

[0098] Exemplarily, the electronic device can determine whether the initial mask image is normal based on the initial mask image and a preset Kalman filter algorithm by utilizing the spatiotemporal continuity of the mask features. For example, it can determine whether the initial mask image is normal by determining whether the Mahalanobis distance between the current observed state and the predicted state is greater than or equal to a threshold.

[0099] Specifically, mask features can be extracted from the initial mask image. The mask features include the coordinates of the initial mask image's centroid, parameters of the bounding rectangle, and its area. Based on the mask features and the Kalman filter algorithm, the current observed state and predicted state are determined. Based on the Mahalanobis distance between the current observed state and the predicted state, it is determined whether the initial mask image is normal. If so, step S207 is executed; if not, step S208 is executed.

[0100] For example, the current observed state represents an estimated value of the observed state of the ice-covered area obtained by a Kalman filter algorithm based on the currently extracted mask features. The predicted state represents the state of the ice-covered area at the current moment, predicted by the Kalman filter algorithm based on the state at the previous moment.

[0101] The electronic device can calculate the centroid coordinates (x, y) of the initial mask image based on the centroid coordinate calculation formula. Simultaneously, a preset function can be used to calculate the minimum bounding rectangle parameters of the initial mask image, such as the length l, width ω, and rotation angle α, as well as the area of ​​the mask region. The area of ​​the mask region can be determined by counting the number of pixels with a pixel value of 1.

[0102] Furthermore, the extracted mask features can be used as elements of the state vector to obtain the state vector X at the target acquisition time t t , X t It can be expressed as and They represent the velocity components of the center of mass coordinates, and Represent the velocity components of the length and width of the minimum circumscribed rectangle, represents the velocity component of the rotation angle, Represents the velocity component of the area; and according to the preset state transfer matrix F, the preset process noise matrix Q and the state estimation of the previous acquisition time t-1 of the target acquisition time According to the state transfer equation Predict the state of the target at the time of acquisition Then, the state vector X that represents the characteristics of the ice-covered area at the time of target acquisition can be extracted. t And the preset observation matrix H, according to the observation equation Z t =HX t , get the current observation state Z at the target acquisition time t ; and according to the predicted state of the target collection time according to Get the predicted status Finally, whether the initial mask image is normal can be determined based on the Mahalanobis distance between the current observed state and the predicted state.

[0103] Optionally, the preset process noise matrix Q and the covariance matrix P of the previous acquisition moment of the target acquisition moment can be used. t-1|t-1 , according to P t|t-1 =FP t-1|t-1 F T +Q, the state covariance matrix P at the time of target acquisition prediction t|t-1 ; and according to K t =P t|t-1 H T (HP t|t-1 H T ) -1 Calculate the Kalman gain K t , which can be based on Get the updated state estimate Based on P t|t =(IK t H)P t|t-1 , and obtain the updated state covariance matrix for subsequent judgment and processing at the acquisition moment.

[0104] S207 , performing erosion and dilation processing on the initial mask image to obtain a mask image.

[0105] For example, since the actual flight and imaging process is often affected by factors such as lighting conditions, background complexity, and the model's own threshold, the segmentation result (initial mask image) will inevitably have a small number of misjudgments or unclear edges. Therefore, it is necessary to erode and dilate the initial mask image to erase isolated noise points or fill small holes in local areas to obtain a more accurate mask image. Specifically, the initial mask image can be eroded and dilated with reference to the following formulas (10) and (11) to obtain a mask image.

[0106]

[0107] Where src(x,y) represents the pixel value of the initial mask image at the coordinate (x,y); dst(x,y) represents the pixel value of the mask image at the coordinate (x,y); (x′,y′) represents the coordinates within the structure element, which can be a rectangle, circle, etc.; element(x′,y′)≠0 defines the position of non-zero values ​​within the structure element.

[0108] Referring to the above formula, based on formula (10), for each pixel point (x, y) in the original image (initial mask image), with it as the center, within the range covered by the structural element, find the maximum value of the pixel value at the corresponding position in the initial mask image, and assign this maximum value to the corresponding pixel point (x, y) of the processed image (mask image); and based on formula (11), find the minimum value of the pixel value at the corresponding position in the original image within the coverage of the structural element, and assign it to the pixel point (x, y) of the processed image.

[0109] S208 , performing correction processing on the initial mask image to obtain a mask image.

[0110] Exemplarily, the electronic device may combine the historical mask image and the initial mask image to generate a mask image by weighting.

[0111] In some possible implementations, a historical mask image before the target acquisition moment is obtained; a predicted mask image is determined according to the predicted state; and the historical mask image, the initial mask image, and the predicted mask image are fused to obtain a mask image.

[0112] Exemplarily, the historical mask image represents a mask image of the ice-covered area obtained by processing the image collected by the drone before the target acquisition moment, and is used to indicate the status information of the ice-covered area before the target acquisition moment. The predicted mask image represents a binary image converted from the predicted features of the ice-covered area (such as centroid coordinates, circumscribed rectangle parameters, area, etc.) according to the predicted state, to represent the predicted distribution of the ice-covered area. It can be understood that in the process of processing the image at each acquisition moment, a mask image at each acquisition moment can be obtained, that is, the historical mask image. The electronic device can extract the historical mask image before the target acquisition moment from the database, such as the first 3 frames of historical mask images before the target acquisition moment. The embodiment of the present application does not limit the number of historical mask images.

[0113] Furthermore, as mentioned above, based on the Kalman filter algorithm, the predicted state is obtained The predicted state and the predicted state at the target acquisition time Based on the predicted state, the predicted ice-covered area features, such as the predicted centroid coordinates, bounding rectangle parameters, and area, can be determined. Based on the predicted ice-covered area features, a predicted mask image is constructed on the blank image. For example, a rectangular area is drawn with the centroid coordinates as the center according to the length, width, and rotation angle of the bounding rectangle. The internal pixels are filled according to the area information, and the pixel values ​​in this area are set to 1, while the pixel values ​​in the remaining areas are set to 0 to generate a predicted mask image.

[0114] Finally, the historical mask image, the initial mask image, and the predicted mask image can be weighted fused according to the following formula (12) to obtain the mask image. For example, for each pixel in the image, a weighted fusion process can be performed according to formula (12) to obtain the initial pixel value in the mask image, and the initial pixel value can be thresholded. If the initial pixel value is greater than or equal to the preset threshold, the pixel value of the mask image at that point is set to 1, otherwise it is set to 0, thereby obtaining the mask image.

[0115]

[0116] in, represents the mask image; M t-n Represents the historical mask image, t represents the target acquisition time, N represents the N-frame historical mask image, n=[0, N-1]; when n=0, M t-n represents the initial mask image; M pred represents the predicted mask image; ω t-n Represents the weight coefficient of the historical mask image; ω pred Represents the weight coefficient for predicting the mask image.

[0117] S209 : Segment the ice-covered area in the target image according to the mask image of the ice-covered area.

[0118] It should be noted that this step is similar to the aforementioned step S104 and will not be repeated here.

[0119] S210: Determine the area ratio of the ice-covered area to the target image.

[0120] For example, the area ratio represents the ratio of the ice-covered area in the mask image to the total area of ​​the target image, reflecting the proportion of the ice-covered area in the image. The electronic device can count the number of pixels N with a pixel value of 1 in the mask image. s , and the total number of pixels N in the target image, according to Calculate the area ratio r.

[0121] S211. Determine the risk value at the target collection time according to the area ratio.

[0122] For example, the risk value represents the degree of icing risk for the transmission line at the target acquisition time, with higher values ​​indicating greater risk. The electronic device can determine the risk value based on the area ratio, based on a correspondence between the area ratio, risk interval, and risk value. For example, if the area ratio is less than 0.05, indicating low risk, and the corresponding risk value is 1, the risk value at the target acquisition time can be determined.

[0123] S212. Generate a test report based on the risk value at each collection moment.

[0124] For example, the electronic device can extract the risk value, target image, ice-covered area, drone posture information, etc. at each collection moment and generate a detection report to clearly display the inspection results.

[0125] Optionally, the ice covered areas can be highlighted in the target image to obtain a segmented coverage image. Figure 4 As shown, the target image can display the transmission lines shown in blue and the ice-covered areas shown in green on the transmission lines, clearly showing the ice-covered areas on each transmission line.

[0126] Optionally, the collection time corresponds to the posture information of the UAV, so the location information of the ice-covered area can be determined, and based on the risk value at each collection time, the key areas of concern and recommended measures can be determined.

[0127] In one example, the test report may be:

[0128] Mission number: 2025-0327-001;

[0129] Flight waypoints: Longitude 111.111, Latitude 22.222, Altitude 500; Longitude 111.115, Latitude 22.226, Altitude 505;

[0130] Collection time: 2025-03-27 10:30:00;

[0131] Processing result: segmented overlay images image1 and image2;

[0132] Textual statistics: A total of 200 images were collected during this inspection;

[0133] Metadata: flight speed 5m / s, line number line-135; shooting device camera 1;

[0134] Risk Assessment: Risk value 1, subsequent impact may lead to increased wire tension;

[0135] Early Warning Report: Focus on areas Tower 21-Tower 25; Recommended measures: Immediate on-site inspection.

[0136] The present invention provides a method for segmenting ice-covered areas of power transmission lines. The electronic device performs denoising, contrast enhancement, and dedistortion processing on the original image captured by the drone to obtain a target image. The electronic device directly extracts an initial mask image of the ice-covered area based on a large visual model. The initial mask image is then combined with a Kalman filter algorithm to determine whether the initial mask image is normal. If the initial mask image is normal, the initial mask image can be directly used as the mask image of the ice-covered area. If the initial mask image is abnormal, the initial mask image can be corrected to obtain a mask image of the ice-covered area. The ice-covered area can then be segmented based on the mask image. The ice-covered area is then determined by combining the area ratio of the ice-covered area to the target image to determine the ice-covered risk value and generate a detection report. This method can eliminate defects in the target image, provide a clear and accurate data foundation for subsequent segmentation processing, and accurately segment the ice-covered area based on the pre-trained large visual model, eliminating the need for additional model training and reducing training costs. At the same time, during the segmentation process, the Kalman filter algorithm utilizes the spatiotemporal continuity of the mask features to detect anomalies in the segmentation results in real time, improving the reliability of the results.

[0137] Figure 5 The schematic diagram of the structure of the device for dividing the ice-covered area of ​​the transmission line provided in this application is as follows: Figure 5 As shown, the device 300 for dividing the ice-covered area of ​​a transmission line provided in this embodiment includes:

[0138] The acquisition module 301 is used to acquire a target image of the UAV at the target acquisition moment; wherein the target image includes the power transmission line;

[0139] An extraction module 302 is configured to extract an initial mask image of the ice-covered area in the target image based on a preset visual macro model;

[0140] The segmentation module 303 is used to determine the mask image of the ice-covered area in the target image based on the initial mask image and a preset Kalman filter algorithm; and segment the ice-covered area in the target image based on the mask image of the ice-covered area.

[0141] In a possible implementation, the segmentation module 303 is specifically configured to:

[0142] Determining whether the initial mask image is normal according to the initial mask image and a preset Kalman filter algorithm;

[0143] If yes, performing corrosion and expansion processing on the initial mask image to obtain the mask image;

[0144] If not, the initial mask image is corrected to obtain the mask image.

[0145] In a possible implementation, the segmentation module 303 is specifically configured to:

[0146] Extracting mask features according to the initial mask image; wherein the mask features include centroid coordinates, circumscribed rectangle parameters, and area of ​​the initial mask image;

[0147] Determining a current observation state and a predicted state according to the mask feature and the Kalman filter algorithm;

[0148] Whether the initial mask image is normal is determined according to the Mahalanobis distance between the current observed state and the predicted state.

[0149] In a possible implementation, the segmentation module 303 is specifically configured to:

[0150] Obtaining a historical mask image before the target acquisition moment;

[0151] Determining a predicted mask image according to the predicted state;

[0152] The historical mask image, the initial mask image, and the predicted mask image are fused to obtain the mask image.

[0153] In a possible implementation, the extraction module 302 is specifically configured to:

[0154] extracting image features from the target image based on an image encoder of the visual macro model;

[0155] Obtaining prompt information, and extracting prompt features from the prompt information based on a prompt encoder of the visual macro model;

[0156] A mask decoder based on the visual large model determines the initial mask image according to the image features and the hint features.

[0157] In a possible implementation, the device further includes a generating module, configured to:

[0158] determining an area ratio of the ice-covered area to the target image;

[0159] determining a risk value at the target acquisition moment according to the area ratio;

[0160] A test report is generated based on the risk value at each collection moment.

[0161] In a possible implementation, the acquisition module 301 is specifically configured to:

[0162] Obtain the original image of the UAV at the target acquisition moment;

[0163] The original image is subjected to denoising, contrast enhancement and dedistortion processing in sequence to obtain the target image.

[0164] The device for dividing the ice-covered area of ​​a transmission line provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.

[0165] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 6 As shown, the electronic device 400 provided in this embodiment includes: at least one processor 401 and a memory 402. Optionally, the electronic device 400 further includes a communication component 403. The processor 401, the memory 402 and the communication component 403 are connected via a bus.

[0166] In a specific implementation process, at least one processor 401 executes the computer-executable instructions stored in the memory 402, so that the at least one processor 401 performs the above method.

[0167] The specific implementation process of the processor 401 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0168] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0169] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0170] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified into address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0171] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0172] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0173] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0174] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0175] The division of units is merely a logical functional division; actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, any direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units, either through an interface, electrical, mechanical, or other means.

[0176] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0177] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0178] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0179] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0180] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.

Claims

1. A method for segmenting ice-covered areas of a transmission line, characterized in that: include: Acquire a target image of the UAV at the target acquisition moment; wherein the target image includes a power transmission line; Extracting an initial mask image of the ice-covered area in the target image based on a preset visual macro model; According to the initial mask image and a preset Kalman filter algorithm, a mask image of the ice-covered area in the target image is determined; and according to the mask image of the ice-covered area, the ice-covered area in the target image is segmented.

2. The method according to claim 1, characterized in that The determining, based on the initial mask image and a preset Kalman filter algorithm, a mask image of the ice-covered area in the target image includes: Determining whether the initial mask image is normal according to the initial mask image and a preset Kalman filter algorithm; If yes, performing corrosion and expansion processing on the initial mask image to obtain the mask image; If not, the initial mask image is corrected to obtain the mask image.

3. The method according to claim 2, characterized in that The determining whether the initial mask image is normal according to the initial mask image and a preset Kalman filter algorithm includes: Extracting mask features according to the initial mask image; wherein the mask features include centroid coordinates, circumscribed rectangle parameters, and area of ​​the initial mask image; Determining a current observation state and a predicted state according to the mask feature and the Kalman filter algorithm; Whether the initial mask image is normal is determined according to the Mahalanobis distance between the current observed state and the predicted state.

4. The method according to claim 3, characterized in that The correcting the initial mask image to obtain the mask image includes: Obtaining a historical mask image before the target acquisition moment; Determining a predicted mask image according to the predicted state; The historical mask image, the initial mask image, and the predicted mask image are fused to obtain the mask image.

5. The method according to claim 1, wherein The step of extracting an initial mask image of the ice-covered area in the target image based on a preset visual macro model includes: extracting image features from the target image based on an image encoder of the visual macro model; Obtaining prompt information, and extracting prompt features from the prompt information based on a prompt encoder of the visual macro model; A mask decoder based on the visual large model determines the initial mask image according to the image features and the hint features.

6. The method according to claim 1, characterized in that The method further comprises: determining an area ratio of the ice-covered area to the target image; determining a risk value at the target acquisition moment according to the area ratio; A test report is generated based on the risk value at each collection moment.

7. The method according to any one of claims 1 to 6, characterized in that The step of obtaining the target image of the UAV at the target acquisition moment includes: Obtain the original image of the UAV at the target acquisition moment; The original image is subjected to denoising, contrast enhancement and dedistortion processing in sequence to obtain the target image.

8. A device for dividing ice-covered areas of power transmission lines, characterized in that: include: An acquisition module, configured to acquire a target image of the UAV at the target acquisition moment; wherein the target image includes a power transmission line; An extraction module, configured to extract an initial mask image of the ice-covered area in the target image based on a preset visual macro model; A segmentation module is used to determine the mask image of the ice-covered area in the target image based on the initial mask image and a preset Kalman filter algorithm; and to segment the ice-covered area in the target image based on the mask image of the ice-covered area.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images