Pull wire state detection method and device based on image analysis, electronic equipment and storage medium

Abstract

The invention discloses a stay wire state detection method and device based on image analysis, electronic equipment and a storage medium, and belongs to the technical field of distribution line detection, and the method comprises the steps: obtaining a multi-view target image of a stay wire, carrying out the foreground segmentation, and generating a foreground region image; inputting each foreground region image into a semantic segmentation model to generate a segmentation mask of a stay wire; removing non-stay wire areas in the foreground area images according to the segmentation masks, and generating key area images only containing stay wires; extracting a stay wire pixel boundary point set from each key area image, and calculating the radian value of each section of the stay wire to form a radian value set; counting the proportion of the radian value of each section of the stay wire in the radian value set exceeding the radian threshold range, and if the proportion is greater than a preset proportion threshold, judging that the to-be-detected stay wire is in a loose state; otherwise, judging that the to-be-detected stay wire is in a normal state. Therefore, the problem of false detection or missing detection caused by the fact that the stay wire is thin and long and is easy to shield in the prior art can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of power distribution line detection, and in particular to a method, device, electronic equipment and storage medium for detecting a wire status based on image analysis. Background Art

[0002] The guy wires in distribution lines are critical structural components that support and stabilize power towers. Their condition directly impacts the safety and stability of the power system. Long-term use or external environmental influences (such as wind loads, temperature and humidity fluctuations, etc.) can cause the guy wires to loosen, affecting the force balance of the power towers and even causing serious accidents such as equipment tilting or collapse. Therefore, monitoring the condition of distribution line guy wires is crucial. Traditional manual inspection methods are inefficient and subject to significant subjective judgment. However, automated inspection methods based on computer vision can significantly improve inspection efficiency and accuracy, and have therefore become a research hotspot.

[0003] Currently, wire slack detection methods based on target detection algorithms (such as YOLO) are widely used. These methods can predict the type and location of wires in distribution line images and determine whether they are slack. However, these methods still face many challenges in practical applications. Specifically, the complex distribution line environment and the presence of numerous background interference factors (such as trees, buildings, and power equipment) can easily affect detection accuracy. Furthermore, wires are small, slender targets that are easily obscured or misidentified in complex scenes, leading to missed or false detections. Summary of the Invention

[0004] The embodiments of the present invention provide a method, device, electronic device and storage medium for detecting the state of a wire based on image analysis, which can solve the problems of false detection or missed detection in the prior art caused by the wire being slender and easily blocked.

[0005] An embodiment of the present invention provides a method for detecting a wire state based on image analysis, comprising:

[0006] Acquire a plurality of target images of the to-be-detected wire at multiple viewing angles;

[0007] Perform foreground segmentation processing on each target image to generate a corresponding foreground area image;

[0008] Inputting each foreground area image into a preset semantic segmentation model in sequence, so that the semantic segmentation model generates a segmentation mask of the line area in each foreground area image according to each foreground area image;

[0009] According to the segmentation mask of the pull-line area, the non-pull-line area in each foreground area image is removed to generate a corresponding key area image containing only the pull-line;

[0010] For each key area image, extract the line pixel boundary point set of the current key area image; calculate the radian value of each line segment corresponding to the current key area image based on the line pixel boundary point set, and add the radian value of each line segment to the radian value set;

[0011] The proportion of the curvature values ​​of each section of the wire in the curvature value concentration that exceeds the preset curvature threshold range is calculated. If the proportion is greater than the preset proportion threshold, it is determined that the wire to be detected is in a relaxed state; otherwise, it is determined that the wire to be detected is in a normal state.

[0012] Furthermore, performing foreground segmentation processing on each target image to generate a corresponding foreground area image includes:

[0013] Analyze the pixel distribution and depth information of each target image and generate a segmentation mask for the corresponding background area;

[0014] The background area of ​​each target image is removed according to the segmentation mask of the background area, and a foreground area image corresponding to each target image is generated.

[0015] Furthermore, the training of the semantic segmentation model includes:

[0016] Acquire a pull-line multi-view image dataset; wherein the pull-line multi-view image dataset includes a plurality of pull-line images and corresponding segmentation mask labels;

[0017] Randomly dividing the pull-wire multi-view image dataset into a number of batches of training samples according to a preset number;

[0018] Each batch of training samples is input into the semantic segmentation model in sequence, and the semantic segmentation model is trained until a preset number of training times is reached; wherein, when the semantic segmentation model receives each batch of training samples, it outputs the predicted segmentation mask corresponding to the training sample; according to the predicted segmentation mask and the corresponding segmentation mask label, the loss function value is calculated through the loss function; and the semantic segmentation model is updated using the optimizer according to the loss function value.

[0019] Furthermore, before inputting the training samples of each batch into the semantic segmentation model in sequence and training the semantic segmentation model until a preset number of training times is reached, the method further includes:

[0020] For each batch of training samples, data augmentation and normalization are performed to generate updated training samples; wherein the data augmentation processing includes any one or a combination of random flipping, random scaling, random translation, random rotation, random adjustment of image brightness, random addition of Gaussian noise and random cropping.

[0021] Furthermore, for each key area image, extracting the line pixel boundary point set of the current key area image includes:

[0022] For each key area image, a set of pixel points of the pull-line area is extracted from the current key area image; based on a preset edge detection algorithm, the edge contour of the pull-line area is extracted from the pixel point set; and the pixel points on the edge contour are used as the pull-line pixel boundary point set of the current key area image.

[0023] Furthermore, the step of calculating the radian value of each segment of the pull line corresponding to the current key area image based on the pull line pixel boundary point set and adding the radian value of each segment of the pull line to the radian value set includes:

[0024] Based on the cubic spline curve fitting algorithm, the shape of the pull line is fitted according to the pull line pixel boundary point set to generate a pull line fitting curve;

[0025] Calculating the curvature of equally spaced points in the line fitting curve to generate a plurality of curvature values;

[0026] According to the several curvature values, the curvature is numerically integrated along the arc length to obtain the bending arc value of each section of the wire;

[0027] Adding the bending radian value of each section of the cable to the radian value set;

[0028] Here, the curvature of equally spaced points is calculated according to the following formula:

[0029]

[0030] Where K(x) is the curvature value of the line fitting curve at point x; s″ i (x) is the second-order derivative of the cubic spline curve at point x; s′ i (x) is the first derivative of the cubic spline curve at point x.

[0031] Furthermore, when it is determined that the pull wire to be detected is in a relaxed state, the method further includes:

[0032] Determining the position of the wire section whose arc value exceeds the arc threshold range, and generating position information of the wire slack section;

[0033] A visual inspection report including slack position markings is generated based on the original target image and the position information of the slack wire section.

[0034] Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0035] An embodiment of the present invention provides a wire pulling state detection device based on image analysis, comprising: a target image acquisition module, a foreground area image generation module, a wire pulling area segmentation mask generation module, a key area image generation module, a radian value calculation module and a wire pulling state detection module.

[0036] The target image acquisition module is used to acquire a plurality of target images of the to-be-detected wire at multiple viewing angles;

[0037] The foreground area image generation module is used to perform foreground segmentation processing on each target image to generate a corresponding foreground area image;

[0038] The pull-line region segmentation mask generation module is used to sequentially input each foreground region image into a preset semantic segmentation model, so that the semantic segmentation model generates a segmentation mask of the pull-line region in each foreground region image based on each foreground region image;

[0039] The key area image generation module is used to remove the non-pull-line area in each foreground area image according to the segmentation mask of the pull-line area, and generate a corresponding key area image containing only the pull-line;

[0040] The radian value calculation module is used to extract the line pixel boundary point set of the current key area image for each key area image; calculate the radian value of each line segment corresponding to the current key area image based on the line pixel boundary point set, and add the radian of each line segment to the radian value set;

[0041] The wire state detection module is used to count the proportion of the curvature values ​​of each section of the wire in the curvature value concentration that exceeds the preset curvature threshold range. If the proportion is greater than the preset proportion threshold, it is determined that the wire to be detected is in a relaxed state; otherwise, it is determined that the wire to be detected is in a normal state.

[0042] Based on the above method embodiment, the present invention provides a corresponding electronic device embodiment.

[0043] One embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the wire pulling state detection method based on image analysis as described in any one of the above method embodiments.

[0044] Based on the above method embodiment, the present invention provides a corresponding storage medium embodiment.

[0045] An embodiment of the present invention provides a storage medium having a computer program stored thereon, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the wire pulling state detection method based on image analysis as described in any one of the above method embodiments.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The embodiments of the present invention provide a method, device, electronic device and storage medium for detecting the state of a wire rope based on image analysis. The method obtains a target image of the wire rope to be detected through multiple perspectives, and performs foreground segmentation on each image to obtain a foreground area image. Subsequently, the foreground area image is processed using a preset semantic segmentation model to generate a segmentation mask for the wire rope area, and the non-wire rope area is removed accordingly to extract a key area image containing only the wire rope. Next, the pixel boundary point set of the wire rope is extracted from the key area image, and the radian value of each segment of the wire rope is calculated based on the boundary point set and summarized into a radian value set. Finally, the proportion of the radian value set that exceeds the preset threshold range is counted. If the proportion exceeds the preset proportion threshold, it is determined that the wire rope is in a relaxed state, otherwise it is determined to be in a normal state.

[0048] This method acquires target images from multiple perspectives and uses foreground segmentation and semantic segmentation models to generate segmentation masks for the pull-wire region, effectively removing complex background interference and improving detection accuracy. Furthermore, by extracting the pull-wire pixel boundary points and calculating the arc value, it can accurately identify the morphological changes of the pull-wire, avoiding false or missed detections caused by the slender and easily obscured pull-wire, thereby improving the stability and reliability of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of a method for detecting a wire pulling state based on image analysis provided by one embodiment of the present invention.

[0050] Figure 2 It is a structural schematic diagram of a wire pulling state detection device based on image analysis provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0052] like Figure 1As shown, in order to solve the problem of false detection or missed detection caused by the slender and easily obstructed pull wire in the prior art, an embodiment of the present invention provides a pull wire status detection method based on image analysis, which includes at least the following steps:

[0053] Step S1: Acquire a plurality of target images of the to-be-detected wire under multiple viewing angles.

[0054] Specifically, high-definition cameras and drones are used to capture images of the distribution line area (including the cables to be inspected) at different times, seasons, weather conditions, voltage levels, terrain, and from multiple perspectives to ensure representative data. Multi-angle imaging effectively reduces detection errors caused by occlusion or perspective distortion from a single perspective, improving the accuracy of cable status analysis. Furthermore, multi-angle images provide richer depth and structural information, providing more reliable foundational data for subsequent foreground segmentation and curvature calculations.

[0055] Step S2: Perform foreground segmentation processing on each target image to generate a corresponding foreground area image.

[0056] In a preferred embodiment, performing foreground segmentation processing on each target image to generate a corresponding foreground area image includes:

[0057] Analyze the pixel distribution and depth information of each target image and generate a segmentation mask for the corresponding background area;

[0058] The background area of ​​each target image is removed according to the segmentation mask of the background area, and a foreground area image corresponding to each target image is generated.

[0059] Specifically, foreground segmentation processing is performed on each target image to extract the main area where the wire is located and remove the interference of the complex background. Specifically, the foreground segmentation algorithm is used to analyze the pixel distribution, color characteristics and depth information of the image to generate a corresponding foreground area image. Through this processing step, the wire target can be effectively highlighted, and the influence of irrelevant elements such as trees, buildings, and power equipment in the background can be reduced, providing clearer and higher-quality input data for subsequent semantic segmentation and wire status analysis. In one embodiment, the RMBG-2.0 depth estimation algorithm is used to estimate the depth information, extract the main target and remove background interference. The RMBG algorithm is an image background removal technology based on deep learning. Its core purpose is to effectively separate the foreground object from the background in the image. When processing the image, it analyzes each pixel in the image to determine whether the pixel belongs to the foreground or the background. In RMBG-2.0, a combination of channel attention and spatial attention is used to better extract the small and complex wires in the actual distribution lines.

[0060] It should be noted that the RMBG-2.0 algorithm used here is a pre-trained model. During the pre-training phase, the RMBG-2.0 model was trained using a large-scale, diverse dataset. During training, the loss function weights were adjusted to balance the impact of different types of losses on the model, ensuring stable and efficient model convergence. In practice, imagery collected from distribution line areas can also be used to fine-tune the RMBG-2.0 algorithm to achieve better results in specific application areas.

[0061] Step S3: input each foreground area image into a preset semantic segmentation model in sequence, so that the semantic segmentation model generates a segmentation mask of the pull-line area in each foreground area image according to each foreground area image.

[0062] Specifically, each foreground region image is sequentially input into a preset semantic segmentation model, which then classifies the image pixel by pixel and generates a segmentation mask for the pull-wire region in each foreground region image. This semantic segmentation model is based on a deep learning algorithm, specifically an encoder-decoder structure. The encoder uses an improved YOLO backbone network to fully utilize its powerful feature extraction capabilities; the decoder uses upsampling modules and deconvolution layers to gradually restore the feature map resolution and achieve accurate segmentation of the pull-wires. At the same time, attention mechanism modules, such as the spatial attention module (SAM) and channel attention module (CAM), are introduced to allow the model to pay more attention to the pull-wire region and improve segmentation accuracy.

[0063] Specifically, after receiving an input image, the model first performs multi-scale feature analysis on the image through a feature extraction network to enhance the detection of slender objects. Subsequently, it combines deep semantic information with shallow detail information to improve the accuracy of the edge recognition of the cable, thereby generating a high-quality segmentation mask. This segmentation mask accurately labels the cable area, laying the foundation for subsequent cable boundary point extraction, morphological analysis, and curvature calculation, ensuring the reliability and stability of cable state detection.

[0064] In a preferred embodiment, the training of the semantic segmentation model includes:

[0065] Acquire a pull-line multi-view image dataset; wherein the pull-line multi-view image dataset includes a plurality of pull-line images and corresponding segmentation mask labels;

[0066] Randomly dividing the pull-wire multi-view image dataset into a number of batches of training samples according to a preset number;

[0067] Each batch of training samples is input into the semantic segmentation model in sequence, and the semantic segmentation model is trained until a preset number of training times is reached; wherein, when the semantic segmentation model receives each batch of training samples, it outputs the predicted segmentation mask corresponding to the training sample; according to the predicted segmentation mask and the corresponding segmentation mask label, the loss function value is calculated through the loss function; and the semantic segmentation model is updated using the optimizer according to the loss function value.

[0068] Specifically, a multi-view image dataset of string lines was obtained. This dataset includes multiple string line images captured at different angles, lighting conditions, and background environments, along with corresponding high-quality segmentation mask labels. To enhance the model's generalization capabilities, the dataset includes string line samples in both normal and relaxed states, and incorporates as many scene factors as possible that may affect detection accuracy, such as occlusion, background interference, and lighting changes.

[0069] The pull-wire multi-view image dataset is randomly divided into several batches of training samples according to a preset number, and optimization strategies such as batch gradient descent are adopted in the model training process to improve training efficiency and stability. The training samples of each batch are sequentially input into the semantic segmentation model, and the semantic segmentation model is trained end-to-end until the preset number of training times is reached or the loss function converges. During the training process, after the semantic segmentation model receives a batch of training samples each time, it uses forward propagation to calculate the corresponding predicted segmentation mask, and compares it with the real segmentation mask label, and calculates the error value through loss functions such as cross entropy loss, Dice loss or IoU loss. Subsequently, an optimizer (such as Adam or SGD) is used to perform gradient updates on the parameters of the semantic segmentation model according to the loss function value, thereby continuously optimizing the feature extraction capability of the model, improving its segmentation accuracy of the pull-wire area, and ultimately achieving accurate recognition of the pull-wire state.

[0070] In a preferred embodiment, before sequentially inputting the training samples of each batch into the semantic segmentation model and training the semantic segmentation model until a preset number of training times is reached, the method further includes:

[0071] For each batch of training samples, data augmentation and normalization are performed to generate updated training samples; wherein the data augmentation processing includes any one or a combination of random flipping, random scaling, random translation, random rotation, random adjustment of image brightness, random addition of Gaussian noise and random cropping.

[0072] Specifically, for each batch of training samples, before inputting into the semantic segmentation model, data augmentation and normalization are first performed to improve the robustness and generalization ability of the model, thereby improving the segmentation accuracy of the pull-down area in complex scenes. Among them, data augmentation processing includes but is not limited to the following operations: random flipping (horizontally or vertically), random scaling (adjusting the image size within a certain ratio range), random translation (translating the image within a set pixel range), random rotation (rotating around the center point within a set angle range), random adjustment of image brightness (simulating different lighting conditions), random addition of Gaussian noise (enhancing anti-interference ability) and random cropping (retaining different parts of the target area). They can be used alone or in any combination to increase data diversity and reduce the model's dependence on fixed patterns.

[0073] In addition, training samples are normalized, mapping the image pixel values ​​to a fixed range (e.g., 0-1 or -1 to 1). This reduces the numerical differences between images and improves the stability and convergence of model training. After data augmentation and normalization, the training samples are fed into the semantic segmentation model as final input data for training, improving the model's ability to accurately identify pull-wire areas in different scenarios.

[0074] Optionally, you can fully evaluate the trained model using a test set on the dataset. By calculating metrics such as segmentation accuracy and recall, you can assess the model's ability to identify and segment wires in different states. You can also visualize the segmentation results to intuitively analyze the model's performance.

[0075] Step S4: removing the non-pull-line areas in each foreground area image according to the segmentation mask of the pull-line area, and generating a corresponding key area image containing only the pull-line.

[0076] Specifically, each foreground image is processed based on the segmentation mask of the pull-wire area, precisely removing non-pull-wire areas and retaining only the target pull-wire area, thereby generating a corresponding key area image containing only the pull-wire area. Specifically, the segmentation mask is first used to filter the pixels in the original foreground image, retaining only those pixels marked as pull-wire areas by the mask, while pixels in other non-target areas are set to background or transparent to prevent complex background interference in subsequent analysis.

[0077] On this basis, morphological processing methods (such as opening and closing operations) can be further applied to optimize the edges of the tension wire area, eliminate possible noise or broken parts, and ensure the integrity of the tension wire area. Furthermore, to improve the efficiency of subsequent calculations, the key region image containing only the tension wire can be appropriately cropped, retaining only the minimum bounding box area where the tension wire is located, thereby reducing computational redundancy and improving processing speed. The resulting key region image will serve as the basis for subsequent tension wire status analysis, helping to improve the accuracy and reliability of slack detection.

[0078] Step S5: for each key area image, extract the line pixel boundary point set of the current key area image; calculate the radian value of each line segment corresponding to the current key area image based on the line pixel boundary point set, and add the radian of each line segment to the radian value set.

[0079] In a preferred embodiment, the step of extracting the line pixel boundary point set of the current key area image for each key area image includes:

[0080] For each key area image, a set of pixel points of the pull-line area is extracted from the current key area image; based on a preset edge detection algorithm, the edge contour of the pull-line area is extracted from the pixel point set; and the pixel points on the edge contour are used as the pull-line pixel boundary point set of the current key area image.

[0081] Specifically, for each key area image, a set of pixel points of the pull-wire area is first extracted from the current key area image. This set contains all pixel point information that may belong to the pull-wire structure. Next, based on a preset edge detection algorithm (such as Canny, Sobel or Laplace operator, etc.), edge feature extraction is performed on the pixel point set to obtain the edge contour of the pull-wire area. Finally, the pixel points on the extracted edge contour are determined as the pull-wire pixel boundary point set of the current key area image, thereby laying the foundation for subsequent pull-wire recognition, feature analysis and target extraction.

[0082] In a preferred embodiment, the step of calculating the radian value of each segment of the pull line corresponding to the current key area image based on the pull line pixel boundary point set and adding the radian value of each segment of the pull line to the radian value set includes:

[0083] Based on the cubic spline curve fitting algorithm, the shape of the pull line is fitted according to the pull line pixel boundary point set to generate a pull line fitting curve;

[0084] Calculating the curvature of equally spaced points in the line fitting curve to generate a plurality of curvature values;

[0085] According to the several curvature values, the curvature is numerically integrated along the arc length to obtain the bending arc value of each section of the wire;

[0086] Adding the bending radian value of each section of the cable to the radian value set;

[0087] Here, the curvature of equally spaced points is calculated according to the following formula:

[0088]

[0089] Where K(x) is the curvature value of the line fitting curve at point x; s″ i (x) is the second-order derivative of the cubic spline curve at point x; s′ i (x) is the first derivative of the cubic spline curve at point x. s′ i (x) reflects the change in the slope of the curve; s″ i (x) is the second-order derivative, reflecting the curvature of the curve. By calculating the curvature at a series of equally spaced points on the curve and integrating the curvature, we can determine the arc value at different positions of the wire, providing a quantitative basis for determining whether the wire is slack.

[0090] Specifically, a cubic spline curve fitting algorithm is used to accurately describe the shape of the pull-line. Based on this algorithm, a set of pixel boundary points in the pull-line region image is first extracted and sorted according to the pull-line topology to form an ordered sequence of key points. Subsequently, a cubic spline curve is constructed between adjacent key points, ensuring that it meets the requirements of continuity, differentiability, and smoothness. This allows for high-precision fitting of the pull-line shape, generating a pull-line fitting curve.

[0091] During the fitting process, natural boundary conditions (i.e., the second-order derivative at both ends of the curve is zero) or chord length parameterization methods can be used to improve the stability and authenticity of the curve. In addition, the least squares method can be combined to optimize the fitting results to further reduce the influence of noise and ensure that the fitting curve can accurately depict the morphological characteristics of the wire. The final fitting curve can be used for subsequent radian calculations to provide accurate data support for the determination of the wire relaxation state. Suppose the key point sequence is {x0, x1, ..., x n}, in each interval [x i , x i+1 ], the cubic spline curve S i (x) is a cubic polynomial of the form:

[0092] S i (x) = a i +b i (xx i )+c i (xx i ) 2 +d i (xx i ) 3

[0093] Among them, a i 、b i 、c i and d i The coefficient to be determined needs to be solved based on the coordinate information of the boundary point set of the line pixel and the boundary conditions to ensure that each segment of the spline curve remains continuous and smooth at the node. i (x) is in the interval [x i , x i+1 ] is used to fit the shape of the line curve within the interval. x is the function variable to be calculated, that is, the horizontal coordinate of the current calculation of the fitting curve value. i It is the horizontal coordinate of the starting key point (known data point) of the current interval. i For the interval [x i , x i+1 ], indicating that the line is at the starting point x i The vertical coordinate value at b i For the interval [x i , x i+1 ]The coefficient of the first-order term in c i For the interval [x i , x i+1 ] the coefficient of the second-order derivative term; d i For the interval [x i , x i+1 ] the coefficient of the third-order derivative term within ;

[0094] Step S6, counting the proportion of the curvature values ​​of each section of the curvature value concentration wire that exceeds the preset curvature threshold range, if the proportion is greater than the preset ratio threshold, it is determined that the wire to be detected is in a relaxed state; otherwise, it is determined that the wire to be detected is in a normal state.

[0095] Specifically, the radian values ​​of each section of the cable are collected and the proportion of values ​​exceeding a preset radian threshold range is calculated; wherein the radian threshold range is used to characterize the radian variation range of the cable under normal stress. If the statistical results show that the proportion of radian values ​​exceeding the radian threshold range is greater than a preset ratio threshold, the cable to be detected is determined to be in a relaxed state; otherwise, the cable to be detected is determined to be in a normal state. Furthermore, in order to improve the reliability of the judgment, a comprehensive analysis can be performed based on the detection results under multiple perspectives. If the relaxation judgment conditions are met under multiple perspectives, the certainty of the relaxed state is further enhanced, thereby improving the detection accuracy and reducing the false positive rate. In order to accurately determine whether the cable is relaxed, it is necessary to determine the reasonable range of the normal cable radian value. First, a large number of cable samples in normal state are collected from distribution lines in different regions and under different environmental conditions. The number of samples should be sufficient (e.g., not less than 1000) to ensure the reliability of the statistical results. The radian value of each sample is accurately measured and recorded, and then these data are analyzed using statistical methods. Calculate the mean μ and standard deviation σ of the sample. The mean reflects the average level of normal wire curvature, and the standard deviation reflects the degree of data dispersion. Based on engineering experience and actual application requirements, the k value is usually between 2 and 3, and μ±kσ is used as the normal curvature range.

[0096] In a preferred embodiment, when it is determined that the pull wire to be detected is in a relaxed state, the method further includes:

[0097] Determining the position of the wire section whose arc value exceeds the arc threshold range, and generating position information of the wire slack section;

[0098] A visual inspection report including slack position markings is generated based on the original target image and the position information of the slack wire section.

[0099] Specifically, the position of the wire section whose arc value exceeds the arc threshold range is determined, and the starting point, end point and length of the section are calibrated to generate the position information of the wire slack section to accurately reflect the specific position and degree of wire slack. Furthermore, the position information of the wire slack section is matched with the original target image, and the slack position annotation is superimposed on the original target image to intuitively display the wire slack situation. Finally, a visual inspection report containing the slack position annotation is generated. The report content may include information such as the inspection time, wire number, specific location of the slack section, and degree of slack, which facilitates operation and maintenance personnel to quickly obtain key inspection results and serves as an important basis for subsequent maintenance decisions.

[0100] Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0101] like Figure 2As shown, an embodiment of the present invention provides a wire pulling state detection device based on image analysis, comprising: a target image acquisition module, a foreground area image generation module, a wire pulling area segmentation mask generation module, a key area image generation module, a radian value calculation module and a wire pulling state detection module;

[0102] The target image acquisition module is used to acquire a plurality of target images of the to-be-detected wire at multiple viewing angles;

[0103] The foreground area image generation module is used to perform foreground segmentation processing on each target image to generate a corresponding foreground area image;

[0104] The pull-line region segmentation mask generation module is used to sequentially input each foreground region image into a preset semantic segmentation model, so that the semantic segmentation model generates a segmentation mask of the pull-line region in each foreground region image based on each foreground region image;

[0105] The key area image generation module is used to remove the non-pull-line area in each foreground area image according to the segmentation mask of the pull-line area, and generate a corresponding key area image containing only the pull-line;

[0106] The radian value calculation module is used to extract the line pixel boundary point set of the current key area image for each key area image; calculate the radian value of each line segment corresponding to the current key area image based on the line pixel boundary point set, and add the radian of each line segment to the radian value set;

[0107] The wire state detection module is used to count the proportion of the curvature values ​​of each section of the wire in the curvature value concentration that exceeds the preset curvature threshold range. If the proportion is greater than the preset proportion threshold, it is determined that the wire to be detected is in a relaxed state; otherwise, it is determined that the wire to be detected is in a normal state.

[0108] It should be noted that the embodiment of the device described above corresponds to the above-mentioned embodiment of the present invention, and it can realize any one of the above-mentioned methods for detecting the state of the wire based on image analysis of the present invention. In addition, the embodiment of the above-mentioned device is merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the embodiment of the device provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0109] Based on the above method embodiment of the present invention, a corresponding electronic device embodiment is provided.

[0110] One embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the wire pulling state detection method based on image analysis described in any one of the present invention is implemented, or when the processor executes the computer program, the functions of the modules in the above-mentioned device embodiments are implemented.

[0111] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0112] The terminal device may be a computing device such as a desktop computer, a notebook computer, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0113] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device using various interfaces and lines.

[0114] The memory can be used to store the computer programs and / or modules, and the processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created based on the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0115] Based on the above method embodiment, the present invention provides a corresponding storage medium embodiment;

[0116] Another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute any one of the above-mentioned wire pulling state detection methods based on image analysis of the present invention.

[0117] The above-mentioned storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0118] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described may be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.

[0119] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting a wire pulling state based on image analysis, characterized in that: include: Acquire a plurality of target images of the to-be-detected wire at multiple viewing angles; Perform foreground segmentation processing on each target image to generate a corresponding foreground area image; Inputting each foreground area image into a preset semantic segmentation model in sequence, so that the semantic segmentation model generates a segmentation mask of the line area in each foreground area image according to each foreground area image; According to the segmentation mask of the pull-line area, the non-pull-line area in each foreground area image is removed to generate a corresponding key area image containing only the pull-line; For each key area image, extract the line pixel boundary point set of the current key area image; calculate the radian value of each line segment corresponding to the current key area image based on the line pixel boundary point set, and add the radian value of each line segment to the radian value set; Counting the percentage of the curvature values ​​of each section of the wire in the curvature value concentration that exceeds a preset curvature threshold range, if the percentage is greater than a preset ratio threshold, it is determined that the wire to be tested is in a slack state; Otherwise, it is determined that the pull wire to be detected is in a normal state.

2. The method for detecting the wire state based on image analysis according to claim 1, wherein: The performing of foreground segmentation processing on each target image to generate a corresponding foreground area image includes: Analyze the pixel distribution and depth information of each target image and generate a segmentation mask for the corresponding background area; The background area of ​​each target image is removed according to the segmentation mask of the background area, and a foreground area image corresponding to each target image is generated.

3. The method for detecting the wire pulling state based on image analysis according to claim 2, wherein: The training of the semantic segmentation model includes: Acquire a pull-line multi-view image dataset; wherein the pull-line multi-view image dataset includes a plurality of pull-line images and corresponding segmentation mask labels; Randomly dividing the pull-wire multi-view image dataset into a number of batches of training samples according to a preset number; Each batch of training samples is input into the semantic segmentation model in sequence, and the semantic segmentation model is trained until a preset number of training times is reached; wherein, when the semantic segmentation model receives each batch of training samples, it outputs the predicted segmentation mask corresponding to the training sample; according to the predicted segmentation mask and the corresponding segmentation mask label, the loss function value is calculated through the loss function; and the semantic segmentation model is updated using the optimizer according to the loss function value.

4. The method for detecting the wire state based on image analysis according to claim 3, wherein: Inputting the training samples of each batch into the semantic segmentation model in sequence and training the semantic segmentation model until a preset number of training times is reached further includes: For each batch of training samples, data augmentation and normalization are performed to generate updated training samples; wherein the data augmentation processing includes any one or a combination of random flipping, random scaling, random translation, random rotation, random adjustment of image brightness, random addition of Gaussian noise and random cropping.

5. The method for detecting the wire pulling state based on image analysis according to claim 4, wherein: The step of extracting the line pixel boundary point set of the current key area image for each key area image includes: For each key area image, a set of pixel points of the pull-line area is extracted from the current key area image; based on a preset edge detection algorithm, the edge contour of the pull-line area is extracted from the pixel point set; and the pixel points on the edge contour are used as the pull-line pixel boundary point set of the current key area image.

6. The method for detecting the state of a wire according to claim 5, wherein: The step of calculating the radian value of each segment of the pull line corresponding to the current key area image according to the pull line pixel boundary point set and adding the radian value of each segment of the pull line to the radian value set includes: Based on the cubic spline curve fitting algorithm, the shape of the pull line is fitted according to the pull line pixel boundary point set to generate a pull line fitting curve; Calculating the curvature of equally spaced points in the line fitting curve to generate a plurality of curvature values; According to the several curvature values, the curvature is numerically integrated along the arc length to obtain the bending arc value of each section of the wire; Adding the bending radian value of each section of the cable to the radian value set; Here, the curvature of equally spaced points is calculated according to the following formula: Where K(x) is the curvature value of the line fitting curve at point x; s″ i (x) is the second-order derivative of the cubic spline curve at point x; s′ i (x) is the first derivative of the cubic spline curve at point x.

7. The method for detecting the wire pulling state based on image analysis according to claim 6, wherein: When it is determined that the pull wire to be detected is in a slack state, the method further includes: Determining the position of the wire section whose arc value exceeds the arc threshold range, and generating position information of the wire slack section; A visual inspection report including slack position markings is generated based on the original target image and the position information of the slack wire section.

8. A wire pulling state detection device based on image analysis, characterized in that: include: Target image acquisition module, foreground area image generation module, wireline area segmentation mask generation module, key area image generation module, radian value calculation module and wireline status detection module; The target image acquisition module is used to acquire a plurality of target images of the to-be-detected wire at multiple viewing angles; The foreground area image generation module is used to perform foreground segmentation processing on each target image to generate a corresponding foreground area image; The pull-line region segmentation mask generation module is used to sequentially input each foreground region image into a preset semantic segmentation model, so that the semantic segmentation model generates a segmentation mask of the pull-line region in each foreground region image based on each foreground region image; The key area image generation module is used to remove the non-pull-line area in each foreground area image according to the segmentation mask of the pull-line area, and generate a corresponding key area image containing only the pull-line; The radian value calculation module is used to extract the line pixel boundary point set of the current key area image for each key area image; calculate the radian value of each line segment corresponding to the current key area image based on the line pixel boundary point set, and add the radian of each line segment to the radian value set; The wire state detection module is used to count the percentage of the radian values ​​of each section of the wire in the radian value concentration that exceeds the preset radian threshold range, and if the percentage is greater than the preset ratio threshold, it is determined that the wire to be detected is in a relaxed state; Otherwise, it is determined that the pull wire to be detected is in a normal state.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for detecting the wire state based on image analysis as described in any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the wire pulling state detection method based on image analysis according to any one of claims 1 to 7.

Images