Unmanned aerial vehicle routing inspection cable surface defect detection method based on artificial intelligence

Abstract

The invention relates to an unmanned aerial vehicle routing inspection cable surface defect detection method based on artificial intelligence. The method comprises the following steps: acquiring first cable polarization image data and second cable polarization image data; the first cable polarization image data is image data obtained by shooting the surface of the cable at a preset polarization angle by the unmanned aerial vehicle; the second cable polarization image data is image data obtained by shooting the cable surface the same as the first cable polarization image data at another preset polarization angle by the unmanned aerial vehicle; performing polarization feature extraction and fusion on the first cable polarization image data and the second cable polarization image data to obtain reflection invariant feature data; physically constraining the reflection invariant feature data to obtain cable reflection invariant image data; according to the cable reflection invariant image data, a support vector machine classifier is adopted to judge the cable defect condition, and a cable defect result corresponding to the cable surface is obtained. The method can improve the success rate of unmanned aerial vehicle inspection defect identification.

Description

Technical Field

[0001] The present invention belongs to the field of image analysis, and in particular relates to a method for detecting surface defects of cables during drone inspection based on artificial intelligence. Background Art

[0002] With the development of image analysis technology, drone inspection technology has emerged for cable defect detection in the power industry. This technology can quickly take photos of cable surfaces and identify the type and location of cable defects based on the photos. Compared with manual inspections, it has effectively solved worker safety issues and improved inspection efficiency.

[0003] Current drone inspections mainly rely on multi-scale convolutional neural networks (such as Faster R-CNN and YOLO series) to perform target detection on RGB images collected by drones. Its technical feature is that it can extract features of typical defects such as cracks and rust through end-to-end training.

[0004] However, the current drone inspection technology still has problems. For example, when the sun's incident angle is large, a reflective area will be formed on the cable surface. During the processing of the existing drone inspection technology, the defective area will overlap with the grayscale value of the reflective area, and the defects in the reflective area cannot be identified, resulting in a greatly reduced success rate in defect identification by drone inspection technology. Summary of the Invention

[0005] Based on this, it is necessary to address the above technical problems and provide an artificial intelligence-based drone inspection cable surface defect detection method that can identify defects in reflective areas and improve the success rate of cable defect identification.

[0006] In the first aspect, the present application provides a method for detecting cable surface defects by drone inspection based on artificial intelligence, comprising:

[0007] Acquire first cable polarization image data and second cable polarization image data; the first cable polarization image data is image data obtained by photographing the cable surface by a drone using a preset polarization angle; the second cable polarization image data is image data obtained by photographing the same cable surface as the first cable polarization image data by a drone using another preset polarization angle;

[0008] Extracting and fusing polarization features of the first cable polarization image data and the second cable polarization image data to obtain reflection-invariant feature data;

[0009] Physical constraints are applied to the reflection-invariant feature data to obtain the reflection-invariant image data of the cable;

[0010] According to the cable reflection invariant image data, the support vector machine classifier is used to judge the cable defects and obtain the cable defect results corresponding to the cable surface.

[0011] Furthermore, polarization feature extraction and fusion are performed on the first cable polarization image data and the second cable polarization image data to obtain reflection invariant feature data, including:

[0012] Calculate the first linear polarization component using the following formula:

[0013] S1=I1cos 2θ1+I2cos 2θ2

[0014] Wherein, S1 is the first linear polarization component, I1 is the first cable polarization image data, θ1 is the polarization angle of the first cable polarization image data, I2 is the second cable polarization image data, and θ2 is the polarization angle of the second cable polarization image data;

[0015] Calculate the second linear polarization component using the following formula:

[0016] S2=I1sin2θ1+I2sin 2θ2

[0017] Wherein, S2 is the second linear polarization component, I1 is the first cable polarization image data, θ1 is the polarization angle of the first cable polarization image data, I2 is the second cable polarization image data, and θ2 is the polarization angle of the second cable polarization image data;

[0018] Calculating the quantized polarized light ratio according to the first linear polarization component and the second linear polarization component;

[0019] Determine the high reflective areas on the cable surface based on the quantified polarized light ratio;

[0020] Based on the reflection suppression algorithm, reflections are suppressed in high-reflection areas, and the reflection weight map is calculated using the following formula:

[0021] W=e -α·DoP

[0022] Where W is the reflection weight map, α is the reflection suppression intensity, and DoP is the quantized polarized light ratio;

[0023] The first cable polarization image data and the second cable polarization image data are fused according to the reflection weight map to obtain reflection invariant feature data.

[0024] Furthermore, according to the reflection weight map, the first cable polarization image data and the second cable polarization image data are fused to obtain reflection invariant feature data, including:

[0025] Performing multi-scale layering on the first cable polarization image data and the second cable polarization image data to obtain corresponding first layered data and second layered data;

[0026] Calculating a detail map of each layer according to the first layer data and the second layer data to obtain first-level detail data and second-level detail data;

[0027] According to the reflectance weight map, the first-level detail data and the second-level detail data are feature-fused to obtain reflectance-invariant feature data;

[0028] Among them, the reflective invariant feature data is obtained by the following formula:

[0029]

[0030] Among them, I fused is the reflective invariant feature data, N is the number of multi-scale layers, Upsample is the upsampling operation function, is the preliminary fusion image of the lth layer;

[0031] Among them, the preliminary fusion image of the lth layer is calculated by the following formula:

[0032]

[0033] in, is the initial fusion image of the lth layer, W is the reflection weight map, is the lth level detail map in the first level detail data, It is the lth level detail map in the second level detail data.

[0034] Furthermore, the method further comprises:

[0035] Use the following formula to adjust the polarization angles of the first cable polarization image data and the second cable polarization image data:

[0036]

[0037] Among them, θ opt To adjust the polarization angle, is to solve the current polarization angle θ, I θ (x, y) is the (x, y) coordinate data in the image data of the current polarization angle, I t (x, y) is the data of the (x, y) coordinates in the defect-free image data.

[0038] Furthermore, physical constraints are applied to the reflection-invariant feature data to obtain the cable reflection-invariant image data, including:

[0039] According to the loss function, the reflection loss entropy of the reflection invariant feature data is calculated;

[0040] Among them, the reflection loss entropy is calculated by the following formula:

[0041]

[0042] in, is the reflection loss entropy, I fused is the reflection invariant feature data, D is the diffuse reflection component in the reflection invariant feature data, S is the specular reflection component in the reflection invariant feature data, λ phys is the physical constraint weight, DoP(D) is the degree of polarization of the specular reflection component, and M is the proportion of the specular reflection component in the reflective invariant feature data;

[0043] According to the reflection loss entropy, an iterative solution is used to update the reflection invariant feature data, and the cable reflection invariant image data is calculated using the following formula:

[0044]

[0045] Among them, I f It is the cable reflection unchanged image data, is the diffuse reflection component of the K-1th iteration, η is the learning rate, Yes beg guide.

[0046] Furthermore, based on the cable reflection invariant image data, a support vector machine classifier is used to judge the cable defects and obtain the cable defect results corresponding to the cable surface, including:

[0047] Extract multi-dimensional defect features from cable reflection-invariant image data to obtain original defect features;

[0048] Normalizing the original defect features to obtain standardized original defect features;

[0049] Among them, the standardized original defect characteristics are obtained by the following formula:

[0050]

[0051] in, is the normalized original defect feature of the i-th dimension, f i is the original defect feature of the i-th dimension, μ i is the mean of the defect feature of the i-th dimension, σ i is the standard deviation of the defect feature in the i-th dimension;

[0052] The support vector machine classifier is used to input the standardized original defect features into the trained defect classification model to obtain the cable defect results.

[0053] Furthermore, the defect classification model is trained by the following method:

[0054] Acquire normal cable defect images;

[0055] Manually label the defect types of normal cable defect images to obtain a circuit defect type atlas;

[0056] The circuit defect type atlas is divided into a training set and a test set according to a preset ratio;

[0057] Based on the support vector machine classifier, the defect classification model is trained using the training set to obtain the weight vector and bias term;

[0058] The weight vector is obtained using the following formula:

[0059]

[0060] Among them, w is the weight vector, N is the number of samples, α i is the influence weight of the i-th training sample on the classification hyperplane, y i is the classification label of the i-th training sample, φ(f i ) is the high-dimensional mapping feature of the i-th training sample;

[0061] The test set is used to train the defect classification model, and the weight vector and bias item are adjusted to obtain the trained defect classification model.

[0062] In a second aspect, the present application also provides an artificial intelligence-based drone inspection cable surface defect detection device, comprising:

[0063] An image data acquisition module is configured to acquire first cable polarization image data and second cable polarization image data; the first cable polarization image data is image data obtained by photographing the cable surface by a drone using a preset polarization angle; the second cable polarization image data is image data obtained by photographing the same cable surface as the first cable polarization image data by a drone using another preset polarization angle;

[0064] An image reflection suppression module is used to extract and fuse polarization features of the first cable polarization image data and the second cable polarization image data to obtain reflection invariant feature data;

[0065] An image regularity constraint module is used to perform physical constraints on the reflection-invariant feature data to obtain the reflection-invariant image data of the cable;

[0066] The image defect determination module is used to judge the cable defect situation based on the cable reflection invariant image data and adopt the support vector machine classifier to obtain the cable defect result corresponding to the cable surface.

[0067] In a third aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements any one of the artificial intelligence-based drone inspection cable surface defect detection methods described in the first aspect of the present application.

[0068] In a fourth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the artificial intelligence-based drone inspection cable surface defect detection methods described in the first aspect of the present application.

[0069] The above-mentioned artificial intelligence-based drone inspection cable surface defect detection method obtains first cable polarization image data and second cable polarization image data; the first cable polarization image data is image data obtained by the drone using a preset polarization angle to shoot the cable surface; the second cable polarization image data is image data obtained by the drone using another preset polarization angle to shoot the same cable surface as the first cable polarization image data; polarization features of the first cable polarization image data and the second cable polarization image data are extracted and fused to obtain reflection-invariant feature data; physical constraints are applied to the reflection-invariant feature data to obtain cable reflection-invariant image data; a support vector machine classifier is used based on the cable reflection-invariant image data to judge the cable defect situation and obtain the cable defect result corresponding to the cable surface, thereby realizing the identification of defects in the reflective area of ​​the cable surface and improving the success rate of defect identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 A flowchart of a method for detecting cable surface defects by drone inspection based on artificial intelligence is provided as an exemplary embodiment of the present application;

[0072] Figure 2 A structural block diagram of an artificial intelligence-based drone inspection cable surface defect detection device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0074] like Figure 1 As shown, a method for detecting cable surface defects by drone inspection based on artificial intelligence is provided. This method is described using a terminal as an example. It is understood that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the following steps S101 to S104 are included. Among them:

[0075] S101, obtaining first cable polarization image data and second cable polarization image data; the first cable polarization image data is image data obtained by photographing the cable surface by a drone using a preset polarization angle; the second cable polarization image data is image data obtained by photographing the same cable surface as the first cable polarization image data by a drone using another preset polarization angle.

[0076] Specifically, the terminal acquires two sets of image data from a drone capturing the cable surface: first cable polarization image data and second cable polarization image data. The first cable polarization image data is captured by the drone using a preset polarization angle; the second cable polarization image data is captured by the drone using a different preset polarization angle for the same cable surface. Illustratively, the polarization angles for the first and second cable polarization image data can be set based on user needs or calculated using a polarization angle optimization formula.

[0077] S102 , extracting and fusing polarization features of the first cable polarization image data and the second cable polarization image data to obtain reflection-invariant feature data.

[0078] Specifically, the terminal processes the acquired polarization image data of the first cable and the second cable, analyzes the characteristics of the two images such as the light intensity distribution at different polarization angles, identifies the high-reflection area on the surface of the photographed cable, and extracts the detail feature data in layers in the high-reflection area. Then, the reflection weight is calculated using the reflection suppression algorithm, and these detail features are weightedly fused according to the reflection weight to generate reflection-invariant feature data.

[0079] S103, performing physical constraints on the reflection-invariant feature data to obtain the cable reflection-invariant image data.

[0080] Specifically, the terminal processes the reflection-invariant feature data based on physical constraints. Based on the physical properties of light reflection on the cable surface, it modifies and optimizes the reflection-invariant feature data to produce the cable reflection-invariant image data. In principle, the reflection-invariant feature data can be iteratively solved by calculating the loss entropy to produce the cable reflection-invariant image data.

[0081] S104, using a support vector machine classifier to judge the cable defect situation based on the cable reflection invariant image data, and obtaining a cable defect result corresponding to the cable surface.

[0082] Specifically, the terminal uses a support vector machine classifier to analyze and judge the cable surface defects based on the cable reflection invariant image data. Based on the features in the cable reflection invariant image data, it determines whether there are defects on the cable surface and the type of defects, and finally obtains the cable defect results corresponding to the cable surface.

[0083] This embodiment provides an artificial intelligence-based drone inspection cable surface defect detection method, which obtains first cable polarization image data and second cable polarization image data, de-reflects the two sets of image data, and performs feature extraction and fusion to obtain reflection-invariant feature data. Physical constraints are imposed on the reflection-invariant feature data to obtain cable reflection-invariant image data. The cable reflection-invariant image data is input into a support vector machine classifier to determine whether the cable has defects and the specific circumstances of the defects, and obtain cable defect results. This method realizes the identification of defects in reflective areas, effectively reduces the impact of cable surface reflections, and improves the accuracy and reliability of defect detection.

[0084] In one embodiment, polarization feature extraction and fusion of the first cable polarization image data and the second cable polarization image data are performed to obtain reflection-invariant feature data, including:

[0085] S201, calculate the first linear polarization component using the following formula:

[0086] S1=I1cos 2θ1+I2cos 2θ2

[0087] Wherein, S1 is the first linear polarization component, I1 is the first cable polarization image data, θ1 is the polarization angle of the first cable polarization image data, I2 is the second cable polarization image data, and θ2 is the polarization angle of the second cable polarization image data.

[0088] Specifically, the polarization states of the first and second cable polarization image data are modulated using a cosine function to extract the horizontal vibration characteristics of the surface defect and calculate a first linear polarization component. Illustratively, the first linear polarization component S1 represents the horizontal polarization intensity of the cable surface defect. Optionally, the first cable polarization image data I1 is the input first cable polarization image data. Illustratively, the polarization angle θ1 of the first cable polarization image data is the polarization angle used to capture the first cable polarization image data. Optionally, the second cable polarization image data I2 is the input second cable polarization image data. Illustratively, the polarization angle θ2 of the second cable polarization image data is the polarization angle used to capture the second cable polarization image data.

[0089] S202, calculate the second linear polarization component using the following formula:

[0090] S2=I1sin2θ1+I2sin 2θ2

[0091] Wherein, S2 is the second linear polarization component, I1 is the first cable polarization image data, θ1 is the polarization angle of the first cable polarization image data, I2 is the second cable polarization image data, and θ2 is the polarization angle of the second cable polarization image data.

[0092] Specifically, the polarization states of the first and second cable polarization image data are modulated using a sine function to extract the vertical vibration characteristics of the surface defect and calculate a second linear polarization component. Illustratively, the first linear polarization component S2 is used to characterize the polarization intensity in the vertical direction of the cable surface defect. Optionally, the first cable polarization image data I1 is the input first cable polarization image data. Illustratively, the polarization angle θ1 of the first cable polarization image data is the polarization angle used to capture the first cable polarization image data. Optionally, the second cable polarization image data I2 is the input second cable polarization image data. Illustratively, the polarization angle θ2 of the second cable polarization image data is the polarization angle used to capture the second cable polarization image data.

[0093] S203 , calculating a quantized polarized light ratio according to the first linear polarization component and the second linear polarization component.

[0094] Specifically, the quantized polarization fraction is calculated based on the first and second linear polarization components according to the formula. Schematically, the quantized polarization fraction reflects the proportion of polarized light in the cable surface reflection. A value closer to 1 indicates stronger specular reflection (such as in metal oxide regions). This quantized polarization fraction can be used as a criterion for high-reflectivity areas.

[0095] S204: Determine a high-reflection area on the cable surface based on the quantified polarized light ratio.

[0096] Specifically, the quantified polarized light ratio is compared with a preset polarized light ratio threshold. When the quantized polarized light ratio exceeds the polarized light ratio threshold, the corresponding cable surface area is determined to be a high-reflection area. In principle, the polarized light ratio threshold can be set based on the user's requirements for reflectivity.

[0097] S205, based on the reflection suppression algorithm, the reflection of the high reflection area is suppressed, and the reflection weight map is calculated using the following formula:

[0098] W=e -α·DoP

[0099] Among them, W is the reflection weight map, α is the reflection suppression intensity, and DoP is the quantized polarized light ratio.

[0100] Specifically, based on the reflection suppression algorithm, the terminal uses an exponential function based on the quantized polarized light percentage to ensure that the reflection weight decreases nonlinearly with the quantized polarized light percentage, resulting in a reflection weight map. Schematically, the reflection weight map W is used to adjust the contribution of reflections from various areas of the cable surface to the overall image characteristics. Optionally, the reflection suppression intensity α is used to control the degree of reflection suppression; larger values ​​result in stronger reflection suppression effects, and can be set based on user needs for reflection suppression. Schematically, the quantized polarized light percentage DoP is the input quantized polarized light percentage.

[0101] S206 , fusing the first cable polarization image data and the second cable polarization image data according to the reflection weight map to obtain reflection invariant feature data.

[0102] Specifically, the first cable polarization image data and the second cable polarization image data are layered to extract features, and the layered features are weighted fused using the reflection weight map. The two polarization image data are evenly fused in low-reflection areas, and noise is suppressed in high-reflection areas to obtain reflection-invariant feature data.

[0103] This embodiment calculates the first linear polarization component and the second linear polarization component, and calculates the quantized polarization intensity based on the first linear polarization component and the second linear polarization component to determine the high-reflection area and the reflection weight map. According to the reflection weight map, the first cable polarization image data and the second cable polarization image data are layered extracted and weighted fused to obtain the reflection-invariant feature data of the cable surface, thereby reducing the interference of mirror reflection and providing more accurate cable surface image data for defect identification.

[0104] In one embodiment, the first cable polarization image data and the second cable polarization image data are fused according to the reflectance weight map to obtain reflectance invariant feature data, including:

[0105] S301 , performing multi-scale layering on the first cable polarization image data and the second cable polarization image data to obtain corresponding first layered data and second layered data.

[0106] Specifically, the terminal performs feature decomposition on the first and second cable polarization image data through multi-scale layered processing to obtain corresponding first and second layered data. Illustratively, the number of multi-scale layers can be set based on the user's requirements for cable surface image processing. Optionally, the dimensions of the first and second layered data are determined by the number of multi-scale layers.

[0107] S302 : Calculate a detail map of each layer according to the first layer data and the second layer data to obtain first-level detail data and second-level detail data.

[0108] Specifically, the terminal uses the Gaussian difference algorithm to perform Gaussian difference calculation on the hierarchical data of each level in the first hierarchical data and the second hierarchical data to obtain a detail map of each layer, and combines the detail maps to obtain corresponding first-level detail data and second-level detail data.

[0109] S303, performing feature fusion on the first-level detail data and the second-level detail data according to the reflectance weight map to obtain reflectance invariant feature data;

[0110] Among them, the reflective invariant feature data is obtained by the following formula:

[0111]

[0112] Among them, I fused is the reflective invariant feature data, N is the number of multi-scale layers, Upsample is the upsampling operation function, is the preliminary fusion image of the lth layer;

[0113] Among them, the preliminary fusion image of the lth layer is calculated by the following formula:

[0114]

[0115] in, is the initial fusion image of the lth layer, W is the reflection weight map, is the lth level detail map in the first level detail data, It is the lth level detail map in the second level detail data.

[0116] Specifically, the reflectance weight map is used to perform weighted fusion of the detail maps of each layer in the first-level detail data and the second-level detail data to obtain a preliminary fused image of each layer. The preliminary fused images of each layer are upsampled and superimposed layer by layer through the upsampling operation function, and the fusion results of each layer are restored to the original resolution to obtain the reflectance invariant feature data. Schematically, the reflectance invariant feature data I fused is the cable surface image data at the original resolution with reduced reflection interference. Optionally, the multi-scale layer number N is the number of layers in the multi-scale layered processing, which can be set according to the user's requirements for cable surface image processing. Schematically, the upsampling operation function Upsample is a bilinear interpolation upsampling function. It is used to gradually restore the low-scale fusion result to the original resolution. Optionally, the preliminary fusion image of the lth layer It is the result of weighted fusion of the first-level detail map in the multi-scale layered detail data and the second-level detail map in the second-level detail data. Schematically, the reflection weight map W is the reflection weight map obtained by calculating the quantized polarized light ratio. The first layer of detail data is obtained by Gaussian difference calculation. It is the detail map of the lth layer obtained by calculating the Gaussian difference of the second layer data.

[0117] This embodiment performs multi-scale layered processing on the first cable polarization image data and the second cable polarization image data to generate corresponding first layered data and second layered data. Based on the two sets of layered data, local feature details of the image are extracted, and detail maps of each layer are calculated respectively to obtain first-level detail data and second-level detail data. The first-level detail data and the second-level detail data are feature fused using a reflection weight map. Reflection-invariant feature data is obtained through weighted fusion and upsampling operations, which effectively suppresses reflection interference and highlights the true features of the cable surface.

[0118] In one embodiment, the method further comprises:

[0119] Use the following formula to adjust the polarization angles of the first cable polarization image data and the second cable polarization image data:

[0120]

[0121] Among them, θ opt To adjust the polarization angle, is to solve the current polarization angle θ, I θ (x, y) is the (x, y) coordinate data in the image data of the current polarization angle, I t (x, y) is the data of the (x, y) coordinates in the defect-free image data.

[0122] Specifically, the polarization angle of the image taken at the current polarization angle is calculated based on the image data taken at the current polarization angle and the obtained flawless image data. opt The optimized polarization angle is calculated, and the polarization angle can be used to adjust the shooting polarization angle of the first cable polarization image data, and the optimal polarization angle difference is calculated. Adjust the shooting polarization angle of the second cable polarization image data, the optimal polarization angle difference Optionally, solve for the current polarization angle θ The minimization operation is performed to find the polarization angle that minimizes the difference between the current polarization image and the standard defect-free image. Schematically, the (x, y) coordinate data I in the image data of the current polarization angle is θ (x, y) is the data corresponding to the coordinates (x, y) in the image data collected with the polarization angle θ. Alternatively, it is the data of the coordinates (x, y) in the defect-free image data. t (x, y) is the data corresponding to the coordinates (x, y) in the acquired defect-free image. The defect-free image can be generated through laboratory calibration or historical inspection defect-free data.

[0123] This embodiment calculates the polarization angle that minimizes the difference between the image data and the defect-free reference image using the optimal polarization angle calculation formula, adjusts the polarization angle of the cable polarization image data, and thus reduces reflection interference.

[0124] In one embodiment, physical constraints are applied to the reflection-invariant feature data to obtain the reflection-invariant image data of the cable, including:

[0125] S501, calculating the reflection loss entropy of the reflection invariant feature data according to the loss function;

[0126] Among them, the reflection loss entropy is calculated by the following formula:

[0127]

[0128] in, is the reflection loss entropy, I fused is the reflection invariant feature data, D is the diffuse reflection component in the reflection invariant feature data, S is the specular reflection component in the reflection invariant feature data, λ phys is the physical constraint weight, DoP(D) is the degree of polarization of the specular reflection component, and M is the proportion of the specular reflection component in the reflection invariant feature data.

[0129] Specifically, based on the reflection invariant feature data, the reflection loss entropy is calculated using a loss function, which consists of two parts: one is the difference between the reflection invariant feature data and the sum of the diffuse reflection and specular reflection components, and the other is the difference between the polarization degree of the specular reflection component and the specular reflection ratio. It is used to optimize the reflection-invariant feature data and obtain the reflection-invariant image data of the cable. fusedis the reflection-invariant feature data outputted by S303. Schematically, the diffuse reflection component D in the reflection-invariant feature data is the diffuse reflection component to be optimized, and is the low-frequency component dominated by the defect feature. Optionally, the specular reflection component S in the reflection-invariant feature data is the specular reflection component to be optimized, and is the high-frequency noise dominated by the reflection. The addition of the diffuse reflection component D and the specular reflection component S can obtain the reflection-invariant feature data I. fused Schematically, the physical constraint weight λ phys is a weight for balancing data reconstruction error and physical model fit. This physical constraint weight can be set based on the user's requirements for matching image data with optical properties. Optionally, the degree of polarization of the specular reflection component DoP(D) is the degree of polarization of the diffuse reflection component and can be calculated in S303. Illustratively, the proportion of specular reflection components in the reflective invariant feature data, M, is the proportion of specular reflection components in the reflective invariant feature data, representing the proportion of specular reflection in the total reflectance.

[0130] S502: Update the reflection-invariant feature data using an iterative solution based on the reflection loss entropy, and obtain the cable reflection-invariant image data using the following formula:

[0131]

[0132] Among them, I f It is the cable reflection unchanged image data, is the diffuse component of the K-1th iteration, η is the learning rate, Yes beg guide.

[0133] Specifically, the terminal iteratively optimizes the reflection-invariant feature data by the gradient descent method to obtain the cable reflection-invariant image data. f is the output cable reflection invariant image data. Optionally, the diffuse reflection component of the K-1th iteration Characterizing the true reflection characteristics of the cable surface can be calculated by the loss function. Schematically, the learning rate η is used to control the parameter update step size, which is determined by cross-validation to avoid oscillation or slow convergence. It can be set according to the user's requirements for fitting image data and optical characteristics. beg guide It is to calculate the gradient of the loss function with respect to the diffuse component, which is used to guide the optimization direction to balance the data reconstruction error and the physical law constraints.

[0134] This embodiment achieves accurate separation of cable surface defects through combined loss optimization and iterative solution, and obtains cable reflection-invariant image data that retains defect characteristics and conforms to the physical laws of polarization.

[0135] In one embodiment, a support vector machine classifier is used to determine cable defects based on the cable reflection invariant image data, and obtain cable defect results corresponding to the cable surface, including:

[0136] S601, extracting multi-dimensional defect features from the cable reflection invariant image data to obtain original defect features.

[0137] Specifically, the terminal analyzes the cable's reflective image data, extracts features from multiple dimensions, and integrates these features to form a comprehensive feature vector, which serves as the original defect signature. Schematically, a multi-dimensional defect feature extraction method can be used to extract features, including features from three dimensions: color, texture, and edge.

[0138] S602, normalizing the original defect features to obtain standardized original defect features;

[0139] Among them, the standardized original defect characteristics are obtained by the following formula:

[0140]

[0141] in, is the normalized original defect feature of the i-th dimension, f i is the original defect feature of the i-th dimension, μ i is the mean of the defect feature of the i-th dimension, σ i is the standard deviation of the defect feature in the i-th dimension.

[0142] Specifically, the data of all dimensions in the original defect feature are standardized by the standardized calculation formula to obtain the standardized original defect feature of each dimension, and then combined into the standardized original defect feature. Schematically, the standardized original defect feature of the i-th dimension is is the normalized original defect feature of the i-th dimension. Optionally, the original defect feature of the i-th dimension f i is the data of the i-th dimension in the original defect feature output by S601. Schematically, the mean value μ of the i-th dimension defect feature is i is the mean of the original defect feature data in the i-th dimension, which can be used to center the data. Alternatively, the standard deviation σ of the defect feature in the i-th dimension i is the standard deviation of the original defect feature data of the i-th dimension, which can be used to standardize the distribution of data.

[0143] S603: Using a support vector machine classifier, the standardized original defect features are input into the trained defect classification model to obtain the cable defect results.

[0144] Specifically, the terminal uses a support vector machine classifier to construct a defect classification model. This model is trained based on a certain number of normal cable defect images to obtain a trained defect classification model. Normalized raw defect features are then input into the trained defect classification model to obtain cable surface defect recognition results. Illustratively, the number of normal cable defect images used for model training can be set based on the user's requirements for model classification refinement. Optionally, the type of cable surface defect recognition results can be set based on the user's cable repair needs.

[0145] This embodiment extracts multi-dimensional defect features from the cable reflection-invariant image data and performs standardization processing to obtain standardized original defect features. The standardized original defect features are then input into a defect classification model constructed using a support vector machine classifier to obtain defect recognition results on the cable surface, thereby enhancing the robustness to reflective scenes and improving the convergence speed and classification accuracy of the classification model.

[0146] In one embodiment, the defect classification model is trained by the following method:

[0147] S701, obtaining a normal cable defect image.

[0148] Specifically, the terminal acquires a certain number of normal cable defect images. In principle, this number can be set based on the user's requirements for model classification refinement. Optionally, the normal cable defect images can be user-photographed images of normal cable defects or images generated through laboratory calibration.

[0149] S702: Manually label the defect types of the normal cable defect image to obtain a circuit defect type atlas.

[0150] Specifically, the terminal manually annotates a normal cable defect image according to the corresponding defect type, generating a circuit defect atlas. Each data entry in the atlas contains the normal cable defect image and the annotated defect type. For example, the number of defect types can be configured based on the user's needs for cable surface defect management.

[0151] S703 , dividing the circuit defect type atlas into a training set and a test set according to a preset ratio.

[0152] Specifically, the terminal divides the circuit defect atlas into a training set and a test set according to a preset ratio. Illustratively, classification can be performed using a stratified random sampling algorithm. Optionally, the ratio of the training set to the test set can be set based on the user's model training requirements.

[0153] S704, based on the support vector machine classifier, using the training set to train the defect classification model, and obtain the weight vector and bias term;

[0154] The weight vector is obtained using the following formula:

[0155]

[0156] Among them, w is the weight vector, N is the number of samples, α i is the influence weight of the i-th training sample on the classification hyperplane, y i is the classification label of the i-th training sample, φ(f i ) is the high-dimensional mapping feature of the i-th training sample.

[0157] Specifically, a weight vector is calculated that will affect the direction of the classification of the input image data. Schematically, the weight vector w is used to characterize the direction of the classification hyperplane in the high-dimensional feature space. Optionally, the number of samples N is the number of samples in the output training set. Schematically, the weight α of the influence of the i-th training sample on the classification hyperplane is i is the weight of the influence of the i-th training sample on the classification hyperplane. Optionally, the classification label y of the i-th training sample i is the defect classification label corresponding to the image data in the i-th training sample in the training set. Schematically, the high-dimensional mapping feature φ(f i ) is the high-dimensional mapping feature obtained by high-dimensional mapping the image data in the i-th training sample in the training set, which can be mapped using the RBF kernel function.

[0158] S705 , using the test set to train the defect classification model, adjusting the weight vector and the bias term, and obtaining a trained defect classification model.

[0159] Specifically, the defect classification model is trained using the test set, and the weight vector and bias term are adjusted to obtain a trained defect classification model. Schematically, the trained defect classification model can be used to input the standardized original defect features to obtain the corresponding defect type results.

[0160] This embodiment uses a support vector separator to construct a defect classification model, realizes the input of defect image features with reflective features removed and fitted with physical laws, obtains defect type results, and improves the efficiency of cable surface defect type identification.

[0161] In the above-mentioned artificial intelligence-based drone inspection cable surface defect detection method, polarization features are extracted and fused from images taken at different polarization angles, a reflection suppression algorithm is used to obtain the removed reflection features, and a support vector machine classifier is used to classify the cable surface image data with the removed reflection features to obtain the types of cable surface defects. This realizes the identification of defects in the reflective area of ​​the cable surface and improves the success rate of drone inspection in identifying cable surface defects.

[0162] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0163] Based on the same inventive concept, the embodiments of the present application also provide an artificial intelligence-based drone inspection cable surface defect detection device for implementing the aforementioned artificial intelligence-based drone inspection cable surface defect detection method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the artificial intelligence-based drone inspection cable surface defect detection device provided below can be found in the above-mentioned limitations of the artificial intelligence-based drone inspection cable surface defect detection method, and will not be repeated here.

[0164] In an exemplary embodiment, Figure 2 As shown, a device 200 for detecting cable surface defects by drone inspection based on artificial intelligence is provided, comprising:

[0165] The image data acquisition module 201 is configured to acquire first cable polarization image data and second cable polarization image data; the first cable polarization image data is image data obtained by photographing the cable surface by a drone using a preset polarization angle; the second cable polarization image data is image data obtained by photographing the same cable surface as the first cable polarization image data by a drone using another preset polarization angle;

[0166] An image reflection suppression module 202 is configured to extract and fuse polarization features of the first cable polarization image data and the second cable polarization image data to obtain reflection invariant feature data;

[0167] An image regularity constraint module 203 is used to perform physical constraints on the reflection-invariant feature data to obtain the cable reflection-invariant image data;

[0168] The image defect determination module 204 is used to determine the cable defect situation based on the cable reflection invariant image data using a support vector machine classifier to obtain a cable defect result corresponding to the cable surface.

[0169] Furthermore, the image reflection suppression module further includes:

[0170] The first linear polarization calculation unit is configured to calculate the first linear polarization component using the following formula:

[0171] S1=I1cos 2θ1+I2cos 2θ2

[0172] Wherein, S1 is the first linear polarization component, I1 is the first cable polarization image data, θ1 is the polarization angle of the first cable polarization image data, I2 is the second cable polarization image data, and θ2 is the polarization angle of the second cable polarization image data;

[0173] The second linear polarization calculation unit is configured to calculate the second linear polarization component using the following formula:

[0174] S2=I1sin2θ1+I2sin 2θ2

[0175] Wherein, S2 is the second linear polarization component, I1 is the first cable polarization image data, θ1 is the polarization angle of the first cable polarization image data, I2 is the second cable polarization image data, and θ2 is the polarization angle of the second cable polarization image data;

[0176] a quantized polarized light ratio calculation unit, configured to calculate the quantized polarized light ratio based on the first linear polarization component and the second linear polarization component;

[0177] A high-reflection area determination unit, configured to determine a high-reflection area on the cable surface based on a quantified polarized light ratio;

[0178] The reflective weight map calculation unit is used to suppress reflective areas based on the reflective suppression algorithm. The reflective weight map is calculated using the following formula:

[0179] W=e -α·DoP

[0180] Where W is the reflection weight map, α is the reflection suppression intensity, and DoP is the quantized polarized light ratio;

[0181] The feature fusion unit is used to fuse the first cable polarization image data and the second cable polarization image data according to the reflection weight map to obtain reflection invariant feature data.

[0182] Furthermore, the feature fusion unit is also used to:

[0183] Performing multi-scale layering on the first cable polarization image data and the second cable polarization image data to obtain corresponding first layered data and second layered data;

[0184] Calculating a detail map of each layer according to the first layer data and the second layer data to obtain first-level detail data and second-level detail data;

[0185] According to the reflectance weight map, the first-level detail data and the second-level detail data are feature-fused to obtain reflectance-invariant feature data;

[0186] Among them, the reflective invariant feature data is obtained by the following formula:

[0187]

[0188] Among them, I fused is the reflective invariant feature data, N is the number of multi-scale layers, Upsample is the upsampling operation function, is the preliminary fusion image of the lth layer;

[0189] Among them, the preliminary fusion image of the lth layer is calculated by the following formula:

[0190]

[0191] in, is the initial fusion image of the lth layer, W is the reflection weight map, is the lth level detail map in the first level detail data, It is the lth level detail map in the second level detail data.

[0192] Furthermore, the device further comprises:

[0193] The polarization angle optimization module is used to adjust the polarization angles of the first cable polarization image data and the second cable polarization image data using the following formula:

[0194]

[0195] Among them, θ opt To adjust the polarization angle, is to solve the current polarization angle θ, I θ (x, y) is the (x, y) coordinate data in the image data of the current polarization angle, I t (x, y) is the data of the (x, y) coordinates in the defect-free image data.

[0196] Furthermore, the image regularity constraint module is also used to:

[0197] According to the loss function, the reflection loss entropy of the reflection invariant feature data is calculated;

[0198] Among them, the reflection loss entropy is calculated by the following formula:

[0199]

[0200] in, is the reflection loss entropy, I fused is the reflection invariant feature data, D is the diffuse reflection component in the reflection invariant feature data, S is the specular reflection component in the reflection invariant feature data, λ phys is the physical constraint weight, DoP(D) is the degree of polarization of the specular reflection component, and M is the proportion of the specular reflection component in the reflective invariant feature data;

[0201] According to the reflection loss entropy, an iterative solution is used to update the reflection invariant feature data, and the cable reflection invariant image data is calculated using the following formula:

[0202]

[0203] Among them, I f It is the cable reflection unchanged image data, is the diffuse reflection component of the K-1th iteration, η is the learning rate, Yes beg guide.

[0204] Furthermore, the image defect determination module is further configured to:

[0205] Extract multi-dimensional defect features from cable reflection-invariant image data to obtain original defect features;

[0206] Normalizing the original defect features to obtain standardized original defect features;

[0207] Among them, the standardized original defect characteristics are obtained by the following formula:

[0208]

[0209] in, is the normalized original defect feature of the i-th dimension, f i is the original defect feature of the i-th dimension, μ i is the mean of the defect feature of the i-th dimension, σ i is the standard deviation of the defect feature in the i-th dimension;

[0210] The support vector machine classifier is used to input the standardized original defect features into the trained defect classification model to obtain the cable defect results.

[0211] Furthermore,

[0212] The defect classification model is trained using the following method:

[0213] Acquire normal cable defect images;

[0214] Manually label the defect types of normal cable defect images to obtain a circuit defect type atlas;

[0215] The circuit defect type atlas is divided into a training set and a test set according to a preset ratio;

[0216] Based on the support vector machine classifier, the defect classification model is trained using the training set to obtain the weight vector and bias term;

[0217] The weight vector is obtained using the following formula:

[0218]

[0219] Among them, w is the weight vector, N is the number of samples, α i is the influence weight of the i-th training sample on the classification hyperplane, y i is the classification label of the i-th training sample, φ(f i ) is the high-dimensional mapping feature of the i-th training sample;

[0220] The test set is used to train the defect classification model, and the weight vector and bias item are adjusted to obtain the trained defect classification model.

[0221] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method for detecting cable surface defects by drone inspection based on artificial intelligence are implemented.

[0222] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0223] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0224] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. A method for detecting cable surface defects by drone inspection based on artificial intelligence, characterized in that: The method comprises: Acquire first cable polarization image data and second cable polarization image data; the first cable polarization image data is image data obtained by photographing the cable surface by a drone using a preset polarization angle; the second cable polarization image data is image data obtained by photographing the same cable surface as the first cable polarization image data by a drone using another preset polarization angle; Extracting and fusing polarization features of the first cable polarization image data and the second cable polarization image data to obtain reflection-invariant feature data; Performing physical constraints on the reflection-invariant feature data to obtain cable reflection-invariant image data; A support vector machine classifier is used to judge the cable defect situation based on the cable reflection invariant image data to obtain a cable defect result corresponding to the cable surface.

2. The method according to claim 1, characterized in that The extracting and fusing polarization features of the first cable polarization image data and the second cable polarization image data to obtain reflection-invariant feature data includes: The first linear polarization component is calculated using the following formula: S1=I1cos 2θ1+I2cos 2θ2 Wherein, S1 is the first linear polarization component, I1 is the first cable polarization image data, θ1 is the polarization angle of the first cable polarization image data, I2 is the second cable polarization image data, and θ2 is the polarization angle of the second cable polarization image data; The second linear polarization component is calculated using the following formula: S2=I1sin 2θ1+I2sin 2θ2 Wherein, S2 is the second linear polarization component, I1 is the first cable polarization image data, θ1 is the polarization angle of the first cable polarization image data, I2 is the second cable polarization image data, and θ2 is the polarization angle of the second cable polarization image data; Calculating a quantized polarized light ratio according to the first linear polarization component and the second linear polarization component; determining a high-reflection area on the cable surface according to the quantified polarized light proportion; Based on the reflection suppression algorithm, the reflection of the high-reflection area is suppressed, and the reflection weight map is calculated using the following formula: W=e -α·DoP Where W is the reflection weight map, α is the reflection suppression intensity, and DoP is the quantized polarized light ratio; The first cable polarization image data and the second cable polarization image data are fused according to the reflection weight map to obtain the reflection invariant feature data.

3. The method according to claim 2, characterized in that The step of fusing the first cable polarization image data and the second cable polarization image data according to the reflection weight map to obtain the reflection invariant feature data includes: Performing multi-scale layering on the first cable polarization image data and the second cable polarization image data to obtain corresponding first layered data and second layered data; Calculating a detail map of each layer based on the first hierarchical data and the second hierarchical data to obtain first-level detail data and second-level detail data; performing feature fusion on the first-level detail data and the second-level detail data according to the reflectance weight map to obtain the reflectance invariant feature data; The reflective invariant feature data is obtained by the following formula: Among them, I fused is the reflective invariant feature data, N is the number of multi-scale layers, Upsample is the upsampling operation function, is the preliminary fusion image of the lth layer; The preliminary fused image of the lth layer is calculated by the following formula: in, is the initial fusion image of the lth layer, W is the reflection weight map, is the lth level detail map in the first level detail data, It is the lth level detail map in the second level detail data.

4. The method according to claim 2, characterized in that The method also includes: The polarization angles of the first cable polarization image data and the second cable polarization image data are adjusted using the following formula: Among them, θ opt To adjust the polarization angle, is to solve the current polarization angle θ, I θ (x, y) is the (x, y) coordinate data in the image data of the current polarization angle, I t (x, y) is the data of the (x, y) coordinates in the defect-free image data.

5. The method according to claim 1, wherein The step of physically constraining the reflection-invariant feature data to obtain the reflection-invariant image data of the cable includes: Calculating the reflection loss entropy of the reflection invariant feature data according to the loss function; The reflection loss entropy is calculated using the following formula: in, is the reflection loss entropy, I fused is the reflection invariant feature data, D is the diffuse reflection component in the reflection invariant feature data, S is the specular reflection component in the reflection invariant feature data, λ phys is the physical constraint weight, DoP(D) is the degree of polarization of the specular reflection component, and M is the proportion of the specular reflection component in the reflective invariant feature data; According to the reflection loss entropy, an iterative solution is used to update the reflection invariant feature data, and the cable reflection invariant image data is calculated using the following formula: Among them, I f It is the cable reflection unchanged image data, is the diffuse reflection component of the K-1th iteration, η is the learning rate, Yes beg guide.

6. The method according to claim 1, characterized in that The method of using a support vector machine classifier to judge the cable defect situation based on the cable reflection invariant image data and obtaining the cable defect result corresponding to the cable surface includes: Extracting multi-dimensional defect features from the cable reflection-invariant image data to obtain original defect features; Normalizing the original defect features to obtain standardized original defect features; The normalized original defect feature is obtained by the following formula: in, is the normalized original defect feature of the i-th dimension, f i is the original defect feature of the i-th dimension, μ i is the mean of the defect feature of the i-th dimension, σ i is the standard deviation of the defect feature in the i-th dimension; A support vector machine classifier is used to input the standardized original defect features into a trained defect classification model to obtain the cable defect results.

7. The method according to claim 6, characterized in that The defect classification model is trained by the following method: Acquire normal cable defect images; Manually label the defect types of the normal cable defect images to obtain a circuit defect type atlas; Dividing the circuit defect type atlas into a training set and a test set according to a preset ratio; Based on a support vector machine classifier, the defect classification model is trained using the training set to obtain a weight vector and a bias term; The weight vector is obtained using the following formula: Among them, w is the weight vector, N is the number of samples, α i is the influence weight of the i-th training sample on the classification hyperplane, y i is the classification label of the i-th training sample, φ(f i ) is the high-dimensional mapping feature of the i-th training sample; The defect classification model is trained using the test set, and the weight vector and the bias term are adjusted to obtain a trained defect classification model.

8. An artificial intelligence-based drone inspection cable surface defect detection device, characterized in that: The device comprises: An image data acquisition module is configured to acquire first cable polarization image data and second cable polarization image data; the first cable polarization image data is image data obtained by photographing a cable surface by a drone using a preset polarization angle; the second cable polarization image data is image data obtained by photographing the same cable surface as the first cable polarization image data by a drone using another preset polarization angle; an image reflection suppression module, configured to extract and fuse polarization features of the first cable polarization image data and the second cable polarization image data to obtain reflection-invariant feature data; An image regularity constraint module, configured to perform physical constraints on the reflection-invariant feature data to obtain cable reflection-invariant image data; The image defect determination module is used to judge the cable defect situation based on the cable reflection invariant image data by using a support vector machine classifier to obtain the cable defect result corresponding to the cable surface.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images