A membrane surface classification and recognition method for cable-membrane structures based on pixel threshold difference

Abstract

The present invention discloses a method for classifying and identifying the membrane surface of a cable-membrane structure based on pixel threshold difference, belonging to the field of membrane structure technology. The method comprises: using a drone front-end camera to capture images of the normal and damaged membrane surfaces of the cable-membrane structure, obtaining initial image data of the captured membrane surface of the cable-membrane structure; using a method for classifying the normal and damaged membrane surfaces of the cable-membrane structure based on pixel threshold difference to classify the normal and damaged membrane surfaces; and using a method for identifying different types of damaged membrane surfaces of the cable-membrane structure based on a YOLOv5s convolutional neural network model to identify damage in images of damaged membrane surfaces. The present invention can quickly distinguish between normal and damaged membrane surfaces, improve the efficiency of damage model recognition, and accurately identify different membrane surface damage types.

Description

Technical Field

[0001] The present invention relates to the technical field of membrane structures, and in particular to a method for classifying and identifying membrane surfaces of cable membrane structures based on pixel threshold difference. Background Art

[0002] With the development and application of coated fabric membranes, membrane structures have rapidly developed in my country since the 1990s. Unlike traditional engineering building structures, membrane structures primarily consist of membrane materials and reinforcing members (steel frames, steel columns, or cables) that apply a certain initial pretension to form a stable spatial shape capable of withstanding external loads. Consequently, membrane structures offer numerous advantages, including large spans, light weight, diverse artistic designs, environmental protection and energy conservation, excellent mechanical properties, strong self-cleaning capabilities, short construction periods, and excellent economic efficiency. They have been hailed as the "modern green buildings of the 21st century." They are currently widely used in large-span buildings such as large stadiums, shopping malls, and public transportation service centers. Because membrane materials are inherently flexible and cannot withstand bending moments, shear forces, and compression, their load-bearing capacity relies on the initial pretension provided by cables and supporting structures. Consequently, the mechanical properties of cable-membrane structures are highly sensitive to damage to the membrane surface. Damage to the membrane surface can cause the structure to lose its equilibrium state. Further external loads can lead to instantaneous fracture. Large-span cable-membrane structures are often iconic local landmarks. Their destruction not only causes significant casualties and economic losses, but also has devastating social impacts. Therefore, to ensure the quality of cable-membrane structures, extend their service life, prevent premature failure due to membrane surface damage, and further reduce the risk of disasters and accidents, regular damage inspection and maintenance of the membrane surface are necessary. Currently, damage detection for cable-membrane structures primarily relies on manual periodic inspection. This method is slow and susceptible to human subjectivity and work experience, reducing the accuracy of damage detection. Therefore, it is necessary to develop automated, unmanned methods for cable-membrane structure damage detection. With the development of artificial intelligence and machine vision technologies, data processing and image recognition methods have been widely applied to civil structure damage detection. Deep learning algorithms, as a typical AI method, offer the advantages of self-learning data features, accurate recognition, and strong applicability, making them an ideal approach for unmanned, intelligent detection of cable-membrane structure damage. However, the membrane surface of the cable-membrane structure has the characteristics of single identification content, simple identification background, and difficulty in identifying early minor damage. When collecting images of it, a large amount of normal membrane surface image information will inevitably be collected. Directly using complex image recognition technology will generate a large amount of redundant information, which will seriously affect the model recognition and prediction efficiency. Summary of the Invention

[0003] The purpose of the present invention is to provide a cable membrane structure membrane surface classification and identification method based on pixel threshold difference, which can realize the rapid classification of normal membrane surface and damaged membrane surface and accurately detect the damage type of the damaged membrane surface.

[0004] To achieve the above-mentioned purpose, the present invention provides a cable-membrane structure membrane surface classification and identification method based on pixel threshold difference, comprising the following steps:

[0005] Step 1: Use the front-end camera of the UAV to collect images of the normal membrane surface and the damaged membrane surface of the cable-membrane structure to obtain the collected initial image data of the membrane surface of the cable-membrane structure;

[0006] Step 2: using a cable-membrane structure normal membrane surface and damaged membrane surface classification method based on pixel threshold difference to classify the normal membrane surface and the damaged membrane surface;

[0007] Step 3: Use the cable-membrane structure different damaged membrane surface types recognition method based on the YOLOv5s convolutional neural network model to identify damage on the damaged membrane surface image.

[0008] Furthermore, the method for classifying the normal membrane surface and the damaged membrane surface of the cable-membrane structure based on pixel threshold difference includes the following steps:

[0009] Step 1: Use the normal membrane surface images collected by the front-end camera of the UAV to establish the initial image database of the cable membrane structure membrane surface;

[0010] Step 2: constructing a semi-supervised convolutional neural network model, wherein the semi-supervised neural network model is only used to train the normal membrane surface data of the cable-membrane structure, and learn to extract the features of the normal membrane surface image of the cable-membrane structure;

[0011] Step 3, based on the semi-supervised convolutional neural network pre-training model, realize encoding and decoding of reconstructing the lossless membrane surface;

[0012] Step 4: input the aerial image data of the cable-membrane structure collected by the drone, calculate the pixel threshold difference with the reconstructed damage-free membrane surface, and determine the damaged membrane surface threshold;

[0013] Step 5: Use the damaged film surface threshold to classify the aerial image into normal film surface and damaged film surface, so as to achieve efficient and intelligent classification detection of normal film surface and damaged film surface.

[0014] Furthermore, a method for identifying different types of damaged membrane surfaces of cable-membrane structures based on the YOLOv5s convolutional neural network model includes the following steps:

[0015] Step 1: Use the front-end camera of the UAV to take aerial images of the membrane surface of the cable-membrane structure to obtain damaged membrane surface images of the cable-membrane structure, mark the damage type and damage location of the damaged membrane surface images, and build a damaged membrane surface database;

[0016] Step 2: constructing a YOLOv5s convolutional neural network model, wherein the YOLOv5s convolutional neural network model is only used to train the marked cable-membrane structure damaged membrane surface database;

[0017] Step 3: The YOLOv5s convolutional neural network model structure includes an input end, a backbone network, a neck and a head, and an output end. The backbone network is used to extract rich picture semantic features from the input image, the neck is used to enhance the detection of multi-scale targets, and the head is used to predict the features transferred from the neck and generate feature maps of three different scales.

[0018] Step 4: training the YOLOv5s convolutional neural network model to identify damage characteristics and types of the cable-membrane structure membrane surface image;

[0019] In step 5, based on the trained YOLOv5s convolutional neural network model, the aerial image of the membrane surface of the cable-membrane structure captured by the drone is input, and the damage type of the damaged membrane surface image is determined. The output result is the damage type recognition result of the damaged membrane surface image taken.

[0020] Furthermore, precision, recall and average precision are used as evaluation indicators to measure the accuracy of damage type detection in the damage type detection task of membrane surface damage images.

[0021] Furthermore, the method of performing image preprocessing on the image database in step 1 and constructing the semi-supervised convolutional neural network model in step 2 includes the following steps:

[0022] Step 1: batch preprocess the initial membrane surface image data to obtain membrane surface grayscale image data with an image resolution of 512 pixels × 512 pixels;

[0023] Step 2: Build a 9-layer semi-supervised convolutional neural network model, where the encoder uses a gradually decreasing convolution kernel size from 11×11 to 3×3 in layers x1 to x5; the decoder structure has convolution kernel sizes in the opposite order of the encoder and uses transposed convolution layers for feature calculation; the encoder output is concatenated with the previous layer x6 before being passed to the decoder layers x7 to x9;

[0024] Step 3: Only unlabeled grayscale image data of normal membrane surface with a resolution of 512 pixels × 512 pixels is provided to the network model for training, ensuring that the model convolutional encoder only learns the feature information of normal membrane surface, and realizing that the network model only encodes and decodes the normal membrane surface, thereby obtaining a trained semi-supervised convolutional neural network pre-training model;

[0025] In step 4, the unlabeled grayscale image data of the damaged membrane surface with a resolution of 512 pixels × 512 pixels is input into the trained normal membrane surface convolutional neural network semi-supervised pre-training model to achieve encoding and decoding of the reconstructed lossless membrane surface.

[0026] Furthermore, the encoding and decoding of the reconstructed lossless film surface is that the convolutional autoencoder of the semi-supervised convolutional neural network model adopts a convolution method to perform a linear transformation on the input film surface image data, and the corresponding weights are shared, and then the output reconstructed film surface image semantic vector is completed based on the linear combination of the basic image blocks of the hidden code; the reconstructed film surface image semantic vector is input into the decoder, and then converted into a three-dimensional vector, and the original information is amplified using a multi-layer deconvolution method to restore the reconstructed film surface image information.

[0027] Furthermore, the method for determining the pixel threshold of the damaged area on the film surface comprises the following steps:

[0028] Step 1: Compare the input membrane surface damage image and the reconstructed membrane surface intact image, and perform pixel threshold difference between the two to obtain the feature residual of the damaged area of ​​the membrane surface damage, thereby realizing pixel segmentation of the damaged area of ​​the membrane surface damage image;

[0029] Step 2: Count the number of pixels in the damaged area after segmentation, determine the pixel threshold of the damaged area on the membrane surface, and implement the semi-supervised convolutional neural network model to automatically identify the normal membrane surface and the damaged membrane surface based on the pixel threshold of the damaged membrane surface; the calculation formula for the pixel threshold difference between the input membrane surface damaged image and the reconstructed membrane surface lossless image is:

[0030] P t =X-ME(X)

[0031] Where P t is the residual, X is the input membrane surface image data, and ME(X) is the reconstructed membrane surface lossless image data output by the semi-supervised convolutional neural network.

[0032] The statistical formula for the number of pixels in the damaged area after segmentation is:

[0033]

[0034] Where T r It is the ratio of the pixels in the segmented damaged area to the total pixels in the reconstructed lossless membrane surface image. The count() function is a counting function.

[0035] Furthermore, in the process of constructing the cable-membrane structure membrane surface damage database, it was determined that the cable-membrane structure membrane surface damage dataset labels include five types: wrinkles, stains, scratches, tears and obstructions.

[0036] Furthermore, in the process of building the YOLOv5s convolutional neural network model, the input images and labels are normalized and preprocessed, and the input image size is uniformly scaled to an image resolution of 640 pixels × 640 pixels. Mosaic image enhancement is performed on the preprocessed image, and image data is randomly scaled, randomly cropped, and randomly arranged to perform image splicing as the training set of the YOLOv5s convolutional neural network.

[0037] Furthermore, the method for enhancing the damaged film surface image database by using the Mosaic image enhancement technology includes:

[0038] Step 1: read four damaged film surface pictures;

[0039] Step 2: Perform operations on the image including flipping, scaling, and color gamut conversion;

[0040] Step 3: Place the four transformed images in different orientations and combine them into one image, which is then put into the YOLOv5s model for training.

[0041] The beneficial effects of the present invention are:

[0042] (1) The present invention can realize automatic screening of damaged membrane surfaces and intelligent detection of five types of membrane surface damage. Aiming at the problems of single membrane surface damage background, slow speed of existing intelligent methods, and lack of dedicated database and intelligent model, targeted improvement strategies are proposed. As a result, the cable-membrane structure damage identification and detection system of the present invention has the characteristics of high efficiency, convenience and low cost, good applicability, and can be applied to engineering practice.

[0043] (2) The present invention is used to quickly distinguish between normal membrane surfaces and damaged membrane surfaces, and can eliminate redundant information of normal membrane surfaces in a large number of detection images, thereby reducing the detection burden of intelligent recognition methods and improving the efficiency of damage model recognition;

[0044] (3) According to the different effects of damage, the membrane surface damage types are divided into five categories, and a corresponding damaged membrane surface image database is established. The database is used to train the YOLOv5s fast classification model, and a cable-membrane structure damage type recognition method based on the YOLOv5 convolutional neural network is proposed to achieve accurate and intelligent membrane surface damage type detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is the main flow chart of the membrane surface intelligent classification and damage type identification method of the present invention;

[0046] Figure 2 It is a topological diagram of the membrane surface intelligent classification and damage type identification method of the present invention;

[0047] Figure 3This is a diagram of the architecture of a semi-supervised convolutional neural network model for classifying normal and damaged membrane surfaces of cable-membrane structures based on pixel threshold difference.

[0048] Figure 4 This is the result of pixel threshold difference calculation on the damaged film surface;

[0049] Figure 5 It is a flowchart of pixel threshold difference on the damaged film surface;

[0050] Figure 6 It is the pixel threshold distribution map of normal film surface and damaged film surface based on pixel threshold difference;

[0051] Figure 7 This is an image showing the recognition of different types of damaged membrane surfaces in a cable-membrane structure based on the YOLOv5s convolutional neural network model.

[0052] Figure 8 This is the output diagram of the prediction accuracy of different damaged membrane surface types of cable-membrane structures based on the YOLOv5s convolutional neural network model; DETAILED DESCRIPTION

[0053] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] The present invention consists of two parts: a method for classifying normal and damaged membrane surfaces of cable-membrane structures based on pixel threshold difference, and a method for identifying different types of damaged membrane surfaces of cable-membrane structures based on a YOLOv5s (a single-stage target detection algorithm) convolutional neural network model. The first part is used to distinguish between normal and damaged membrane surfaces, thereby eliminating redundant normal membrane surface information from a large number of detection images, reducing the detection burden of intelligent recognition methods, and improving the recognition efficiency of damage models. The second part classifies membrane surface damage types into five categories based on the different effects of the damage, establishes a corresponding damaged membrane surface image database, and conducts damage recognition training based on the damaged membrane surface database, which can accurately identify different membrane surface damage types.

[0055] like Figure 1 and Figure 2 The figure shows a flow chart of a cable-membrane structure membrane surface classification and recognition method based on pixel threshold difference. The specific steps of the method are as follows:

[0056] Step 1: Use the front-end camera of the UAV to collect images of the normal membrane surface and the damaged membrane surface of the cable-membrane structure to obtain the collected initial image data of the membrane surface of the cable-membrane structure;

[0057] Step 2: using a cable-membrane structure normal membrane surface and damaged membrane surface classification method based on pixel threshold difference to classify the normal membrane surface and the damaged membrane surface;

[0058] Step 3: Use the cable-membrane structure different damaged membrane surface types recognition method based on the YOLOv5s convolutional neural network model to identify damage on the damaged membrane surface image.

[0059] The specific implementation process of the classification method of normal membrane surface and damaged membrane surface of cable-membrane structure based on pixel threshold difference is as follows:

[0060] Step 1: Use the normal membrane surface images collected by the front-end camera of the UAV to establish the initial image database of the cable membrane structure membrane surface;

[0061] Specifically, the front-end camera of a drone was used to collect images of the normal membrane surface of the cable-membrane structure, and 190 initial images of the normal membrane surface of the cable-membrane structure were obtained. The initial membrane surface image data were batch preprocessed to construct an unlabeled grayscale image database of the normal membrane surface with an image resolution of 512 pixels × 512 pixels.

[0062] Step 2: Construct a semi-supervised convolutional neural network model. This model is used only to learn and extract features from the normal membrane surface images of cable-membrane structures based on the normal membrane surface data. The semi-supervised convolutional neural network model is trained based on 60% of the sample data from the constructed normal membrane surface database of cable-membrane structures. After training, the network model is tested based on the remaining 40% of the sample data from the database.

[0063] like Figure 3 As shown, a semi-supervised convolutional neural network model with 9 layers is constructed, in which the encoder (layers x1 to x5) uses a gradually decreasing convolution kernel from 11×11 to 3×3. The use of this gradually decreasing convolution kernel is to provide the network model with a larger receptive field, thereby better extracting image features. Considering that deeper neural networks are prone to problems such as overfitting and poor generalization ability, the present invention only constructs a convolutional neural network model with 9 layers. The decoder structure has a convolution kernel size in the opposite order to that of the encoder and uses a transposed convolution layer for feature calculation. The output of the encoder is connected to the previous layer x6 before the decoder is passed to layers x7 to x9. For each convolution transposition process, the parameters displayed are the convolution kernel size, step size, and number of convolution kernels of that layer. Batch normalization method and ReLU activation function are used after each layer. When the input image resolution is 512 pixels × 512 pixels, the corresponding convolutional neural network model outputs a reconstructed image of the same specification of 512 pixels × 512 pixels. Moreover, the network model is only trained on 190 normal membrane surface data samples.

[0064] Step 3: Based on the semi-supervised convolutional neural network pre-training model, the encoding and decoding of the reconstructed lossless membrane surface are realized.

[0065] The encoding and decoding of the lossless film surface reconstruction is done by a convolutional autoencoder within a semi-supervised convolutional neural network model. The convolutional autoencoder uses convolution to linearly transform the input film surface image data, with the corresponding weights shared. The output is a reconstructed film surface image semantic vector based on a linear combination of the underlying encoded basic image blocks. The reconstructed film surface image semantic vector is input to the decoder, converted into a three-dimensional vector, and then amplified using multi-layer deconvolution to restore the original information.

[0066] Step 4: Input the aerial image data of the cable-membrane structure collected by the UAV, calculate the pixel threshold difference with the reconstructed damage-free membrane surface, and determine the damaged membrane surface threshold. The model calculates the pixel threshold difference of the damaged membrane surface as follows: Figure 4 For the specific process, please refer to Figure 5 .

[0067] The calculation formula for the pixel threshold difference between the input damaged film surface image and the reconstructed lossless film surface image is:

[0068] P t =X-ME(X)

[0069] Where P t is the residual, X is the input membrane surface image data, and ME(X) is the reconstructed membrane surface lossless image data output by the semi-supervised convolutional neural network.

[0070] The statistical formula for the number of pixels in the damaged area after segmentation is:

[0071]

[0072] Where T r It is the ratio of the pixels in the segmented damaged area to the total pixels in the reconstructed lossless membrane surface image. The count() function is a counting function.

[0073] In step 5, the damaged film surface threshold value obtained in step 4 is used to classify the aerial image into normal film surface and damaged film surface, thereby achieving efficient and intelligent classification detection of normal film surface and damaged film surface.

[0074] like Figure 6 As shown in the figure, the number of pixels in the damaged area after segmentation is counted to determine the pixel threshold of the damaged area on the membrane surface. After statistical analysis, the feature residual range of the damaged area for five different damaged membrane surfaces is [0.0073, 0.0880]. Selecting the pixel threshold of 0.0073 can realize the pixel threshold of membrane surface damage based on the semi-supervised convolutional neural network model to automatically identify normal membrane surfaces and damaged membrane surfaces.

[0075] like Figure 1 and Figure 2As shown in the figure, the specific implementation process of the method for identifying different damaged membrane surface types of cable-membrane structures based on the YOLOv5s convolutional neural network model is as follows:

[0076] Step 1: Use the front-end camera of the UAV to take aerial images of the cable membrane structure membrane surface to obtain the membrane surface damage image of the cable membrane structure, and mark the damage type and damage location of the membrane surface damage defect image to build a damaged membrane surface database.

[0077] In the process of constructing a database of cable-membrane structure membrane surface damage, the types of defects in the images of cable-membrane structure damaged membrane surfaces include the following features: Based on the requirements for daily inspection and maintenance items of cable-membrane structure membrane surfaces in the Technical Specifications for Membrane Structures (CECS 158: 2015), the present invention summarizes the inspection and maintenance indicators of mildew, looseness, damage, wear, deformation, dirt, breakage, peeling and aging required by the specification into four types, namely: wrinkles, stains, scratches and tears. Since there will be obstructions in the membrane surface inspection process during daily use, the membrane surface damage dataset constructed by the present invention includes the damage type of obstructions. In summary, the cable-membrane structure membrane surface damage dataset constructed by the present invention has five types of labels, namely wrinkles, stains, scratches, tears and obstructions, with a total of 2,500 damaged membrane surface image data.

[0078] In the process of constructing the YOLOv5s convolutional neural network model, the input images and labels are normalized and preprocessed, and the input image size is uniformly scaled to an image resolution of 640 pixels × 640 pixels. The preprocessed images are then subjected to Mosaic image enhancement, and the image data is randomly scaled, randomly cropped, and randomly arranged for image splicing, which is used as the training set for the YOLOv5s convolutional neural network. Among them, the damaged membrane surface image database is enhanced by using Mosaic image enhancement technology, which mainly includes: (1) reading four damaged membrane surface images; (2) performing operations such as flipping, scaling, and color gamut conversion on the images. Image flipping is to flip the original damaged membrane surface image left and right, image scaling is to scale the original damaged membrane surface image, and color gamut conversion is to change the brightness, saturation, and hue of the original damaged membrane surface, thereby simulating the damaged membrane surface images of the actual cable membrane structure under different weather and time conditions; (3) the four transformed images are placed in different orientations and combined into one image, which is then put into the YOLOv5s model for training. The main purpose of using Mosaic image enhancement technology is to randomly crop four images and then splice them into a new damaged membrane surface image. The new damaged membrane surface image is used as training data to enrich the background of the damaged membrane surface of the cable-membrane structure. In addition, the data of four images can be calculated at one time during calculation. In this way, even if a smaller database is used, good training and recognition effects can still be obtained.

[0079] Step 2: Construct a YOLOv5s convolutional neural network model. The YOLOv5s convolutional neural network model is used to train the YOLOv5s convolutional neural network model based on 80% of the sample data of the marked cable-membrane structure membrane surface damage database. After the training is completed, the network model is tested based on the remaining 20% ​​sample data of the database.

[0080] In step 3, the YOLOv5s convolutional neural network model consists of five parts: input, backbone, neck, head, and output. The backbone, which primarily includes the Focus module, Conv module, C3 module, and SPP module, extracts rich semantic features from the input image. The neck uses FPN and PAN to generate a feature pyramid to enhance multi-scale object detection. The head predicts features transmitted from the neck and generates feature maps at three different scales.

[0081] Step 4: Train the membrane surface damage image recognition YOLOv5s convolutional neural network model to identify the damage characteristics and types of the cable membrane structure membrane surface image. Figure 7 This is a diagram showing the recognition effect of different damaged membrane surface types of cable-membrane structures based on the YOLOv5s convolutional neural network model. To evaluate the detection effect of the YOLOv5s model, precision (P), recall (R), and mean average precision (mAP@0.5) are used as evaluation indicators to measure the accuracy of damage type detection in the damage type detection task of membrane surface damage images. Figure 8 The figure shows the prediction accuracy of different damaged membrane surface types of cable-membrane structures based on the YOLOv5s convolutional neural network model. Figure 8 The Precision-Recall curve (PR curve) effectively reflects the trade-off between the YOLOv5s convolutional neural network model's recognition accuracy and coverage for positive examples of different damaged membrane surface types. The mean average precision (mAP@0.5) curve, which takes into account both recall and precision, takes the average AP value of all damaged membrane surface types and measures the average performance of the YOLOv5s convolutional neural network model across all damaged membrane surface types.

[0082] In step 5, based on the trained YOLOv5s convolutional neural network model, the aerial images of the cable-membrane structure surface captured by the drone are input to determine the damage type of the damaged membrane surface image. The output result is the damage type identification result of the damaged membrane surface image taken, realizing efficient and intelligent membrane surface damage type identification and detection.

[0083] The present invention not only solves the problems of slow speed and low efficiency in detecting artificial damage to cable-membrane structures, but also can realize automatic screening of damaged membrane surfaces and intelligent detection of five types of membrane surface damage. Aiming at the problems of single membrane surface damage background, slow speed of existing intelligent methods, and lack of dedicated databases and intelligent models, targeted improvement strategies are proposed, so that the cable-membrane structure damage identification and detection system of the present invention has the characteristics of high efficiency, convenience and low cost, has good applicability to cable-membrane structures, and can be widely used in engineering practice.

[0084] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto. Various changes that can be made within the scope of knowledge possessed by technicians in the relevant technical field without departing from the spirit of the present invention are all within the scope of protection of the claims of the present invention.

Claims

1. A cable-membrane structure membrane surface classification and recognition method based on pixel threshold difference, characterized in that: The steps include: Step 1: Use the front-end camera of the UAV to collect images of the normal membrane surface and the damaged membrane surface of the cable-membrane structure to obtain the collected initial image data of the membrane surface of the cable-membrane structure; Step 2: Classify the normal membrane surface and the damaged membrane surface using a classification method for the normal membrane surface and the damaged membrane surface of the cable-membrane structure based on pixel threshold difference, which specifically includes the following steps: Step 2-1: Use the normal membrane surface images collected by the front-end camera of the UAV to establish an initial image database of the cable membrane structure membrane surface, and perform batch preprocessing on the initial membrane surface image data in the initial image database to obtain membrane surface grayscale image data with an image resolution of 512 pixels × 512 pixels; Step 2-2: Construct a semi-supervised convolutional neural network model, which is only used to train the normal membrane surface data of the cable-membrane structure and learn to extract the features of the normal membrane surface image of the cable-membrane structure; wherein, the method for constructing the semi-supervised convolutional neural network model is as follows: construct a semi-supervised convolutional neural network model with 9 layers, wherein, in layers x1 to x5, the encoder uses a gradually decreasing convolution kernel from 11×11 to 3×3; the decoder structure has a convolution kernel size in the opposite order to that of the encoder, and uses a transposed convolution layer for feature calculation; the output of the encoder is connected to the previous layer x6 before the decoder is passed to layers x7 to x9; the network model is only provided with unlabeled normal membrane surface grayscale image data with a resolution of 512 pixels × 512 pixels for training, to ensure that the model convolution encoder only learns the normal membrane surface feature information, so that the network model only encodes and decodes the normal membrane surface, and obtains a trained semi-supervised convolutional neural network pre-training model; Step 2-3: Based on the semi-supervised convolutional neural network pre-trained model, encoding and decoding of the reconstructed lossless membrane surface are realized, specifically: unlabeled grayscale image data of the damaged membrane surface with a resolution of 512 pixels × 512 pixels is input into the trained normal membrane surface semi-supervised convolutional neural network pre-trained model to realize encoding and decoding of the reconstructed lossless membrane surface; Step 2-4: Input the aerial image data of the cable-membrane structure collected by the drone, calculate the pixel threshold difference with the reconstructed damage-free membrane surface, and determine the damaged membrane surface threshold. Specifically, the following steps are included: Step 2-4-1: Compare the input membrane surface damage image and the reconstructed membrane surface intact image, and perform pixel threshold difference between the two to obtain the feature residual of the damaged area of ​​the membrane surface damage, and realize the pixel segmentation of the damaged area of ​​the membrane surface damage image; Step 2-4-2: Count the number of pixels in the damaged area after segmentation, determine the pixel threshold of the damaged area on the membrane surface, and implement the semi-supervised convolutional neural network model to automatically identify the normal membrane surface and the damaged membrane surface based on the pixel threshold of the damaged membrane surface; the calculation formula for the pixel threshold difference between the input damaged membrane surface image and the reconstructed intact membrane surface image is: ， In the formula P t is the residual, X is the input film surface image data, It is the reconstructed lossless image data of the membrane surface output by the semi-supervised convolutional neural network; The statistical formula for the number of pixels in the damaged area after segmentation is: ， In the formula T r It is the ratio of the pixels in the damaged area after segmentation to the total pixels in the reconstructed lossless membrane surface image. Function is a counting function Step 2-5: using the damaged film surface threshold to classify the aerial image into normal film surface and damaged film surface, thereby achieving efficient and intelligent classification detection of normal film surface and damaged film surface; Step 3: Use the cable-membrane structure different damaged membrane surface types recognition method based on the YOLOv5s convolutional neural network model to identify damage on the damaged membrane surface image.

2. The cable-membrane structure membrane surface classification and identification method based on pixel threshold difference according to claim 1 is characterized in that: In step 3, the method for identifying different types of damaged membrane surfaces of cable-membrane structures based on the YOLOv5s convolutional neural network model includes the following steps: Step 3-1: Use the front-end camera of the UAV to take aerial images of the membrane surface of the cable-membrane structure to obtain damaged membrane surface images of the cable-membrane structure, mark the damage type and damage location of the damaged membrane surface images, and build a damaged membrane surface database; Step 3-2: constructing a YOLOv5s convolutional neural network model, wherein the YOLOv5s convolutional neural network model is only used to train the marked cable-membrane structure damaged membrane surface database; Step 3-3: The YOLOv5s convolutional neural network model structure includes an input end, a backbone network, a neck and a head, and an output end. The backbone network is used to extract rich image semantic features from the input image, the neck is used to enhance the detection of multi-scale objects, and the head is used to predict the features transferred from the neck and generate feature maps of three different scales. Step 3-4: training the YOLOv5s convolutional neural network model to identify damage features and types of the cable-membrane structure membrane surface image; Step 3-5: Based on the trained YOLOv5s convolutional neural network model, the aerial image of the membrane surface of the cable-membrane structure captured by the drone is input, and the damage type of the damaged membrane surface image is determined. The output result is the damage type recognition result of the damaged membrane surface image taken by the drone.

3. The cable-membrane structure membrane surface classification and identification method based on pixel threshold difference according to claim 2 is characterized in that: In steps 3-4, precision, recall and average precision are used as evaluation indicators to measure the accuracy of damage type detection in the damage type detection task of membrane surface damage images.

4. The cable-membrane structure membrane surface classification and identification method based on pixel threshold difference according to claim 1 is characterized in that: The encoding and decoding of the reconstructed lossless membrane surface is that the convolutional autoencoder of the semi-supervised convolutional neural network model adopts a convolution method to perform a linear transformation on the input membrane surface image data, and the corresponding weights are shared, and then the output reconstructed membrane surface image semantic vector is completed based on the linear combination of the basic image blocks of the hidden code; the reconstructed membrane surface image semantic vector is input into the decoder, and then converted into a three-dimensional vector, and the original information is amplified using a multi-layer deconvolution method to restore the reconstructed membrane surface image information.

5. The cable-membrane structure membrane surface classification and identification method based on pixel threshold difference according to claim 2 is characterized in that: In step 3-1, in the process of constructing the cable-membrane structure membrane surface damage database, it is determined that the cable-membrane structure membrane surface damage dataset labels include five types: wrinkles, stains, scratches, tears, and obstructions.

6. The cable-membrane structure membrane surface classification and identification method based on pixel threshold difference according to claim 2 is characterized by: In the process of building the YOLOv5s convolutional neural network model, the input images and labels are normalized and preprocessed, and the input image size is uniformly scaled to an image resolution of 640 pixels × 640 pixels. Mosaic image enhancement is performed on the preprocessed images, and image data is randomly scaled, randomly cropped, and randomly arranged to perform image splicing, which serves as the training set of the YOLOv5s convolutional neural network.

7. The cable-membrane structure membrane surface classification method and identification method based on pixel threshold difference according to claim 6 is characterized in that: The method for enhancing the damaged film surface image database by using the Mosaic image enhancement technology includes: Step 1: read four damaged film surface pictures; Step 2: Perform operations on the image including flipping, scaling, and color gamut conversion; Step 3: Place the four transformed images in different orientations and combine them into one image, which is then put into the YOLOv5s model for training.

Images