A method for rapid extraction and matching of cable-membrane structure damage identification based on knowledge base

Abstract

The present invention discloses a method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base, comprising the following steps: S1, establishing a cable-membrane structure damage knowledge base, and using a main feature extraction method to extract key feature information according to the damage type; S2, photographing a cable-membrane structure in service, using an industrial camera to reciprocate and photograph a local video of the cable-membrane structure in service, extracting a video key frame with a step length of 2 seconds, and using an image segmentation method to perform image segmentation on the key frame to obtain image information; S3, using a similarity matching method to match the key frame after the image segmentation in step S2 with the key feature information in step S1, and completing the damage identification of the cable-membrane structure in service. The present invention can reduce the storage space and information volume of the knowledge base, improve the matching efficiency during detection, and improve the efficiency and accuracy of intelligent detection of cable-membrane structures.

Description

Technical Field

[0001] The invention belongs to the technical field of full-life monitoring of membrane structure building materials, and in particular relates to a method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base. Background Art

[0002] Cable-membrane structure is a new type of building material in the field of civil engineering construction. It started late. Representative domestic buildings include the Water Cube, Shanghai 80,000-seat Gymnasium, Shanghai Hongkou Football Stadium, Qingdao Yizhong Stadium, Wuhu Olympic Sports Center, etc. It has the advantages of diverse artistic design, environmental protection and energy saving, good mechanical properties, strong self-cleaning ability, short construction period and good economy. It has become an important building structure. Related architectural sketches are integrated into all aspects of life, such as large sports stadiums, exhibition halls, aviation, railways, culture and entertainment, carports and other public places.

[0003] Although the development trend of cable-membrane structure buildings is gradually rising, compared with other structural buildings, its research in related fields such as health monitoring is still lagging behind. At present, the health diagnosis of cable-membrane structure buildings is completely dependent on manual regular inspections. However, most commonly used cable-membrane structure buildings are public resources, and manual inspections are affected by weather, mood, carelessness and other factors. Cable-membrane damage that is not discovered in time poses a major safety hazard. In addition, this type of structure is still in the emerging start-up stage, and there is a big performance difference from buildings with commonly used material structures. The professional knowledge level of the inspectors also greatly affects the accuracy of cable-membrane damage. Therefore, it is of great significance to study unmanned and intelligent cable-membrane structure damage detection methods.

[0004] Image recognition technology provides some ideas for intelligent health monitoring in this field. Patent authorization number CN213208934U provides a pole tower deformation detection device based on image recognition. The device judges whether the pole tower is deformed by taking fixed-point images every day and comparing them with the target position in the original image. Patent publication number CN113920075A provides a simple defect detection method and system based on object recognition. The method uses a large number of object databases and deep neural network training to obtain object recognition capabilities, and judges whether there are defects by comparing the recognized object with the complete object.

[0005] The above patent uses image recognition related methods to detect damage to other objects and material buildings. When used for the cable membrane structure damage detection and monitoring of the present invention, the following problems exist: 1. There are many types of damage to cable membrane structure buildings, including aging deformation, tearing, perforation, dirt, corrosion, etc. Different types of damage should be treated differently, and the damage cannot be considered from a single aspect; 2. Cable membrane structure buildings have diverse design styles, and the damage to their materials is also random. Different service states will produce different damages, and the same type of damage does not have a completely consistent appearance; 3. The use of deep learning methods has high hardware requirements and is time-consuming. The semantic segmentation method used also requires a long time of training. The membrane material structure belongs to an emerging material structure. Although the use of deep learning related methods may realize damage identification, it is time-consuming, and it is necessary to re-establish a database and train for different membrane structures and their damage types. The portability is poor, which is not conducive to the implementation of the project. Summary of the invention

[0006] Technical problem to be solved: In response to the above technical problem, the present invention provides a method for quickly extracting and matching to identify cable-membrane structure damage based on a knowledge base, which can effectively solve the shortcomings of the above image recognition technology when applied to cable-membrane structure health monitoring, such as excessively large database information, high hardware requirements, and long time consumption.

[0007] Technical solution: A method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base, comprising the following steps:

[0008] S1, establish a knowledge base of cable-membrane structure damage, and use the main feature extraction method to extract key feature information according to damage type;

[0009] S2, photographing the cable-membrane structure in service, using an industrial camera to reciprocately photograph a local video of the cable-membrane structure in service, extracting video key frames with a step length of 2 seconds, and using an image segmentation method to segment the key frames to obtain image information;

[0010] S3, using a similarity matching method to match the key frame after the image segmentation in step S2 with the key feature information in step S1, to complete the damage identification of the cable-membrane structure in service.

[0011] 2. According to claim 1, a method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base is characterized in that: the cable-membrane structure damage knowledge base described in step S1 is composed of a plurality of cable-membrane structure damage images, which are classified and stored according to the cable-membrane structure damage type, and each type of image is no less than 150, and the pixels of each image are 256×256.

[0012] 3. A method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to claim 2, characterized in that the cable-membrane structure damage types include tearing, perforation, deformation, corrosion, dirt and broken wires.

[0013] 4. The method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to claim 1, characterized in that the step of extracting key feature information in step S1 comprises the following steps:

[0014] S11, grayscale the color image in the cable-membrane structure damage knowledge base using the following formula to obtain a grayscale image:

[0015]

[0016] In the formula, Gray i,j is the gray value after conversion, with a value range of 0-255, i = 1, ..., 256, j = 1, ..., 256, R, G, B represent the values ​​of the red, green, and blue channels of the color image in the cable-membrane structure damage knowledge base, Gray 256×256 It is a 256×256 grayscale image matrix;

[0017] S12, the grayscale image matrix Gray obtained in step S11 256×256 Reset to 1 column and arrange the same type of damage images in columns to obtain the following formula:

[0018]

[0019] Where, Total is the overall matrix of grayscale image data corresponding to the same type of damage image in the cable-membrane structure damage knowledge base after reset and arranged in columns. The number of rows of Total is 65536, and the number of columns is the total number K of damage images of this type in the cable-membrane structure damage knowledge base;

[0020] S13, calculate the average gray value of each row of the overall matrix Total in step S12, and obtain:

[0021] GM 65536×1 =[gm 1 …gm 65536 ] T

[0022] In the formula, GM represents the average gray value matrix of Total obtained by row, which is a 65536×1 column matrix, gm is the average gray value of a row of Total, the subscript of gm is the row number, and T represents transposition;

[0023] Among them, the calculation formula of the average gray value gm is as follows:

[0024]

[0025] Where n represents the row number in Total corresponding to the average value, k represents the current image number, i.e., the column number corresponding to Total, i and j represent the row and column numbers of the grayscale image respectively, and N is an integer set;

[0026] S14, decentralize Total, and subtract the average gray value gm of the corresponding row from each element of the overall matrix Total, as shown in the following formula:

[0027]

[0028] Where U is the decentralized matrix after the overall matrix Total is decentralized by row;

[0029] S15, calculating the eigenvalues ​​and eigenvectors of the center-decentralized matrix U, and sorting the eigenvalues ​​from large to small, and retaining the eigenvectors according to the standard of 90% contribution rate of the main features;

[0030] S16, multiplying the retained eigenvector with the overall matrix Total to obtain the grayscale image overall matrix with reduced dimensionality;

[0031] S17, according to 1 column as 1 image, reversely restore the grayscale image to obtain a reduced-dimensional cable-membrane structure damage knowledge base;

[0032] S18, taking the ratio of the characteristic values ​​of the grayscale image inversely restored in step S17 as the image number, the ratio is:

[0033]

[0034] In the formula, β a is the proportion of the eigenvalues ​​of the a-th inverse restored grayscale image, M is the number of eigenvalues ​​retained in step S15, and λ a is the current eigenvalue, is the sum of all eigenvalues.

[0035] 5. A method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to claim 4, characterized in that: the main feature contribution rate in step S15 is that after calculating the eigenvalues ​​and eigenvectors of the decentralized matrix U, the sum of all its eigenvalues ​​is 1 according to the matrix theory idea, and 90% is used as the standard, the eigenvalues ​​are sorted from large to small, and the ratio is continuously accumulated until it exceeds 90%, and the several groups of eigenvalues ​​and their corresponding eigenvectors involved in the accumulation are retained, that is, the main features are retained.

[0036] 6. The method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to claim 4, characterized in that the image segmentation method in step S2 comprises the following steps:

[0037] S21, graying the key frame extracted from the video using the formula in S11 to obtain a grayscale image;

[0038] S22, using Gaussian filtering to filter the grayscale image and remove noise points in the image;

[0039] S23, using the canny operator to extract the edge of the damaged part image filtered in step S22, and completing the damaged part segmentation.

[0040] 7. The method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to claim 1, characterized in that the similarity matching method in step S3 comprises the following steps:

[0041] S31, randomly select 5 images from each damage type in the knowledge base;

[0042] S32, using a sequential similarity detection algorithm to match the key frames after image segmentation with the extracted images in step S31 one by one, and record the similarity μ;

[0043] S33, using the following formula to calculate the average value of the similarity between the current image and each of the extracted damage type images:

[0044]

[0045] In the formula, γ c is the average similarity of the current image to each type, subscript c is the type number of the corresponding damage type, β d represents the feature proportion of the feature image in the dth knowledge base, μ d Indicates the similarity of the d-th image;

[0046] S34, similarity average γ c Sort by the largest γ c The damage type is the damage type of the cable-membrane structure currently detected.

[0047] Beneficial effects: (1) The present invention can effectively and quickly extract the main features of different types of damage knowledge bases, reduce the amount of useless data stored in the cable-membrane structure damage knowledge base, retain the main features, reduce the knowledge base storage space and information volume, and improve the matching efficiency during detection.

[0048] (2) The present invention makes full use of the background uniformity of the membrane material structure, uses a simple and mature calculation method, and quickly determines the damage type of the current detected membrane surface through weighted similarity, thereby improving the efficiency and accuracy of intelligent detection of cable membrane structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of a method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of a method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of reciprocating shooting of industrial cameras;

[0052] Figure 4 The present invention proposes a sample extraction result of a method for quickly extracting and matching and identifying damage of cable-membrane structures based on a knowledge base. DETAILED DESCRIPTION

[0053] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0054] Example 1

[0055] like Figure 1-4 As shown, a method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base includes the following steps:

[0056] Step S1, establishing a cable-membrane structure damage knowledge base, and using a main feature extraction method to extract key feature information according to damage type;

[0057] The cable-membrane structure damage knowledge base is composed of a number of cable-membrane structure damage images. The cable-membrane structure damage knowledge base is a reference for judging the type of cable-membrane structure damage in detection. The images in the cable-membrane structure damage knowledge base are classified and stored according to the cable-membrane damage type, including: tearing, perforation, deformation, corrosion, dirt, broken wire, etc.; in order to ensure that the feature extraction has a certain robustness, each type of image is not less than 150; in order to take into account the amount of image information and computational efficiency, the pixels of each image are 256×256, and all image libraries together constitute the cable-membrane structure damage knowledge base.

[0058] The above-mentioned extraction of key feature information includes the following steps:

[0059] Step S11, grayscale the color image in the cable-membrane structure damage knowledge base using the following formula to obtain a grayscale image:

[0060]

[0061] In the formula, Gray i,j is the gray value after conversion, with a value range of 0-255, i = 1, ..., 256, j = 1, ..., 256, R, G, B represent the values ​​of the red, green, and blue channels of the color image in the cable-membrane structure damage knowledge base, Gray 256×256It is a 256×256 grayscale image matrix.

[0062] Step S12: convert the grayscale image matrix Gray obtained in step S11 256×256 Reset to 1 column and arrange the same type of damage images in columns to obtain the following formula:

[0063]

[0064] Where Total is the overall matrix of grayscale image data corresponding to the same type of damage image in the cable-membrane structure damage knowledge base after reset and arranged in columns. The number of rows of Total is 65536, and the number of columns is the total number K of damage images of this type in the cable-membrane structure damage knowledge base.

[0065] Step S13, calculate the average gray value of each row of the overall matrix Total in step S12, and obtain:

[0066] GM 65536×1 =[gm 1 …gm 65536 ] T

[0067] In the formula, GM represents the average gray value matrix of Total obtained by row, which is a 65536×1 column matrix, gm is the average gray value of a row of Total, the subscript of gm is the row number, and T represents transposition;

[0068] Among them, the calculation formula of the average gray value gm is as follows:

[0069]

[0070] Where n represents the row number in Total corresponding to the average value, k represents the current image number, i.e., the column number corresponding to Total, i and j represent the row and column numbers of the grayscale image respectively, and N is an integer set;

[0071] S14, decentralize Total, and subtract the average gray value gm of the corresponding row from each element of the overall matrix Total, as shown in the following formula:

[0072]

[0073] Where U is the decentralized matrix after the overall matrix Total is decentralized by row.

[0074] Step S15, calculating the eigenvalues ​​and eigenvectors of the center-decentralized matrix U, and sorting the eigenvalues ​​from large to small, and retaining the eigenvectors according to the standard of 90% main feature contribution rate;

[0075] The contribution rate of the above main features is: after calculating the eigenvalues ​​and eigenvectors of the decentralized matrix U, according to the idea of ​​matrix theory, the sum of all its eigenvalues ​​is 1, but the main features are several groups of image data with larger eigenvalues. Therefore, based on 90% as the standard, the eigenvalues ​​are sorted from large to small, and the ratio is continuously accumulated until it exceeds 90%, and the several groups of eigenvalues ​​and their corresponding eigenvectors involved in the accumulation are retained, that is, the main features are saved.

[0076] Step S16, multiplying the retained eigenvector with the overall matrix Total to obtain the reduced-dimensional grayscale image overall matrix.

[0077] Step S17, inversely restore the grayscale image according to one column as one image, and obtain a cable-membrane structure damage knowledge base after dimensionality reduction. The cable-membrane structure damage knowledge base after dimensionality reduction reduces the number of images and extracts the main feature information of the same damage type.

[0078] Step S18, taking the ratio of the characteristic values ​​of the grayscale image inversely restored in step S17 as the image number, the ratio is:

[0079]

[0080] In the formula, β a is the proportion of the eigenvalues ​​of the a-th inverse restored grayscale image, M is the number of eigenvalues ​​retained in step S15, and λ a is the current eigenvalue, is the sum of all eigenvalues.

[0081] Step S2, photographing the cable-membrane structure in service, using an industrial camera to reciprocately photograph a local video of the cable-membrane structure in service, extracting video key frames with a step length of 2 seconds, and performing image segmentation on the key frames using an image segmentation method to obtain image information; the image segmentation method comprises the following steps:

[0082] S21, graying the key frame extracted from the video using the formula in S11 to obtain a grayscale image;

[0083] S22, using Gaussian filtering to filter the grayscale image and remove noise points in the image;

[0084] S23, using the canny operator to extract the edge of the damaged part image filtered in step S22, and completing the damaged part segmentation.

[0085] Step S3, using a similarity matching method to match the key frame after the image segmentation in step S2 with the key feature information in step S1, to complete the damage identification of the cable-membrane structure in service;

[0086] The similarity matching method comprises the following steps:

[0087] S31, randomly select 5 images from each damage type in the knowledge base;

[0088] S32, using a sequential similarity detection algorithm to match the key frames after image segmentation with the extracted images in step S31 one by one, and record the similarity μ;

[0089] S33, using the following formula to calculate the average value of the similarity between the current image and each of the extracted damage type images:

[0090]

[0091] In the formula, γ c is the average similarity of the current image to each type, subscript c is the type number of the corresponding damage type, β d represents the feature proportion of the feature image in the dth knowledge base, μ d Indicates the similarity of the d-th image;

[0092] S34, similarity average γ c Sort by the largest γ c The damage type is the damage type of the cable-membrane structure currently detected.

[0093] In summary, the present invention can effectively and quickly extract the main features of different types of damage knowledge bases, reduce the amount of useless data stored in the cable-membrane structure damage knowledge base, retain the main features, reduce the knowledge base storage space and information volume, and improve the matching efficiency during detection. In addition, the background singleness of the membrane material structure is fully utilized, and a simple and mature calculation method is used to quickly determine the damage type of the current detection membrane surface through weighted similarity, thereby improving the efficiency and accuracy of intelligent detection of cable-membrane structures.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base, characterized in that: The following steps are involved: S1, establishing a cable-membrane structure damage knowledge base, and using a main feature extraction method to extract key feature information according to damage type, wherein extracting key feature information includes the following steps: S11, grayscale the color image in the cable-membrane structure damage knowledge base using the following formula to obtain a grayscale image: In the formula, Gray i,j is the gray value after conversion, with a value range of 0-255, i = 1, ..., 256, j = 1, ..., 256, R, G, B represent the values ​​of the red, green, and blue channels of the color image in the cable-membrane structure damage knowledge base, Gray 256×256 It is a 256×256 grayscale image matrix; S12, the grayscale image matrix Gray obtained in step S11 256×256 Reset to 1 column and arrange the same type of damage images in columns to obtain the following formula: Where, Total is the overall matrix of grayscale image data corresponding to the same type of damage image in the cable-membrane structure damage knowledge base after reset and arranged in columns. The number of rows of Total is 65536, and the number of columns is the total number K of damage images of this type in the cable-membrane structure damage knowledge base; S13, calculate the average gray value of each row of the overall matrix Total in step S12, and obtain: GM 65536×1 =[gm1 … gm 65536 ] T In the formula, GM represents the average gray value matrix of Total obtained by row, which is a 65536×1 column matrix, gm is the average gray value of a row of Total, the subscript of gm is the row number, and T represents transposition; Among them, the calculation formula of the average gray value gm is as follows: Where n represents the row number in Total corresponding to the average value, k represents the current image number, i.e., the column number corresponding to Total, i and j represent the row and column numbers of the grayscale image respectively, and N is an integer set; S14, decentralize Total, and subtract the average gray value gm of the corresponding row from each element of the overall matrix Total, as shown in the following formula: Where U is the decentralized matrix after the overall matrix Total is decentralized by row; S15, calculating the eigenvalues ​​and eigenvectors of the center-decentralized matrix U, and sorting the eigenvalues ​​from large to small, and retaining the eigenvectors according to the standard of 90% contribution rate of the main features; S16, multiplying the retained eigenvector with the overall matrix Total to obtain the grayscale image overall matrix with reduced dimensionality; S17, according to 1 column as 1 image, reversely restore the grayscale image to obtain a reduced-dimensional cable-membrane structure damage knowledge base; S18, taking the ratio of the characteristic values ​​of the grayscale image inversely restored in step S17 as the image number, the ratio is: In the formula, β a is the proportion of the eigenvalues ​​of the ath inverse restored grayscale image, M is the number of eigenvalues ​​retained in step S15, and λ a is the current eigenvalue, is the sum of all eigenvalues; S2, photographing the cable-membrane structure in service, using an industrial camera to reciprocately photograph a local video of the cable-membrane structure in service, extracting video key frames with a step length of 2 seconds, and using an image segmentation method to segment the key frames to obtain image information; S3, using a similarity matching method to match the key frame after the image segmentation in step S2 with the key feature information in step S1, to complete the damage identification of the cable-membrane structure in service.

2. According to claim 1, a method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base is characterized by: The cable-membrane structure damage knowledge base described in step S1 is composed of a number of cable-membrane structure damage images, which are classified and stored according to the cable-membrane structure damage types. There are no less than 150 images of each type, and the pixels of each image are 256×256.

3. The method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to claim 2, characterized in that: The cable membrane structure damage types include tearing, perforation, deformation, corrosion, dirt and broken wires.

4. The method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to claim 1, characterized in that: The main feature contribution rate described in step S15 is that after calculating the eigenvalues ​​and eigenvectors of the decentralized matrix U, according to the idea of ​​matrix theory, the sum of all its eigenvalues ​​is 1. Taking 90% as the standard, the eigenvalues ​​are sorted from large to small, and the ratio is continuously accumulated until it exceeds 90%, and the several groups of eigenvalues ​​and their corresponding eigenvectors involved in the accumulation are retained, that is, the main features are retained.

5. The method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to claim 1 is characterized in that: The image segmentation method in step S2 comprises the following steps: S21, graying the key frame extracted from the video using the formula in S11 to obtain a grayscale image; S22, using Gaussian filtering to filter the grayscale image and remove noise points in the image; S23, using the canny operator to extract the edge of the damaged part image filtered in step S22, and completing the damaged part segmentation.

6. The method for quickly extracting and matching and identifying cable-membrane structure damage based on a knowledge base according to claim 1 is characterized in that: The similarity matching method in step S3 comprises the following steps: S31, randomly select 5 images from each damage type in the knowledge base; S32, using a sequential similarity detection algorithm to match the key frames after image segmentation with the extracted images in step S31 one by one, and record the similarity μ; S33, using the following formula to calculate the average value of the similarity between the current image and each of the extracted damage type images: In the formula, γ c is the average similarity of the current image to each type, subscript c is the type number of the corresponding damage type, β d represents the feature proportion of the feature image in the dth knowledge base, μ d Indicates the similarity of the d-th image; S34, similarity average γ c Sort by the largest γ c The damage type is the damage type of the cable-membrane structure currently detected.

Images