Image recognition method for damaged position of tower and mast structure

Abstract

The present application discloses a method for identifying damaged positions of tower mast structures, which uses computer vision technology based on deep learning to analyze and process tower mast structure images, captures global semantic features of tower mast structure images, and captures local position areas from tower mast structure images, thereby identifying damaged positions based on the correlation matching between tower mast structure features of the local position areas and global semantic features of tower mast structure images. In this way, the accuracy and efficiency of damaged position identification can be improved, while reducing the reliance on manual inspections, and reducing detection costs and risks.

Description

Technical Field

[0001] The present application relates to the field of intelligent identification, and more specifically, to an image identification method for damaged locations of tower mast structures. Background Art

[0002] As a tall building structure, masts are widely used in the fields of electricity, communication, broadcasting, traffic monitoring, etc., and they play an important supporting and transmission role. With the rapid development of communication technology, masts are important supporting facilities for communication base stations, and their safety and stability have a great impact on the normal operation of communication networks. However, due to the long-term exposure of masts to the natural environment, they are affected by wind loads, temperature changes, corrosion and other factors. The mast structure is prone to breakage or damage. If it is not discovered and repaired in time, it may cause serious safety accidents or even lead to the failure of the entire structure.

[0003] Traditional methods of detecting damage to tower mast structures mainly rely on manual inspections, which is not only time-consuming and labor-intensive, but also inefficient, and the accuracy of identification is also affected by the experience and subjective judgment of the inspectors. In addition, tower masts are often located in remote areas or high altitudes, which further increases the difficulty and risk of manual inspections.

[0004] At present, with the rapid development of computer vision technology, tower mast structure damage detection methods based on image recognition have gradually become a research hotspot. However, some existing tower mast structure damage detection methods usually use traditional image processing algorithms, such as edge detection and threshold segmentation. These methods are easily affected by factors such as lighting changes, shadows, and noise, resulting in false detection or missed detection. For example, the edge detection algorithm may misjudge the shadows or textures of non-damaged areas as damaged edges, or ignore some damaged areas with low contrast with the background. In addition, traditional image processing algorithms usually require manual setting of threshold parameters and lack adaptability, which limits the improvement of detection efficiency and automation level.

[0005] Therefore, an optimized image recognition method for damaged locations of tower and mast structures is desired. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a tower mast structure damaged position image recognition method, which analyzes and processes the tower mast structure image by using computer vision technology based on deep learning, captures the global semantic features of the tower mast structure image, and intercepts the local position area from the tower mast structure image, thereby realizing the recognition of the damaged position based on the correlation matching between the tower mast structure features of the local position area and the global semantic features of the tower mast structure image. In this way, the accuracy and efficiency of the damaged position recognition can be improved, while reducing the reliance on manual inspections and reducing the detection costs and risks.

[0007] According to one aspect of the present application, a method for image recognition of damaged positions of a tower mast structure is provided, comprising:

[0008] Acquire a tower mast structure image captured by a camera;

[0009] Extracting and dispersing features of the surface state of the tower mast structure from the tower mast structure image to obtain a sequence of local feature vectors of the surface state of the tower mast structure;

[0010] intercepting a first candidate position image to be identified from the tower mast structure image;

[0011] Using the surface state feature of the tower mast structure in the first candidate position image to be identified as a query feature, and performing one-way interactive matching with the sequence of local feature vectors of the surface state of the tower mast structure to obtain a one-way matching feature vector;

[0012] Based on the one-way matching feature vector, it is determined whether the first candidate position to be identified is a damaged position.

[0013] Compared with the prior art, the present application provides a method for identifying damaged position images of a tower mast structure, which uses computer vision technology based on deep learning to analyze and process the tower mast structure image, captures the global semantic features of the tower mast structure image, and intercepts the local position area from the tower mast structure image, thereby realizing the identification of the damaged position based on the correlation matching between the tower mast structure features of the local position area and the global semantic features of the tower mast structure image. In this way, the accuracy and efficiency of damaged position identification can be improved, while reducing the reliance on manual inspections, and reducing detection costs and risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0015] Figure 1 is a flow chart of a method for image recognition of damaged positions of a tower mast structure according to an embodiment of the present application;

[0016] Figure 2 A system architecture diagram of a tower mast structure damage position image recognition method according to an embodiment of the present application;

[0017] Figure 3 It is a flowchart of sub-step S2 of the tower mast structure damage position image recognition method according to an embodiment of the present application;

[0018] Figure 4 It is a flowchart of sub-step S4 of the method for image recognition of damaged positions of tower mast structures according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0020] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0021] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0022] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0024] Traditional tower and mast structure damage detection methods mainly rely on manual inspections, which is not only time-consuming and labor-intensive, but also inefficient, and the recognition accuracy is also affected by the experience and subjective judgment of the inspectors. In addition, towers and masts are often located in remote areas or high altitudes, which further increases the difficulty and risk of manual detection. At present, with the rapid development of computer vision technology, tower and mast structure damage detection methods based on image recognition have gradually become a hot topic of research. However, some existing tower and mast structure damage detection methods usually use traditional image processing algorithms, such as edge detection and threshold segmentation, which are easily affected by factors such as illumination changes, shadows, and noise, resulting in false detection or missed detection. For example, the edge detection algorithm may misjudge the shadow or texture of the non-damaged area as a damaged edge, or ignore some damaged areas with low contrast with the background. In addition, traditional image processing algorithms usually require manual setting of threshold parameters, lack of adaptability, and limit the improvement of detection efficiency and automation level. Therefore, an optimized image recognition method for damaged position of tower and mast structure is expected.

[0025] In the technical solution of the present application, a method for image recognition of damaged positions of a tower mast structure is proposed. Figure 1 Flow chart of a method for image recognition of damaged positions of a tower mast structure according to an embodiment of the present application. Figure 2 : is a system architecture diagram of a tower mast structure damage location image recognition method according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the method for image recognition of damaged positions of tower mast structures includes the following steps: S1, acquiring a tower mast structure image captured by a camera; S2, performing tower mast structure surface state feature extraction and feature dispersion processing on the tower mast structure image to obtain a sequence of local feature vectors of the tower mast structure surface state; S3, intercepting a first candidate position image to be identified from the tower mast structure image; S4, using the tower mast structure surface state feature in the first candidate position image to be identified as a query feature, and performing one-way interactive matching with the sequence of local feature vectors of the tower mast structure surface state to obtain a one-way matching feature vector; S5, determining whether the first candidate position to be identified is a damaged position based on the one-way matching feature vector.

[0026] In particular, the S1, obtains the tower mast structure image captured by the camera. It should be understood that in the tower mast structure damage detection, for areas that are far away or difficult to reach directly, using a high-definition camera to capture the tower mast structure image, so as to remotely identify the damaged location, is an effective technical means. Among them, the tower mast structure image captured by the camera contains the overall appearance and detailed information of the tower mast structure, which provides an important image basis for the identification of the damaged location of the tower mast structure. By analyzing the tower mast structure image, it is helpful to timely discover potential safety hazards.

[0027] In particular, the step S2 extracts and disperses the surface state features of the tower mast structure image to obtain a sequence of local feature vectors of the surface state of the tower mast structure. In particular, in a specific example of the present application, Figure 3 As shown, the S2 includes: S21, performing boundary information compensated global feature extraction on the tower mast structure image to obtain a global tower mast structure surface state feature map; S22, performing local feature dispersion processing on the global tower mast structure surface state feature map along the channel dimension to obtain a sequence of local feature vectors of the tower mast structure surface state.

[0028] Specifically, the S21 performs boundary information compensation global feature extraction on the tower mast structure image to obtain a global tower mast structure surface state feature map. In a specific example of the present application, the tower mast structure image is input into a dual-branch boundary information compensation network including a backbone network and a boundary feature extraction branch to obtain the global tower mast structure surface state feature map. The backbone network in the dual-branch boundary information compensation network includes a continuous convolution layer, a first activation layer based on a ReLU activation function, and a pooling layer, wherein the convolution kernel size of the continuous convolution layer is 3x3, the step size is 1, and the padding is 1, and the pooling kernel size of the pooling layer is 2x2, and the step size is 2; and the boundary feature extraction branch in the dual-branch boundary information compensation network includes a point convolution layer, a second activation layer based on a ReLU activation function, and an upsampling layer, wherein the convolution kernel size of the point convolution layer is 1x1, the step size is 1, and the padding is 0. It should be understood that, in the process of feature extraction of the tower mast structure image, as the number of network layers increases, the spatial detail information of the image will gradually be lost, causing the boundary information of the damaged location to become blurred, thereby affecting the recognition accuracy of the damaged location. Therefore, in the technical solution of the present application, a dual-branch boundary information compensation network including a backbone network and a boundary feature extraction branch is used to process the tower mast structure image to obtain a global tower mast structure surface state feature map containing rich boundary information. Among them, the backbone network of the dual-branch boundary information compensation network is mainly used to extract the global semantic features of the tower mast structure image, while the boundary feature extraction branch is used to capture the boundary information of the tower mast structure image. Furthermore, by fusing the features of the two to form a comprehensive description of the tower mast structure, the accuracy of damaged location recognition is improved.

[0029] Specifically, in S22, the global tower mast structure surface state feature map is processed by local feature dispersion along the channel dimension to obtain a sequence of local feature vectors of the tower mast structure surface state. It should be understood that the damaged position of the tower mast structure often corresponds to a specific local area in the global tower mast structure surface state feature map. Therefore, in order to locate the specific location of the damage more accurately, the global tower mast structure surface state feature map is further processed by local feature dispersion along the channel dimension to obtain a sequence of local feature vectors of the tower mast structure surface state corresponding to different local areas in the tower mast structure image, so as to analyze the surface state of the tower mast structure more finely, thereby improving the recognition accuracy of the damaged position. It is worth mentioning that local feature dispersion refers to dividing the continuous local feature space into discrete areas or barrels so as to more effectively represent and process the local features. The local feature dispersion process maps continuous feature values ​​to a set of discrete values, which is usually achieved by setting a threshold or using a clustering method. In the local feature dispersion process, the continuous feature space is divided into several barrels or areas, each barrel representing a discrete value range. Local feature discretization is a technique that converts continuous features into discrete features. This conversion can simplify problems, improve efficiency and robustness, and provide better feature representation for the training and reasoning of machine learning models.

[0030] It is worth mentioning that in other specific examples of the present application, the tower mast structure image can also be subjected to tower mast structure surface state feature extraction and feature dispersion processing in other ways to obtain a sequence of local feature vectors of the tower mast structure surface state, for example: inputting the tower mast structure image; preprocessing the tower mast structure image, including denoising, edge detection, image enhancement and other operations, to perform feature extraction; using appropriate feature extraction technology to extract surface state features from the tower mast structure image; dispersing continuous surface state features, and using a clustering algorithm (such as K-means clustering) or manually setting a threshold to divide the continuous features into different discrete categories or intervals; converting each local surface state feature into a local feature vector; each local feature vector may contain a dispersed feature value, representing the state of the local area; organizing all local feature vectors into a sequence according to their position in the image or other rules to obtain a sequence of local feature vectors of the tower mast structure surface state.

[0031] In particular, the S3 intercepts the first candidate position image to be identified from the tower mast structure image. In the technical solution of the present application, in order to further narrow the scope and identify the specific location of the damage, in the technical solution of the present application, the first candidate position image to be identified is intercepted from the original tower mast structure image, so that it can be determined whether the first candidate position to be identified is a damaged position based on the correlation matching between the tower mast structure state features in the intercepted candidate position image to be identified and the tower mast structure surface state features of each local area in the global image of the tower mast structure.

[0032] In particular, in S4, the surface state feature of the tower mast structure in the first candidate position image to be identified is used as a query feature, and a sequence of local feature vectors of the surface state of the tower mast structure is subjected to one-way interactive matching to obtain a one-way matching feature vector. In particular, in a specific example of the present application, Figure 4 As shown, the S4 includes: S41, extracting the surface state features of the tower mast structure of the first candidate position to be identified image to obtain a first candidate position to be identified state feature vector; S42, using the first candidate position to be identified state feature vector as a query feature vector, and inputting the query feature vector and the sequence of the tower mast structure surface state local feature vector into a one-way interactive matching network to obtain the one-way matching feature vector.

[0033] Specifically, the S41 extracts the surface state features of the tower mast structure of the first candidate position image to be identified to obtain the first candidate position state feature vector. In particular, in a specific example of the present application, the first candidate position image to be identified is input into a surface state feature extractor based on a dilated convolutional neural network model to obtain the first candidate position state feature vector. That is, the first candidate position image to be identified is input into a surface state feature extractor based on a dilated convolutional neural network model for image feature extraction, and the unique dilated convolution structure of the dilated convolutional neural network model is used to capture the detail information in the image with high computational efficiency, and effectively extract the surface state features of the tower mast structure in the first candidate position image to be identified to obtain the first candidate position state feature vector. It is worth mentioning that the dilated convolutional neural network is a convolutional neural network structure that increases the receptive field by introducing spacing (or "expansion rate") between convolution kernels. This technology allows the network to effectively expand the receptive field while keeping the number of parameters relatively small, thereby better capturing the global information in the image, and is particularly suitable for processing tasks with a wide range of contextual information requirements. The following are the core concepts and features of the dilated convolutional neural network: The role of dilated convolution: dilated convolution introduces a parameter called "dilation rate" to control the spacing inside the convolution kernel; by increasing the dilation rate, the effective receptive field of the convolution kernel can be increased while keeping the size of the convolution kernel unchanged, thereby capturing a wider range of contextual information; Expanding the receptive field: The dilated convolutional network can expand the receptive field without increasing the number of parameters, allowing the network to better understand the global structure and contextual information in the image. This is very useful for many computer vision tasks such as semantic segmentation, object detection, and image classification; Maintaining resolution: The dilated convolution can maintain the spatial resolution of the input image while expanding the receptive field, avoiding the information loss problem caused by downsampling the image in traditional convolution.

[0034] Specifically, in S42, the state feature vector of the first candidate position to be identified is used as a query feature vector, and the sequence of the query feature vector and the local feature vector of the surface state of the tower mast structure is input into a one-way interactive matching network to obtain the one-way matching feature vector. In the technical solution of the present application, a feature matching analysis is performed on the first candidate position to be identified, and the state feature vector of the first candidate position to be identified is used as a query feature vector, and the sequence of the query feature vector and the local feature vector of the surface state of the tower mast structure is input into a one-way interactive matching network to obtain a one-way matching feature vector. Among them, the one-way interactive matching network calculates the similarity between the query feature vector and the sequence of local feature vectors of the surface state of the tower mast structure to determine the degree of matching between the image features of the first candidate position to be identified and the features of each local area in the global image, and thereby dynamically assigns weights to the sequence of local feature vectors of the surface state of the tower mast structure, performs weighted fusion on the features of each local area, and calculates the differential features between the state features of the first candidate position to be identified and the fused features of each local area to remove similar features, thereby obtaining a one-way matching feature vector with high discrimination, which is used to determine whether the first candidate position to be identified is a damaged position.

[0035] In an embodiment of the present application, the query feature vector and the sequence of local feature vectors of the tower mast structure surface state are input into a one-way interactive matching network to obtain the one-way matching feature vector, including: calculating the semantic measurement coefficients between the query feature vector and each local feature vector of the tower mast structure surface state in the sequence of local feature vectors of the tower mast structure surface state to obtain a sequence of one-way matching semantic measurement coefficients; normalizing the sequence of one-way matching semantic measurement coefficients through a softmax function to obtain a sequence of one-way matching semantic measurement weights; calculating the weighted sum of the sequence of local feature vectors of the tower mast structure surface state using the sequence of one-way matching semantic measurement weights as weights to obtain a one-way matching interactive reshaping feature vector; and calculating the differential feature vector between the query feature vector and the one-way matching interactive reshaping feature vector as the one-way matching feature vector.

[0036] Furthermore, the process of calculating the semantic metric coefficients between the query feature vector and each tower mast structure surface state local feature vector in the sequence of tower mast structure surface state local feature vectors to obtain a sequence of one-way matching semantic metric coefficients includes: calculating the square of each eigenvalue of each tower mast structure surface state local feature vector in the sequence of tower mast structure surface state local feature vectors to obtain a sequence of nonlinear transformation tower mast structure surface state local feature vectors; cascading the query feature vector with each nonlinear transformation tower mast structure surface state local feature vector in the sequence of nonlinear transformation tower mast structure surface state local feature vectors to obtain a sequence of semantic association feature vectors; multiplying each semantic association feature vector in the sequence of semantic association feature vectors using a predetermined weight vector to obtain a sequence of semantic similarities; adding a predetermined bias parameter to each semantic similarity in the sequence of semantic similarities and then passing through a sigmoid activation function to obtain a sequence of one-way matching semantic metric coefficients.

[0037] In summary, in the above embodiment, the query feature vector and the sequence of the local feature vector of the surface state of the tower mast structure are input into a one-way interactive matching network to obtain the one-way matching feature vector, including: processing the query feature vector and the sequence of the local feature vector of the surface state of the tower mast structure with the following one-way interactive matching formula to obtain the one-way matching feature vector, wherein the one-way interactive matching formula is:

[0038]

[0039] Wherein, v1 represents the query feature vector, v j represents the jth local feature vector of the surface state of the tower mast structure in the sequence of local feature vectors of the surface state of the tower mast structure, [·,·] represents a cascade operation, W is a predetermined weight vector, b is a predetermined bias parameter, σ represents a Sigmoid activation function, r 1j represents the semantic similarity between the query feature vector and the j-th local feature vector of the surface state of the tower mast structure, m is the number of local feature vectors of the surface state of the tower mast structure, exp(·) represents the exponential function operation with e as the base, e 1j represents the one-way matching semantic metric coefficient between the query feature vector and the j-th tower mast structure surface state local feature vector, v 1p represents the one-way matching interaction reshaping feature vector, Indicates subtraction by position, v m Represents the one-way matching feature vector.

[0040] It is worth mentioning that in other specific examples of the present application, other methods can be used to use the surface state features of the tower mast structure in the first candidate position image to be identified as query features, and perform one-way interactive matching with the sequence of local feature vectors of the tower mast structure surface state to obtain a one-way matching feature vector, for example: input the first candidate position image to be identified and the sequence of local feature vectors of the tower mast structure surface state; extract the surface state features of the tower mast structure from the first candidate position image to be identified as query features; preprocess and extract features of the query features to obtain a feature vector describing the surface state of the image; perform one-way interactive matching between the query features and the sequence of local feature vectors of the tower mast structure surface state; for each local feature vector, a similarity measurement method such as cosine similarity, Euclidean distance, etc. can be used to measure its similarity with the query feature; for each local feature vector, calculate its similarity score with the query feature; combine these similarity scores into a one-way matching feature vector, in which each element represents the similarity between the corresponding local feature vector and the query feature; to obtain the one-way matching feature vector.

[0041] In particular, the S5 determines whether the first candidate position to be identified is a damaged position based on the one-way matching feature vector. In particular, in a specific example of the present application, the one-way matching feature vector is input into a classifier-based image recognizer to obtain a recognition result, and the recognition result is used to indicate whether the first candidate position to be identified is a damaged position. That is, a classification algorithm is used to perform binary classification on the one-way matching feature vector to determine whether the first candidate position to be identified is a damaged position. If the recognition result indicates that the first candidate position to be identified is a damaged position, the position can be immediately marked as an area that needs maintenance, and the information can be fed back to the monitoring system in real time, so that maintenance measures can be taken in time to prevent safety accidents.

[0042] In the technical solution of the present application, the first candidate position state feature vector to be identified represents the image semantic features of the first candidate position image to be identified captured from the tower mast structure image, and the sequence of the tower mast structure surface state local feature vectors represents a sequence set of image local semantic features of the global image of the tower mast structure image based on a dual-branch boundary information compensation network including a backbone network and a boundary feature extraction branch. In the process of using the first candidate position state feature vector to be identified as a query feature vector and inputting the query feature vector and the sequence of the tower mast structure surface state local feature vector into a one-way interactive matching network, the first candidate position state feature vector to be identified is used as a query unit and the sequence of the tower mast structure surface state local feature vector is used as a feature unit database to perform one-way interactive global matching to obtain a feature matching result, namely, the one-way matching feature vector. However, considering that the encoding methods of the image semantic features of the first alternative position state feature vector to be identified and the global tower mast structure surface state feature map are different, the feature distribution difference caused by the encoding difference will cause the feature semantic interaction weight distribution difference, and the global feature distribution of the one-way matching feature vector will have the problem of insufficient feature distribution aggregation, thereby affecting its classification convergence efficiency through the classifier, that is, affecting the efficiency of classification training and the accuracy of classification results.

[0043] Preferably, inputting the one-way matching feature vector into a classifier-based image recognizer to obtain a recognition result comprises:

[0044] After multiplying each eigenvalue of the one-way matching feature vector by the length of the one-way matching feature vector, subtracting a norm of the one-way matching feature vector, and calculating the square root of the absolute value of the subtraction result to obtain a first one-way matching intermediate vector;

[0045] After multiplying each eigenvalue of the one-way matching feature vector by the square root of the length of the one-way matching feature vector, subtracting the bi-norm of the one-way matching feature vector, and calculating the square root of the absolute value of the subtraction result to obtain a second one-way matching intermediate vector;

[0046] Respectively calculating the logarithmic value with base 2 of each eigenvalue of the first one-way matching intermediate vector and the second one-way matching intermediate vector to obtain a first one-way matching intermediate information vector and a second one-way matching intermediate information vector;

[0047] Calculating a weighted sum of the first one-way matching intermediate information vector and the second one-way matching intermediate information vector to obtain a corrected one-way matching feature vector; and

[0048] The corrected one-way matching feature vector is input into a classifier-based image recognizer to obtain a recognition result.

[0049] That is, in this preferred embodiment, the step of correcting the characteristic distribution of the one-way matching characteristic vector is expressed by the formula:

[0050]

[0051] Wherein, V represents the one-way matching feature vector, ‖V‖1 represents the first norm of the one-way matching feature vector, ||V||2 represents the second norm of the one-way matching feature vector, L represents the length of the one-way matching feature vector, |·| ⊙1 / 2 represents the square root of the absolute value of each eigenvalue of the eigenvector, log represents the logarithmic function operation with base 2, ω is the balance hyperparameter, ⊙ represents the dot product, Indicates point reduction, represents point addition, and V' represents the corrected one-way matching feature vector.

[0052] Accordingly, in this preferred embodiment, the applicant of the present application takes the one-way matching feature vector as a feature set and represents the change semantics in units of the feature value of the one-way matching feature vector. In order to dynamically aggregate the semantic set as a whole composed of different change semantics of the one-way matching feature vector without ignoring individual semantic change information, the collective expression of the individual features of the one-way matching feature vector and their aggregation scale is used as the feature full amplitude and half amplitude, and the low-rank negative correlation of different dimensions of the overall semantics of the feature set of the one-way matching feature vector is used as the phase and scaled to dynamically adjust the change relationship of the semantic content of the one-way matching feature vector, thereby improving the aggregation of the semantic information expression of the overall feature set of the one-way matching feature vector, and improving the classification convergence efficiency of the one-way matching feature vector through the classifier, that is, improving the efficiency of classification training and the accuracy of classification results.

[0053] In summary, the method for identifying damaged position images of a tower mast structure according to the embodiment of the present application is explained, which analyzes and processes the tower mast structure image by using computer vision technology based on deep learning, captures the global semantic features of the tower mast structure image, and intercepts the local position area from the tower mast structure image, thereby realizing the identification of the damaged position based on the correlation matching between the tower mast structure features of the local position area and the global semantic features of the tower mast structure image. In this way, the accuracy and efficiency of damaged position identification can be improved, while reducing the reliance on manual inspections, and reducing detection costs and risks.

[0054] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for identifying damaged positions of tower and mast structures, characterized in that: include: Acquire a tower mast structure image captured by a camera; Extracting and dispersing features of the surface state of the tower mast structure from the tower mast structure image to obtain a sequence of local feature vectors of the surface state of the tower mast structure; intercepting a first candidate position image to be identified from the tower mast structure image; Using the surface state feature of the tower mast structure in the first candidate position image to be identified as a query feature, and performing one-way interactive matching with the sequence of local feature vectors of the surface state of the tower mast structure to obtain a one-way matching feature vector; The method of using the surface state feature of the tower mast structure in the first candidate position image to be identified as a query feature and performing one-way interactive matching with the sequence of local feature vectors of the surface state of the tower mast structure to obtain a one-way matching feature vector includes: Extracting surface state features of the tower mast structure of the first candidate position to be identified image to obtain a first candidate position to be identified state feature vector; Taking the first candidate position state feature vector to be identified as a query feature vector, inputting the query feature vector and a sequence of the local feature vector of the tower mast structure surface state into a one-way interactive matching network to obtain the one-way matching feature vector; The step of extracting the surface state feature of the tower mast structure of the first candidate position to be identified image to obtain the state feature vector of the first candidate position to be identified includes: Inputting the first candidate position image to be identified into a surface state feature extractor based on a dilated convolutional neural network model to obtain a state feature vector of the first candidate position to be identified; The step of inputting the query feature vector and the sequence of the local feature vector of the surface state of the tower mast structure into a one-way interactive matching network to obtain the one-way matching feature vector comprises: Calculating semantic metric coefficients between the query feature vector and each tower mast structure surface state local feature vector in the sequence of tower mast structure surface state local feature vectors to obtain a sequence of one-way matching semantic metric coefficients; Normalizing the sequence of one-way matching semantic metric coefficients through a softmax function to obtain a sequence of one-way matching semantic metric weights; The weighted sum of the sequence of local feature vectors of the surface state of the tower mast structure is calculated by using the sequence of the one-way matching semantic measurement weights as weights to obtain a one-way matching interactive reshaping feature vector; Calculating a differential feature vector between the query feature vector and the one-way matching interactive reshaping feature vector as the one-way matching feature vector; Based on the one-way matching feature vector, it is determined whether the first candidate position to be identified is a damaged position.

2. The tower mast structure damage position image recognition method according to claim 1, characterized in that: Extracting and dispersing the surface state features of the tower mast structure on the tower mast structure image to obtain a sequence of local feature vectors of the surface state of the tower mast structure includes: Performing boundary information compensation global feature extraction on the tower mast structure image to obtain a global tower mast structure surface state feature map; The global tower mast structure surface state feature map is processed by local feature dispersion along the channel dimension to obtain a sequence of local feature vectors of the tower mast structure surface state.

3. The tower mast structure damage position image recognition method according to claim 2, characterized in that: Performing boundary information compensation global feature extraction on the tower mast structure image to obtain a global tower mast structure surface state feature map, including: The tower mast structure image is input into a dual-branch boundary information compensation network including a backbone network and a boundary feature extraction branch to obtain the global tower mast structure surface state feature map.

4. The tower mast structure damage position image recognition method according to claim 3, characterized in that: The backbone network in the dual-branch boundary information compensation network includes a continuous convolution layer, a first activation layer based on a ReLU activation function, and a pooling layer, wherein the convolution kernel size of the continuous convolution layer is 3x3, the step size is 1, and the padding is 1, and the pooling kernel size of the pooling layer is 2x2, and the step size is 2.

5. The tower mast structure damage position image recognition method according to claim 4, characterized in that: The boundary feature extraction branch in the dual-branch boundary information compensation network includes a point convolution layer, a second activation layer based on a ReLU activation function, and an upsampling layer, wherein the convolution kernel size of the point convolution layer is 1x1, the step size is 1, and the padding is 0.

6. The method for image recognition of damaged positions of tower mast structures according to claim 5, characterized in that: Calculating the semantic metric coefficients between the query feature vector and each tower mast structure surface state local feature vector in the sequence of tower mast structure surface state local feature vectors to obtain a sequence of one-way matching semantic metric coefficients, including: Calculating the square of each eigenvalue of each local eigenvalue of the surface state of each tower mast structure in the sequence of local eigenvalues ​​of the surface state of the tower mast structure to obtain a sequence of local eigenvalues ​​of the surface state of the tower mast structure; Cascading the query feature vector and each nonlinear transformation tower mast structure surface state local feature vector in the sequence of nonlinear transformation tower mast structure surface state local feature vectors to obtain a sequence of semantically associated feature vectors; Multiplying each semantic association feature vector in the sequence of semantic association feature vectors by a predetermined weight vector to obtain a sequence of semantic similarities; Each semantic similarity in the sequence of semantic similarities is respectively added with a predetermined bias parameter and then passed through a sigmoid activation function to obtain a sequence of one-way matching semantic measurement coefficients.

7. The tower mast structure damage position image recognition method according to claim 6, characterized in that: Determining whether the first candidate position to be identified is a damaged position based on the one-way matching feature vector includes: The one-way matching feature vector is input into a classifier-based image recognizer to obtain a recognition result, and the recognition result is used to indicate whether the first candidate position to be recognized is a damaged position.

Images