Communication iron tower with image recognition device

Abstract

The invention provides a communication iron tower with an image recognition device, and belongs to the technical field of image recognition. The communication iron tower with the image recognition device comprises an optical flow calculation module used for inputting a target iron tower image sequence into a target model to obtain an optical flow field between adjacent target iron tower images; the key feature extraction module is used for performing feature extraction on the first target iron tower image to obtain key feature points; the feature matching module is used for matching the key feature points with the second target iron tower image based on the optical flow field to obtain feature coordinates of the key feature points on the second target iron tower image; and the displacement judgment module is used for obtaining the vibration displacement of the iron tower based on the coordinates of the key feature points between the adjacent frames of target iron tower images. According to the invention, the requirement of real-time and accurate monitoring of the communication iron tower can be met.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and particularly to a communication tower with an image recognition device. Background Art

[0002] As an important infrastructure of the communication network, the stability of the communication tower directly affects the communication quality and safety. Traditional tower monitoring methods mainly rely on installing various sensors, such as acceleration sensors, displacement sensors, etc. These methods have many limitations. On the one hand, the installation and maintenance costs of sensors are relatively high, and they are prone to failure due to the influence of harsh environments. On the other hand, sensors can only obtain information at limited positions and it is difficult to comprehensively reflect the overall operating state of the tower.

[0003] With the development of image recognition technology, using images to monitor the tower state has become a new idea. However, existing image monitoring methods are relatively low in efficiency and accuracy in calculating image motion information, extracting features, and matching features, and cannot meet the requirements for real-time and accurate monitoring of communication towers. Summary of the Invention

[0004] Embodiments of the present disclosure provide a communication tower with an image recognition device to meet the requirements for real-time and accurate monitoring of communication towers.

[0005] Embodiments of the present disclosure provide a communication tower with an image recognition device, including: An optical flow calculation module, configured to input a target tower image sequence into a target model to obtain an optical flow field between adjacent target tower images, where the adjacent frame target tower images are adjacent target tower images in the target tower image sequence; A key feature extraction module, configured to extract key feature points from a first target tower image, where the first target tower image is the first frame target tower image in the target tower image sequence; A feature matching module, configured to match the key feature points with a second target tower image based on the optical flow field to obtain the feature coordinates of the key feature points on the second target tower image, where the second target tower image is any frame target tower image in the target tower image sequence except the first frame target tower image; A displacement judgment module, configured to obtain the vibration displacement of the tower based on the coordinates of the key feature points between adjacent frame target tower images.

[0006] In an exemplary embodiment of the present disclosure, it further includes: a target model construction module; The target model construction module is specifically configured to: respectively extract features from any two frames of target tower images to obtain a first feature map and a second feature map; Obtain an initial optical flow estimate based on the similarity between the first feature map and the second feature map; Evaluate the initial optical flow estimate based on a target loss function; Determine the target model based on the evaluation result.

[0007] In an exemplary embodiment of the present disclosure, it further includes: a similarity calculation module; The similarity calculation module is specifically configured to: calculate the similarity between the first feature map and the second feature map based on a first formula; The first formula is:

[0008] Wherein, represents the similarity between the first feature map and the second feature map, represents the feature vector of the first feature map, represents the feature vector of the second feature map, represents the Euclidean distance between the first feature map and the second feature map, represents the weight coefficient corresponding to the Euclidean distance, represents the cosine similarity between the first feature map and the second feature map, represents the weight coefficient corresponding to the cosine similarity, represents the feature distribution weight.

[0009] In an exemplary embodiment of the present disclosure, the key feature extraction module is specifically configured to: Spatially align the first target iron tower image and the first infrared iron tower image to obtain a preprocessed image; Based on the association weight between each pixel point in the preprocessed image and its surrounding pixel points, obtain an initial feature map; Determine the key feature points based on the importance of each target feature in the initial feature map.

[0010] In an exemplary embodiment of the present disclosure, it further includes: a feature enhancement module; The feature enhancement module is used to: In response to the importance of the target feature in the initial feature map being greater than a first preset value, increase the resolution of the target feature; In response to the importance of the target feature in the initial feature map being less than or equal to the first preset value, reduce the resolution of the target feature.

[0011] In an exemplary embodiment of the present disclosure, the feature matching module is specifically configured to: Obtain the motion vector corresponding to each key feature point from the optical flow field based on the coordinates of each key feature point in the first target iron tower image; Determine the initial coordinates of the key feature points in the second target iron tower image based on the motion vector; With the initial coordinates as the center, determine the search window based on the vibration amplitude of the communication iron tower; In response to being within the search window range, calculate the descriptors of all key feature points in the corresponding search window ranges of the first target iron tower image and the second target iron tower image respectively; Match the descriptors of the key feature points in the first target iron tower image with the descriptors of the key feature points in the second target iron tower image to obtain the feature coordinates of the key feature points on the second target iron tower image.

[0012] In an exemplary embodiment of the present disclosure, the feature matching module is further specifically configured to: In response to being within the search window range, use the feature point closest to the descriptor of each key feature point in the first target iron tower image among the descriptors of each key feature point in the corresponding second target iron tower image as the matching point.

[0013] In an exemplary embodiment of the present disclosure, the displacement judgment module is specifically configured to: Calculate the displacement of the key feature points between adjacent frame target iron tower images; Obtain the vibration displacement of the iron tower based on the average value of the displacements of all the key feature points.

[0014] In an exemplary embodiment of the present disclosure, the displacement judgment module is further specifically configured to: Calculate the average value of the displacements of all the key feature points based on the second formula; The second calculation formula is:

[0015] Wherein, represents the average value of the displacements of all key feature points, represents the importance weight of the i-th key feature point, represents the modulus of the displacement vector of the i-th key feature point, represents the cosine value of the included angle between the displacement vector of each key feature point and the reference vector, represents the total number of key feature points.

[0016] In an exemplary embodiment of the present disclosure, it further includes: A vibration analysis module, configured to analyze the iron tower based on the vibration displacement of the iron tower to obtain the operating condition of the iron tower.

[0017] The beneficial effects of a communication tower with an image recognition device provided by an embodiment of the present disclosure are as follows: The optical flow calculation module in the embodiment of the present disclosure can calculate the optical flow field between adjacent target tower images with the help of a target model, clearly presenting the movement information of the tower between adjacent frames and providing a basis for subsequent analysis. The key feature extraction module extracts key feature points from the first frame image to establish an initial feature reference. These representative and unique points are easy to identify and track in different frames. The feature matching module uses the movement information of the optical flow field to narrow the matching search range, greatly improving the efficiency and accuracy of key feature point matching. The displacement judgment module obtains the displacement of each point by comparing the coordinates of key feature points in adjacent frames, and then obtains the vibration displacement of the tower through comprehensive analysis. These vibration displacement data can effectively monitor the operation state of the tower, timely detect abnormal vibration conditions, and thus ensure the stable and safe operation of the communication tower. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 is a schematic structural diagram of a communication tower with an image recognition device provided by an embodiment of the present disclosure; Figure 2 is a schematic structural diagram of a communication tower with an image recognition device provided by another embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] In order to enable those skilled in the art to better understand this solution, the following will clearly describe the technical solutions in the embodiments of this solution in conjunction with the drawings in the embodiments of this solution. Obviously, the described embodiments are some, rather than all, of the embodiments of this solution. Based on the embodiments in this solution, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this solution.

[0021] The term "including" and any other variations in the description and claims of this solution and the above-mentioned drawings mean "including but not limited to", intending to cover non-exclusive inclusion and not limited to the examples listed in the text. In addition, the terms "first" and "second" etc. are used to distinguish different objects, rather than to describe a specific order.

[0022] The following will describe the implementation of the present disclosure in detail in conjunction with specific drawings: Figure 1The structural schematic diagram of a communication tower with an image recognition device provided by an embodiment of the present disclosure. Refer to Figure 1 , the communication tower with an image recognition device includes: An optical flow calculation module, configured to input a target tower image sequence into a target model to obtain an optical flow field between adjacent target tower images, where the adjacent frame target tower images are adjacent target tower images in the target tower image sequence; A key feature extraction module, configured to perform feature extraction on a first target tower image to obtain key feature points, where the first target tower image is the first frame target tower image in the target tower image sequence; A feature matching module, configured to match the key feature points with a second target tower image based on the optical flow field to obtain the feature coordinates of the key feature points on the second target tower image, where the second target tower image is any frame target tower image in the target tower image sequence except the first frame target tower image; A displacement judgment module, configured to obtain the vibration displacement of the tower based on the coordinates of the key feature points between adjacent frame target tower images.

[0023] In this embodiment, the communication tower with an image recognition device can be applied to a communication tower. The optical flow calculation module takes the target tower image sequence as input and calculates the optical flow field between adjacent target tower images. The optical flow field is the motion information of the communication tower in the image between adjacent frames, and the optical flow vector of each point represents the displacement direction and magnitude of the point between two frames of images.

[0024] The target tower image sequence is input into the target model. The target model can be a trained optical flow estimation algorithm model, such as deep learning models like RAFT, PWC-Net, etc. The target model can analyze the brightness change and correlation of pixels in adjacent frame images, and can calculate the motion vector of each pixel point based on assumptions such as brightness constancy hypothesis (i.e., the brightness of the same object remains unchanged in adjacent frames) and temporal continuity hypothesis (the object moves slowly). The set of these motion vectors constitutes the optical flow field.

[0025] In this embodiment, the key feature extraction module takes the first frame target tower image in the target tower image sequence as input, extracts key feature points, and establishes an initial feature reference. The key feature points are representative and unique points in the image, such as corner points and edge points of the communication tower. The key feature points are relatively easy to be recognized and tracked in different image frames.

[0026] Algorithms such as scale-invariant feature transform, speeded-up robust features, or Harris corner detection can be used to find representative and stable key feature points in the first target tower image.

[0027] In this embodiment, after obtaining the optical flow field between adjacent target tower images and the key feature points of the first frame image, the feature matching module matches the key feature points in the first frame image with the second target tower image based on the motion information provided by the optical flow field. The motion vectors in the optical flow field can help predict the approximate positions of the key feature points in the second target tower image, thereby reducing the search range of the matching and improving the efficiency and accuracy of the matching. Through the matching process, the corresponding positions of the key feature points on the second target tower image are found, and their feature coordinates are recorded. These feature coordinates represent the position changes of the key feature points in different image frames.

[0028] In this embodiment, the displacement judgment module compares the coordinates of the key feature points between adjacent frame target tower images. For each key feature point, the coordinate difference between the two adjacent frames of images is calculated, and this difference is the displacement of the feature point during this time period.

[0029] By analyzing and synthesizing the displacements of multiple key feature points, such as calculating statistical quantities such as the average value and maximum value of the displacements of all key feature points, the vibration displacement of the overall tower or specific parts can be obtained. The vibration displacement data can be used to monitor the operating state of the tower and determine whether there is abnormal vibration in the tower.

[0030] It can be concluded from the above that the optical flow calculation module in this embodiment can calculate the optical flow field between adjacent target tower images with the help of the target model, clearly presenting the motion information of the tower between adjacent frames and providing a basis for subsequent analysis. The key feature extraction module extracts key feature points from the first frame image to establish an initial feature reference. These representative and unique points are easy to identify and track in different frames. The feature matching module uses the motion information of the optical flow field to narrow the matching search range, greatly improving the efficiency and accuracy of the key feature point matching. The displacement judgment module obtains the displacement of each point by comparing the coordinates of the key feature points in adjacent frames, and then obtains the tower vibration displacement through comprehensive analysis. These vibration displacement data can effectively monitor the operating state of the tower, timely detect abnormal vibration conditions, and thus ensure the stable and safe operation of the communication tower.

[0031] Reference Figure 2 , in an embodiment of the present disclosure, a communication tower with an image recognition device further includes: a target model construction module; The target model construction module is specifically used for: respectively performing feature extraction on any two frames of target tower images to obtain a first feature map and a second feature map; Obtaining an initial optical flow estimate based on the similarity between the first feature map and the second feature map; Evaluating the initial optical flow estimate based on the target loss function; Determining the target model based on the evaluation result.

[0032] In this embodiment, a convolutional neural network can be used to extract features from two input images and and can automatically extract features that are meaningful for optical flow estimation from the images. Let the feature extraction network be , then the first feature map and the second feature map extracted are respectively expressed as:

[0033]

[0034] where represents the first feature map, represents the second feature map, represents the parameters of the feature extraction network, which are continuously learned and optimized during the training process so that the extracted features can better reflect the essential information of the images and provide a basis for subsequent similarity calculation and optical flow estimation.

[0035] In this embodiment, the similarity of features between the first feature map and the second feature map is measured based on the correlation volume. By calculating the dot product of the feature vectors at each position in the feature map and dividing by the vector norm, the correlation volume is obtained, and the correlation volume reflects the similarity degree between features at different positions. Considering the direction and magnitude of the feature vectors, it can measure the similarity between features more accurately.

[0036] For each position in the first feature map and each position in the second feature map , the calculation formula of the correlation volume is:

[0037] where represents the vector dot product, represents the norm of the vector.

[0038] Based on the similarity between the first feature map and the second feature map, the motion situation of each point in the image is initially speculated to obtain an initial optical flow estimation. The high-similarity regions in the correlation volume correspond to the motion trajectories of objects in the image, thus providing a basis for optical flow estimation.

[0039] In this embodiment, the optical flow estimation result is updated based on a recurrent update module. Let be the hidden state at the t-th iteration, be the optical flow estimation value at the t-th iteration, and the calculation steps of the recurrent update module are as follows: The current optical flow Perform bilinear sampling onto the correlation volume C to obtain the sampled correlation features , and then , and are concatenated together as the input :

[0040]

[0041] In this embodiment, a gated recurrent unit is used to update the hidden state , and the calculation formula of the gated recurrent unit is as follows:

[0042]

[0043]

[0044]

[0045] Wherein, represents the reset gate, represents the update gate, represents the Sigmoid function, and represent learnable weight matrices, represents the candidate hidden state, represents the hyperbolic tangent function.

[0046] In this embodiment, based on a small convolutional network update the optical flow according to the hidden state :

[0047]

[0048] Wherein, represents the parameters of the convolutional network .

[0049] The convolutional network can learn the mapping relationship between the hidden state and the optical flow update amount, thereby continuously optimizing the optical flow estimation result.

[0050] In this embodiment, the initial optical flow estimation is evaluated based on a multi-scale mean absolute error loss function; Assume that N optical flow estimation results are obtained after N iterations , and the corresponding true optical flow is , and the loss function L is defined as:

[0051] Among them, represents the weight of the optical flow loss at the i-th scale, which can be adjusted according to the importance of the optical flow at different scales, enabling the model to pay more attention to the optical flow estimation at important scales during the training process. The mean absolute error loss function can measure the error between the optical flow estimation result and the true optical flow. By minimizing the loss function, the optical flow estimation of the model can be made more accurate.

[0052] Based on the evaluation results of the loss function, optimization algorithms (such as stochastic gradient descent, Adam, etc.) can be used to adjust the parameters of the model and Make adjustments. Continuously iterate and update the parameters until the loss function converges to a smaller value. At this time, the obtained model is the target model. The target model can accurately estimate the optical flow between adjacent target tower images, providing support for subsequent image recognition and tower vibration monitoring.

[0053] It can be concluded from the above that in this embodiment, by extracting feature maps from any two frames of target tower images, the key information of the images can be accurately captured. Based on the similarity of the feature maps, an initial optical flow estimation is obtained, providing a basis for analyzing the movement of the tower. Using the target loss function to evaluate the initial optical flow estimation can timely detect deviations. Determining the target model based on the evaluation results can effectively improve the accuracy of the optical flow estimation, thereby more accurately monitoring the vibration state of the tower and ensuring the stable operation of the communication tower.

[0054] In an embodiment of the present disclosure, a communication tower with an image recognition device further includes: a similarity calculation module; The similarity calculation module is specifically used for: calculating the similarity between the first feature map and the second feature map based on the first formula; The first formula is:

[0055] Among them, represents the similarity between the first feature map and the second feature map, represents the feature vector of the first feature map, represents the feature vector of the second feature map, represents the Euclidean distance between the first feature map and the second feature map, represents the weight coefficient corresponding to the Euclidean distance, represents the cosine similarity between the first feature map and the second feature map, represents the weight coefficient corresponding to the cosine similarity, represents the feature distribution weight.

[0056] In this embodiment, let the feature vectors extracted from two frames of images be and , where n is the dimension of the feature vector. These feature vectors are the numerical representations of image features, containing key information of the image, such as texture, shape, etc.

[0057] Define a comprehensive distance D, and the calculation formula of the comprehensive distance D is:

[0058] where, represents the Euclidean distance, which reflects the accumulation of numerical differences in each dimension of the feature vector. The larger the Euclidean distance, the farther the positions of the two feature vectors in space, and the greater the possible difference between the images; represents the cosine similarity, and the cosine similarity focuses on the direction similarity of the two feature vectors; is then converted to the cosine distance, and the larger the value, the greater the difference in vector directions.

[0059] The comprehensive distance D combines the Euclidean distance and the cosine distance, and balances the importance of the two in the calculation of the comprehensive distance through the weight coefficients and . This can consider both the spatial position and direction information of the feature vector, and avoid the limitations of a single distance metric.

[0060] In this embodiment, in order to further consider the distribution of features in the entire feature space, a weight W based on feature distribution is introduced. Assume that the overall distribution of the feature vectors and can be described by the mean vector and the covariance matrix .

[0061] First, calculate the Mahalanobis distance from the feature vector to the overall mean:

[0062]

[0063] Then, define the weight W of the feature distribution:

[0064] where, is an adjustable parameter used to control the influence degree of the Mahalanobis distance difference on the weight.

[0065] When the difference in the Mahalanobis distance from the two feature vectors to the overall mean is small, the weight W of the feature distribution is close to 1, indicating that the distributions of the two feature vectors in the feature space are relatively similar; when the difference is large, the weight W of the feature distribution will become smaller, reducing the influence of the comprehensive distance in the similarity calculation.

[0066] Finally, the combined distance D and the weight W of the feature distribution are combined to obtain the final similarity measure S:

[0067] As can be seen from the above, in this embodiment, by combining the Euclidean distance and the cosine distance, both the spatial position and the direction information of the feature vectors are considered, avoiding the limitations of a single distance measure. Using the Mahalanobis distance and the overall feature distribution information, the similarity calculation is adjusted according to the distribution of the feature vectors in the feature space, so that even in the case of uneven feature distribution, the similarity between the features of two frames of images can be measured more accurately.

[0068] In an embodiment of the present disclosure, the key feature extraction module is specifically used for: Spatially align the first target iron tower image and the first infrared iron tower image to obtain a preprocessed image; Based on the association weights between each pixel point in the preprocessed image and its surrounding pixel points, obtain an initial feature map; Determine key feature points based on the importance of each target feature in the initial feature map.

[0069] In this embodiment, the first infrared iron tower image can be obtained by an infrared camera.

[0070] Obtain a high-resolution visible light image of the first target iron tower to ensure that the image is clear and the iron tower structure is fully presented. Synchronously collect the infrared image of the same iron tower. The infrared image can reflect the temperature distribution on the surface of the iron tower, which helps to discover potential abnormal heating points, and these points may correspond to hidden dangers in the iron tower structure.

[0071] Use a feature-based registration algorithm, such as SIFT combined with the random sample consensus algorithm, to accurately register the visible light image and the infrared image, so that the two are spatially aligned to obtain a preprocessed image.

[0072] Traditional convolutional neural networks often only focus on local information when extracting features. In this embodiment, a context-aware convolutional neural network is introduced to calculate the association weights between each pixel point and its surrounding pixel points, thereby enhancing the perception of global information. For example, when extracting the features of the iron tower, the relationship between the iron tower and the surrounding environment (such as terrain, other buildings) can be considered.

[0073] Input the preprocessed image into the context-aware convolutional neural network, and output an initial feature map. The initial feature map not only contains the local features of the iron tower, but also integrates the context information, which helps to more accurately locate and identify the key structures of the iron tower.

[0074] Evaluate each target feature in the initial feature map. The importance evaluation can be based on multiple factors, such as the variance of the feature, information entropy, position in the image, etc. Features with larger variance usually contain more variation information, and features with higher information entropy have higher uncertainty and uniqueness. For features located in key structural parts of the iron tower (such as the tower top, tower corners, etc.), higher importance weights can also be assigned. According to the importance scores obtained from the evaluation, set a threshold, and determine the pixel points corresponding to the features with importance scores higher than the threshold as key feature points.

[0075] It can be concluded from the above that the key feature extraction module in this embodiment makes full use of the information of multimodal images, excavates the context relationship between pixels, and thus accurately extracts representative key feature points by performing multimodal image spatial alignment, generating an initial feature map based on correlation weights, and determining key feature points, providing strong support for subsequent image analysis and iron tower status monitoring.

[0076] In an embodiment of the present disclosure, a communication iron tower with an image recognition device further includes: a feature enhancement module; The feature enhancement module is used for: In response to the importance of the target feature in the initial feature map being greater than a first preset value, increase the resolution of the target feature; In response to the importance of the target feature in the initial feature map being less than or equal to the first preset value, reduce the resolution of the target feature.

[0077] In this embodiment, the feature enhancement module is used to perform targeted resolution adjustment on the target features in the initial feature map to highlight important features and suppress unimportant features, thereby improving the accuracy of subsequent image analysis and recognition.

[0078] Before performing the feature enhancement operation, it is necessary to first evaluate the importance of the target features in the initial feature map. The importance evaluation can be based on multiple factors, such as the variance of the feature, information entropy, contribution degree to the judgment of the iron tower status, etc., and assign an importance score to each target feature.

[0079] In this embodiment, when it is detected that the importance of the target feature in the initial feature map is greater than the first preset value, it indicates that the target feature has a relatively high value for image recognition and iron tower status monitoring. The first preset value is a preset threshold used to distinguish important features from unimportant features.

[0080] To capture and analyze the detailed information of these important features more clearly, it is necessary to increase the resolution of the target features. For example, it can be based on deep learning super-resolution convolutional neural networks, enhanced super-resolution generative adversarial networks, etc. Enlarge and enhance the details of the low-resolution target feature image to improve its resolution, making the texture, edges and other information of the important features more clearly distinguishable.

[0081] When the importance of the target feature is less than or equal to the first preset value, it indicates that the feature has a relatively small effect on image recognition and tower status monitoring, such as some noise or irrelevant information. To reduce the interference of these unimportant features on subsequent analysis, the resolution of the target feature is reduced. Image downsampling methods such as average pooling, max pooling, etc. can be used. Reduce the detailed information of the feature, reduce the data volume, and at the same time highlight the dominant position of the important feature.

[0082] It can be concluded from the above that in this embodiment, the resolution of target features with different importance in the initial feature map is adjusted by the feature enhancement module, which can effectively improve the quality of the image and the distinguishability of features. Enhance the expression ability of important features, enabling the subsequent image recognition device to more accurately identify the status and features of the communication tower; at the same time, suppress the interference of unimportant features, reduce the amount of calculation and the possibility of misjudgment, thereby improving the performance and efficiency of the entire communication tower image recognition system.

[0083] In an embodiment of the present disclosure, the feature matching module is specifically used for: Obtain the motion vector corresponding to each key feature point from the optical flow field based on the coordinates of each key feature point in the first target tower image; Determine the initial coordinates of the key feature point in the second target tower image based on the motion vector; Taking the initial coordinates as the center, determine the search window based on the vibration amplitude of the communication tower; In response to being within the search window range, calculate the descriptors of all key feature points in the corresponding search window range of the first target tower image and the second target tower image respectively; Match the descriptors of the key feature points in the first target tower image with the descriptors of the key feature points in the second target tower image to obtain the feature coordinates of the key feature points on the second target tower image.

[0084] In this embodiment, the optical flow field describes the motion information of each point in the image between adjacent frames, which includes motion vectors. These vectors represent the displacement direction and magnitude of a certain point in the image between two frames. The feature matching module first finds the corresponding motion vectors from the pre-calculated optical flow field according to the coordinates of each key feature point in the first target iron tower image. These motion vectors reflect the possible motion of the key feature points from the first frame image to the second frame image, and can provide a basis for predicting the positions of the key feature points in the second target iron tower image subsequently.

[0085] In this embodiment, based on the obtained motion vectors, the positions of the key feature points in the second target iron tower image can be preliminarily predicted. Since the motion vectors represent the displacements of the feature points, the approximate positions in the second target iron tower image can be estimated by adding the corresponding motion vectors to the coordinates of the feature points in the first target iron tower image.

[0086] Let the coordinate of a certain key feature point in the first target iron tower image be , and its corresponding motion vector be , then the initial coordinate of this key feature point in the second target iron tower image can be calculated by the formula , .

[0087] In this embodiment, due to certain errors in the calculation of the optical flow field and the vibration of the communication iron tower in practice, the actual positions of the key feature points in the second target iron tower image may deviate from the initial coordinates. Therefore, it is necessary to determine a search window centered on the initial coordinates and perform more accurate feature matching within this range.

[0088] The size of the search window can be determined according to the vibration amplitude of the communication iron tower. If the vibration amplitude of the iron tower is large, the search window needs to be set larger to ensure that the possible positions of the key feature points can be covered; on the contrary, if the vibration amplitude is small, the search window can be correspondingly reduced, which can not only ensure the accuracy of the matching but also reduce unnecessary computational complexity.

[0089] In this embodiment, the feature descriptor is a vector that describes the local features of the key feature points, with characteristics such as rotation invariance and scale invariance, and can accurately represent the feature information of the feature points under different image conditions. By calculating the feature descriptors, the features of the feature points can be quantified, which is convenient for subsequent matching operations.

[0090] Within the range of the search window, the feature descriptors of all key feature points in the first target iron tower image and the second target iron tower image are calculated respectively. The feature descriptors can be calculated by Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF) or Oriented FAST and Rotated BRIEF (ORB) algorithms.

[0091] In this embodiment, by comparing the similarity between the descriptors of the key feature points in the first target tower image and the descriptors of the key feature points in the second target tower image, the correspondence between the first target tower image and the second target tower image can be found, so as to determine the accurate feature coordinates of the key feature points on the second target tower image.

[0092] Match the descriptors of the key feature points in the first target tower image with the descriptors of the key feature points in the second target tower image. After successful matching, record the coordinates of the corresponding key feature points in the second target tower image, which are the feature coordinates of the key feature points on the second target tower image.

[0093] As can be seen from the above, the feature matching module predicts the initial positions of the key feature points by using the optical flow field information, determines the search window in combination with the vibration amplitude of the communication tower, and then realizes the accurate matching of the key feature points in the first target tower image to the second target tower image by calculating and matching the feature descriptors, providing key data for subsequent calculation of the vibration displacement of the tower.

[0094] In an embodiment of the present disclosure, the feature matching module is further specifically configured to: In response to being within the search window range, take the feature point closest to the descriptor of each key feature point in the first target tower image among each key feature point in the corresponding second target tower image as the matching point.

[0095] In this embodiment, the search window is a region determined based on the initial coordinates of the key feature points in the first target tower image in the second target tower image and the vibration amplitude of the communication tower. Due to possible errors in optical flow field calculation and the vibration of the tower, the actual positions of the key feature points in the second target tower image may deviate from the initial coordinates. The setting of the search window is to find the accurate matching positions of the key feature points within a reasonable range, reduce unnecessary global searches, and improve the matching efficiency.

[0096] For each key feature point in the first target tower image, find the feature point with the closest distance to its descriptor within the search window range of the second target tower image as the matching point. In this embodiment, the Euclidean distance can be used to calculate the descriptor distance.

[0097] Let the descriptor of a certain key feature point in the first target tower image be the vector , and the descriptor of a certain feature point within the search window of the second target tower image be the vector , then the Euclidean distance between them is:

[0098] Where Denote the Euclidean distance from the descriptor of the key feature point in the first target tower image to the descriptor of the key feature point in the second target tower image. n is the dimension of the descriptor vector. and are respectively and the i-th component of the vectors

[0099] Within the search window of the second target tower image, calculate the distance between the descriptor of each key feature point in the first target tower image and the descriptors of all feature points within the search window. Select the feature point with the minimum distance as the matching point for this key feature point. If the distances between the descriptors of two feature points are very close, it indicates that they correspond to the same physical point in the image, that is, they are matched.

[0100] It can be concluded from the above that in this embodiment, by calculating the distances between its descriptors and the descriptors of all feature points within the search window, the matching points are found according to the nearest neighbor matching principle. The key feature points in the first target tower image can be accurately corresponded to the second target tower image, providing a key correspondence relationship for subsequent calculation of the vibration displacement of the tower, thereby realizing effective monitoring of the state of the communication tower.

[0101] In an embodiment of the present disclosure, the displacement judgment module is specifically configured to: Calculate the displacement of the key feature points between adjacent frame target tower images; Obtain the vibration displacement of the tower based on the average value of the displacements of all key feature points.

[0102] In this embodiment, for each key feature point, its displacement between adjacent frame target tower images can be obtained by calculating the difference in coordinates of this feature point in the two frames of images. Assume that the coordinate of a certain key feature point in the first frame of image is , and the coordinate in the second frame of image is , then the displacement vector of this key feature point can be expressed as:

[0103] The magnitude (modulus) of the displacement is:

[0104] In this way, the displacement situation of each key feature point between adjacent frames can be obtained.

[0105] In this embodiment, the displacement of a single key feature point may be affected by factors such as local noise and image acquisition errors, and cannot fully and accurately represent the vibration condition of the entire iron tower. The displacement data of all key feature points contains the motion information of different parts of the iron tower. By calculating their average value, the displacement conditions of each key feature point can be comprehensively considered, and the interference of individual abnormal displacements can be reduced, so as to obtain a value that can better reflect the overall vibration state of the iron tower.

[0106] In an embodiment of the present disclosure, the displacement judgment module is further specifically configured to: Calculate the average value of the displacements of all key feature points based on the second formula; The second calculation formula is:

[0107] Wherein, represents the average value of the displacements of all key feature points, represents the importance weight of the i-th key feature point, represents the modulus of the displacement vector of the i-th key feature point, represents the cosine value of the included angle between the displacement vector of each key feature point and the reference vector, represents the total number of key feature points.

[0108] The traditional method for calculating the average value of the displacements of all key feature points is usually to simply add the displacement vectors of each key feature point between adjacent frames and then divide by the number of key feature points. This method does not consider the importance differences of different key feature points in reflecting the vibration characteristics of the iron tower, nor does it consider the comprehensive influence of the displacement direction on the overall vibration condition.

[0109] In this embodiment, assume that we have n key feature points. In two adjacent frames of images (the first frame and the second frame), the coordinates of the i-th key feature point in the first frame are , and the coordinates in the second frame are .

[0110] The displacement vector of the i-th key feature point can be expressed as:

[0111] The modulus of its displacement (i.e., the displacement magnitude) is:

[0112] Key feature points at different positions contribute differently to reflecting the overall vibration characteristics of the iron tower. The simple averaging method of the traditional method cannot reflect this difference, and may cause the calculation result to not accurately reflect the actual vibration condition of the iron tower. Therefore, this embodiment defines an importance weight for each key feature point , importance weight It can be determined according to factors such as the position of the feature points in the tower structure and the sensitivity to vibration. For example, the key feature points located at the top of the tower are more sensitive to vibration, and their weights can be set higher; while the key feature points located in the relatively stable area at the bottom of the tower, the weights can be set lower. The weights need to satisfy .

[0113] To comprehensively consider the direction of displacement, the dot product operation of vectors can be introduced. Select a reference vector , reference vector can be determined according to the main vibration direction or structural characteristics of the tower. Calculate the cosine value of the angle between each displacement vector and the reference vector , and the calculation formula is:

[0114] Finally, the formula for calculating the average value of the displacements of all key feature points is:

[0115] It can be concluded from the above that in this embodiment, by multiplying the displacement magnitude of each key feature point by its importance weight and the cosine value of the angle, and then summing up all key feature points, the obtained average value can more comprehensively and accurately reflect the overall vibration situation of the tower. It not only considers the importance differences of different key feature points, but also comprehensively considers the influence of the displacement direction on the overall vibration, and has higher accuracy and reliability compared with the traditional simple average method.

[0116] Reference Figure 2 , in an embodiment of the present disclosure, a communication tower with an image recognition device further includes: A vibration analysis module for analyzing the tower based on the vibration displacement of the tower to obtain the operating condition of the tower.

[0117] In this embodiment, multiple different levels of vibration displacement safety thresholds can be preset according to the design standards, the environment where the tower is located, and past operating experience of the tower. For example, a slight vibration threshold, a moderate vibration threshold, and a severe vibration threshold are set.

[0118] Compare the currently obtained vibration displacement of the iron tower with these preset thresholds in real time. If the vibration displacement is lower than the slight vibration threshold, it can be preliminarily judged that the iron tower is in a normal and stable operating state, with a solid structure and no obvious abnormal vibration. If the vibration displacement exceeds the slight vibration threshold but is lower than the moderate vibration threshold, it indicates that there may be some slight abnormalities in the iron tower, such as being disturbed by external factors such as slight wind force and surrounding construction, but it is still within the acceptable operating range and continuous attention is required. When the vibration displacement exceeds the moderate vibration threshold or even reaches the severe vibration threshold, it means that there may be relatively serious problems in the iron tower, such as structural looseness, component damage, etc., and further inspection and maintenance are required immediately.

[0119] From the above, it can be concluded that the vibration analysis module can comprehensively evaluate the operating conditions of the iron tower and obtain specific evaluation conclusions, such as "normal operation", "slight abnormalities, attention required", "serious problems, immediate repair required", etc. These conclusions can be displayed to the operation and maintenance personnel through a visual interface or corresponding reports can be generated, providing strong support for the operation and maintenance decision-making of communication iron towers and ensuring the stable operation of iron towers and the reliability of communication services.

[0120] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

1. A communication tower with an image recognition device, characterized in that: include: An optical flow calculation module is used to input a target tower image sequence into a target model to obtain an optical flow field between adjacent target tower images, wherein the adjacent frame target tower images are adjacent target tower images in the target tower image sequence; A key feature extraction module is used to extract features from a first target tower image to obtain key feature points, wherein the first target tower image is a first frame of a target tower image in a target tower image sequence; A feature matching module is used to match the key feature point with a second target tower image based on the optical flow field to obtain feature coordinates of the key feature point on the second target tower image, wherein the second target tower image is any frame of the target tower image except the first frame of the target tower image in the target tower image sequence; The displacement judgment module is used to obtain the vibration displacement of the tower based on the coordinates of the key feature points between the target tower images in adjacent frames.

2. A communication tower with an image recognition device as claimed in claim 1, characterized in that: Also includes: Target model building module; The target model building module is specifically used to: extract features from any two frames of target tower images respectively to obtain a first feature map and a second feature map; Obtaining an initial optical flow estimate based on a similarity between the first feature map and the second feature map; evaluating the initial optical flow estimate based on a target loss function; The target model is determined based on the evaluation result.

3. A communication tower with an image recognition device as claimed in claim 2, characterized in that: Also includes: Similarity calculation module; The similarity calculation module is specifically used to: calculate the similarity between the first feature map and the second feature map based on a first formula; The first formula is: in, represents the similarity between the first feature map and the second feature map, represents the eigenvector of the first eigenmap, represents the eigenvector of the second eigenmap, represents the Euclidean distance between the first feature map and the second feature map, Represents the weight coefficient corresponding to the Euclidean distance, represents the cosine similarity between the first feature map and the second feature map, Represents the weight coefficient corresponding to the cosine similarity, Represents the feature distribution weight.

4. A communication tower with an image recognition device as claimed in claim 1, characterized in that: The key feature extraction module is specifically used for: Spatially aligning the first target tower image and the first infrared tower image to obtain a preprocessed image; Obtaining an initial feature map based on the association weights of each pixel in the preprocessed image and its surrounding pixels; The key feature points are determined based on the importance of each target feature in the initial feature map.

5. A communication tower with an image recognition device as claimed in claim 4, characterized in that: Also includes: feature enhancement module; The feature enhancement module is used to: In response to the importance of the target feature in the initial feature map being greater than a first preset value, increasing the resolution of the target feature; In response to the importance of the target feature in the initial feature map being less than or equal to a first preset value, the resolution of the target feature is reduced.

6. A communication tower with an image recognition device as claimed in claim 1, characterized in that: The feature matching module is specifically used for: Acquire a motion vector corresponding to each key feature point from the optical flow field based on the coordinates of each key feature point in the first target tower image; Determine the initial coordinates of the key feature point in the second target tower image based on the motion vector; Determine a search window based on the vibration amplitude of the communication tower with the initial coordinates as the center; In response to being within the search window range, respectively calculating the descriptors of all key feature points of the first target iron tower image and the second target iron tower image within the corresponding search window range; The descriptor of the key feature point in the first target tower image is matched with the descriptor of the key feature point in the second target tower image to obtain the feature coordinates of the key feature point on the second target tower image.

7. A communication tower with an image recognition device as claimed in claim 6, characterized in that: The feature matching module is also specifically used for: In response to the search window, the feature point closest to the descriptor of each key feature point in the first target iron tower image and the corresponding key feature point in the second target iron tower image is taken as a matching point.

8. A communication tower with an image recognition device as claimed in claim 1, characterized in that: The displacement judgment module is specifically used for: Calculating the displacement of the key feature points between adjacent frame target tower images; The vibration displacement of the iron tower is obtained based on the average value of the displacements of all the key characteristic points.

9. A communication tower with an image recognition device as claimed in claim 8, characterized in that: The displacement judgment module is also specifically used for: Calculating the average value of the displacements of all the key feature points based on the second formula; The second calculation formula is: in, represents the average displacement of all key feature points, represents the importance weight of the i-th key feature point, represents the modulus of the displacement vector of the i-th key feature point, Represents the cosine value of the angle between the displacement vector of each key feature point and the reference vector, Indicates the total number of key feature points.

10. The communication tower with an image recognition device as claimed in claim 1, characterized in that: Also includes: The vibration analysis module is used to analyze the tower based on its vibration displacement to obtain the operating status of the tower.

Images