Vibration measurement method, device and equipment of cable structure and medium

Abstract

The embodiment of the invention provides a vibration measurement method and device of a cable structure, equipment and a medium, and relates to the technical field of computer vision. The method comprises the following steps: firstly, extracting features (such as corner features, texture features and edge features) of a cable structure from a target image sequence to generate a feature image; and then, according to an area including the cable structure in the feature image, determining spatial position information of the cable knot in a real physical space. And finally, according to the time sequence information and the spatial position information of the target image, analyzing vibration conditions of the cable structure, including vibration frequency, vibration amplitude and the like. Therefore, according to the vibration measurement method of the cable structure disclosed by the embodiment of the invention, the limitation that a traditional sensor needs to be in physical contact with the cable structure is avoided, the possible interference or damage to the cable structure due to sensor installation, maintenance or replacement is reduced, and the flexibility of vibration measurement of the cable structure is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular, to a method, device, equipment and medium for measuring the vibration of a cable structure. Background Art

[0002] A cable structure is a structure often used in projects such as photovoltaic modules, bridges, and towers. It mainly relies on high-strength steel cables to bear the tensile force to achieve large-span space coverage. Due to being exposed to the outdoor environment for a long time, the cable structure is prone to vibration caused by meteorological conditions such as strong winds, and long-term vibration may lead to fatigue damage of the cable structure and even may cause safety problems. Therefore, it is of great significance to measure the vibration of the cable structure in real time to ensure its safety.

[0003] In related technologies, the vibration measurement of a cable structure usually relies on traditional sensor technologies, such as acceleration sensors, strain gauges, etc. These sensors are directly installed on the cable structure to measure the vibration of the cable structure by physical contact.

[0004] However, sensors are very easy to be damaged in the outdoor environment and need to be maintained and replaced regularly, which not only consumes a large amount of manpower and material resources, but also may affect the normal operation of the cable structure if the maintenance or replacement is improper. Summary of the Invention

[0005] Based on the above problems, the present application provides a method, device, equipment and medium for measuring the vibration of a cable structure, which can reduce the interference or damage that may be caused to the cable structure itself due to the installation, maintenance or replacement of sensors, and increase the flexibility of the vibration measurement of the cable structure.

[0006] The embodiments of the present application disclose the following technical solutions: In a first aspect, the present application discloses a method for measuring the vibration of a cable structure, the method comprising: Obtaining a target image sequence, the target image sequence comprising a plurality of target images arranged in time sequence, each of the target images comprising a cable structure; Generating a feature image corresponding to the target image by extracting features in the target image, the features comprising at least one of corner features, texture features and edge features; Determining the spatial position information of the cable structure in the real physical space by determining the positive sample region in the feature image, the positive sample region being the region comprising the cable structure; Determining the vibration frequency and vibration amplitude of the cable structure according to the time sequence information of the target image and the spatial position information.

[0007] Optionally, determining the spatial position information of the cable structure in the real physical space by determining the positive sample regions in the feature image includes: Generating a mask in the feature image by determining the positive sample regions in the feature image; Determining the spatial position information of the cable structure in the real physical space according to the coordinate information of the mask in the feature image and the camera parameters for collecting the target image sequence.

[0008] Optionally, generating a mask in the feature image by determining the positive sample regions in the feature image includes: Determining the probability distribution of the cable structure by determining the positive sample regions and negative sample regions in the feature image, where the negative sample regions are regions that do not include the cable structure; Generating a mask in the feature image according to the probability distribution of the cable structure.

[0009] Optionally, determining the positive sample regions and negative sample regions in the feature image includes: Allocating a plurality of preset anchor boxes in the feature image, and determining the real target box in the feature image, where the real target box is the target box including the cable structure; Determining the preset anchor boxes with an intersection over union greater than the first similarity threshold with the real target box as positive sample regions, and determining the preset anchor boxes with an intersection over union less than the second similarity threshold with the real target box as negative sample regions, where the first similarity threshold is higher than the second similarity threshold.

[0010] Optionally, the feature image includes a first feature image and a second feature image, and determining the positive sample regions in the feature image includes: If the resolutions of both the first feature image and the second feature image are higher than the preset resolution threshold, and the similarity between the first feature image and the second feature image is higher than the third similarity threshold, then determining the positive sample regions in the first feature image or the second feature image.

[0011] Optionally, generating a feature image corresponding to the target image by extracting features from the target image includes: Extracting a plurality of features from the target image through a multi-layer convolutional neural network to generate multiple sub-feature images, where each layer of the convolutional neural network is used to extract one feature of the target image; Performing path aggregation on the multiple sub-feature images to obtain the feature image corresponding to the target image.

[0012] Optionally, after performing the determination of the vibration frequency and vibration amplitude of the cable structure, the method further includes: When the vibration frequency is greater than a preset frequency threshold, and / or the vibration amplitude is greater than a preset amplitude threshold, a warning indication is triggered.

[0013] In a second aspect, the present application discloses a vibration measurement device for a cable structure, the device including: an image acquisition module, a feature generation module, a position determination module, and a vibration measurement module; The image acquisition module is configured to acquire a target image sequence, the target image sequence including a plurality of target images arranged in time sequence, and each of the target images includes a cable structure; The feature generation module is configured to generate a feature image corresponding to the target image by extracting features in the target image, where the features include at least one of corner features, texture features, and edge features; The position determination module is configured to determine the spatial position information of the cable structure in the real physical space by determining a positive sample region in the feature image, where the positive sample region is a region including the cable structure; The vibration measurement module is configured to determine the vibration frequency and vibration amplitude of the cable structure according to the time sequence information of the target image and the spatial position information.

[0014] Optionally, the position determination module includes: a mask generation module and an information determination module; The mask generation module is configured to generate a mask in the feature image by determining a positive sample region in the feature image; The information determination module is configured to determine the spatial position information of the cable structure in the real physical space according to the coordinate information of the mask in the feature image and the camera parameters for acquiring the target image sequence.

[0015] Optionally, the mask generation module includes: a distribution determination sub-module and a mask generation sub-module; The distribution determination sub-module is configured to determine the probability distribution of the cable structure by determining a positive sample region and a negative sample region in the feature image, where the negative sample region is a region not including the cable structure; The mask generation sub-module is configured to generate a mask in the feature image according to the probability distribution of the cable structure.

[0016] Optionally, the distribution determination sub-module is specifically configured to: allocate a plurality of preset anchor boxes in the feature image, and determine a true target box in the feature image, where the true target box is a target box including a cable structure; determine a preset anchor box with an intersection over union (IoU) greater than a first similarity threshold with the true target box as a positive sample region, and determine a preset anchor box with an IoU less than a second similarity threshold with the true target box as a negative sample region, where the first similarity threshold is higher than the second similarity threshold.

[0017] Optionally, the feature image includes a first feature image and a second feature image. The position determination module is specifically configured to: if the resolutions of both the first feature image and the second feature image are higher than a preset resolution threshold, and the similarity between the first feature image and the second feature image is higher than a third similarity threshold, then determine the positive sample region in the first feature image or the second feature image.

[0018] Optionally, the feature generation module includes: a first generation module and a second generation module; The first generation module is configured to extract a plurality of features in the target image through a multi-layer convolutional neural network and generate a plurality of sub-feature images, where each layer of the convolutional neural network is used to extract one feature of the target image; The second generation module is configured to perform path aggregation on the plurality of sub-feature images to obtain a feature image corresponding to the target image.

[0019] Optionally, the vibration measurement device of the cable structure further includes: an alarm indication module; The alarm indication module is configured to trigger an alarm indication when the vibration frequency is greater than a preset frequency threshold and / or the vibration amplitude is greater than a preset amplitude threshold.

[0020] In a third aspect, the present application discloses a vibration measurement device for a cable structure, and the device includes: a memory and a processor; The memory is configured to store a program; The processor is configured to execute the program to implement each step of the vibration measurement method for the cable structure as described in the first aspect.

[0021] In a fourth aspect, the present application discloses a computer-readable medium, on which a computer program is stored. When the computer program is executed by a processor, each step of the vibration measurement method for the cable structure as described in the first aspect is implemented.

[0022] Compared with the prior art, the present application has the following beneficial effects: The embodiments of the present application provide a method, device, equipment and medium for measuring the vibration of a cable structure. First, by extracting the features of the cable structure (such as corner features, texture features and edge features) from the target image sequence, a feature image is generated. Subsequently, according to the region including the cable structure in the feature image, the spatial position information of the cable knot in the real physical space is determined. Finally, according to the timing information and spatial position information of the target image, the vibration condition of the cable structure is analyzed, including vibration frequency, vibration amplitude, etc. Thus, the method for measuring the vibration of the cable structure disclosed in the embodiments of the present application avoids the limitation that traditional sensors need to be physically in contact with the cable structure, reduces the possible interference or damage to the cable structure itself caused by sensor installation, maintenance or replacement, and increases the flexibility of the vibration measurement of the cable structure. Description of the Drawings

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0024] Figure 1 It is a flowchart of a method for measuring the vibration of a cable structure provided by an embodiment of the present application; Figure 2 It is a flowchart of another method for measuring the vibration of a cable structure provided by an embodiment of the present application; Figure 3 It is a schematic diagram of a device for measuring the vibration of a cable structure provided by an embodiment of the present application; Figure 4 It is a schematic diagram of a computer-readable medium provided by an embodiment of the present application. Detailed Embodiments

[0025] As described above, the vibration measurement of cable structures usually relies on traditional sensor technologies, such as acceleration sensors, strain gauges, etc. These sensors are directly installed on the cable structure to measure the vibration condition of the cable structure by physical contact. However, sensors are very vulnerable to damage in outdoor environments and need to be maintained and replaced regularly, which not only consumes a lot of manpower and material resources, but also may affect the normal operation of the cable structure if the maintenance or replacement is improper.

[0026] After research, the inventor proposed a vibration measurement method, device, equipment and medium for a cable structure. First, by extracting the features of the cable structure (such as corner features, texture features and edge features) from the target image sequence, a feature image is generated. Subsequently, according to the region including the cable structure in the feature image, the spatial position information of the cable knot in the real physical space is determined. Finally, according to the timing information and spatial position information of the target image, the vibration condition of the cable structure is analyzed, including vibration frequency, vibration amplitude, etc. Thus, the vibration measurement method of the cable structure disclosed in the embodiments of the present application avoids the limitation that traditional sensors need to be physically in contact with the cable structure, reduces the possible interference or damage to the cable structure itself caused by sensor installation, maintenance or replacement, and increases the flexibility of the vibration measurement of the cable structure. Moreover, the vibration measurement method of the cable structure disclosed in the embodiments of the present application uses a multi-layer convolutional neural network and path aggregation technology, which can automatically extract and process the features of the cable structure, improving the accuracy and intelligent level of the vibration measurement of the cable structure. In addition, the vibration measurement method of the cable structure disclosed in the embodiments of the present application can monitor the vibration state in real time, and automatically trigger an alarm when the vibration frequency and / or vibration amplitude exceed a preset threshold, ensuring timely warning of potential safety hazards.

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0028] See Figure 1 , which is a flowchart of a vibration measurement method for a cable structure provided by an embodiment of the present application. The method includes: S101: Obtain a target image sequence, where the target image sequence includes multiple target images arranged in time sequence, and each target image includes a cable structure.

[0029] The target image sequence is obtained through a camera, a drone or other vision sensors. The target image sequence is a set composed of multiple target images arranged in time sequence, and each target image includes a cable structure. Moreover, these target images can be obtained by shooting at fixed intervals. For example, 10 photos can be taken every 1 s.

[0030] It can be understood that the resolution of the target image is crucial for subsequent feature extraction. Therefore, a high-resolution camera is usually used to ensure that the target image remains clear in complex environments (such as light changes, background interference, etc.). The specific resolution is not limited in the present application.

[0031] S102: Generate a feature image corresponding to the target image by extracting features from the target image, where the features include at least one of corner features, texture features, and edge features.

[0032] Features refer to specific attributes or structures that can be extracted from the target image and used to describe the content of the image, including corner features, texture features, and edge features, etc. Among them, corner features indicate obvious corners in the target image and can help identify the shape and structure of objects (such as cable structures) in the target image. Texture features indicate the repetitive patterns or surface characteristics of a certain area in the target image, such as smoothness, roughness, etc. Edge features indicate the contour lines of objects in the target image and can help identify the boundaries of objects in the target image.

[0033] The feature image is a new image generated based on the extracted features, which only contains the extracted feature information (such as corners, textures, or edges), while ignoring other redundant information. Thus, by extracting the features from the target image, the original target image is converted into a more concise and meaningful representation form (i.e., the feature image), which is convenient for subsequent processing.

[0034] S103: Determine the spatial position information of the cable structure in the real physical space by determining the positive sample region in the feature image, where the positive sample region is the region including the cable structure.

[0035] In some specific implementation manners, the method for determining the positive sample region in the feature image is as follows: First, allocate multiple preset anchor boxes in the feature image. These preset anchor boxes are used as candidate regions to detect possible targets (i.e., cable structures). Second, determine the ground truth box containing the cable structure in the feature image through the manual annotation method. Subsequently, determine the preset anchor box whose intersection over union (IoU) with the ground truth box is greater than the first similarity threshold as the positive sample region. Among them, IoU is an index to measure the overlapping degree of two anchor boxes, and its value range is [0, 1]. The larger the IoU, the more similar the two anchor boxes are. Exemplarily, if the IoU value of a certain preset anchor box with any ground truth box is greater than the first similarity threshold (such as 0.8), then it is marked as the positive sample region.

[0036] After determining the positive sample regions in the feature image, the probability that each positive sample region contains the true cable structure can be calculated, and the probability distribution of the cable structure can be obtained. Subsequently, based on the probability distribution of the cable structure, a target mask is generated in the feature image. The target mask is a binary image that is used to identify the exact position and shape of the cable structure in the feature image. It provides a clear boundary to distinguish the cable structure from the background or other structures.

[0037] Finally, by determining the position information of the cable structure in the target mask, the spatial position information of the cable structure in the real physical space can be determined. In some specific implementation manners, first, the minimum bounding rectangle of the target mask is determined, and the upper-left coordinate (x_min, y_min) and the lower-right coordinate (x_max, y_max) of the minimum bounding rectangle are recorded, so as to determine the position information of the target mask. Subsequently, according to the internal parameters (such as focal length, principal point, etc.) and external parameters (such as rotation matrix, translation vector, etc.) of the camera, the position information of the target mask is converted into the spatial position information of the cable structure in the real physical space. Exemplarily, assuming that the position information of the target mask is (x_img, y_img), the spatial position information (X_real, Y_real, Z_real) of the cable structure in the real physical space can be determined through the projection matrix composed of the internal and external parameters of the camera and the position information of the target mask.

[0038] S104: Determine the vibration frequency and vibration amplitude of the cable structure according to the timing information and spatial position information of the target image.

[0039] In step S104, for each target image in the target image sequence, the spatial position information of the cable structure in the real physical space needs to be determined, and then the spatial position change of the cable structure in the real physical space is determined. This spatial position change can be described by a displacement vector, which represents the moving direction and distance of the cable structure from one position to another.

[0040] Subsequently, based on the displacement vector of the cable structure and the actual dimension information (such as parameters including length, diameter, etc., which can be obtained through design documents, CAD models or on-site measurements), the vibration frequency and vibration amplitude of the cable structure can be determined.

[0041] In summary, the embodiment of the present application provides a method for measuring the vibration of a cable structure. First, by extracting the features of the cable structure (such as corner features, texture features, and edge features) from the target image sequence, a feature image is generated. Subsequently, according to the region including the cable structure in the feature image, the spatial position information of the cable knot in the real physical space is determined. Finally, according to the timing information and spatial position information of the target image, the vibration condition of the cable structure is analyzed, including vibration frequency, vibration amplitude, etc. Thus, the method for measuring the vibration of the cable structure disclosed in the embodiment of the present application avoids the limitation that traditional sensors need to be in physical contact with the cable structure, reduces the possible interference or damage to the cable structure itself caused by sensor installation, maintenance, or replacement, and increases the flexibility of the vibration measurement of the cable structure.

[0042] See Figure 2 , which is a flowchart of another method for measuring the vibration of a cable structure provided by the embodiment of the present application. The method includes: S201: Obtain a target image sequence, where the target image sequence includes multiple target images arranged in time sequence, and each target image includes a cable structure.

[0043] It can be understood that this step is similar to step S101 and will not be elaborated here.

[0044] S202: Generate a feature map by extracting features in the target image.

[0045] In order to extract features efficiently and accurately, Convolutional Neural Networks (CNN) technology can be adopted. Specifically, the method for measuring the vibration of the cable structure provided by the embodiment of the present application slides a convolution kernel on the target image through a convolutional layer, performs a convolution operation on the target image, extracts edge features, corner features, texture features, etc. of the target image, and generates a set of feature maps (each convolutional neural network layer is used to extract a feature of the target image, and each feature map is used to characterize a feature of the target image). Subsequently, the spatial size of this set of feature maps is increased through a deconvolution layer.

[0046] It should be noted that during the process of extracting features through convolutional neural network technology, multiple convolutional layers can be connected with residual connections. By directly transmitting the output of the previous convolutional layer to the subsequent convolutional layer, the problem of gradient disappearance in deep networks can be avoided, ensuring that features can be effectively transmitted to deeper network layers.

[0047] It should also be noted that a bottleneck structure can also be used to reduce the number of channels in the intermediate layer in the convolutional layer, reduce the number of parameters and computational complexity of the convolutional neural grid, so as to reduce the consumption of computing resources while maintaining the feature extraction ability.

[0048] It should also be noted that in the process of extracting features through convolutional neural network technology, a CXF unit (an optimized convolutional module) can also be used to improve the efficiency of feature extraction by reducing redundant parameters and optimizing the network structure.

[0049] It should also be noted that features can also be extracted through Depthwise Separable Convolution technology. The Depthwise Separable Convolution technology decomposes the standard convolution into two steps: first, perform a convolution operation on each input channel separately, and second, use a 1×1 convolution to fuse the channel information. Thus, the computational amount can be greatly reduced while maintaining the ability of feature extraction.

[0050] It should also be noted that features can also be extracted through Dilated Convolution technology, that is, by inserting holes in the convolution kernel to expand the receptive field of the convolution kernel and capture a larger range of context information, so as to be able to extract spatial features in a larger range.

[0051] S203: Perform scale unification processing on the feature map to obtain the processed feature map.

[0052] It should be noted that the cable structure may have different sizes. If only feature maps of a single scale are used, it may not be possible to effectively capture all sizes of targets (i.e., the cable structure) at the same time.

[0053] In some specific implementation manners, first, for the feature maps output by each convolutional layer, an appropriate pooling strategy is applied to adjust their sizes. Then, the feature maps of different scales after pooling are spliced together. Thus, by splicing, the high-resolution features of the shallow layer (suitable for detecting small-sized targets) and the low-resolution features of the deep layer (suitable for detecting large-sized targets) can be utilized simultaneously, thereby improving the detection ability for targets of different sizes.

[0054] S204: Determine the positive sample region and negative sample region in the processed feature map, as well as the weights corresponding to each region respectively, where the positive sample region is the region including the cable structure, and the negative sample region is the region not including the cable structure.

[0055] In the vibration measurement method of the cable structure provided by the embodiments of the present application, the positive sample region and the negative sample region in the processed feature map, as well as the weights corresponding to each region, can be determined by intelligently allocating anchor boxes. Specifically, first, calculate the IoU between each anchor box and the ground truth box (the box that has been determined to contain the cable structure). Subsequently, determine the positive sample region and the negative sample region according to the IoU value. For example, if the IoU value of a certain anchor box and any ground truth box is greater than the first similarity threshold (such as 0.8), it is marked as a positive sample region. If the IoU value of a certain anchor box and any ground truth box is less than the second similarity threshold (such as 0.2), it is marked as a negative sample region.

[0056] It should be noted that the present application does not limit the above first similarity threshold and the second similarity threshold, and only needs to ensure that the first similarity threshold is greater than the second similarity threshold. Moreover, the first similarity threshold and the second similarity threshold can be adaptively adjusted, so as to reduce the probability that some actual cable structures are ignored (missed detection) or some background regions are misidentified as cable structures (false detection), thereby improving the detection accuracy.

[0057] It should also be noted that during the process of determining the positive sample region and the negative sample region from each processed feature map, the calculation accuracy can also be dynamically adjusted. Exemplarily, if the resolution of the processed feature map is 1280×900 (i.e., the resolution is relatively high), if the positive sample region and the negative sample region of each processed feature map are accurately determined, a large amount of computing resources will be consumed. Therefore, in one implementation, in order to avoid unnecessary repeated calculations, a similarity comparison mechanism can be introduced. If the resolutions of two processed feature maps are both higher than the preset resolution threshold, and the similarity between the two processed feature maps is higher than the third similarity threshold (such as 0.9), it is considered that they have high similarity, and only one of them needs to be determined for the positive sample region and the negative sample region. Thus, the amount of calculation can be significantly reduced while maintaining a high detection accuracy. In another implementation, selective processing can also be performed based on time frequency. For example, if 4 processed feature maps are obtained every 1s, in order to reduce the amount of calculation, a certain number of processed feature maps can be selected at intervals for a detailed determination of the positive and negative sample regions. For example, the first and third (or the second and fourth) processed feature maps can be selected from the 4 processed feature maps for the determination of the positive sample region and the negative sample region.

[0058] S205: Perform path aggregation on the processed feature map to obtain the fused feature map.

[0059] The purpose of Path Aggregation is to combine processed feature maps at different levels, which can capture both low-level details (such as edges and textures) and high-level semantic information (such as object categories and overall structures) in the target image, thereby enhancing the detection and recognition capabilities for targets at different scales.

[0060] Path Aggregation generally includes two main parts: the Bottom-Up Path and the Top-Down Path. Among them, the Bottom-Up Path refers to starting from the processed feature map of the input, and gradually extracting more and more abstract high-level features through a series of convolutional layers. These high-level features usually contain more semantic information but may lose some details. The Top-Down Path refers to starting from the high-level feature map, and gradually restoring the spatial resolution through upsampling (such as deconvolution or bilinear interpolation), combining the high-level semantic information with the low-level detail information. This can restore more detail information while maintaining the semantic information. To better fuse features at different levels, lateral connections are usually added between the Bottom-Up Path and the Top-Down Path. These connections fuse the feature maps at the same scale by element-wise addition or concatenation to obtain the fused feature map.

[0061] S206: Determine the probability distribution of the cable structure according to the fused feature map, the positive sample region, the negative sample region, and the weights corresponding to each region respectively.

[0062] It can be understood that through the processing of steps S201 to S205, the feature map has been obtained and it has been determined which regions may be the locations of the cable structure. In step S206, this information can be integrated to calculate the probability that each region includes the real cable structure.

[0063] S207: Generate a target mask in the fused feature map according to the probability distribution of the cable structure.

[0064] If the probability distribution of the cable structure in a certain region determined in step S206 is higher than a preset probability threshold (such as 0.5), then determine that this region is the cable structure and mark it as 1; otherwise, mark it as 0 (indicating that this region is not the cable structure). Subsequently, convert the above binarization result into a target mask, where 1 represents the target region and 0 represents the background region.

[0065] It should be noted that in order to improve the quality of the target mask, some refinement and optimization operations are usually required. For example, morphological operations such as dilation (expanding the foreground region, which helps to connect broken parts) and erosion (shrinking the foreground region, which helps to remove small noise) are used to remove noise in the mask and fill holes, making the mask smoother and more continuous. This application does not make any limitations on this.

[0066] S208: Determine the spatial position information of the cable structure in the real physical space by determining the position information of the target mask.

[0067] It should be noted that during the process of determining the spatial position information of the cable structure in the real physical space, it is also necessary to consider the influence of environmental factors such as temperature and humidity on the internal and external parameters of the camera. This is because temperature changes can cause the camera lens material to expand or contract, thereby changing the focal length and other optical parameters. In addition, a high-humidity environment may cause condensation on the lens surface, affecting the quality of the acquired target image and thus the accuracy of the camera parameters. For the influence of temperature, the internal and external parameters of the camera at different temperatures can be obtained through experiments, and the corresponding calibration curves can be established, so as to monitor the environmental temperature using a temperature sensor in actual applications and adjust the camera parameters in real time according to the pre-established calibration curves. For the influence of humidity, the quality of the target image can be improved through image preprocessing techniques (such as defogging algorithms) to avoid errors caused by humidity. Moreover, it is also necessary to consider the deviations caused by reasons such as equipment aging and improper installation. This is because if the camera is improperly installed, it may cause deviations in the measured angle and distance. Therefore, the camera can be checked and reinstalled every preset time interval to ensure its correct position and angle, and the perspective of the target image can be corrected through geometric transformation methods to eliminate the deviations caused by improper camera installation. This application does not make any limitations on this.

[0068] S209: Determine the displacement vector of the cable structure according to the timing information of multiple target images and the spatial position information of the cable structure corresponding to the multiple target images in the real physical space.

[0069] The above steps S201 - S208 are the steps to determine the spatial position information of the cable structure in the real physical space through target images. In S209, it is necessary to analyze multiple target images arranged in chronological order to determine the change in the spatial position of the cable structure in the real physical space, and then calculate the displacement vector of the cable structure (i.e., the vector transformation result). This displacement vector describes the moving direction and distance of the cable structure.

[0070] S210: Determine the vibration frequency and vibration amplitude of the cable structure according to the actual size information and displacement vector of the cable structure.

[0071] In a specific implementation manner, the determination method of the vibration frequency of the cable structure is as follows: First, denoise and normalize the time series data of the displacement vector to obtain preprocessed data, thereby ensuring that the preprocessed data is smooth and free of outliers. Second, apply the fast Fourier transform algorithm to the preprocessed data to convert the time-domain signal into a frequency-domain signal and obtain a frequency spectrum. Finally, identify the peak points in the frequency spectrum whose amplitudes exceed the preset amplitude threshold, and the frequency corresponding to the peak point is the vibration frequency of the cable structure.

[0072] In a specific implementation manner, the determination method of the vibration amplitude of the cable structure is as follows: First, based on the actual size information of the cable structure (such as the diameter D of the cable structure) and the pixel width W of the cable structure in the image, establish a proportionality coefficient α between the pixel distance and the actual physical distance. Where α = D / W.

[0073] Second, based on the proportionality coefficient α and the horizontal and vertical components of the displacement vector, determine the actual physical displacement (including the actual physical displacement dx in the horizontal direction and the actual physical displacement dy in the vertical direction). Where the actual physical displacement dx in the horizontal direction is the product of the proportionality coefficient α and the horizontal component of the displacement vector, and the actual physical displacement dy in the vertical direction is the product of the proportionality coefficient α and the vertical component of the displacement vector.

[0074] Subsequently, based on the actual physical displacement dx in the horizontal direction and the actual physical displacement dy in the vertical direction, determine the instantaneous vibration amplitude A. Where the determination formula of the instantaneous vibration amplitude A is specifically shown in the following formula (1): (1) Subsequently, after determining the instantaneous vibration amplitudes corresponding to multiple time points, take the maximum value as the vibration amplitude of the cable structure. Or, after determining the instantaneous vibration amplitudes corresponding to multiple time points, calculate the corresponding root mean square value A RMS as the vibration amplitude of the cable structure. Where the root mean square value A RMS The calculation formula is shown in the following formula (2): (2) Where A RMS is the root mean square value, N is the total number of time points, and A i is the instantaneous vibration amplitude at the i-th time point.

[0075] S211: When the vibration frequency is greater than the preset frequency threshold and / or the vibration amplitude is greater than the preset amplitude threshold, trigger an alarm indication.

[0076] It should be noted that the above-mentioned preset frequency threshold can be 5 Hz, and the above-mentioned preset amplitude threshold can be 0.05 m. The present application does not limit the specific preset frequency threshold and preset amplitude threshold. By setting reasonable preset frequency thresholds and / or preset amplitude thresholds, an alarm can be issued in a timely manner when abnormal vibrations are detected, so as to take corresponding measures for maintenance or processing. The alarm indication can be realized in various ways, such as sending an email, a text message notification, a pop-up window prompt on the interface, etc.

[0077] In summary, the present application discloses a method for measuring the vibration of a cable structure. First, by extracting the features of the cable structure (such as corner features, texture features, and edge features) from the target image sequence, a feature image is generated. Subsequently, according to the region including the cable structure in the feature image, the spatial position information of the cable knot in the real physical space is determined. Finally, according to the timing information and spatial position information of the target image, the vibration conditions of the cable structure are analyzed, including the vibration frequency, vibration amplitude, etc. Thus, the method for measuring the vibration of the cable structure disclosed in the embodiments of the present application avoids the limitation that traditional sensors need to be in physical contact with the cable structure, reduces the possible interference or damage to the cable structure itself caused by sensor installation, maintenance, or replacement, and increases the flexibility of the vibration measurement of the cable structure. Moreover, the method for measuring the vibration of the cable structure disclosed in the embodiments of the present application uses a multi-layer convolutional neural network and a path aggregation technology, which can automatically extract and process the features of the cable structure, improving the accuracy and intelligent level of the vibration measurement of the cable structure. In addition, the method for measuring the vibration of the cable structure disclosed in the embodiments of the present application can monitor the vibration state in real time, and automatically trigger an alarm when the vibration frequency and / or vibration amplitude exceed the preset threshold, ensuring timely warning of potential safety hazards.

[0078] See Figure 3 , which is a schematic diagram of a device for measuring the vibration of a cable structure provided by an embodiment of the present application. The device 300 for measuring the vibration of the cable structure includes: an image acquisition module 301, a feature generation module 302, a position determination module 303, and a vibration measurement module 304.

[0079] The image acquisition module 301 is configured to acquire a target image sequence, where the target image sequence includes a plurality of target images arranged in time sequence, and each target image includes a cable structure; The feature generation module 302 is configured to generate a feature image corresponding to the target image by extracting features in the target image, and the features include at least one of corner features, texture features, and edge features; The position determination module 303 is configured to determine the spatial position information of the cable structure in the real physical space by determining the positive sample region in the feature image, and the positive sample region is the region including the cable structure; The vibration measurement module 304 is configured to determine the vibration frequency and vibration amplitude of the cable structure according to the timing information and spatial position information of the target image.

[0080] In some specific implementation manners, the position determination module 303 includes: a mask generation module and an information determination module; The mask generation module is configured to generate a mask in the feature image by determining the positive sample region in the feature image; The information determination module is configured to determine the spatial position information of the cable structure in the real physical space according to the coordinate information of the mask in the feature image and the camera parameters for collecting the target image sequence.

[0081] In some specific implementation manners, the mask generation module includes: a distribution determination sub-module and a mask generation sub-module; The distribution determination sub-module is configured to determine the probability distribution of the cable structure by determining the positive sample region and the negative sample region in the feature image, where the negative sample region is the region that does not include the cable structure; The mask generation sub-module is configured to generate a mask in the feature image according to the probability distribution of the cable structure.

[0082] In some specific implementation manners, the distribution determination sub-module is specifically configured to: allocate a plurality of preset anchor boxes in the feature image, and determine the real target box in the feature image, where the real target box is the target box that includes the cable structure; determine the preset anchor boxes with an intersection over union ratio greater than the first similarity threshold with the real target box as the positive sample region, and determine the preset anchor boxes with an intersection over union ratio less than the second similarity threshold with the real target box as the negative sample region, and the first similarity threshold is higher than the second similarity threshold.

[0083] In some specific implementation manners, the feature image includes a first feature image and a second feature image, and the position determination module 303 is specifically configured to: if the resolutions of both the first feature image and the second feature image are higher than the preset resolution threshold, and the similarity between the first feature image and the second feature image is higher than the third similarity threshold, then determine the positive sample region in the first feature image or the second feature image.

[0084] In some specific implementation manners, the feature generation module 302 includes: a first generation module and a second generation module; The first generation module is configured to extract a plurality of features in the target image through a multi-layer convolutional neural network to generate a plurality of sub-feature images, where each layer of the convolutional neural network is used to extract one feature of the target image; The second generation module is configured to perform path aggregation on the plurality of sub-feature images to obtain the feature image corresponding to the target image.

[0085] In some specific implementation manners, the vibration measurement device 300 of the cable structure further includes: an alarm indication module; The alarm indication module is configured to trigger an alarm indication when the vibration frequency is greater than a preset frequency threshold and / or the vibration amplitude is greater than a preset amplitude threshold.

[0086] In summary, the present application discloses a vibration measurement device for a cable structure. First, by extracting features of the cable structure (such as corner features, texture features, and edge features) from a target image sequence, a feature image is generated. Subsequently, according to the region including the cable structure in the feature image, the spatial position information of the cable knot in the real physical space is determined. Finally, according to the timing information and spatial position information of the target image, the vibration condition of the cable structure is analyzed, including vibration frequency, vibration amplitude, etc. Thus, the vibration measurement device for the cable structure disclosed in the embodiments of the present application avoids the limitation that traditional sensors need to be physically in contact with the cable structure, reduces the possible interference or damage to the cable structure itself caused by sensor installation, maintenance, or replacement, and increases the flexibility of vibration measurement of the cable structure. Moreover, the vibration measurement device for the cable structure disclosed in the embodiments of the present application uses a multi-layer convolutional neural network and a path aggregation technology, which can automatically extract and process the features of the cable structure, improving the accuracy and intelligent level of vibration measurement of the cable structure. In addition, the vibration measurement device for the cable structure disclosed in the embodiments of the present application can monitor the vibration state in real time and automatically trigger an alarm when the vibration frequency and / or vibration amplitude exceed a preset threshold, ensuring timely warning of potential safety hazards.

[0087] The embodiments of the present application further provide a corresponding vibration measurement device for a cable structure and a computer-readable medium for implementing the vibration measurement method for a cable structure provided in the embodiments of the present application.

[0088] Among them, the vibration measurement device for a cable structure includes a memory and a processor. The memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes a vibration measurement method for a cable structure according to any one of the embodiments of the present application.

[0089] See Figure 4 , this figure is a schematic diagram of a computer-readable medium provided by an embodiment of the present application. A computer program 411 is stored on the computer-readable medium 400. When the computer program 411 is executed by a processor, the steps of the vibration measurement method for the cable structure described above are implemented. Figure 1 of the vibration measurement method for the cable structure.

[0090] Note that in the context of this application, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0091] Note that the machine-readable medium described above in this application may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may, for example, but not be limited to, be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. And in this application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0092] The above computer-readable medium may be included in the above electronic device; or may exist separately without being assembled into the electronic device.

[0093] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims.

[0094] Although a number of specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of the present application. Certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0095] The above description is only a preferred embodiment of the present application and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, technical solutions formed by mutually replacing the above features with technical features (but not limited to) having similar functions disclosed in the present application.

Claims

1. A method for measuring the vibration of a cable structure, characterized in that: The method comprises: Acquire a target image sequence, wherein the target image sequence includes a plurality of target images arranged in time sequence, and each of the target images includes a cable structure; Generate a feature image corresponding to the target image by extracting features from the target image, wherein the features include at least one of corner features, texture features, and edge features; Determine the spatial position information of the cable structure in the real physical space by determining a positive sample area in the feature image, wherein the positive sample area is an area including the cable structure; The vibration frequency and vibration amplitude of the cable structure are determined according to the time sequence information and the spatial position information of the target image.

2. The method according to claim 1, characterized in that The determining of the spatial position information of the cable structure in the real physical space by determining the positive sample area in the feature image includes: By determining a positive sample area in the feature image, generating a mask in the feature image; The spatial position information of the cable structure in the real physical space is determined according to the coordinate information of the mask in the feature image and the camera parameters used to collect the target image sequence.

3. The method according to claim 2, characterized in that The step of generating a mask in the feature image by determining a positive sample area in the feature image comprises: Determine the probability distribution of the cable structure by determining a positive sample area and a negative sample area in the feature image, wherein the negative sample area is an area that does not include the cable structure; A mask is generated in the feature image according to the probability distribution of the cable structure.

4. The method according to claim 3, characterized in that The determining of the positive sample area and the negative sample area in the feature image includes: Allocating a plurality of preset anchor frames in the feature image, and determining a real target frame in the feature image, wherein the real target frame is a target frame including a cable structure; A preset anchor frame whose intersection-over-union ratio with the true target frame is greater than a first similarity threshold is determined as a positive sample area, and a preset anchor frame whose intersection-over-union ratio with the true target frame is less than a second similarity threshold is determined as a negative sample area, and the first similarity threshold is higher than the second similarity threshold.

5. The method according to claim 1, characterized in that The feature image includes a first feature image and a second feature image, and determining a positive sample area in the feature image includes: If the resolutions of the first feature image and the second feature image are both higher than a preset resolution threshold, and the similarity between the first feature image and the second feature image is higher than a third similarity threshold, a positive sample area in the first feature image or the second feature image is determined.

6. The method according to claim 1, characterized in that The step of extracting features from the target image to generate a feature image corresponding to the target image includes: Extracting multiple features in the target image through a multi-layer convolutional neural network to generate multiple sub-feature images, wherein each layer of the convolutional neural network is used to extract one feature of the target image; Path aggregation is performed on the multiple sub-feature images to obtain a feature image corresponding to the target image.

7. The method according to claim 1, characterized in that After performing the determining of the vibration frequency and the vibration amplitude of the cable structure, the method further comprises: When the vibration frequency is greater than a preset frequency threshold, and / or the vibration amplitude is greater than a preset amplitude threshold, an alarm indication is triggered.

8. A vibration measuring device for a cable structure, characterized in that: The device comprises: an image acquisition module, a feature generation module, a position determination module and a vibration measurement module; The image acquisition module is used to acquire a target image sequence, wherein the target image sequence includes a plurality of target images arranged in time sequence, and each of the target images includes a cable structure; The feature generation module is used to generate a feature image corresponding to the target image by extracting features from the target image, wherein the features include at least one of corner features, texture features and edge features; The position determination module is used to determine the spatial position information of the cable structure in the real physical space by determining the positive sample area in the feature image, wherein the positive sample area is the area including the cable structure; The vibration measurement module is used to determine the vibration frequency and vibration amplitude of the cable structure according to the time sequence information and the spatial position information of the target image.

9. A vibration measuring device for a cable structure, characterized in that: The device comprises: a memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the vibration measurement method of the cable structure according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the vibration measurement method of a cable structure according to any one of claims 1 to 7 is implemented.

Images