Vibration measurement method, device, equipment and medium for cable structure

Abstract

The embodiments of the present application provide a method, apparatus, device, and medium for measuring the vibration of a cable structure, relating to the field of computer vision technology. The method first generates a feature image by extracting features of the cable structure (such as corner features, texture features, and edge features) from a target image sequence. Subsequently, the spatial position information of the cable knot in the real physical space is determined based on the area of the feature image that includes the cable structure. Finally, the vibration of the cable structure, including the vibration frequency and amplitude, is analyzed based on the timing information and spatial position information of the target image. Therefore, the vibration measurement method for a cable structure disclosed in the embodiments of the present application avoids the limitation that traditional sensors need to be in physical contact with the cable structure, reduces interference or damage to the cable structure itself that may be caused by sensor installation, maintenance, or replacement, and increases the flexibility of vibration measurement of the cable structure.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device, equipment, and medium for measuring the vibration of a cable structure. Background Art

[0002] Cable structures are commonly used in projects such as photovoltaic panels, bridges, and towers. They rely primarily on high-strength steel cables to withstand tension and achieve large-span coverage. Due to prolonged exposure to outdoor environments, cable structures are susceptible to vibrations caused by strong winds and other meteorological conditions. Long-term vibration can cause fatigue damage to the cable structure and even lead to safety issues. Therefore, real-time vibration measurement of cable structures is crucial to ensuring their safety.

[0003] In related technologies, vibration measurement of cable structures usually relies on traditional sensor technologies, such as accelerometers, strain gauges, etc. These sensors are directly installed on the cable structure and measure the vibration of the cable structure through physical contact.

[0004] However, sensors are very susceptible to damage in outdoor environments and require regular maintenance and replacement, which not only consumes a lot of manpower and material resources, but also may affect the normal operation of the cable structure if the maintenance or replacement is not done properly. Summary of the Invention

[0005] Based on the above problems, the present application provides a vibration measurement method, device, equipment and medium for a cable structure, which can reduce the interference or damage to the cable structure itself caused by sensor installation, maintenance or replacement, and increase the flexibility of vibration measurement of the cable structure.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In a first aspect, the present application discloses a method for measuring vibration of a cable structure, the method comprising:

[0008] Acquire a target image sequence, wherein the target image sequence includes a plurality of target images arranged in time sequence, and each of the target images includes a cable structure;

[0009] Generating a feature image corresponding to the target image by extracting features from the target image, wherein the features include at least one of corner features, texture features, and edge features;

[0010] Determining spatial position information of the cable structure in real physical space by determining a positive sample region in the feature image, wherein the positive sample region is a region including the cable structure;

[0011] The vibration frequency and vibration amplitude of the cable structure are determined according to the time sequence information and the spatial position information of the target image.

[0012] Optionally, determining the spatial position information of the cable structure in the real physical space by determining the positive sample area in the feature image includes:

[0013] Generating a mask in the feature image by determining a positive sample area in the feature image;

[0014] The spatial position information of the cable structure in the real physical space is determined according to the coordinate information of the mask in the feature image and the camera parameters used to acquire the target image sequence.

[0015] Optionally, generating a mask in the feature image by determining a positive sample area in the feature image includes:

[0016] Determining the probability distribution of the cable structure by determining a positive sample area and a negative sample area in the feature image, wherein the negative sample area is an area that does not include the cable structure;

[0017] A mask is generated in the feature image according to the probability distribution of the cable structure.

[0018] Optionally, determining the positive sample area and the negative sample area in the feature image includes:

[0019] Allocating a plurality of preset anchor frames in the feature image, and determining a true target frame in the feature image, wherein the true target frame is a target frame including a cable structure;

[0020] Determine a preset anchor frame whose intersection-over-union ratio with the true target frame is greater than a first similarity threshold as a positive sample area, and determine a preset anchor frame whose intersection-over-union ratio with the true target frame is less than a second similarity threshold as a negative sample area, where the first similarity threshold is higher than the second similarity threshold.

[0021] Optionally, the feature image includes a first feature image and a second feature image, and determining the positive sample area in the feature image includes:

[0022] If the resolutions of the first feature image and the second feature image are both higher than a preset resolution threshold, and the similarity between the first feature image and the second feature image is higher than a third similarity threshold, a positive sample area in the first feature image or the second feature image is determined.

[0023] Optionally, the extracting features from the target image to generate a feature image corresponding to the target image includes:

[0024] Extracting multiple features from the target image through a multi-layer convolutional neural network to generate multiple sub-feature images, wherein each layer of the convolutional neural network is used to extract one feature of the target image;

[0025] Path aggregation is performed on the multiple sub-feature images to obtain a feature image corresponding to the target image.

[0026] Optionally, after determining the vibration frequency and vibration amplitude of the cable structure, the method further includes:

[0027] When the vibration frequency is greater than a preset frequency threshold, and / or the vibration amplitude is greater than a preset amplitude threshold, an alarm indication is triggered.

[0028] In a second aspect, the present application discloses a vibration measurement device for a cable structure, the device comprising: an image acquisition module, a feature generation module, a position determination module, and a vibration measurement module;

[0029] The image acquisition module is used to acquire a target image sequence, wherein the target image sequence includes a plurality of target images arranged in time sequence, and each target image includes a cable structure;

[0030] The feature generation module is configured to generate a feature image corresponding to the target image by extracting features from the target image, wherein the features include at least one of corner features, texture features, and edge features;

[0031] The position determination module is configured to determine the spatial position information of the cable structure in the real physical space by determining a positive sample area in the feature image, wherein the positive sample area is an area including the cable structure;

[0032] The vibration measurement module is used to determine the vibration frequency and vibration amplitude of the cable structure according to the time sequence information and the spatial position information of the target image.

[0033] Optionally, the position determination module includes: a mask generation module and an information determination module;

[0034] The mask generation module is used to generate a mask in the feature image by determining a positive sample area in the feature image;

[0035] The information determination module is used to determine the spatial position information of the cable structure in the real physical space based on the coordinate information of the mask in the feature image and the camera parameters used to acquire the target image sequence.

[0036] Optionally, the mask generation module includes: a distribution determination submodule and a mask generation submodule;

[0037] The distribution determination submodule is configured to determine the probability distribution of the cable structure by determining a positive sample region and a negative sample region in the feature image, wherein the negative sample region is a region that does not include the cable structure;

[0038] The mask generation submodule is used to generate a mask in the feature image according to the probability distribution of the cable structure.

[0039] Optionally, the distribution determination submodule is specifically used to: allocate multiple preset anchor frames in the feature image, and determine a true target frame in the feature image, wherein the true target frame is a target frame including a rope structure; determine a preset anchor frame whose intersection-with-union ratio with the true target frame is greater than a first similarity threshold as a positive sample area, and determine a preset anchor frame whose intersection-with-union ratio with the true target frame is less than a second similarity threshold as a negative sample area, and the first similarity threshold is higher than the second similarity threshold.

[0040] Optionally, the feature image includes a first feature image and a second feature image, and the position determination module is specifically used to: if the resolutions of the first feature image and the second feature image are both higher than a preset resolution threshold, and the similarity between the first feature image and the second feature image is higher than a third similarity threshold, then determine the positive sample area in the first feature image or the second feature image.

[0041] Optionally, the feature generation module includes: a first generation module and a second generation module;

[0042] The first generating module is configured to extract multiple features from the target image through a multi-layer convolutional neural network to generate multiple sub-feature images, wherein each layer of the convolutional neural network is configured to extract one feature of the target image;

[0043] The second generating module is configured to perform path aggregation on the multiple sub-feature images to obtain a feature image corresponding to the target image.

[0044] Optionally, the vibration measurement device for the cable structure further comprises: an alarm indication module;

[0045] The alarm indication module is configured to trigger an alarm indication when the vibration frequency is greater than a preset frequency threshold and / or the vibration amplitude is greater than a preset amplitude threshold.

[0046] In a third aspect, the present application discloses a vibration measurement device for a cable structure, the device comprising: a memory and a processor;

[0047] The memory is used to store programs;

[0048] The processor is used to execute the program to implement the various steps of the vibration measurement method of the cable structure as described in the first aspect.

[0049] In a fourth aspect, the present application discloses a computer-readable medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the various steps of the vibration measurement method of the cable structure as described in the first aspect.

[0050] Compared with the existing technology, this application has the following beneficial effects:

[0051] The embodiments of the present application provide a method, apparatus, device, and medium for measuring the vibration of a cable structure. The method first generates a feature image by extracting features of the cable structure (such as corner features, texture features, and edge features) from a target image sequence. Subsequently, the spatial position information of the cable knot in real physical space is determined based on the region of the feature image that includes the cable structure. Finally, the vibration of the cable structure, including the vibration frequency and amplitude, is analyzed based on the timing information and spatial position information of the target image. Thus, the cable structure vibration measurement method disclosed in the embodiments of the present application avoids the limitation of traditional sensors requiring physical contact with the cable structure, reduces interference or damage to the cable structure itself that may be caused by sensor installation, maintenance, or replacement, and increases the flexibility of vibration measurement of the cable structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 A flow chart of a vibration measurement method for a cable structure provided in an embodiment of the present application;

[0054] Figure 2 A flow chart of another method for measuring vibration of a cable structure provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of a vibration measuring device for a cable structure provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of a computer-readable medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] As previously mentioned, vibration measurement of cable structures typically relies on traditional sensor technologies, such as accelerometers and strain gauges. These sensors are mounted directly on the cable structure, measuring its vibration through physical contact. However, these sensors are highly susceptible to damage in outdoor environments and require regular maintenance and replacement, which not only consumes significant labor and resources but also can affect the normal operation of the cable structure if improperly maintained or replaced.

[0058] After research, the inventors have proposed a method, apparatus, device, and medium for measuring the vibration of cable structures. First, a feature image is generated by extracting cable structure features (such as corner features, texture features, and edge features) from a target image sequence. Subsequently, the spatial location information of the cable knot in real physical space is determined based on the region containing the cable structure in the feature image. Finally, the vibration of the cable structure, including frequency and amplitude, is analyzed based on the temporal and spatial location information of the target image. Thus, the cable structure vibration measurement method disclosed in the embodiments of the present application avoids the limitation of traditional sensors requiring physical contact with the cable structure, reduces potential interference or damage to the cable structure caused by sensor installation, maintenance, or replacement, and increases the flexibility of cable structure vibration measurement. Furthermore, the cable structure vibration measurement method disclosed in the embodiments of the present application utilizes a multi-layer convolutional neural network and path aggregation technology to automatically extract and process the extracted cable structure features, improving the accuracy and intelligence of cable structure vibration measurement. Furthermore, the cable structure vibration measurement method disclosed in the embodiments of the present application can monitor the vibration status in real time and automatically trigger an alarm when the vibration frequency and / or amplitude exceed preset thresholds, ensuring timely warning of potential safety hazards.

[0059] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0060] See also Figure 1 , which is a flow chart of a vibration measurement method for a cable structure provided in an embodiment of the present application. The method includes:

[0061] S101: Acquire a target image sequence, where the target image sequence includes a plurality of target images arranged in time sequence, and each target image includes a cable structure.

[0062] A target image sequence is acquired using a camera, drone, or other visual sensor. A target image sequence consists of multiple target images arranged in time sequence, each of which includes a cable structure. Furthermore, these target images can be captured at fixed intervals, for example, 10 images can be taken every 1 second.

[0063] It is understood that the resolution of the target image is crucial for subsequent feature extraction. Therefore, a high-resolution camera is typically used to ensure that the target image remains clear in complex environments (such as lighting changes, background interference, etc.). This application does not impose any restrictions on the specific resolution.

[0064] S102: Generate a feature image corresponding to the target image by extracting features from the target image, where the features include at least one of corner features, texture features, and edge features.

[0065] Features are specific attributes or structures that can be extracted from a target image to describe its content. These include corner features, texture features, and edge features. Corner features identify distinct corners in a target image and can help identify the shape and structure of objects (e.g., cable structures) within the image. Texture features identify repetitive patterns or surface characteristics within a region of the image, such as smoothness or roughness. Edge features identify the outlines of objects within the image and can help identify their boundaries.

[0066] A feature image is a new image generated based on the extracted features. It contains only the extracted feature information (such as corners, textures, or edges), while ignoring other redundant information. Thus, by extracting features from the target image, the original target image is converted into a more concise and meaningful representation (i.e., a feature image) for easier subsequent processing.

[0067] S103: Determine the spatial position information of the cable structure in the real physical space by determining a positive sample region in the feature image, where the positive sample region is a region including the cable structure.

[0068] In some specific implementations, the method for determining positive sample regions in a feature image is as follows: First, multiple preset anchor boxes are allocated in the feature image. These preset anchor boxes are used as candidate regions to detect possible targets (i.e., cable structures). Second, a ground truth box containing the cable structure is determined in the feature image through a manual annotation method. Subsequently, the preset anchor box whose intersection over union (IoU) with the ground truth box is greater than a first similarity threshold is determined to be a positive sample region. Here, IoU is an indicator that measures the degree of overlap between two anchor boxes and has a value range of [0, 1]. The larger the IoU, the more similar the two anchor boxes are. For example, if the IoU value between a preset anchor box and any ground truth box is greater than a first similarity threshold (e.g., 0.8), it is marked as a positive sample region.

[0069] After determining the positive sample regions in the feature image, the probability that each positive sample region contains a true cable structure is calculated, and the probability distribution of the cable structure is obtained. Subsequently, based on the probability distribution of the cable structure, a target mask is generated in the feature image. The target mask is a binary image that identifies the precise location and shape of the cable structure in the feature image. It provides a clear boundary to distinguish the cable structure from the background or other structures.

[0070] Finally, by determining the position of the cable structure within the target reticle, the spatial position of the cable structure in real physical space can be determined. In some specific implementations, the minimum bounding rectangle of the target reticle is first determined, and the coordinates of the upper left corner (x_min, y_min) and the lower right corner (x_max, y_max) of the minimum bounding rectangle are recorded to determine the position of the target reticle. Subsequently, the position of the target reticle is converted into the spatial position of the cable structure in real physical space based on the camera's intrinsic parameters (such as focal length, principal point, etc.) and extrinsic parameters (such as rotation matrix, translation vector, etc.). For example, assuming the position of the target reticle is (x_img, y_img), the spatial position of the cable structure in real physical space (X_real, Y_real, Z_real) can be determined using the projection matrix composed of the camera's intrinsic and extrinsic parameters and the target reticle's position information.

[0071] S104: Determine the vibration frequency and vibration amplitude of the cable structure according to the time sequence information and spatial position information of the target image.

[0072] In step S104, for each target image in the target image sequence, the spatial position information of the cable structure in real physical space must be determined, and then the spatial position change of the cable structure in real physical space must be determined. This spatial position change can be described by a displacement vector, which represents the direction and distance the cable structure moves from one location to another.

[0073] Subsequently, the vibration frequency and amplitude of the cable structure can be determined based on the displacement vector and actual size information of the cable structure (such as parameters such as length and diameter, which can be obtained through design documents, CAD models or on-site measurements).

[0074] In summary, the embodiments of the present application provide a method for measuring the vibration of a cable structure. This method first generates a feature image by extracting features of the cable structure (such as corner features, texture features, and edge features) from a target image sequence. Subsequently, the spatial position information of the cable knot in real physical space is determined based on the area of the feature image that includes the cable structure. Finally, the vibration of the cable structure, including the vibration frequency and amplitude, is analyzed based on the timing information and spatial position information of the target image. As a result, the method for measuring the vibration of a cable structure disclosed in the embodiments of the present application avoids the limitation that traditional sensors need to be in physical contact with the cable structure, reduces interference or damage to the cable structure itself that may be caused by sensor installation, maintenance, or replacement, and increases the flexibility of vibration measurement of the cable structure.

[0075] See also Figure 2 , which is a flow chart of another method for measuring the vibration of a cable structure provided in an embodiment of the present application. The method includes:

[0076] S201: Acquire a target image sequence, where the target image sequence includes a plurality of target images arranged in time sequence, and each target image includes a cable structure.

[0077] It is understandable that this step is similar to step S101 and will not be described in detail here.

[0078] S202: Generate a feature map by extracting features from the target image.

[0079] To efficiently and accurately extract features, convolutional neural network (CNN) technology can be employed. Specifically, the cable structure vibration measurement method provided in the embodiments of this application uses a convolutional layer to slide a convolution kernel over a target image, performing a convolution operation on the target image. This operation extracts edge features, corner features, texture features, and other features from the target image, and generates a set of feature maps (each convolutional neural network layer extracts a feature of the target image, and each feature map represents a feature of the target image). Subsequently, the spatial size of this set of feature maps is increased through a deconvolution layer.

[0080] It should be noted that in the process of extracting features through convolutional neural network technology, multiple convolutional layers can be connected with residual connections (Residual Connection). By directly passing the output of the previous convolutional layer to the subsequent convolutional layer, the gradient vanishing problem in the deep network is avoided, ensuring that the features can be effectively transferred to deeper network layers.

[0081] It should also be noted that the bottleneck structure can also be used to reduce the number of channels in the middle layer of the convolutional layer, reduce the number of parameters and computational complexity of the convolutional neural network, and thus reduce the consumption of computing resources while maintaining the feature extraction capability.

[0082] It should also be noted that in the process of extracting features through convolutional neural network technology, the CXF unit (an optimized convolution module) can also be used to improve the efficiency of feature extraction by reducing redundant parameters and optimizing the network structure.

[0083] It's also worth noting that features can also be extracted using depthwise separable convolution. This technique breaks down the standard convolution into two steps: first, performing a separate convolution operation on each input channel, and second, fusing the channel information using a 1×1 convolution. This significantly reduces computational effort while maintaining feature extraction capabilities.

[0084] It should also be noted that features can also be extracted through the dilated convolution technique, that is, by inserting holes in the convolution kernel, the receptive field of the convolution kernel is expanded, and a wider range of contextual information is captured, thereby being able to extract a wider range of spatial features.

[0085] S203: Performing scale unification processing on the feature map to obtain a processed feature map.

[0086] It should be noted that the cable structure may have different sizes. If only a single-scale feature map is used, it may not be possible to effectively capture objects of all sizes (i.e., the cable structure) at the same time.

[0087] In some specific implementations, an appropriate pooling strategy is first applied to the feature maps output by each convolutional layer to adjust their size. Next, the pooled feature maps of different scales are concatenated. This concatenation allows for the simultaneous utilization of high-resolution features in shallow layers (suitable for detecting small objects) and low-resolution features in deep layers (suitable for detecting large objects), thereby improving detection capabilities for objects of varying sizes.

[0088] S204: Determine the positive sample region and the negative sample region in the processed feature map, and the weight corresponding to each region, wherein the positive sample region is the region including the cable structure, and the negative sample region is the region not including the cable structure.

[0089] In the vibration measurement method of the cable structure provided in an embodiment of the present application, the positive sample area and the negative sample area in the processed feature map, as well as the weight corresponding to each area, can be determined by intelligently allocating anchor boxes. Specifically, first, the IoU between each anchor box and the true target box (the target box that has been determined to contain the cable structure) is calculated. Subsequently, the positive sample area and the negative sample area are determined based on the IoU value. For example, if the IoU value between an anchor box and any true target box is greater than a first similarity threshold (such as 0.8), it is marked as a positive sample area. If the IoU value between an anchor box and any true target box is less than a second similarity threshold (such as 0.2), it is marked as a negative sample area.

[0090] It should be noted that this application does not impose any restrictions on the first and second similarity thresholds. It is sufficient to ensure that the first similarity threshold is greater than the second similarity threshold. Furthermore, the first and second similarity thresholds can be adaptively adjusted to reduce the probability of overlooking existing cable structures (missed detection) or misidentifying background areas as cable structures (false detection), thereby improving detection accuracy.

[0091] It should also be noted that in the process of determining the positive sample area and the negative sample area from each processed feature map, the calculation accuracy can also be dynamically adjusted. For example, if the resolution of the processed feature map is 1280×900 (i.e., the resolution is relatively high), if the positive sample area and the negative sample area are accurately determined for each processed feature map, a large amount of computing resources will be consumed. Therefore, in one implementation, in order to avoid unnecessary repeated calculations, a similarity comparison mechanism can be introduced. If the resolutions of the two processed feature maps are both higher than the preset resolution threshold, and the similarity of the two processed feature maps is higher than the third similarity threshold (such as 0.9), they are considered to be highly similar, and only one of them needs to be used to determine the positive sample area and the negative sample area. In this way, the amount of calculation can be significantly reduced while maintaining a high detection accuracy. In another implementation, selective processing can also be performed based on time frequency. For example, if four processed feature maps are obtained every second, in order to reduce the amount of calculation, you can choose to perform a detailed positive and negative sample area determination every certain number of processed feature maps. For example, you can select the first and third (or second and fourth) of the four processed feature maps to determine the positive and negative sample areas.

[0092] S205: Perform path aggregation on the processed feature map to obtain a fused feature map.

[0093] The purpose of path aggregation is to combine the processed feature maps at different levels, which can simultaneously capture low-level details (such as edges and textures) and high-level semantic information (such as object categories and overall structures) in the target image, thereby improving the detection and recognition capabilities of targets of different scales.

[0094] Path aggregation typically consists of two main parts: a bottom-up path and a top-down path. The bottom-up path begins with the processed input feature map and gradually extracts increasingly abstract high-level features through a series of convolutional layers. These high-level features typically contain more semantic information but may lose some details. The top-down path begins with high-level feature maps and gradually restores spatial resolution through upsampling (such as deconvolution or bilinear interpolation), combining high-level semantic information with low-level detail information. This preserves semantic information while recovering more detail. To better fuse features at different levels, lateral connections are often added between the bottom-up and top-down paths. These connections fuse feature maps of the same scale through element-wise addition or concatenation to produce a fused feature map.

[0095] S206: Determine the probability distribution of the cable structure according to the fused feature map, the positive sample area, the negative sample area, and the weight corresponding to each area.

[0096] It is understood that the feature map has been obtained and which regions are likely to be locations of cable structures through the processing of steps S201 to S205. In step S206, this information can be integrated to calculate the probability that each region contains a true cable structure.

[0097] S207: Generate a target mask in the fused feature map according to the probability distribution of the cable structure.

[0098] If the probability distribution of a cable structure in a certain area determined in step S206 is higher than a preset probability threshold (e.g., 0.5), the area is determined to be a cable structure and marked as 1; otherwise, it is marked as 0 (indicating that the area is not a cable structure). Subsequently, the above binarization result is converted into a target mask, where 1 represents the target area and 0 represents the background area.

[0099] It should be noted that to improve the quality of the target mask, some refinement and optimization operations are usually required. For example, morphological operations such as dilation (expanding the foreground area to help connect broken parts) and erosion (shrinking the foreground area to help remove small noise points) are used to remove noise in the mask and fill holes to make the mask smoother and more continuous. This application does not limit this.

[0100] S208: Determine the spatial position information of the cable structure in the real physical space by determining the position information of the target mask.

[0101] It should be noted that when determining the spatial position of the cable structure in real physical space, it is also necessary to consider the impact of environmental factors such as temperature and humidity on the camera's intrinsic and extrinsic parameters. This is because temperature changes can cause the camera lens material to expand or contract, thereby changing the focal length and other optical parameters. Furthermore, high humidity can cause condensation on the lens surface, affecting the quality of the captured target image and, in turn, the accuracy of the camera parameters. To address the impact of temperature, experimentally determine the camera's intrinsic and extrinsic parameters at different temperatures and establish corresponding calibration curves. In practical applications, a temperature sensor can be used to monitor the ambient temperature and adjust the camera parameters in real time based on the pre-established calibration curves. To address the impact of humidity, image preprocessing techniques (such as dehazing algorithms) can be used to improve the quality of the target image and avoid errors caused by humidity. Furthermore, it is necessary to consider deviations caused by equipment aging and improper installation. Improper camera installation can lead to deviations in measured angles and distances. Therefore, the camera can be checked and reinstalled at preset intervals to ensure its correct position and angle. Geometric transformation methods can also be used to correct the perspective of the target image to eliminate deviations caused by improper camera installation. This is not a limitation of this application.

[0102] S209: Determine a displacement vector of the cable structure according to the time sequence information of the multiple target images and the spatial position information of the cable structure corresponding to the multiple target images in the real physical space.

[0103] Steps S201-S208 above determine the spatial position of the cable structure in real physical space using the target image. In S209, the cable structure's spatial position change in real physical space is determined by analyzing multiple chronologically arranged target images. This in turn calculates the cable structure's displacement vector (i.e., the result of the vector transformation). This displacement vector describes the direction and distance of movement of the cable structure.

[0104] S210: Determine the vibration frequency and vibration amplitude of the cable structure according to the actual size information and displacement vector of the cable structure.

[0105] In a specific implementation, the vibration frequency of the cable structure is determined as follows:

[0106] First, the time series data of the displacement vectors is denoised and normalized to obtain preprocessed data, ensuring smoothness and the absence of outliers. Second, a fast Fourier transform algorithm is applied to the preprocessed data to convert the time-domain signal into a frequency-domain signal, obtaining a frequency spectrum. Finally, peaks in the frequency spectrum with amplitudes exceeding a preset threshold are identified. The frequency corresponding to these peaks is the vibration frequency of the cable structure.

[0107] In a specific implementation, the vibration amplitude of the cable structure is determined as follows:

[0108] First, based on the actual size information of the cable structure (such as the diameter D of the cable structure) and the pixel width W of the cable structure in the image, a proportional coefficient α is established between the pixel distance and the actual physical distance, where α = D / W.

[0109] Next, the actual physical displacement (including the actual physical displacement dx in the horizontal direction and the actual physical displacement dy in the vertical direction) is determined based on the proportional coefficient α and the horizontal and vertical components of the displacement vector. The actual physical displacement dx in the horizontal direction is the product of the proportional coefficient α and the horizontal component of the displacement vector, and the actual physical displacement dy in the vertical direction is the product of the proportional coefficient α and the vertical component of the displacement vector.

[0110] Subsequently, the instantaneous vibration amplitude A is determined based on the actual physical displacement dx in the horizontal direction and the actual physical displacement dy in the vertical direction. The formula for determining the instantaneous vibration amplitude A is specifically shown in the following formula (1):

[0111] (1)

[0112] Then, after determining the instantaneous vibration amplitudes corresponding to multiple time points, the maximum value is taken as the vibration amplitude of the cable structure. Alternatively, after determining the instantaneous vibration amplitudes corresponding to multiple time points, the corresponding root mean square value A is calculated. RMS As the vibration amplitude of the cable structure. RMS The calculation formula is shown in the following formula (2):

[0113] (2)

[0114] Among them, A RMS is the root mean square value, N is the total number of time points, A i is the instantaneous vibration amplitude at the i-th time point.

[0115] S211: When the vibration frequency is greater than a preset frequency threshold, and / or the vibration amplitude is greater than a preset amplitude threshold, triggering an alarm indication.

[0116] It should be noted that the above-mentioned preset frequency threshold can be 5Hz, and the above-mentioned preset amplitude threshold can be 0.05m. This application does not limit the specific preset frequency threshold and preset amplitude threshold. By setting a reasonable preset frequency threshold and / or preset amplitude threshold, an alarm can be issued in time when abnormal vibration is detected so that appropriate measures can be taken for maintenance or processing. The alarm indication can be implemented in a variety of ways, such as sending an email, SMS notification, interface pop-up prompt, etc.

[0117] In summary, this application discloses a method for measuring the vibration of a cable structure. First, a feature image is generated by extracting cable structure features (such as corner features, texture features, and edge features) from a target image sequence. Subsequently, the spatial location information of the cable knot in real physical space is determined based on the region containing the cable structure in the feature image. Finally, the vibration of the cable structure, including frequency and amplitude, is analyzed based on the temporal and spatial location information of the target image. Thus, the cable structure vibration measurement method disclosed in the embodiments of this application avoids the limitation of traditional sensors requiring physical contact with the cable structure, reduces potential interference or damage to the cable structure caused by sensor installation, maintenance, or replacement, and increases the flexibility of cable structure vibration measurement. Furthermore, the cable structure vibration measurement method disclosed in the embodiments of this application utilizes a multi-layer convolutional neural network and path aggregation technology to automatically extract and process the extracted cable structure features, improving the accuracy and intelligence of cable structure vibration measurement. Furthermore, the cable structure vibration measurement method disclosed in the embodiments of this application can monitor the vibration status in real time and automatically trigger an alarm when the vibration frequency and / or amplitude exceed preset thresholds, ensuring timely warning of potential safety hazards.

[0118] See also Figure 3 , which is a schematic diagram of a cable structure vibration measurement device provided by an embodiment of the present application. The cable structure vibration measurement device 300 includes: an image acquisition module 301, a feature generation module 302, a position determination module 303 and a vibration measurement module 304.

[0119] An image acquisition module 301 is configured to acquire a target image sequence, where the target image sequence includes a plurality of target images arranged in a time sequence, and each target image includes a cable structure;

[0120] A feature generation module 302 is configured to generate a feature image corresponding to the target image by extracting features from the target image, where the features include at least one of corner features, texture features, and edge features;

[0121] A position determination module 303 is configured to determine the spatial position information of the cable structure in the real physical space by determining a positive sample region in the feature image, where the positive sample region is a region including the cable structure;

[0122] The vibration measurement module 304 is used to determine the vibration frequency and vibration amplitude of the cable structure according to the time sequence information and spatial position information of the target image.

[0123] In some specific implementations, the position determination module 303 includes: a mask generation module and an information determination module;

[0124] A mask generation module is used to generate a mask in the feature image by determining a positive sample area in the feature image;

[0125] The information determination module is used to determine the spatial position information of the cable structure in the real physical space based on the coordinate information of the mask in the feature image and the camera parameters used to collect the target image sequence.

[0126] In some specific implementations, the mask generation module includes: a distribution determination submodule and a mask generation submodule;

[0127] a distribution determination submodule, configured to determine a probability distribution of the cable structure by determining a positive sample region and a negative sample region in the feature image, wherein the negative sample region is a region that does not include the cable structure;

[0128] The mask generation submodule is used to generate a mask in the feature image according to the probability distribution of the cable structure.

[0129] In some specific implementations, the distribution determination submodule is specifically used to: allocate multiple preset anchor frames in the feature image, and determine the true target frame in the feature image, wherein the true target frame is a target frame including a rope structure; determine the preset anchor frame whose intersection-with-union ratio with the true target frame is greater than a first similarity threshold as a positive sample area, and determine the preset anchor frame whose intersection-with-union ratio with the true target frame is less than a second similarity threshold as a negative sample area, and the first similarity threshold is higher than the second similarity threshold.

[0130] In some specific implementations, the feature image includes a first feature image and a second feature image, and the position determination module 303 is specifically used to: if the resolutions of the first feature image and the second feature image are both higher than a preset resolution threshold, and the similarity between the first feature image and the second feature image is higher than a third similarity threshold, then determine the positive sample area in the first feature image or the second feature image.

[0131] In some specific implementations, the feature generation module 302 includes: a first generation module and a second generation module;

[0132] A first generation module is configured to extract multiple features from a target image through a multi-layer convolutional neural network to generate multiple sub-feature images, wherein each layer of the convolutional neural network is configured to extract one feature of the target image;

[0133] The second generation module is used to perform path aggregation on multiple sub-feature images to obtain a feature image corresponding to the target image.

[0134] In some specific implementations, the cable structure vibration measurement device 300 further includes: an alarm indication module;

[0135] The alarm indication module is used to trigger an alarm indication when the vibration frequency is greater than a preset frequency threshold and / or the vibration amplitude is greater than a preset amplitude threshold.

[0136] In summary, this application discloses a cable structure vibration measurement device. First, a feature image is generated by extracting cable structure features (such as corner features, texture features, and edge features) from a target image sequence. Subsequently, the spatial location information of the cable knot in real physical space is determined based on the region containing the cable structure in the feature image. Finally, the vibration of the cable structure, including frequency and amplitude, is analyzed based on the temporal and spatial location information of the target image. Thus, the cable structure vibration measurement device disclosed in the embodiments of this application avoids the limitation of traditional sensors requiring physical contact with the cable structure, reduces potential interference or damage to the cable structure caused by sensor installation, maintenance, or replacement, and increases the flexibility of cable structure vibration measurement. Furthermore, the cable structure vibration measurement device disclosed in the embodiments of this application utilizes a multi-layer convolutional neural network and path aggregation technology to automatically extract and process the extracted cable structure features, improving the accuracy and intelligence of cable structure vibration measurement. Furthermore, the cable structure vibration measurement device disclosed in the embodiments of this application can monitor the vibration status in real time and automatically trigger an alarm when the vibration frequency and / or amplitude exceed preset thresholds, ensuring timely warning of potential safety hazards.

[0137] The embodiments of the present application also provide a corresponding cable structure vibration measurement device and a computer-readable medium for implementing the cable structure vibration measurement method provided in the embodiments of the present application.

[0138] The cable structure vibration measurement device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute instructions or codes so that the device executes a cable structure vibration measurement method according to any embodiment of the present application.

[0139] See also Figure 4 , which is a schematic diagram of a computer readable medium provided by an embodiment of the present application. The computer readable medium 400 stores a computer program 411, which implements the above-mentioned Figure 1 Steps of the vibration measurement method of the cable structure.

[0140] It should be noted that in the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or used in conjunction with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] It should be noted that the machine-readable medium mentioned above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0142] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0143] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

[0144] Although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this application. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0145] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for measuring the vibration of a cable structure, characterized in that: The method comprises: Acquire a target image sequence, wherein the target image sequence includes a plurality of target images arranged in time sequence, and each of the target images includes a cable structure; A multi-layer convolutional neural network is used to extract multiple features from the target image using a depthwise separable convolution technique to generate multiple sub-feature images, wherein the multiple convolutional layers of the multi-layer convolutional neural network are constructed through residual connections, the multi-layer convolutional neural network has a bottleneck structure, and the convolution kernels of the multi-layer convolutional neural network have holes, and each layer of the convolutional neural network is used to extract a feature of the target image, wherein the feature includes at least one of a corner feature, a texture feature, and an edge feature; Performing top-down and bottom-up path aggregation on the multiple sub-feature images to obtain a feature image corresponding to the target image; generating a mask in the feature image by determining a positive sample region in the feature image, wherein the positive sample region is a region including a cable structure; Determining spatial position information of the cable structure in real physical space based on coordinate information of the mask in the feature image and camera parameters used to acquire the target image sequence; determining a vibration frequency and a vibration amplitude of the cable structure according to the time sequence information and the spatial position information of the target image; The vibration frequency is determined by: determining a displacement vector of the cable structure based on the time series information of the target image and the spatial position information; applying a fast Fourier transform algorithm to the time series data of the displacement vector to obtain a frequency spectrum; and determining a peak point in the frequency spectrum whose amplitude exceeds a preset amplitude threshold as the vibration frequency; The method for determining the vibration amplitude includes: determining a proportional coefficient based on actual size information of the cable structure and a pixel width of the cable structure in the target image; determining the actual physical displacement of the cable structure based on the proportional coefficient and the horizontal component and the vertical component of the displacement vector; determining the instantaneous vibration amplitude of the cable structure based on the actual physical displacement; and determining the maximum value of multiple instantaneous vibration amplitudes as the vibration amplitude.

2. The method according to claim 1, characterized in that The step of generating a mask in the feature image by determining a positive sample area in the feature image includes: Determining the probability distribution of the cable structure by determining a positive sample area and a negative sample area in the feature image, wherein the negative sample area is an area that does not include the cable structure; A mask is generated in the feature image according to the probability distribution of the cable structure.

3. The method according to claim 2, characterized in that The determining of the positive sample area and the negative sample area in the feature image includes: Allocating a plurality of preset anchor frames in the feature image, and determining a true target frame in the feature image, wherein the true target frame is a target frame including a cable structure; Determine a preset anchor frame whose intersection-over-union ratio with the true target frame is greater than a first similarity threshold as a positive sample area, and determine a preset anchor frame whose intersection-over-union ratio with the true target frame is less than a second similarity threshold as a negative sample area, where the first similarity threshold is higher than the second similarity threshold.

4. The method according to claim 1, wherein The feature image includes a first feature image and a second feature image, and determining a positive sample area in the feature image includes: If the resolutions of the first feature image and the second feature image are both higher than a preset resolution threshold, and the similarity between the first feature image and the second feature image is higher than a third similarity threshold, a positive sample area in the first feature image or the second feature image is determined.

5. The method according to claim 1, wherein After performing the step of determining the vibration frequency and the vibration amplitude of the cable structure, the method further comprises: When the vibration frequency is greater than a preset frequency threshold, and / or the vibration amplitude is greater than a preset amplitude threshold, an alarm indication is triggered.

6. A vibration measuring device for a cable structure, characterized in that: The device includes: an image acquisition module, a first generation module, a second generation module, a mask generation module, an information determination module and a vibration measurement module; The image acquisition module is used to acquire a target image sequence, wherein the target image sequence includes a plurality of target images arranged in time sequence, and each target image includes a cable structure; The first generation module is configured to extract multiple features from the target image using a depthwise separable convolution technique through a multi-layer convolutional neural network to generate multiple sub-feature images, wherein the multiple convolutional layers of the multi-layer convolutional neural network are constructed through residual connections, the multi-layer convolutional neural network has a bottleneck structure, and the convolution kernels of the multi-layer convolutional neural network have holes, and each layer of the convolutional neural network is configured to extract a feature of the target image, wherein the feature includes at least one of a corner feature, a texture feature, and an edge feature; The second generating module is configured to perform top-down and bottom-up path aggregation on the plurality of sub-feature images to obtain a feature image corresponding to the target image; The mask generation module is used to generate a mask in the feature image by determining a positive sample area in the feature image, wherein the positive sample area is an area including the cable structure; The information determination module is used to determine the spatial position information of the cable structure in the real physical space based on the coordinate information of the mask in the feature image and the camera parameters used to acquire the target image sequence; The vibration measurement module is used to determine the vibration frequency and vibration amplitude of the cable structure based on the time sequence information and the spatial position information of the target image; The vibration frequency determination unit is configured to: determine a displacement vector of the cable structure based on the time series information of the target image and the spatial position information; apply a fast Fourier transform algorithm to the time series data of the displacement vector to obtain a frequency spectrum; and determine a peak point in the frequency spectrum whose amplitude exceeds a preset amplitude threshold as the vibration frequency; The vibration amplitude determination unit is used to: determine a proportional coefficient based on actual size information of the cable structure and a pixel width of the cable structure in the target image; determine an actual physical displacement of the cable structure based on the proportional coefficient and a horizontal component and a vertical component of the displacement vector; determine an instantaneous vibration amplitude of the cable structure based on the actual physical displacement; and determine a maximum value among a plurality of instantaneous vibration amplitudes as the vibration amplitude.

7. A vibration measuring device for a cable structure, characterized in that: The device includes: a memory and a processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the vibration measurement method for a cable structure according to any one of claims 1 to 5.

8. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the vibration measurement method of a cable structure according to any one of claims 1 to 5 is implemented.

Images