Cable force monitoring method and device, storage medium and product
Through improved linear detection algorithms and computer vision technology, the problem of insufficient real-time and accuracy of traditional zoom force detection methods is solved, and efficient and accurate cable force monitoring is achieved, which is suitable for engineering applications.
Patent Information
- Application Number
- CN202410981347.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The traditional zoom force detection method lacks real-time and accuracy, especially in contactless measurement technology, there are problems such as high equipment cost, poor applicability, and susceptibility to environmental and lighting conditions.
Using an improved linear detection algorithm, the cable center line is determined through multiple search steps, including thick search, line search and sub-pixel level detailed search, ensuring the accuracy and efficiency of cable force calculation, and using computer vision technology to extract the cable from video images The vibration displacement time curve of the cable and calculate the cable force.
It realizes high-precision and real-time cable force monitoring, which can synchronously detect the cable force of multiple cables, improves monitoring efficiency and accuracy, and is suitable for real-time and long-term monitoring needs in engineering applications.
Smart Images

Figure CN118913503A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of bridge monitoring, and in particular relates to a cable force monitoring method, equipment, storage medium and product. Background Art
[0002] As the main load-bearing component of the cable-stayed bridge, the cable provides multi-point elastic support for the main beam, and together with the cable towers and auxiliary piers, it forms a hyperstatic structural system. The vibration characteristics of the cable reflect and affect the performance of the cable-stayed bridge structure to a certain extent. The cable of the cable-stayed bridge faces the phenomenon of cable force degradation under long-term environmental corrosion and vehicle cyclic loads. The vibration frequency is a key indicator for evaluating the cable force. How to accurately and efficiently obtain the vibration characteristics of the cable under natural conditions or vehicle excitation is of great significance and practical engineering value for the structural health status and performance evaluation of the cable-stayed bridge.
[0003] Traditional methods for extracting structural vibration characteristic parameters (such as natural frequency, modal vibration shape, etc.) usually rely on placing contact sensors on the surface of the structure, such as accelerometers, displacement sensors, etc., to collect the dynamic response of the structure under environmental or artificial excitation, and then perform frequency domain conversion on the measured acceleration or displacement time history curves to obtain its vibration frequencies of various orders.
[0004] With the continuous development of image acquisition equipment and video image processing technology, vibration identification methods based on computer vision have attracted extensive attention from researchers at home and abroad as an emerging non-contact measurement technology. Compared with traditional measurement methods, it has many advantages such as non-contact, high efficiency and convenience, high applicability and strong real-time performance. It can quickly obtain parameters such as structural dynamic displacement, natural frequency, modal vibration shape, etc., and provides a new solution for bridge displacement monitoring and modal identification.
[0005] At present, non-contact measurement methods for cable tension include microwave radar monitoring, Doppler laser vibrometer, template matching algorithm, optical flow estimation, digital image correlation (DIC) technology, etc. Microwave radar and laser Doppler vibrometer can realize multi-point synchronous vibration measurement, and have also been applied to the vibration test of cable-stayed bridge cables. However, when measuring large structures, multiple devices need to be arranged to work together; before actual measurement, it is also necessary to manually set up target and other characteristic areas. In a strict sense, non-contact measurement of the entire process is not achieved. In addition, the high price of the equipment leads to an increase in the detection cost, and the economy and applicability need to be further considered. The accuracy of the calculation results of the template matching algorithm depends on the quality of the image, which is easily affected by lighting conditions, complex background, and target occlusion. The displacement response obtained is pixel-level, which does not meet the accuracy required in actual engineering. DIC technology can achieve close-range displacement measurement without a target, but for objects that lack obvious texture or structural features, it is necessary to create speckles on their surfaces, which conflicts with the aesthetic design in engineering. In addition, DIC technology is easily affected by environmental factors and is generally suitable for close-range measurements in stable environments and obvious surface features. It is difficult to achieve long-term stable displacement monitoring and measurement, and it is difficult to be universal in engineering applications. The optical flow estimation method needs to calculate the displacement of each pixel in the image, which requires a huge amount of calculation. Its calculation accuracy is affected by factors such as the complexity of motion in the shooting scene, occlusion, and image noise. It is suitable for scenes with small changes in background and lighting conditions, and cannot well meet the needs of real-time monitoring in actual engineering. Summary of the invention
[0006] The purpose of the present invention is to provide a cable force monitoring method, device, storage medium and product to solve the problem of insufficient real-time and accuracy of traditional cable force detection methods.
[0007] The present invention solves the above technical problems through the following technical solutions: a cable force monitoring method, the monitoring method comprising:
[0008] Get Lasso ROI;
[0009] A first rectangular window and a second rectangular window of the same size and in units of pixels are defined, and the first rectangular window is used to search in a vertical direction on the left side of the Lasso ROI to obtain a left candidate point; and the second rectangular window is used to search in a vertical direction on the right side of the Lasso ROI to obtain a right candidate point;
[0010] Determine a candidate straight line according to the left candidate point and the right candidate point;
[0011] A third rectangular window whose unit is pixel is defined, and each candidate straight line is searched by using the third rectangular window to obtain a candidate center line;
[0012] Determine the search range on the left and right sides of the candidate center line according to the pixel value of the cable diameter;
[0013] A fourth rectangular window and a fifth rectangular window of the same size and in sub-pixel units are defined, and a vertical search is performed within a search range on the left side of the candidate center line using the fourth rectangular window to obtain a left endpoint; a vertical search is performed within a search range on the right side of the candidate center line using the fifth rectangular window to obtain a right endpoint;
[0014] Determine the center line of the cable according to the left end point and the right end point;
[0015] Taking any point on the center line of the cable as a tracking point, tracking the tracking point using the cable ROI of continuous frames to obtain a vibration displacement time history curve of the cable;
[0016] The cable force is calculated according to the vibration displacement time history curve of the cable.
[0017] Further, using a first rectangular window to search in a vertical direction on the left side of the Lasso ROI, or using a second rectangular window to search in a vertical direction on the right side of the Lasso ROI, includes:
[0018] Step A1: when the first rectangular window or the second rectangular window is located at the corresponding side of the Lasso ROI, calculating the grayscale image intensity value within the first rectangular window or the second rectangular window;
[0019] Step A2: determining whether the grayscale image intensity value in the first rectangular window or the second rectangular window is less than a first intensity threshold; if not, when the corresponding side search of the Lasso ROI is not completed, the first rectangular window or the second rectangular window is moved one pixel in the vertical direction, and the process goes to step A1;
[0020] If yes, the center point of the edgemost pixel in the first rectangular window or the second rectangular window is used as the left candidate point or the right candidate point, and the left candidate point or the right candidate point is stored in the current left sub-block point set or the current right sub-block point set, and the process goes to step A3;
[0021] Step A3: when the search for the corresponding side of the Lasso ROI is not completed, the first rectangular window or the second rectangular window is moved one pixel in the vertical direction;
[0022] Step A4: calculating the grayscale image intensity value within the first rectangular window or the second rectangular window;
[0023] Step A5: determining whether the grayscale image intensity value in the first rectangular window or the second rectangular window is less than a first intensity threshold; if not, taking the center point of the current left sub-block point set as the left candidate point or taking the center point of the current right sub-block point set as the right candidate point, and proceeding to step A6;
[0024] If yes, the center point of the edgemost pixel in the first rectangular window or the second rectangular window is used as the left candidate point or the right candidate point, and the left candidate point or the right candidate point is stored in the current left sub-block point set or the current right sub-block point set, and the process goes to step A3;
[0025] Step A6: When the search for the corresponding side of the Lasso ROI is not completed, the first rectangular window or the second rectangular window is moved one pixel in the vertical direction and the process goes to step A1.
[0026] Further, when the left candidate point is not found on the left side of the Lasso ROI using the first rectangular window, the first rectangular window is moved to the right, and then the first rectangular window is continued to be used to search in a vertical direction on the left side of the Lasso ROI until the left candidate point is found;
[0027] When the right candidate point is not found on the right side of the Lasso ROI using the second rectangular window, the second rectangular window is moved to the left, and then the second rectangular window is continuously used to search in the vertical direction on the right side of the Lasso ROI until the right candidate point is found.
[0028] Furthermore, each candidate line is searched using the third rectangular window, including:
[0029] When the third rectangular window is located on the candidate straight line, calculating the grayscale image intensity value within the third rectangular window;
[0030] Determine whether the third rectangular window has completed the search for the candidate straight line. If not, move the third rectangular window one pixel along the candidate straight line and repeat the step of calculating the grayscale image intensity value within the third rectangular window.
[0031] If so, selecting a maximum intensity value from all grayscale image intensity values within the third rectangular window, and determining whether the maximum intensity value is less than a second intensity threshold;
[0032] If the maximum intensity value is less than the second intensity threshold, the candidate straight line is a candidate center line; if the maximum intensity value is greater than or equal to the second intensity threshold, the candidate straight line is not a candidate center line and the candidate straight line is discarded.
[0033] Further, using the fourth rectangular window to search in the vertical direction within the search range on the left side of the candidate center line, or using the fifth rectangular window to search in the vertical direction within the search range on the right side of the candidate center line, includes:
[0034] Step B1: when the fourth rectangular window or the fifth rectangular window is located in the search range on the corresponding side of the candidate center line, calculating the grayscale image intensity value within the fourth rectangular window or the fifth rectangular window;
[0035] Step B2: determining whether the grayscale image intensity value in the fourth rectangular window or the fifth rectangular window is less than the third intensity threshold; if not, when the search for the side corresponding to the candidate center line is not completed, the fourth rectangular window or the fifth rectangular window is moved one sub-pixel point in the vertical direction, and the process goes to step B1; when the search for the side corresponding to the candidate center line is completed, the process goes to step B4;
[0036] If yes, the center point of the edgemost pixel in the fourth rectangular window or the fifth rectangular window is used as the left candidate endpoint or the right candidate endpoint, and the left candidate endpoint or the right candidate endpoint is stored in the left candidate endpoint set or the right candidate endpoint set, and the process goes to step B3;
[0037] Step B3: When the search for the side corresponding to the candidate center line is not completed, the fourth rectangular window or the fifth rectangular window is moved one sub-pixel point in the vertical direction, and the process goes to step B1; when the search for the side corresponding to the candidate center line is completed, the process goes to step B4;
[0038] Step B4: Use the center point of the left endpoint set as the left endpoint or use the center point of the right endpoint set as the right endpoint.
[0039] Further, the cable force is calculated according to the vibration displacement time history curve of the cable, including:
[0040] Performing a fast Fourier transform on the vibration displacement time history curve of the cable to obtain a vibration spectrum diagram of the cable;
[0041] Extracting the natural frequency of the cable from the vibration spectrum of the cable;
[0042] The cable force is calculated according to the natural frequency of the cable, and the specific calculation formula is:
[0043]
[0044] Where T is the cable force, m is the linear density of the cable, l is the length of the cable, and f is n represents the nth natural frequency of the cable, EI represents the structural stiffness, and n represents the order of the natural frequency.
[0045] Furthermore, the Lasso ROI is obtained, including:
[0046] Get cable monitoring video;
[0047] Performing frame extraction processing on the cable monitoring video to obtain multiple frames of original cable images;
[0048] Calling a background segmentation model, and using the background segmentation model to perform background segmentation on each frame of the original cable image;
[0049] The Lasso region is marked on the image after background segmentation to obtain the Lasso ROI of each frame.
[0050] Based on the same concept, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instruction stored in the memory, wherein the processor executes the computer program / instruction to implement the cable force monitoring method as described above.
[0051] Based on the same concept, the present invention also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the cable force monitoring method as described above is implemented.
[0052] Based on the same concept, the present invention also provides a computer program product, including a computer program / instruction, which implements the cable force monitoring method as described above when executed by a processor.
[0053] Beneficial Effects
[0054] Compared with the prior art, the advantages of the present invention are:
[0055] The present invention adopts an improved straight line detection algorithm to determine the center line of the cable. The improved straight line detection algorithm includes three searches, namely, using a first rectangular window and a second rectangular window to perform a coarse search to determine candidate points on the left and right sides of the cable, using a third rectangular window to perform a line search to determine the candidate center line of the cable, and using a sub-pixel fourth rectangular window and a fifth rectangular window to perform a fine search to accurately detect the center line of the cable, thereby ensuring the calculation accuracy of the cable force; for multiple cable ROIs of multiple cables in a single frame, the cable center lines can be detected synchronously, thereby realizing synchronous monitoring of the cable forces of multiple cables, thereby improving the monitoring efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0057] Figure 1 is a flow chart of a cable force monitoring method in an embodiment of the present invention;
[0058] Figure 2 is a schematic diagram of the dimensions of a cable-stayed bridge in an embodiment of the present invention; wherein the dimensions are in mm;
[0059] Figure 3 is a schematic diagram of ROIs of two cables under a single frame image in an embodiment of the present invention;
[0060] Figure 4 is a schematic diagram of searching for a Lasso ROI using a first rectangular window and a second rectangular window in an embodiment of the present invention;
[0061] Figure 5 is a flowchart of searching in the vertical direction on the left side of the Lasso ROI using the first rectangular window in an embodiment of the present invention;
[0062] Figure 6 is a schematic diagram of candidate straight lines in an embodiment of the present invention;
[0063] Figure 7 is a schematic diagram of searching for candidate straight lines using the fourth rectangular window and the fifth rectangular window in an embodiment of the present invention;
[0064] Figure 8 is the vibration displacement time history curve of the cable S4 in the embodiment of the present invention
[0065] Fig. 9 is a vibration displacement time history curve of the cable S5 in the embodiment of the present invention;
[0066] Fig.10 is a vibration spectrum diagram of the cable S4 in an embodiment of the present invention;
[0067] Fig.11 is a vibration spectrum diagram of the cable S5 in the embodiment of the present invention. DETAILED DESCRIPTION
[0068] The following is a clear and complete description of the technical solutions in the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0069] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0070] like Figure 1 As shown, a cable force monitoring method provided by an embodiment of the present invention comprises the following steps:
[0071] Step S1: Get the Lasso ROI.
[0072] In a specific embodiment of the present invention, obtaining the Lasso ROI includes:
[0073] Step S11: Obtain cable monitoring video;
[0074] Step S12: extracting frames from the cable monitoring video to obtain multiple frames of original cable images;
[0075] Step S13: calling the background segmentation model, and using the background segmentation model to perform background segmentation on each frame of the original cable image;
[0076] Step S14: marking the Lasso region of the image after background segmentation to obtain the Lasso ROI (Region of interest) of each frame.
[0077] In step S11, an image acquisition device (such as an industrial camera) is used to acquire monitoring videos of multiple cables under different environmental backgrounds and different vehicle load excitations.
[0078] In order to collect monitoring videos under different environmental backgrounds and different vehicle load excitations, the embodiment of the present invention is based on Figure 2 Based on the cable-stayed bridge shown in the figure, an indoor scaled bridge model is built. The background of the indoor scaled bridge model is changed to simulate different environmental backgrounds; a rubber hammer is used to hammer the end of the cable to artificially simulate the external vehicle load excitation, and the excitation time interval is usually 6 to 7 seconds. In this embodiment, the pixel resolution of the image acquisition device is 4096×2160pixel; according to the Nyquist sampling theorem, the highest target frequency that can be captured by video shooting is half of the sampling frequency. In order to accurately identify the vibration frequency of the cable, the sampling frequency is set to 60Hz; the acquisition time of the monitoring video is 2 minutes, and the monitoring video is saved in common video formats such as mp4, avi, and mov. The size of the acquisition area can be changed by changing the focal length of the image acquisition device, thereby changing the number of cables in the acquisition area.
[0079] In step S12, a certain frame of the original image of the cable without excitation is selected as the initial frame, and the image resolution and size of the subsequent frames are kept consistent with the initial frame. When a single frame of the original image of the cable includes multiple cables, the number of cable ROIs in the single frame image obtained by the background segmentation model and marking is the same as the number of cables. Figure 3 As shown, the original image of a single-frame cable includes two cables (i.e. Figure 2 The background segmentation model and marker output two Lasso ROIs, namely ROI1 and ROI2.
[0080] In actual projects, the cable monitoring video can be shot using an unmanned aerial vehicle equipped with an image acquisition device, or the image acquisition device can be installed at a certain location on the actual bridge to obtain the cable monitoring video in real time.
[0081] In this embodiment, the background segmentation model adopts U 2-Net deep learning network, using the trained deep learning network to perform background segmentation on the original image of the cable, remove the background, and obtain a target detection frame that only includes the cable. In a specific embodiment of the present invention, the training process of the deep learning network includes:
[0082] Step S21: construct a Lasso sample data set, where each sample in the Lasso sample data set includes images of Lassos under different backgrounds and manually annotated Lasso labels.
[0083] The cable image can be obtained by an experimental method similar to step S11, or it can be obtained from an existing data set. The cable image can be enhanced by rotating, enlarging, cropping, and adding noise, so as to expand the cable sample data set and improve the training accuracy of the deep learning network.
[0084] Step S22: Divide the cable sample data set into a training set and a test set.
[0085] Step S23: Construct U 2 -Net deep learning network, using the training set to 2 -Net deep learning network is trained and the trained U 2 -Net deep learning network for testing and get the final U 2 -Net deep learning network training parameters and weights.
[0086] Using the final U 2 -Net deep learning network performs background segmentation on the original image of the monitored cable to obtain the image after background segmentation, and then performs ROI marking on the image to obtain the cable ROI.
[0087] Since the grayscale image intensity value needs to be calculated in the subsequent process of determining the cable centerline using the improved straight line detection algorithm, the cable image output by the background segmentation model is grayed. The improved straight line detection algorithm is used to search the cable ROIs output by the background segmentation model and the markers to obtain the cable centerline corresponding to the cable ROI.
[0088] In actual engineering, the cables on the same side of the bridge usually do not cross each other, so the two adjacent cable ROIs in a single frame will not cross or overlap, and it is assumed that the cables in the cable ROI are straight and run from left to right.
[0089] Step S2: define a first rectangular window and a second rectangular window of the same size and in pixels, use the first rectangular window to search vertically on the left side of the Lasso ROI to obtain a left candidate point; use the second rectangular window to search vertically on the right side of the Lasso ROI to obtain a right candidate point.
[0090] The first rectangular window and the second rectangular window are used to perform a rough search on the left and right sides of the cable ROI respectively to determine the candidate endpoints of the left and right ends of the cable, and then determine the candidate straight line of the cable. In order to ensure the accuracy of the detection result, the width of the first rectangular window and the second rectangular window should be smaller than the pixel value of the cable diameter in the cable ROI. In this embodiment, the size of the first rectangular window and the second rectangular window is 5×5 pixels, that is, Figure 4 The dashed box in the figure is roughly searched by moving the first rectangular window and the second rectangular window in the vertical direction on the left and right sides of the cable ROI respectively. In a specific embodiment of the present invention, Figure 4 and Figure 5 As shown, a vertical search is performed on the left side of the Lasso ROI using the first rectangular window, including:
[0091] Step A1: when the first rectangular window is located on the left side of the Lasso ROI, the grayscale image intensity value within the first rectangular window is calculated;
[0092] Step A2: determining whether the grayscale image intensity value in the first rectangular window is less than the first intensity threshold; if not, when the left search of the Lasso ROI is not completed, the first rectangular window is moved one pixel in the vertical direction, and the process goes to step A1;
[0093] If yes, the leftmost pixel center point in the first rectangular window is used as the left candidate point, and the left candidate point is stored in the current left sub-block point set, and the process goes to step A3;
[0094] Step A3: when the left search of the Lasso ROI is not completed, the first rectangular window is moved one pixel in the vertical direction;
[0095] Step A4: Calculate the grayscale image intensity value within the first rectangular window;
[0096] Step A5: determine whether the grayscale image intensity value in the first rectangular window is less than the first intensity threshold; if not, take the center point of the current left sub-block point set as the left candidate point and proceed to step A6;
[0097] If yes, the leftmost pixel center point in the first rectangular window is used as the left candidate point, and the left candidate point is stored in the current left sub-block point set, and the process goes to step A3;
[0098] Step A6: When the left search of the Lasso ROI is not completed, the first rectangular window is moved one pixel in the vertical direction and the process goes to step A1.
[0099] In this embodiment, the first intensity threshold is the average value of the background intensity of the Lasso ROI. The initial position of the first rectangular window can be the top of the Lasso ROI (e.g. Figure 4As shown in the figure, the search is then performed from top to bottom, moving one pixel each time, until the first rectangular window is at the bottom of the Lasso ROI, thus completing the search on the left side of the Lasso ROI. The initial position of the first rectangular window may also be at the bottom of the Lasso ROI, and then the search is performed from bottom to top, moving one pixel each time, until the first rectangular window is at the top of the Lasso ROI, thus completing the search on the left side of the Lasso ROI.
[0100] Figure 4 a1~a5 in the above are all left candidate points in the same left sub-block point set. Multiple left candidate points in the same left sub-block point set are continuous pixel points. When multiple left candidate points are determined, if the grayscale image intensity value in the first rectangular window is greater than or equal to the first intensity threshold, it indicates that the current left sub-block has been searched, and the next left sub-block is searched. The subsequently determined left candidate points are stored in the next left sub-block point set. The center point of multiple left candidate points in each left sub-block point set is the left candidate point. For example Figure 4 Point P in m (a3). In the actual shooting process, due to the complexity of the background, the Lasso image obtained by the background segmentation model cannot completely segment the Lasso background. The grayscale value of the incompletely separated background may be close to or smaller than the grayscale value of the Lasso at the edge of the Lasso ROI. Therefore, when searching the left side of the Lasso ROI using the first rectangular window, multiple left candidate points may appear.
[0101] Similarly, the second rectangular window is used to search vertically on the right side of the Lasso ROI to obtain the right candidate point. The first rectangular window and the second rectangular window are used to roughly search on the left and right sides of the Lasso ROI to obtain at least one left candidate point and at least one right candidate point.
[0102] When the left candidate point is not found on the left side of the Lasso ROI using the first rectangular window, the first rectangular window is moved one pixel to the right, and then the first rectangular window is continued to be used to search vertically on the left side of the Lasso ROI according to steps A1 to A6; if the left candidate point is still not found, the first rectangular window is continued to be moved one pixel to the right until the left candidate point is found. Similarly, when the right candidate point is not found on the right side of the Lasso ROI using the second rectangular window, the second rectangular window is moved to the left, and then the second rectangular window is continued to be used to search vertically on the right side of the Lasso ROI until the right candidate point is found.
[0103] Step S3: Determine a candidate line based on the left candidate point and the right candidate point.
[0104] Connect the left candidate point with the right candidate point to obtain a candidate line; when there are multiple left candidate points and / or right candidate points, multiple candidate lines are obtained. For example, Figure 6 As shown, there are 2 left candidate points Pm1 , P m2 and 3 right candidate points P n1 , P n2 , P n3 , there are 6 candidate lines.
[0105] Step S4: define a third rectangular window whose unit is pixel, and use the third rectangular window to search each candidate straight line to obtain a candidate center line.
[0106] In order to determine the candidate center line from multiple candidate lines, it is necessary to use the third rectangular window to perform pixel-level line search to remove candidate lines that are not the center line of the cable. In a specific embodiment of the present invention, the third rectangular window is used to search each candidate line, including:
[0107] Step S4.1: when the third rectangular window is located on the candidate straight line, calculating the grayscale image intensity value within the third rectangular window;
[0108] Step S4.2: Determine whether the third rectangular window has completed the search for the candidate straight line. If not, move the third rectangular window one pixel along the candidate straight line and proceed to step S4.1.
[0109] If yes, select the maximum intensity value from all the grayscale image intensity values in the third rectangular window and go to step S4.3;
[0110] Step S4.3: Determine whether the maximum intensity value is less than the second intensity threshold; if the maximum intensity value is less than the second intensity threshold, the candidate straight line is the candidate center line; if the maximum intensity value is greater than or equal to the second intensity threshold, the candidate straight line is not the candidate center line and the candidate straight line is discarded.
[0111] The initial position of the third rectangular window can be the leftmost end of the candidate line, and then the candidate line is searched from left to right, moving one pixel each time, until the third rectangular window is located at the rightmost end of the candidate line, and the search for the candidate line is completed. The initial position of the third rectangular window can also be the rightmost end of the candidate line, and then the candidate line is searched from right to left, moving one pixel each time, until the third rectangular window is located at the leftmost end of the candidate line, and the search for the candidate line is completed.
[0112] like Figure 6As shown in the figure, there are 6 candidate lines. The smaller the intensity value (0-255), the darker the color. When the maximum intensity value of the candidate line is less than the second intensity threshold, the intensity value of each part of the candidate line is less than the second intensity threshold. When the candidate line contains the background, since the background part is lighter in color and has a larger intensity value, the maximum intensity value of the candidate line must be located in the background part. Therefore, the maximum intensity value of the candidate line must be greater than or equal to the second intensity threshold, indicating that the candidate line contains the background part. The candidate line is not the center line of the cable and is discarded.
[0113] In order to avoid erroneous search, the size of the third rectangular window is smaller than the pixel value of the cable diameter. In this embodiment, the size of the third rectangular window is 1×7 pixels. When the background is simple, the second intensity threshold can be the same as the first intensity threshold; when the background is complex, in order to improve the detection accuracy, the second intensity threshold is set to different values according to the background.
[0114] Step S5: Determine the search range on the left and right sides of the candidate center line according to the pixel value of the cable diameter.
[0115] The units of the first rectangular window, the second rectangular window and the third rectangular window are pixels, so the point search in step S2 and the line search in step S4 are both pixel-level searches, and the accuracy of the pixel-level search cannot meet the requirements for extracting the vibration frequency of the cable. In order to further improve the detection accuracy of the cable centerline, it is necessary to detect the candidate centerline at the sub-pixel level to obtain the cable centerline and ensure the reliability of the method of the present invention.
[0116] In this embodiment, the search ranges on the left and right sides of the candidate center line are both larger than the pixel value of the cable diameter. Figure 7 The red dashed box in the figure is the search range on the left and right of the candidate centerline.
[0117] Step S6: Define a fourth rectangular window and a fifth rectangular window of the same size and in sub-pixel units, and use the fourth rectangular window to perform a vertical search within the search range on the left side of the candidate center line to obtain the left endpoint; use the fifth rectangular window to perform a vertical search within the search range on the right side of the candidate center line to obtain the right endpoint.
[0118] The fourth rectangular window and the fifth rectangular window should be covered by the cable, so the size of the fourth rectangular window and the fifth rectangular window should be smaller than the pixel value of the cable diameter. In this embodiment, the size of the fourth rectangular window and the fifth rectangular window is 7×7 (eg Figure 7 The blue grid in the figure adopts 0.5 pixel sub-pixel resolution and 0.5 pixel as the search step size.
[0119] In a specific embodiment of the present invention, using the fourth rectangular window to search in the vertical direction within the search range on the left side of the candidate center line includes:
[0120] Step B1: when the fourth rectangular window is located in the search range on the left side of the candidate center line, calculating the grayscale image intensity value within the fourth rectangular window;
[0121] Step B2: Determine whether the grayscale image intensity value in the fourth rectangular window is less than the third intensity threshold value. If not, when the search on the left side of the candidate center line is not completed, the fourth rectangular window is moved one sub-pixel (i.e., 0.5 pixel) in the vertical direction, and the process goes to step B1; when the search on the left side of the candidate center line is completed, the process goes to step B4;
[0122] If yes, the leftmost pixel center point in the fourth rectangular window is used as the left candidate endpoint, the left candidate endpoint is stored in the left candidate endpoint set, and the process goes to step B3;
[0123] Step B3: When the search on the left side of the candidate center line is not completed, the fourth rectangular window is moved one sub-pixel in the vertical direction, and the process goes to step B1; when the search on the left side of the candidate center line is completed, the process goes to step B4;
[0124] Step B4: Take the center point of the left candidate endpoint set as the left endpoint.
[0125] The initial position of the fourth rectangular window may be the top of the search range on the left side of the candidate center line, and then the search is performed from top to bottom, moving one sub-pixel point each time, until the fourth rectangular window is located at the bottom of the search range on the left side of the candidate center line, that is, the search for the left side of the candidate center line is completed. The initial position of the fourth rectangular window may also be the bottom of the search range on the left side of the candidate center line, and then the search is performed from bottom to top, moving one sub-pixel point each time, until the fourth rectangular window is located at the top of the search range on the left side of the candidate center line, that is, the search for the left side of the candidate center line is completed.
[0126] Since the sub-pixel point search is performed after the pixel-level point search (i.e., the search using the first rectangular window and the second rectangular window), the search range is small and the grayscale value variation range is relatively small. Therefore, the left candidate endpoint set is usually adjacent and continuous, and there will be no sub-blocks or multiple left candidate endpoint sets. Only one left endpoint is obtained. If the grayscale image intensity value corresponding to each left candidate endpoint in the left candidate endpoint set is less than the third intensity threshold, it is considered that the left candidate endpoint set is completely covered by the cable, and the center point of the left candidate endpoint set is used as the left endpoint. Figure 7 P centre1 .
[0127] Similarly, by using the fifth rectangular window to search vertically within the search range on the right side of the candidate center line, the right endpoint P can be obtained. centre2.
[0128] Step S7: Determine the center line of the cable according to the left end point and the right end point.
[0129] Connect the left endpoint with the right endpoint to get the cable centerline. Considering that there may be a situation where the slope of the straight line does not exist in the search results, the cable centerline is represented by the linear expression in Hough transform. The specific expression is:
[0130] r=xcosθ+ysinθ (1)
[0131] Among them, r represents the vertical distance from the origin of the coordinate system to the center line of the cable, (x, y) represents the coordinates of the point on the center line of the cable, and θ represents the angle between the vertical line from the origin of the coordinate system to the center line of the cable and the x-axis. The coordinate system takes the upper left corner of the cable ROI as the origin, the top edge of the cable ROI to the right as the X-axis, and the left side of the cable ROI to the bottom as the Y-axis.
[0132] Step S8: Taking any point on the center line of the cable as a tracking point, the tracking point is tracked using the cable ROI of continuous frames to obtain a vibration displacement time history curve of the cable.
[0133] Take any point P on the center line of the cable track (x k ,y p ) as the tracking point, and record the tracking point P in the Lasso ROI of the continuous frame track of y p Change, y p The change is the y value of the Lasso ROI in each frame relative to the Lasso ROI in the initial frame. p Change, that is, vibration displacement, and then the curve of vibration displacement with respect to time can be obtained, that is, the vibration displacement time history curve of the cable. Figure 3 The vibration displacement time history curves of the cables S4 and S5 are shown in Figure 8 and Fig. 9 shown.
[0134] Step S9: Calculate the cable force according to the vibration displacement time history curve of the cable.
[0135] In a specific embodiment of the present invention, the cable force is calculated according to the vibration displacement time history curve of the cable, including:
[0136] Step S9.1: Perform fast Fourier transform on the vibration displacement time history curve of the cable to obtain the vibration spectrum of the cable, such as Fig.10 and Fig.11 As shown;
[0137] Step S9.2: Extracting the natural frequency of the cable from the vibration spectrum of the cable.
[0138] The fixed frequency is related to the cable's own mass m, structural stiffness EI and cable force T. Different cables correspond to different natural frequencies.
[0139] Step S9.3: According to the natural frequency of the cable, based on the vibration frequency method, the cable force is calculated by calculating the relationship between each order of vibration frequency and the cable force. The specific calculation formula is:
[0140]
[0141] Where T represents the cable force, m represents the linear density of the cable (i.e. the mass per unit length of the cable), l represents the cable length, and f n represents the nth natural frequency of the cable, EI represents the structural stiffness, and n represents the order of the natural frequency.
[0142] In order to evaluate the detection accuracy of the method of the present invention, the cable force value obtained by the method of the present invention is compared with the cable force value measured and calculated by the vibration acceleration sensor. The comparison results are shown in Table 1.
[0143] Table 1 Comparison results of the cable force values measured and calculated by the present invention and the vibration acceleration sensor
[0144] Cable No. The cable force value calculated by the present invention Vibration acceleration sensor measures and calculates the cable force value The error between the two S4 83.904N 85.59N 1.97% S5 112.15N 115.47N 2.88%
[0145] According to the comparison results in Table 1, the improved straight line detection algorithm used in the present invention can obtain a more accurate cable vibration displacement time history curve, and has better detection accuracy.
[0146] Example 3
[0147] An embodiment of the present invention further provides an electronic device, which includes: a memory, a processor, and a computer program / instructions stored in the memory, and the processor executes the computer program / instructions to implement the rope force monitoring method in the embodiment of the present application.
[0148] Although not shown, the electronic device includes a processor, which can perform various appropriate operations and processes according to the program and / or data stored in the read-only memory (ROM) or the program and / or data loaded from the storage portion into the random access memory (RAM). The processor can be a multi-core processor, or it can include multiple processors. In some embodiments, the processor can include a general main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.
[0149] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.
[0150] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the rope force monitoring method in the embodiment of the present application.
[0151] Storage media in embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be implemented by any method or technology to store information. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
[0152] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.
[0153] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the cable force monitoring method in the embodiment of the present application.
[0154] What is disclosed above is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, which should be included in the protection scope of the present invention.
Claims
1. A cable force monitoring method, characterized in that: The monitoring method comprises: Get Lasso ROI; A first rectangular window and a second rectangular window of the same size and in units of pixels are defined, and the first rectangular window is used to search in a vertical direction on the left side of the Lasso ROI to obtain a left candidate point; and the second rectangular window is used to search in a vertical direction on the right side of the Lasso ROI to obtain a right candidate point; Determine a candidate straight line according to the left candidate point and the right candidate point; A third rectangular window whose unit is pixel is defined, and each candidate straight line is searched by using the third rectangular window to obtain a candidate center line; Determine the search range on the left and right sides of the candidate center line according to the pixel value of the cable diameter; A fourth rectangular window and a fifth rectangular window of the same size and in sub-pixel units are defined, and a vertical search is performed within a search range on the left side of the candidate center line using the fourth rectangular window to obtain a left endpoint; a vertical search is performed within a search range on the right side of the candidate center line using the fifth rectangular window to obtain a right endpoint; Determine the center line of the cable according to the left end point and the right end point; Taking any point on the center line of the cable as a tracking point, tracking the tracking point using the cable ROI of continuous frames to obtain a vibration displacement time history curve of the cable; The cable force is calculated according to the vibration displacement time history curve of the cable.
2. The cable force monitoring method according to claim 1, characterized in that: Using a first rectangular window to search in a vertical direction on the left side of the Lasso ROI, or using a second rectangular window to search in a vertical direction on the right side of the Lasso ROI, includes: Step A1: when the first rectangular window or the second rectangular window is located at the corresponding side of the Lasso ROI, calculating the grayscale image intensity value within the first rectangular window or the second rectangular window; Step A2: determining whether the grayscale image intensity value in the first rectangular window or the second rectangular window is less than a first intensity threshold; if not, when the corresponding side search of the Lasso ROI is not completed, the first rectangular window or the second rectangular window is moved one pixel in the vertical direction, and the process goes to step A1; If yes, the center point of the edgemost pixel in the first rectangular window or the second rectangular window is used as the left candidate point or the right candidate point, and the left candidate point or the right candidate point is stored in the current left sub-block point set or the current right sub-block point set, and the process goes to step A3; Step A3: when the search for the corresponding side of the Lasso ROI is not completed, the first rectangular window or the second rectangular window is moved one pixel in the vertical direction; Step A4: calculating the grayscale image intensity value within the first rectangular window or the second rectangular window; Step A5: determining whether the grayscale image intensity value in the first rectangular window or the second rectangular window is less than a first intensity threshold; if not, taking the center point of the current left sub-block point set as the left candidate point or taking the center point of the current right sub-block point set as the right candidate point, and proceeding to step A6; If yes, the center point of the edgemost pixel in the first rectangular window or the second rectangular window is used as the left candidate point or the right candidate point, and the left candidate point or the right candidate point is stored in the current left sub-block point set or the current right sub-block point set, and the process goes to step A3; Step A6: When the search for the corresponding side of the Lasso ROI is not completed, the first rectangular window or the second rectangular window is moved one pixel in the vertical direction and the process goes to step A1.
3. The cable force monitoring method according to claim 2, characterized in that: When the left candidate point is not found on the left side of the Lasso ROI using the first rectangular window, the first rectangular window is moved to the right, and then the first rectangular window is continued to be used to search in a vertical direction on the left side of the Lasso ROI until the left candidate point is found; When the right candidate point is not found on the right side of the Lasso ROI using the second rectangular window, the second rectangular window is moved to the left, and then the second rectangular window is continuously used to search in the vertical direction on the right side of the Lasso ROI until the right candidate point is found.
4. The cable force monitoring method according to claim 1, characterized in that: Use the third rectangular window to search each candidate line, including: When the third rectangular window is located on the candidate straight line, calculating the grayscale image intensity value within the third rectangular window; Determine whether the third rectangular window has completed the search for the candidate straight line. If not, move the third rectangular window one pixel along the candidate straight line and repeat the step of calculating the grayscale image intensity value within the third rectangular window. If so, selecting a maximum intensity value from all grayscale image intensity values within the third rectangular window, and determining whether the maximum intensity value is less than a second intensity threshold; If the maximum intensity value is less than the second intensity threshold, the candidate straight line is a candidate center line; if the maximum intensity value is greater than or equal to the second intensity threshold, the candidate straight line is not a candidate center line and the candidate straight line is discarded.
5. The cable force monitoring method according to claim 1, characterized in that: Using the fourth rectangular window to search in the vertical direction within the search range on the left side of the candidate center line, or using the fifth rectangular window to search in the vertical direction within the search range on the right side of the candidate center line, includes: Step B1: when the fourth rectangular window or the fifth rectangular window is located in the search range on the corresponding side of the candidate center line, calculating the grayscale image intensity value within the fourth rectangular window or the fifth rectangular window; Step B2: determining whether the grayscale image intensity value in the fourth rectangular window or the fifth rectangular window is less than the third intensity threshold; if not, when the search for the side corresponding to the candidate center line is not completed, the fourth rectangular window or the fifth rectangular window is moved one sub-pixel point in the vertical direction, and the process goes to step B1; when the search for the side corresponding to the candidate center line is completed, the process goes to step B4; If yes, the center point of the edgemost pixel in the fourth rectangular window or the fifth rectangular window is used as the left candidate endpoint or the right candidate endpoint, and the left candidate endpoint or the right candidate endpoint is stored in the left candidate endpoint set or the right candidate endpoint set, and the process goes to step B3; Step B3: When the search for the side corresponding to the candidate center line is not completed, the fourth rectangular window or the fifth rectangular window is moved one sub-pixel point in the vertical direction, and the process goes to step B1; when the search for the side corresponding to the candidate center line is completed, the process goes to step B4; Step B4: Use the center point of the left endpoint set as the left endpoint or use the center point of the right endpoint set as the right endpoint.
6. The cable force monitoring method according to claim 1, characterized in that: Calculating the cable force according to the vibration displacement time history curve of the cable includes: Performing a fast Fourier transform on the vibration displacement time history curve of the cable to obtain a vibration spectrum diagram of the cable; Extracting the natural frequency of the cable from the vibration spectrum of the cable; The cable force is calculated according to the natural frequency of the cable, and the specific calculation formula is: Where T is the cable force, m is the linear density of the cable, l is the length of the cable, and f is n represents the nth natural frequency of the cable, EI represents the structural stiffness, and n represents the order of the natural frequency.
7. The cable force monitoring method according to any one of claims 1 to 6, characterized in that: Get Lasso ROI, including: Get cable monitoring video; Performing frame extraction processing on the cable monitoring video to obtain multiple frames of original cable images; Calling a background segmentation model, and using the background segmentation model to perform background segmentation on each frame of the original cable image; The Lasso region is marked on the image after background segmentation to obtain the Lasso ROI of each frame.
8. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the cable force monitoring method according to any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the cable force monitoring method according to any one of claims 1 to 7 is implemented.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the cable force monitoring method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Inhaul cable time-variant cable force process recognition data driving method based on acceleration speed monitoring
CN103698071A
Bridge cable force measuring method based on video image recognition
CN108106541A
Cable force testing method based on mobile phone image recognition
CN112381190A
Bridge cable force automatic cruise monitoring system and method based on pan-tilt camera
CN115144102A
Stay cable force determination method, device and equipment based on linear tracking algorithm
CN115683431A
