A method and system for intelligent hoisting monitoring of a suspension bridge based on machine vision

Abstract

The application discloses a kind of intelligent hoisting monitoring method and system of suspension bridge based on machine vision, which comprises: picture is collected by visual unit being arranged around main cable, sling and component hoisting point, component three-dimensional posture vector of the component to be hoisted is generated, the displacement of multiple sampling points of the component to be hoisted is obtained, the displacement measure in time interval is calculated, the weighted sum of the displacement measure of all sampling points is used as dynamic deformation index, the time sequence stability index of the component to be hoisted is calculated according to component three-dimensional posture vector and dynamic deformation index;The displacement spectrum of each sling is generated, the direction consistency index and peak mutation degree are calculated according to displacement time sequence, and the frequency spectrum cluster degree is calculated according to displacement spectrum;The weighted sum of direction consistency index, peak mutation degree and frequency spectrum cluster degree is used as sling stress trend factor, the sling stress trend factor is coupled with time sequence stability index, the risk quantization index of hoisting process is generated, and monitoring is carried out according to risk quantization index.

Description

Technical Field

[0001] This invention belongs to the field of suspension bridge hoisting technology, and more specifically, relates to a machine vision-based intelligent hoisting monitoring method and system for suspension bridges. Background Technology

[0002] Currently, in the hoisting of large components, especially in the hoisting of long-span suspension bridges, mainstream methods for monitoring component attitude and determining cable stress largely rely on various traditional sensor systems, including: inertial measurement units (IMUs), total stations, laser rangefinders, single-view or dual-view cameras, optical flow calculations, target tracking systems, or force measuring devices such as stress gauges and tension gauges embedded in the mechanical structure. While these technologies can provide component position and attitude information to some extent, they generally suffer from several inherent problems.

[0003] First, existing visual monitoring typically relies on manually placed targets, reflective points, or high-precision QR codes as the basis for recognition, and requires the prior establishment of feature libraries or 3D templates. If the component's shape changes, its surface is occluded, or its reflectivity is excessive, recognition will fail. Furthermore, the parallax capability of single or dual cameras is insufficient, making it impossible to maintain stable 3D reconstruction accuracy in large scenes and difficult to continuously and stably track components under conditions of wind and wave disturbances, changes in lighting, or changes in the hoisting path. This results in drawbacks such as strong functional dependence, low robustness, high deployment costs, and sensitivity to environmental factors.

[0004] Secondly, existing attitude estimation methods based on optical flow or feature point tracking have high requirements for texture and are severely affected by strong winds, vibrations, occlusion, and motion blur. When the component scale is large, the texture is monotonous, or the lighting is unstable, the optical flow field is prone to breakage, drift, and local failure, making the final attitude estimation uncontrollable. In addition, most optical flow algorithms are two-dimensional motion analysis, which cannot fundamentally guarantee stable and reliable three-dimensional attitude information. To obtain three-dimensional results, additional equipment such as structured light and lidar must be superimposed, increasing system complexity and cost.

[0005] Third, in terms of monitoring dynamic deformation of components, traditional technologies generally rely on contact or semi-contact sensors such as strain gauges, displacement gauges, and laser scanners. These require the placement of points or reflective strips on the component surface, making the installation process cumbersome. They are also sensitive to the surface condition of the structure and, due to limitations in the number of points, it is difficult to obtain the global deformation pattern. Furthermore, they cannot provide real-time analysis of complex nonlinear deformations such as torsion, warping, and out-of-plane vibration commonly seen in wind farm hoisting. In addition, most existing visual deformation monitoring methods are based on fixed texture tracking, which is unsuitable for steel structural components with untextured or reflective surfaces.

[0006] Therefore, there is an urgent need for a technical solution that can establish a mapping between the stability of the component's posture and the stress trend of the sling, and construct a unified risk quantification index, so as to achieve rapid early warning of component swaying, deflection, critical stress and abnormal deformation during actual hoisting. Summary of the Invention

[0007] To address the above technical problems, this invention proposes a machine vision-based intelligent hoisting monitoring method for suspension bridges, comprising:

[0008] Synchronous images are captured by visual units deployed around the main cable, slings, and component lifting points to form a visual tensor field covering the lifting operation area. The visual tensor field is then clustered into pixels to remove the visual tensor fields of components not to be lifted, thus obtaining the visual tensor field of the components to be lifted.

[0009] Generate the three-dimensional attitude vector of the component to be lifted, obtain the displacement of multiple sampling points of the component to be lifted, and calculate the displacement measure within the time interval. The weighted sum of the displacement measures of all sampling points is used as the dynamic deformation index. The temporal stability index of the component to be lifted is calculated based on the three-dimensional attitude vector of the component and the dynamic deformation index.

[0010] The displacement time series of each cable is magnified and compared by visual tensor field to generate displacement spectrum of each cable. The directional consistency index and peak change degree are calculated based on displacement time series, and the spectral clustering degree is calculated based on displacement spectrum.

[0011] The weighted sum of directional consistency index, peak mutation degree and spectral clustering degree is used as the sling stress trend factor. The sling stress trend factor is coupled with the time series stability index to generate a risk quantification index for the hoisting process, and monitoring is carried out based on the risk quantification index.

[0012] Furthermore, the visual tensor field forming the area covering the hoisting operation includes:

[0013] ,

[0014] in, For time Scene grid points The visual tensor field, The number of visual units, To make scene grid points Map back to the first Inverse projection of the pixel coordinates of the synchronized image of each visual unit For the first After distortion correction and brightness normalization of the synchronized images of each visual unit, the pixel coordinates are... The pixel value at that location.

[0015] Furthermore, generating the three-dimensional pose vector of the component to be lifted includes: back-projecting the visual tensor field of the component to be lifted into a three-dimensional point cloud fragment through multi-baseline triangulation;

[0016] A local reference coordinate system is fitted on a 3D point cloud fragment to obtain the local coordinate system of the component to be lifted. Under the local coordinate system of the component to be lifted, the 3D coordinates of the direction vector and the centroid of the outline of the component to be lifted are obtained by fitting the component.

[0017] Obtain the Euler angles of the component to be lifted on each direction vector. Combine the Euler angles on each direction vector with the three-dimensional coordinates of the centroid of the profile to form the three-dimensional attitude vector of the component to be lifted. The direction vector includes the X-axis, Y-axis and Z-axis of the component to be lifted. The Euler angles on each direction vector include: the roll angle corresponding to the X-axis of the component to be lifted, the pitch angle corresponding to the Y-axis of the component to be lifted and the yaw angle corresponding to the Z-axis of the component to be lifted.

[0018] Furthermore, the displacement measure within the calculation time interval includes:

[0019] ,

[0020] in, For time Time The first component to be lifted One sampling point, For time intervals, For time Time The first component to be lifted The displacement of each sampling point For time Time The first component to be lifted The displacement of each sampling point For time Time The first component to be lifted The baseline length between each sampling point and the component reference point To prevent dividing by zero and positive numbers.

[0021] Furthermore, the temporal stability index of the component to be lifted is calculated based on the component's three-dimensional attitude vector and dynamic deformation index, including:

[0022] ,

[0023] in, For time Time The temporal stability index of each component to be lifted. The weights of the norm and standard deviation, For window length Time within Time The three-dimensional attitude vector of the component to be lifted. The norm standard deviation, The weights of the moving average, For window length Time within Time Dynamic deformation index of a component to be lifted The moving average.

[0024] Furthermore, by amplifying and comparing the displacement time series of each cable through the visual tensor field, the displacement spectrum of each cable is generated by sampling the cable pixels marked in the visual tensor field along the cable direction, extracting the centroid position of the pixels in the time series, and obtaining the displacement time series of each cable.

[0025] The displacement time series of the sling is bandpass filtered and short-time Fourier transform is used to obtain the local energy spectrum, which is defined as the displacement spectrum of the sling.

[0026] Furthermore, the directional consistency index and peak abruptness are calculated based on the displacement time series, and the spectral clustering degree is calculated based on the displacement spectrum, including:

[0027] The consistency index for calculation direction includes:

[0028] ,

[0029] in, For time Time sling Indicators of directional consistency To calculate the window length, For indicator functions, For time Time sling The first time derivative of the displacement time series, For symbolic functions, For comparison with lag;

[0030] Calculating the peak abrupt change includes:

[0031] ,

[0032] in, For time Time sling The peak abrupt change, From time Time The standard deviation of the peak value of the internal displacement time series. From time Time The average of the peak values ​​of the internal displacement time series. From time Time Internal slings displacement time series, To prevent dividing by zero and positive numbers;

[0033] Calculate the spectral clustering degree:

[0034] ,

[0035] in, For time Time sling Spectrum clustering degree, The frequency of the sling's vibration. For time At the time of suspension vibration frequency The normalized displacement spectrum of the suspension cable. This represents the number of frequency points.

[0036] Furthermore, by coupling the sling stress trend factor with the time-series stability index, a risk quantification index for the lifting process is generated, including:

[0037] ,

[0038] in, For time Time The slings for the component to be lifted Risk quantification indicators The weights of the sling stress trend factor, For time Time sling The stress tendency factor of the suspension cable The weights for the time series stability index, For time Time The temporal stability index of each component to be lifted.

[0039] Furthermore, monitoring based on risk quantification indicators includes: if time Time The slings for the component to be lifted Risk quantification indicators If the first warning threshold is exceeded, a high-risk warning will be issued. If the second warning threshold is exceeded, emergency control will be triggered. The second warning threshold is greater than the first warning threshold.

[0040] This invention also proposes a machine vision-based intelligent hoisting monitoring system for suspension bridges, comprising:

[0041] The fusion module is used to acquire synchronous images through visual units deployed around the main cable, slings and component lifting points, forming a visual tensor field covering the lifting operation area. The visual tensor field is then clustered into pixels to remove the visual tensor fields of components not to be lifted, thus obtaining the visual tensor field of the component to be lifted.

[0042] The attitude construction module is used to generate the three-dimensional attitude vector of the component to be lifted, obtain the displacement of multiple sampling points of the component to be lifted, calculate the displacement measure within the time interval, use the weighted sum of the displacement measures of all sampling points as the dynamic deformation index, and calculate the temporal stability index of the component to be lifted based on the component's three-dimensional attitude vector and the dynamic deformation index.

[0043] The module for calculating spectral clustering is used to amplify and compare the displacement time series of each cable through a visual tensor field, generate the displacement spectrum of each cable, calculate the directional consistency index and peak change degree based on the displacement time series, and calculate the spectral clustering degree based on the displacement spectrum.

[0044] The monitoring module is used to take the weighted sum of the directional consistency index, peak mutation degree and spectral clustering degree as the sling stress trend factor, couple the sling stress trend factor with the time series stability index to generate a risk quantification index for the hoisting process, and monitor according to the risk quantification index.

[0045] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art:

[0046] This invention establishes a mapping between the stability of the component's posture and the stress trend of the sling, and constructs a unified risk quantification index. This enables rapid early warning of component swaying, deflection, critical stress, and abnormal deformation during actual hoisting, thereby significantly improving the safety, stability, and intelligent control level of high-altitude hoisting operations for suspension bridges. Attached Figure Description

[0047] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention;

[0048] Figure 2 This is a system structure diagram of Embodiment 2 of the present invention. Detailed Implementation

[0049] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0050] The method provided by this invention can be implemented in a terminal environment that may include one or more of the following components: a processor, a storage medium, and a display screen. The storage medium stores at least one instruction, which is loaded and executed by the processor to implement the method described in the following embodiments.

[0051] A processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the terminal, and performs various functions and processes data by running or executing instructions, programs, code sets or instruction sets stored in the storage medium, and by calling data stored in the storage medium.

[0052] Storage media can include random access memory (RAM) or read-only memory (ROM). Storage media can be used to store instructions, programs, code, code sets, or instructions.

[0053] The display screen is used to show the user interface of each application.

[0054] In addition, those skilled in the art will understand that the structure of the terminal described above does not constitute a limitation on the terminal. The terminal may include more or fewer components, or combine certain components, or have different component arrangements. For example, the terminal may also include radio frequency circuits, input units, sensors, audio circuits, power supplies, and other components, which will not be described in detail here.

[0055] Example 1

[0056] like Figure 1 As shown in the figure, this embodiment proposes a machine vision-based intelligent hoisting monitoring method for suspension bridges, including:

[0057] Step 101: Collect synchronized images by visual units (e.g., cameras) deployed around the main cable, slings and component lifting points to form a visual tensor field covering the lifting operation area. Perform pixel clustering on the visual tensor field, remove the visual tensor fields of non-components to be lifted, and obtain the visual tensor field of the component to be lifted.

[0058] Specifically, the visual tensor field that forms the area covering the hoisting operation includes:

[0059] ,

[0060] in, For time Scene grid points The visual tensor field, The number of visual units, To make scene grid points Map back to the first Inverse projection of the pixel coordinates of the synchronized image of each visual unit For the first After distortion correction and brightness normalization of the synchronized images of each visual unit, the pixel coordinates are... The pixel value at that location.

[0061] Step 102: Generate the three-dimensional attitude vector of the component to be lifted, obtain the displacement of multiple sampling points of the component to be lifted, calculate the displacement measure within the time interval, use the weighted sum of the displacement measures of all sampling points as the dynamic deformation index, and calculate the temporal stability index of the component to be lifted based on the three-dimensional attitude vector of the component and the dynamic deformation index.

[0062] Specifically, generating the 3D pose vector of the component to be lifted includes: back-projecting the visual tensor field of the component to be lifted into a 3D point cloud fragment through multi-baseline triangulation;

[0063] A local reference coordinate system is fitted on a 3D point cloud fragment to obtain the local coordinate system of the component to be lifted. Under the local coordinate system of the component to be lifted, the 3D coordinates of the direction vector and the centroid of the outline of the component to be lifted are obtained by fitting the component.

[0064] Obtain the Euler angles of the component to be lifted on each direction vector. Combine the Euler angles on each direction vector with the three-dimensional coordinates of the centroid of the profile to form the three-dimensional attitude vector of the component to be lifted. The direction vector includes the X-axis, Y-axis and Z-axis of the component to be lifted. The Euler angles on each direction vector include: the roll angle corresponding to the X-axis of the component to be lifted (i.e., the rotation of the component to be lifted about its own X-axis), the pitch angle corresponding to the Y-axis of the component to be lifted (i.e., the rotation of the component to be lifted about its own Y-axis), and the yaw angle corresponding to the Z-axis of the component to be lifted (i.e., the rotation of the component to be lifted about its own Z-axis).

[0065] Specifically, the displacement measure within the calculation time interval includes:

[0066] ,

[0067] in, For time Time The first component to be lifted One sampling point, For time intervals, For time Time The first component to be lifted The displacement of each sampling point For time Time The first component to be lifted The displacement of each sampling point For time Time The first component to be lifted The baseline length between each sampling point and the component reference point To prevent dividing by zero and positive numbers.

[0068] Specifically, the time-series stability index of the component to be lifted, calculated based on its three-dimensional attitude vector and dynamic deformation index, includes:

[0069] ,

[0070] in, For time Time The temporal stability index of each component to be lifted. The weights of the norm and standard deviation, For window length Time within Time The three-dimensional attitude vector of the component to be lifted. The norm standard deviation, The weights of the moving average, For window length Time within Time Dynamic deformation index of a component to be lifted The moving average.

[0071] Step 103: The displacement time series of each cable is magnified and compared by visual tensor field to generate displacement spectrum of each cable. The directional consistency index and peak change degree are calculated based on displacement time series, and the spectral clustering degree is calculated based on displacement spectrum.

[0072] Specifically, the displacement time series of each cable is magnified and compared by visual tensor field to generate displacement spectrum of each cable. This includes sampling the cable pixels marked in visual tensor field along the cable direction, extracting the centroid position of the pixels in the time series, and obtaining the displacement time series of each cable.

[0073] The displacement time series of the sling is bandpass filtered and short-time Fourier transform is used to obtain the local energy spectrum, which is defined as the displacement spectrum of the sling.

[0074] Specifically, the calculation of directional consistency index and peak abruptness based on displacement time series, and the calculation of spectral clustering degree based on displacement spectrum include:

[0075] The consistency index for calculation direction includes:

[0076] ,

[0077] in, For time Time sling Indicators of directional consistency To calculate the window length, For indicator functions, For time Time sling The first time derivative of the displacement time series, For symbolic functions, To compare lag (number of frames or time interval);

[0078] Calculating the peak abrupt change includes:

[0079] ,

[0080] in, For time Time sling The peak abrupt change, From time Time The standard deviation of the peak value of the internal displacement time series. From time Time The average of the peak values ​​of the internal displacement time series. From time Time Internal slings displacement time series, To prevent dividing by zero and positive numbers;

[0081] Calculate the spectral clustering degree:

[0082] ,

[0083] in, For time Time sling Spectrum clustering degree, The frequency of the sling's vibration. For time At the time of suspension vibration frequency The normalized displacement spectrum of the suspension cable. This represents the number of frequency points.

[0084] Step 104: The weighted sum of the directional consistency index, peak mutation degree and spectral clustering degree is used as the sling stress trend factor. The sling stress trend factor is coupled with the time series stability index to generate a risk quantification index for the hoisting process, and the risk quantification index is used for monitoring.

[0085] Specifically, by coupling the sling stress trend factor with the time-series stability index, the resulting quantitative risk indicators for the lifting process include:

[0086] ,

[0087] in, For time Time The slings for the component to be lifted Risk quantification indicators The weights of the sling stress trend factor, For time Time sling The stress tendency factor of the suspension cable The weights for the time series stability index, For time Time The temporal stability index of each component to be lifted.

[0088] Specifically, monitoring based on risk quantification indicators includes: if time Time The slings for the component to be lifted Risk quantification indicators If the first warning threshold is exceeded, a high-risk warning will be issued. If the second warning threshold is exceeded, emergency control will be triggered. The second warning threshold is greater than the first warning threshold.

[0089] Example 2

[0090] like Figure 2 As shown, this embodiment proposes a machine vision-based intelligent hoisting monitoring system for suspension bridges, including:

[0091] The fusion module is used to acquire synchronous images through visual units deployed around the main cable, slings and component lifting points, forming a visual tensor field covering the lifting operation area. The visual tensor field is then clustered into pixels to remove the visual tensor fields of components not to be lifted, thus obtaining the visual tensor field of the component to be lifted.

[0092] Specifically, the visual tensor field that forms the area covering the hoisting operation includes:

[0093] ,

[0094] in, For time Scene grid points The visual tensor field, The number of visual units, To make scene grid points Map back to the first Inverse projection of the pixel coordinates of the synchronized image of each visual unit For the first After distortion correction and brightness normalization of the synchronized images of each visual unit, the pixel coordinates are... The pixel value at that location.

[0095] The attitude construction module is used to generate the three-dimensional attitude vector of the component to be lifted, obtain the displacement of multiple sampling points of the component to be lifted, calculate the displacement measure within the time interval, use the weighted sum of the displacement measures of all sampling points as the dynamic deformation index, and calculate the temporal stability index of the component to be lifted based on the component's three-dimensional attitude vector and the dynamic deformation index.

[0096] Specifically, generating the 3D pose vector of the component to be lifted includes: back-projecting the visual tensor field of the component to be lifted into a 3D point cloud fragment through multi-baseline triangulation;

[0097] A local reference coordinate system is fitted on a 3D point cloud fragment to obtain the local coordinate system of the component to be lifted. Under the local coordinate system of the component to be lifted, the 3D coordinates of the direction vector and the centroid of the outline of the component to be lifted are obtained by fitting the component.

[0098] Obtain the Euler angles of the component to be lifted on each direction vector. Combine the Euler angles on each direction vector with the three-dimensional coordinates of the centroid of the profile to form the three-dimensional attitude vector of the component to be lifted. The direction vector includes the X-axis, Y-axis and Z-axis of the component to be lifted. The Euler angles on each direction vector include the roll angle corresponding to the X-axis, the pitch angle corresponding to the Y-axis and the yaw angle corresponding to the Z-axis.

[0099] Specifically, the displacement measure within the calculation time interval includes:

[0100] ,

[0101] in, For time Time The first component to be lifted One sampling point, For time intervals, For time Time The first component to be lifted The displacement of each sampling point For time Time The first component to be lifted The displacement of each sampling point For time Time The first component to be lifted The baseline length between each sampling point and the component reference point To prevent dividing by zero and positive numbers.

[0102] Specifically, the time-series stability index of the component to be lifted, calculated based on its three-dimensional attitude vector and dynamic deformation index, includes:

[0103] ,

[0104] in, For time Time The temporal stability index of each component to be lifted. The weights of the norm and standard deviation, For window length Time within Time The three-dimensional attitude vector of the component to be lifted. The norm standard deviation, The weights of the moving average, For window length Time within Time Dynamic deformation index of a component to be lifted The moving average.

[0105] The module for calculating spectral clustering is used to amplify and compare the displacement time series of each cable through a visual tensor field, generate the displacement spectrum of each cable, calculate the directional consistency index and peak change degree based on the displacement time series, and calculate the spectral clustering degree based on the displacement spectrum.

[0106] Specifically, the displacement time series of each cable is magnified and compared by visual tensor field to generate displacement spectrum of each cable. This includes sampling the cable pixels marked in visual tensor field along the cable direction, extracting the centroid position of the pixels in the time series, and obtaining the displacement time series of each cable.

[0107] The displacement time series of the sling is bandpass filtered and short-time Fourier transform is used to obtain the local energy spectrum, which is defined as the displacement spectrum of the sling.

[0108] Specifically, the calculation of directional consistency index and peak abruptness based on displacement time series, and the calculation of spectral clustering degree based on displacement spectrum include:

[0109] The consistency index for calculation direction includes:

[0110] ,

[0111] in, For time Time sling Indicators of directional consistency To calculate the window length, For indicator functions, For time Time sling The first time derivative of the displacement time series, For symbolic functions, For comparison with lag;

[0112] Calculating the peak abrupt change includes:

[0113] ,

[0114] in, For time Time sling The peak abrupt change, From time Time The standard deviation of the peak value of the internal displacement time series. From time Time The average of the peak values ​​of the internal displacement time series. From time Time Internal slings displacement time series, To prevent dividing by zero and positive numbers;

[0115] Calculate the spectral clustering degree:

[0116] ,

[0117] in, For time Time sling Spectrum clustering degree, The frequency of the sling's vibration. For time At the time of suspension vibration frequency The normalized displacement spectrum of the suspension cable. This represents the number of frequency points.

[0118] The monitoring module is used to take the weighted sum of the directional consistency index, peak mutation degree and spectral clustering degree as the sling stress trend factor, couple the sling stress trend factor with the time series stability index to generate a risk quantification index for the hoisting process, and monitor according to the risk quantification index.

[0119] Specifically, by coupling the sling stress trend factor with the time-series stability index, the resulting quantitative risk indicators for the lifting process include:

[0120] ,

[0121] in, For time Time The slings for the component to be lifted Risk quantification indicators The weights of the sling stress trend factor, For time Time sling The stress tendency factor of the suspension cable The weights for the time series stability index, For time Time The temporal stability index of each component to be lifted.

[0122] Specifically, monitoring based on risk quantification indicators includes: if time Time The slings for the component to be lifted Risk quantification indicators If the first warning threshold is exceeded, a high-risk warning will be issued. If the second warning threshold is exceeded, emergency control will be triggered. The second warning threshold is greater than the first warning threshold.

[0123] Example 3

[0124] This invention also proposes a storage medium storing multiple instructions for implementing the aforementioned machine vision-based intelligent hoisting monitoring method for suspension bridges.

[0125] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0126] Optionally, in this embodiment, the storage medium is configured to store program code for performing the method steps as described in Embodiment 1.

[0127] Example 4

[0128] This invention also proposes an electronic device, including a processor and a storage medium connected to the processor. The storage medium stores multiple instructions, which can be loaded and executed by the processor to enable the processor to execute the aforementioned intelligent suspension bridge hoisting monitoring method based on machine vision.

[0129] Specifically, the electronic device in this embodiment can be a computer terminal, which may include one or more processors and a storage medium.

[0130] The storage medium can be used to store software programs and modules, such as the machine vision-based intelligent suspension bridge hoisting monitoring method in this embodiment of the invention. The corresponding program instructions / modules are executed by the processor through running the software programs and modules stored in the storage medium, thereby performing various functional applications and data processing, thus realizing the aforementioned machine vision-based intelligent suspension bridge hoisting monitoring method. The storage medium may include high-speed random access storage media, and may also include non-volatile storage media, such as one or more magnetic storage systems, flash memory, or other non-volatile solid-state storage media. In some instances, the storage medium may further include storage media remotely configured relative to the processor, which can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0131] The processor can call the information and application stored in the storage medium through the transmission system to execute the method steps as described in Example 1.

[0132] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0133] In the several embodiments provided by this invention, it should be understood that the disclosed technical content can be implemented in other ways. The system embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0136] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, optical disks, and other media capable of storing program code.

[0137] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A machine vision-based intelligent hoisting monitoring method for suspension bridges, characterized in that, include: Synchronous images are captured by visual units deployed around the main cable, slings, and component lifting points to form a visual tensor field covering the lifting operation area. The visual tensor field is then clustered into pixels to remove the visual tensor fields of components not to be lifted, thus obtaining the visual tensor field of the components to be lifted. Generate the three-dimensional attitude vector of the component to be lifted, obtain the displacement of multiple sampling points of the component to be lifted, and calculate the displacement measure within the time interval. The weighted sum of the displacement measures of all sampling points is used as the dynamic deformation index. The temporal stability index of the component to be lifted is calculated based on the three-dimensional attitude vector of the component and the dynamic deformation index. The displacement time series of each cable is magnified and compared by visual tensor field to generate displacement spectrum of each cable. The directional consistency index and peak change degree are calculated based on displacement time series, and the spectral clustering degree is calculated based on displacement spectrum. The weighted sum of directional consistency index, peak mutation degree and spectral clustering degree is used as the sling stress trend factor. The sling stress trend factor is coupled with the time series stability index to generate a risk quantification index for the hoisting process, and monitoring is carried out based on the risk quantification index.

2. The intelligent suspension bridge hoisting monitoring method based on machine vision as described in claim 1, characterized in that, The visual tensor field that forms the area covering the hoisting operation includes: ， in, For time Scene grid points The visual tensor field, The number of visual units, To make scene grid points Map back to the first Inverse projection of the pixel coordinates of the synchronized image of each visual unit For the first After distortion correction and brightness normalization of the synchronized images of each visual unit, the pixel coordinates are... The pixel value at that location.

3. The intelligent suspension bridge hoisting monitoring method based on machine vision as described in claim 1, characterized in that, The process of generating the 3D pose vector of the component to be lifted includes: back-projecting the visual tensor field of the component to be lifted into a 3D point cloud fragment through multi-baseline triangulation; A local reference coordinate system is fitted on a 3D point cloud fragment to obtain the local coordinate system of the component to be lifted. Under the local coordinate system of the component to be lifted, the 3D coordinates of the direction vector and the centroid of the outline of the component to be lifted are obtained by fitting the component. Obtain the Euler angles of the component to be lifted on each direction vector. Combine the Euler angles on each direction vector with the three-dimensional coordinates of the centroid of the profile to form the three-dimensional attitude vector of the component to be lifted. The direction vector includes the X-axis, Y-axis and Z-axis of the component to be lifted. The Euler angles on each direction vector include: the roll angle corresponding to the X-axis of the component to be lifted, the pitch angle corresponding to the Y-axis of the component to be lifted and the yaw angle corresponding to the Z-axis of the component to be lifted.

4. The intelligent suspension bridge hoisting monitoring method based on machine vision as described in claim 1, characterized in that, The displacement measure within the calculation time interval includes: ， in, For time Time The first component to be lifted One sampling point, For time intervals, For time Time The first component to be lifted The displacement of each sampling point For time Time The first component to be lifted The displacement of each sampling point For time Time The first component to be lifted The baseline length between each sampling point and the component reference point To prevent dividing by zero and positive numbers.

5. The intelligent suspension bridge hoisting monitoring method based on machine vision as described in claim 4, characterized in that, The time-series stability indices of the component to be lifted are calculated based on its three-dimensional attitude vector and dynamic deformation index, including: ， in, For time Time The temporal stability index of each component to be lifted. The weights of the norm and standard deviation, For window length Time within Time The three-dimensional attitude vector of the component to be lifted. The norm standard deviation, The weights of the moving average, For window length Time within Time Dynamic deformation index of a component to be lifted The moving average.

6. The intelligent suspension bridge hoisting monitoring method based on machine vision as described in claim 1, characterized in that, The displacement time series of each cable is magnified and compared by visual tensor field to generate displacement spectrum of each cable. This includes sampling the cable pixels marked in visual tensor field along the cable direction, extracting the centroid position of the pixels in the time series, and obtaining the displacement time series of each cable. The displacement time series of the sling is bandpass filtered and short-time Fourier transform is used to obtain the local energy spectrum, which is defined as the displacement spectrum of the sling.

7. The intelligent suspension bridge hoisting monitoring method based on machine vision as described in claim 1, characterized in that, The directional consistency index and peak abruptness are calculated based on the displacement time series, and the spectral clustering degree is calculated based on the displacement spectrum, including: The consistency index for calculation direction includes: ， in, For time Time sling Indicators of directional consistency To calculate the window length, For indicator functions, For time Time sling The first time derivative of the displacement time series, For symbolic functions, For comparison with lag; Calculating the peak abrupt change includes: ， in, For time Time sling The peak abruptness, From time Time The standard deviation of the peak value of the internal displacement time series. From time Time The average of the peak values ​​of the internal displacement time series. From time Time Internal slings displacement time series, To prevent dividing by zero and positive numbers; Calculate the spectral clustering degree: ， in, For time Time sling Spectrum clustering degree, The frequency of the sling's vibration. For time At the time of suspension vibration frequency The normalized displacement spectrum of the suspension cable. This represents the number of frequency points.

8. The intelligent suspension bridge hoisting monitoring method based on machine vision as described in claim 1, characterized in that, By coupling the sling stress trend factor with the time-series stability index, a risk quantification index for the lifting process is generated, including: ， in, For time Time The slings for the component to be lifted Risk quantification indicators The weights of the sling stress trend factor, For time Time sling The stress tendency factor of the suspension cable The weights for the time series stability index, For time Time The temporal stability index of each component to be lifted.

9. The intelligent suspension bridge hoisting monitoring method based on machine vision as described in claim 2, characterized in that, Monitoring based on risk quantification indicators includes: if time Time The slings for the component to be lifted Risk quantification indicators If the first warning threshold is exceeded, a high-risk warning will be issued. If the second warning threshold is exceeded, emergency control will be triggered. The second warning threshold is greater than the first warning threshold.

10. A machine vision-based intelligent hoisting monitoring system for suspension bridges, characterized in that, include: The fusion module is used to acquire synchronous images through visual units deployed around the main cable, slings and component lifting points, forming a visual tensor field covering the lifting operation area. The visual tensor field is then clustered into pixels to remove the visual tensor fields of components not to be lifted, thus obtaining the visual tensor field of the component to be lifted. The attitude construction module is used to generate the three-dimensional attitude vector of the component to be lifted, obtain the displacement of multiple sampling points of the component to be lifted, calculate the displacement measure within the time interval, use the weighted sum of the displacement measures of all sampling points as the dynamic deformation index, and calculate the temporal stability index of the component to be lifted based on the component's three-dimensional attitude vector and the dynamic deformation index. The module for calculating spectral clustering is used to amplify and compare the displacement time series of each cable through a visual tensor field, generate the displacement spectrum of each cable, calculate the directional consistency index and peak change degree based on the displacement time series, and calculate the spectral clustering degree based on the displacement spectrum. The monitoring module is used to take the weighted sum of the directional consistency index, peak mutation degree and spectral clustering degree as the sling stress trend factor, couple the sling stress trend factor with the time series stability index to generate a risk quantification index for the hoisting process, and monitor according to the risk quantification index.

Images