A method and apparatus for installing bridge guardrails in windy and dusty environments.
By acquiring hoisting environment parameters and multi-view image sequences during bridge guard plate hoisting, and utilizing a pendulum motion model and image restoration technology, the accuracy and safety issues of bridge guard plate installation in strong wind and dusty environments were solved, and the guard plate was smoothly and smoothly positioned.
Patent Information
- Application Number
- CN202611570569.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-12
- Publication Date
- 2026-11-10
AI Technical Summary
In windy and dusty environments, during the hoisting of bridge guardrails, image degradation in the vision system and swaying of the guardrail suspension lead to unstable position measurement, making it difficult to accurately position the guardrails and prone to interference or collisions, thus failing to meet the installation accuracy requirements.
By acquiring hoisting environment state parameters and multi-view image sequences, the pose is predicted using a pendulum motion model, image restoration processing is performed, edge features are extracted and visual pose fusion is optimized, and contact status is detected by combining edge grayscale profile curvature changes. Control parameters are adaptively adjusted to ensure the guard plate is smoothly positioned.
Maintaining image processing accuracy in windy and dusty environments, avoiding structural damage caused by hard contact, and achieving smooth and stable placement of the protective plate improves installation accuracy and safety.
Smart Images

Figure CN123064098A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of bridge construction technology, and more specifically, to a method and apparatus for installing bridge guardrails in windy and dusty environments. Background Technology
[0002] Bridge guardrail installation typically involves hoisting large guardrails from the ground to their installation position on the bridge deck using lifting equipment. A vision system is then used to acquire the guardrail's positional information to guide the hoisting equipment in positioning and adjustment. Since the installation height usually exceeds eight meters, the construction area generates significant dust due to bridge deck operations. This dust, amplified by high-altitude gusts of wind, permeates the installation area, causing severe degradation of the images captured by the vision system. Simultaneously, the guardrails, suspended from steel cables, are prone to complex swaying in strong winds and dusty conditions, further exacerbating the instability of the vision measurement and leading to jitter in the positional output. With these interfering factors combined, a significant error exists between the actual position of the guardrail and the vision measurement results, making it difficult for the guardrail to accurately reach its installation position. Interference or collisions between the guardrail and the mounting base are also likely to occur, failing to meet the required positioning accuracy. Summary of the Invention
[0003] The purpose of this invention is to provide a method and apparatus for installing bridge guardrails in windy and dusty environments, thereby improving the aforementioned problems. To achieve the above objective, the technical solution adopted by this invention is as follows:
[0004] Firstly, this application provides a method for installing bridge guardrails in windy and dusty environments, including:
[0005] Acquire the hoisting environment state parameters and the multi-view image sequence of the protective plate; based on the hoisting environment state parameters, predict the pose of the protective plate using a pendulum motion model.
[0006] The multi-view image sequence of the protective plate is subjected to image restoration processing to obtain the restored image;
[0007] Extract the edge features of the protective plate from the restored image, and optimize the visual pose of the protective plate based on the edge features and reprojection error;
[0008] The visual pose is fused with the predicted pose, and the position of the guard plate is adjusted according to the deviation between the fused pose and the target pose.
[0009] The grayscale profile curvature change of the edge of the guard plate is detected by using the edge features of the guard plate to determine the contact state between the guard plate and the mounting base. The control parameters are adaptively adjusted according to the contact state to make the guard plate smoothly positioned.
[0010] Secondly, this application also provides a bridge guard plate installation device suitable for strong wind and dusty environments, comprising:
[0011] The acquisition module is used to acquire hoisting environment state parameters and multi-view image sequences of the protective plate, and to predict the pose of the protective plate by using a pendulum motion model based on the hoisting environment state parameters.
[0012] The image restoration module is used to perform image restoration processing on the multi-view image sequence of the protective plate to obtain the restored image;
[0013] The visual pose calculation module is used to extract the edge features of the protective plate on the restored image, and optimize the visual pose of the protective plate based on the edge features and reprojection error.
[0014] The pose fusion module is used to fuse the visual pose with the predicted pose and adjust the position of the guard plate according to the deviation between the fused pose and the target pose.
[0015] The positioning control module is used to detect the grayscale profile curvature change of the edge of the guard plate using the edge features of the guard plate to determine the contact state between the guard plate and the mounting base, and adaptively adjust the control parameters according to the contact state to make the guard plate smoothly positioned.
[0016] Thirdly, this application also provides a bridge guard plate installation device suitable for strong wind and dusty environments, comprising:
[0017] Memory, used to store computer programs;
[0018] A processor is used to execute the computer program to implement the steps of the bridge guard plate installation method suitable for strong wind and dust environments.
[0019] Fourthly, this application also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described bridge guard plate installation method suitable for strong wind and dusty environments.
[0020] The beneficial effects of this invention are as follows:
[0021] 1. This invention automatically selects different image restoration modes based on the energy concentration of the swing spectrum, which can maintain optimal performance under different hoisting swing conditions and solves the problem of decreased pose calculation accuracy caused by image degradation in strong wind and dust environments.
[0022] 2. This invention senses the contact state by detecting changes in the curvature of the grayscale profile at the edge of the protective plate, and adaptively adjusts the control parameters according to the contact state to achieve a smooth transition from rigidity to smooth sliding, thus avoiding structural damage caused by hard contact.
[0023] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description
[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a schematic diagram of the bridge guard plate installation method suitable for strong wind and dust environments as described in this embodiment of the invention;
[0026] Figure 2 This is a schematic diagram of the hoisting scenario described in an embodiment of the present invention;
[0027] Figure 3 This is a schematic diagram of the swing coordinate system of the guard plate as described in an embodiment of the present invention;
[0028] Figure 4 This is a schematic diagram of the bridge guard plate installation device suitable for strong wind and dust environments as described in this embodiment of the invention;
[0029] Figure 5 This is a schematic diagram of the bridge guard plate installation equipment suitable for strong wind and dust environments, as described in an embodiment of the present invention.
[0030] Marked in the image:
[0031] 01. Suspension rope; 02. Guard plate; 03. Mounting base; 04. Bridge deck; 05. Industrial camera; 06. Vibration damping bracket.
[0032] 800. Bridge guard plate installation equipment suitable for strong wind and dusty environments; 801. Processor; 802. Memory; 803. Multimedia components; 804. I / O interface; 805. Communication components. Detailed Implementation
[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0034] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0035] Example 1:
[0036] This embodiment provides a bridge guard plate installation method suitable for strong wind and dusty environments, applicable to bridge guard plate hoisting and installation construction scenarios in high-altitude strong wind and dusty environments.
[0037] See Figure 1 The figure shows that this method includes:
[0038] S1. During the process of lifting the guard plate 02 from the ground to the installation height of the bridge deck 04 by the hoisting equipment, the hoisting environment state parameters and the multi-view image sequence of the guard plate 02 are acquired. Based on the hoisting environment state parameters, the predicted pose of the guard plate 02 is predicted by a pendulum motion model.
[0039] Preferred, such as Figure 2 As shown, before the installation of the bridge guard plate 02 begins, a multi-view vision system is deployed and calibrated. Specifically, one industrial camera 05 is placed on each side and directly in front of the installation area. The three industrial cameras 05 are fixed to the bridge deck 04 via vibration damping brackets 06, and their optical axes converge at the center of the installation area.
[0040] Specifically, the hoisting environment parameters include the length of the hoisting rope. The horizontal and vertical components of the suspension rope, and the swing angle of the guard plate 02. and angular rate of oscillation The swing angle of the guard plate 02 is... and angular rate of oscillation Due to the length of the suspension rope The horizontal and vertical components of the suspension rope 01 were calculated.
[0041] Furthermore, based on the current hoisting environment parameters, the swing angle and swing rate of the guard plate 02 at the next moment are predicted, using the following formula:
[0042] (1)
[0043] In the formula, , Indicates in The subsequent oscillation angle and oscillation angular rate, The time interval between two adjacent frames. It is the acceleration due to gravity. Indicates the current time The swing angle and swing angular rate.
[0044] Specifically, such as Figure 3 As shown, a swing coordinate system is established with the lifting point as the origin. The horizontal direction of the guard plate 02 in the swing plane is the x-axis, and the vertical downward direction is the z-axis. Then the position vector of the guard plate 02 relative to the lifting point is:
[0045] (2)
[0046] In the formula, The predicted position vector for guard plate 02. This is the position vector of the lifting point.
[0047] Furthermore, based on the swing angle The predicted attitude rotation matrix of guard plate 02 is obtained. Predicted location With the predicted attitude rotation matrix By combining the results, we obtain the predicted six-DOF pose of guard plate 02 at the next moment. .
[0048] Based on the above embodiments, this method further includes:
[0049] S2. Perform image restoration processing on the multi-view image sequence of the protective plate 02 to obtain the restored image;
[0050] Specifically, step S2 includes:
[0051] S21. Calculate the oscillation spectrum energy concentration based on the multi-view image sequence;
[0052] Specifically, a time-domain Fourier transform is performed on the N-frame image sequence to obtain the spectrum of each pixel along the time axis. For each frame, the energy spectrum of all its pixels is calculated. ,in, Represents frequency, and its value range is: , This represents the camera's sampling frame rate.
[0053] Furthermore, the total energy across the entire frequency range is calculated. ;
[0054] Meanwhile, based on the simple pendulum motion model, the fundamental frequency of the oscillation is determined by the length of the suspension rope:
[0055] (3)
[0056] In the formula, For the fundamental frequency of the swing, This is the acceleration due to gravity.
[0057] energy spectrum The swing fundamental frequency mentioned in The energy value at the location is used as the peak value. , by the peak and total energy The energy concentration of the oscillating spectrum was calculated. :
[0058] (4)
[0059] Among them, the energy concentration of the oscillating spectrum This reflects the degree of concentration of the swing energy of the guard plate 02 in the frequency spectrum: The larger the value, the more pronounced the periodic narrow-band characteristics of the oscillation. The smaller the value, the more non-periodic the oscillation.
[0060] S22. Based on the energy concentration of the oscillation spectrum Select image restoration mode:
[0061] when When selecting the frequency domain comb filtering dust removal mode, this mode utilizes the narrow-band frequency domain characteristics of the oscillating signal to separate the target signal from broadband dust noise using a comb filter; wherein, As a first preset threshold, and as an optional example, .
[0062] when When selecting the spatial domain dust probability weighted defogging mode, the following is provided: As a second preset threshold, and as an optional example, .
[0063] when When selecting the frequency domain comb filtering dust removal mode and the spatial domain dust probability weighted defogging mode, a fusion mode is chosen.
[0064] Preferred, The closer The greater the weight of the frequency domain dust removal result, The closer The greater the weight of the spatial domain defogging result, the smoother the dust removal effect will be during mode switching.
[0065] S23. Remove dust from the multi-view image sequence using the selected dust removal mode to obtain the restored image;
[0066] Specifically, in step S23, "removing dust from the multi-view image sequence using the selected frequency domain comb filtering dust removal mode" includes:
[0067] S231. Perform a time-domain Fourier transform on the multi-view image sequence to obtain the frequency-domain energy distribution. ;
[0068] S232. Determine the spectrum signal-to-peak ratio from the frequency domain energy distribution, and select the time domain window length based on the spectrum signal-to-peak ratio;
[0069] Specifically, the average energy of the non-peak region in the spectrum is determined based on the frequency domain energy distribution. According to average energy and peak Calculate the signal-to-peak ratio of the spectrum :
[0070] (5)
[0071] when When the value is large, a shorter time-domain window is used to improve the real-time performance of dust removal; when When the size is small, a longer time-domain window is used to accumulate more frame energy and enhance spectral resolution.
[0072] S233. Extract the hoisting rope length from the hoisting environment state parameters to determine the swing fundamental frequency, and construct a comb filter based on the swing fundamental frequency;
[0073] In this embodiment, the calculation method of the swing fundamental frequency has been described in detail in step S21, and will not be repeated here.
[0074] Specifically, using the swing fundamental frequency The comb filter is constructed using its various harmonics as the passband center frequency. The passband width of the comb filter is adaptively adjusted based on the time-domain stability of the oscillation frequency.
[0075] S234. The comb filter is used to perform frequency domain dust removal on the multi-view image sequence to obtain the restored image;
[0076] Specifically, the frequency domain signal obtained after applying a comb filter to the time-domain Fourier transform is... The filtered frequency domain signal is obtained after filtering out non-target frequency components. :
[0077] (6)
[0078] For the filtered frequency domain signal Perform an inverse Fourier transform to obtain the restored image.
[0079] Specifically, in step S23, "removing dust from the multi-view image sequence using the selected spatial domain dust probability weighted defogging mode" includes:
[0080] S235. Perform preliminary transmittance estimation on a single frame image in the multi-view image sequence to obtain an initial transmittance value, and calculate the degree of transmittance mutation based on the initial transmittance value. ;
[0081] S236. Calculate the local gray-level variance of the single-frame image, and map the local gray-level variance and the degree of abrupt change in transmittance to the probability of dust accumulation. :
[0082] (7)
[0083] In the formula, and These represent the local grayscale variance reference value and the transmittance abrupt change reference value for the dust-free area, respectively. , The adjustment coefficient is determined by coupling the vibration amplitude measured by the accelerometer on the camera mount.
[0084] S237. Determine the edge visibility index based on the multi-view image sequence, and use the edge visibility index to correct the atmospheric scattering coefficient. Specifically, the atmospheric scattering coefficient is used to characterize the intensity of light scattering and extinction caused by dust. When the edge visibility index is low, it indicates that dust is causing severe image degradation. In this case, increasing the atmospheric scattering coefficient can enhance the defogging intensity.
[0085] Furthermore, obtain the pixel-by-pixel scene depth of a single frame image. ,in, Indicates pixel position;
[0086] According to the dust probability Pixel-by-pixel scene depth and atmospheric scattering coefficient Update transmittance :
[0087] (8)
[0088] In the formula, The dust enhancement factor is determined by the average dust probability across the entire map. is the atmospheric scattering coefficient.
[0089] S238. The single-frame image is dehazed using the updated transmittance to obtain the restored image:
[0090] (9)
[0091] In the formula, For pixels Image intensity after dehazing at the location, For the original image in pixels The intensity value at that location, This represents the global atmospheric light value.
[0092] Based on the above embodiments, this method further includes:
[0093] S3. Extract the edge features of the protective plate 02 from the restored image, and optimize the visual pose of the protective plate 02 based on the edge features of the protective plate 02 and the reprojection error;
[0094] Specifically, step S3 includes:
[0095] S31. Perform edge detection on the restored image to obtain pixel-level edge points; preferably, the Canny operator can be used to perform pixel-level edge detection on the restored image.
[0096] S32. Take multiple neighboring points along the gradient direction at the pixel-level edge point, and determine the weighting weight of each neighboring point based on the consistency between the gradient direction of each neighboring point and the gradient direction of the center point.
[0097] Taking each pixel-level edge point as the center point, take along its gradient direction For each of the ...
[0098] (10)
[0099] In the formula, The normalized consistency weight, with a value range of [0,1], reflects the consistency between the neighborhood points and the center point in the gradient direction. The closer it is to 1, the more consistent the gradient directions of the two gradients are; For the first Gradient direction angle of each neighboring point; The gradient direction angle of the center pixel.
[0100] S33. Calculate the sub-pixel offset based on the weighted weights to obtain the sub-pixel level edge:
[0101] (11)
[0102] In the formula, Subpixel-level edges, For the first The gray-level difference between neighboring points along the gradient direction and the center pixel. It is the cosine value.
[0103] S34. Obtain the three-dimensional model of the protective plate 02, project the edges of the three-dimensional model onto the image planes of each camera, and construct an optimization objective function based on the reprojection error between the projected edges and the sub-pixel level edges;
[0104] Specifically, a three-dimensional CAD model of the protective plate 02 is obtained. The three-dimensional CAD model includes the geometric dimensions and edge contours of the protective plate 02. The edge contours of the three-dimensional CAD model are projected onto the image planes of each camera to obtain the projected edges from each viewpoint.
[0105] Furthermore, an optimization objective function is constructed based on the reprojection error between the projected edge and the sub-pixel level edge:
[0106] (12)
[0107] In the formula, To optimize the objective function value, For the first The reprojection error of each matching edge point For the first The confidence weight of each matching edge point For the number of cameras, For the first Number of matching edge points under each camera viewpoint.
[0108] S35. Iteratively solve the optimization objective function to obtain the visual pose of the protective plate 02;
[0109] Preferably, the Levenberg-Marquardt algorithm is used to iteratively solve the objective function, wherein the iteration variables are the six-degree-of-freedom pose parameters of the guard plate 02, including three translational components and three rotational components.
[0110] In each iteration, the pose parameters of the protective plate 02 are updated, and the reprojection error between the projected edge and the extracted edge is recalculated until the objective function converges, thus obtaining the visual pose of the protective plate 02. .
[0111] Based on the above embodiments, this method further includes:
[0112] S4. Fuse the visual pose with the predicted pose, and adjust the position of the guard plate 02 according to the deviation between the fused pose and the target pose;
[0113] In this embodiment, a Kalman filter framework is used to achieve adaptive fusion of visual pose and model predicted pose, including a prediction stage and an update stage.
[0114] Specifically, in the prediction phase, the pose will be predicted. As the predicted values for the Kalman filter, construct the state transition matrix. Covariance estimate at the current time The state transition matrix The linearized formula (1) is used to recursively calculate the pose and swing state of the guard plate 02 from the current moment to the next moment.
[0115] Based on the state transition matrix Covariance Estimation Calculate the predicted covariance :
[0116] (13)
[0117] In the formula, The noise covariance matrix during the oscillation process is... To predict covariance, reflecting the uncertainty of the model's prediction results.
[0118] During the update phase of the Kalman filter, the visual pose is... The predicted pose is corrected based on the observed values.
[0119] Furthermore, based on the predicted covariance Determine the Kalman gain:
[0120] (14)
[0121] In the formula, For Kalman gain, To predict covariance, The observation covariance is the visual pose.
[0122] During the fusion process, the fused pose is calculated based on the Kalman gain:
[0123] (15)
[0124] In the formula, The pose is the fused six-degree-of-freedom pose.
[0125] After obtaining the fused pose, calculate the deviation between the fused pose and the pre-stored target mounting pose:
[0126] (16)
[0127] In the formula, The deviations are indicated, including positional and orientation deviations. Control commands are generated based on these deviations to drive the hoisting equipment to perform corresponding positional adjustments in the horizontal and vertical directions, gradually bringing the guard plate 02 closer to the target installation position.
[0128] Based on the above embodiments, this method further includes:
[0129] S5. Detect the grayscale profile curvature change of the edge of the guard plate 02 using the edge features of the guard plate 02 to determine the contact state between the guard plate 02 and the mounting base 03, and adaptively adjust the control parameters according to the contact state to make the guard plate 02 smoothly positioned.
[0130] Specifically, in step S5, "detecting the change in grayscale profile curvature of the edge of the protective plate 02 using the edge features of the protective plate 02 to determine the contact state between the protective plate 02 and the mounting base 03" includes:
[0131] S51. Based on the reference value of the gray-scale profile curvature of the edge of the guard plate 02 in the non-contact state, define the contact state index according to the amount of change of the current gray-scale profile curvature of the edge of the guard plate 02 relative to the reference value.
[0132] Specifically, images of the edges of the protective plate 02 and the mounting base 03 are acquired, and for each edge point in the image, a grayscale profile is sampled along its normal direction. ,in, This is the position variable along the normal direction.
[0133] Calculate the grayscale profile at the edge center position Curvature at:
[0134] (17)
[0135] in, For curvature, , These are the first and second derivatives of the grayscale profile along the normal direction, respectively.
[0136] Iterate through all edge points and use the maximum curvature value among all edge points as the curvature reference value in the non-contact state. The curvature reference value in a non-contact state. Based on this, define the contact status index at the current moment. :
[0137] (18)
[0138] In the formula, This represents the maximum curvature of each edge point at the current moment.
[0139] S52. Calculate the rate of change of the contact state index over time: ;
[0140] S53. Determine the contact state between the protective plate 02 and the mounting base 03 based on the contact state index and its rate of change:
[0141] 1) When and This indicates that the protective plate 02 and the mounting base 03 are in a state of being far from contact. In this embodiment, the first indicator threshold is preset. ;
[0142] 2) When and This indicates that the protective plate 02 is approaching the mounting base 03, and the degree of contact is continuously increasing but has not yet reached a strong contact state; it is a state approaching contact. In this embodiment, the second indicator threshold is preset. ;
[0143] 3) When and When the edge of the guard plate 02 has made physical contact with the edge of the mounting base 03, it indicates that the contact state has been entered.
[0144] 4) When and When this occurs, it indicates that the guard plate 02 has entered the contact state but the contact is weakening, which is the contact release state.
[0145] S54. Execute the corresponding control strategy according to the contact state determination result to make the guard plate 02 smoothly take place.
[0146] 1) When the condition is determined to be far from contact, the distance between the guard plate 02 and the mounting base 03 is large, and there is no risk of physical interference. The speed command at this time is:
[0147] (19)
[0148] In the formula, For speed commands, For position control gain, This refers to the pose deviation.
[0149] 2) When the state is determined to be approaching contact, the speed command is:
[0150] (20)
[0151] In the formula, The initial position control gain, For position control gain, depending on the contact state index It decreases linearly as it increases.
[0152] 3) When it is determined that the contact state has been entered, the edge of the guard plate 02 has made physical contact with the edge of the mounting base 03. The speed command at this time is:
[0153] ;(twenty one)
[0154] In the formula, The damping coefficient is... For stiffness gain, both vary with contact state indices. They change continuously according to the following rules:
[0155] ;(twenty two)
[0156] In the formula, The initial damping coefficient, The preset damping adjustment coefficient, This represents the initial stiffness gain.
[0157] Among them, the speed command Subject to maximum permissible speed The constraints, namely:
[0158] ;(twenty three)
[0159] Specifically, the maximum permissible speed The calculation formula is:
[0160] ;(twenty four)
[0161] In the formula, The initial maximum permissible speed, This is the preset speed attenuation coefficient.
[0162] Following the aforementioned control commands, the guard plate 02 smoothly and gradually slides into the mounting base 03, contacting the status indicator. It gradually increases and tends to stabilize at a high value.
[0163] 4) When it is determined that the contact is being released, maintain the current control command unchanged until... Stable close to zero or Falling back to the following.
[0164] Once the protective plate 02 is fully in place, the contact status indicators... Stabilize at a value close to 1, and maintain control parameters in a compliant positioning state with low stiffness and high damping to complete the installation of guard plate 02.
[0165] Example 2:
[0166] like Figure 4 As shown, this embodiment provides a bridge guard plate 02 installation device suitable for strong wind and dusty environments. The device includes:
[0167] The acquisition module is used to acquire hoisting environment state parameters and multi-view image sequences of the protective plate 02, and to predict the pose of the protective plate 02 by using a pendulum motion model based on the hoisting environment state parameters.
[0168] The image restoration module is used to perform image restoration processing on the multi-view image sequence of the protective plate 02 to obtain the restored image;
[0169] The visual pose calculation module is used to extract the edge features of the protective plate 02 on the restored image, and optimize the visual pose of the protective plate 02 based on the edge features of the protective plate 02 and the reprojection error.
[0170] The pose fusion module is used to fuse the visual pose with the predicted pose and adjust the position of the guard plate 02 according to the deviation between the fused pose and the target pose.
[0171] The positioning control module is used to detect the grayscale profile curvature change of the edge of the guard plate 02 using the edge features of the guard plate 02, so as to determine the contact state between the guard plate 02 and the mounting base 03, and adaptively adjust the control parameters according to the contact state to make the guard plate 02 smoothly positioned.
[0172] Based on the above embodiments, the image restoration module includes:
[0173] The calculation unit is used to calculate the oscillation spectrum energy concentration based on the multi-view image sequence;
[0174] The dust removal unit is used to select the frequency domain comb filter dust removal mode when the energy concentration of the oscillating spectrum is greater than or equal to a first preset threshold.
[0175] The defogging unit is used to select a spatial domain dust probability weighted defogging mode when the energy concentration of the oscillating spectrum is less than or equal to a second preset threshold.
[0176] The fusion processing unit is used to select a fusion mode of the frequency domain comb filtering dust removal mode and the spatial domain dust probability weighted defogging mode when the energy concentration of the oscillating spectrum is less than the first preset threshold and greater than the second preset threshold.
[0177] Based on the above embodiments, the dust removal unit includes:
[0178] The transformation unit is used to perform a time-domain Fourier transform on the multi-view image sequence to obtain the frequency-domain energy distribution;
[0179] A window selection unit is used to determine the spectrum signal-to-peak ratio from the frequency domain energy distribution and select the time domain window length based on the spectrum signal-to-peak ratio.
[0180] A filter construction unit is used to extract the hoisting rope length from the hoisting environment state parameters to determine the swing fundamental frequency, and to construct a comb filter based on the swing fundamental frequency.
[0181] The first restoration unit is used to perform frequency domain dust removal processing on the multi-view image sequence using the comb filter to obtain the restored image.
[0182] Based on the above embodiments, the defogging unit includes:
[0183] The preliminary estimation unit is used to perform a preliminary estimate of the transmittance of a single frame image in the multi-view image sequence to obtain an initial value of transmittance, and to calculate the degree of transmittance change based on the initial value of transmittance.
[0184] The probability mapping unit is used to calculate the local gray-level variance of the single-frame image and map the local gray-level variance and the degree of abrupt change in transmittance into the probability of dust.
[0185] The transmittance update unit is used to determine the edge visibility index based on the multi-view image sequence, correct the atmospheric scattering coefficient using the edge visibility index, and update the transmittance using the dust probability and the atmospheric scattering coefficient.
[0186] The second restoration unit is used to perform dehazing on the single-frame image using the updated transmittance to obtain the restored image.
[0187] Based on the above embodiments, the visual pose calculation module includes:
[0188] An edge detection unit is used to perform edge detection on the restored image to obtain pixel-level edge points;
[0189] The weight determination unit is used to take multiple neighborhood points along the gradient direction at the pixel-level edge point, and determine the weight of each neighborhood point based on the consistency between the gradient direction of each neighborhood point and the gradient direction of the center point.
[0190] A positioning unit is used to calculate the sub-pixel offset based on the weighted weights to obtain the sub-pixel level edge.
[0191] The function construction unit is used to obtain the three-dimensional model of the protective plate 02, project the edges of the three-dimensional model onto the image planes of each camera, and construct an optimization objective function based on the reprojection error between the projected edges and the sub-pixel level edges.
[0192] The iterative solution unit is used to iteratively solve the optimization objective function to obtain the visual pose of the protective plate 02.
[0193] Based on the above embodiments, the positioning control module includes:
[0194] The index definition unit is used to define the contact state index based on the reference value of the gray-scale profile curvature of the edge of the guard plate 02 in the non-contact state, and according to the amount of change of the current gray-scale profile curvature of the edge of the guard plate 02 relative to the reference value.
[0195] A rate of change calculation unit is used to calculate the rate of change of the contact state index over time.
[0196] The contact state determination unit is used to determine the contact state between the guard plate 02 and the mounting base 03 based on the contact state index and its rate of change.
[0197] Based on the above embodiments, the contact state determination unit includes:
[0198] The first determination subunit is used to determine a state of being far from contact when the contact state index is less than the first index threshold and the rate of change of the contact state index is close to zero.
[0199] The second determination subunit is used to determine that the contact state is approaching the contact state when the contact state index is between the first index threshold and the second index threshold and the rate of change of the contact state index is positive.
[0200] The third determination subunit is used to determine that a contact state has been entered when the contact state index exceeds the second index threshold.
[0201] It should be noted that the specific manner in which each module performs its operation in the apparatus described in the above embodiments has been described in detail in the embodiments of the method, and will not be elaborated here.
[0202] Example 3:
[0203] Corresponding to the above method embodiments, this embodiment also provides a bridge guard plate installation device suitable for strong wind and dusty environments. The bridge guard plate installation device suitable for strong wind and dusty environments described below and the bridge guard plate installation method suitable for strong wind and dusty environments described above can be referred to in correspondence.
[0204] Figure 5 This is a block diagram illustrating a bridge guard plate installation device 800 suitable for strong wind and dusty environments, according to an exemplary embodiment. Figure 5 As shown, the bridge guard plate installation device 800 suitable for strong wind and dusty environments may include: a processor 801 and a memory 802. The bridge guard plate installation device 800 may also include one or more of the following: a multimedia component 803, an I / O interface 804, and a communication component 805.
[0205] The processor 801 controls the overall operation of the bridge guardrail installation device 800 suitable for strong wind and dusty environments to complete all or part of the steps in the bridge guardrail installation method for strong wind and dusty environments described above. The memory 802 stores various types of data to support the operation of the bridge guardrail installation device 800. This data may include, for example, instructions for any application or method operating on the bridge guardrail installation device 800, as well as application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 802 can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 802 or transmitted via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. I / O interface 804 provides an interface between processor 801 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical. Communication component 805 is used for wired or wireless communication between the bridge guard plate mounting device 800 (suitable for windy and dusty environments) and other devices. Wireless communication includes Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof. Therefore, the corresponding communication component 805 may include a Wi-Fi module, a Bluetooth module, or an NFC module.
[0206] In an exemplary embodiment, the bridge guard plate installation device 800 suitable for strong wind and dusty environments can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described bridge guard plate installation method suitable for strong wind and dusty environments.
[0207] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, these program instructions implement the steps of the bridge guard plate installation method for strong wind and dusty environments described above. For example, the computer-readable storage medium may be the memory 802 including the program instructions described above. These program instructions may be executed by the processor 801 of the bridge guard plate installation device 800 for strong wind and dusty environments to complete the bridge guard plate installation method for strong wind and dusty environments described above.
[0208] Example 4:
[0209] Corresponding to the above method embodiments, this embodiment also provides a readable storage medium. The readable storage medium described below can be referred to in conjunction with the bridge guard plate installation method suitable for strong wind and dust environments described above.
[0210] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the bridge guard plate installation method applicable to strong wind and dust environments described in the above method embodiments.
[0211] Specifically, the readable storage medium can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other readable storage medium capable of storing program code.
[0212] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
[0213] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for installing bridge guardrails in windy and dusty environments, characterized in that, include: Acquire the hoisting environment state parameters and the multi-view image sequence of the guard plate (02). Based on the hoisting environment state parameters, predict the pose of the guard plate (02) by using a pendulum motion model. The multi-view image sequence of the protective plate (02) is subjected to image restoration processing to obtain the restored image; The edge features of the protective plate (02) are extracted from the restored image, and the visual pose of the protective plate (02) is optimized and calculated based on the edge features of the protective plate (02) and the reprojection error. The visual pose is fused with the predicted pose, and the position of the guard plate (02) is adjusted according to the deviation between the fused pose and the target pose. The edge features of the guard plate (02) are used to detect the change in grayscale profile curvature of the edge of the guard plate (02) to determine the contact state between the guard plate (02) and the mounting base (03). The control parameters are adaptively adjusted according to the contact state to make the guard plate (02) smoothly positioned.
2. The bridge guard plate installation method suitable for strong wind and dusty environments according to claim 1, characterized in that, Image restoration processing is performed on the multi-view image sequence of the protective plate (02), including: Calculate the oscillation spectrum energy concentration based on the multi-view image sequence; When the energy concentration of the swing spectrum is greater than or equal to the first preset threshold, the frequency domain comb filter dust removal mode is selected. When the energy concentration of the swing spectrum is less than or equal to the second preset threshold, the spatial domain dust probability weighted defogging mode is selected. When the energy concentration of the oscillating spectrum is less than the first preset threshold and greater than the second preset threshold, the fusion mode of the frequency domain comb filtering dust removal mode and the spatial domain dust probability weighted defogging mode is selected. The multi-view image sequence is cleaned using the selected dust removal mode to obtain the restored image.
3. The bridge guard plate installation method suitable for strong wind and dusty environments according to claim 2, characterized in that, The multi-view image sequence is dusted using the selected frequency domain comb filtering dust removal mode, including: The frequency domain energy distribution is obtained by performing a time-domain Fourier transform on the multi-view image sequence; The frequency domain energy distribution determines the spectral signal-to-peak ratio, and the time domain window length is selected based on the spectral signal-to-peak ratio. The hoisting environment state parameters are used to extract the hoisting rope length to determine the swing fundamental frequency, and a comb filter is constructed based on the swing fundamental frequency. The comb filter is used to perform frequency domain dust removal on the multi-view image sequence to obtain the restored image.
4. The bridge guard plate installation method suitable for strong wind and dusty environments according to claim 2, characterized in that, The multi-view image sequence is dehazed using a selected spatial domain dust probability-weighted dehazing mode, including: A preliminary transmittance estimate is performed on a single frame image in the multi-view image sequence to obtain an initial transmittance value. The degree of transmittance mutation is then calculated based on the initial transmittance value. Calculate the local grayscale variance of the single-frame image, and map the local grayscale variance and the degree of abrupt change in transmittance to the probability of dust accumulation; The edge visibility index is determined based on the multi-view image sequence, and the atmospheric scattering coefficient is corrected using the edge visibility index. The transmittance is then updated using the dust probability and the atmospheric scattering coefficient. The single-frame image is dehazed using the updated transmittance to obtain the restored image.
5. The bridge guard plate installation method suitable for strong wind and dusty environments according to claim 1, characterized in that, Extracting edge features of the protective plate (02) from the restored image, and optimizing the visual pose of the protective plate (02) based on the edge features and reprojection error, including: Edge detection is performed on the restored image to obtain pixel-level edge points; Multiple neighboring points are taken along the gradient direction at the pixel-level edge point, and the weighting weight of each neighboring point is determined based on the consistency between the gradient direction of each neighboring point and the gradient direction of the center point. The sub-pixel offset is calculated based on the weighted weights to obtain the sub-pixel level edge. Obtain the three-dimensional model of the protective plate (02), project the edges of the three-dimensional model onto the image plane of each camera, and construct an optimization objective function based on the reprojection error between the projected edges and the sub-pixel level edges; The visual pose of the guard plate (02) is obtained by iteratively solving the optimization objective function.
6. A bridge guard plate (02) installation device suitable for strong wind and dusty environments, characterized in that, include: The acquisition module is used to acquire hoisting environment state parameters and multi-view image sequences of the guard plate (02), and to predict the pose of the guard plate (02) by using a pendulum motion model based on the hoisting environment state parameters. The image restoration module is used to perform image restoration processing on the multi-view image sequence of the protective plate (02) to obtain the restored image; The visual pose calculation module is used to extract the edge features of the protective plate (02) on the restored image, and optimize the visual pose of the protective plate (02) based on the edge features of the protective plate (02) and the reprojection error. The pose fusion module is used to fuse the visual pose with the predicted pose and adjust the position of the guard plate (02) according to the deviation between the fused pose and the target pose. The positioning control module is used to detect the grayscale profile curvature change of the edge of the guard plate (02) by utilizing the edge features of the guard plate (02) to determine the contact state between the guard plate (02) and the mounting base (03), and adaptively adjust the control parameters according to the contact state to make the guard plate (02) smoothly positioned.
7. The bridge guard plate (02) installation device suitable for strong wind and dusty environments according to claim 6, characterized in that, The image restoration module includes: The calculation unit is used to calculate the oscillation spectrum energy concentration based on the multi-view image sequence; The dust removal unit is used to select the frequency domain comb filter dust removal mode when the energy concentration of the oscillating spectrum is greater than or equal to a first preset threshold. The defogging unit is used to select a spatial domain dust probability weighted defogging mode when the energy concentration of the oscillating spectrum is less than or equal to a second preset threshold. The fusion processing unit is used to select a fusion mode of the frequency domain comb filtering dust removal mode and the spatial domain dust probability weighted defogging mode when the energy concentration of the oscillating spectrum is less than the first preset threshold and greater than the second preset threshold.
8. The bridge guard plate (02) installation device suitable for strong wind and dusty environments according to claim 7, characterized in that, The dust removal unit includes: The transformation unit is used to perform a time-domain Fourier transform on the multi-view image sequence to obtain the frequency-domain energy distribution; A window selection unit is used to determine the spectrum signal-to-peak ratio from the frequency domain energy distribution and select the time domain window length based on the spectrum signal-to-peak ratio. A filter construction unit is used to extract the hoisting rope length from the hoisting environment state parameters to determine the swing fundamental frequency, and to construct a comb filter based on the swing fundamental frequency. The first restoration unit is used to perform frequency domain dust removal processing on the multi-view image sequence using the comb filter to obtain the restored image.
9. The bridge guard plate (02) installation device suitable for strong wind and dusty environments according to claim 7, characterized in that, The defogging unit includes: The preliminary estimation unit is used to perform a preliminary estimate of the transmittance of a single frame image in the multi-view image sequence to obtain an initial transmittance value, and to calculate the degree of transmittance change based on the initial transmittance value. The probability mapping unit is used to calculate the local gray-level variance of the single-frame image and map the local gray-level variance and the degree of abrupt change in transmittance into the probability of dust. The transmittance update unit is used to determine the edge visibility index based on the multi-view image sequence, correct the atmospheric scattering coefficient using the edge visibility index, and update the transmittance using the dust probability and the atmospheric scattering coefficient. The second restoration unit is used to perform dehazing on the single-frame image using the updated transmittance to obtain the restored image.
10. The bridge guard plate (02) installation device suitable for strong wind and dusty environments according to claim 6, characterized in that, The visual pose calculation module includes: An edge detection unit is used to perform edge detection on the restored image to obtain pixel-level edge points; The weight determination unit is used to take multiple neighborhood points along the gradient direction at the pixel-level edge point, and determine the weight of each neighborhood point based on the consistency between the gradient direction of each neighborhood point and the gradient direction of the center point. A positioning unit is used to calculate the sub-pixel offset based on the weighted weights to obtain the sub-pixel level edge. The function construction unit is used to obtain the three-dimensional model of the protective plate (02), project the edge of the three-dimensional model onto the image plane of each camera, and construct an optimization objective function based on the reprojection error between the projected edge and the sub-pixel level edge. The iterative solution unit is used to iteratively solve the optimization objective function to obtain the visual pose of the guard plate (02).
