Steel bridge u rib inner side dust recognition and cleaning control system based on neural network

Abstract

The application provides a neural network-based steel bridge U-rib inner side dust identification and cleaning control system and relates to the technical field of intelligent detection.The application associates the geometric distribution characteristics of dust with the mechanical response characteristics by visual-mechanical coupling analysis equivalent compression stiffness, can directly reflect the hardness, thickness and adhesion strength of dust, provides data support for cleaning, cooperates with the stiffness gradient weight index to carry out path planning, can dynamically adjust the cleaning step and operation mode according to the stiffness spatial distribution, reduces the invalid operation mileage under the premise of ensuring the cleaning quality, and balances the energy consumption and cleaning efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology, specifically to a neural network-based system for identifying and cleaning dust on the inner side of the U-rib of a steel bridge. Background Technology

[0002] The inspection and cleaning of the inner wall of the U-rib of steel bridges often uses crawling robots as the carrier. During the operation, the robot enters the interior through the manhole of the steel box girder, and the cleaning path is planned according to the uniformity of dust coverage.

[0003] Current cleaning operations mostly rely on visual recognition results to plan paths, which can only obtain the geometric distribution information of dust and cannot quantify its mechanical adhesion characteristics. This results in a low degree of matching between cleaning parameters and actual working conditions, and easily leads to problems such as energy waste or incomplete cleaning.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a neural network-based dust recognition and cleaning control system for the inner side of the U-rib of a steel bridge, in order to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A neural network-based dust recognition and cleaning control system for the inner side of the U-rib of a steel bridge includes:

[0008] The mapping module is used to drive the crawling robot. The crawling robot integrates a binocular vision module and a micro-force probing module. The crawling robot enters the interior of the U-rib and moves along the inner wall of the U-rib to construct a three-dimensional point cloud topology mesh of the inner wall of the U-rib and establish a mapping relationship between the robot's body coordinate system and the world coordinate system.

[0009] The segmentation module is used to acquire images of the inner wall of the U-rib through the binocular vision module, perform semantic segmentation on the images using a pre-trained lightweight neural network, identify dust areas and extract feature point sets, and map the feature point sets onto a three-dimensional point cloud topology grid to generate a spatial feature point cloud.

[0010] The probe module is used to drive the micro-force probe module to extend and contact the dust area corresponding to the spatial feature point cloud with a preset contact force, collect sequential microscopic images before and after contact, track the displacement of feature points based on digital image correlation, and obtain the equivalent compressive stiffness field of the dust layer.

[0011] The planning module is used to analyze the stiffness gradient modulus based on the equivalent compressive stiffness field, and generate a stiffness gradient weight index in combination with the reference stiffness value. Based on the stiffness gradient weight index, the cleaning area is divided into several priority levels.

[0012] The execution module is used to match the corresponding cleaning step length and path mode according to the priority level, drive the cleaning execution mechanism to perform operations on the inner wall of the U-rib, and generate data of the post-cleaning inspection area.

[0013] The correction module repeatedly executes the segmentation module and the probing module on the data of the re-inspection area, obtains the equivalent compressive stiffness of the re-inspection area, determines the cleanup qualification based on the preset threshold, and performs secondary residual verification.

[0014] Furthermore, in the mapping module, the three-dimensional point cloud topological mesh extracts image pair feature points through a scale-invariant feature transformation algorithm, restores depth information through triangulation, and establishes the mapping relationship through a hand-eye calibration matrix.

[0015] Furthermore, the input of the lightweight U-Net neural network is an RGB image, and the output is a dust region mask. The spatial feature point cloud is composed of the three-dimensional coordinates of the mask edge points and the internal centroid points in the world coordinate system.

[0016] Furthermore, the preset contact force is calibrated by a force sensor, the feature point displacement is obtained by tracking the pixel offset of dust particles in the sequence of microscopic images, and the equivalent compressive stiffness field characterizes the dust layer's resistance to compressive deformation.

[0017] Furthermore, in the probe module, the equivalent compressive stiffness is calculated by fusing the square mean of the feature points with the theoretical elastic deformation correction term, which is obtained by Hooke's law based on the elastic modulus of steel and the probe contact area.

[0018] Furthermore, in the planning module, the stiffness gradient modulus is obtained by performing a differential operation on the equivalent compressive stiffness values ​​of adjacent grid cells and dividing by the grid cell size, and the reference stiffness value is the measured value of the clean steel substrate under the same penetration conditions.

[0019] Furthermore, the priority levels include Level 1, Level 2, and Level 3, which correspond to the small step spiral cleaning mode, the medium step reciprocating cleaning mode, and the large step fast sweeping mode, respectively.

[0020] Furthermore, in the correction module, the preset threshold is set based on the reference stiffness value. If the equivalent compressive stiffness of the re-inspection area is not less than the preset threshold, the cleaning is deemed qualified.

[0021] Compared with the prior art, the beneficial effects of the present invention are:

[0022] 1. This invention analyzes the equivalent compressive stiffness through visual-mechanical coupling, and associates the geometric distribution characteristics of dust with mechanical response characteristics. It can directly reflect the softness, thickness and adhesion strength of dust, providing data support for cleaning. Combined with the stiffness gradient weight index for path planning, it can dynamically adjust the cleaning step length and operation mode according to the stiffness spatial distribution, reduce the ineffective operation mileage while ensuring cleaning quality, and achieve a balance between energy consumption and cleaning efficiency.

[0023] 2. A secondary residue verification is added, which verifies the trace residue risk based on hysteresis energy dissipation. It detects qualified areas that meet the stiffness standard but have trace amounts of highly hazardous residues, thereby improving the reliability of cleaning quality. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the system module framework structure of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0026] Example:

[0027] Please see Figure 1 The present invention provides a technical solution:

[0028] The neural network-based dust recognition and cleaning control system for the inner side of the U-rib of a steel bridge specifically includes:

[0029] The mapping module integrates a binocular vision module and a micro-force probing module on the crawling robot body. The crawling robot enters the interior of the U-rib and moves along the inner wall of the U-rib to construct a three-dimensional point cloud topology mesh of the inner wall of the U-rib, and establishes the mapping relationship between the robot body coordinate system and the world coordinate system.

[0030] The interior of the U-rib is a long, narrow, enclosed cavity with a clearance of only about 26 centimeters, making it difficult for humans to enter. Conventional inspection uses a crawling robot as a carrier, which crawls and performs the work simultaneously. The process is as follows: the crawling robot is placed inside the steel box girder through a manhole. The robot uses magnetic attraction or its tracks to adhere to the wall and enters the U-rib cavity, moving longitudinally along the U-rib. The robot's front end integrates a binocular vision module, including two industrial cameras. The baseline distance is adapted to the width of the U-rib's interior. The end effector is equipped with a retractable micro-force probing module, employing visual-tactile sensors to measure surface roughness and contamination levels. The binocular vision module is activated to acquire stereoscopic images of the inner wall. A scale-invariant feature transform algorithm is used to extract feature points from the left and right images. The depth information of the feature points is calculated using triangulation principles, and the resulting images are stitched together to generate an initial 3D point cloud topology mesh of the U-rib's inner wall. Meanwhile, the camera intrinsic parameters and distortion coefficients are obtained through Zhang Zhengyou's calibration method, and the transformation matrix from the robot end flange coordinate system to the camera coordinate system is obtained through hand-eye calibration, thereby establishing the mapping relationship between the robot body coordinate system and the world coordinate system.

[0031] The segmentation module acquires images of the inner wall of the U-rib through a binocular vision module, performs semantic segmentation on the images using a pre-trained lightweight U-Net neural network, identifies dust areas and extracts feature point sets, and maps the feature point sets onto a 3D point cloud topology grid to generate a spatial feature point cloud.

[0032] The lightweight U-Net neural network is an existing technology. The encoder uses the MobileNetV2 backbone network, and the decoder uses skip connections. It is trained on a dataset containing dust samples from the U-ribs of a steel bridge, achieving high-precision semantic segmentation with low computational overhead. The specific process is as follows: a crawling robot moves along the inner wall of the U-rib, a binocular vision module acquires RGB images and inputs them into the neural network, the network outputs a binary mask of the dust area, and the edge points and internal centroids of the mask are extracted as the image feature point set. Using the established mapping relationship and camera projection model, the pixel coordinates of the image feature point set are back-projected onto a 3D point cloud mesh to obtain a spatial feature point cloud with physical scale.

[0033] The probe module drives the micro-force probe module to extend and contact the dust area corresponding to the spatial feature point cloud with a preset contact force. It collects sequential microscopic images before and after contact, tracks the displacement of feature points based on digital image correlation, and calculates and generates the equivalent compressive stiffness field of the dust layer.

[0034] Micro-force probing for dust quantification: Micro-force measurement can measure forces at the micro-Newton level and is used for detecting the cleanliness of metal surfaces. Visual-tactile sensors can transmit the magnitude and distribution of contact force through the deformation of an elastic body, thereby measuring the degree of contamination. The robot's end effector is controlled to extend the micro-force probing module, which integrates a microscope lens, using a preset contact force calibrated by a precision force sensor. For the surface corresponding to the feature point cloud in vertical contact space (e.g., 0.5N), a sequence of microscopic images were acquired before contact, after contact stabilization, and after unloading. Based on digital image correlation, dust particles were used as natural speckle patterns to track the displacement of feature points on the image plane. Based on mechanical equilibrium and geometric relationships, the equivalent compressive stiffness of the dust layer was constructed. The formula used is:

[0035]

[0036] in, The preset contact force is calibrated by a precision force sensor, reflecting the externally applied detection load. Characterizing the physical size of a single pixel, obtained through camera macro calibration, it reflects the conversion relationship between image pixels and actual physical size. The area of ​​the dust region segmented by the neural network is determined by the total number of mask pixels and The product of squares reflects the extent of dust coverage. Let be the pixel deformation of the i-th feature point, obtained by subpixel matching using digital image correlation, reflecting the local compressive deformation of the dust layer under the action of probing force, and n be the total number of feature points involved in the calculation. The effective contact area of ​​the probe reflects the area of ​​the probing action. For steel in force The theoretical elastic deformation under action, according to Hooke's Law Calculated is the elastic modulus constant of steel, reflecting the baseline deformation of clean steel substrate under the same penetration test conditions.

[0037] The denominator is empirically coupled with the two displacements, which effectively characterizes the relationship between the total compressive displacement and the stiffness. The average square deformation of the dust surface reflects the softness of the dust; the greater the deformation, the higher the value. This is a theoretical elastic deformation correction term. `max` is used to take the maximum value. When the dust coverage area is less than or equal to the probe contact area, the effective bearing area is limited by the dust area, and the steel base deformation increases as the dust area decreases. When the dust coverage area is greater than the probe contact area, the effective bearing area is limited by the probe area, and the steel base deformation remains unchanged at the baseline theoretical value. This establishes the measurement benchmark and the equivalent compressive stiffness of the dust layer. This reflects the dust layer's resistance to compression deformation. A higher value indicates a denser, thinner, and harder dust layer. When the thickness approaches the theoretical stiffness of the steel plate, it indicates that the dust is extremely thin or has hardened. A smaller value indicates that the dust is a soft, thick layer or floating dust. The softer the dust, the larger the pixel deformation of the feature points. The smaller the value, the faster and more accurate the quantification of dust status can be achieved by constructing an equivalent compressive stiffness, compared to existing technologies that rely solely on visual grayscale to determine thickness.

[0038] The planning module calculates the stiffness gradient modulus based on the equivalent compressive stiffness field and generates a stiffness gradient weight index in combination with the reference stiffness value. Based on the stiffness gradient weight index, the cleaning area is divided into several priority levels.

[0039] Existing technologies often employ strategies based on uniform coverage cleaning, which can easily lead to energy waste or incomplete cleaning. Therefore, the inner wall of the U-rib is divided into a regular grid. The side length h of the grid cell is determined by the U-rib size and the required detection accuracy, with a typical value of 10 mm. The equivalent compressive stiffness of each grid node is calculated using bilinear interpolation, and the difference in stiffness values ​​between adjacent nodes is calculated. Dividing this difference by the grid cell size h yields the stiffness gradient vector. In terms of modulus As a measure of the rate of change of stiffness, a stiffness gradient weight exponent is introduced. The formula used as an indicator of cleaning is:

[0040]

[0041] in, The gradient modulus of the equivalent compressive stiffness is obtained by performing a difference operation on the equivalent compressive stiffness values ​​of adjacent mesh elements and dividing by the mesh element size. It reflects the degree of spatial variation of the stiffness field. For reference stiffness values, the measured equivalent compressive stiffness of the clean steel substrate under the same penetration test conditions is taken. The equivalent compressive stiffness of the current mesh element is given by β, which is the attenuation coefficient. The rate at which the control exponential gating term decays with increasing relative stiffness is determined by: preparing standard dust samples with different adhesion strengths in the laboratory and measuring the cleaning energy consumption. The relationship is determined by taking the value corresponding to the inflection point of unit energy consumption, with a typical value of 1.0.

[0042] Areas with significant changes in stiffness often correspond to complex working conditions where hard grime encases soft dust or rust pits trap dirt, making cleaning difficult. As an exponential gating term, when the dust is generally very hard, When the value is large, the exponential term decays, reducing the priority and preventing the robot from ineffectively over-cleaning hardened areas, thus saving energy. When the dust is soft but unevenly distributed, Larger When the value is moderate, the exponent term is close to 1, the exponent value is the largest, and the priority is the highest. This is the stiffness gradient weight exponent. Gradient modulus of equivalent compressive stiffness reflects the overall difficulty and priority of cleaning. The more drastic the change, the greater the stiffness gradient weight exponent. The larger the layer, the more attention is needed to the equivalent compressive stiffness of the dust layer. The larger the value, the stiffer the overall structure; therefore, the lower the priority, based on the stiffness gradient weighting index. The values ​​divide the region into three levels: Level 1 priority ( ), corresponding to small step size (5mm) spiral cleaning; secondary priority ( ), corresponding to medium step length (20mm) reciprocating cleaning; three priority levels ( ), corresponding to a large stride (50mm) for rapid sweep.

[0043] The execution module matches the corresponding cleaning step length and path mode according to the priority level, drives the cleaning execution mechanism to perform operations on the inner wall of the U-rib, and generates data of the area to be inspected after cleaning.

[0044] The robot controller reads the priority map. For first-priority areas, it controls the cleaning brush to use a spiral converging path with a step size of 5mm to increase the dwell time per unit area. For second-priority areas, it uses a reciprocating straight path with a step size of 20mm. For third-priority areas, it uses a unidirectional fast-passing path with a step size of 50mm. During cleaning, force sensors monitor the contact force in real time to ensure smooth operation. After cleaning is completed, the robot moves to the next station and records the position coordinates of the currently cleaned area, marking it as a re-inspection area.

[0045] The correction module repeatedly segments and probes the data in the re-inspection area, calculates the equivalent compressive stiffness of the re-inspection area, and determines the cleanup qualification based on the preset threshold.

[0046] The robot travels to the re-inspection area and repeatedly executes the segmentation and probing modules to collect images and microscopic images of the re-inspection area. It calculates the equivalent compressive stiffness of the re-inspection area and sets a judgment threshold of 0.85 times the reference stiffness value. If the equivalent compressive stiffness of the re-inspection area is not less than the judgment threshold, the area is judged to be cleaned successfully and the robot continues to move forward. If it is less than the judgment threshold, the cleaning is judged to be incomplete, triggering a secondary cleaning subroutine and storing the coordinates of the area and the equivalent compressive stiffness value in the anomaly database.

[0047] After the initial qualification judgment, a second residual verification is performed on the re-inspection area that was initially qualified. The specific process is as follows: retrieve the collected full-cycle data of micro-force penetration force-displacement, calculate the hysteresis area enclosed by the loading and unloading sections to obtain the penetration dissipation energy. Calculate the risk value It is constructed based on the hysteresis dissipation theory and the interface adhesion energy theory in contact mechanics.

[0048] The formula is as follows:

[0049]

[0050] The penetration test response of the clean steel substrate is purely elastic. When the micro-force penetration module applies a force F0 to the clean steel substrate, the system behaves as a purely elastic body. The force-displacement curve of the loading section is a straight line, and its slope is determined by the stiffness of the steel substrate. The force-displacement curve of the unloading section completely coincides with that of the loading section, and the hysteresis area is 0, meaning that the penetration energy dissipation is zero. The response of the probe with trace residues is hysteretic dissipation. When there are trace residues (oil, micro-rust, dust adhesion layer) of 1~5μm on the surface, the probe process exhibits nonlinear hysteresis. The loading stage is when the probe compresses the residue layer, and the residue layer undergoes viscoelastic deformation / interface slip / microcrack initiation, consuming energy. The unloading stage is when the probe recovers, but the residue layer cannot completely return to its original state (plastic deformation, irreversible interface slip). The loading and unloading stages do not coincide, forming a hysteresis loop, and the area of ​​the loop is the dissipated energy. The value is greater than 0, therefore a normalized stiffness term is constructed. Normalized dissipative energy term It reflects the ratio of abnormal dissipated energy to the baseline elastic potential energy, and the residual risk index is obtained by coupling the stiffness term and the dissipation term.

[0051] in, This is the residual adhesion risk index; a higher value indicates a higher risk of trace amounts of harmful deposits in the re-inspected area. To re-examine the equivalent compressive stiffness of the region, κ is the dissipation sensitivity coefficient. The calibration method is as follows: 20 standard samples with stiffness meeting the 0.85 times threshold condition but containing 1~5μm trace harmful residues (including oil stains and rust, two typical working conditions) are prepared in the laboratory. The hysteresis dissipation energy of each sample is measured. Take the κ value corresponding to the 90th percentile sample. To measure the hysteretic dissipation energy of the micro-force probe loading-unloading cycle, the force-displacement curves were integrated to calculate the hysteretic energy (force-displacement integral of the loading segment minus the force-displacement integral of the unloading segment). The threshold C was taken from a clean steel sample. The 95th percentile value ensures a false positive rate of less than 5%. The final determination was that the cleaning was qualified. Even if it is judged to be qualified, it is still classified as incomplete cleaning, triggering a second cleaning, and the area is marked as a trace attachment risk area and stored in the anomaly database.

[0052] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0053] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0054] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0055] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A neural network-based dust recognition and cleaning control system for the inner side of the U-rib of a steel bridge, characterized in that, include: The mapping module is used to drive the crawling robot. The crawling robot integrates a binocular vision module and a micro-force probing module. The crawling robot enters the interior of the U-rib and moves along the inner wall of the U-rib to construct a three-dimensional point cloud topology mesh of the inner wall of the U-rib and establish a mapping relationship between the robot's body coordinate system and the world coordinate system. The segmentation module is used to acquire images of the inner wall of the U-rib through the binocular vision module, perform semantic segmentation on the images using a pre-trained lightweight neural network, identify dust areas and extract feature point sets, and map the feature point sets onto a three-dimensional point cloud topology grid to generate a spatial feature point cloud. The probe module is used to drive the micro-force probe module to extend and contact the dust area corresponding to the spatial feature point cloud with a preset contact force, collect sequential microscopic images before and after contact, track the displacement of feature points based on digital image correlation, and obtain the equivalent compressive stiffness field of the dust layer. The planning module is used to analyze the stiffness gradient modulus based on the equivalent compressive stiffness field, and generate a stiffness gradient weight index in combination with the reference stiffness value. Based on the stiffness gradient weight index, the cleaning area is divided into several priority levels. The execution module is used to match the corresponding cleaning step length and path mode according to the priority level, drive the cleaning execution mechanism to perform operations on the inner wall of the U-rib, and generate data of the post-cleaning inspection area. The correction module repeatedly executes the segmentation module and the probing module on the data of the re-inspection area, obtains the equivalent compressive stiffness of the re-inspection area, determines the cleanup qualification based on the preset threshold, and performs secondary residual verification.

2. The neural network-based dust identification and cleaning control system for the inner side of the U-rib of a steel bridge according to claim 1, characterized in that: In the mapping module, the three-dimensional point cloud topological mesh extracts image pair feature points through the scale-invariant feature transformation algorithm, restores depth information through the triangulation principle, and establishes the mapping relationship through the hand-eye calibration matrix.

3. The neural network-based dust identification and cleaning control system for the inner side of the U-rib of a steel bridge according to claim 1, characterized in that: The lightweight U-Net neural network takes an RGB image as input and outputs a dust region mask. The spatial feature point cloud is composed of the three-dimensional coordinates of the mask edge points and the internal centroid points in the world coordinate system.

4. The neural network-based dust identification and cleaning control system for the inner side of the U-rib of a steel bridge according to claim 1, characterized in that: The preset contact force is calibrated by a force sensor, the feature point displacement is obtained by tracking the pixel offset of dust particles in the sequence of microscopic images, and the equivalent compressive stiffness field characterizes the dust layer's resistance to compressive deformation.

5. The neural network-based dust identification and cleaning control system for the inner side of the U-rib of a steel bridge according to claim 1, characterized in that: In the probe module, the equivalent compressive stiffness is calculated by fusing the square mean of the feature points with the theoretical elastic deformation correction term. The theoretical elastic deformation correction term is obtained by Hooke's law based on the elastic modulus of steel and the probe contact area.

6. The neural network-based dust identification and cleaning control system for the inner side of the U-rib of a steel bridge according to claim 1, characterized in that: In the planning module, the stiffness gradient modulus is obtained by performing a differential operation on the equivalent compressive stiffness values ​​of adjacent grid cells and dividing by the grid cell size. The reference stiffness value is the measured value of a clean steel substrate under the same penetration conditions.

7. The neural network-based dust identification and cleaning control system for the inner side of the U-rib of a steel bridge according to claim 1, characterized in that: Priority levels include Level 1, Level 2, and Level 3, which correspond to the small step spiral cleaning mode, the medium step reciprocating cleaning mode, and the large step fast sweeping mode, respectively.

8. The neural network-based dust identification and cleaning control system for the inner side of the U-rib of a steel bridge according to claim 1, characterized in that: In the correction module, the preset threshold is set based on the reference stiffness value. If the equivalent compressive stiffness of the re-inspection area is not less than the preset threshold, the cleaning is deemed qualified.

Images