Bridge support disease detection system and method based on rail type inspection robot

Abstract

The invention discloses a bridge support disease detection system and method based on a rail-mounted inspection robot, and belongs to the technical field of bridge detection. The system comprises a robot walking guide mechanism, the rail-mounted inspection robot and a self-adaptive following system; the rail type inspection robot comprises a chassis, and a mechanical arm is arranged on the chassis; the self-adaptive following system comprises a robot control system, an image processing system and a network communication system; the robot control system comprises a main control unit which is mainly used for controlling movement of a chassis and self-adaptive adjustment of a mechanical arm; the image processing system comprises an image acquisition device, an image processing and storage device and an edge computing device; and the image acquisition equipment is arranged on the mechanical arm and is mainly used for acquiring image data. According to the invention, intelligent identification and adaptive following can be carried out on the bridge support disease image, so that the image acquisition equipment can be ensured to lock and track the disease area from the front side, and high-quality and omission-free photographing detection can be realized.

Description

Technical Field

[0001] The present invention relates to bridge detection technology, and in particular to a bridge bearing disease detection system and method based on an orbital inspection robot. Background Art

[0002] At present, during the construction process of bridge engineering, bearings are an important part of the lower structure of a bridge, and the safety condition of the bearings directly affects the structural safety and stability of the bridge. The damage and failure of bridge bearings have become one of the main diseases of existing bridges in China. Its typical diseases are manifested as local voids, excessive shear deformation, rubber surface cracking, and uneven bulging deformation. All kinds of diseases have a great impact on the stability and safety of the bearings and even the health of the entire bridge, and must be prevented and treated. However, since the bearings account for a relatively small proportion in the bridge project cost, they are often not taken seriously by engineering technicians and managers, and are extremely likely to become the weak links of the bridge structure during use.

[0003] Regular manual inspection is a common means to obtain the health condition of bridge bearings. However, manual inspection is not only time-consuming and laborious, but also affects traffic. Especially for bridges in remote or special sections, the difficulty and danger of manual inspection are greatly increased. Moreover, due to the concealment of the installation position of bridge bearings, the reserved position of the bearings is relatively small and the light is dim, making it very difficult for workers to accurately observe the bearings, and there is a large error compared with the actual results. For some bridges across rivers and with high piers, it is even necessary to use equipment such as bridge inspection vehicles, which is expensive. Therefore, the manual inspection method has the disadvantages of difficult detection, high cost, subjective determination of damage degree, and inability to give early warnings. Therefore, it is imperative to develop an automated and efficient bridge bearing disease detection method.

[0004] At present, the intelligent bearings with real-time monitoring mainly monitor the stress conditions of the bearings by arranging various sensors and through data transmission and processing, but they cannot effectively monitor the apparent diseases of the bearings and the surrounding environment of the bearings, such as the surface crack conditions of the bearings, whether the bearings are skewed, whether the bearings are rusted, whether there is water accumulation in the bearing pads, and whether there is accumulation of construction waste. CN202310364013.6 discloses a real-time monitoring system and method for bridge bearings that combines inspection and routine inspection. Although this system and method combine drones or bridge inspection crawling robots for apparent disease detection and develop a bridge bearing monitoring system with remote real-time monitoring and warning functions to achieve real-time monitoring of the internal stress of the bearings and effective detection of external diseases, and timely grasp the health status and potential risks of the in-service bearings and bridges, which has obvious economic value for ensuring the health of bridges and extending the service life of bridges, this system and method must use drones to send the bridge inspection crawling robots to the piers around the bridge bearings. Due to the different shapes of the piers of different bridges, the reserved platform space is narrow, and the reserved space around the bridge bearings is also relatively small, which poses high requirements for the load operation of drones and also poses extremely high requirements for the chassis structure and motion planning of the bridge inspection crawling robots, making it difficult to implement on the engineering site. In addition, it is impossible to intelligently identify and adaptively follow the bridge bearing disease images to ensure that the depth camera can positively lock and track the disease area to achieve high-quality and non-missing photo detection.

[0005] In summary, there is an urgent need to find a new bridge bearing disease detection system and method to solve the above problems. Summary of the Invention

[0006] The technical problem to be solved by the present invention is, aiming at the deficiencies of the above-mentioned prior art, to provide a bridge bearing disease detection system and method based on an orbital inspection robot that is easy to implement on the engineering site, can intelligently identify and adaptively follow the bridge bearing disease images to ensure that the image acquisition device can positively lock and track the disease area, and achieve high-quality and non-missing photo detection.

[0007] The present invention accurately arranges and installs specific orbits on the bridge deck and the piers around the bridge bearings. The inspection personnel only need to place the robot on the orbit located on the bridge deck, and the robot can automatically and efficiently run along the orbit to the bridge bearing detection area for high-precision image acquisition and disease identification. After the detection is completed, the system will automatically generate a detailed detection report and push it to the mobile phone of the inspection personnel in real time, so as to be able to grasp the health status of the bridge bearings at any time and provide strong support for the subsequent maintenance work.

[0008] The technical solution adopted by the present invention to solve its technical problems is: a bridge bearing disease detection system based on an orbital inspection robot, including a robot walking guiding mechanism, an orbital inspection robot, and an adaptive following system; The orbital inspection robot includes a chassis, and a robotic arm is provided on the chassis; The adaptive following system includes a robot control system, an image processing system, and a network communication system; The robot control system includes a main control unit, which is mainly used to control the movement of the chassis and the adaptive adjustment of the robotic arm. When inspecting the bridge bearing for problems, the robot internally controls the movement of the chassis for positioning by recording the corresponding coordinates of the bearing during the inspection, and controls the movement of each joint of the robotic arm for adaptive adjustment at the observation position, and performs multi-scheme configuration according to factors such as distance, light, and imaging angle to obtain detailed data of the warning part of the bearing; The image processing system includes an image acquisition device, an image processing and storage device, and an edge computing device; the image acquisition device is arranged on the robotic arm and is mainly used for the acquisition of image data; the image processing and storage device uses a cloud server and is mainly used for the processing, analysis, and application of image data; the edge computing device refers to the main control unit, which is mainly used for the processing, analysis, and application of image data, and transmits the image data to the cloud server through the network communication system.

[0009] Further, the chassis includes a chassis body, side baffles and drive wheels are installed on both the left and right sides of the chassis body, guide wheel mounting brackets are installed at both the front and rear ends of the chassis body, guide wheels are installed on both sides of the guide wheel mounting brackets, and a top plate is installed on the top of the chassis body; a motor, a reducer, a steering mechanism, a chassis control unit, etc. are also provided on the chassis body. The motor is connected to the drive wheel through the reducer to form a chassis drive system, and the forward, backward, turning and other actions of the robot can be accurately controlled through the chassis drive system.

[0010] Further, the robotic arm includes a robotic arm base, a first joint component, a second joint servo, a second joint servo mount, a first extension arm, a third joint servo, a third joint servo mount, a second extension arm, a fourth joint servo, a fourth joint servo mount, an image acquisition device connection bracket, an image acquisition device, and a terminal spare mount base. The first joint component is mounted on the robotic arm base and connected to a drive motor. Under the action of the drive motor, the first joint component can rotate along the robotic arm base on a horizontal plane. The second joint servo is mounted on the first joint component, and the second joint servo mount is mounted on the second joint servo. Under the action of the second joint servo, the second joint servo mount can rotate along the second joint servo. The first extension arm is mounted on the second joint servo mount. The third joint servo is mounted on the top of the first extension arm, and the third joint servo mount is mounted on the third joint servo. Under the action of the third joint servo, the third joint servo mount can rotate along the third joint servo. The second extension arm is mounted on the third joint servo mount. The fourth joint servo is mounted on the top of the second extension arm, and the fourth joint servo mount is mounted on the fourth joint servo. Under the action of the fourth joint servo, the fourth joint servo mount can rotate along the fourth joint servo. The image acquisition device is mounted on the fourth joint servo mount through the image acquisition device connection bracket, and the terminal spare mount base is mounted on the fourth joint servo mount. A robotic arm control unit is provided inside the robotic arm base. The robotic arm control unit is respectively connected to the drive motor of the first joint component, the second joint servo, the third joint servo, and the fourth joint servo through lines, and controls the drive motor of the first joint component, the second joint servo, the third joint servo, and the fourth joint servo to work, constituting a robotic arm control system.

[0011] Further, the image acquisition device uses a depth camera, namely the Gemini Plus 3D depth camera. Based on the binocular stereo vision principle, this camera takes pictures of the same scene from different angles through two infrared cameras, and then uses an algorithm to calculate the position deviation (i.e., parallax) between corresponding points of the images, thereby obtaining the three-dimensional geometric information of the object. At the same time, to solve the problems of insufficient light, uneven light intensity, and alternating light and dark in the bridge bearing, supplementary light source LED lights are configured around the depth camera, which can effectively supplement the light in the dark area and balance the overall lighting conditions, ensuring the accuracy and efficiency of the detection work.

[0012] Further, the image processing and storage device is used for regular inspection and problem checking; the robot will capture a large amount of picture data on-site and upload the collected images to the cloud server in real time. The cloud server is equipped with a GPU (Graphics Processing Unit) acceleration card, which can accelerate the execution of image processing algorithms. At the same time, the cloud server also includes a cloud storage service for storing the original image data, processed images, and analysis results, supporting long-term data preservation and fast access.

[0013] Further, the edge computing device refers to the main control unit, which is mainly used for regular inspection; in order to reduce the data transmission volume and latency, a lightweight bridge bearing disease feature recognition model is deployed on the main control unit, and the GPU in the edge processor is used to analyze and judge the images captured by the robot in real time.

[0014] Further, the architecture core of the network communication system consists of components such as routers, 4G IoT cards, and wireless network communication modules. They work together to establish a stable and efficient network connection; the router, as the center, is responsible for the routing selection and forwarding of data packets to ensure the smooth flow of information between different devices; the 4G IoT card provides wireless wide area network access capabilities, enabling the system to be flexibly deployed in scenarios without a fixed broadband network to achieve remote data transmission and monitoring; while the wireless network communication module focuses on building a wireless local area network within a short distance, facilitating instant communication and data sharing between nearby devices; these components work together to build a wide-coverage and flexible network communication environment, ensuring the instant and accurate transmission of information between the image processing system and its various components, laying a solid foundation for the stable operation and efficient cooperation of the system.

[0015] The bridge bearing disease detection method implemented using the system includes the following processes: The regular task of the robot is regular inspection; at the beginning, the robot moves from the bridge deck to a certain bridge bearing through the walking track by itself. The image acquisition device on the robotic arm obtains video stream information in real time and extracts picture frames for image recognition in the main control unit of the robot to ensure that any potential disease parts of the bridge bearing can be detected comprehensively and accurately; at the same time, the pictures obtained by the image acquisition device will be uploaded to the cloud server through the network communication system, and secondary analysis will be carried out in the cloud server to ensure that no disease parts of the bridge bearing are missed; if no warning parts of the bridge bearing are found during the entire inspection process, the robot will send a message and generate a detection report to be sent to the mobile APP, and the inspection of this bearing is completed, and the inspection of the next bearing will continue; If a warning area of the bridge bearing is found during the inspection process, or if the cloud server analyzes and finds a warning part in a certain bearing, then problem inspection will be carried out for that bearing. The problem inspection mainly involves the adaptive following system related to the robotic arm. Inside the robot, the corresponding coordinates of the bearing are controlled by the inspection record to control the movement of the chassis for positioning. At the observation position, the joints of the robotic arm are controlled to move for adaptive adjustment. Multiple schemes are configured according to factors such as distance, light, and imaging angle to obtain detailed data of the warning area of the bearing. During this process, manual intervention can be carried out to optimize the imaging quality, and combined judgment is made through re-inspection and manual expert inspection. Finally, a test report is generated, and subsequent work is carried out manually.

[0016] Furthermore, the mobile phone APP has the functions of manually controlling the movement of the robot, starting the automatic detection of the robot, displaying the real-time detection images and detection results of the main control unit, and displaying the complete detection report after the detection is completed.

[0017] Furthermore, when carrying out problem inspection, the precise deviation amount between the current viewing angle of the image acquisition device and the center position of the disease image is calculated through an intelligent algorithm. Based on this deviation data, the system immediately applies the inverse kinematics principle of the robotic arm of the robot to precisely calculate the optimal movement angles required for each joint of the robotic arm, so as to precisely control the robotic arm to drive the image acquisition device for adaptive adjustment, ensuring that the image acquisition device always positively locks and tracks the warning area. If there are obstacles or the warning area is incomplete after the initial image adjustment, the system will try to move the robot and keep the robotic arm and the image acquisition device positively lock the warning area from different positions. The entire process will continuously upload the image set to achieve high-quality and non-missing photo detection.

[0018] Compared with the prior art, the present invention has the following advantages: The present invention integrates image processing technology and precise robot control technology to achieve intelligent linkage operation. First, it is necessary to use image processing technology to efficiently identify and analyze the disease images of bridge bearings, and accurately extract information such as the center coordinates and key feature points of the images. Subsequently, the precise deviation amount between the current viewing angle of the image acquisition device and the center position of the disease image is calculated through an intelligent algorithm. Based on this deviation data, the system immediately applies the inverse kinematics principle of the robotic arm of the robot to precisely calculate the optimal movement angles required for each joint of the robotic arm, so as to precisely control the robotic arm to drive the image acquisition device for adaptive adjustment, ensuring that the image acquisition device always positively locks and tracks the disease area and achieving high-quality and non-missing photo detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is the block diagram of the composition structure of the bridge bearing disease detection system based on the rail-type inspection robot of the present invention; Figure 2 is Figure 1Schematic diagram of the structure of the robot of the system shown Figure 3 is Figure 2 Schematic diagram of the robotic arm structure of the robot shown Figure 4 Flowchart of the method for detecting diseases of bridge bearings based on an orbital inspection robot of the present invention Figure 5 Flowchart of adaptive adjustment during problem inspection Explanation of reference numerals in the figure 1. Side baffle, 2. Driving wheel, 3. Guide wheel mounting bracket, 4. Guide wheel, 5. Top plate, 6. Main control unit, 7. Robotic arm control unit, 8. Robotic arm 9. Robotic arm base, 10. First joint component, 11. Second joint servo, 12. Second joint servo mounting part, 13. First extension arm, 14. Third joint servo, 15. Third joint servo mounting part, 16. Second extension arm, 17. Fourth joint servo, 18. Fourth joint servo mounting part, 19. Image acquisition device connecting frame, 20. Image acquisition device, 21. End spare mounting base Specific implementation manner

[0020] The technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments, rather than all of the embodiments. Based on these embodiments, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention

[0021] Refer to Figures 1 to 3 , a bridge bearing disease detection system based on an orbital inspection robot, including a robot walking guiding mechanism, an orbital inspection robot, and an adaptive following system The orbital inspection robot includes a chassis, and a robotic arm is provided on the chassis The adaptive following system includes a robot control system, an image processing system, and a network communication system The robot control system includes a main control unit, which is mainly used to control the movement of the chassis and the adaptive adjustment of the robotic arm. When conducting problem inspection on the bridge bearing, the robot controls the movement of the chassis for positioning through the inspection record of the corresponding coordinates of the bearing inside, and controls the movement of each joint of the robotic arm for adaptive adjustment at the observation position, and performs multi-scheme configuration according to factors such as distance, light, and imaging angle to obtain detailed data of the warning part of the bearing The described image processing system includes an image acquisition device, an image processing and storage device, and an edge computing device; the image acquisition device is installed on the robotic arm and is mainly used for collecting image data; the image processing and storage device uses a cloud server and is mainly used for processing, analyzing, and applying image data; the edge computing device refers to the main control unit and is mainly used for processing, analyzing, and applying image data, and transmits the image data to the cloud server through a network communication system.

[0022] In this embodiment, the robot walking guiding mechanism is composed of a walking track, a mounting bracket, a track fixing block, etc. (specifically, refer to the patent with the patent number "ZL202420625427.X" and the patent name "A Bridge Bearing Inspection Track Structure"), which is installed on the surface of the bridge pier bearing and provides a guiding function for the movement of the robot. One end of the track extends to the bridge deck, facilitating the installation and retrieval of the robot by inspection personnel on the bridge deck. The design and manufacture of the walking track should follow the dimensions of the inspection robot and the requirements of the actual inspection working environment. The track is made of aluminum alloy as a whole, with an internal width of not less than 28 cm, a minimum turning radius of not less than 40 cm, and a maximum slope of not exceeding 35°. The specific shape will be determined according to the actual task scenario. Ultrasonic ranging sensors MFR70753 are installed on both the left and right sides of the track-type inspection robot. This type of sensor has the characteristics of stable, accurate, and high-precision distance measurement. Configuring ultrasonic ranging sensors on both the left and right sides will provide the position of the robot in the walking track in real time to ensure that the robot does not deviate from the track.

[0023] In this embodiment, the chassis includes a chassis body. On both the left and right sides of the chassis body, side baffles 1 and drive wheels 2 are installed. On the front and rear ends of the chassis body, guide wheel mounting brackets 3 are installed. On both sides of the guide wheel mounting bracket 3, guide wheels 4 are installed. On the top of the chassis body, a top plate 5 is installed, as Figure 2 shown. The chassis body is mechanically connected to components such as the side baffles 1, drive wheels 2, guide wheel mounting brackets 3, guide wheels 4, and top plate 5 to jointly form the execution body for the robot to walk. Different selection and matching schemes are available for both the drive wheels 2 and the guide wheels 4 to adapt to the complex and changeable inspection environment; the side baffles 1 and the top plate 5 can effectively prevent sand, dust, and local water accumulation from damaging the motion motor and improve the reliability of the robot's field work. The chassis body is also provided with a motor, a reducer, a steering mechanism, a chassis control unit, etc. The motor is connected to the drive wheel through the reducer to form a chassis drive system. Through the chassis drive system, the forward, backward, and turning actions of the robot can be precisely controlled, which is prior art. At the same time, the chassis drive system also has the functions of autonomous navigation and obstacle avoidance to ensure the safe and stable driving of the robot on the bridge.

[0024] In this embodiment, the robotic arm 8 includes a robotic arm base 9, a first joint component 10, a second joint servo 11, a second joint servo mounting member 12, a first extension arm 13, a third joint servo 14, a third joint servo mounting member 15, a second extension arm 16, a fourth joint servo 17, a fourth joint servo mounting member 18, an image acquisition device connecting bracket 19, an image acquisition device 20, and a terminal spare mounting base 21, as Figure 3 shown, the first joint component 10 is mounted on the robotic arm base 9 and is connected to a driving motor. Under the action of the driving motor, the first joint component 10 can rotate along the robotic arm base 9 on a horizontal plane. The second joint servo 11 is mounted on the first joint component 10, and the second joint servo mounting member 12 is mounted on the second joint servo 11. Under the action of the second joint servo 11, the second joint servo mounting member 12 can rotate along the second joint servo 11. The first extension arm 13 is mounted on the second joint servo mounting member 12. The third joint servo 14 is mounted on the top of the first extension arm 13, and the third joint servo mounting member 15 is mounted on the third joint servo 14. Under the action of the third joint servo 14, the third joint servo mounting member 15 can rotate along the third joint servo 14. The second extension arm 16 is mounted on the third joint servo mounting member 15. The fourth joint servo 17 is mounted on the top of the second extension arm 16, and the fourth joint servo mounting member 18 is mounted on the fourth joint servo 17. Under the action of the fourth joint servo 17, the fourth joint servo mounting member 18 can rotate along the fourth joint servo 17. The image acquisition device 20 is mounted on the fourth joint servo mounting member 18 through the image acquisition device connecting bracket 19, and the terminal spare mounting base 21 is mounted on the fourth joint servo mounting member 18.

[0025] A robotic arm control unit 7 is provided inside the robotic arm base 9. The robotic arm control unit 7 is respectively connected to the driving motor of the first joint component 10, the second joint servo 11, the third joint servo 14, and the fourth joint servo 17 through circuits, and controls the driving motor of the first joint component 10, the second joint servo 11, the third joint servo 14, and the fourth joint servo 17 to work, constituting a robotic arm control system.

[0026] The robotic arm control system is responsible for controlling the precise movement of the robotic arm and the shooting of the image acquisition device. When the chassis drive system stops running, the robotic arm control system will drive the robotic arm for precise positioning and adjustment according to the disease location information provided by the image processing system, so that the camera can accurately capture the image of the disease center. The robotic arm control system has the control ability of multiple degrees of freedom (such as pitching, rotating, telescoping, etc.) to adapt to the disease detection requirements of different shapes and positions. In addition, the robotic arm control system is also equipped with high-precision sensors and feedback mechanisms to ensure the stable operation and precise control of the robotic arm in complex environments.

[0027] The control algorithm of the robotic arm is integrated in the main control unit 6. The main control unit 6 is connected to the robotic arm control unit 7 through a line and makes adaptive adjustments to the robotic arm, forming an adaptive system. The main application scenario of this system is in the problem inspection stage. After the rail-type inspection robot detects a warning area during inspection, it will play its function. The specific form is to optimize and allocate according to the existing robot posture and imaging conditions, and control the changes in the robot's walking, the joint postures of the robotic arm, and the parameters of the image acquisition device, so as to ensure that when there are cables, bridge structures or sundries blocking the bearing in some cases, the whole picture of the warning area can be obtained maximally and the disease can be identified.

[0028] The structure of the robotic arm base 9 has a large density and volume, providing sufficient stability for the robot during inspection. No matter how the posture of the robotic arm is adjusted, the center of gravity of the robot can always be kept at a relatively low horizontal line.

[0029] The terminal spare mounting base 21 can be installed with a variety of expansion parts. In this invention, a supplementary lighting device is installed. During the inspection and detection of bridge bearings, the bearing area directly below the bridge is often difficult to observe clearly due to insufficient light or being blocked by sundries; while the bearings on both sides of the bridge may have uneven light intensity due to direct sunlight, forming an alternating light and dark phenomenon. The supplementary lighting device can effectively supplement the light in the dark area and balance the overall lighting conditions, ensuring the accuracy and efficiency of the detection work. When the imaging recognizes that the light is dim and uneven, the supplementary lighting device can be controlled to turn on through the main control unit 6.

[0030] When the robot patrols around the bridge bearing on the walking track, the chassis drive system will continuously drive the robot to walk, update and feedback the current movement coordinates. The image acquisition device will conduct real-time detection on the bridge bearing, obtain the bearing image information and analyze it, so as to identify the disease conditions such as cracks, deformations, displacements, etc.

[0031] The design of the rail-type inspection robot fully considers the field work standards. Off-road tires can be assembled on the robot, which has strong grip when walking, ignores gravel and sand, and can also work under wet conditions with less accumulated water. The materials of the robot chassis and the robotic arm have different densities. Combined with the structural design, the center of gravity of the robot is finally made lower, ensuring that the robot will not tip over when walking on slopes and when the robotic arm is in various postures.

[0032] The following designs are also carried out to improve the robot's field work ability: A wireless communication module and a spare battery installation box are set on the chassis, which can provide installation positions for the communication module and additional batteries, improving the robot's communication ability and long-term working ability.

[0033] An anti-static housing for the main control unit is set on the chassis, which can effectively protect the stability of the main control unit and prevent damage to the main control unit by sand, stone, rain, static electricity, etc.

[0034] Spare connecting rods for the robotic arm are set, which can increase the length of the robot's robotic arm, improve the image acquisition ability, and are applicable to some conditions where the distance between the support and the track is too far.

[0035] A spare support mounting bracket for the image acquisition device is set to add light sources or other sensors, improving the image acquisition ability in extreme environments.

[0036] In this embodiment, in addition to the main control unit 6, the robot control system further includes a power integration unit (not shown in the figure); the main control unit 6 and the power integration unit are arranged on the top plate 5 of the chassis. The main control unit 6 is respectively connected to the chassis control unit of the chassis drive system and the robotic arm control unit of the robotic arm control system through lines.

[0037] The main control unit 6 is a Jetson Nano control board, which is a high-performance and low-power edge AI computing platform launched by NVIDIA, designed specifically for machine learning, deep learning, and inference. In the bridge bearing disease robot control and detection system, the Jetson Nano control board serves as the main controller, responsible for receiving real-time data from the image processing system, including the disease image information of the bridge bearing. It runs complex image processing and machine learning algorithms to analyze these images and identify disease conditions such as cracks, deformations, and displacements. The Jetson Nano control board is also responsible for coordinating the work of each subsystem, including controlling the chassis drive system, the robotic arm control system, and the automatic adjustment of the supplementary light, ensuring the efficiency and accuracy of the entire detection process.

[0038] The chassis control unit is an STM32 control board. The control of the chassis drive system is carried out by the STM32 control board, but the control strategy is issued by the Jetson Nano control board. Under the instructions of the Jetson Nano control board, the chassis drive system can accurately control the forward, backward, turning and other actions of the robot. In addition, the Jetson Nano control board controls the wireless communication module, robotic arm, image acquisition device, etc. of the robot through the IIC protocol.

[0039] The power module of the power integration unit uses a lithium battery. After internal voltage conversion and step-down processing through an integrated control circuit, it is supplied to the main control unit, and then the main control unit distributes it to each corresponding circuit device for the second time.

[0040] In this embodiment, the image acquisition device uses a depth camera, namely the Gemini Plus 3D depth camera. Based on the binocular stereo vision principle, this camera takes pictures of the same scene from different angles through two infrared cameras, and then uses an algorithm to calculate the position deviation (i.e., parallax) between corresponding points in the images, so as to obtain the three-dimensional geometric information of the object. It has the advantages of low cost and strong environmental adaptability. At the same time, in order to solve the problems of insufficient light, uneven light intensity, alternating light and dark, etc. of bridge bearings, supplementary light source LED lights are configured around the depth camera, which can effectively supplement the light in the dark area and balance the overall lighting conditions, ensuring the accuracy and efficiency of the detection work.

[0041] In this embodiment, the image processing and storage device mainly relies on the cloud server and is applied to problem inspection. The robot will take a large number of picture data on site and upload the collected images to the cloud server in real time. The cloud server is installed with a GPU (Graphics Processing Unit) acceleration card, which can accelerate the execution of image processing algorithms. At the same time, the cloud server also includes a cloud storage service for storing the original image data, processed images and analysis results, supporting long-term preservation and fast access of data.

[0042] In this embodiment, the edge computing device refers to the main control unit and is mainly applied to regular inspections. In order to reduce the data transmission volume and latency, a lightweight bridge bearing disease feature recognition model is deployed on the main control unit, and the GPU in this edge processor is used to analyze and judge the images taken by the robot in real time.

[0043] In this embodiment, the core architecture of the network communication system consists of components such as routers, 4G IoT cards, and wireless network communication modules. They work together to establish a stable and efficient network connection. The router, as the central hub, is responsible for routing and forwarding data packets to ensure the smooth flow of information between different devices. The 4G IoT card provides wireless wide area network access capabilities, enabling the system to be flexibly deployed in scenarios without a fixed broadband network to achieve remote data transmission and monitoring. The wireless network communication module focuses on building a wireless local area network within a short distance, facilitating instant communication and data sharing between nearby devices. These components work together to build a widely covered and flexible network communication environment, ensuring the instant and accurate transmission of information between the image processing system and its various components, laying a solid foundation for the stable operation and efficient collaboration of the system.

[0044] Referring to Figure 4 , the bridge bearing disease detection method implemented using the system includes the following process: The regular task of the robot is to conduct regular inspections. At the beginning, the robot moves from the bridge deck to a certain bearing of the bridge through the walking track by itself. The image acquisition device on the robotic arm continuously obtains video stream information and extracts picture frames for image recognition in the main control unit of the robot to ensure that any potential disease parts of the bridge bearing can be comprehensively and accurately detected. At the same time, the pictures obtained by the image acquisition device are uploaded to the cloud server through the network communication system, and secondary analysis will be carried out in the cloud server (the secondary warning is a mixture of computer enhanced algorithms and manual expert judgment) to ensure that no disease parts of the bridge bearing are missed. If no warning parts of the bridge bearing are found during the entire inspection process, the robot will send a message and generate an inspection report to be sent to the mobile APP, and the inspection of this bearing is completed, and the inspection of the next bearing will continue; If a warning part of the bridge bearing is found during the inspection process, or the cloud server analyzes and finds that a certain bearing has a warning part, then problem inspection will be carried out for this bearing. The problem inspection mainly involves the adaptive following system related to the robotic arm. The robot internally controls the movement of the chassis for positioning according to the coordinates corresponding to the bearing in the inspection record, and controls the movement of the joints of the robotic arm for adaptive adjustment at the observation position, and performs multi-scheme configuration according to factors such as distance, light, and imaging angle to obtain detailed data of the warning part of this bearing. During this process, manual intervention can be carried out to optimize the imaging quality, and the results are comprehensively judged through re-inspection and manual expert inspection. Finally, an inspection report will be generated, and manual work will be carried out for the follow-up.

[0045] The mobile phone APP has the functions of manually controlling the movement of the robot, starting the automatic detection of the robot, displaying the real-time detection images and detection results of the main control unit, and displaying the complete detection report after the detection is completed. Among them, the function of manually controlling the movement of the robot means that through the corresponding buttons on the mobile phone APP, the forward, backward, and pause of the robot chassis can be controlled, and the independent rotation of each joint of the robotic arm and the forward, backward, left, right, up, down, and rotation of the end of the entire robotic arm can also be controlled. This function is mainly applied in the following situations: one is when the robot fails and cannot automatically return or continue to complete the detection task; the second is when the support is too large or there are too many surrounding obstacles beyond the detection range of the robot; the third is when there is a doubt about the accuracy of the detection results and it is necessary to conduct a fine detection of the local position of some supports again. The function of starting the automatic detection of the robot is mainly applied when the robot is initially placed on the track on the bridge surface and starts to run, and when it is necessary to restart the automatic operation of the robot again after manual intervention during the detection. At the initial start, the robot will adjust the robot chassis and robotic arm to the optimal posture according to the preset program to ensure smooth operation and collision avoidance when going downhill; when the automatic operation of the robot is restarted again after manual intervention during the detection, the robot will automatically adjust to the optimal posture according to the current environment and continue to automatically execute the remaining detection tasks. During the entire detection, the mobile phone APP can display the images processed by the main control unit in real time and display the detection results. After the detection is completed, the detection results and detailed detection reports of each support on the entire bridge pier can be viewed through the mobile phone APP.

[0046] When conducting problem inspection, the precise deviation amount between the current perspective of the image acquisition device and the center position of the disease image is calculated through an intelligent algorithm. Based on this deviation data, the system immediately applies the principle of robot inverse kinematics to precisely calculate the optimal movement angles required for each joint of the robotic arm, thereby precisely controlling the robotic arm to drive the image acquisition device for adaptive adjustment to ensure that the image acquisition device always locks and tracks the warning area frontally; if there are obstacles or the warning area is incomplete after the initial image adjustment, the system will try to move the robot and keep the robotic arm and the image acquisition device locked to the warning area frontally from different positions. The entire process will continuously upload the image set to achieve high-quality and non-missing photo detection.

[0047] Refer to Figure 5 , the adaptive adjustment process is as follows: First, subscribe to the topic message published by the bridge bearing disease image recognition node to obtain the recognized bridge bearing disease information; Then, when a disease image is recognized, obtain the center position of the disease image; Finally, the angle required for aligning the center position of the current viewing angle of the image acquisition device with the center position of the disease image is solved through inverse kinematics, a corresponding topic message is published, and the robotic arm is controlled to move, so that the robotic arm drives the image acquisition device to move to lock the center position of the disease image for taking pictures from the front.

[0048] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. The bridge bearing defect detection system based on the rail inspection robot is characterized by: It includes robot walking guide mechanism, track inspection robot and adaptive following system; The track-type inspection robot comprises a chassis, and a mechanical arm is provided on the chassis; The adaptive following system includes a robot control system, an image processing system and a network communication system; The robot control system includes a main control unit, which is mainly used to control the movement of the chassis and the adaptive adjustment of the robot arm. When inspecting the bridge bearing, the robot controls the movement of the chassis to locate the corresponding coordinates of the bearing by patrolling and recording, and controls the movement of each joint of the robot arm to make adaptive adjustments at the observation position. Multiple schemes are configured according to factors such as distance, light, and imaging angle to obtain detailed data of the warning part of the bearing. The image processing system includes an image acquisition device, an image processing and storage device, and an edge computing device; the image acquisition device is installed on the robotic arm and is mainly used for acquiring image data; the image processing and storage device uses a cloud server and is mainly used for processing, analyzing and applying image data; the edge computing device refers to a main control unit, which is mainly used for processing, analyzing and applying image data, and transmitting image data to the cloud server through a network communication system.

2. The bridge bearing defect detection system based on the rail inspection robot according to claim 1 is characterized in that: The chassis includes a chassis body, side baffles and driving wheels are installed on the left and right sides of the chassis body, guide wheel mounting brackets are installed on the front and rear ends of the chassis body, guide wheels are installed on both sides of the guide wheel mounting bracket, and a top plate is installed on the top of the chassis body; the chassis body is also provided with a motor, a reducer, a steering mechanism and a chassis control unit, the motor is connected to the driving wheel through the reducer to form a chassis drive system, and the chassis drive system can accurately control the forward, backward and steering movements of the robot.

3. The bridge bearing defect detection system based on the rail inspection robot according to claim 1 or 2, characterized in that: The mechanical arm includes a mechanical arm base, a first joint component, a second joint steering gear, a second joint steering gear mounting piece, a first extension arm, a third joint steering gear, a third joint steering gear mounting piece, a second extension arm, a fourth joint steering gear, a fourth joint steering gear mounting piece, an image acquisition device connecting frame, an image acquisition device and a terminal spare mounting base. The first joint component is installed on the mechanical arm base and is connected to a drive motor. Under the action of the drive motor, the first joint component can rotate along the mechanical arm base on a horizontal plane. The second joint steering gear is installed on the first joint component. The second joint steering gear mounting piece is installed on the second joint steering gear. Under the action of the second joint steering gear, the second joint steering gear mounting piece can rotate along the second joint steering gear. The first extension arm is installed on the second joint steering gear mounting piece. The third joint steering gear is installed on the top of the first extension arm. The third joint steering gear mounting piece is installed on the third joint steering gear. Under the action of the third joint steering gear, the third joint steering gear mounting piece can rotate along the third joint steering gear, the second extension arm is installed on the third joint steering gear mounting piece, the fourth joint steering gear is installed on the top of the second extension arm, and the fourth joint steering gear mounting piece is installed on the fourth joint steering gear. Under the action of the fourth joint steering gear, the fourth joint steering gear mounting piece can rotate along the fourth joint steering gear, the image acquisition device is installed on the fourth joint steering gear mounting piece through the image acquisition device connecting frame, and the end spare mounting base is installed on the fourth joint steering gear mounting piece; a robotic arm control unit is provided in the robotic arm base, and the robotic arm control unit is connected to the driving motor of the first joint component, the second joint steering gear, the third joint steering gear, and the fourth joint steering gear through lines respectively, and controls the driving motor of the first joint component, the second joint steering gear, the third joint steering gear, and the fourth joint steering gear to work, thereby forming a robotic arm control system.

4. The bridge bearing defect detection system based on the rail inspection robot according to claim 1 or 2, characterized in that: The image acquisition device uses a depth camera, namely the Gemini Plus 3D depth camera, which is based on the principle of binocular stereo vision. It uses two infrared cameras to shoot the same scene from different angles, and then uses an algorithm to calculate the position deviation between corresponding points in the image to obtain the three-dimensional geometric information of the object; at the same time, a supplementary light source LED lamp is configured around the depth camera.

5. The bridge bearing defect detection system based on the rail inspection robot according to claim 1 or 2, characterized in that: The image processing and storage equipment is used for regular inspections and problem inspections; the robot will take a large amount of image data on site and upload the collected images to the cloud server in real time. The cloud server is equipped with a GPU accelerator card to accelerate the execution of the image processing algorithm. At the same time, the cloud server also includes a cloud storage service for storing original image data, processed images and analysis results, supporting long-term storage and fast access to data.

6. The bridge bearing defect detection system based on the rail inspection robot according to claim 1 or 2, characterized in that: The edge computing device is mainly used for regular inspections. In order to reduce data transmission volume and delay, a lightweight bridge bearing disease feature recognition model is deployed on the main control unit, and the GPU in the edge processor is used to perform real-time analysis and judgment on the images taken by the robot.

7. The bridge bearing defect detection system based on the rail inspection robot according to claim 1 or 2, characterized in that: The core architecture of the network communication system is composed of a router, a 4G IoT card and a wireless network communication module, which work together to establish a stable and efficient network connection; As the hub, the router is responsible for routing and forwarding data packets to ensure the smooth flow of information between different devices; the 4G IoT card provides wireless WAN access capabilities, enabling the system to be flexibly deployed in scenarios without fixed broadband networks to achieve remote data transmission and monitoring; and the wireless network communication module focuses on the construction of wireless LANs within short distances, facilitating instant communication and data sharing between close-range devices.

8. A bridge bearing defect detection method implemented by the system of claims 1 to 7, characterized in that: The process includes: The robot's routine task is regular inspection. At the beginning, the robot moves from the bridge deck to a bridge support by itself via the walking track. The image acquisition device on the robot arm obtains video stream information in real time, and extracts picture frames for image recognition in the robot's main control unit to ensure that any potential defective parts of the bridge support can be fully and accurately detected. At the same time, the pictures obtained by the image acquisition device will be uploaded to the cloud server through the network communication system, and the cloud server will perform secondary analysis to ensure that no defective parts of the bridge support are missed. If the entire inspection process does not find the warning part of the bridge support, the robot will send a message and generate a test report to the mobile phone APP. The inspection of the support is completed and the next support inspection will continue. If a warning part of a bridge bearing is found during the inspection process, or the cloud server analysis finds that a certain bearing has a warning part, a problem inspection will be carried out on the bearing. The problem inspection mainly involves the adaptive following system related to the robotic arm; the robot controls the movement of the chassis for positioning by recording the corresponding coordinates of the bearing through inspection, and controls the movement of each joint of the robotic arm for adaptive adjustment at the observation position. Multiple schemes are configured according to factors such as distance, light, and imaging angle to obtain detailed data on the warning part of the bearing. Manual intervention can be made in this process to optimize the imaging quality, and re-inspection and comprehensive analysis by manual expert inspection can be carried out. Finally, the results will be generated into a test report, and subsequent work will be carried out manually.

9. The bridge bearing defect detection method according to claim 8, characterized in that: The mobile phone APP has the functions of manually controlling the robot movement, starting the robot's automatic detection, displaying the real-time detection image and detection results of the main control unit, and displaying a complete detection report after the detection is completed.

10. The bridge bearing defect detection method according to claim 8, characterized in that: When performing problem inspection, the precise deviation between the current viewing angle of the image acquisition device and the center position of the disease image is calculated through an intelligent algorithm. Based on this deviation data, the system immediately uses the inverse kinematics principle of the robot to accurately calculate the optimal movement angle required for each joint of the robotic arm, thereby accurately controlling the robotic arm to drive the image acquisition device to make adaptive adjustments, ensuring that the image acquisition device always locks on and tracks the warning area; If there are obstructions or the warning area is incomplete after the initial adjustment of the image, the system will try to move the robot and keep the robotic arm and image acquisition device locking the warning area from different points. The image set will be continuously uploaded during the whole process to achieve high-quality and complete photo detection.

Images