Control method and device of inspection and maintenance integrated robot for sound barrier

Abstract

The invention discloses a control method and device of a sound barrier inspection and maintenance integrated robot, and relates to the field of robot control. According to the method, a three-dimensional digital twinborn model is constructed according to structural feature data and material attribute data of a sound barrier; generating an initial inspection path, constructing a dynamic obstacle avoidance strategy, correcting path planning parameters in real time to obtain a current inspection path, and collecting an image, bolt resonant frequency and environmental parameters of each target area of the sound barrier based on the current inspection path; determining a corrosion target area according to the image, determining a corrosion area ratio and a loosening displacement according to the corrosion target area, a bolt resonant frequency and a three-dimensional digital twinborn model, and determining an environmental corrosion factor according to the environmental parameters; and the damage index of the target area is determined according to the corrosion area ratio, the loosening displacement and the environment corrosion factors, and the main mechanical arm and the auxiliary mechanical arm are controlled to maintain the target area according to the damage index. By implementing the technical scheme provided by the invention, the inspection and maintenance efficiency is remarkably improved.

Description

Technical Field

[0001] This application relates to the technical field of robot control, and particularly to a control method and device for an integrated inspection and maintenance robot for sound barriers. Background Art

[0002] As an important sound insulation facility in the transportation and industrial fields, the stability and durability of the performance of sound barriers directly affect the noise control effect of the surrounding environment. With the increasing requirements for environmental protection in society, the maintenance and inspection of sound barriers have received more and more attention.

[0003] In the prior art, to solve the problems of inspection and maintenance of sound barriers, fixed cameras are usually used to monitor the status of sound barriers or manual inspections are carried out regularly. The appearance and fastener status of the sound barriers are recorded and manual repairs are performed. Fixed cameras cannot cover all angles and are difficult to identify subtle damages; the manual inspection and maintenance methods are time-consuming and laborious, with low efficiency, increased operation risks under adverse weather conditions, and are difficult to meet the maintenance needs of large-scale sound barrier networks.

[0004] Therefore, there is an urgent need for a technical means that can comprehensively detect the status of sound barriers and provide accurate maintenance solutions. Summary of the Invention

[0005] This application provides a control method and device for an integrated inspection and maintenance robot for sound barriers, achieving the effects of accurately detecting the damage condition of sound barriers and intelligently planning maintenance solutions, significantly improving the inspection and maintenance efficiency, and reducing labor costs and potential safety hazards.

[0006] In the first aspect of this application, a control method for an integrated inspection and maintenance robot for sound barriers is provided, which is applied to the integrated inspection and maintenance robot. The method includes: Obtain the structural feature data and material property data of the sound barrier, and construct a three-dimensional digital twin model according to the structural feature data and the material property data; Generate an initial inspection path through a preset algorithm, construct a dynamic obstacle avoidance strategy through an infrared ranging sensor, and real-time correct the path planning parameters to obtain the current inspection path. Based on the current inspection path, collect images, bolt resonance frequencies, and environmental parameters of each target area of the sound barrier. The environmental parameters include temperature, humidity, and salt spray concentration; Determine the corrosion target area according to the image, determine the corrosion area ratio according to the corrosion target area and the three-dimensional digital twin model, determine the loosening displacement amount according to the bolt resonance frequency and the three-dimensional digital twin model, and determine the environmental corrosion factor according to the environmental parameters; Determine the damage index of the target area according to the corrosion area ratio, the loosening displacement amount, and the environmental corrosion factor, determine the maintenance mode according to the damage index, and control the main robotic arm and the auxiliary robotic arm to perform maintenance on the target area according to the maintenance mode.

[0007] Optionally, the constructing the three-dimensional digital twin model according to the structural feature data and the material property data includes: Based on the lidar point cloud data, the multi-spectral polarization image, and the building information modeling design parameters in the structural feature data, segment each component of the sound barrier, and determine the structural information of the sound barrier according to the structural data and the connection relationship of each component; Based on the bolt elastic modulus, the coating thickness, the porosity of the sound-absorbing material, and the preset material degradation model in the material property data, determine the material property change information of each component of the sound barrier under different environmental conditions; Obtain the precise spatial position information of the sound barrier, and generate a continuous surface model of the sound barrier through the Poisson surface reconstruction algorithm; Integrate the structural information, the material property change information, and the continuous surface model to construct a three-dimensional digital twin model.

[0008] Optionally, the constructing the dynamic obstacle avoidance strategy through the infrared distance measurement sensor and real-time correcting the path planning parameters to obtain the current inspection path includes: When moving along the initial inspection path and the infrared distance measurement sensor detects an obstacle and needs to adjust the path, generate multiple potential paths, and determine the estimated energy consumption of each potential path according to the current motion model, the current load condition, and the energy consumption characteristics of the terrain of the potential path; Select the target potential path with the minimum estimated energy consumption, obtain the actual energy consumption, and compare it with the estimated energy consumption. When the difference between the actual energy consumption and the estimated energy consumption is greater than the threshold, determine the influencing factors, and correct the target potential path according to the influencing factors.

[0009] Optionally, the determining the corrosion area ratio according to the corrosion target area and the three-dimensional digital twin model includes: Extract the contour of the corrosion target area, and map the image pixel coordinates of the contour to the three-dimensional digital twin model; Obtain the radius of curvature of the corrosion target area from the three-dimensional digital twin model, calculate the corrosion depth using the terahertz reflection time difference, and determine the actual corrosion area according to the radius of curvature and the corrosion depth; Determine the corrosion area ratio according to the actual corrosion area and the projected area of the corrosion target area.

[0010] Optionally, the determination of the loosening displacement amount based on the bolt resonance frequency and the three-dimensional digital twin model includes: Determining the actual resonance peak frequency by using the Gaussian fitting method based on the bolt design frequency, and calculating the frequency offset according to the actual resonance peak frequency and the bolt design frequency; Correcting the elastic modulus according to the current temperature value, and calculating the loosening displacement amount according to the frequency offset, the corrected elastic modulus, the bolt dynamic parameters, and the bolt cross-sectional area, where the bolt dynamic parameters include the bolt length and the material density.

[0011] Optionally, the determination of the damage index of the target area based on the corrosion area ratio, the loosening displacement amount, and the environmental corrosion factor, the determination of the maintenance mode according to the damage index, and the control of the main robotic arm and the auxiliary robotic arm to repair the target area according to the maintenance mode include: Performing a weighted sum of the corrosion area ratio, the loosening displacement amount, and the environmental corrosion factor to obtain the damage index, When the damage index is less than the first preset threshold, determining the maintenance mode as the minor damage repair mode, controlling the main robotic arm to remove the surface corrosion products of the target area, and controlling the auxiliary robotic arm to spray an anti-corrosion coating on the target area; When the damage index is greater than or equal to the first preset threshold and less than the second preset threshold, determining the maintenance mode as the moderate damage repair mode, controlling the main robotic arm to grind the target area to remove the corrosion layer, and controlling the auxiliary robotic arm to perform filling repair and spray an anti-corrosion coating on the ground target area; When the damage index is greater than or equal to the second preset threshold, determining the maintenance mode as the severe damage replacement mode, controlling the main robotic arm and the auxiliary robotic arm to work together to remove the damaged components in the target area and install new components.

[0012] Optionally, the control of the main robotic arm to grind the target area to remove the corrosion layer, and the control of the auxiliary robotic arm to perform filling repair and spray an anti-corrosion coating on the ground target area includes: Adjusting the grinding force of the main robotic arm in real time according to the force information fed back by the six-axis force sensor, and combining the vision system to monitor the grinding progress and surface quality in real time; Adjusting the welding parameters of the auxiliary robotic arm according to the shape and depth of the damaged area, where the welding parameters include the welding current, the welding time, and the welding pressure, and determining the spraying rate and flow rate of the coating according to the distance between the auxiliary robotic arm and the target area.

[0013] In a second aspect of the present application, a control system for an integrated inspection and maintenance robot for a sound barrier is provided, including a model module, an inspection module, a calculation module, and a maintenance module, where: The model module is configured to obtain the structural feature data and material property data of the sound barrier, and construct a three-dimensional digital twin model according to the structural feature data and the material property data; The inspection module is configured to generate an initial inspection path through a preset algorithm, construct a dynamic obstacle avoidance strategy through an infrared ranging sensor, and correct the path planning parameters in real time to obtain the current inspection path, and collect images, bolt resonance frequencies, and environmental parameters of each target area of the sound barrier based on the current inspection path, where the environmental parameters include temperature, humidity, and salt fog concentration; The calculation module is configured to determine the corrosion target area according to the image, determine the corrosion area ratio according to the corrosion target area and the three-dimensional digital twin model, determine the loosening displacement amount according to the bolt resonance frequency and the three-dimensional digital twin model, and determine the environmental corrosion factor according to the environmental parameters; The maintenance module is configured to determine the damage index of the target area according to the corrosion area ratio, the loosening displacement amount, and the environmental corrosion factor, determine the maintenance mode according to the damage index, and control the main robotic arm and the auxiliary robotic arm to repair the target area according to the maintenance mode.

[0014] In a third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, and both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method described in any one of the above.

[0015] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions that, when executed, execute the method described in any one of the above.

[0016] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By obtaining the structural feature data and material property data of the sound barrier to construct a three-dimensional digital twin model, the actual state of the sound barrier can be simulated with high precision. This model not only contains the geometric structure information of the sound barrier, but also integrates the performance change data of the material under different environmental conditions. This enables accurate analysis and prediction based on the model in subsequent damage assessment and maintenance planning, allowing for early understanding of potential problems with the sound barrier and providing strong support for formulating scientific and reasonable maintenance strategies. With the help of the three-dimensional digital twin model, maintenance personnel can simulate and verify different maintenance plans in a virtual environment. By comparing the effects and costs of various plans, the optimal maintenance plan can be selected, thereby improving the maintenance efficiency and quality, and reducing the maintenance cost and risk; 2. By presetting an algorithm to generate an initial inspection path and combining it with an infrared ranging sensor to construct a dynamic obstacle avoidance strategy, the path planning parameters can be corrected in real time, enabling the robot to quickly and accurately reach each target area of the sound barrier. This avoids path detours and repeated inspections caused by obstacles or environmental changes, greatly improving the inspection efficiency and shortening the inspection cycle. The dynamic obstacle avoidance strategy can ensure that the robot timely avoids obstacles during the inspection process, preventing collisions with surrounding objects, safeguarding the safety of the robot itself, and reducing potential damage to the surrounding environment and facilities of the sound barrier; 3. Using the collected images to determine the corrosion target area, and combining with the three-dimensional digital twin model to calculate the corrosion area ratio; determining the loosening displacement according to the bolt resonance frequency; determining the environmental corrosion factor according to the environmental parameters. By comprehensively considering these factors to determine the damage index of the target area, the damage degree of the sound barrier can be evaluated comprehensively and accurately. Compared with the traditional single-factor evaluation method, this method is more scientific and reliable, and can provide an accurate basis for subsequent maintenance decisions; by real-time monitoring and analyzing the corrosion area ratio, loosening displacement, and environmental corrosion factor, early damage signs of the sound barrier can be detected in a timely manner; 4. According to the damage index, different maintenance modes are determined, such as minor damage repair mode, moderate damage repair mode, and severe damage replacement mode, and corresponding maintenance measures can be taken for different damage degrees. This targeted maintenance method can ensure the maximization of the maintenance effect, avoid problems of over-maintenance or under-maintenance, and improve the utilization efficiency of maintenance resources. Controlling the main manipulator and the auxiliary manipulator to repair the target area according to the maintenance mode realizes the efficient collaborative operation between the components of the robot. The main manipulator and the auxiliary manipulator can accurately and quickly complete their respective maintenance tasks according to the preset task division and cooperation strategy, improving the maintenance efficiency and quality, reducing manual intervention, and lowering the labor intensity. Description of the Drawings

[0017] Figure 1 is a schematic flowchart of the control method of the inspection and maintenance integrated robot for the sound barrier disclosed in the embodiment of the present application; Figure 2 It is a schematic diagram of the modules of the control system of the integrated inspection and maintenance robot for the sound barrier disclosed in the embodiments of the present application; Figure 3 It is a schematic diagram of the structure of an electronic device disclosed in the embodiments of the present application.

[0018] Explanation of reference numerals: 201, model module; 202, inspection module; 203, calculation module; 204, maintenance module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Specific embodiments

[0019] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0020] In the description of the embodiments of the present application, words such as "for example" or "for illustration" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for illustration" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "for example" or "for illustration" is intended to present relevant concepts in a specific manner.

[0021] In the description of the embodiments of the present application, the meaning of the term "a plurality" refers to two or more. For example, a plurality of systems refers to two or more systems, and a plurality of screen terminals refers to two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the technical features indicated. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0022] This embodiment discloses a control method for an integrated inspection and maintenance robot for a sound barrier, which is applied to the integrated inspection and maintenance robot. The integrated inspection and maintenance robot includes a main robotic arm and at least one sub-robotic arm. Figure 1 It is a schematic flowchart of the control method for the integrated inspection and maintenance robot for the sound barrier disclosed in the embodiments of the present application, as Figure 1 shown, the method includes the following steps: S101. Obtain the structural characteristic data and material property data of the sound barrier, and construct a three-dimensional digital twin model according to the structural characteristic data and the material property data; S102. Generate an initial inspection path through a preset algorithm, construct a dynamic obstacle avoidance strategy through an infrared ranging sensor, and correct the path planning parameters in real time to obtain the current inspection path. Based on the current inspection path, collect images, bolt resonance frequencies, and environmental parameters of each target area of the sound barrier. The environmental parameters include temperature, humidity, and salt fog concentration; S103. Determine the corrosion target area according to the image, determine the corrosion area ratio according to the corrosion target area and the three-dimensional digital twin model, determine the loosening displacement according to the bolt resonance frequency and the three-dimensional digital twin model, and determine the environmental corrosion factor according to the environmental parameters; S104. Determine the damage index of the target area according to the corrosion area ratio, loosening displacement, and environmental corrosion factor, determine the maintenance mode according to the damage index, and control the main robotic arm and the auxiliary robotic arm to repair the target area according to the maintenance mode.

[0023] The sound barrier is scanned in all directions by using LiDAR to obtain a large amount of point cloud data on its surface. These data contain the three-dimensional geometric information of the sound barrier, such as shape, size, position, etc., and can accurately depict the external contour and surface features of the sound barrier. The image of the sound barrier is captured by a multispectral polarization camera. Images of different spectral bands and polarization states can provide rich information such as the surface material, color, texture, etc. of the sound barrier, which helps to identify different components and surface conditions of the sound barrier. The BIM model parameters of the sound barrier in the design stage are obtained, including the geometric dimensions, connection methods, material specifications, etc. of each component. These parameters provide an accurate design benchmark for building a digital twin model. The elastic modulus of the bolt material is obtained by referring to the material manual. This parameter reflects the stiffness characteristics of the bolt when it is under force, which is crucial for evaluating the connection state and looseness of the bolt. The coating thickness of components such as bolts is measured using professional measuring equipment (such as a coating thickness gauge). The thickness of the coating will affect the corrosion resistance and service life of the component. The porosity of the sound-absorbing material is determined by referring to the material manual. Porosity is one of the key factors affecting the performance of the sound-absorbing material and is closely related to the noise reduction effect of the sound barrier. A three-dimensional digital twin model is constructed based on the structural feature data and material property data. The model not only reflects the geometric shape of the sound barrier, but also contains the physical properties and performance change information of the material, and can simulate the actual state of the sound barrier in real time. The initial inspection path is generated by using a suitable path planning algorithm (such as A* algorithm, Dijkstra algorithm, etc.). The algorithm comprehensively considers factors such as the structural layout of the sound barrier, the distribution of the target area, and the robot's motion ability to plan an optimal path that can cover all target areas. The distance of the obstacle in front of the robot is detected in real time according to the infrared ranging sensor installed on the robot. When the robot moves along the initial inspection path, if the infrared ranging sensor detects an obstacle, it means that the current path needs to be adjusted. The path is adjusted according to the preset obstacle avoidance strategy. Based on the current inspection path, a high-definition camera is used to collect images of each target area of the sound barrier. These images can be used for subsequent corrosion target area identification and damage assessment. The resonant frequency of the bolt is measured by a special sensor (such as an accelerometer). The resonant frequency of the bolt is closely related to its tightening state and connection quality. When the bolt is loose, the resonant frequency will change. The environmental parameters of the target area, including temperature, humidity and salt spray concentration, are collected using temperature and humidity sensors and salt spray concentration sensors. These environmental parameters will affect the corrosion rate and damage degree of the sound barrier material. The collected images are preprocessed, such as denoising and contrast enhancement, to improve image quality. The images are then analyzed using image recognition algorithms (such as convolutional neural networks) to identify the corrosion areas in the images. The corrosion area ratio is determined based on the corrosion target area and the three-dimensional digital twin model, and the loose displacement is determined based on the bolt resonance frequency and the three-dimensional digital twin model.Based on the collected environmental parameters such as temperature, humidity, and salt fog concentration, combined with the material corrosion mechanism and experimental data, comprehensively analyze and determine the environmental corrosion factor. For example, high temperature, high humidity, and high salt fog concentration will accelerate the corrosion of materials, and the corresponding environmental corrosion factor will be larger. Obtain the damage index of the target area based on the corrosion area ratio, loosening displacement amount, and environmental corrosion factor. Determine the maintenance mode according to the damage index, and control the main robotic arm and the auxiliary robotic arm to repair the target area according to the maintenance mode.

[0024] Optionally, the constructing the three-dimensional digital twin model according to the structural feature data and the material property data includes: Based on the lidar point cloud data, multi-spectral polarization images, and building information modeling design parameters in the structural feature data, segment each component of the sound barrier, and determine the structural information of the sound barrier according to the structural data and connection relationship of each component; Based on the bolt elastic modulus, coating thickness, porosity of the sound-absorbing material, and the preset material degradation model in the material property data, determine the material property change information of each component of the sound barrier under different environmental conditions; Obtain the accurate spatial position information of the sound barrier, and generate a continuous surface model of the sound barrier through the Poisson surface reconstruction algorithm; Integrate the structural information, the material property change information, and the continuous surface model to construct a three-dimensional digital twin model.

[0025] LiDAR point cloud data: LiDAR can quickly and accurately obtain the three-dimensional coordinate information of a large number of points on the surface of the sound barrier. These point cloud data can reflect the overall shape and contour of the sound barrier. By processing and analyzing the point cloud data, different components of the sound barrier, such as panels, columns, etc., can be identified. Multispectral polarization images: Multispectral images can provide spectral information in different bands, and polarization information can enhance the ability to identify the surface features of objects. Using multispectral polarization images, the material, texture, and other characteristics of each component of the sound barrier can be obtained, which helps to more accurately segment the components. Building Information Modeling (BIM) design parameters: BIM design parameters contain detailed information about the sound barrier in the design stage, such as the dimensions, shapes, connection methods, etc. of each component. These parameters provide important reference bases for segmenting components and determining structural information. Use point cloud processing algorithms (such as clustering algorithms, region growing algorithms, etc.) to process the LiDAR point cloud data, and classify the point cloud data belonging to the same component into one category, so as to achieve the preliminary segmentation of the components. Combine the characteristic information of the multispectral polarization images to further optimize the results of the preliminary segmentation and improve the accuracy of segmentation. According to the BIM design parameters, extract the structural information of the segmented components, including the dimensions, positions, connection relationships, etc. of each component. The elastic modulus is an important indicator to measure the ability of a material to resist elastic deformation. Changes in the elastic modulus of bolts will affect the connection strength and stability of the sound barrier. The coating can play roles such as anti-corrosion and wear resistance. Changes in the coating thickness will directly affect the protection performance of the sound barrier. The porosity is a key factor affecting the sound absorption performance of sound-absorbing materials. Changes in the porosity will cause changes in the sound absorption effect of the sound barrier. The preset material degradation model describes the performance degradation laws of materials under different environmental conditions (such as temperature, humidity, salt spray, etc.). Through this model, the material performance changes of each component of the sound barrier under different environmental conditions can be predicted. According to the preset material degradation model, combined with environmental parameters (which can be obtained from the subsequent collected environmental data), calculate the material performance change amounts of each component under different environmental conditions, such as the reduction value of the elastic modulus, the corrosion rate of the coating, etc. Precise spatial position information is the basis for constructing a three-dimensional digital twin model, which determines the accurate position and attitude of the sound barrier in three-dimensional space. Only by obtaining precise spatial position information can the generated model be ensured to be highly consistent with the actual sound barrier. The Poisson surface reconstruction algorithm is a commonly used method to generate a continuous surface model from point cloud data. It can generate a smooth and continuous surface model according to the point cloud data of the sound barrier, better reflecting the actual shape of the sound barrier. First, preprocess the obtained point cloud data, such as denoising, filtering, etc., to improve the data quality. Then, use the Poisson surface reconstruction algorithm to calculate the processed point cloud data to generate a continuous surface model of the sound barrier. The structural information, material performance change information, and continuous surface model respectively describe the characteristics of the sound barrier from different aspects.The structural information reflects the geometry and connection relationships of the sound barrier; the information on material property changes embodies the performance evolution of the sound barrier under different environmental conditions; the continuous surface model visually displays the appearance of the sound barrier. Only by integrating these three aspects can a complete and accurate three-dimensional digital twin model be constructed. Correlate and fuse the structural information, the information on material property changes, and the continuous surface model. For example, map the positions and connection relationships of components in the structural information onto the continuous surface model, and at the same time bind the information on material property changes to the corresponding components. In this way, the three-dimensional digital twin model can comprehensively and accurately simulate the sound barrier.

[0026] Using lidar point cloud data, multi-spectral polarization images, and building information modeling design parameters to segment the components of the sound barrier can achieve an accurate division of the complex structure of the sound barrier. The lidar point cloud data provides the three-dimensional spatial coordinate information of the sound barrier, the multi-spectral polarization images can capture the spectral characteristic differences of components made of different materials, and the building information modeling design parameters include the design dimensions and position information of the components. By integrating these information, the various components of the sound barrier can be accurately distinguished, laying a foundation for subsequent structural analysis. Based on material property data such as the elastic modulus of bolts, the coating thickness, and the porosity of the sound-absorbing material, combined with a preset material degradation model, the information on material property changes of each component of the sound barrier under different environmental conditions can be determined. This helps to understand the performance evolution law of the material during actual use, predict the remaining life of the material, and provide a scientific basis for the maintenance and replacement of the sound barrier. Considering the influence of different environmental conditions on material properties enables the digital twin model to more realistically reflect the working state of the sound barrier in the actual environment. Different environmental factors such as temperature, humidity, and salt fog concentration will cause phenomena such as corrosion and aging of the material. Through the material degradation model, these effects can be simulated, providing targeted maintenance suggestions for the use of the sound barrier in different regions and seasons. Obtaining the accurate spatial position information of the sound barrier and generating a continuous surface model of the sound barrier through the Poisson surface reconstruction algorithm can highly accurately restore the geometry of the sound barrier. The Poisson surface reconstruction algorithm can generate a smooth and continuous surface based on the discrete point cloud data, avoiding the problems of sharp corners and discontinuity that may occur in traditional modeling methods, and improving the fidelity and accuracy of the model.

[0027] Optionally, a dynamic obstacle avoidance strategy is constructed through an infrared ranging sensor to correct the path planning parameters in real time, so as to obtain the current inspection path including: When moving along the initial inspection path and the infrared ranging sensor detects an obstacle that requires path adjustment, multiple potential paths are generated. According to the current motion model, the current load condition, and the energy consumption characteristics of the terrain of the potential paths, the estimated energy consumption of each potential path is determined. Select the target potential path with the minimum estimated energy consumption, obtain the actual energy consumption, and compare it with the estimated energy consumption. When the difference between the actual energy consumption and the estimated energy consumption is greater than the threshold, determine the influencing factors and correct the target potential path according to the influencing factors.

[0028] When the robot moves along the initial inspection path, the infrared ranging sensor works continuously to detect in real time whether there are obstacles ahead. Once an obstacle is detected and the path needs to be adjusted, the system will activate the dynamic obstacle avoidance strategy. After detecting an obstacle, the system will quickly generate multiple potential paths based on the current position information and the environmental map. These potential paths are feasible routes to bypass the obstacle, and they may have different directions, lengths, and terrain features. For example, some potential paths may be relatively flat but have a longer distance; while some paths may be shorter but pass through some undulating terrain. For each potential path, the system will comprehensively consider the current motion model, the current load condition, and the energy consumption characteristics of the terrain of the potential path to calculate the estimated energy consumption of each potential path. Current motion model: It includes motion parameters such as the speed and acceleration of the inspection device. Different motion states will result in different energy consumption. For example, the acceleration or deceleration process consumes more energy. Current load condition: The load carried by the inspection device, such as equipment and tools, will affect its energy consumption. The heavier the load, the more energy is required for the device to move. Terrain's energy consumption characteristics: Different terrains, such as flat ground, slopes, and rough roads, have different effects on the energy consumption of the inspection device. For example, climbing slopes will significantly increase energy consumption, while flat roads are relatively energy-efficient. Calculating the estimated energy consumption: By comprehensively considering the above factors, the system uses a specific algorithm or model to calculate the estimated energy consumption of each potential path. This estimated value provides an important basis for selecting the optimal path subsequently. The system will select the target potential path with the minimum estimated energy consumption as the current preferred path. This is because choosing the path with the minimum energy consumption can improve the battery life of the inspection device, reduce the number of charging times, and thus improve the inspection efficiency. During the movement of the inspection device along the target potential path, the system will monitor and record the actual energy consumption in real time. Compare the actual energy consumption with the estimated energy consumption. When the difference between the two is greater than the set threshold, it indicates that there is a large deviation between the estimated energy consumption and the actual situation. The system will analyze the influencing factors that cause this deviation. Possible influencing factors include more complex terrain changes than expected, changes in the device load during movement, inaccurate motion model parameters, etc. According to the determined influencing factors, the system will correct the target potential path. For example, if it is found that the energy consumption increases due to terrain changes, the system may re-plan a more suitable path to avoid those high-energy-consuming terrain areas; if it is caused by load changes, the system may adjust the motion parameters to adapt to the new load condition. By continuously correcting the path, the inspection device can complete the task more efficiently.

[0029] By means of a dynamic obstacle avoidance strategy and real-time correction of path planning parameters, the robot can operate stably in a complex and ever-changing environment, ensuring the smooth completion of the inspection task. Even in case of emergencies, it can adjust the path in a timely manner to avoid inspection interruptions caused by obstacles or energy consumption problems, improving the reliability and stability of the inspection. Optimized energy utilization and accurate path selection can reduce the energy consumption of the robot, decrease the number of charging times and the frequency of battery replacement, thereby reducing the operation and maintenance costs. At the same time, avoiding robot damage and repair costs caused by improper path selection further improves the economic benefits. Efficient path planning and energy management enable the robot to complete more inspection tasks in a shorter time, improving the inspection efficiency. At the same time, a stable inspection process can ensure the accuracy and reliability of the collected data, improving the inspection quality.

[0030] Optionally, the determining the corrosion area ratio according to the corrosion target area and the three-dimensional digital twin model includes: Extracting the contour of the corrosion target area and mapping the image pixel coordinates of the contour to the three-dimensional digital twin model; Obtaining the curvature radius of the corrosion target area from the three-dimensional digital twin model, calculating the corrosion depth using the terahertz reflection time difference, and determining the actual corrosion area according to the curvature radius and the corrosion depth; Determining the corrosion area ratio according to the actual corrosion area and the projected area of the corrosion target area.

[0031] Use image processing techniques to extract the contour of the corroded target area from the collected sound barrier images. This usually involves edge detection algorithms such as the Canny edge detection algorithm. This algorithm finds the edge points with obvious gray value changes by calculating the gradient change of the pixel gray values in the image, thus outlining the boundary of the corroded area. For example, in an image of a sound barrier, the corroded area may appear as a darker-colored and abnormally-textured part, and the edge detection algorithm can accurately identify the boundary between these areas and the normal areas, forming the contour of the corroded target area. Map the pixel coordinates of the image of the extracted corroded target area contour to a 3D digital twin model. Since the image is two-dimensional and the 3D digital twin model has spatial information, coordinate transformation is required. This usually involves camera calibration and projection transformation techniques. Camera calibration can determine the internal parameters of the camera (such as focal length, optical center, etc.) and external parameters (such as the position and orientation of the camera). By establishing the mapping relationship between the image coordinate system and the world coordinate system, the pixel coordinates in the image are converted into coordinates in 3D space. Then, use projection transformation to map these 3D coordinates onto the 3D digital twin model so that the contour of the corroded target area can be accurately presented in the model. Obtain the radius of curvature of the corroded target area from the 3D digital twin model. The radius of curvature reflects the degree of curvature of the sound barrier surface and is crucial for accurately calculating the corroded area. In the 3D digital twin model, each point has its corresponding curvature information, and the radius of curvature of each point in the corroded target area can be extracted through the analysis tools or algorithms of the model. For example, for an arc-shaped sound barrier component, the radius of curvature at different positions on its surface may be different, and these specific values can be obtained through the model. Calculate the corrosion depth using terahertz reflection time difference. Terahertz waves have the characteristics of strong penetration and high resolution and can detect internal defects and corrosion conditions of materials. When terahertz waves irradiate the surface of the sound barrier, reflection and refraction occur inside the material, and the corrosion depth can be calculated by measuring the time difference of the reflected waves. Specifically, the terahertz emitter emits terahertz waves towards the corroded target area, and the receiver receives the reflected waves. According to the wave propagation speed and time difference, the corrosion depth is calculated using the formula d = (c × Δt) / 2 (where d is the corrosion depth, c is the propagation speed of terahertz waves in the material, and Δt is the reflection time difference). Determine the actual corroded area based on the radius of curvature and the corrosion depth. Since the surface of the sound barrier may be curved, the corroded area cannot be simply calculated by multiplying the corrosion depth by a fixed width. The influence of the radius of curvature needs to be considered, and the corroded area is regarded as a part of a curved surface. Through calculus methods, the corroded area can be divided into countless tiny surface elements, and the area of each surface element can be calculated based on its radius of curvature and corrosion depth, and then the areas of all surface elements are integrated to obtain the actual corroded area. Calculate the projected area of the corroded target area. The projected area refers to the area of the plane projection of the corroded target area in the direction perpendicular to the line of sight.The projected area can be obtained by projecting the contour of the corroded target area onto a plane and then calculating the area of the planar figure. For example, if the corroded target area is an irregular curved surface, projecting it onto a horizontal plane results in a two-dimensional figure, and the area of this figure is calculated using geometric calculation methods (such as the polygon area formula). The corrosion area ratio is determined based on the actual corrosion area and the projected area. The corrosion area ratio is defined as the ratio of the actual corrosion area to the projected area, and is expressed by the formula: corrosion area ratio = (actual corrosion area / projected area) × 100%. The corrosion area ratio can intuitively reflect the severity of corrosion. If the corrosion area ratio is close to 100%, it indicates that the corrosion almost covers the entire projected area, and the corrosion situation is relatively severe; if the corrosion area ratio is small, it means that the corrosion is mainly concentrated in local areas, and the degree of corrosion is relatively light.

[0032] Identifying the boundary of the corroded area through image processing technology provides a clear target object for subsequent analysis. Mapping the image pixel coordinates of the contour to the three-dimensional digital twin model realizes the precise correspondence from the two-dimensional image to the three-dimensional model, enabling the position and shape of the corroded area in the three-dimensional space to be determined. Obtaining the curvature radius of the corroded target area from the three-dimensional digital twin model fully considers the influence of the geometric characteristics of the sound barrier structure on corrosion. Areas with different curvature radii may exhibit different behaviors during the corrosion process. Areas with larger curvatures may be more likely to accumulate corrosion media, resulting in an accelerated corrosion rate. By obtaining the curvature radius, the corrosion mechanism and degree can be analyzed more accurately. Calculating the corrosion depth using terahertz reflection time difference provides a high-precision, non-contact measurement method. Terahertz waves have good penetrability and resolution, and can accurately detect the corrosion situation inside the sound barrier material, avoiding the damage and errors that may be caused by traditional measurement methods. Determining the actual corrosion area based on the curvature radius and the corrosion depth takes into account the actual distribution of corrosion in the three-dimensional space. Due to the complexity of the sound barrier structure, corrosion is not a simple two-dimensional planar phenomenon, but is closely related to the curvature of the structure. By combining the curvature radius and the corrosion depth, the actual area of corrosion on the three-dimensional surface can be calculated more accurately. Determining the corrosion area ratio based on the actual corrosion area and the projected area of the corroded target area quantifies the degree of corrosion. The corrosion area ratio is an intuitive indicator that can clearly reflect the proportion of corrosion in the target area, facilitating the comparison and analysis of the corrosion situations in different areas.

[0033] Optionally, the determining the loosening displacement amount according to the bolt resonance frequency and the three-dimensional digital twin model includes: Determining the actual resonance peak frequency using the Gaussian fitting method based on the bolt design frequency, and calculating the frequency offset according to the actual resonance peak frequency and the bolt design frequency; The elastic modulus is corrected according to the current temperature value, and the loosening displacement is calculated based on the frequency offset, the corrected elastic modulus, the bolt dynamic parameters, and the bolt cross-sectional area, where the bolt dynamic parameters include the bolt length and the material density.

[0034] Gaussian fitting is a mathematical statistical method used to fit data to conform to the characteristics of a Gaussian distribution (normal distribution). In the analysis of the resonant frequency of bolts, due to the possible presence of noise and interference in the actually measured frequency data, the resonant peaks may not be obvious or accurate enough. The Gaussian fitting method can be used to smooth these data and more accurately determine the frequency of the actual resonant peaks. For example, in spectral analysis, the measured bolt vibration frequency data may exhibit multiple peaks, some of which may be spurious peaks caused by noise. Through Gaussian fitting, these data can be fitted to obtain a Gaussian curve that better conforms to the actual resonant characteristics, thereby accurately determining the frequency of the actual resonant peaks. The bolt design frequency is the theoretical frequency determined based on factors such as its material, dimensions, and expected usage conditions during the bolt design stage. Based on the bolt design frequency for Gaussian fitting, it can provide a reference standard for determining the actual resonant peak frequency. By comparing with the design frequency, it is easier to identify abnormal frequencies in actual measurements and improve the accuracy of frequency determination. For instance, if the actually measured frequency data differs significantly from the bolt design frequency, it may indicate that the bolt is loose, damaged, or has other abnormal conditions, and further analysis of the reasons is required. The frequency offset is the difference between the actual resonant peak frequency and the bolt design frequency. It reflects the deviation between the actual vibration state of the bolt and the designed state and is an important indicator for measuring whether the bolt is loose or abnormal. For example, when the bolt is tightly fastened, the actual resonant peak frequency should be close to the bolt design frequency, and the frequency offset is small; while when the bolt becomes loose, the actual resonant peak frequency changes, and the frequency offset increases. Calculating the frequency offset can provide a key parameter for subsequent calculation of the loosening displacement. The magnitude of the frequency offset is closely related to the degree of bolt loosening. By analyzing the frequency offset, the loosening condition of the bolt can be preliminarily judged and a basis for further calculation of the loosening displacement can be provided. For example, the larger the frequency offset, the more severe the loosening of the bolt may be, and the larger the loosening displacement that needs to be calculated may be. The elastic modulus of a material changes with temperature. Generally, an increase in temperature leads to a decrease in the elastic modulus of the material, while a decrease in temperature causes the elastic modulus to increase. In the calculation of the bolt loosening displacement, if the influence of temperature on the elastic modulus is not considered, the calculation result will be inaccurate. For example, in a high-temperature environment, the elastic modulus of the bolt material decreases, and the stiffness of the bolt also decreases accordingly, which will affect the vibration characteristics and loosening displacement of the bolt. Therefore, it is necessary to correct the elastic modulus according to the current temperature value. Usually, a relationship model between the elastic modulus and temperature can be established through experimental data or empirical formulas, and then the elastic modulus can be corrected according to the current temperature value. The corrected elastic modulus can more accurately reflect the actual mechanical properties of the bolt at the current temperature, thereby improving the accuracy of the loosening displacement calculation.For example, given the elastic modulus data of a certain bolt material at different temperatures, the functional relationship between the elastic modulus of the material and temperature can be obtained through interpolation or fitting methods, and then the corrected elastic modulus can be calculated based on the current temperature value. The dynamic parameters of the bolt include the bolt length and material density, etc. These parameters describe the geometric characteristics and material properties of the bolt and have an important impact on the vibration characteristics and loosening displacement of the bolt. The bolt length affects the vibration frequency and mode of the bolt, while the material density is related to the mass and inertia of the bolt. For example, a longer bolt may generate more modes during vibration, and its vibration characteristics are different from those of a short bolt; while a bolt with a larger material density has a greater mass under the same conditions, and its vibration response will also be different. Therefore, these dynamic parameters need to be considered when calculating the loosening displacement. The bolt cross-sectional area determines the load-bearing capacity and stiffness of the bolt. When calculating the loosening displacement, the bolt cross-sectional area is an important parameter, which acts together with the elastic modulus of the bolt to affect the deformation of the bolt under force. For example, a bolt with a larger cross-sectional area produces less deformation under the same stress, while a bolt with a smaller cross-sectional area is prone to greater deformation. By considering the bolt cross-sectional area, the loosening displacement of the bolt can be calculated more accurately. By integrating parameters such as the frequency offset, corrected elastic modulus, bolt dynamic parameters, and bolt cross-sectional area, through a certain mechanical model and calculation formula, the loosening displacement of the bolt can be calculated. This loosening displacement can intuitively reflect the degree of bolt loosening and provide a basis for bolt condition assessment and maintenance decision-making. For example, by establishing a mechanical relationship model between bolt vibration and loosening displacement, substituting the above parameters into the model for calculation, the loosening displacement of the bolt in the current state can be obtained. If the loosening displacement exceeds the specified threshold, maintenance measures such as tightening or replacing the bolt need to be taken.

[0035] The Gaussian fitting method is used to determine the actual resonance peak frequency based on the bolt design frequency, which can accurately extract the actual resonance frequency of the bolt from complex vibration signals. The Gaussian fitting method is an effective data processing technique that can determine the peak position by fitting a Gaussian curve according to the distribution characteristics of the data, thereby improving the determination accuracy of the resonance peak frequency. Calculating the frequency offset based on the actual resonance peak frequency and the bolt design frequency can quantify the frequency change of the bolt during use. The frequency offset is an important indicator reflecting the degree of bolt loosening because bolt loosening will cause a change in its natural frequency. By calculating the frequency offset, the loosening trend of the bolt can be intuitively understood. Correcting the elastic modulus according to the current temperature value fully considers the influence of temperature on the material properties of the bolt. The elastic modulus is an important mechanical property index of the material, which will change with the change of temperature. In actual engineering, the ambient temperature of the bolt may change. If the influence of temperature on the elastic modulus is not considered, the calculation result of the loosening displacement will be inaccurate. Calculating the loosening displacement based on the frequency offset, the corrected elastic modulus, the bolt dynamic parameters, and the bolt cross-sectional area comprehensively considers multiple factors affecting bolt loosening. The bolt dynamic parameters (including bolt length and material density) and the bolt cross-sectional area reflect the structural and material characteristics of the bolt itself, the frequency offset reflects the loosening state of the bolt, and the corrected elastic modulus considers the influence of environmental factors on the material properties. By comprehensively considering these factors, the loosening displacement of the bolt can be calculated more accurately.

[0036] Optionally, determining the damage index of the target area according to the corrosion area ratio, the loosening displacement, and the environmental corrosion factor, determining the maintenance mode according to the damage index, and controlling the main robotic arm and the auxiliary robotic arm to perform maintenance on the target area according to the maintenance mode includes: Performing a weighted sum on the corrosion area ratio, the loosening displacement, and the environmental corrosion factor to obtain the damage index. When the damage index is less than the first preset threshold, determining the maintenance mode as the minor damage repair mode, controlling the main robotic arm to remove the surface corrosion products of the target area, and controlling the auxiliary robotic arm to spray an anti-corrosion coating on the target area; When the damage index is greater than or equal to the first preset threshold and less than the second preset threshold, determining the maintenance mode as the moderate damage repair mode, controlling the main robotic arm to perform grinding treatment on the target area to remove the corrosion layer, and controlling the auxiliary robotic arm to perform filling repair and spray an anti-corrosion coating on the ground target area; When the damage index is greater than or equal to the second preset threshold, determining the maintenance mode as the severe damage replacement mode, controlling the main robotic arm and the auxiliary robotic arm to work together to remove the damaged components in the target area and install new components.

[0037] Corrosion area ratio: It reflects the degree of corrosion of the target area, that is, the ratio of the corrosion area to the total area of the target area. The larger this ratio, the more serious the corrosion situation. Loose displacement amount: It reflects the degree of looseness of connecting components such as bolts in the target area. The larger the loose displacement amount, the worse the stability of the connecting components and the greater the impact on the overall performance of the structure. Environmental corrosion factor: It considers the influence of the environment where the target area is located on corrosion. Different environmental factors (such as humidity, temperature, concentration of corrosive gases, etc.) will accelerate or slow down the corrosion rate. Different weights are assigned according to the importance of each factor's influence on damage, and then they are weighted and summed to obtain the damage index. This method can comprehensively consider multiple factors and more accurately evaluate the damage degree of the target area. When the damage index is less than the first preset threshold, it indicates that the damage degree of the target area is relatively light, and only surface treatment and simple anti-corrosion measures are needed to restore its performance. Control the main robotic arm to remove the surface corrosion products in the target area. Removing the surface corrosion layer can prevent further corrosion expansion; control the auxiliary robotic arm to spray an anti-corrosion coating on the target area. The anti-corrosion coating can isolate the environment from the metal surface, play a protective role, and extend the service life of the target area. When the damage index is greater than or equal to the first preset threshold and less than the second preset threshold, the damage degree of the target area is relatively serious, and the surface corrosion has developed to a certain extent, and more in-depth repair treatment is needed. Control the main robotic arm to grind the target area to remove the corrosion layer. Grinding can completely remove the corroded part and make the metal surface flat; control the auxiliary robotic arm to fill and repair the ground target area and spray an anti-corrosion coating. Filling and repairing can make up for the material loss caused by corrosion, and the anti-corrosion coating further enhances the protection effect. When the damage index is greater than or equal to the second preset threshold, it indicates that the damage to the target area is very serious and its performance cannot be restored through repair, and the damaged components need to be replaced. Control the main robotic arm and the auxiliary robotic arm to work together to remove the damaged components in the target area and install new components. Working together can improve work efficiency and ensure the smooth progress of the replacement process.

[0038] The damage index is obtained by weighted summation of the corrosion area ratio, the loosening displacement amount, and the environmental corrosion factor, which can comprehensively consider various damage factors in the target area and evaluate its damage degree comprehensively and accurately. The corrosion area ratio reflects the degree of corrosion coverage on the surface of the target area, the loosening displacement amount reflects the loosening condition of connecting components such as bolts, and the environmental corrosion factor considers the accelerating or inhibiting effect of the external environment on the corrosion of the target area. Through weighted summation, these factors are integrated into a single damage index, which is convenient for subsequent analysis and decision-making. The quantification of the damage index enables the damage state of the target area to be represented by a specific numerical value, which is convenient for comparison and classification. Different ranges of the damage index correspond to different degrees of damage, providing a clear basis for determining the subsequent maintenance mode. Determining the corresponding maintenance mode according to different ranges of the damage index can provide a targeted maintenance strategy for the target area. Different degrees of damage require different maintenance methods. Minor damage may only require surface treatment and anti-corrosion coating spraying, moderate damage requires grinding, filling, and coating spraying, while severe damage requires replacing damaged components. By reasonably determining the maintenance mode, the maintenance efficiency and quality can be improved, and the maintenance cost can be reduced. Different maintenance modes require different allocation of maintenance resources and time. By determining the maintenance mode according to the damage index, maintenance resources can be reasonably allocated to avoid waste and shortage of resources. For minor damage, less manpower and time can be arranged for maintenance; for severe damage, more resources and time need to be concentrated for processing. Controlling the main robotic arm and the auxiliary robotic arm to perform maintenance on the target area according to the maintenance mode realizes the automation of the maintenance operation. The robotic arm can accurately complete various maintenance tasks according to the preset programs and instructions, such as removing surface corrosion products, grinding treatment, filling repair, anti-corrosion coating spraying, and component removal and installation. The automated maintenance operation improves the accuracy and consistency of maintenance and reduces the influence of human factors. The collaborative operation of the robotic arms can greatly improve the maintenance efficiency and shorten the maintenance time. The main robotic arm and the auxiliary robotic arm can perform different operations simultaneously. For example, in the moderate damage repair mode, the main robotic arm performs grinding treatment while the auxiliary robotic arm simultaneously prepares for filling repair, thus saving maintenance time. In addition, the operation of the robotic arm can prevent maintenance personnel from directly contacting the dangerous environment and improve the safety of maintenance.

[0039] Optionally, controlling the main robotic arm to perform grinding treatment on the target area to remove the corrosion layer and controlling the auxiliary robotic arm to perform filling repair and anti-corrosion coating spraying on the ground target area includes: According to the force information feedback by the six-axis force sensor, the grinding force of the main robotic arm is adjusted in real time, and the grinding progress and surface quality are monitored in real time in combination with the vision system; Adjust the welding parameters of the auxiliary robotic arm according to the shape and depth of the damaged area. The welding parameters include welding current, welding time, and welding pressure, and determine the spraying rate and flow rate of the coating according to the distance between the auxiliary robotic arm and the target area.

[0040] The six-axis force sensor can real-time sense the force information between the main robotic arm and the target area during the grinding process, including the magnitude and direction of the force, etc. By analyzing this force information, the control system can real-time adjust the grinding force of the main robotic arm. For example, when it is detected that the grinding force is too large, the system will automatically reduce the output power of the robotic arm to reduce the grinding force and avoid over-damaging the target area; conversely, when the grinding force is insufficient, the system will appropriately increase the power to ensure that the corrosion layer can be effectively removed. The vision system can real-time capture the image information of the target area. By analyzing and processing the images, it can accurately judge the progress and surface quality of the grinding. For example, the system can identify the areas that have not been ground yet, and whether the surface is flat and whether there are residual corrosion products after grinding. According to this information, the operator or the automatic control system can timely adjust the movement trajectory and grinding parameters of the main robotic arm to ensure the quality and efficiency of the grinding work. Different damaged areas have different shapes and depths, which will have an important impact on the effect of filling and repairing. Therefore, it is necessary to adjust the welding parameters of the auxiliary robotic arm according to the specific conditions of the damaged area, including welding current, welding time, and welding pressure. For example, for deeper damaged areas, it may be necessary to increase the welding current and extend the welding time to ensure that the filling material can fully fill the damaged part; for areas with complex shapes, it is necessary to precisely control the welding pressure to ensure good bonding between the filling material and the surrounding materials. The spraying rate and flow rate of the coating will directly affect the quality and uniformity of the anti-corrosion coating. The distance between the auxiliary robotic arm and the target area is a key factor. Different distances will result in different spraying effects of the coating material. When the auxiliary robotic arm is close to the target area, the spraying rate and flow rate can be appropriately reduced to avoid too thick or uneven coating; when the distance is far, it is necessary to appropriately increase the spraying rate and flow rate to ensure that the coating can cover all parts of the target area.

[0041] Adjust the grinding force of the main robotic arm in real time according to the force information feedback by the six-axis force sensor, which can ensure the precise control of the force during the grinding process. Different corrosion layer thicknesses and materials may require different grinding forces. Through the real-time feedback of the force sensor, the main robotic arm can dynamically adjust the force according to the actual situation, avoiding over-grinding or under-grinding. This precise force control helps to improve the grinding quality and ensure that the surface flatness and roughness of the target area meet the requirements. Combining with the vision system to monitor the grinding progress and surface quality in real time, the operator can timely understand the progress of the grinding work. The vision system can clearly capture the images of the target area, and by analyzing the images, it can judge whether the grinding is completed, whether there are defects on the surface, etc. This real-time monitoring helps to improve the grinding efficiency and quality and reduce the workload and error of manual inspection. Adjust the welding parameters of the auxiliary robotic arm according to the shape and depth of the damaged area, including welding current, welding time, and welding pressure, which can achieve personalized filling repair. Different damage conditions require different welding parameters to ensure good bonding between the filling material and the target area. By adjusting the welding parameters personalized, the quality of the filling repair can be improved, and the structural strength of the target area can be enhanced. Determine the spraying rate and flow rate of the coating according to the distance between the auxiliary robotic arm and the target area, which can ensure that the anti-corrosion coating is evenly and accurately sprayed on the target area. Different distances will affect the spraying effect of the coating. If the distance is too close, the coating may be too thick and flow; if the distance is too far, the coating may be too thin and incomplete coverage.

[0042] This embodiment also discloses a control system for an integrated inspection and maintenance robot for a sound barrier. Figure 2 It is a schematic diagram of the modules of the control system for an integrated inspection and maintenance robot for a sound barrier disclosed in an embodiment of the present application, as Figure 2 shown. The system includes a model module 201, an inspection module 202, a calculation module 203, and a maintenance module 204, where: The model module 201 is configured to obtain the structural feature data and material property data of the sound barrier, and construct a three-dimensional digital twin model according to the structural feature data and the material property data; The inspection module 202 is configured to generate an initial inspection path through a preset algorithm, construct a dynamic obstacle avoidance strategy through an infrared ranging sensor, and correct the path planning parameters in real time to obtain the current inspection path. Based on the current inspection path, it collects images, bolt resonance frequencies, and environmental parameters of each target area of the sound barrier. The environmental parameters include temperature, humidity, and salt spray concentration; The calculation module 203 is configured to determine the corrosion target area according to the image, determine the corrosion area ratio according to the corrosion target area and the three-dimensional digital twin model, determine the loosening displacement amount according to the bolt resonance frequency and the three-dimensional digital twin model, and determine the environmental corrosion factor according to the environmental parameters; The maintenance module 204 is configured to determine the damage index of the target area according to the corrosion area ratio, the loosening displacement amount, and the environmental corrosion factor, determine the maintenance mode according to the damage index, and control the main robotic arm and the auxiliary robotic arm to perform maintenance on the target area according to the maintenance mode.

[0043] Optionally, the model module 201 is configured to: Based on the lidar point cloud data, the multi-spectral polarization image, and the building information modeling design parameters in the structural feature data, segment each component of the sound barrier, and determine the structural information of the sound barrier according to the structural data and connection relationship of each component; Based on the bolt elastic modulus, the coating thickness, the porosity of the sound-absorbing material, and the preset material degradation model in the material property data, determine the material property change information of each component of the sound barrier under different environmental conditions; Obtain the accurate spatial position information of the sound barrier, and generate a continuous surface model of the sound barrier through the Poisson surface reconstruction algorithm; Integrate the structural information, the material property change information, and the continuous surface model to construct a three-dimensional digital twin model.

[0044] Optionally, the inspection module 202 is configured to: When moving along the initial inspection path and the infrared ranging sensor detects an obstacle that requires path adjustment, generate multiple potential paths, and determine the estimated energy consumption of each potential path according to the current motion model, the current load condition, and the energy consumption characteristics of the terrain of the potential path; Select the target potential path with the minimum estimated energy consumption, obtain the actual energy consumption, and compare it with the estimated energy consumption. When the difference between the actual energy consumption and the estimated energy consumption is greater than the threshold, determine the influencing factors, and correct the target potential path according to the influencing factors.

[0045] Optionally, the calculation module 203 is configured to: Extract the contour of the corroded target area, and map the image pixel coordinates of the contour to the three-dimensional digital twin model; Obtain the radius of curvature of the corroded target area from the three-dimensional digital twin model, calculate the corrosion depth using the terahertz reflection time difference, and determine the actual corrosion area according to the radius of curvature and the corrosion depth; Determine the corrosion area ratio according to the actual corrosion area and the projected area of the corroded target area.

[0046] Optionally, the calculation module 203 is configured to: The actual resonant peak frequency is determined by using the Gaussian fitting method based on the bolt design frequency, and the frequency offset is calculated according to the actual resonant peak frequency and the bolt design frequency; The elastic modulus is corrected according to the current temperature value, and the loosening displacement is calculated according to the frequency offset, the corrected elastic modulus, the bolt dynamic parameters and the bolt cross-sectional area, where the bolt dynamic parameters include the bolt length and the material density.

[0047] Optionally, the maintenance module 204 is configured to: Perform a weighted sum of the corrosion area ratio, the loosening displacement and the environmental corrosion factor to obtain a damage index, When the damage index is less than the first preset threshold, determine the maintenance mode as the minor damage repair mode, control the main robotic arm to remove the surface corrosion products in the target area, and control the secondary robotic arm to spray an anti-corrosion coating on the target area; When the damage index is greater than or equal to the first preset threshold and less than the second preset threshold, determine the maintenance mode as the moderate damage repair mode, control the main robotic arm to grind the target area to remove the corrosion layer, and control the secondary robotic arm to perform filling repair and spray an anti-corrosion coating on the ground target area; When the damage index is greater than or equal to the second preset threshold, determine the maintenance mode as the severe damage replacement mode, control the main robotic arm and the secondary robotic arm to work together to remove the damaged components in the target area and install new components.

[0048] Optionally, the maintenance module 204 is configured to: Real-time adjust the grinding force of the main robotic arm according to the force information fed back by the six-axis force sensor, and combine the vision system to monitor the grinding progress and surface quality in real time; Adjust the welding parameters of the secondary robotic arm according to the shape and depth of the damaged area, where the welding parameters include the welding current, the welding time and the welding pressure, and determine the spraying rate and flow rate of the coating according to the distance between the secondary robotic arm and the target area.

[0049] It should be noted that: when the device provided in the above embodiment realizes its functions, only the above-mentioned division of each functional module is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0050] This embodiment also discloses an electronic device, refer to Figure 3, the electronic device may include: at least one processor 301, at least one communication bus 302, a user interface 303, a network interface 304, and at least one memory 305.

[0051] Among them, the communication bus 302 is used to realize the connection and communication between these components.

[0052] Among them, the user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may further include a standard wired interface and a wireless interface.

[0053] Among them, the network interface 304 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0054] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts within the entire server. By running or executing instructions, programs, code sets, or instruction sets stored in the memory, and by calling data stored in the memory, it executes various functions of the server and processes data. Optionally, the processor 301 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 301 may integrate one or a combination of several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs, etc.; the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 301 and may be implemented separately by a single chip.

[0055] Among them, the memory 305 may include a Random Access Memory (RAM), or may include a Read-Only Memory. Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area can store the data involved in the above-mentioned method embodiments. Optionally, the memory 305 may also be at least one storage device located far from the aforementioned processor 301. As Figure 3 shown, in a memory as a computer storage medium, it may include an operating system, a network communication module, a user interface module, and an application program for the control method of the integrated inspection and maintenance robot for the sound barrier.

[0056] In Figure 3 the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user to obtain the data input by the user; and the processor 301 can be used to call the application program for the control method of the integrated inspection and maintenance robot for the sound barrier stored in the memory. When executed by one or more processors 301, the electronic device is caused to execute the method as described in one or more of the above embodiments.

[0057] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0058] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0059] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0060] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0061] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0062] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present application. And the aforementioned memory includes: various media such as USB flash drives, mobile hard disks, magnetic disks or optical discs that can store program codes.

[0063] The above are only exemplary embodiments of the present disclosure and should not be used to limit the scope of the present disclosure. That is, all equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. Those skilled in the art will easily think of other implementation schemes of the present disclosure after considering the disclosure of the specification. The present application aims to cover any variations, uses or adaptive changes of the present disclosure. These variations, uses or adaptive changes follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A control method for an integrated inspection and maintenance robot of a sound barrier, characterized in that, Applied to the integrated inspection and maintenance robot, the method includes: Obtain the structural feature data and material property data of the sound barrier, and construct a three-dimensional digital twin model according to the structural feature data and the material property data; Generate an initial inspection path through a preset algorithm, construct a dynamic obstacle avoidance strategy through an infrared ranging sensor, and correct the path planning parameters in real time to obtain the current inspection path. Based on the current inspection path, collect images, bolt resonance frequencies, and environmental parameters of each target area of the sound barrier. The environmental parameters include temperature, humidity, and salt fog concentration; Determine the corrosion target area according to the image, determine the corrosion area ratio according to the corrosion target area and the three-dimensional digital twin model, determine the loosening displacement according to the bolt resonance frequency and the three-dimensional digital twin model, and determine the environmental corrosion factor according to the environmental parameters; Determine the damage index of the target area according to the corrosion area ratio, loosening displacement, and environmental corrosion factor, determine the maintenance mode according to the damage index, and control the main robotic arm and the auxiliary robotic arm to repair the target area according to the maintenance mode.

2. The control method of the integrated inspection and maintenance robot for the sound barrier according to claim 1, wherein, The constructing a three-dimensional digital twin model according to the structural feature data and the material property data includes: Based on the lidar point cloud data, multi-spectral polarization image, and building information modeling design parameters in the structural feature data, segment each component of the sound barrier, and determine the structural information of the sound barrier according to the structural data and connection relationship of each component; Based on the bolt elastic modulus, coating thickness, and sound absorption material porosity in the material property data and a preset material degradation model, determine the material property change information of each component of the sound barrier under different environmental conditions; Obtain the accurate spatial position information of the sound barrier, and generate a continuous surface model of the sound barrier through the Poisson surface reconstruction algorithm; Integrate the structural information, the material property change information, and the continuous surface model to construct a three-dimensional digital twin model.

3. The control method of the integrated inspection and maintenance robot for the sound barrier according to claim 1, characterized in that, The constructing a dynamic obstacle avoidance strategy through an infrared ranging sensor and correcting the path planning parameters in real time to obtain the current inspection path includes: When moving along the initial inspection path and the infrared ranging sensor detects an obstacle and needs to adjust the path, generate multiple potential paths, and determine the estimated energy consumption of each potential path according to the current motion model, the current load condition, and the energy consumption characteristics of the terrain of the potential path; Select the target potential path with the minimum estimated energy consumption, obtain the actual energy consumption, and compare it with the estimated energy consumption. When the difference between the actual energy consumption and the estimated energy consumption is greater than the threshold, determine the influencing factor, and correct the target potential path according to the influencing factor.

4. The control method of the integrated inspection and maintenance robot for the sound barrier according to claim 1, characterized in that, The determining the corrosion area ratio according to the corrosion target area and the three-dimensional digital twin model includes: Extract the contour of the corrosion target area, and map the image pixel coordinates of the contour to the three-dimensional digital twin model; Obtain the radius of curvature of the corrosion target area from the three-dimensional digital twin model, calculate the corrosion depth using terahertz reflection time difference, and determine the actual corrosion area according to the radius of curvature and the corrosion depth; Determine the corrosion area ratio based on the actual corrosion area and the projected area of the corrosion target area.

5. The control method of the integrated inspection and maintenance robot for the sound barrier according to claim 1, characterized in that, The determining the loosening displacement amount according to the bolt resonance frequency and the three-dimensional digital twin model includes: Determine the actual resonance peak frequency by using the Gaussian fitting method based on the bolt design frequency, and calculate the frequency offset according to the actual resonance peak frequency and the bolt design frequency; Correct the elastic modulus according to the current temperature value, and calculate the loosening displacement amount according to the frequency offset, the corrected elastic modulus, the bolt dynamic parameters, and the bolt cross-sectional area, where the bolt dynamic parameters include the bolt length and the material density.

6. The control method of the integrated inspection and maintenance robot according to claim 1, characterized in that The determining the damage index of the target area according to the corrosion area ratio, the loosening displacement amount, and the environmental corrosion factor, determining the maintenance mode according to the damage index, and controlling the main robotic arm and the auxiliary robotic arm to perform maintenance on the target area according to the maintenance mode includes: Perform a weighted sum of the corrosion area ratio, the loosening displacement amount, and the environmental corrosion factor to obtain the damage index. When the damage index is less than the first preset threshold, determine the maintenance mode as the minor damage repair mode, control the main robotic arm to remove the surface corrosion products of the target area, and control the auxiliary robotic arm to spray an anti-corrosion coating on the target area; When the damage index is greater than or equal to the first preset threshold and less than the second preset threshold, determine the maintenance mode as the moderate damage repair mode, control the main robotic arm to perform grinding treatment on the target area to remove the corrosion layer, and control the auxiliary robotic arm to perform filling repair and spray an anti-corrosion coating on the ground target area; When the damage index is greater than or equal to the second preset threshold, determine the maintenance mode as the severe damage replacement mode, control the main robotic arm and the auxiliary robotic arm to work together to remove the damaged components of the target area and install new components.

7. The control method of the integrated inspection and maintenance robot according to claim 6, characterized in that, The controlling the main robotic arm to perform grinding treatment on the target area to remove the corrosion layer, and controlling the auxiliary robotic arm to perform filling repair and spray an anti-corrosion coating on the ground target area includes: Adjust the grinding force of the main robotic arm in real time according to the force information fed back by the six-axis force sensor, and combine the vision system to monitor the grinding progress and surface quality in real time; Adjust the welding parameters of the auxiliary robotic arm according to the shape and depth of the damaged area, where the welding parameters include welding current, welding time, and welding pressure, and determine the spraying rate and flow rate of the coating according to the distance between the auxiliary robotic arm and the target area.

8. A control system for an integrated inspection and maintenance robot of a sound barrier, characterized in that, Includes a model module, an inspection module, a calculation module, and a maintenance module, where: The model module is configured to obtain the structural feature data and material property data of the sound barrier, and construct a three-dimensional digital twin model according to the structural feature data and the material property data; The inspection module is configured to generate an initial inspection path through a preset algorithm, construct a dynamic obstacle avoidance strategy through an infrared distance sensor, and correct the path planning parameters in real time to obtain the current inspection path, and collect images, bolt resonance frequencies, and environmental parameters of each target area of the sound barrier based on the current inspection path, where the environmental parameters include temperature, humidity, and salt fog concentration; A calculation module, configured to determine a corrosion target area according to the image, determine a corrosion area ratio according to the corrosion target area and the three-dimensional digital twin model, determine a loosening displacement amount according to the bolt resonance frequency and the three-dimensional digital twin model, and determine an environmental corrosion factor according to the environmental parameters; A maintenance module, configured to determine a damage index of the target area according to the corrosion area ratio, the loosening displacement amount, and the environmental corrosion factor, determine a maintenance mode according to the damage index, and control a main robotic arm and a secondary robotic arm to perform maintenance on the target area according to the maintenance mode.

9. An electronic device, characterized in that, It includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. Both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method according to any one of claims 1-7.

Images