A hybrid-driven underwater small-scale monitoring system based on deep learning

Abstract

The application discloses a kind of hybrid drive underwater small monitoring systems based on deep learning, the system is constituted by underwater robot and land computing platform, wherein underwater robot is small in size, flexible, has two movement modes of elastic start and propeller propulsion, can enter the underwater position that human is difficult to reach;Land computing platform uses deep learning network, collects sample picture of underwater concrete damage in situ, including three typical underwater concrete damages of crack, corrosion and exposed reinforcement, according to the size of maintenance rate formulated by maintenance party, algorithm can output different identification results in real time, finally quantitative score situation is given to structural damage degree by analytic hierarchy process, and early classification warning is made to the safety state of underwater structure.The application combines underwater robot (hybrid drive) with artificial intelligence (deep learning), and provides an innovative solution for underwater structural health monitoring task.

Description

Technical Field

[0001] The present invention belongs to the intersection of underwater robots and artificial intelligence, and specifically relates to a hybrid-driven small underwater monitoring system based on deep learning. Background Art

[0002] The 21st century marks a new era in which humanity has begun to significantly develop and utilize the ocean. The maturity of marine engineering technology determines a country's ability to develop and utilize marine resources. my country has approximately 3 million square kilometers of ocean area, and the ocean has begun to provide a significant driving force for its economic development. Marine engineering structures deteriorate over time. Without effective monitoring and assessment, the safety status of marine engineering structures during operation remains unknown, potentially resulting in damage to human life and property. In some engineering environments where direct human observation is inconvenient, the use of underwater robots has become an option.

[0003] With the development of underwater robotics, soft robots have become a growing area of ​​robotics research and a hot topic. Their excellent flexibility and adaptability hold great promise in fields such as underwater robotics. High-speed soft actuators based on combustion-induced transient actuation can generate significant driving force in a very short period of time, enabling high-speed robot motion. By collecting video footage and engineering photos, underwater robots can rapidly inspect underwater structures for corrosion and cracks, guiding pipeline construction and underwater bridge pier maintenance. The emergence of underwater robot monitoring technology has addressed several challenges in structural health monitoring.

[0004] With the popularization of machine learning, more and more deep learning algorithms have been introduced into the field of concrete damage identification. However, current technologies are still mainly based on image classification tasks represented by convolutional neural networks (CNNs), which have the following problems: 1. In order to ensure the accuracy of detection, the complexity of the model is relatively high, resulting in the algorithm model being too large and the calculation speed being slow, which is not conducive to practical application in engineering. 2. Existing detection methods often cannot achieve simultaneous detection while collecting data, and cannot perform real-time detection, which affects work efficiency. 3. The recognition function is mainly based on image classification, and lacks the function of target detection for a variety of damage types and multiple damage areas. In addition, after the identification is completed, the subsequent corresponding disaster warning is not in place, the warning is not accurate and rigorous enough, and the factors considered are not comprehensive enough. Summary of the Invention

[0005] The purpose of the present invention is to address the shortcomings of the existing technology and propose a hybrid-driven small underwater monitoring system based on deep learning.

[0006] In response to the needs of disaster prevention and mitigation in the field of marine engineering, the purpose of the present invention is achieved through the following technical solutions: The present invention proposes a hybrid-driven small underwater monitoring system based on deep learning, which is composed of an underwater robot and a land computing platform, wherein the underwater robot includes a transient speed drive module, a propeller propulsion module, a control cabin and a monitoring module; the propeller propulsion module is connected to the control cabin to realize propeller propulsion movement; the propeller propulsion module can be retracted into the underwater robot, at which time the transient speed drive module acts as a drive to realize ejection starting movement through an instantaneous energy release reaction; the monitoring module is used to collect underwater concrete damage images and transmit them to the land computing platform; the land computing platform recognizes and analyzes underwater concrete damage images based on a deep neural network, and the confidence threshold conf of the deep neural network threh It is obtained by the maintenance rate h set by the user. The specific formula is as follows:

[0007]

[0008] Among them, 0 <h≤1,检修率h越大,置信度阈值conf threh The smaller it is, the higher the recall rate of deep neural network prediction is.

[0009] Furthermore, the transient speed drive module is provided with a raw material replenishment port, a raw material storage unit, a raw material input port, an exergonic reaction excitation device, a reaction chamber and a flexible membrane;

[0010] The reaction raw materials are added to the raw material storage unit through the raw material replenishment port. When the transient speed drive module is working, the reaction raw materials in the raw material storage unit enter the reaction chamber through the raw material input port. The excitation device triggers the reaction raw materials in the reaction chamber through electric sparks to produce an instantaneous excitation reaction. The flexible membrane is arranged at the bottom of the reaction chamber, and the flexible membrane quickly deforms to produce an instantaneous driving force in the opposite direction for the robot.

[0011] Furthermore, the propeller propulsion module consists of four symmetrically installed propellers, a fairing and a brushless motor. The brushless motor is installed on the top of the propeller, and the two rotate coaxially. The fairing is installed on the outside of the propeller. The propeller propulsion module uses a microcomputer to distribute the duty cycle of the PWM wave of each propeller to achieve underwater vector propulsion of the robot.

[0012] Furthermore, a communication unit, a microcomputer and a power supply unit are provided in the control cabin, wherein the communication unit establishes a wireless data transmission connection with the land control end, and is used to receive control commands and send the robot's underwater sensor information and monitoring analysis results; the microcomputer serves as the main processor, and is used to process the control commands issued by the land computing platform and store video information; the power supply unit includes a power supply and a power management board, and is used to power the entire system.

[0013] Furthermore, the monitoring module is provided with a laser scanning device, an auxiliary light source device, a camera and a pan / tilt platform;

[0014] The laser scanning device acquires spatial point cloud data, couples it with the underwater high-definition images obtained by the camera, and establishes a three-dimensional visualization model; the auxiliary light source device automatically adjusts the light intensity according to the brightness of the underwater environment to improve the quality of underwater shooting images; the pan-tilt head is a supporting device for the fixed camera, and the collection results of the monitoring module are transmitted to the microcomputer through the communication unit and sent to the land computing platform for real-time monitoring of the underwater situation.

[0015] Furthermore, the land computing platform is mainly composed of underwater communication modules and high-performance computers. It uses deep neural network-based recognition and analysis, takes 5 frames of images per second, and performs target detection on three typical underwater concrete damages: cracks, corrosion, and exposed steel bars, and gives a corresponding confidence level for each prediction box.

[0016] Furthermore, the training process of a deep neural network includes the following steps:

[0017] (1) Data acquisition: photograph and prepare underwater concrete damage images, including three types of damage: cracks, corrosion, and exposed steel bars; and divide them into a data set and a test set;

[0018] (2) Labeling: Organize experts to calibrate the damaged area on the original image of the dataset. The calibration area is a rectangular box;

[0019] (3) Training the network: Input the data set into the neural network for training and set the learning rate;

[0020] (4) Test network: Input the test set into the trained network and calculate the prediction accuracy and recall rate.

[0021] (5) Improvement and optimization: Change the network structure, including adding attention, residual, and feature pyramid modules to verify the changes in model accuracy; adjust the composition of the training set, including deleting difficult examples and supplementing sample images for features that have not been learned; change the depth of the neural network, including testing the relationship between the computational accuracy and efficiency of networks of different depths; compare the recognition accuracy and recall rate data of three typical injuries under different conditions, and finally obtain the optimal model for application.

[0022] Furthermore, the prediction results of the damaged areas in the captured images are obtained through the 80×80×15 tensor output by the neural network, where 80×80 means that the image is evenly divided into 80×80 square grids, one grid corresponds to one prediction box, that is, a total of 6400 prediction boxes, and 15 means 5×3, which means three types of damage: cracks, corrosion, and exposed steel bars. Each type of damage is represented by 5 numbers, such as T iAs shown in the formula, i represents the category code, (x1, y1) represents the coordinates of the upper left corner of the prediction box, (x2, y2) represents the coordinates of the lower right corner of the prediction box, and conf i Represents the confidence probability that the prediction box belongs to category i;

[0023] T i ={x1,y1,x2,y2,conf i}(i=1,2 or 3)

[0024] After getting the output results, filter out the conf i >conf threh The predicted box is used as the final deep learning prediction result and marked in the original image to complete the recognition process.

[0025] Furthermore, the inspection rate is affected by human resources, maintenance importance, and weather conditions, and is determined by the funding maintenance party. It represents the maintenance demand of the project area. The larger the inspection rate, the more urgent the inspection and maintenance of the area, and it is suitable for emergency maintenance tasks. The smaller the inspection rate, the less maintenance demand of the area, and it is suitable for annual inspection tasks. threh The setting is best within the range of 0.74-0.85.

[0026] Furthermore, after locating the damaged area and retrieving relevant video clips, the hierarchical analysis method is used to determine the degree of damage to the concrete structure, and to provide early warning of the safety status of the underwater structure.

[0027] The beneficial effects of the present invention are as follows: the present invention proposes the concept of catapult start, which enriches the driving mode of the underwater monitoring robot; proposes an underwater concrete damage target detection algorithm, which is divided into three categories: cracks, corrosion and exposed steel bars, and quantitatively describes the severity of damage to underwater concrete; proposes the concept of "repair rate", and conducts focused and secondary repairs and inspections on different engineering areas, which can help the repair party select the areas that are most in need of and most suitable for repair, and significantly improve the work efficiency of large-scale engineering repair tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the overall structure of the underwater robot of the present invention;

[0029] Figure 2 This is a schematic diagram of the instantaneous speed drive module of the underwater robot of the present invention;

[0030] Figure 3 This is a schematic diagram of the driving principle of the underwater robot's instantaneous speed drive module of the present invention;

[0031] Figure 4 This is a schematic diagram of the ejection start of the underwater robot of the present invention;

[0032] Figure 5 This is a schematic diagram of the propeller propulsion module of the underwater robot of the present invention;

[0033] Figure 6 This is a schematic diagram of the control cabin of the underwater robot of the present invention;

[0034] Figure 7 This is a schematic diagram of the underwater robot monitoring module of the present invention.

[0035] Figure 8 This is a schematic diagram of the composition of the land computing platform of the present invention.

[0036] Figure 9 This is a sample diagram of the data set used for deep learning training in the present invention.

[0037] Figure 10 This is a structural diagram of the deep neural network of the present invention.

[0038] Figure 11 This is a flowchart of the target detection model established based on deep learning in the present invention.

[0039] Figure 12 Schematic diagram of the workflow of the present invention.

[0040] Figure 13 Schematic diagram of the application scenario of the present invention.

[0041] Figure 14 A diagram explaining the concept of "maintenance rate".

[0042] In the figure: transient speed drive module 1; propeller propulsion module 2; control cabin 3; monitoring module 4; raw material replenishment port 101; raw material storage unit 102; raw material input port 103; exergy reaction excitation device 104; reaction chamber 105; flexible membrane 106; propeller 201; fairing 202; brushless motor 203; communication unit 301; microcomputer 302; power supply unit 303; laser scanning device 401; auxiliary light source device 402; camera 403; pan / tilt head 404; underwater communication module 501; high-performance computer 502. DETAILED DESCRIPTION

[0043] The present invention will be further described below with reference to the accompanying drawings.

[0044] by Figure 1 The figure is used as an example to explain the appearance and structure of the underwater robot. It can be seen that the structure of the robot mainly includes a transient speed drive module 1, a propeller propulsion module 2, a control cabin 3 and a monitoring module 4.

[0045] by Figure 2 、 Figure 3The following example illustrates the principles of the underwater robot's transient speed drive module 1 and chemical reaction drive. The transient speed drive module 1 is equipped with a raw material supply port 101, a raw material storage unit 102, a raw material input port 103, an exergonic reaction stimulating device 104, a reaction chamber 105, and a flexible membrane 106. Reaction raw materials (i.e., oxygen and ethane) are added to the raw material storage unit 102 via the raw material supply port 101. During operation, the transient speed drive module 1 enters the reaction chamber 105 via the raw material input port 103. The exergonic reaction stimulating device 104 triggers the reaction raw materials in the reaction chamber 105 through an electric spark, producing an instantaneous exergonic reaction. The flexible membrane 106, made of silicone and located at the bottom of the reaction chamber 105, rapidly deforms, generating an instantaneous propulsion force in the opposite direction. The present invention uses oxygen and ethane as the reaction raw materials because gaseous fuels can easily adjust their mixing ratio to achieve different explosion effects. Using solid or liquid fuels makes it difficult to control the explosion intensity, thus affecting the launch. Soft robots using transient speed drive can usually achieve a higher driving speed in a very short time, which solves the problem of generally poor motion performance of soft robots.

[0046] by Figure 4 The following example illustrates the launch process of this underwater robot. The underwater robot field has long faced the problem of insufficient driving force, making it unable to properly complete motion requirements in special situations such as starting, sharp turns, and escape attempts. To address this issue, the underwater robot involved in this system addresses this issue in two ways. First, a chemical reaction drive module is introduced to provide the underwater robot with tremendous driving force within an extremely short timeframe. Second, to reduce fluid resistance, a cavity is designed. During the launch, the robot's four propellers are retracted into the main body of the robot, transforming it into a bullet-like shape. This significantly reduces the direct contact area with the water resistance, enabling an instant launch through a chemical exothermic reaction. Launching can be used not only for robot acceleration but also for situations such as entanglement in water plants, silt, and attacks by underwater creatures. The launch system improves the underwater robot's driving efficiency and enhances its adaptability to complex underwater conditions.

[0047] by Figure 5The following figure illustrates the propeller propulsion module 2 of the underwater robot. It consists of four symmetrically mounted propellers 201, a shroud 202, and a brushless motor 203. The brushless motor 203 is mounted on top of the propellers 201, rotating coaxially with the shroud 202. The shroud 202 is mounted on the outside of the propellers 201. The propeller propulsion module 2 receives control commands via a communication unit 301. A microcomputer 302 processes the control protocol and outputs PWM signals with varying duty cycles to control the direction and speed of the four propellers, thereby maneuvering the robot underwater.

[0048] by Figure 6 The control cabin 3 of the underwater robot is shown as an example. The control cabin 3 houses a communication unit 301, a microcomputer 302 serving as the main processor, and a power supply unit 303, which provides and distributes voltage for the entire system. Communication unit 301 establishes a wireless data transmission connection with the land-based control terminal, receiving control commands and transmitting information from the robot's underwater sensors and monitoring and analysis results. Microcomputer 302, serving as the main processor, processes control commands from the land-based computing platform 5 and stores video information. Power supply unit 303, including a power supply and power management board, supplies power to the entire system.

[0049] by Figure 7 The following illustrates the monitoring module 4 of the underwater robot. Monitoring module 4 is equipped with a laser scanner 401, an auxiliary light source 402, a camera 403, and a pan / tilt head 404. The laser scanner 401 acquires spatial point cloud data, which is coupled with the high-definition underwater images captured by the camera 403 to create a three-dimensional visualization model. The auxiliary light source 402 automatically adjusts the illumination level based on the brightness of the underwater environment, improving underwater image quality and making the camera's images clearer and easier to process. The pan / tilt head 404 serves as a support device for the fixed camera. Monitoring module 4 transmits the collected data to the microcomputer 302 via the communication unit 301, which then transmits it to a land-based computing platform for real-time monitoring of the underwater situation. After locating damaged areas and retrieving relevant video footage, the analytic hierarchy process (AHP) is used to determine the damage level and score the system, providing a graded early warning of the underwater structure's safety status.

[0050] by Figure 8 The following example illustrates the components of the underwater robot's land computing platform. The land computing platform 5 primarily consists of an underwater communication module 501 and a high-performance computer 502. The high-performance computer uses a deep neural network for recognition and analysis, capturing five frames of image per second. It detects three typical underwater concrete damage types: cracks, corrosion, and exposed rebar, and assigns a corresponding confidence score to each predicted frame.

[0051] by Figure 9The deep learning image dataset used by the system of the underwater robot is shown as an example for explanation. The deep learning image dataset used by the present application is a concrete damage image shot underwater, and the damage types include cracks, corrosion and exposed steel bars. When selecting the dataset pictures, special cases and difficult cases should be avoided, and damage pictures with typical characteristics should be selected to improve the accuracy of the deep learning model.

[0052] With Figure 10 The deep learning network structure used by the monitoring system is shown as an example for explanation. The neural network designed by the present application has an input of a 640x640x3 RGB color picture (using picture training does not affect subsequent video recognition), and after the picture is input into the network, it passes through n convolutional layers, 1 max pooling layer and 2 fully connected layers to finally obtain an output result, that is, a 80x80x15 tensor. Among them, 80x80 means that the picture is evenly divided into 80x80 square grids, and one grid corresponds to one prediction box, that is, a total of 6400 prediction boxes, and 15 is 5x3, that is, three damage types of cracks, corrosion and exposed steel bars, and each damage type is represented by 5 numbers, such as T i As shown in the formula, i represents the category code, (x1, y1) represents the coordinates of the upper left corner of the prediction box, (x2, y2) represents the coordinates of the lower right corner of the prediction box, and conf i represents the confidence probability that the prediction box belongs to the i-th category.

[0053] T i ={x1, y1, x2, y2, conf i}(i=1, 2 or 3)

[0054] After obtaining the output result, the conf i >conf threhThe prediction box is used as the final deep learning prediction result and marked in the original image, thus completing the recognition process. It is worth noting that the deep learning model structure used by the system of the present invention is relatively simple. This is because the underwater robot is small in size and has limited computing power. In order to ensure the real-time recognition speed of the video, a deep learning model with a relatively simple structure is selected. However, this does not affect the realization of the monitoring effect of cracks, corrosion and exposed steel damage targets. The present invention introduces the "repair rate" h(0 <h≤1),代表工程区域在下一段时间内适合维修的程度,出资维护方即责任方根据天气、资金、人力和检查性质四个方面综合考量,在0-1之间给出检修率h的值。检修率受人力资源、维护重要程度、天气条件等因素的影响,检修率h的值与深度学习算法中的置信度阈值一一对应,h越大,置信度阈值越小,即更多可能是损伤区域的预测框将被深度学习网络输出,从而提高预警等级。h越小,置信度阈值越大,较少可能是损伤区域的预测框将被深度学习网络输出,从而降低预警等级。检修率越大,代表此区域急需检查维修,适合紧急维修等任务;检修率小,代表此区域维护需求不大,适用于年度检修等任务。根据已有经验结合专家协助,可以知道当检修率h接近0或1时,置信度阈值的变化应当是比较剧烈的。当检修率h从0向1增大时,置信度阈值将以指数函数的变化规律减小,从而使更多的可能损伤进入预警系统中,另外考虑到置信度阈值大致的可信度范围在0.74-0.85之间,所以对指数项进行适当的衰减之后加入公式,最后得到的检修率h与conf threh The functional relationship is:

[0055]

[0056] The funding and maintenance party can determine the maintenance quality of the underwater engineering structure according to the actual situation. If the situation is urgent or the maintenance personnel are free to carry out a comprehensive and thorough repair, more prediction frames will be output; if it is just routine maintenance, only fewer prediction frames need to be output. In the actual experiment, threh The upper limit of 0.85 is set moderately, resulting in fewer missed damaged areas and sufficient for routine inspections. The introduction of a maintenance rate changes the previous approach of fixed confidence thresholds, focusing on prioritized and secondary maintenance and troubleshooting of different engineering areas. This significantly improves efficiency for large-scale engineering maintenance tasks and allows for the selection of repair technicians for the areas most in need and most suitable for maintenance.

[0057] by Figure 11The following example illustrates the steps for establishing a deep learning model for this system. This small monitoring system targets underwater concrete damage and uses a deep neural network for training. The training process includes the following steps:

[0058] (1) Data acquisition: Capture and prepare a large number of underwater concrete damage images, including three types of damage: cracks, corrosion, and exposed steel bars.

[0059] (2) Labeling: Organize experts to calibrate the damaged areas on the original images.

[0060] (3) Training the network: Input the prepared data set into the deep neural network for training, set the learning rate, and observe the degree of fitting.

[0061] (4) Test network: Input the test set into the trained network and calculate the prediction accuracy and recall rate.

[0062] (5) Improvement and optimization: Change the network structure, including adding attention, residual, and feature pyramid modules to verify the changes in model accuracy; adjust the composition of the training set, including deleting difficult examples and supplementing sample images for features that have not been learned; change the depth of the neural network, including testing the relationship between the computational accuracy and efficiency of networks of different depths; compare the recognition accuracy and recall rate data of three typical injuries under different conditions, and finally obtain the optimal model for application.

[0063] by Figure 12 、 Figure 13 The following example illustrates the overall structure and application scenarios of the present system. Specifically, the present invention relates to a hybrid-driven underwater small-scale monitoring system based on deep learning, which consists of an underwater robot and a land-based computing platform. The underwater robot is responsible for carrying a camera to shoot underwater videos of concrete structures and transmit them to the land-based computing platform. The land-based computing platform uses a deep learning algorithm to identify and locate damaged areas. After retrieving relevant video clips, it uses the hierarchical analysis method to discriminate and score the degree of damage to the system, and provide early graded warnings for the safety status of underwater structures.

[0064] by Figure 14 The following example illustrates the concept of "maintenance rate" in this system. The maintenance rate represents the degree to which the project area is suitable for maintenance in the next period of time. The maintenance funder, i.e., the responsible party, gives the maintenance rate h value they believe in between 0 and 1 based on comprehensive considerations of weather, funds, manpower, and the nature of the inspection. Figure 14As shown in , the more favorable the weather for maintenance work, the more available funding and manpower, and the more urgent the maintenance work, the higher the maintenance rate h should be. The maintenance rate h corresponds to the confidence threshold in the deep learning algorithm. A larger h corresponds to a smaller confidence threshold, meaning that more prediction boxes likely to indicate damaged areas will be output by the deep learning network, thereby improving the warning level. The introduction of a maintenance rate changes the previous approach of using a fixed confidence threshold and instead implements focused and prioritized maintenance and troubleshooting in different engineering areas. For large-scale engineering maintenance tasks, this significantly improves work efficiency and allows for the selection of repair technicians for the areas most in need and most suitable for repair.

[0065] Currently developed hybrid drive methods often lack the ability to quickly generate high acceleration. However, this invention leverages the rapid and intense nature of chemical exothermic reactions. The massive thrust generated by the soft actuator allows the robot to jump in milliseconds, achieving high acceleration in a fraction of a second.

[0066] Bridges, dams, buildings, television towers, pipelines, wind farms, and highway tunnels can all be subject to extreme environmental conditions. Strong winds, heavy rain, high humidity, large temperature fluctuations, or catastrophic events such as earthquakes, hurricanes, and floods can severely damage the health of these structures and potentially cause life-threatening events such as collapse. Offshore structures are also subject to the impact of extreme weather conditions such as typhoons. Therefore, monitoring the health indicators of offshore structures and evaluating the resulting data will ensure the safe operation of offshore structures, reduce maintenance costs, and provide more room for the development of structural novelties in offshore structures.

[0067] In structural health monitoring of marine engineering, current methods for damage identification are primarily categorized into ultrasonic testing, piezoelectric ceramics, and computer vision. In many countries with a long history of urbanization, there is a growing need for automated inspections based on computer vision to replace conventional, labor-intensive visual inspections. Damage identification in complex underwater environments is a recognized challenge worldwide, and computer vision methods offer unique advantages in detecting damage under complex conditions. Some concrete structures, such as bridge piers and the bottom of drilling platforms, are difficult for humans to reach. Traditional manual damage detection methods are inefficient, costly, and dangerous. Computer vision methods offer high accuracy, speed, and automation. In recent years, powerful deep learning methods have rapidly developed and become a mainstream research direction in computer vision, providing a new path for damage image recognition.

[0068] The present invention belongs to the intersection of underwater robots and artificial intelligence, and specifically relates to a hybrid-driven underwater small-scale monitoring system based on deep learning. The system consists of an underwater robot and a land computing platform. The underwater robot is small in size, flexible in operation, and can enter underwater locations that are difficult for humans to reach. The land computing platform uses a self-designed deep learning algorithm to collect sample images of underwater concrete damage on the spot, covering three typical underwater concrete damages: cracks, corrosion, and exposed steel bars. Through training and optimization, the most suitable deep learning model is obtained. The robot transmits the underwater scene to a high-performance computer. The algorithm can output the recognition results in real time. Finally, a hierarchical analysis method is used to give a quantitative score for the degree of structural damage, and an early graded warning is made for the safety status of the underwater structure. The present invention combines an underwater robot (hybrid drive) with artificial intelligence (deep learning) to establish a hybrid-driven underwater small-scale monitoring system based on deep learning, providing an innovative solution for underwater structural health monitoring tasks.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hybrid drive underwater small monitoring system based on deep learning, characterized by: The system is composed of an underwater robot and a land computing platform, wherein the underwater robot includes a transient speed drive module (1), a propeller propulsion module (2), a control cabin (3) and a monitoring module (4); the propeller propulsion module (2) is connected to the control cabin (3) to realize propeller propulsion movement; the propeller propulsion module (2) can be retracted into the interior of the underwater robot, at which time the transient speed drive module (1) acts as a drive to realize ejection start movement through instantaneous energy release reaction; the monitoring module (4) is used to collect underwater concrete damage images and transmit them to the land computing platform (5); the land computing platform (5) recognizes and analyzes the underwater concrete damage images based on a deep neural network, and the confidence threshold conf of the deep neural network is conf. threh It is obtained by the maintenance rate h set by the user. The specific formula is as follows: where \(0 < h\leqslant1\), the larger the maintenance rate \(h\) is, the smaller the confidence threshold \(conf\) threh is, that is, the higher the recall rate predicted by the deep neural network is; The prediction result of the damaged area in the collected image is obtained by the 80×80×15 tensor output by the neural network, where 80×80 means that the image is evenly divided into 80×80 square grids, one grid corresponds to one prediction box, that is, a total of 6400 prediction boxes, and 15 means 5×3, which means three types of damage: cracks, corrosion, and exposed steel bars. Each type of damage is represented by 5 numbers, such as T i As shown in the formula, i represents the category code, (x1, y1) represents the coordinates of the upper left corner of the prediction box, (x2, y2) represents the coordinates of the lower right corner of the prediction box, and conf i Represents the confidence probability that the prediction box belongs to category i; T i ={x1,y1,x2,y2,conf i }, i = 1, 2 or 3 After getting the output results, filter out the conf i >conf threh The predicted box is used as the final deep learning prediction result and marked in the original image to complete the recognition process.

2. A hybrid drive underwater small monitoring system based on deep learning according to claim 1, characterized in that: The transient speed drive module (1) is provided with a raw material replenishing port (101), a raw material storage unit (102), a raw material input port (103), an exergy reaction excitation device (104), a reaction chamber (105) and a flexible membrane (106); Reaction raw materials are added to the raw material storage unit (102) through the raw material replenishing port (101). When the transient speed drive module (1) is working, the reaction raw materials in the raw material storage unit (102) enter the reaction chamber (105) through the raw material input port (103). The exergonic reaction excitation device (104) triggers the reaction raw materials in the reaction chamber (105) to produce an instantaneous exergonic reaction through electric sparks. The flexible membrane (106) is arranged at the bottom of the reaction chamber (105). The flexible membrane quickly deforms to generate an instantaneous driving force in the opposite direction for the robot.

3. The hybrid drive underwater small monitoring system based on deep learning according to claim 1 is characterized in that: The propeller propulsion module (2) is composed of four symmetrically installed propellers (201), a shroud (202) and a brushless motor (203). The brushless motor (203) is installed on the top of the propeller (201) and the two rotate coaxially. The shroud (202) is installed outside the propeller (201). The propeller propulsion module (2) distributes the duty ratio of the PWM wave of each propeller (201) through a microcomputer (302) to realize underwater vector propulsion of the robot.

4. The hybrid drive underwater small monitoring system based on deep learning according to claim 1 is characterized in that: A communication unit (301), a microcomputer (302) and a power supply unit (303) are provided in the control cabin (3), wherein the communication unit (301) establishes a wireless data transmission connection with a land control terminal for receiving control commands and sending robot underwater sensor information and monitoring analysis results; the microcomputer (302) serves as a main processor for processing control commands issued by a land computing platform (5) and storing video information; and the power supply unit (303) includes a power supply and a power management board for supplying power to the entire system.

5. The hybrid drive underwater small monitoring system based on deep learning according to claim 1 is characterized in that: The monitoring module (4) is provided with a laser scanning device (401), an auxiliary light source device (402), a camera (403) and a pan / tilt platform (404); The laser scanning device (401) acquires spatial point cloud data, couples it with the underwater high-definition image acquired by the camera (403), and establishes a three-dimensional visualization model; the auxiliary light source device (402) automatically adjusts the light intensity according to the brightness of the underwater environment, thereby improving the quality of underwater shooting images; the pan-tilt platform (404) is a supporting device for the fixed camera, and the acquisition results of the monitoring module are transmitted to the microcomputer (302) through the communication unit (301), and sent to the land computing platform for real-time monitoring of the underwater situation.

6. The hybrid drive small underwater monitoring system based on deep learning according to claim 1, characterized in that: The land computing platform (5) is mainly composed of an underwater communication module (501) and a high-performance computer (502). It uses a deep neural network-based recognition and analysis, takes 5 frames of images per second, and performs target detection on three typical underwater concrete damages: cracks, corrosion, and exposed steel bars. A corresponding confidence level is given for each prediction frame.

7. The hybrid drive underwater small monitoring system based on deep learning according to claim 1, characterized in that: The training process of a deep neural network consists of the following steps: (1) Data acquisition: photograph and prepare underwater concrete damage images, including three types of damage: cracks, corrosion, and exposed steel bars; and divide them into a data set and a test set; (2) Labeling: Organize experts to calibrate the damaged area on the original image of the dataset. The calibration area is a rectangular box; (3) Training the network: Input the data set into the neural network for training and set the learning rate; (4) Test network: Input the test set into the trained network and calculate the prediction accuracy and recall rate; (5) Improvement and optimization: Change the network structure, including adding attention, residual, and feature pyramid modules to verify the changes in model accuracy; adjust the composition of the training set, including deleting difficult examples and supplementing sample images for features that have not been learned; change the depth of the neural network, including testing the relationship between the computational accuracy and efficiency of networks of different depths; compare the recognition accuracy and recall rate data of three typical injuries under different conditions, and finally obtain the optimal model for application.

8. The hybrid drive underwater small monitoring system based on deep learning according to claim 1 is characterized in that: The inspection rate is affected by factors such as human resources, maintenance importance, and weather conditions. It is determined by the funding party and represents the maintenance demand of the project area. The higher the inspection rate, the more urgent the inspection and maintenance needs of the area, and the more suitable it is for emergency maintenance tasks. The maintenance rate is low, which means that the maintenance demand in this area is not large, and it is suitable for annual maintenance tasks. threh The setting is best within the range of 0.74-0.

85.

9. The hybrid-driven underwater small-scale monitoring system based on deep learning according to claim 1, characterized in that: After locating the damaged area and retrieving relevant video clips, the analytic hierarchy process is used to determine the extent of damage to the concrete structure, providing early warning of the safety status of the underwater structure.

Images