Control fault-tolerant evaluation method and device for underwater robot multi-sensor fusion environment perception

Abstract

The invention relates to the technical field of autonomous underwater robots, in particular to a control fault-tolerant evaluation method for underwater robot multi-sensor fusion environment perception. The method comprises the following steps: generating a searching route according to an obtained area environment of an ocean area where a target object is located; under the condition that the target object is found along the finding route, a fusion model is constructed according to the received data of the multiple sensors; based on the fusion model, generating a plurality of detection routes for detecting the target object by combining the sensing performance and the motion performance of the underwater robot, and evaluating the fault-tolerant capability of each detection route; after the target object is detected along the detection route with the optimal fault-tolerant capability, a detection result is obtained and sent according to received data in the detection process; the method avoids the problem that the detailed information of the search and rescue object cannot be obtained.

Description

Technical Field

[0001] The present application relates to the technical field of autonomous underwater robots, and in particular to a control fault-tolerant evaluation method and device for multi-sensor fusion environmental perception of an underwater robot. Background Art

[0002] Autonomous Underwater Vehicle (AUV) is equipped with power system, sensor system and control system. It can independently complete the predetermined tasks in complex underwater environment without human real-time control. Therefore, it has been widely used in the fields of marine scientific research, marine resource development and marine security.

[0003] When dealing with accidents in the ocean, using underwater robots to search the sea can not only improve the efficiency of locating the crashed objects, but also obtain the environmental conditions of the sea area where they are located, providing guidance for the rescue operations of rescue personnel.

[0004] At present, when underwater robots are used for underwater search and rescue, they often only locate the search and rescue object, but cannot conduct detailed detection of the search and rescue object, so there is a problem that detailed information of the search and rescue object cannot be obtained. Summary of the invention

[0005] In order to solve the problems existing in the above-mentioned prior art, the purpose of this application is to provide a control fault tolerance evaluation method and device for multi-sensor fusion environmental perception of an underwater robot.

[0006] In a first aspect, the present application provides a control fault tolerance evaluation method for underwater robot multi-sensor fusion environment perception, the method comprising:

[0007] Generate a search route based on the acquired regional environment of the ocean area where the target object is located; the regional environment includes the regional range, underwater depth and regional weather;

[0008] When the target object is found along the search route, a fusion model is constructed based on the received data of multiple sensors; the received data includes sound wave information and image information; the fusion model is a three-dimensional model including the target object, its environment, and the underwater robot;

[0009] Based on the fusion model, combined with the underwater robot's sensing performance and motion performance, multiple detection routes for detecting the target object are generated, and the fault tolerance of each detection route is evaluated; the fault tolerance includes the degree of distortion of environmental perception and the collision risk during the detection process;

[0010] After detecting the target object along the detection route with the best fault tolerance, the detection result is obtained and sent according to the received data in the detection process; the detection result includes the position and shape of the target object.

[0011] In one embodiment, the step of constructing a fusion model based on the received data of multiple sensors includes:

[0012] Generate the morphology of the target object and the three-dimensional model of the surrounding landforms and water flow elements based on the sound wave information and image information;

[0013] Perform image recognition on the image information to obtain the environmental factors of the target object's environment; the environmental factors include visibility and obstacle distribution; visibility is expressed as the farthest distance at which the underwater robot can clearly image the target object;

[0014] The fusion model is obtained by combining the three-dimensional model, water flow elements and environmental elements.

[0015] In one embodiment, the steps of generating multiple detection routes for detecting a target object based on a fusion model and combining the sensing performance and motion performance of the underwater robot, and evaluating the fault tolerance of each detection route include:

[0016] Based on visibility and fusion model, multiple detection routes for automatic obstacle avoidance are obtained;

[0017] According to the acquisition distance and sensor performance on the detection route, the degree of distortion of each detection route is predicted; and the collision risk of each detection route is predicted based on the motion performance of the underwater robot; the acquisition distance represents the distance between the data acquisition location and the target object;

[0018] The distortion degree and collision risk are normalized respectively, and the fault tolerance of the corresponding detection route is obtained through weighted operation.

[0019] In one embodiment, the step of predicting the collision risk of each detection route based on the motion performance of the underwater robot includes:

[0020] According to the preset motion performance, the position, speed and posture of the underwater robot along the detection route in the fusion model are simulated to obtain the simulated driving process;

[0021] Based on the position and posture of the underwater robot during the simulated driving process, the minimum distance between the underwater robot and other objects is obtained; other objects include seabed reefs, target objects and obstacles;

[0022] According to the speed and posture of the underwater robot when it reaches the minimum distance, the hazard coefficient of collision is obtained;

[0023] The collision risk is calculated by calculating the ratio of the hazard coefficient to the corresponding minimum distance.

[0024] In one embodiment, the method further comprises:

[0025] According to the simulated driving process and sensor performance, the aberration coefficient of the image collected by the underwater robot during the simulated driving process is predicted; the aberration coefficient includes the degree of distortion and the degree of blur;

[0026] Predicting the fusion distortion coefficient caused by fusion of collected images according to the number of collected images during the simulated driving process;

[0027] The degree of distortion is obtained by multiplying the aberration coefficient and the fusion distortion coefficient.

[0028] In one embodiment, the number of sensors of the underwater robot is configured according to the load of the underwater robot and the regional environment.

[0029] In one embodiment, the fault tolerance of the detection route also includes the energy consumption fault tolerance of the detection route; and the method further includes:

[0030] During the process of traveling along the search route, the limit communication range of the underwater robot is detected; the limit communication range indicates the area in which the underwater robot can establish communication with the host computer in the sea water;

[0031] If the area where the detection route is located exceeds the limit communication range, the return energy consumption of the underwater robot returning to the nearest communication signal coverage position is calculated;

[0032] When calculating the fault tolerance, the energy tolerance of the detection route is predicted based on the simulated driving process and the return energy consumption.

[0033] In a second aspect, the present application provides a control fault tolerance evaluation device for underwater robot multi-sensor fusion environment perception, comprising:

[0034] The route generation module is used to generate a search route based on the regional environment of the ocean area where the target object is located; the regional environment includes the regional range, underwater depth and regional weather; it is also used to generate multiple detection routes for detecting the target object based on the fusion model, combined with the sensing performance and motion performance of the underwater robot, and evaluate the fault tolerance of each detection route; the fault tolerance includes the degree of distortion of environmental perception and the collision risk of the detection process;

[0035] The data processing module is used to build a fusion model based on the received data of multiple sensors when the target object is found along the search route; the received data includes sound wave information and image information; the fusion model is a three-dimensional model including the target object, its environment, and the underwater robot; it is also used to obtain and send the detection results based on the received data during the detection process after detecting the target object along the detection route with the best fault tolerance; the detection results include the position and shape of the target object.

[0036] In a third aspect, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the methods in the first aspect of the present application are implemented.

[0037] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect of the present application.

[0038] The present application provides a control fault-tolerant evaluation method for multi-sensor fusion environmental perception of an underwater robot. By generating a search route for the underwater robot according to the sea parameters and meteorological conditions of the area where the target object may be located, the search efficiency can be improved. After the target object is found, a fusion model of the target object and the surrounding environment can be generated by fusing the received data of multiple sensors, providing a simulation basis for path planning for detailed detection of the target object. Through the motion performance of the underwater robot, the sensing performance of the sensors carried by the underwater robot and the fusion model, a detection route that can capture a clear image of the target object while avoiding collision is obtained. After the detection of the target object is completed according to the detection route, the received data is fused to obtain detailed information of the target object and send it, effectively avoiding the problem of not being able to obtain detailed information of the search and rescue object. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flowchart of the steps of a control fault tolerance evaluation method for multi-sensor fusion environment perception of an underwater robot in an embodiment;

[0040] Figure 2 is a flowchart of the steps of building a fusion model according to received data in an embodiment;

[0041] Figure 3 is a flowchart of steps for generating and evaluating the fault tolerance of each detection route in an embodiment;

[0042] Figure 4 is a flowchart of steps for predicting the collision risk of each detection route in an embodiment;

[0043] Figure 5 is a flow chart of steps for predicting the degree of distortion of each detection route in one embodiment;

[0044] Figure 6 is a flowchart of steps for predicting the energy consumption fault tolerance of each detection route in an embodiment;

[0045] Figure 7 The present invention is a structural block diagram of a control fault tolerance evaluation device for multi-sensor fusion environment perception of an underwater robot in an embodiment. DETAILED DESCRIPTION

[0046] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0048] When used herein, the singular forms "a", "an" and "the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" etc. specify the presence of stated features, integers, steps or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, integers, steps or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.

[0049] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0050] In one embodiment, Figure 1 As shown, the present application provides a control fault tolerance evaluation method for underwater robot multi-sensor fusion environment perception, the method comprising the following steps S22 to S28:

[0051] S22, generating a search route according to the acquired regional environment of the ocean area where the target object is located; the regional environment includes regional range, underwater depth and regional weather.

[0052] Specifically, the approximate location of the accident is determined based on the last signal sent by the target object, and then the water flow direction in the area around the accident location is simulated through the ocean environment numerical model according to the meteorological conditions at the time, and the sea area where the target object may currently be located is predicted. Then, based on the existing ocean database, the seabed topography and underwater depth of the above-mentioned sea area are obtained; the above-mentioned sea area and underwater depth for finding the target object are input into the underwater robot, which can generate a search route with high coverage and high search efficiency according to the search path generation rules.

[0053] S24, when the target object is found along the search route, a fusion model is constructed according to the received data of the multiple sensors; the received data includes sound wave information and image information; the fusion model is a three-dimensional model including the target object and its environment, as well as the underwater robot.

[0054] Specifically, the underwater robot searches for the target object along the search route. During the search process, it can determine its own position by wirelessly communicating with the server or through the onboard inertial navigation system. When the onboard sensor receives an abnormal signal, it determines that the abnormal signal comes from the target object based on the sonar signal and image recognition.

[0055] Furthermore, when it is determined that the target object is found, the location information of the target object is saved, and the target object and its environment are modeled; the modeling step specifically generates a three-dimensional model of the shape of the target object, the seabed topography around the target object, and the obstacle distribution based on the sonar data and image data, and obtains environmental factors based on the sound wave information and image recognition; the environmental factors include the visibility of the sea water around the target object, the direction of the water flow, the speed of the water flow, and the type and distribution of obstacles.

[0056] Preferably, the underwater robot is equipped with a binocular vision sensor, which can obtain the depth information of each object in the image, and combine the depth information and sonar information for three-dimensional modeling; the underwater robot is equipped with an acoustic Doppler current meter, which can obtain water flow elements based on the Doppler principle.

[0057] S26, based on the fusion model, combined with the sensing performance and motion performance of the underwater robot, generates multiple detection routes for detecting the target object, and evaluates the fault tolerance of each detection route; the fault tolerance includes the degree of distortion of environmental perception and the collision risk of the detection process.

[0058] It is understandable that detailed information of the target object, such as the current appearance and structure of the target object, can help search and rescue personnel quickly determine the possible cause of the accident and the current state of the target object, which plays an important role in the search and rescue operation. In order to detect the detailed information of the target object, the underwater robot needs to conduct detailed detection around the target object under the limitations of visibility and sensing performance to obtain the clearest possible detailed information of the target object. The three-dimensional model of the target object can also be further modeled based on the obtained detailed information to obtain the detection results.

[0059] Specifically, based on the visibility around the target object and the sensing performance of the onboard sensors, the limited distance at which the underwater robot can obtain clear detailed information of the target object is obtained; within the limited distance, taking into account the underwater terrain and the distribution of surrounding obstacles, a plurality of different detection routes that can automatically avoid obstacles are generated based on different detection requirements through a preset detection route planning method; the detection requirements include speed requirements and clarity requirements.

[0060] Furthermore, a water flow model within a limited distance of the target object is established according to the fusion model, and then the position change of the underwater robot during detection along the detection route is simulated according to the kinematic model of the underwater robot; and the collision risk of the underwater robot is further predicted according to the distance between the underwater robot and the target object and the distance with other obstacles; at the same time, the sound wave data and image data collected by the underwater robot and the degree of distortion of the modeling based on the two data are predicted according to the distance between the underwater robot and the target object, the visibility of the sea water and the sensing performance of the sensor; the collision risk is taken as the motion fault tolerance, and the distortion degree is taken as the perception fault tolerance, and the two are normalized respectively, assigned weights and added to obtain the fault tolerance capability of the detection route.

[0061] S28, after detecting the target object along the detection route with the best fault tolerance, a detection result is obtained and sent according to the received data in the detection process; the detection result includes the position and shape of the target object.

[0062] Specifically, the underwater robot selects a detection route with the best fault tolerance to detect around the target object, fuses and models the data collected during the detection process, and sends it as the detection result of the target object.

[0063] Preferably, the detection result may also include water flow element data and environmental images around the target object.

[0064] The above-mentioned control fault-tolerant evaluation method of underwater robot multi-sensor fusion environmental perception can improve the search efficiency by generating the search route of the underwater robot according to the sea parameters and meteorological conditions of the area where the target object may be located; after discovering the target object, a fusion model of the target object and the surrounding environment can be generated by fusing the received data of multiple sensors, providing a simulation basis for the path planning of detailed detection of the target object; through the motion performance of the underwater robot, the sensing performance of the sensors carried by the underwater robot and the fusion model, a detection route that can collect a clear image of the target object while avoiding collision is obtained; after completing the detection of the target object according to the detection route, the received data is fused to obtain detailed information of the target object and send it, effectively avoiding the problem of not being able to obtain detailed information of the search and rescue object.

[0065] In an exemplary embodiment, Figure 2 As shown, building a fusion model based on the received data of multiple sensors includes the following steps S242 to S246:

[0066] S242, generating a three-dimensional model of the target object's shape and surrounding landforms and water flow elements based on the sound wave information and the image information.

[0067] Specifically, the complementary distance information of the target object and the surrounding ground relative to the underwater robot is first obtained through the sound wave information received by the sonar and the image information collected by the visual sensor; then, a three-dimensional model reflecting the size, shape and terrain of the target object and the surrounding area is constructed according to the complementary distance information through filtering technology; based on the Doppler principle, the direction and velocity of the surrounding water flow are obtained according to the frequency shift of the transmitted and received sound waves.

[0068] S244, performing image recognition on the image information to obtain environmental factors of the environment where the target object is located; the environmental factors include visibility and obstacle distribution; visibility is expressed as the farthest distance at which the underwater robot can clearly image the target object.

[0069] Preferably, a pre-trained neural network model for image recognition is used to classify and recognize image information.

[0070] Specifically, the neural network model is first used to identify objects in the image that are not target objects and seabed terrain and judge them as obstacles. Secondly, the obstacles are classified according to their edge features, texture features, and color features. The obstacle types may include fish, seaweed, and floating objects. Finally, the sonar information and image depth information of each obstacle are integrated to obtain the obstacle location distribution. At the same time, the visibility of the surrounding seawater is obtained based on the depth of the farthest object identifiable in the image information.

[0071] Preferably, the flow direction and velocity of the surrounding seawater can also be estimated by the movement trajectory of the tiny objects in the time-series image, which is complementary to the flow elements obtained by the sound wave information; the tiny objects may include plankton and sand and dust.

[0072] S246, combining the three-dimensional model, water flow elements and environmental elements to obtain a fusion model.

[0073] Specifically, the three-dimensional model of the target object and the surrounding landforms as well as the obstacle location distribution are combined, and the water flow elements are input as the physical field to obtain the fusion model.

[0074] In an exemplary embodiment, Figure 3 As shown, based on the fusion model, combined with the sensing performance and motion performance of the underwater robot, multiple detection routes for detecting the target object are generated, and the fault tolerance capability of each detection route is evaluated, including the following steps S262 to S266:

[0075] S262, obtaining multiple automatic obstacle avoidance detection routes based on visibility and fusion model.

[0076] It is understandable that the clarity of the image obtained by the visual sensor is restricted by the turbidity of the seawater, so the distance between the visual sensor and the collected object needs to be kept within the visibility; if the distance between the visual sensor and the collected object exceeds the visibility, a clear and recognizable image cannot be obtained.

[0077] Specifically, in order to detect detailed information of the target object, the underwater robot needs to circle the target object to collect detailed data; therefore, based on the fusion model, multiple feasible detection routes are generated; the range of the detection route is based on visibility as the farthest circling distance, which can ensure clear information. At the same time, the detection route must be able to avoid obstacles near the target object to prevent collisions that affect the detection task.

[0078] S264, predicting the degree of distortion of each detection route based on the collection distance and sensor performance on the detection route; and predicting the collision risk of each detection route based on the motion performance of the underwater robot; the collection distance represents the distance between the data collection position and the target object.

[0079] It is understandable that when selecting a detection route, it is necessary to consider both the motion fault tolerance and perception fault tolerance of the underwater robot in performing detection tasks along the detection route; the perception fault tolerance refers to the fault tolerance of the perception data obtained along the detection route, and the main reason for the distortion of the perception data is the image distortion of the image information and the fusion distortion during subsequent image fusion; and the motion fault tolerance refers to the risk of collision that may occur during detection along the detection route.

[0080] Specifically, the aberrations of image information, such as distortion and blur, are related to the sensing performance and acquisition distance of the visual sensor. The degree of image distortion of the visual sensor carried by the underwater robot at different acquisition distances is obtained through experiments. Combined with the distortion coefficient of the image fusion algorithm, the degree of perception distortion of the underwater robot when detecting along the detection route can be estimated, that is, the perception fault tolerance capability. On the other hand, according to the distance between each position on the detection route and the target object, various obstacles and seabed reefs, the collision risk can be predicted, that is, the motion fault tolerance capability.

[0081] S266, normalizing the distortion degree and the collision risk respectively, and obtaining the fault tolerance of the corresponding detection route through weighted calculation.

[0082] It is understandable that the closer to the target object, the clearer the sensor data can be obtained, but there will also be a greater risk of collision. Therefore, when evaluating the fault tolerance of the detection route, it is necessary to set the weights of distortion degree and collision risk according to the specific task requirements to obtain the optimal detection path for the current task.

[0083] Specifically, the distortion degree of each position on the simulated driving route is normalized respectively, and the collision risk of each position is normalized at the same time; the fault tolerance is obtained by weighted calculation based on the normalized total distortion degree and total collision risk.

[0084] In an exemplary embodiment, Figure 4 As shown, predicting the collision risk of each detection route based on the motion performance of the underwater robot includes the following steps S2662 to S2668:

[0085] S2662, based on the preset motion performance, simulate the position, speed and posture of the underwater robot in the fusion model during the driving process along the detection route to obtain a simulated driving process.

[0086] It can be understood that based on the principles of fluid mechanics, simulating the flow of water around a target object and obtaining the water flow elements surrounding the target object can provide conditions for the motion simulation of an underwater robot.

[0087] Specifically, based on the principles of kinematics and according to the motion performance of the underwater robot, such as maximum speed and minimum steering angle, the posture and speed change process of the underwater robot along the detection route in the fusion model is simulated, that is, the driving process is simulated.

[0088] S2664, based on the position and posture of the underwater robot during the simulated driving process, obtain the minimum distance between the underwater robot and other objects; the other objects include seabed reefs, target objects and obstacles.

[0089] Specifically, according to the position and posture of the underwater robot during the simulated driving process, several minimum distances between the boundary of the underwater robot model and the boundary of the target object, the seabed reefs and the boundaries of various obstacles are obtained.

[0090] S2666, obtain the hazard coefficient of collision according to the speed and posture of the underwater robot when it reaches the minimum distance.

[0091] Specifically, when the underwater robot appears at each minimum distance position, the speed of the underwater robot at this time is obtained, and the part closest to other objects is obtained based on its posture; then, the preset hazard score of the part is obtained according to the importance and stability of the part; the speed of the underwater robot is normalized and multiplied by the hazard score to obtain the hazard coefficient corresponding to the minimum distance.

[0092] S2668, calculate the ratio of the hazard coefficient to the corresponding minimum distance to obtain the collision risk.

[0093] It is understandable that the greater the relative distance between the underwater robot and the obstacle, the smaller the possibility of a collision; the greater the speed of the underwater robot, the greater the kinetic energy, the stronger the impact force when a collision occurs, and the greater the possibility of a failure; the importance and stability of the collision part of the underwater robot determines whether the underwater robot can work normally after a collision.

[0094] Specifically, the ratio of the hazard coefficient to the corresponding minimum distance is used as the collision risk of the minimum distance; and the average value of the collision risks of each minimum distance is calculated as the collision risk of the detection route.

[0095] In an exemplary embodiment, Figure 5 As shown, the method further includes the following steps S32 to S36:

[0096] S32, predicting the aberration coefficient of the image collected by the underwater robot during the simulated driving process according to the simulated driving process and the sensing performance; the aberration coefficient includes the degree of distortion and the degree of blur.

[0097] Specifically, the corresponding relationship between the distortion degree and the acquisition distance of the visual sensor carried by the underwater robot, as well as the corresponding relationship between different movement speeds and the degree of motion blur, is obtained through experiments; in the fault-tolerance evaluation of the detection route, the aberration in the collected image in the detection route can be predicted based on the distance and driving speed between the underwater robot and the target object during the simulated driving process.

[0098] S34, predicting a fusion distortion coefficient caused by fusion of the collected images according to the number of images collected during the simulated driving process.

[0099] It is understandable that the sensor sampling interval during the simulated driving process can also be adjusted according to the detection requirements; if the sampling interval is small, more collected images can be obtained, thereby improving the quality of the fused image; if fewer images are collected, the fused image will have greater distortion.

[0100] Specifically, according to the size of the target object and the predicted number of collected images, a fusion distortion coefficient is obtained through a preset fusion distortion coefficient rule.

[0101] S36, obtaining the degree of distortion by multiplying the aberration coefficient and the fusion distortion coefficient.

[0102] Specifically, although the original image is geometrically calibrated before image fusion, the influence of aberration cannot be completely eliminated. Therefore, when calculating perceptual error tolerance, the predicted aberration coefficient is multiplied by the predicted fusion distortion coefficient to obtain the degree of distortion.

[0103] In an exemplary embodiment, the number of sensors of the underwater robot is configured according to the load of the underwater robot and the regional environment.

[0104] Specifically, before the underwater robot performs a mission, if the ocean weather is bad or the underwater depth is deep, the underwater robot's sensors are more likely to fail due to collision and pressure. If the underwater robot's load permits, the underwater robot's fault tolerance can be increased by configuring redundant sensors.

[0105] In an exemplary embodiment, Figure 6 As shown, the fault tolerance of the detection route also includes the energy consumption fault tolerance of the detection route; the method also includes the following steps S42 to S46:

[0106] S42, during the driving process along the search route, detecting the limit communication range of the underwater robot; the limit communication range indicates the area in the sea water where the underwater robot can establish communication with the host computer.

[0107] It is understandable that the communication signal between the underwater robot and the host computer is affected by the sea water and weather and cannot be accurately measured. Therefore, the underwater robot needs to obtain it based on the signal strength during the search process.

[0108] Specifically, when the underwater robot is searching for the target object along the search route, it pays attention to the communication signal strength between itself and the host computer; when the communication signal strength is lower than a preset value, the distance between the underwater robot and the host computer is used as the limit communication range.

[0109] S44, if the area where the detection route is located exceeds the limit communication range, the return energy consumption of the underwater robot returning to the nearest communication signal coverage position is calculated.

[0110] It is understandable that, since the underwater robot may have consumed a lot of energy during the search process, the limitation of remaining energy needs to be considered when selecting the detection route, so as to prevent the underwater robot from running out of energy before sending out detection information of the target object.

[0111] Specifically, after the underwater robot finds the target object, the return distance between the target object and the boundary of the limit communication range is calculated, and the return energy consumption required for the return distance is calculated according to the energy parameters of the underwater robot.

[0112] S46, when calculating the fault tolerance, predicting the energy consumption fault tolerance of the detection route according to the simulated driving process and the return energy consumption.

[0113] Specifically, the distance of the detection route is obtained according to the simulated driving route, and then the detection energy consumption required for the detection route is calculated based on the distance; the remaining energy is subtracted from the return energy consumption to obtain the available energy, and then the ratio of the detection energy consumption to the available energy is calculated as the energy consumption tolerance of the detection route.

[0114] Furthermore, a weight is assigned to the energy consumption fault tolerance capability and a weighted calculation is performed with the perception fault tolerance capability and the motion fault tolerance capability to obtain the fault tolerance capability of the detection route.

[0115] It should be understood that although Figure 1-Figure 6 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-Figure 6 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0116] In one embodiment, Figure 7 As shown, the present application provides a control fault tolerance evaluation device 70 for underwater robot multi-sensor fusion environment perception, comprising:

[0117] The route generation module 71 is used to generate a search route according to the regional environment of the ocean area where the target object is located; the regional environment includes the regional range, underwater depth and regional weather; it is also used to generate multiple detection routes for detecting the target object based on the fusion model, combined with the sensing performance and motion performance of the underwater robot, and evaluate the fault tolerance of each detection route; the fault tolerance includes the degree of distortion of environmental perception and the collision risk of the detection process;

[0118] The data processing module 72 is used to build a fusion model based on the received data of multiple sensors when the target object is found along the search route; the received data includes sound wave information and image information; the fusion model is a three-dimensional model including the target object, its environment, and the underwater robot; it is also used to obtain and send the detection results based on the received data during the detection process after detecting the target object along the detection route with the best fault tolerance; the detection results include the position and shape of the target object.

[0119] For the specific definition of a control fault tolerance assessment device for multi-sensor fusion environmental perception of an underwater robot, please refer to the definition of a control fault tolerance assessment method for multi-sensor fusion environmental perception of an underwater robot mentioned above, which will not be repeated here. Each module in the above-mentioned control fault tolerance assessment device for multi-sensor fusion environmental perception of an underwater robot can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0120] In one embodiment, a computer device provided by the present application includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the steps of any one of the control fault tolerance assessment methods for multi-sensor fusion environmental perception of an underwater robot provided by the present application.

[0121] In one embodiment, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any one of the control fault-tolerance evaluation methods for multi-sensor fusion environmental perception of an underwater robot provided in the present application are implemented.

[0122] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0123] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A control fault tolerance evaluation method for underwater robot multi-sensor fusion environment perception, characterized in that: include: Generate a search route based on the acquired regional environment of the ocean area where the target object is located; The regional environment includes the regional scope, underwater depth and regional weather; When the target object is found along the search route, a fusion model is constructed according to the received data of the multiple sensors; the received data includes sound wave information and image information; the fusion model is a three-dimensional model including the target object and its environment, and the underwater robot; Based on the fusion model, combined with the sensing performance and motion performance of the underwater robot, multiple detection routes for detecting the target object are generated, and the fault tolerance of each detection route is evaluated; the fault tolerance includes the degree of distortion of environmental perception and the collision risk of the detection process; After detecting the target object along the detection route with the best fault tolerance, a detection result is obtained and sent according to the received data in the detection process; the detection result includes the position and shape of the target object.

2. The method according to claim 1, characterized in that: The step of constructing a fusion model based on the received data of multiple sensors includes: Generate a three-dimensional model of the shape of the target object and surrounding landforms and water flow elements according to the sound wave information and the image information; Performing image recognition on the image information to obtain environmental factors of the environment where the target object is located; the environmental factors include visibility and obstacle distribution; the visibility is expressed as the farthest distance at which the underwater robot can clearly image the target object; The fusion model is obtained by combining the three-dimensional model, the water flow elements and the environmental elements.

3. The method according to claim 2, characterized in that: The steps of generating a plurality of detection routes for detecting the target object based on the fusion model and combining the sensing performance and motion performance of the underwater robot, and evaluating the fault tolerance of each detection route include: According to the visibility and the fusion model, a plurality of automatic obstacle avoidance detection routes are obtained; According to the acquisition distance on the detection route and the sensing performance, the degree of distortion of each detection route is predicted; and based on the motion performance of the underwater robot, the collision risk of each detection route is predicted; the acquisition distance represents the distance between the data acquisition position and the target object; The distortion degree and the collision risk are normalized respectively, and the fault tolerance of the corresponding detection route is obtained through weighted calculation.

4. The method according to claim 3, characterized in that: The step of predicting the collision risk of each detection route based on the motion performance of the underwater robot comprises: According to the preset motion performance, the position, speed and posture of the underwater robot in the fusion model during the driving process along the detection route are simulated to obtain a simulated driving process; Based on the position and posture of the underwater robot during the simulated driving process, obtaining the minimum distance between the underwater robot and other objects; the other objects include seabed reefs, the target object and the obstacle; Obtaining a collision risk coefficient according to the speed and posture of the underwater robot when it reaches the minimum distance; The collision risk is obtained by calculating the ratio of the hazard coefficient to the corresponding minimum distance.

5. The method according to claim 4, characterized in that: The method further comprises: According to the simulated driving process and the sensing performance, predicting the aberration coefficient of the image collected by the underwater robot during the simulated driving process; the aberration coefficient includes the degree of distortion and the degree of blur; Predicting a fusion distortion coefficient caused by fusing the collected images according to the number of images collected during the simulated driving process; The distortion degree is obtained by multiplying the aberration coefficient and the fusion distortion coefficient.

6. The method according to claim 1, characterized in that: The number of sensors of the underwater robot is configured according to the load of the underwater robot and the regional environment.

7. The method according to claim 4, characterized in that: The fault tolerance of the detection route also includes the energy consumption fault tolerance of the detection route; the method further includes: During the driving process along the search route, the limit communication range of the underwater robot is detected; the limit communication range indicates the area in which the underwater robot can establish communication with the host computer in the sea water; If the area where the detection route is located exceeds the limit communication range, then calculating the return energy consumption of the underwater robot returning to the nearest communication signal coverage position; When calculating the fault tolerance, the energy consumption fault tolerance of the detection route is predicted according to the simulated driving process and the return energy consumption.

8. A control fault tolerance evaluation device for underwater robot multi-sensor fusion environment perception, characterized in that: The device comprises: A route generation module is used to generate a search route according to the acquired regional environment of the ocean area where the target object is located; the regional environment includes the regional range, underwater depth and regional weather; it is also used to generate multiple detection routes for detecting the target object based on the fusion model and in combination with the sensing performance and motion performance of the underwater robot, and evaluate the fault tolerance of each detection route; the fault tolerance includes the degree of distortion of environmental perception and the collision risk of the detection process; The data processing module is used to construct a fusion model based on the received data of multiple sensors when the target object is found along the search route; the received data includes sound wave information and image information; the fusion model is a three-dimensional model including the target object and its environment, and the underwater robot; and after detecting the target object along the detection route with the best fault tolerance, obtain and send the detection result based on the received data during the detection process; the detection result includes the position and shape of the target object.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images