A navigation method and device for an underwater environment, an underwater electronic device, and a computer-readable storage medium

Abstract

The application provides a navigation method and device for an underwater environment, an underwater electronic device and a computer readable storage medium. The method comprises: acquiring underwater environment data collected in real time; preprocessing the underwater environment data by using a pre-trained machine learning model to obtain an environment perception result; constructing an underwater map based on position information and category information of underwater obstacles; acquiring a target position of the underwater electronic device, performing path planning based on the current position, the target position and the underwater map to obtain a first navigation path; determining a feasibility evaluation result of the first navigation path based on size information and a turning radius of the underwater electronic device, and the minimum path width, the minimum path length of the first navigation path and the minimum turning width in the first navigation path; and controlling the underwater electronic device to move based on the navigation path when the evaluation result is that the underwater electronic device can pass through. Through the application, the dependence on manual work for navigation in the underwater environment can be reduced, the navigation accuracy can be improved, and autonomous navigation in the underwater scene can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to navigation technology, in particular to a navigation method and device for underwater environment, an underwater electronic device and a computer readable storage medium. BACKGROUND

[0002] In related art, the navigation of a conventional underwater electronic device relies on manual operation, which is dangerous and complicated. The underwater electronic device only moves along a predetermined route and lacks real-time adjustment capability. These defects increase the operation difficulty, affect the navigation accuracy and limit the autonomous navigation capability of the underwater electronic device in a complex underwater environment. SUMMARY

[0003] The navigation method and device for underwater environment, the underwater electronic device and the computer readable storage medium provided by the embodiments of the present application can improve the navigation accuracy and realize autonomous navigation in an underwater scene.

[0004] The technical solutions of the embodiments of the present application are implemented as follows:

[0005] The navigation method for underwater environment provided by the embodiments of the present application comprises the following steps:

[0006] Obtaining underwater environment data collected in real time, wherein the underwater environment data at least comprises image data, sonar data and inertial navigation data;

[0007] Preprocessing the underwater environment data by using a pre-trained machine learning model to obtain an environment perception result, wherein the environment perception result comprises position information and category information of underwater obstacles, first current position and attitude information of the underwater electronic device;

[0008] Constructing an underwater map based on the position information and the category information of the underwater obstacles;

[0009] Obtaining a target position of the underwater electronic device, and performing path planning based on the first current position, the target position and the underwater map to obtain a first navigation path;

[0010] Obtaining size information and turning radius of the underwater electronic device, and minimum path width, minimum path length and minimum turning width in the first navigation path, and determining that the feasibility evaluation result of the first navigation path is evaluation pass when the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius;

[0011] Controlling the underwater electronic device to move based on the first navigation path.

[0012] The embodiment of the present application provides a navigation device of an underwater environment, comprising:

[0013] A data acquisition module is configured to acquire underwater environment data collected in real time, wherein the underwater environment data at least includes image data, sonar data and inertial navigation data;

[0014] A map construction module is configured to pre-process the underwater environment data by using a pre-trained machine learning model to obtain environment perception results, wherein the environment perception results include position information and category information of underwater obstacles, first current position and attitude information of underwater electronic equipment; and an underwater map is constructed based on the position information and the category information of the underwater obstacles;

[0015] A first path planning module is configured to acquire a target position of the underwater electronic equipment, and perform path planning based on the first current position, the target position and the underwater map to obtain a first navigation path;

[0016] A feasibility evaluation module is configured to acquire size information and turning radius of the underwater electronic equipment, and minimum path width, minimum path length of the first navigation path and minimum turning width in the first navigation path; when the minimum path width is greater than the equipment width, the minimum path length is greater than the equipment length, and the minimum turning width is greater than the turning radius, determining that a feasibility evaluation result of the first navigation path is evaluation pass;

[0017] A navigation control module is configured to control the underwater electronic equipment to move based on the first navigation path.

[0018] The embodiment of the present application provides an underwater electronic equipment, comprising:

[0019] A memory is configured to store computer executable instructions;

[0020] A processor is configured to execute the computer executable instructions stored in the memory, and implement a navigation method of an underwater environment provided by the embodiment of the present application.

[0021] The embodiment of the present application provides a computer readable storage medium, which stores computer executable instructions, and is configured to implement a navigation method of an underwater environment provided by the embodiment of the present application when executed by a processor.

[0022] The embodiment of the present application provides a computer program product, which comprises a computer program or computer executable instructions, and implements a navigation method of an underwater environment provided by the embodiment of the present application when executed by a processor.

[0023] The embodiment of the present application has the following beneficial effects:

[0024] The embodiment of the present application realizes a navigation method of underwater environment. After obtaining the underwater environment data collected in real time, the pre-trained machine learning model is used to perform prediction processing based on the underwater environment data to obtain the environment perception result, so as to realize accurate perception and understanding of the real-time underwater environment, so that the underwater electronic device can better identify and classify underwater objects and obstacles. The improved environment perception capability helps the underwater electronic device to better adapt to the complex and changeable underwater environment. Then, a real-time underwater map is constructed based on the environment perception result, and the current position and target position of the underwater electronic device are obtained according to the underwater map. The path planning is performed based on the current position, target position and underwater map to obtain a navigation path, so as to control the underwater electronic device to move based on the navigation path. Since the underwater map is a static description of the underwater environment, it is a grid map including all known obstacles and environmental features. When the path planning is performed based on the underwater map, the obstacles and environmental features on the underwater map are considered. In this way, while realizing the autonomous path planning of the underwater electronic device to improve the autonomy and working efficiency of the underwater electronic device, a safe and efficient navigation path can also be obtained to improve the safety and stability of the underwater environment navigation. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a structural schematic diagram of a first architecture of a navigation system of underwater environment provided by the embodiment of the present application;

[0026] Figure 2 is a structural schematic diagram of a navigation device of underwater environment provided by the embodiment of the present application;

[0027] Figure 3A is a first flowchart of a navigation method of underwater environment provided by the embodiment of the present application;

[0028] Figure 3B is a second flowchart of a method of obtaining environment perception result provided by the embodiment of the present application;

[0029] Figure 3C is a third flowchart of a map construction method provided by the embodiment of the present application;

[0030] Figure 3D is a fourth flowchart of a path planning method provided by the embodiment of the present application;

[0031] Figure 4 is a fifth flowchart of a method of re-planning path provided by the embodiment of the present application;

[0032] Figure 5 is a sixth flowchart of a power prompt method provided by the embodiment of the present application;

[0033] Figure 6is a schematic diagram of a business process of navigation of an underwater environment provided by an embodiment of the present application;

[0034] Figure 7 is a structural schematic diagram of a second architecture of a navigation system of an underwater environment provided by an embodiment of the present application.

[0035] It should be noted that the above-mentioned "first", "second" are only used to distinguish different schemes, and do not represent the advantages or disadvantages of the schemes or the priority in the implementation process. DETAILED DESCRIPTION

[0036] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings, and the described embodiments should not be regarded as limiting the present application. All other embodiments obtained by a person of ordinary skill in the art without making creative labor fall within the scope of protection of the present application.

[0037] In the following description, "some embodiments" are related to a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0038] In the following description, the terms "first\second\third" are only used to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first\second\third" can interchange the specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0039] "Module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as processing circuit or memory) or combination thereof. Similarly, one processor (or multiple processors or memory) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the function of the module or unit.

[0040] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by a person skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0041] The relevant data collection and processing in the embodiments of the present application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0042] Before further detailing the embodiments of the present application, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations.

[0043] 1) Underwater electronic equipment: electronic equipment capable of autonomous or remote control, performing various tasks in underwater environment. These devices combine artificial intelligence, detection and identification, information fusion and intelligent control technologies, and can replace or assist humans to complete underwater operations. For example, underwater drones, unmanned underwater vehicles and underwater boosters are all underwater electronic equipment, which are widely used in ocean exploration, archaeology, fishing, rescue, environmental protection and other fields, providing underwater monitoring, shooting, search and rescue functions.

[0044] 2) Inertial navigation system: a self-contained navigation system that does not rely on external information and does not radiate energy to the outside. It is based on Newton's laws of motion, by measuring the acceleration of the carrier in the inertial reference frame, and integrating it over time, to calculate the velocity, position and attitude of the carrier. For example, in fishing, fishermen use sonar systems to emit sound waves and receive echoes from fish schools, to determine the position and quantity of fish schools. Sonar technology is widely used in ocean exploration, underwater surveying, ship navigation and other fields.

[0045] 3) Grid map: a grid-based map representation method that divides the map area into uniform grid cells and assigns specific attribute information to each grid cell, such as feature types, heights, obstacles, etc. This map is commonly used in autonomous driving, robot navigation, game development and other fields. For example, in an autonomous driving system, a grid map can contain detailed information such as road geometry, lane boundaries, traffic signs, etc., to help vehicles perceive and understand the road environment.

[0046] Traditional underwater electronic equipment navigation relies on manual operation, which is dangerous and complex to set the path, and results in the underwater electronic equipment only following the predetermined route, lacking real-time adjustment capability; the number of sensors carried is limited, the data information collected is less, the operation difficulty is increased, the normal operation of the navigation system is affected, and the autonomous navigation capability of the underwater electronic equipment in complex underwater environment is limited.

[0047] Specifically, the embodiment of the present application provides a kind of underwater electronic equipment navigation system design based on machine learning, underwater electronic equipment carries multiple sensors to collect underwater data, carries out map construction and path planning, and in the design of navigation system, the server that machine learning algorithm is applied and deployed, server is annotated and annotated to the underwater data collected by multiple sensors, establishes training data set, for training machine learning model, so that underwater electronic equipment can be according to underwater environment Real-time path planning and obstacle avoidance, reduce manual intervention, reduce the danger of manual underwater operation, improve the autonomy of underwater electronic equipment navigation system, so as to adapt to complex and changeable underwater environment, improve work efficiency.

[0048] The embodiment of the present application provides a kind of underwater environment navigation method, device, underwater electronic equipment and computer readable storage medium, which can improve the autonomy of underwater electronic equipment navigation system, adapt to underwater complex environment, the following describes the exemplary application of underwater electronic equipment provided by the embodiment of the present application, the underwater electronic equipment provided by the embodiment of the present application can be implemented as underwater unmanned aerial vehicle, underwater robot, deep-sea detector and various terminal applied to underwater scene.The following will illustrate the exemplary application when the device is implemented as a terminal.

[0049] Referring to Figure 1 , Figure 1 is the structure diagram of the first architecture of the navigation system 100 of underwater environment provided by the embodiment of the present application, to realize the support of one underwater environment navigation application, underwater electronic equipment 400 is connected to server 200 through network 300, network 300 can be wide area network or local area network, or combination of the two.

[0050] Underwater electronic equipment 400 is used to implement underwater environment navigation, including data acquisition, preprocessing, environment perception, map construction, path planning and navigation, server 200 is used for data annotation, model training, data storage and analysis.Underwater electronic equipment 400 sends underwater environment data to server 200 after collecting underwater environment data through sensor, server 200 annotates the received underwater environment data, and trains the machine learning model to be trained based on back propagation algorithm using the annotated underwater environment data, to obtain the pre-trained machine learning model.Server 200 sends the pre-trained machine learning model to underwater electronic equipment 400.Underwater electronic equipment 400 acquires real-time collected underwater environment data after entering water, predicts the underwater environment data using the pre-trained machine learning model, obtains environment perception result, and constructs underwater map based on the environment perception result;Then, the current position and target position of underwater electronic equipment are acquired, the path planning is carried out based on the current position, target position and underwater map, and the navigation path is obtained;Finally, underwater electronic equipment is controlled to move based on the navigation path.

[0051] The underwater electronic device 400 can also avoid obstacles and optimize paths according to real-time underwater environmental information during navigation in an underwater environment. During movement based on the first navigation path, the underwater electronic device 400 acquires underwater environmental data collected in real time; then, the underwater environmental data is predicted and processed by using a pre-trained machine learning model, and when an unmarked obstacle is monitored, the underwater electronic device is controlled to stop moving; at this time, the position information of the unmarked obstacle and the second current position of the underwater electronic device are acquired; then, an obstacle avoidance algorithm is used, and the position information of the unmarked obstacle, the second current position of the underwater electronic device, and the target position are used to re-plan a path to obtain a second navigation path; finally, the underwater electronic device is controlled to move based on the second navigation path, thereby improving the task execution capability and flexibility of the underwater electronic device.

[0052] In some embodiments, the server 200 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the embodiments of the present application.

[0053] Referring to Figure 2 , Figure 2 is a structural schematic diagram of the underwater electronic device 400 provided by the embodiments of the present application, Figure 2 The underwater electronic device 400 shown in the figure includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the underwater electronic device 400 are coupled together through a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between the components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 440 in the figure. Figure 2 The processor 410 can be an integrated circuit chip with signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0054] The processor 410 can be an integrated circuit chip with signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0055] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432 that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0056] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, and the like. The memory 450 optionally includes one or more storage devices remotely located from the processor(s) 410.

[0057] The memory 450 includes volatile memory or nonvolatile memory, and can also include both volatile and nonvolatile memory. Nonvolatile memory can be read only memory (ROM), volatile memory can be random access memory (RAM). The memory 450 described herein is intended to include any suitable type of memory.

[0058] In some embodiments, the memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or a subset or superset thereof, examples of which are illustrated below.

[0059] The operating system 451 includes systems programs for handling various basic system services and for performing hardware dependent tasks, such as a firmware layer, a kernel layer, a driver layer, and the like, for implementing various basic services and handling hardware dependent tasks;

[0060] The network communication module 452 is for communicating to other electronic devices via one or more (wired or wireless) network interfaces 420, examples of which include Bluetooth, wireless fidelity (WiFi), universal serial bus (USB), and the like;

[0061] The presentation module 453 is for enabling presentation of information via one or more output devices 431 associated with the user interface 430 (e.g., display screens, speakers, and the like) (e.g., user interfaces for operating peripheral devices and displaying content and information);

[0062] The input processing module 454 is for detecting and interpreting one or more user inputs or interactions from one or more input devices 432.

[0063] In some embodiments, the apparatus provided by the embodiments of the present application can be implemented in software, Figure 2 The navigation apparatus 455 of the underwater environment stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: a data acquisition module 4551, a map construction module 4552, a first path planning module 4553, a feasibility evaluation module 4554, and a navigation control module 4555, which are logical, and thus can be combined or further split according to the implemented functions. The functions of each module will be described below.

[0064] In some other embodiments, the apparatus provided by the embodiments of the present application can be implemented in hardware, for example, the apparatus provided by the embodiments of the present application can be a hardware decoding processor programmed to execute the underwater environment navigation method provided by the embodiments of the present application. For example, the hardware decoding processor can use one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic elements.

[0065] The underwater environment navigation method provided by the embodiments of the present application will be described below. As described above, the electronic device implementing the underwater environment navigation method provided by the embodiments of the present application is an underwater electronic device, and thus the execution subject of each step will not be repeated below. Referring to Figure 3A Figure 3A is the first flowchart of the underwater environment navigation method provided by the embodiments of the present application, which will be described below in combination with the steps shown in Figure 3A Figure 3A The subject of the step is the underwater electronic device.

[0066] In step 101, real-time collected underwater environment data is acquired.

[0067] ​​In some embodiments, the underwater environment data at least includes image data, sonar data and inertial navigation data, which can be collected by sensors deployed on the underwater electronic device, including a camera, a sonar, an inertial navigation system, a depth sensor, a water quality sensor, etc., and the corresponding real-time collected underwater environment data includes image data, sonar data, inertial navigation data and water quality data; wherein the image data can be underwater images collected by the camera, and the image data can further include the size, channel number, etc. of the image. The camera image collection coverage can be within 20 meters in front of the camera; the sonar data includes echo data, for example, echo data within a range of 50 meters collected by the sonar; the inertial navigation data includes position information and attitude information of the underwater electronic device, the position information being the specific coordinates [x, y, z] of the underwater electronic device, and the attitude data being the attitude information [roll, pitch, yaw] of the underwater electronic device; the water quality sensor data includes physical and chemical parameters of the water body, such as temperature, pH value, dissolved oxygen, etc., to evaluate the underwater water quality environment and ensure that the underwater electronic device works in a suitable environment. The frequency of data collection by the sensors can be pre-set, for example, it can be 10 seconds / time, or it can be adjusted according to the accuracy of the underwater environment data.

[0068] As an example of step 101, the obtained underwater environment data can include: the image data collected by the camera is in RGB format, the size is 224*224 pixels, the channel number is 3, and it contains red, green and blue; the sonar data is an echo signal containing information such as the position, shape, size of the target object and the seabed topography; the inertial navigation data obtains the position information of the underwater electronic device as [10, 5, 6], and the attitude information of the underwater electronic device as [roll=10°, pitch=20°, yaw=30°].

[0069] In some embodiments, after obtaining the underwater environment data, a pre-trained machine learning model can be obtained, which is used to predict and process the underwater environment data to obtain the environment perception result. The pre-trained machine learning model can be trained by using the data collected by the underwater electronic device. Before step 101, the underwater environment data can be collected by the underwater electronic device, and then the collected underwater environment data can be labeled and annotated, the labeled and annotated data set is used as a training data set, which is input into the machine learning model for training, so as to obtain the pre-trained machine learning model.

[0070] In step 102, the pre-trained machine learning model is used to pre-process the underwater environment data to obtain the environment perception result.

[0071] In some embodiments, referring to Figure 3B ,Figure 3A The step 102 shown can be implemented by the following steps 1021 to 1025, which are explained in detail below.

[0072] In step 1021, the image data, the sonar data and the inertial navigation data are aligned to obtain fused data.

[0073] In some embodiments, the image data, the sonar data and the inertial navigation data are time-synchronized and spatially aligned, and a filtering algorithm such as Kalman filtering algorithm is used for data fusion to obtain fused data. The fused data includes the aligned image data, the aligned sonar data and the aligned inertial navigation data.

[0074] In step 1022, based on the aligned image data, the pre-trained machine learning model is called for prediction processing to obtain the category information of the underwater obstacle.

[0075] In some embodiments, the category information of the underwater obstacle is a living body or a non-living body. Here, the living body includes all aquatic life, such as fish, coral, shellfish and seaweed, etc., and the non-living body includes rock, sunken ship, artificial structure such as underwater pipeline, platform, etc.

[0076] In step 1023, based on the aligned sonar data, the pre-trained machine learning model is called for prediction processing to obtain the position information of the underwater obstacle.

[0077] In some embodiments, the position information of the underwater obstacle includes the distance and direction of the underwater obstacle relative to the underwater electronic device. The direction can be described by the angle of azimuth and pitch, for example, southeast direction 45° pitch 30°.

[0078] In step 1024, based on the aligned inertial navigation data, the pre-trained machine learning model is called for prediction processing to obtain the attitude information and the first current position of the underwater electronic device.

[0079] In some embodiments, the attitude information [roll, pitch, yaw] of the underwater electronic device and the first current position [x, y, z] of the underwater electronic device can be obtained by analyzing the aligned inertial navigation data.

[0080] As an example of steps 1022 to 1024, the underwater obstacle is a non-living object, the underwater obstacle is 1 meter away from the underwater electronic device in the southeast direction at an angle of 45 degrees and a pitch angle of 30 degrees. The first current position of the underwater electronic device is [0, 2, 19]. It should be noted that in actual applications, the relative distance and the coordinate form are used in combination, the relative description is used in real-time obstacle avoidance, and the coordinate description is used in map construction and path planning.

[0081] In step 1025, one or more features of the position information of the underwater obstacle, the category information, the attitude information, and the first current position are determined as the environment perception result.

[0082] Through steps 1021 to 1025, further processing of the underwater environment data obtained in step 101 is achieved, the data processed in step 101 is applied to the pre-trained machine learning model, the underwater environment information is predicted, the underwater complex environment is adapted, the degree of artificial dependence is reduced, and the underwater electronic device can learn autonomously and obtain the environment perception result.

[0083] Continuing to refer to Figure 3A In step 103, a underwater map is constructed based on the position information and the category information of the underwater obstacle.

[0084] In some embodiments, referring to Figure 3C , Figure 3A The step 103 shown can be implemented through the following steps 1031 to 1033, which are described in detail below.

[0085] In step 1031, initial environment information is obtained, and an initial grid map is generated based on the initial environment information.

[0086] In some embodiments, the obtained initial environment information is consistent with the content contained in the above-mentioned environment perception result, and the construction of the underwater map uses a simultaneous localization and mapping technology, and a real-time underwater map is constructed according to the environment perception result. The specific form of the underwater map is a grid map, which divides the underwater environment into equal-sized grids, and each grid contains environment information, such as whether there is an object, an obstacle, etc. Each grid (cell) can be represented as "occupied" or "free". By fusing sonar data and camera image data, the state of each grid is marked. In some embodiments, in addition to the grid map, the form of the map can also be a topological map or a three-dimensional point cloud map.

[0087] As an example of step 1031, the obtained initial environmental information includes the position of the target object [32, 54, 23], the category of the target object is biological coral, the position information of the underwater obstacle is [18, 66, 63], the distance is 83.87 meters, the position information of the underwater electronic device is [10, 5, 6], the attitude information of the underwater electronic device is [roll = 10°, pitch = 20°, yaw = 30°], the underwater map in the form of a grid is constructed using the simultaneous localization and mapping technology, the grid where the object or obstacle exists is marked as "occupied" and represented by the number 1, and the rest is marked as "free" and represented by the number 0.

[0088] In step 1032, the first target grid of the underwater obstacle in the initial grid map is determined based on the position information and category information of the underwater obstacle.

[0089] The position information includes the distance and direction of the underwater obstacle relative to the underwater electronic device, and the category information is used to represent whether the underwater obstacle is a biological object or a non-biological object.

[0090] In some embodiments, the position information of the obstacle is first converted into a position in the grid map, and the position of the obstacle in the grid map is marked, including whether the grid is occupied and the category information of the obstacle occupying the grid. The marked grid is the first target grid of the underwater obstacle in the initial grid map.

[0091] In step 1033, the state of the first target grid in the initial grid map is set to an occupied state to obtain the underwater map.

[0092] As an example of step 1032 and step 1033, the obstacle position information is [18, 66, 63], the corresponding first target grid is marked as 1, i.e., in the "occupied" state, and the category of the obstacle is marked as non-biological. It should be noted that environmental perception is performed in real time, and the environmental perception results obtained in step 1031 are updated in real time to the underwater map. The real-time perception results are integrated into the map using the simultaneous localization and mapping technology to update the map. The constructed map data is stored locally or in the cloud, and a visualization tool is used to display the environmental map to support navigation and analysis.

[0093] Through steps 1031 to 1033, dynamic perception of the underwater environment is realized, the perception results are updated in real time in the map, and the underwater map is obtained. The underwater electronic device can autonomously learn the underwater environment to improve the autonomous learning ability of the underwater electronic device in a complex underwater environment and realize dynamic environmental perception.

[0094] Continuing to refer to Figure 3AIn step 104, a target position of the underwater electronic device is obtained, and a first navigation path is planned based on the first current position, the target position, and the underwater map.

[0095] In some embodiments, referring to Figure 3D , Figure 3A The step 104 shown can be implemented by the following steps 1041 to 1042, which will be described below in conjunction with Figure 3D .

[0096] In step 1041, the position information of the first target grid in the underwater map is obtained.

[0097] In some embodiments, the position information of the map grid can be marked using a serial number such as 0, 1, 2, … or a coordinate such as [x, y]; the serial number is assigned to the grid from left to right and from top to bottom, starting from the first grid in the upper left corner of the grid map, and the coordinate is selected from one corner of the grid map, such as the upper left corner, as the coordinate origin, and the coordinate axes are defined with the origin as the starting point, with the horizontal right as the positive direction of the x-axis and the vertical downward as the positive direction of the y-axis.

[0098] As an example of step 1041, the position information of the first target grid in the underwater map is obtained, which is 0 if marked by a serial number, or [0, 0] if marked by a coordinate, both indicating that the first target grid is located at the first position in the upper left corner of the grid map.

[0099] In step 1042, a first navigation path is generated based on the first current position, the target position, and the position information of the target grid.

[0100] In some embodiments, the first current position of the underwater electronic device is obtained by an inertial navigation system; the target position is marked in the grid map and obtained by a sonar; and the position information of the target grid is obtained by step 1041.

[0101] As an example of step 1042, assume that a path is needed to be planned from the current electronic device position coordinate [10, 5, 6] to the target position coordinate [18, 66, 63], avoiding the obstacle position [32, 54, 23], the position information of the grid map through which the path passes does not include the grid position of the obstacle, and the path planning is implemented using the classic path planning algorithm Dijkstra.

[0102] Through steps 1041 to 1042, the data obtained by the underwater electronic device using the machine learning model is used to autonomously plan a path, improving the autonomy of navigation.

[0103] Continuing to refer to Figure 3AIn step 105, the size information and turning radius of the underwater electronic device and the minimum path width, the minimum path length of the first navigation path and the minimum turning width in the first navigation path are obtained.

[0104] As an example of step 105, the size information of the underwater electronic device is obtained, the length, width and height are all 1 meter, the turning radius is 2 meters, the minimum path width of the first navigation path is 3 meters, the minimum path length is 5 meters and the minimum turning width in the first navigation path is 2.5 meters.

[0105] In step 106, when the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius, it is determined that the feasibility evaluation result of the first navigation path is evaluation pass.

[0106] In some embodiments, before step 106, it is necessary to first determine whether the size information and the turning radius of the underwater electronic device are less than the minimum path width, the minimum path length and the minimum turning width of the first navigation path. The specific content of the determination includes the size relationship between the minimum path width and the device width, the minimum path length and the device length, and the minimum turning width and the turning radius. When the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius, step 106 is entered. When the minimum path width is less than or equal to the device width, the minimum path length is less than or equal to the device length, and the minimum turning width is less than or equal to the turning radius, it is determined that the feasibility evaluation result of the first navigation path is evaluation fail. The content of the feasibility evaluation of the first navigation path includes the physical feasibility of the underwater electronic device moving on the first navigation path, that is, only when the size information and the turning radius of the underwater electronic device are less than the minimum path width, the minimum path length and the minimum turning width of the first navigation path, it can be ensured that the underwater electronic device moving on the first navigation path will not encounter physical obstacles and keep smooth movement.

[0107] As an example of step 106, it is assumed that the length and width of the underwater electronic device are both 1 meter, the turning radius is 1.5 meters, the minimum path width of the first navigation path is 6 meters, the minimum path length is 10 meters, and the minimum turning width is 2.3 meters. At this time, the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius, so the underwater electronic device can pass through the first navigation path smoothly, that is, the feasibility evaluation result of the first navigation path is evaluation pass.

[0108] Through steps 105 to 106, it can be evaluated whether the underwater electronic device can pass through the first navigation path smoothly, thereby improving the success rate of the underwater electronic device performing tasks on the first navigation path.

[0109] In step 107, the underwater electronic device is controlled to move based on the first navigation path.

[0110] In some embodiments, when the feasibility evaluation result of the first navigation path is evaluation failure, the first navigation path needs to be re-planned. The re-planning of the first navigation path can be achieved by the following technical solutions: obtaining a first sub-path and a second sub-path in the first navigation path, the first sub-path being a sub-path that does not need to be optimized, and the second sub-path being a sub-path that needs to be optimized; performing path planning based on the starting position of the second sub-path, the target position, and the underwater map to obtain a third sub-path; determining a feasibility evaluation result of the third sub-path based on the size information and the turning radius of the underwater electronic device, and the minimum path width, the minimum path length of the third sub-path, and the minimum turning width in the third sub-path; and when the feasibility evaluation result of the third sub-path is evaluation success, combining the first sub-path and the third sub-path to obtain a second navigation path. The underwater electronic device is controlled to move based on the second navigation path. Through re-planning, the problematic part of the first navigation path can be optimized, ensuring that the underwater electronic device can safely navigate based on the navigation path, and further improving the accuracy of navigation.

[0111] For example, assuming that the first navigation path is a path from A to E, passing through B, C, and D in order, where the grid map state corresponding to C is updated to "occupied" or the grid map state corresponding to C is "free", but the position of C is a position that the underwater electronic device cannot pass through, i.e., the feasibility evaluation result of the path of C is evaluation failure, then the second sub-path is from B to D, passing through C. The optimization of the second sub-path uses the kinematic model of the underwater electronic device and the dynamic obstacle avoidance algorithm. Here, for the real-time updated underwater map, the kinematic model of the underwater electronic device and the dynamic obstacle avoidance algorithm are combined to obtain a third sub-path; assuming that E satisfies the corresponding grid map state being "free" and the position of E being a position that the underwater electronic device can pass through, i.e., the feasibility evaluation result of the path of E is evaluation success, then the third sub-path is from B to D, passing through E. The determined second navigation path is from A to E, passing through B, E, and D in order.

[0112] In some embodiments, during the movement of the underwater electronic device based on the first navigation path, the following steps 108 to 112 can also be performed. Figure 4 The steps 108 to 112 are shown below, and the following describes the steps 108 to 112 in combination with the above description. Figure 4

[0113] In step 108, real-time underwater environment data is acquired. ​

[0114] In step 109, the pre-trained machine learning model is used to predict the underwater environment data, and it is determined that an unmarked obstacle is monitored, and the underwater electronic device is controlled to stop moving.

[0115] The unmarked obstacle is an obstacle that is not marked in the underwater map. It should be clear that due to the dynamic changes of the underwater environment, new obstacles will appear, so the unmarked obstacle monitored here is inconsistent with the obstacles marked when the underwater map is constructed.

[0116] In step 110, the position information of the unmarked obstacle and the second current position of the underwater electronic device are obtained.

[0117] In some embodiments, the position information and distance of the unmarked obstacle are obtained by analyzing the sonar data in the underwater environment data, and the second current position of the underwater electronic device is provided by the inertial navigation system.

[0118] As an example of step 110, the second current position of the underwater electronic device is [15, 20, 13], the position of the unmarked obstacle is [8, 21, 19], and the distance between the unmarked obstacle and the underwater electronic device is 9.27 meters.

[0119] In step 111, the path is re-planned according to the underwater map to obtain a third navigation path.

[0120] In some embodiments, when step 111 is implemented, first, based on the position information of the unmarked obstacle, the second target grid corresponding to the unmarked obstacle in the underwater map is determined; the state corresponding to the second target grid in the underwater map is updated to the occupied state to obtain the updated underwater map; based on the second current position, the target position and the updated underwater map, the path is re-planned to obtain the third navigation path.

[0121] In step 112, the underwater electronic device is controlled to move based on the third navigation path.

[0122] Through steps 108 to 112, dynamic perception of changes in the underwater environment is achieved, real-time obstacle avoidance is performed, and the autonomous navigation capability of the underwater electronic device is further improved to adapt to complex underwater environments.

[0123] In some embodiments, after step 112, when an unmarked obstacle is monitored, the statistical number of unmarked obstacles is updated; when it is determined that the underwater electronic device successfully avoids the unmarked obstacle, the number of successful avoidance is updated; based on the statistical number and the number of successful avoidance, the probability of successful avoidance is determined; when the probability of successful avoidance is less than a preset probability threshold, the resolution of the sensor used to collect underwater environment data is increased, or the obstacle avoidance algorithm used by the underwater electronic device is updated.

[0124] In some embodiments, during the movement of the underwater electronic device based on the first navigation path, the actual movement path of the underwater electronic device can also be recorded, and the path deviation is determined based on the actual movement path and the first navigation path; when the path deviation is greater than a preset deviation threshold, the calibration frequency of the sensor used to collect data is increased, the path planning algorithm is updated, the SLAM algorithm is improved, or the filtering algorithm used for data fusion is optimized.

[0125] In some embodiments, referring to Figure 5 , before controlling the movement of the underwater electronic device, steps 201 to 205 can also be performed, which will be described below in combination with Figure 5 .

[0126] In step 201, a first power value of the underwater electronic device is obtained.

[0127] In some embodiments, the power value can be expressed in mAh (milliampere-hour), Wh (watt-hour), power remaining percentage, or icon form. Here, the icon form refers to using an icon to represent the power, and the filled part of the icon represents the remaining amount of power. As the power decreases, the filled part also decreases accordingly.

[0128] As an example of step 201, it is assumed here that the power usage percentage is represented, and the first power value is 100%.

[0129] It should be noted that the power representation is not unique, and in general, the icon form will be combined with other power value representation methods to facilitate intuitive perception of power changes.

[0130] In step 202, during the movement of the underwater electronic device, when it is determined that the power collection condition is met, a second power value of the underwater device is obtained.

[0131] In some embodiments, the power collection condition can be met when the task is completed or when a certain time is run for collection, or the power collection can be performed after sensing the environmental change. The power collection condition is not unique and can be set artificially or learned by the underwater electronic device.

[0132] As an example of step 202, it is assumed that the power acquisition is performed after the underwater electronic device works for 2 hours, and the second power value at this time is 50%.

[0133] In step 203, the power consumption rate of the underwater electronic device is determined based on the first power value and the second power value.

[0134] In step 204, it is determined whether the power consumption rate is greater than a preset rate threshold.

[0135] Wherein, when the power consumption rate is greater than the preset rate threshold, step 205 is entered; when the power consumption rate is less than or equal to the rate threshold, step 202 can be entered.

[0136] In step 205, the first prompt information of the power consumption being too fast is sent.

[0137] In some embodiments, the rate threshold of the power consumption rate is set, the performance of the battery is evaluated, if the power consumption rate is too large, a battery with a lower power consumption rate can be considered to be replaced, or the energy management algorithm of the underwater electronic device can be optimized to improve the battery use efficiency; the content of the first prompt information of the power consumption being too fast includes the current power consumption rate, the current power value and the prompt information.

[0138] In some embodiments, the underwater electronic device can also be monitored in real time using sensors, and the state information and environmental information of the underwater robot during navigation can be collected, if abnormal conditions or task target changes occur, the system can timely send an alarm or adjust the navigation strategy; the abnormal conditions here can be that the underwater electronic device detects an unmarked obstacle on the predetermined path, the underwater electronic device will immediately stop and alarm, re-plans the path by using an obstacle avoidance algorithm, bypasses the obstacle, and records the obstacle position to update the map data; it can also be a sensor failure, low battery, navigation system error, etc.

[0139] Here, the water quality sensor in the sensor will collect water quality data in real time, including physical and chemical parameters of the water body, such as temperature, pH value, dissolved oxygen, etc., evaluate the underwater water quality environment, and evaluate the environment, if the evaluation result is not suitable for the underwater electronic device to work, the current work will be immediately ended and the underwater electronic device will return.

[0140] Through real-time monitoring, the state information of the underwater electronic device can be obtained, and the abnormal conditions of the underwater electronic device can be processed in time, so that the underwater operation can be safely and efficiently carried out.

[0141] Below, an exemplary application of the embodiments of the present application in an actual application scenario will be described. The navigation method for an underwater environment provided by the embodiments of the present application can be implemented through the following five parts:

[0142] The first part is target detection and recognition. Deep learning target detection and recognition is an important technology in the design of underwater robot navigation systems. By using deep learning algorithms, robots can learn and recognize target objects in underwater environments, such as marine life, potential obstacles, etc. This can help robots make appropriate decisions during navigation, avoiding collisions with target objects or interacting with them. Deep learning models can be trained through a large amount of labeled data to improve the accuracy of target detection and recognition.

[0143] The second part is path planning. Deep learning path planning is a key technology for efficient underwater navigation. By analyzing the characteristics and historical data of the underwater environment, robots can learn and predict the best navigation path. Deep learning models can learn the patterns of environmental characteristics and path selection from large-scale underwater navigation data, and then select the optimal path according to the current task and conditions. Such path planning methods can improve the efficiency and safety of navigation.

[0144] The third part is sensor fusion. Sensor fusion is one of the key technologies for precise underwater navigation. Underwater robots are usually equipped with multiple sensors, such as sonar, cameras, inertial navigation systems, etc. Through deep learning technology, the data of these sensors can be fused and optimized to improve the robot's perception and understanding of the underwater environment. Deep learning models can learn the relevance and weight distribution between sensor data, thereby achieving more accurate navigation and positioning.

[0145] The fourth part is dynamic environment modeling. Dynamic environment modeling is also one of the key technologies in the design of underwater robot navigation systems. The underwater environment often has changes and uncertainties, such as water flow, movement of marine life, etc. Through deep learning technology, robots can model and predict the dynamic changes of the environment in real time to adapt to and cope with different underwater navigation challenges. Deep learning models can learn the patterns and trends of environmental changes from historical data, and then perform real-time environment modeling and prediction.

[0146] The fifth part is autonomous control. Autonomous control is a key technology to realize the autonomy of underwater robot navigation system. Through deep learning algorithm, the robot can learn and optimize the autonomous control strategy to realize more accurate and stable underwater navigation. Deep learning model can analyze and make decisions on environment and tasks in real time, thereby improving the autonomous navigation ability of the robot. This autonomous control method can make the robot have adaptability and flexibility in complex underwater environment. The application of these technologies can improve the navigation ability and autonomy of underwater robots in complex underwater environment, bringing important application value to underwater exploration, marine scientific research and resource development, etc.

[0147] Traditional underwater electronic device navigation relies on manual operation, manual underwater operation is dangerous, the path setting method is complex, and it can only follow the established route, lacking real-time adjustment capability; the number of sensors carried is limited, and the collected data information is less. These defects increase the operation difficulty, affect the normal operation of the navigation system, and limit the autonomous navigation ability of the underwater electronic device in complex environment.

[0148] Based on this, the embodiment of the present application provides a kind of underwater electronic device navigation system design based on machine learning, underwater electronic device carries multiple sensors to collect underwater data, carries out map construction and path planning, and in the design of navigation system, server of machine learning algorithm is applied and deployed, server is marked and annotated to the underwater data collected by multiple sensors, establishes training data set, for training machine learning model, so that underwater electronic device can carry out path planning and obstacle avoidance according to underwater environment in real time, reduce artificial intervention, reduce the danger of manual underwater operation, improve the autonomy of underwater electronic device navigation system, so as to adapt to complex and changeable underwater environment, improve work efficiency. The embodiment of the present application provides a kind of underwater electronic device navigation system design based on machine learning, which can be applied to marine scientific research, marine resource development and marine environment monitoring tasks.

[0149] Reference Figure 6 , Figure 6 It is the schematic diagram of the business process of the navigation of underwater environment provided by the embodiment of the present application. The implementation process of the navigation method of underwater environment provided by the embodiment of the present application will be described below. Figure 6 The implementation process of the navigation method of underwater environment provided by the embodiment of the present application will be described below.

[0150] Step 301, data acquisition and pretreatment. Underwater electronic device collects underwater environment data through sensors carried, including camera, sonar, inertial navigation system, depth sensor, water quality sensor, including image data, echo data, inertial navigation data and water quality data, etc. Information, pretreatment operation is carried out on data, including denoising, image enhancement, and data fusion and calibration are carried out using Kalman filter.

[0151] Data calibration mainly includes four aspects. The first aspect is camera calibration, which uses a checkerboard or calibration board to calibrate the camera's intrinsic and extrinsic parameters. The second aspect is sonar calibration, which adjusts the sonar parameters to match the actual situation by using known distances and objects. The third aspect is inertial navigation system calibration, which includes static calibration such as six-face calibration method and dynamic calibration such as rotating table method to obtain the offset and scale factor of accelerometer and gyroscope. The fourth aspect is depth sensor calibration, which adjusts the measurement value to match the true depth in an environment with known depth.

[0152] Water quality sensors are mainly used to monitor physical and chemical parameters of water bodies, such as temperature, pH value, dissolved oxygen, etc. In the navigation system of underwater robots, the data of water quality sensors can be used for environmental assessment, task planning, and ensuring that the robot works in suitable conditions. It is worth noting that the data collection frequency is 10 seconds per time, the coverage of image acquisition is within 20 meters in front of the camera, and the coverage of echo data acquisition is within 50 meters around the sonar.

[0153] Step 302, data labeling and annotation. Use data labeling tools to label the preprocessed data with underwater object types, establish a training data set, and use it for model training and verification. The types of objects include biological objects such as all aquatic organisms, such as fish, coral, shellfish, seaweed, etc., and non-biological objects such as rocks, sunken ships, artificial structures underwater pipelines, platforms, etc. In some specific scenarios, more detailed classification is required. Data labeling uses LabelMe or VGG Image Annotator for image labeling, and sonar data labeling is done through custom tools, labeling target objects and obstacles. The labeled categories are underwater objects such as rocks, sunken ships, marine life, etc., obstacles such as waste, other underwater electronic devices, etc., and terrain such as sand, coral reefs, etc.

[0154] Step 303, model training and optimization. Use the training data set to train and optimize the machine learning model, iteratively train and adjust the parameter model, improve the model's ability to perceive and recognize in underwater environment. The pre-trained model is deployed in underwater electronic devices for real-time perception and path planning. The training process of the machine learning model is as follows: the image data collected by the camera is labeled using LabelMe or VGG Image Annotator (VIA), a machine learning model is trained to recognize object types in underwater images such as fish, coral, rock, etc. The input data is an RGB image with a size of 224*224 pixels and a channel number of 3, containing red, green and blue. The output is a pre-trained machine learning model; target detection is achieved using the YOLOv5 algorithm, the training data set is 5000 labeled underwater image data sets, the Adam algorithm is used to optimize and update the parameters of the target detection model, and the learning rate is set to 0.001; path planning is achieved using the deep reinforcement learning model DQN, inputting the underwater environment data fused by multiple sensors, training 100000 steps, using a reward function based on distance to target and collision, dynamically adjusting the training strategy.

[0155] Step 304, environment perception and map construction. Use machine learning models to analyze and process real-time collected data to achieve perception and understanding of underwater environment; identify and classify underwater objects and obstacles, and construct real-time underwater map. Among them, objects refer to all identifiable entities in underwater environment, including natural and man-made entities such as fish, coral, rock, wreck, etc. Objects can be task targets such as fish for biological monitoring, coral and rock for environmental feature terrain features; obstacles refer to objects that need to be avoided during underwater electronic device task execution, which may hinder the movement of underwater electronic device or cause damage, such as large rocks, waste, other submersibles, etc. Obstacles are objects that need to be avoided during navigation to ensure safety and smooth task execution. The underwater map can be a grid map, a topological map or a three-dimensional point cloud map. The grid map divides the underwater environment into equal-sized grids, each grid contains environmental information such as whether there are objects and obstacles, each grid (cell) can be represented as "occupied" or "free", by fusing sonar data and camera image data, marking the state of each grid. The underwater map can be a topological map, which defines nodes and edges in the topological map. The underwater map can also be a three-dimensional point cloud map, which generates a three-dimensional environment model by collecting and splicing point cloud data. As the underwater electronic device moves, the environment map is continuously updated, and an optimization algorithm is used to reduce errors and improve map accuracy. The constructed map data is stored locally or in the cloud, and a visualization tool is used to display the environment map, supporting navigation and analysis.

[0156] The process of analysis and processing is first to collect data, the camera collects underwater RGB images, the sonar collects echo data, the inertial navigation system collects position information and attitude data, denoising and image enhancement are performed on the collected data, sonar data is used for filtering processing, sensor data synchronization and calibration are performed, a pre-trained convolutional neural network (CNN) is used for target detection and classification of images; sonar data is used for obstacle detection and distance measurement; the inertial navigation system data is used for position information correction; the analysis and processing results include target object position and category, which are obtained from image data, and are coordinate positions in the image, represented by [x, y, width, height]. The category of the object is marked as fish, coral, rock, etc.; the position and distance of the obstacle are obtained from the sonar data, where the position is the distance and direction of the obstacle relative to the underwater electronic device, and the distance is the distance between the obstacle and the underwater electronic device, such as 20 meters; the position information and attitude data are obtained from the inertial navigation system (INS) data, the position data is the specific coordinate [x, y, z] of the underwater electronic device in the water, and the attitude data is the attitude information [roll, pitch, yaw] of the robot. Relative distance is usually in meters (m). Relative to the orientation of the robot, the azimuth (azimuth) and elevation (elevation) are used to describe. For example: the obstacle is 1 meter away from the robot and located at a southeast direction of 45° elevation and 30°. In practical applications, both description methods are used. For example, relative distance is used for real-time obstacle avoidance, and coordinate form is used for map construction and path planning.

[0157] In the process of underwater map construction, first, data fusion is performed, and Kalman filtering and other algorithms are used to fuse the data of the camera, sonar and inertial navigation system to obtain more accurate environmental perception information. Second, the environmental perception results are integrated, the position and category of the target object, the position and distance of the obstacle, the position information and attitude data of the underwater electronic device are integrated together to form a unified environmental perception result. Finally, map construction is performed, and simultaneous localization and mapping (SLAM) technology is used to construct a real-time underwater map according to the environmental perception result. The SLAM algorithm can be vision-based, such as ORB-SLAM2 or sonar-based, such as sonar-based SLAM.

[0158] Step 305, path planning and navigation. Based on the underwater map and perception results, the navigation system plans the travel path of the underwater electronic device, adjusts the path in real time to avoid obstacles, selects the optimal path, and avoids potential risks. Among them, the perception results are useful information extracted from the underwater environment through machine learning models and multi-sensor fusion technology, including: the category of objects such as fish, coral, rock, etc., the size, shape, and position and coordinate information of the object. The map is a static description of the underwater environment, including all known objects, obstacles and environmental features grid map. The path is the travel route of the underwater electronic device from the starting point to the end point planned on the map. When planning the path, the environmental information on the map should be considered to avoid obstacles during navigation.

[0159] According to the perception results, the underwater map is constructed or updated. Using the simultaneous localization and mapping (SLAM) technology, the real-time perception results are integrated into the map. The classic path planning algorithm Dijkstra algorithm is used for path planning. Combined with the kinematic model of the underwater electronic device and the dynamic obstacle avoidance algorithm, considering the mass, friction, inertia and other factors of the underwater electronic device, it is suitable for more complex motion control, and ensures that the planned path is physically feasible and meets the motion limits of the underwater electronic device, such as maximum speed, turning radius, etc. The preliminary planned path is optimized to ensure the feasibility and safety of the path. During the movement of the underwater electronic device, the environmental changes are continuously perceived, and the path is adjusted in real time to deal with new obstacles or environmental changes.

[0160] Suppose the underwater robot is performing a task and needs to move from starting point A to target point B. The preliminary path planning has generated a path. However, during the movement process, the robot detects that a new obstacle such as a moving rock appears in front. At this time, the system will adjust the path according to the following steps: environmental perception: the robot detects the position and size of the rock through sonar and camera. Map update: update the environment map using SLAM technology, mark the position of the newly appeared rock. Re-plan the path: use A* algorithm to re-plan the path according to the updated map, avoid new obstacles. Path optimization: smooth and feasibility evaluation of the new path to ensure that the robot can safely pass through. Execute adjustment: the robot adjusts its motion parameters according to the new path and continues to move towards the target point B.

[0161] Step 306, real-time monitoring. The navigation system monitors the state information of the underwater electronic device and the environmental information in real time through sensors. If there is an abnormal situation or a change in the target, the system can issue an alarm or adjust the navigation strategy in time. The state information of the underwater electronic device includes position, attitude, speed, acceleration, battery status, system temperature, and the environmental information includes water depth, water flow speed and direction, ambient light intensity, etc. Abnormal situations include detecting unmarked obstacles, sensor failure, low battery, navigation system error, etc. For example, if the underwater electronic device detects an unmarked obstacle on the planned path, it may affect navigation safety, so the underwater electronic device will stop immediately and alarm, use obstacle avoidance algorithm to re-plan path, bypass the obstacle, and record the obstacle position, update the map data. Real-time monitoring uses real-time video streaming, uses H.264 encoding, and sensor data is uploaded to the monitoring center in real time, with an update frequency of 10 times per second.

[0162] Step 307, data storage and analysis. The data during navigation is stored and analyzed for subsequent performance evaluation and improvement. Storage uses Hadoop or Spark for big data processing, storing navigation data, sensor data and video data for subsequent analysis. A large amount of collected data, including sensor and system information, is uploaded and stored in the cloud, and a server deploying machine learning algorithms cleans and preprocesses the data, evaluates the performance of the underwater electronic device through data analysis, including the following indicators: navigation accuracy: comparing the deviation between the planned path and the actual path; obstacle avoidance success rate: the ratio of the number of times the underwater electronic device successfully avoids obstacles to the total number of times obstacles are detected; battery usage efficiency: battery consumption per unit time or unit distance; analysis method uses Python and R for data analysis and model evaluation, and data visualization tools use Matplotlib and Seaborn.

[0163] The server deploying machine learning algorithms makes improvement suggestions based on the results obtained by analyzing the performance of the underwater electronic device. If the navigation accuracy error is large, consider improving the SLAM algorithm or increasing the sensor calibration frequency or optimizing the filtering algorithm. If the obstacle avoidance success rate is low, optimize the obstacle avoidance algorithm or increase the sensor resolution or add sensors and perform multi-sensor fusion. If the battery usage efficiency is too high, consider replacing the battery with a lower power consumption or optimizing the energy management algorithm of the underwater electronic device to improve battery usage efficiency.

[0164] Reference Figure 7 , Figure 7FIG. 1 is a structural schematic diagram of a second architecture of a navigation system for an underwater environment provided by an embodiment of the present application. The sensor 401 is deployed on the underwater electronic device 400, and data acquisition, preprocessing, environment perception, map construction, path planning, and navigation are performed by the underwater electronic device. The main body of the improved execution is performed on the server side 402, the navigation system is deployed inside the robot, and the map construction and path planning are both generated in real time by pre-trained models. Data labeling and annotation, model training and optimization, real-time monitoring, data storage and analysis are completed by the server. The map construction and path planning are both generated in real time by pre-trained models.

[0165] The underwater environment navigation system based on deep learning provided by the embodiments of the present application brings significant benefits to enterprises in many aspects, and has important significance for the application and promotion of underwater electronic devices. Traditional underwater navigation systems usually rely on manually set paths or remote control instructions. The navigation system based on deep learning can realize autonomous path planning and decision-making, enabling underwater robots to independently complete tasks. This can reduce the need for human intervention and improve the autonomy and efficiency of underwater robots. Deep learning algorithms can analyze and process underwater images and sensor data in real time, enabling the perception and understanding of underwater environments. This enables underwater robots to better recognize and classify underwater objects and obstacles, and to construct real-time underwater maps. This improved environmental perception capability helps underwater robots better adapt to complex and variable underwater environments. Deep learning enables intelligent path planning and dynamic obstacle avoidance, enabling underwater robots to complete tasks more efficiently. Through analysis of the underwater environment by deep learning algorithms, the robot can choose the optimal path, avoid obstacles, and adjust in real time according to task requirements. This improves the ability and flexibility of the robot to perform tasks.

[0166] Traditional underwater tasks usually require human divers to operate, which poses certain risks and costs. The underwater robot navigation system based on deep learning can reduce the dependence on human divers, thereby reducing the risks and costs of diving operations. Underwater robots can replace humans to perform dangerous tasks while providing more stable and reliable execution capabilities. Through the navigation method for an underwater environment provided by the embodiments of the present application, not only the navigation capability of underwater robots is improved, but also the application and development of deep learning technology in the underwater field are promoted. By continuously improving and optimizing deep learning algorithms, the perception, decision-making, and navigation capabilities of underwater robots can be further improved, bringing more innovation and progress to the fields of underwater scientific research, marine resource development, and environmental protection. In addition, it also brings about the improvement of navigation accuracy, the enhancement of task execution capability, the reduction of risks and costs, and the promotion of scientific research and application development. These beneficial effects enable underwater robots to better adapt to complex underwater environments.

[0167] The following description continues to illustrate the exemplary structure of the underwater environment navigation device 455 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2 As shown, the software modules stored in the underwater environment navigation device 455 in the memory 450 may include: a data acquisition module 4551, used to acquire real-time underwater environment data, the underwater environment data including at least image data, sonar data, and inertial navigation data; a map construction module 4552, used to preprocess the underwater environment data using a pre-trained machine learning model to obtain environmental perception results, the environmental perception results including the location and category information of underwater obstacles, the first current position and attitude information of the underwater electronic device; and to construct an underwater map based on the location and category information of the underwater obstacles; and a first path planning module 4553, used to acquire the underwater electronic device's... The target location is determined by performing path planning based on the first current location, the target location, and the underwater map to obtain a first navigation path. A feasibility assessment module 4554 is used to acquire the size information and turning radius of the underwater electronic device, as well as the minimum path width, minimum path length, and minimum turning width of the first navigation path. When the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius, the feasibility assessment result of the first navigation path is determined to be passed. A navigation control module 4555 is used to control the underwater electronic device to move based on the first navigation path.

[0168] In some embodiments, the map building module 4552 is further configured to align the image data, the sonar data, and the inertial navigation data to obtain fused data, the fused data including aligned image data, aligned sonar data, and aligned inertial navigation data; based on the aligned image data, call the pre-trained machine learning model to perform prediction processing to obtain category information of underwater obstacles; based on the aligned sonar data, call the pre-trained machine learning model to perform prediction processing to obtain position information of underwater obstacles; based on the aligned inertial navigation data, call the pre-trained machine learning model to perform prediction processing to obtain attitude information and a first current position of the underwater electronic device; and determine one or more features from the position information, category information, attitude information, and first current position of the underwater obstacle as environmental perception results.

[0169] In some embodiments, the map construction module 4552 is further configured to acquire initial environment information, and generate an initial grid map based on the initial environment information; determine a first target grid corresponding to the underwater obstacle in the initial grid map based on position information and category information of the underwater obstacle, the position information comprising a distance and a direction of the underwater obstacle relative to the underwater electronic device, and the category information being used to represent whether the underwater obstacle is a living body or a non-living body; set a state corresponding to the first target grid in the initial grid map to an occupied state to obtain the underwater map.

[0170] In some embodiments, the feasibility evaluation module 4554 is further configured to determine that the feasibility evaluation result is evaluation passed when the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius; and determine that the feasibility evaluation result is evaluation failed when at least one of the following conditions is met: the minimum path width is less than or equal to the device width, the minimum path length is less than or equal to the device length, and the minimum turning width is less than or equal to the turning radius.

[0171] In some embodiments, the navigation device for the underwater environment further comprises a second path planning module configured to, when the feasibility evaluation result is evaluation failed, acquire a first sub-path and a second sub-path in the first navigation path, the first sub-path being a sub-path that does not need to be optimized, and the second sub-path being a sub-path that needs to be optimized; perform path planning based on a starting position of the second sub-path, the target position, and the underwater map to obtain a third sub-path; acquire size information and a turning radius of the underwater electronic device, a minimum path width of the third sub-path, a minimum path length of the third sub-path, and a minimum turning width in the third sub-path, and determine a feasibility evaluation result of the third sub-path; when the feasibility evaluation result of the third sub-path is evaluation passed, combine the first sub-path and the third sub-path to obtain a second navigation path; and control the underwater electronic device to move based on the second navigation path.

[0172] In some embodiments, the navigation device of the underwater environment further comprises a third path planning module configured to, during movement of the underwater electronic device based on the first navigation path, acquire underwater environment data collected in real time; perform prediction processing on the underwater environment data by using the pre-trained machine learning model, determine that an unmarked obstacle is monitored, control the underwater electronic device to stop moving, the unmarked obstacle being an obstacle that is not marked in the underwater map; acquire position information of the unmarked obstacle and a second current position of the underwater electronic device, and perform path planning again according to the underwater map to obtain a third navigation path; and control the underwater electronic device to move based on the third navigation path.

[0173] In some embodiments, the third path planning module is further configured to, based on the position information of the unmarked obstacle, determine a second target grid corresponding to the unmarked obstacle in the underwater map; update a state corresponding to the second target grid in the underwater map to an occupied state to obtain an updated underwater map; and perform path planning again based on the second current position, the target position, and the updated underwater map to obtain the third navigation path.

[0174] In some embodiments, the navigation device of the underwater environment further comprises an obstacle avoidance optimization module configured to, when the unmarked obstacle is monitored, update a statistical number of unmarked obstacles; when it is determined that the underwater electronic device successfully avoids the unmarked obstacle, update a successful avoidance number; based on the statistical number and the successful avoidance number, determine a successful avoidance probability; and when the successful avoidance probability is less than a preset probability threshold, increase a resolution of a sensor used to collect underwater environment data, or update an obstacle avoidance algorithm used by the underwater electronic device.

[0175] In some embodiments, the navigation device of the underwater environment further comprises a path deviation correction module configured to, during movement of the underwater electronic device based on the first navigation path, record an actual movement path of the underwater electronic device; determine a path deviation based on the actual movement path and the first navigation path; and when the path deviation is greater than a preset deviation threshold, increase a calibration frequency of a sensor used to collect data or update a path planning algorithm.

[0176] In some embodiments, the navigation device for underwater environment further comprises a power prompt module, configured to acquire a first power value of the underwater electronic device before controlling the underwater electronic device to move; acquire a second power value of the underwater electronic device when it is determined that the power collection condition is met during the control of the movement of the underwater electronic device; determine a power consumption rate of the underwater electronic device based on the first power value and the second power value; and send a first prompt information of power consumption being too fast when the power consumption rate is greater than a preset rate threshold. When the power consumption rate is less than or equal to the rate threshold, a third power value of the underwater electronic device when it reaches the target position is determined based on the power consumption rate; and a second prompt information of power remaining being insufficient is sent when the third power value is less than a preset power threshold.

[0177] In some embodiments, the navigation device for underwater environment further comprises a water quality evaluation module, configured to acquire water quality data collected by a water quality sensor; perform environment evaluation by using the water quality data to obtain an environment evaluation result; and control the underwater electronic device to return when the environment evaluation result is that the evaluation is failed.

[0178] Embodiments of the present application provide a computer program product, which comprises a computer program or computer executable instructions stored in a computer readable storage medium. The computer executable instructions are read by a processor of an underwater electronic device from the computer readable storage medium, and the processor executes the computer executable instructions to cause the underwater electronic device to perform the navigation method for underwater environment provided by the embodiments of the present application.

[0179] Embodiments of the present application provide a computer readable storage medium, which stores computer executable instructions or computer programs. When the computer executable instructions or computer programs are executed by a processor, the processor will execute the navigation method for underwater environment provided by the embodiments of the present application.

[0180] In some embodiments, the computer readable storage medium can be RAM, ROM, flash memory, magnetic surface memory, optical disc, or CD-ROM memory, etc. It can also be various devices comprising one or any combination of the above-mentioned memories. The computer executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0181] By way of example, computer executable instructions can, but need not, reside in a file system's files, can, but need not, be stored on a file system's file, can, but need not, be stored in a part of a file that is used by the operating system, applications, or other software programs, or in a file that has been stored in a location based on a file type (e.g., files stored with other source or object code files). Computer executable instructions can, but need not, be stored in files dedicated to the program executing them, or in files shared with other computer programs or with temporary files. Computer executable instructions can, but need not, be stored in the memory of or on the data storage of a computer, or in a computer readable medium. Computer executable instructions can, but need not, be written in a compiled or interpreted language. Computer executable instructions may, but need not, be written in machine code, assembly language, object code, or source code.

[0182] To sum up, by means of the embodiments of the present application, the underwater electronic device can perform path planning and obstacle avoidance in real time according to the underwater environment, reduce manual intervention, reduce the danger of manual underwater operation, and improve the autonomy of the navigation system of the underwater electronic device, so as to adapt to the complex and changeable underwater environment and improve work efficiency.

[0183] The above merely illustrates the embodiments of the present application, but is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, and improvement within the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. A method of navigation in an underwater environment, characterized by, The method comprises: acquiring real-time collected underwater environment data, the underwater environment data at least comprising image data, sonar data and inertial navigation data; using a pre-trained machine learning model to preprocess the underwater environment data to obtain an environment perception result, the environment perception result comprising position information and category information of underwater obstacles, first current position and attitude information of underwater electronic equipment; constructing an underwater map based on the position information and category information of the underwater obstacles; acquiring a target position of the underwater electronic equipment, and performing path planning based on the first current position, the target position and the underwater map to obtain a first navigation path; acquiring size information and turning radius of the underwater electronic equipment, and minimum path width, minimum path length of the first navigation path and minimum turning width in the first navigation path, and determining that a feasibility evaluation result of the first navigation path is evaluation pass when the minimum path width is greater than the width of the underwater electronic equipment, the minimum path length is greater than the length of the underwater electronic equipment, and the minimum turning width is greater than the turning radius; controlling the underwater electronic equipment to move based on the first navigation path.

2. The method of claim 1, wherein, The pre-trained machine learning model is trained by using underwater environment training data through a back propagation algorithm, and the preprocessing of the underwater environment data by using the pre-trained machine learning model to obtain the environment perception result comprises: aligning the image data, the sonar data and the inertial navigation data to obtain fusion data, the fusion data comprising aligned image data, aligned sonar data and aligned inertial navigation data; based on the aligned image data, calling the pre-trained machine learning model to perform prediction processing to obtain category information of underwater obstacles; based on the aligned sonar data, calling the pre-trained machine learning model to perform prediction processing to obtain position information of the underwater obstacles; based on the aligned inertial navigation data, calling the pre-trained machine learning model to perform prediction processing to obtain attitude information and first current position of the underwater electronic equipment; determining one or more features in the position information, the category information, the attitude information and the first current position of the underwater obstacles as the environment perception result.

3. The method of claim 1, wherein, The construction of the underwater map based on the position information and category information of the underwater obstacles comprises: acquiring initial environment information, and generating an initial grid map based on the initial environment information; determining a first target grid corresponding to the underwater obstacle in the initial grid map based on the position information and category information of the underwater obstacle; the position information comprises distance and direction of the underwater obstacle relative to the underwater electronic equipment, and the category information is used to represent whether the underwater obstacle is a living body or a non-living body; setting a state corresponding to the first target grid in the initial grid map to an occupied state to obtain the underwater map.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: When the feasibility evaluation result is evaluation failure, a first sub-path and a second sub-path in the first navigation path are obtained, the first sub-path being a sub-path that does not need to be optimized, and the second sub-path being a sub-path that needs to be optimized; path planning is performed based on a starting position of the second sub-path, the target position, and the underwater map, to obtain a third sub-path; a feasibility evaluation result of the third sub-path is determined based on size information and a turning radius of the underwater electronic device, and a minimum path width, a minimum path length, and a minimum turning width in the third sub-path; when the feasibility evaluation result of the third sub-path is evaluation success, the first sub-path and the third sub-path are combined to obtain a second navigation path; the underwater electronic device is controlled to move based on the second navigation path.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: during movement of the underwater electronic device based on the first navigation path, underwater environment data collected in real time is obtained; the underwater environment data is processed by using the pre-trained machine learning model, to determine that an unmarked obstacle is monitored, the underwater electronic device is controlled to stop moving, and the unmarked obstacle is an obstacle that is not marked in the underwater map; position information of the unmarked obstacle and a second current position of the underwater electronic device are obtained, and path planning is performed again based on the underwater map to obtain a third navigation path; the underwater electronic device is controlled to move based on the third navigation path.

6. The method of claim 5, wherein, The path planning based on the underwater map again to obtain the third navigation path includes: based on the position information of the unmarked obstacle, a second target grid corresponding to the unmarked obstacle in the underwater map is determined; a state corresponding to the second target grid in the underwater map is updated to an occupied state to obtain an updated underwater map; based on the second current position, the target position, and the updated underwater map, path planning is performed again to obtain the third navigation path.

7. The method of claim 6, wherein, The method further includes: when the unmarked obstacle is monitored, a statistical number of unmarked obstacles is updated; when it is determined that the underwater electronic device successfully avoids the unmarked obstacle, a successful avoidance number is updated; based on the statistical number and the successful avoidance number, a successful avoidance probability is determined; when the successful avoidance probability is less than a preset probability threshold, a resolution of a sensor used to collect underwater environment data is improved, or an obstacle avoidance algorithm used by the underwater electronic device is updated.

8. The method according to any one of claims 1 to 4, characterized in that, The method further includes: before the underwater electronic device is controlled to move, a first power value of the underwater electronic device is obtained; during movement of the underwater electronic device, when it is determined that a power collection condition is met, a second power value of the underwater electronic device is obtained; based on the first power value and the second power value, a power consumption rate of the underwater electronic device is determined; when the power consumption rate is greater than a preset rate threshold, first prompt information that power is consumed too quickly is sent.

9. A navigation device for an underwater environment, characterized in that The device includes: The data acquisition module is configured to acquire underwater environment data collected in real time, wherein the underwater environment data at least includes image data, sonar data, and inertial navigation data. The map construction module is configured to pre-process the underwater environment data by using a pre-trained machine learning model to obtain environment perception results, wherein the environment perception results include position information and category information of underwater obstacles, first current position and attitude information of underwater electronic equipment; and construct an underwater map based on the position information and category information of the underwater obstacles. The first path planning module is configured to acquire a target position of the underwater electronic equipment, and perform path planning based on the first current position, the target position, and the underwater map to obtain a first navigation path. The feasibility evaluation module is configured to acquire size information and turning radius of the underwater electronic equipment, and minimum path width, minimum path length of the first navigation path, and minimum turning width in the first navigation path; and determine that a feasibility evaluation result of the first navigation path is evaluation pass when the minimum path width is greater than a width of the underwater electronic equipment, the minimum path length is greater than a length of the underwater electronic equipment, and the minimum turning width is greater than the turning radius. The navigation control module is configured to control the underwater electronic equipment to move based on the first navigation path.

10. An underwater electronic device, characterized by The underwater electronic equipment includes: A memory configured to store computer executable instructions. A processor configured to execute the computer executable instructions stored in the memory to implement the method in any one of claims 1 to 8.

11. A computer-readable storage medium storing computer-executable instructions, characterized in that, The computer executable instructions, when executed by the processor, implement the method in any one of claims 1 to 8.

Images