Navigation method and device for underwater environment, underwater electronic equipment and computer readable storage medium

Abstract

The invention provides a navigation method and device for an underwater environment, underwater electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring underwater environment data collected in real time; preprocessing the underwater environment data by using a pre-trained machine learning model to obtain an environment sensing result; constructing an underwater map based on the position information and the category information of the underwater obstacles; acquiring a target position of the underwater electronic equipment, and performing path planning based on the current position, the target position and the underwater map to obtain a first navigation path; and controlling the underwater electronic equipment to move based on the navigation path when determining that the feasibility evaluation result of the first navigation path is that the evaluation is passed based on the size information and the steering radius of the underwater electronic equipment, the minimum path width and the minimum path length of the first navigation path and the minimum steering width in the first navigation path. According to the invention, the artificial dependence of underwater environment navigation can be reduced, the navigation precision is improved, and autonomous navigation of an underwater scene is realized.

Description

Technical Field

[0001] The present application relates to navigation technology, and in particular to a navigation method, device, underwater electronic equipment and computer-readable storage medium in an underwater environment. Background Art

[0002] In the related technologies, the navigation of traditional underwater electronic equipment relies on manual work. Manual underwater operations are highly dangerous, the path setting method is relatively complicated, and underwater electronic equipment only follows a predetermined route and lacks the ability to adjust in real time. These defects increase the difficulty of operation, affect the navigation accuracy, and limit the autonomous navigation ability of underwater electronic equipment in complex underwater environments. Summary of the invention

[0003] The embodiments of the present application provide a navigation method, device, underwater electronic equipment and computer-readable storage medium for an underwater environment, which can improve navigation accuracy and realize autonomous navigation of underwater scenes.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a method for navigating an underwater environment, the method comprising:

[0006] Acquiring underwater environmental data collected in real time, wherein the underwater environmental data at least includes image data, sonar data and inertial navigation data;

[0007] Preprocessing the underwater environment data using a pretrained machine learning model to obtain an environment perception result, wherein the environment perception result includes location information and category information of underwater obstacles, and a first current location and posture information of the underwater electronic device;

[0008] Building an underwater map based on the location information and category information of the underwater obstacles;

[0009] Acquire a target position of the underwater electronic device, perform path planning based on the first current position, the target position and the underwater map, and obtain a first navigation path;

[0010] Acquire the size information and turning radius of the underwater electronic device, as well as the minimum path width, the minimum path length, and the minimum turning width in the first navigation path of the first navigation path, and determine that the feasibility assessment result of the first navigation path is passed when the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius;

[0011] The underwater electronic device is controlled to move based on the first navigation path.

[0012] The present application provides a navigation device for an underwater environment, including:

[0013] A data acquisition module, used to acquire underwater environmental data collected in real time, wherein the underwater environmental data at least includes image data, sonar data and inertial navigation data;

[0014] A map construction module, used to pre-process the underwater environment data using a pre-trained machine learning model to obtain an environment perception result, wherein the environment perception result includes location information and category information of underwater obstacles, and a first current location and posture information of underwater electronic equipment; and construct an underwater map based on the location information and category information of the underwater obstacles;

[0015] A first path planning module, used to obtain a target position of the underwater electronic device, perform path planning based on the first current position, the target position and the underwater map, and obtain a first navigation path;

[0016] a feasibility assessment module, configured to obtain the size information and the turning radius of the underwater electronic device, as well as the minimum path width, the minimum path length and the minimum turning width in the first navigation path of the first navigation path, and determine that the feasibility assessment result of the first navigation path is passed when the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius;

[0017] The navigation control module is used to control the underwater electronic device to move based on the first navigation path.

[0018] The present application provides an underwater electronic device, including:

[0019] A memory for storing computer executable instructions;

[0020] The processor is used to implement a navigation method for an underwater environment provided in an embodiment of the present application when executing the computer executable instructions stored in the memory.

[0021] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing a navigation method for an underwater environment provided by an embodiment of the present application when executed by a processor.

[0022] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, a navigation method for an underwater environment provided in an embodiment of the present application is implemented.

[0023] The embodiments of the present application have the following beneficial effects:

[0024] The embodiment of the present application realizes a navigation method for an underwater environment. After acquiring underwater environmental data collected in real time, a pre-trained machine learning model is used to perform prediction processing based on the underwater environmental data to obtain an environmental perception result, thereby realizing accurate perception and understanding of the underwater real-time environment, so that underwater electronic equipment can better identify and classify underwater objects and obstacles. This improved environmental perception capability helps underwater electronic equipment to better adapt to complex and changeable underwater environments; then a real-time underwater map is constructed based on the environmental perception result, and the current position and target position of the underwater electronic equipment are obtained according to the underwater map, and a path is planned based on the current position, the target position and the underwater map to obtain a navigation path, thereby controlling the underwater electronic equipment to move based on the navigation path. Since the underwater map is a static description of the underwater environment and a grid map including all known obstacles and environmental features, the obstacles and environmental features on the underwater map need to be considered when performing path planning based on the underwater map. In this way, while realizing autonomous path planning of underwater electronic equipment to improve the autonomy and work efficiency of underwater electronic equipment, it can also ensure that a safe and efficient navigation path is obtained, thereby improving the safety and stability of underwater environment navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a structural diagram of a first architecture of a navigation system for an underwater environment provided in an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of the structure of a navigation device for an underwater environment provided by an embodiment of the present application;

[0027] Figure 3A It is a first flow chart of the underwater environment navigation method provided by an embodiment of the present application;

[0028] Figure 3B is a second flow chart of the method for obtaining an environmental perception result provided in an embodiment of the present application;

[0029] Figure 3C is a third flow chart of the map construction method provided in the embodiment of the present application;

[0030] Figure 3D is a fourth flow chart of the path planning method provided in an embodiment of the present application;

[0031] Figure 4 is a fifth flow chart of the method for re-planning a path provided in an embodiment of the present application;

[0032] Figure 5 This is a sixth flow chart of the power prompt method provided in an embodiment of the present application;

[0033] Figure 6It is a schematic diagram of the business process of navigation in an underwater environment provided by an embodiment of the present application;

[0034] Figure 7 It is a structural diagram of the second architecture of the navigation system for an underwater environment provided in an embodiment of the present application.

[0035] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of superiority or inferiority of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0037] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0038] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0039] A "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0040] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0041] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0042] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0043] 1) Underwater electronic equipment: electronic equipment that can autonomously or remotely control and perform various tasks in underwater environments. These devices combine technologies such as artificial intelligence, detection and identification, information fusion and intelligent control, and can replace or assist humans in completing underwater operations. For example, underwater drones, unmanned submersibles and underwater boosters are all underwater electronic equipment. They are widely used in many fields such as ocean exploration, archaeology, fisheries, rescue, environmental protection, etc., providing underwater monitoring, filming, search and rescue and other functions.

[0044] 2) Inertial Navigation System: It is an autonomous navigation system that does not rely on external information and does not radiate energy to the outside. It is based on Newton's laws of mechanics. By measuring the acceleration of the carrier in the inertial reference system and integrating it over time, it can infer information such as the carrier's speed, position, and attitude. For example, in fishing, fishermen use sonar systems to emit sound waves and receive echoes from fish schools to determine the location and number of fish schools. Sonar technology is widely used in ocean exploration, underwater mapping, ship navigation and other fields.

[0045] 3) Grid map: A grid-based map representation method that divides the map area into uniform grid cells and assigns specific attribute information to each grid cell, such as object type, height, obstacles, etc. This type of map is often used in fields such as autonomous driving, robot navigation, and game development. For example, in an autonomous driving system, a grid map can contain detailed information such as road geometry, lane boundaries, and traffic signs to help vehicles perceive and understand the road environment.

[0046] Traditional underwater electronic equipment navigation relies on manual work. Manual underwater operations are highly dangerous, and the path setting method is relatively complicated. As a result, underwater electronic equipment only follows a predetermined route and lacks the ability to make real-time adjustments. The number of sensors carried is limited, and less data and information are collected, which increases the difficulty of operation, affects the normal operation of the navigation system, and limits the autonomous navigation capabilities of underwater electronic equipment in complex underwater environments.

[0047] Specifically, an embodiment of the present application provides a design of an underwater electronic device navigation system based on machine learning. The underwater electronic device is equipped with a variety of sensors to collect underwater data, perform map construction and path planning, and apply a server deployed with a machine learning algorithm in the design of the navigation system. The server labels and annotates the underwater data collected by the various sensors, and establishes a training data set for training the machine learning model, so that the underwater electronic device can perform path planning and obstacle avoidance in real time according to the underwater environment, reduce manual intervention, reduce the danger of manual underwater operations, and enhance the autonomy of the underwater electronic device navigation system, so as to adapt to the complex and changeable underwater environment and improve work efficiency.

[0048] The embodiments of the present application provide a navigation method, device, underwater electronic device and computer-readable storage medium in an underwater environment, which can improve the autonomy of the underwater electronic device navigation system and adapt to the complex underwater environment. The exemplary application of the underwater electronic device provided by the embodiments of the present application is described below. The underwater electronic device provided by the embodiments of the present application can be implemented as various terminals applied to underwater scenes, such as underwater drones, underwater robots, and deep-sea detectors. The following will describe an exemplary application when the device is implemented as a terminal.

[0049] See also Figure 1 , Figure 1 It is a structural diagram of the first architecture of the navigation system 100 for an underwater environment provided in an embodiment of the present application. To realize a navigation application supporting an underwater environment, an underwater electronic device 400 is connected to a server 200 via a network 300. The network 300 may be a wide area network or a local area network, or a combination of the two.

[0050] The underwater electronic device 400 is used to implement underwater environment navigation, including data collection, preprocessing, environmental perception, map construction, path planning and navigation, and the server 200 is used for data annotation, model training, data storage and analysis. After the underwater electronic device 400 collects underwater environmental data through sensors, it sends the underwater environmental data to the server 200. The server 200 annotates the received underwater environmental data and uses the annotated underwater environmental data to train the machine learning model to be trained based on the back propagation algorithm to obtain a pre-trained machine learning model. The server 200 sends the pre-trained machine learning model to the underwater electronic device 400. After entering the water, the underwater electronic device 400 obtains the underwater environmental data collected in real time, predicts the underwater environmental data using the pre-trained machine learning model, obtains the environmental perception result, and constructs an underwater map based on the environmental perception result; then, the current position and target position of the underwater electronic device are obtained, and the path planning is performed based on the current position, target position and underwater map to obtain the navigation path; finally, the underwater electronic device is controlled to move based on the navigation path.

[0051] The underwater electronic device 400, during the navigation process in the underwater environment, will also perform obstacle avoidance and path optimization based on the real-time underwater environmental information. In the process of moving based on the first navigation path, the underwater electronic device 400 acquires real-time underwater environmental data; then, using the pre-trained machine learning model, the underwater environmental data is predicted and processed, and when an unmarked obstacle is detected, the underwater electronic device is controlled to stop moving; at this time, the location information of the unmarked obstacle and the second current location of the underwater electronic device are acquired; then, using the obstacle avoidance algorithm, and based on the location information of the unmarked obstacle, the second current location of the underwater electronic device and the target location, the path is replanned to obtain the second navigation path; finally, the underwater electronic device is controlled to move based on the second navigation path, thereby improving the task execution capability and flexibility of the underwater electronic device.

[0052] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0053] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an underwater electronic device 400 provided in an embodiment of the present application, Figure 2 The underwater electronic device 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the underwater electronic device 400 are coupled together through a bus system 440. It can be understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0054] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0055] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0056] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0057] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0058] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0059] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0060] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0061] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0062] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0063] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The navigation device 455 of the underwater environment stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: data acquisition module 4551, map construction module 4552, first path planning module 4553, feasibility assessment module 4554 and navigation control module 4555. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0064] In other embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute a navigation method for an underwater environment provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.

[0065] Next, the underwater environment navigation method provided by the embodiment of the present application is described. As mentioned above, the electronic device implementing the underwater environment navigation method of the embodiment of the present application is an underwater electronic device, so the execution subject of each step will not be repeatedly described below. Figure 3A , Figure 3A This is a first flow chart of the underwater environment navigation method provided by the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained, Figure 3A The main body of the steps is underwater electronic equipment.

[0066] In step 101, underwater environment data collected in real time is obtained.

[0067] In some embodiments, the underwater environment data at least includes image data, sonar data and inertial navigation data. The underwater environment data can be collected by sensors deployed on underwater electronic devices. The sensors can include cameras, sonars, inertial navigation systems, depth sensors, water quality sensors, etc. The corresponding underwater environment data collected in real time include image data, sonar data, inertial navigation data and water quality data; wherein the image data can be underwater images collected by the camera, and the image data can also include information such as the size of the image and the number of channels. The camera can collect images within 20 meters in front of the camera; the sonar data includes echo data, for example, it can be echo data within a range of 50 meters collected by sonar; the inertial navigation data includes the position information and attitude information of the underwater electronic device, the position information is the specific coordinates [x, y, z] of the underwater electronic device, and the attitude data is the attitude information [roll, pitch, yaw] of the underwater electronic device; the water quality sensor data includes the physical and chemical parameters of the water body, such as temperature, pH value, dissolved oxygen, etc., to evaluate the underwater water quality environment and ensure that the underwater electronic device works in a suitable environment. The frequency of data collection using the sensor may be pre-set, for example, 10 seconds per time, or may be adjusted according to the accuracy of the underwater environment data.

[0068] As an example of step 101, the underwater environment data obtained may include: image data collected by the camera is in RGB format, with a size of 224*224 pixels, 3 channels, and includes red, green, and blue; sonar data is an echo signal of information such as the position, shape, size, and seabed topography of the target object; the position information of the underwater electronic device obtained by inertial navigation data is [10,5,6], and the attitude information of the underwater electronic device is [roll=10°, pitch=20°, yaw=30°].

[0069] In some embodiments, after acquiring the underwater environment data, a pre-trained machine learning model may be acquired, and the pre-trained machine learning model is used to perform predictive processing on the underwater environment data to obtain an environmental perception result. The pre-trained machine learning model may be trained using data collected by underwater electronic equipment. Prior to step 101, underwater environment data may be collected using underwater electronic equipment, and then the collected underwater environment data may be labeled and annotated. The labeled and annotated data set is used as a training data set and input into the machine learning model for training, thereby obtaining a pre-trained machine learning model.

[0070] In step 102, the underwater environment data is preprocessed using a pre-trained machine learning model to obtain an environment perception result.

[0071] In some embodiments, see Figure 3B , Figure 3A The illustrated step 102 may be implemented by following steps 1021 to 1025, which are described in detail below.

[0072] In step 1021, the image data, the sonar data and the inertial navigation data are aligned to obtain fused data.

[0073] In some embodiments, the image data, the sonar data and the inertial navigation data are subjected to time synchronization and spatial alignment, and a filtering algorithm such as a Kalman filtering algorithm is used to perform data fusion to obtain fused data, wherein the fused data includes aligned image data, aligned sonar data and aligned inertial navigation data.

[0074] In step 1022, based on the aligned image data, the pre-trained machine learning model is called to perform prediction processing to obtain category information of underwater obstacles.

[0075] In some embodiments, the category information of underwater obstacles is biological or non-biological, where biological includes all aquatic organisms, such as fish, corals, shellfish and seaweed, and non-biological includes rocks, shipwrecks, artificial structures such as underwater pipelines, platforms, etc.

[0076] In step 1023, based on the aligned sonar data, the pre-trained machine learning model is called to perform prediction processing to obtain the location information of the underwater obstacle.

[0077] In some embodiments, the location information of the underwater obstacle includes the distance and direction of the underwater obstacle relative to the underwater electronic device. The direction can be described by the azimuth and the pitch angle, for example, the southeast direction is 45° and the pitch angle is 30°.

[0078] In step 1024, based on the aligned inertial navigation data, the pre-trained machine learning model is called to perform prediction processing to obtain the posture information and the first current position of the underwater electronic device.

[0079] In some embodiments, the attitude information [roll, pitch, yaw] of the underwater electronic device and the first current position [x, y, z] of the underwater electronic device may be obtained by analyzing the aligned inertial navigation data.

[0080] As an example of step 1022 to step 1024, the underwater obstacle is a non-living thing, and the underwater obstacle is 1 meter away from the underwater electronic device, and is located at a 45° pitch angle of 30° in the southeast direction. The first current position of the underwater electronic device is [0, 2, 19]. It should be noted that in actual applications, relative distance and coordinate forms are used in combination, relative description is used in real-time obstacle avoidance, and coordinate description is used in map construction and path planning.

[0081] In step 1025, the position information, category information, posture information and one or more features of the first current position of the underwater obstacle are determined as an environmental perception result.

[0082] Through steps 1021 to 1025, the underwater environmental data obtained in step 101 is further processed, and the data processed in step 101 is applied to the pre-trained machine learning model to predict the underwater environmental information, adapt to the complex underwater environment, reduce the degree of manual dependence, and enable underwater electronic equipment to learn autonomously and obtain environmental perception results.

[0083] Continue to see Figure 3A In step 103, an underwater map is constructed based on the location information and category information of the underwater obstacles.

[0084] In some embodiments, see Figure 3C , Figure 3A The illustrated step 103 may be implemented by following steps 1031 to 1033, which are described in detail below.

[0085] In step 1031, initial environment information is acquired, and an initial grid map is generated based on the initial environment information.

[0086] In some embodiments, the initial environmental information obtained is consistent with the content contained in the above-mentioned environmental perception results. The underwater map is constructed using synchronous positioning and map construction technology. A real-time underwater map is constructed based on the environmental perception results. The specific form of the underwater map is a grid map, which divides the underwater environment into grids of equal size. Each grid contains environmental information, such as whether there are objects or obstacles. Each grid (cell) can be represented as "occupied" or "free". The state of each grid is marked by fusing sonar data and camera image data. In some embodiments, the map can be in the form of a topological map or a three-dimensional point cloud map in addition to a grid map.

[0087] As an example of step 1031, the initial environmental information is obtained, including the position of the target object [32, 54, 23], the category of the target object is organism coral, red coral, the position information of the underwater obstacle is [18, 66, 63], the distance is 83.87 meters, the position information of the underwater electronic device [10, 5, 6], the posture information of the underwater electronic device [roll = 10°, pitch = 20°, yaw = 30°], and the underwater map in grid form is constructed using synchronous positioning and map construction technology. The grids where there are objects or obstacles are marked as "occupied" and represented by the number 1, and the rest are marked as "idle" and represented by the number 0.

[0088] In step 1032, a first target grid corresponding to the underwater obstacle in the initial grid map is determined based on the position information and category information of the underwater obstacle.

[0089] The location information includes the distance and direction of the underwater obstacle relative to the underwater electronic device, and the category information is used to characterize whether the underwater obstacle is a living object or a non-living object.

[0090] In some embodiments, the location information of the obstacle is first converted into a location in a grid map, and the location in the grid map corresponding to the obstacle is marked, and the marking content includes whether the grid is occupied and the category information of the obstacle occupying the grid. The marked grid is the first target grid of the underwater obstacle in the initial grid map.

[0091] In step 1033, the state corresponding to the first target grid in the initial grid map is set to an occupied state to obtain an underwater map.

[0092] As an example of step 1032 and step 1033, the obstacle location information [18,66,63], the corresponding first target grid is marked as 1, that is, the "occupied" state, and the obstacle category is marked as non-biological. It should be noted that environmental perception is carried out in real time. The environmental perception results obtained in step 1031 will update the underwater map in real time. The real-time perception results are integrated into the map using synchronous positioning and map construction technology to update the map; the constructed map data is stored locally or in the cloud, and the environmental map is displayed using visualization tools to support navigation and analysis.

[0093] Through step 1031 to step 1033, dynamic perception of the underwater environment is achieved, and the perception results are updated in real time in the map to obtain an underwater map. The underwater electronic device can autonomously learn the underwater environment, improve the autonomous learning ability of the underwater electronic device in a complex underwater environment, and achieve dynamic environmental perception.

[0094] Continue to see Figure 3AIn step 104, the target position of the underwater electronic device is obtained, and path planning is performed based on the first current position, the target position and the underwater map to obtain a first navigation path.

[0095] In some embodiments, see Figure 3D , Figure 3A Step 104 shown can be implemented by the following steps 1041 to 1042, as shown below. Figure 3D Provide explanation.

[0096] In step 1041, the position information of the first target grid in the underwater map is obtained.

[0097] In some embodiments, the location information of the map grid can be marked with serial numbers such as 0, 1, 2... or coordinates such as [x, y]; the serial number marking starts from the first grid in the upper left corner of the grid map, and from left to right, from top to bottom, the grids are assigned serial numbers using 0, 1, 2,...; the coordinate marking selects a corner of the grid map, such as the upper left corner, as the coordinate origin, and the coordinate axis starts from the origin, and defines the horizontal right as the positive direction of the x-axis and the vertical downward as the positive direction of the y-axis.

[0098] As an example of step 1041, the position information of the first target grid in the underwater map is obtained, which is 0 if using serial number marking and [0,0] if using coordinate marking, both indicating that the first target grid is located at the first position in the upper left corner of the grid map.

[0099] In step 1042, a first navigation path is generated based on the first current position, the target position, and the position information of the target grid.

[0100] In some embodiments, the first current position of the underwater electronic device is obtained by an inertial navigation system; the target position is marked in a grid map and obtained by sonar; and the position information of the target grid is obtained by step 1041.

[0101] As an example of step 1042, assume that a path needs to be planned from the current electronic device's position coordinates [10,5,6] to the target position coordinates [18,66,63], and avoiding the obstacle position [32,54,23]. The position information of the grid map passed by this path does not include the grid position of the obstacle. The path planning is implemented using the classic path planning algorithm Dijkstra algorithm.

[0102] Through steps 1041 to 1042, the underwater electronic device can use the data obtained by the machine learning model to autonomously plan the path, thereby improving the autonomy of navigation.

[0103] Continue to see Figure 3AIn step 105, the size information and turning radius of the underwater electronic device, as well as the minimum path width, minimum path length and minimum turning width in the first navigation path of the first navigation path are obtained.

[0104] As an example of step 105, the size information of the underwater electronic device is obtained, the length, width and height are all 1 meter, the turning radius is 2 meters, the minimum path width of the first navigation path is 3 meters, the minimum path length is less than 5 meters, and the minimum turning width in the first navigation path is 2.5 meters.

[0105] Step 106: When the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius, it is determined that the feasibility evaluation result of the first navigation path is passed.

[0106] In some embodiments, before step 106, it is necessary to first determine whether the size information and turning radius of the underwater electronic device are less than the minimum path width, the minimum path length and the minimum turning width of the first navigation path. The specific content of the judgment includes the size relationship between the minimum path width and the device width, the minimum path length and the device length, and the minimum turning width and the turning radius. When the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius, step 106 is entered. When the minimum path width is less than or equal to the device width, the minimum path length is less than or equal to the device length, and the minimum turning width is less than or equal to the turning radius, it is determined that the feasibility evaluation result of the first navigation path is not passed. The content of the feasibility evaluation of the first navigation path includes evaluating the physical feasibility of the underwater electronic device when it moves on the first navigation path, that is, only when the size information and turning radius of the underwater electronic device are less than the minimum path width, the minimum path length and the minimum turning width of the first navigation path, can it be ensured that the underwater electronic device will not encounter physical obstacles when moving on the first navigation path and maintain smooth movement.

[0107] As an example of step 106, assuming that the length and width of the underwater electronic device are both 1 meter, the turning radius is 1.5 meters, the minimum path width of the first navigation path is 6 meters, the minimum path length is 10 meters, and the minimum turning width is 2.3 meters. At this time, the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius. Therefore, the underwater electronic device can pass the first navigation path smoothly, that is, the feasibility assessment result of the first navigation path is passed.

[0108] Through step 105 to step 106, it is possible to evaluate whether the underwater electronic device can successfully pass through the first navigation path, thereby improving the success rate of the underwater electronic device when performing tasks on the first navigation path.

[0109] In step 107, the underwater electronic device is controlled to move based on the first navigation path.

[0110] In some embodiments, when the feasibility assessment result of the first navigation path is that the assessment fails, the first navigation path needs to be replanned. Replanning the first navigation path can be achieved by the following technical solutions: obtaining the first sub-path and the second sub-path in the first navigation path, the first sub-path is a sub-path that does not need to be optimized, and the second sub-path is a sub-path that needs to be optimized; performing path planning based on the starting position of the second sub-path, the target position and the underwater map to obtain a third sub-path; determining the feasibility assessment result of the third sub-path based on the size information and turning radius of the underwater electronic device, as well as the minimum path width, the minimum path length and the minimum turning width in the third sub-path; when the feasibility assessment result of the third sub-path is that the assessment passes, combining the first sub-path and the third sub-path to obtain the second navigation path. Control the underwater electronic device to move based on the second navigation path. Through replanning, the part of the first navigation path where there are planning problems can be optimized to ensure that the underwater electronic device can achieve safe navigation based on the navigation path, further improving the accuracy of navigation.

[0111] Exemplarily, assuming that the first navigation path is a path from A to E, and passes through B, C and D in sequence, wherein the grid map state corresponding to C is updated to "occupied" or the grid map state corresponding to C is "idle", but the location of C is inaccessible to underwater electronic equipment, that is, the feasibility evaluation result of the C path is the evaluation failed, then the second sub-path is now from B to D, and passes through C. The second sub-path is optimized by combining the kinematic model of underwater electronic equipment and the dynamic obstacle avoidance algorithm. Here, for the underwater map updated in real time, it is necessary to combine the kinematic model of underwater electronic equipment and the dynamic obstacle avoidance algorithm to obtain the third sub-path; assuming that E satisfies the corresponding grid map state of "idle" and the location of E is accessible to underwater electronic equipment, that is, the feasibility evaluation result of the E path is the evaluation passed, then the third sub-path is from B to D, and passes through E. The determined second navigation path is from A to E, and passes through B, E and D in sequence.

[0112] In some embodiments, during the process of the underwater electronic device moving based on the first navigation path, the following steps may also be performed: Figure 4 Steps 108 to 112 are shown below. Figure 4 Provide explanation.

[0113] In step 108, underwater environment data is acquired in real time.

[0114] In step 109, the pre-trained machine learning model is used to perform predictive processing on the underwater environment data, determine that an unmarked obstacle is monitored, and control the underwater electronic device to stop moving.

[0115] The unmarked obstacles are obstacles that are not marked in the underwater map; it should be noted that due to the dynamic changes in the underwater environment, new obstacles will appear, so the unmarked obstacles monitored here are inconsistent with the obstacles marked when constructing the underwater map.

[0116] In step 110, the position information of the unmarked obstacle and the second current position of the underwater electronic device are obtained.

[0117] In some embodiments, the location information and distance of the unmarked obstacle are obtained by analyzing sonar data in the underwater environment data, and the second current position of the underwater electronic device is provided by an inertial navigation system.

[0118] As an example of step 110, the second current position of the underwater electronic device is [15, 20, 13], the position of the unmarked obstacle is [8, 21, 19], and the distance between the unmarked obstacle and the underwater electronic device is 9.27 meters.

[0119] In step 111, the path is replanned according to the underwater map to obtain a third navigation path.

[0120] In some embodiments, when step 111 is implemented, first, based on the position information of the unmarked obstacle, the second target grid corresponding to the unmarked obstacle in the underwater map is determined; the state corresponding to the second target grid in the underwater map is updated to an occupied state to obtain the updated underwater map; based on the second current position, the target position and the updated underwater map, the path is re-planned to obtain a third navigation path.

[0121] In step 112, the underwater electronic device is controlled to move based on the third navigation path.

[0122] Through steps 108 to 112, dynamic perception of underwater environmental changes is achieved, real-time obstacle avoidance is performed, and the autonomous navigation capability of underwater electronic equipment is further improved to adapt to complex underwater environments.

[0123] In some embodiments, after step 112, when an unmarked obstacle is monitored, the statistical number of unmarked obstacles is updated; when it is determined that the underwater electronic device successfully avoids the unmarked obstacle, the number of successful avoidance is updated; based on the statistical number and the number of successful avoidance, the probability of successful avoidance is determined; when the probability of successful avoidance is less than a preset probability threshold, the resolution of the sensor used to collect underwater environmental data is increased, or the obstacle avoidance algorithm used by the underwater electronic device is updated.

[0124] In some embodiments, during the movement of the underwater electronic device based on the first navigation path, the actual movement path of the underwater electronic device can also be recorded, and the path deviation can be determined based on the actual movement path and the first navigation path. When the path deviation is greater than a preset deviation threshold, the calibration frequency of the sensor used to collect data is increased, the path planning algorithm is updated, the SLAM algorithm is improved, or the filtering algorithm used to optimize the fusion data is improved.

[0125] In some embodiments, see Figure 5 Before controlling the underwater electronic device to move, steps 201 to 205 may also be performed. Figure 5 Provide explanation.

[0126] In step 201, a first power value of the underwater electronic device is obtained.

[0127] In some embodiments, the power value can be expressed as mAh (milliampere-hour), Wh (watt-hour), remaining power percentage or icon form. Here, icon form refers to using an icon to represent the power level, and the filled part of the icon represents the remaining power. As the power level decreases, the filled part will also decrease accordingly.

[0128] As an example of step 201 , it is assumed here that the power usage is expressed in percentage, and the first power value is 100%.

[0129] It should be noted that there is not only one way to represent the power level. Usually, the icon form will be combined with other ways of representing power values ​​to facilitate intuitive perception of power changes.

[0130] In step 202, during the process of controlling the movement of the underwater electronic device, when it is determined that a power collection condition is met, a second power value of the underwater device is acquired.

[0131] In some embodiments, the power collection condition can be collected at the end of a task or at a specific time, or it can be collected after sensing environmental changes. The power collection condition here is not unique and can be set manually or by autonomous learning and setting of the underwater electronic device.

[0132] As an example of step 202, it is assumed that the power value of the underwater electronic device is collected after working for 2 hours, and the second power value at this time is 50%.

[0133] In step 203, the power consumption rate of the underwater electronic device is determined based on the first power value and the second power value.

[0134] In step 204, it is determined whether the power consumption rate is greater than a preset rate threshold.

[0135] When the power consumption rate is greater than a preset rate threshold, the process proceeds to step 205 ; when the power consumption rate is less than or equal to the rate threshold, the process proceeds to step 202 .

[0136] In step 205, a first prompt message indicating that the power consumption is too fast is sent.

[0137] In some embodiments, a rate threshold of the power consumption rate is set to evaluate the performance of the battery. If the power consumption rate is too high, you can consider replacing the battery with a lower power consumption rate, or optimizing the energy management algorithm of the underwater electronic device to improve the battery usage efficiency; the first prompt information sent when the power is consumed too quickly includes the current power consumption rate, the current power value and the prompt information.

[0138] In some embodiments, sensors can also be used to monitor underwater electronic equipment in real time and collect status information and environmental information of the underwater robot during navigation. If an abnormal situation occurs or the mission objective changes, the system can promptly issue an alarm or adjust the navigation strategy; the abnormal situation here can be that the underwater electronic equipment detects an unmarked obstacle on the predetermined path, in which case the underwater electronic equipment will immediately stop and sound an alarm, use an obstacle avoidance algorithm to re-plan the path, bypass the obstacle, record the location of the obstacle, and update the map data; it can also be a sensor failure, low battery, navigation system error, etc.

[0139] Here, the water quality sensor in the sensor will collect water quality data in real time, including physical and chemical parameters of the water body, such as temperature, pH value, dissolved oxygen, etc., to evaluate the underwater water quality environment. If the evaluation result is not suitable for underwater electronic equipment to work, it will immediately end the current work and return. For example, if the current water temperature is 30 degrees, the pH value is acidic, and the dissolved oxygen content is 3 mg per liter, the underwater environment is high temperature and acidic, and the oxygen content is low, which is not suitable for underwater electronic equipment to work. The underwater electronic equipment will immediately end the current work and return.

[0140] Through real-time monitoring, the status information of underwater electronic equipment can be obtained, which facilitates timely handling of abnormal situations of underwater electronic equipment, thereby ensuring that underwater operations can be carried out safely and efficiently.

[0141] The following is an exemplary application of the embodiment of the present application in a practical application scenario. The navigation method for an underwater environment provided by the embodiment of the present application can be implemented by the following five parts:

[0142] The first part is target detection and recognition. Deep learning target detection and recognition is an important technology in the design of underwater robot navigation systems. By using deep learning algorithms, robots can learn and recognize target objects in underwater environments, such as marine life, potential obstacles, etc. This can help robots make appropriate decisions during navigation and avoid collisions or interactions with target objects. Deep learning models can be trained with large amounts of labeled data to improve the accuracy of target detection and recognition.

[0143] The second part is path planning. Deep learning path planning is a key technology for achieving efficient underwater navigation. By analyzing the characteristics and historical data of the underwater environment, the robot can learn and predict the best navigation path. The deep learning model can learn environmental characteristics and path selection patterns from large-scale underwater navigation data, and then select the optimal path based on the current task and conditions. Such a path planning method can improve the efficiency and safety of navigation.

[0144] The third part is sensor fusion. Sensor fusion is one of the key technologies to achieve accurate underwater navigation. Underwater robots are usually equipped with a variety of sensors, such as sonar, cameras, inertial navigation systems, etc. Through deep learning technology, the data of these sensors can be fused and optimized to improve the robot's perception and understanding of the underwater environment. The deep learning model can learn the correlation and weight distribution between sensor data, thereby achieving more accurate navigation and positioning.

[0145] The fourth part is dynamic environment modeling. Dynamic environment modeling is also one of the key technologies in the design of underwater robot navigation systems. The underwater environment often has changes and uncertainties, such as water flow and the movement of marine life. Through deep learning technology, robots can model and predict the dynamic changes of the environment in real time to adapt to and respond to different underwater navigation challenges. Deep learning models can learn the patterns and trends of environmental changes from historical data, and then perform real-time environmental modeling and prediction.

[0146] The fifth part is autonomous control. Autonomous control is the key technology to achieve the autonomy of underwater robot navigation system. Through deep learning algorithms, robots can learn and optimize autonomous control strategies to achieve more accurate and stable underwater navigation. Deep learning models can analyze and make decisions on the environment and tasks in real time, thereby improving the robot's autonomous navigation capabilities. This autonomous control method can make the robot adaptable and flexible in complex underwater environments. The application of these technologies can improve the navigation capabilities and autonomy of underwater robots in complex underwater environments, and bring important application value to underwater exploration, marine scientific research, and resource development.

[0147] Traditional underwater electronic equipment navigation relies on manual work. Manual underwater operations are highly dangerous, and the path setting method is relatively complicated. This will cause underwater electronic equipment to only follow a predetermined route and lack the ability to make real-time adjustments. The number of sensors carried is limited, and the data collected is relatively small. These defects increase the difficulty of operation, affect the normal operation of the navigation system, and limit the autonomous navigation capabilities of underwater electronic equipment in complex environments.

[0148] Based on this, the embodiment of the present application provides a design of an underwater electronic device navigation system based on machine learning. The underwater electronic device is equipped with multiple sensors to collect underwater data, perform map construction and path planning, and a server with a machine learning algorithm is applied and deployed in the design of the navigation system. The server labels and annotates the underwater data collected by multiple sensors, establishes a training data set, and is used to train the machine learning model, so that the underwater electronic device can perform path planning and obstacle avoidance in real time according to the underwater environment, reduce human intervention, reduce the danger of manual underwater operations, and enhance the autonomy of the underwater electronic device navigation system, so as to adapt to complex and changeable underwater environments and improve work efficiency. The embodiment of the present application provides a design of an underwater electronic device navigation system based on machine learning, which can be applied to tasks such as marine scientific research, marine resource development, and marine environment monitoring.

[0149] See also Figure 6 , Figure 6 Schematic diagram of the business process of underwater navigation provided by the embodiment of the present application. Figure 6 The implementation process of the underwater environment navigation method provided in the embodiment of the present application is explained.

[0150] Step 301, data collection and preprocessing. The underwater electronic device collects underwater environmental data, including image data, echo data, inertial navigation data, water quality data and other information, through the sensors carried by it, including cameras, sonars, inertial navigation systems, depth sensors, and water quality sensors, and performs preprocessing operations on the data, including denoising and image enhancement, and uses Kalman filtering for data fusion and calibration.

[0151] Data calibration mainly includes four aspects. The first aspect is camera calibration, which uses a chessboard or calibration plate to calibrate the camera's intrinsic and extrinsic parameters; the second aspect is sonar calibration, which uses known distances and objects to adjust the sonar parameters to match the actual situation; the third aspect is inertial navigation system calibration, static calibration, such as the six-face calibration method and dynamic calibration, such as the rotating table method, to obtain the offset and scale factor of the accelerometer and gyroscope; the fourth aspect is depth sensor calibration, which calibrates the sensor in an environment with a known depth and adjusts the measurement value to match the actual depth.

[0152] Water quality sensors are mainly used to monitor the physical and chemical parameters of water bodies, such as temperature, pH value, dissolved oxygen, etc. In the underwater robot navigation system, the data of water quality sensors can be used for environmental assessment, mission planning, and ensuring that the robot works under suitable conditions. It should be noted that the data acquisition frequency is 10 seconds / time, the image acquisition coverage is within 20 meters in front of the camera, and the echo data acquisition coverage is within 50 meters around the sonar.

[0153] Step 302, data labeling and annotation. Use data labeling tools to label the types of underwater objects on the preprocessed data and establish a training data set for model training and verification. Among them, the types of objects include organisms such as all aquatic organisms, such as fish, corals, shellfish, seaweed, etc., and non-biological objects such as rocks, sunken ships, artificial structures, underwater pipes, platforms, etc. More detailed classification is required in some specific scenarios. Data labeling uses LabelMe or VGG Image Annotator for image labeling, and sonar data is labeled using custom tools to label target objects and obstacles. The labeled categories are underwater objects such as rocks, sunken ships, marine life, etc., obstacles such as waste, other underwater electronic equipment, etc., and terrain such as sand, coral reefs, etc.

[0154] Step 303: Model training and optimization. Use the training data set to train and optimize the machine learning model, repeatedly iterate the training and adjust the parameter model to improve the model's perception and recognition capabilities in the underwater environment. The pre-trained model is deployed in underwater electronic equipment for real-time perception and path planning. The training process of the machine learning model is as follows: the image data collected by the camera is annotated using LabelMe or VGG Image Annotator (VIA), and a machine learning model is trained to identify the types of objects in underwater images, such as fish, corals, rocks, etc. The input data is an RGB format image with an image size of 224*224 pixels and 3 channels, including red, green, and blue. The output result is a pre-trained machine learning model; target detection is implemented using the YOLOv5 algorithm, and the training data set is a data set of 5,000 annotated underwater images. The Adam algorithm is used to optimize and update the parameters of the target detection model, and the learning rate is set to 0.001; path planning is implemented using the deep reinforcement learning model DQN, and the underwater environment data after multi-sensor fusion is input. A total of 100,000 steps of training are performed, and a reward function based on the distance to the target and the collision situation is used to dynamically adjust the training strategy.

[0155] Step 304, environmental perception and map construction. Use machine learning models to analyze and process the real-time collected data to achieve perception and understanding of the underwater environment; identify and classify underwater objects and obstacles to build a real-time underwater map. Among them, objects refer to all identifiable entities in the underwater environment, including natural and artificial entities, such as fish, corals, rocks, shipwrecks, etc. Objects can be mission targets, such as fish for biological monitoring, corals and rocks with environmental and topographical features; obstacles refer to those objects that need to be avoided during the execution of underwater electronic equipment tasks. These objects may hinder the movement of underwater electronic equipment or cause damage, such as large rocks, waste, other submersibles, etc. Obstacles are objects that underwater electronic equipment needs to avoid during navigation to ensure safety and smooth mission progress. The underwater map can be a grid map, a topological map, or a three-dimensional point cloud map. The grid map divides the underwater environment into grids of equal size, each grid contains environmental information, such as whether there are objects and obstacles, and each grid (cell) can be expressed as "occupied" or "free". By fusing sonar data and camera image data, the status of each grid is marked. The underwater map can be a topological map, which requires nodes and edges to be defined in the topological map. The underwater map can also be a three-dimensional point cloud map, which generates a three-dimensional environment model by collecting and splicing point cloud data. As the underwater electronic equipment moves, the environment map is continuously updated, and optimization algorithms are used to reduce errors and improve map accuracy. The constructed map data is stored locally or in the cloud, and visualization tools are used to display the environment map to support navigation and analysis.

[0156] The analysis and processing process is as follows: first, data is collected. The camera collects underwater RGB images, the sonar collects echo data, and the inertial navigation system collects position information and attitude data. The collected data is denoised and image enhanced. The sonar data is used for filtering, sensor data is synchronized and calibrated, and the pre-trained convolutional neural network (CNN) is used to detect and classify the image; obstacle detection and distance measurement are performed using sonar data; and position information is corrected using inertial navigation system data. The analysis and processing results include the position and category of the target object, which are obtained from the image data and are the coordinate positions in the image, represented by [x, y, width, height]. The categories of objects are marked as fish, coral, rock, etc.; the position and distance of obstacles are obtained from sonar data, where the position is the distance and direction of the obstacle relative to the underwater electronic device, and the distance is the distance between the obstacle and the underwater electronic device, such as 20 meters; the position information and attitude data are obtained from the inertial navigation system (INS) data, where the position data is the specific coordinates [x, y, z] of the underwater electronic device underwater, and the attitude data is the attitude information of the robot [roll, pitch, yaw]. Relative distance is usually expressed in meters (m). Relative to the robot's position, it is described using azimuth and elevation. For example, the obstacle is 1 meter away from the robot, 45° southeast and 30° elevation. In practical applications, both descriptions are used in combination. For example, relative distance is used for real-time obstacle avoidance, and coordinates are used for map building and path planning.

[0157] In the process of underwater map construction, data fusion is first performed, using algorithms such as Kalman filtering to fuse the data from cameras, sonars, and inertial navigation systems to obtain more accurate environmental perception information. The second step is to integrate the environmental perception results, integrating the location and category of the target object, the location and distance of obstacles, and the location information and posture data of underwater electronic equipment to form a unified environmental perception result. The last step is map construction, using simultaneous localization and mapping (SLAM) technology to build a real-time underwater map based on the environmental perception results. SLAM algorithms can be vision-based, such as ORB-SLAM2, or sonar-based, such as sonar-based SLAM.

[0158] Step 305, path planning and navigation. Based on the underwater map and perception results, the navigation system plans the driving path of the underwater electronic device, adjusts the path in real time to avoid obstacles, selects the optimal path, and avoids potential risks. Among them, the perception result is useful information extracted from the underwater environment through machine learning models and multi-sensor fusion technology, including: the category of objects such as fish, corals, rocks, etc., the location and specific size, shape, and location and coordinate information of the object. The map is a static description of the underwater environment, including a raster map of all known objects, obstacles, and environmental features. The path is the route of the underwater electronic device from the starting point to the end point planned on the map. The environmental information on the map should be considered when planning the path, so as to avoid obstacles during navigation.

[0159] Build or update the underwater map based on the perception results. Use the simultaneous localization and mapping (SLAM) technology to integrate the real-time perception results into the map. Use the classic path planning algorithm Dijkstra algorithm for path planning. Combined with the kinematic model of underwater electronic equipment and the dynamic obstacle avoidance algorithm, taking into account factors such as the mass, friction, and inertia of underwater electronic equipment, it is suitable for more complex motion control, ensuring that the planned path is physically feasible and meets the motion limitations of underwater electronic equipment, such as maximum speed, turning radius, etc. Optimize the initially planned path to ensure the feasibility and safety of the path. During the movement of underwater electronic equipment, continuously perceive environmental changes and adjust the path in real time to cope with new obstacles or environmental changes.

[0160] Suppose an underwater robot is performing a task and needs to move from starting point A to target point B. Preliminary path planning has generated a path. However, during the movement, the robot detects a new obstacle in front of it, such as a moving rock. At this point, the system will adjust the path according to the following steps: Environmental perception: The robot detects the position and size of the rock through sonar and camera. Map update: Use SLAM technology to update the environmental map and mark the location of the newly appeared rock. Re-plan the path: Use the A* algorithm to re-plan the path according to the updated map to avoid new obstacles. Path optimization: Smooth and evaluate the feasibility of the new path to ensure that the robot can pass safely. Execution adjustment: The robot adjusts its own motion parameters according to the new path and continues to move towards target point B.

[0161] Step 306, real-time monitoring. The navigation system monitors the status information and environmental information of underwater electronic devices in real time through sensors. If an abnormal situation occurs or the target person changes, the system can issue an alarm or adjust the navigation strategy in time. The status information of underwater electronic devices includes position, attitude, speed, acceleration, battery status, system temperature, and environmental information includes water depth, water flow speed and direction, ambient light intensity, etc. Abnormal situations include detection of unmarked obstacles, sensor failure, low battery, navigation system error, etc. For example, if an underwater electronic device detects an unmarked obstacle on a predetermined path, it may affect navigation safety. The underwater electronic device will immediately stop and alarm, use an obstacle avoidance algorithm to replan the path, bypass the obstacle, record the obstacle location, and update the map data. Real-time monitoring uses real-time video streaming transmission, H.264 encoding, and sensor data is uploaded to the monitoring center in real time, with an update frequency of 10 times per second.

[0162] Step 307, data storage and analysis. The data during the navigation process is stored and analyzed for subsequent performance evaluation and improvement. Storage uses Hadoop or Spark for big data processing, storing navigation data, sensor quantity and video data for subsequent analysis. The large amount of data collected, including sensor and system information, is uploaded and stored in the cloud. The server deploys the machine learning algorithm to clean and preprocess the data. The performance of underwater electronic equipment is evaluated through data analysis, including the following indicators: Navigation accuracy: compare the deviation between the planned path and the actual path; Obstacle avoidance success rate: the ratio of the number of times underwater electronic equipment successfully avoids obstacles to the total number of obstacles detected; Battery efficiency: battery consumption per unit time or unit distance; Analysis methods use Python and R for data analysis and model evaluation, and data visualization tools use Matplotlib and Seaborn.

[0163] The server that deploys the machine learning algorithm makes suggestions for improving the results obtained by analyzing and evaluating the performance of underwater electronic equipment. If the navigation accuracy error is large, you can consider improving the SLAM algorithm or increasing the sensor calibration frequency or optimizing the filtering algorithm. If the obstacle avoidance success rate is low, you can optimize the obstacle avoidance algorithm or increase the sensor resolution or add sensors and perform multi-sensor fusion. If the battery usage efficiency is too high, you can consider replacing the battery with a lower power consumption rate, or optimizing the energy management algorithm of the underwater electronic equipment to improve the battery usage efficiency.

[0164] See also Figure 7 , Figure 7It is a structural diagram of the second architecture of the navigation system for the underwater environment provided by the embodiment of the present application. Sensor 401 is deployed on the underwater electronic device 400, and data collection, preprocessing, environmental perception, map construction, path planning and navigation are performed by the underwater electronic device. The main body of the improved execution is carried out on the server side 402, and the navigation system is deployed inside the robot. Map construction and path planning are both generated in real time by the pre-trained model. Data labeling and annotation, model training and optimization, real-time monitoring, data storage and analysis are completed by the server. Map construction and path planning are both generated in real time by the pre-trained model.

[0165] The underwater environment navigation system based on deep learning provided by the embodiment of the present application has brought many significant benefits to enterprises and is of great significance to the application and promotion of underwater electronic equipment. Traditional underwater navigation systems usually rely on manually set paths or remote control instructions. The navigation system based on deep learning can realize autonomous path planning and decision-making, so that underwater robots can complete tasks independently. This can reduce the need for manual intervention and improve the autonomy and work efficiency of underwater robots. The deep learning algorithm can analyze and process underwater images and sensor data in real time, thereby realizing the perception and understanding of the underwater environment. This enables underwater robots to better identify and classify underwater objects and obstacles, and build real-time underwater maps. This improved environmental perception ability helps underwater robots better adapt to complex and changeable underwater environments. Deep learning can realize intelligent path planning and dynamic obstacle avoidance, so that underwater robots can complete tasks more efficiently. Through the analysis of the underwater environment by the deep learning algorithm, the robot can choose the optimal path, avoid obstacles, and adjust in real time according to the task requirements. Improve the ability and flexibility of robot task execution.

[0166] Traditional underwater tasks usually require human divers to operate, which has certain risks and costs. The underwater robot navigation system based on deep learning can reduce the dependence on human divers, thereby reducing the risks and costs of diving operations. Underwater robots can replace humans to perform dangerous tasks while providing more stable and reliable execution capabilities. The navigation method for the underwater environment provided by the embodiment of the present application not only improves the navigation ability of the underwater robot, but also promotes the application and development of deep learning technology in the underwater field. By continuously improving and optimizing the deep learning algorithm, the perception, decision-making and navigation capabilities of the underwater robot can be further improved, bringing more innovations and progress to the fields of underwater scientific research, marine resource development and environmental protection. In addition, it also brings about the improvement of navigation accuracy, the enhancement of task execution capabilities, the reduction of risks and costs, and the promotion of scientific research and application development. These beneficial effects enable underwater robots to better adapt to complex underwater environments.

[0167] The following is a description of an exemplary structure of the underwater environment navigation device 455 provided in the embodiment of the present application as a software module. In some embodiments, Figure 2 As shown, the software modules in the underwater environment navigation device 455 stored in the memory 450 may include: a data acquisition module 4551, used to obtain underwater environment data collected in real time, and the underwater environment data at least includes image data, sonar data and inertial navigation data; a map construction module 4552, used to pre-process the underwater environment data using a pre-trained machine learning model to obtain an environment perception result, and the environment perception result includes the location information and category information of underwater obstacles, and the first current location and posture information of underwater electronic equipment; construct an underwater map based on the location information and category information of the underwater obstacles; a first path planning module 4553, used to obtain the underwater electronic equipment The target position is configured to plan a path based on the first current position, the target position and the underwater map to obtain a first navigation path; a feasibility assessment module 4554 is configured to obtain the size information and turning radius of the underwater electronic device, as well as the minimum path width, the minimum path length and the minimum turning width in the first navigation path of the underwater electronic device. When the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius, the feasibility assessment result of the first navigation path is determined to be passed; a navigation control module 4555 is configured to control the underwater electronic device to move based on the first navigation path.

[0168] In some embodiments, the map construction module 4552 is also used to align the image data, the sonar data and the inertial navigation data to obtain fused data, wherein the fused data includes aligned image data, aligned sonar data and aligned inertial navigation data; based on the aligned image data, calling the pre-trained machine learning model for prediction processing to obtain category information of underwater obstacles; based on the aligned sonar data, calling the pre-trained machine learning model for prediction processing to obtain location information of the underwater obstacles; based on the aligned inertial navigation data, calling the pre-trained machine learning model for prediction processing to obtain attitude information and a first current position of the underwater electronic device; determining one or more features of the location information, category information, attitude information and the first current position of the underwater obstacle as the environmental perception result.

[0169] In some embodiments, the map construction module 4552 is also used to obtain initial environmental information and generate an initial grid map based on the initial environmental information; determine the first target grid corresponding to the underwater obstacle in the initial grid map based on the position information and category information of the underwater obstacle; the position information includes the distance and direction of the underwater obstacle relative to the underwater electronic device, and the category information is used to characterize whether the underwater obstacle is a living body or a non-living body; set the state corresponding to the first target grid in the initial grid map to an occupied state to obtain an underwater map.

[0170] In some embodiments, the feasibility assessment module 4554 is also used to determine that the feasibility assessment result is a passed assessment when the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius; and to determine that the feasibility assessment result is a failed assessment when at least one of the following is met: the minimum path width is less than or equal to the device width, the minimum path length is less than or equal to the device length, and the minimum turning width is less than or equal to the turning radius.

[0171] In some embodiments, the navigation device for the underwater environment also includes a second path planning module, which is used to obtain the first sub-path and the second sub-path in the first navigation path when the feasibility assessment result is that the assessment fails, the first sub-path is a sub-path that does not need to be optimized, and the second sub-path is a sub-path that needs to be optimized; perform path planning based on the starting position of the second sub-path, the target position and the underwater map to obtain a third sub-path; obtain the size information and turning radius of the underwater electronic device, as well as the minimum path width, minimum path length and minimum turning width in the third sub-path to determine the feasibility assessment result of the third sub-path; when the feasibility assessment result of the third sub-path is that the assessment passes, combine the first sub-path and the third sub-path to obtain a second navigation path; and control the underwater electronic device to move based on the second navigation path.

[0172] In some embodiments, the navigation device for the underwater environment also includes a third path planning module, which is used to obtain real-time collected underwater environment data during the movement of the underwater electronic device based on the first navigation path; use the pre-trained machine learning model to predict and process the underwater environment data, determine that an unmarked obstacle is monitored, and control the underwater electronic device to stop moving, wherein the unmarked obstacle is an obstacle that is not marked in the underwater map; obtain the location information of the unmarked obstacle and the second current position of the underwater electronic device, and re-plan the path according to the underwater map to obtain a third navigation path; control the underwater electronic device to move based on the third navigation path.

[0173] In some embodiments, the third path planning module is also used to determine the second target grid corresponding to the unmarked obstacle in the underwater map based on the location information of the unmarked obstacle; update the state corresponding to the second target grid in the underwater map to an occupied state to obtain the updated underwater map; based on the second current position, the target position and the updated underwater map, re-plan the path to obtain a third navigation path.

[0174] In some embodiments, the navigation device for the underwater environment also includes an obstacle avoidance optimization module, which is used to update the statistical number of unmarked obstacles when unmarked obstacles are monitored; when it is determined that the underwater electronic device successfully avoids the unmarked obstacles, update the number of successful avoidances; based on the statistical number and the number of successful avoidances, determine the probability of successful avoidance; when the probability of successful avoidance is less than a preset probability threshold, increase the resolution of the sensor used to collect underwater environment data, or update the obstacle avoidance algorithm used by the underwater electronic device.

[0175] In some embodiments, the navigation device for an underwater environment also includes a path deviation correction module, which is used to record the actual moving path of the underwater electronic device during the movement of the underwater electronic device based on the first navigation path; determine the path deviation based on the actual moving path and the first navigation path; when the path deviation is greater than a preset deviation threshold, increase the calibration frequency of the sensor used to collect data or update the path planning algorithm.

[0176] In some embodiments, the navigation device for an underwater environment further includes a power prompt module, which is used to obtain a first power value of the underwater electronic device before controlling the movement of the underwater electronic device; in the process of controlling the movement of the underwater electronic device, when it is determined that the power collection condition is met, obtain a second power value of the underwater device; determine the power consumption rate of the underwater electronic device based on the first power value and the second power value; when the power consumption rate is greater than a preset rate threshold, send a first prompt message indicating that the power consumption is too fast. When the power consumption rate is less than or equal to the rate threshold, determine a third power value when the underwater electronic device reaches the target position based on the power consumption rate; when the third power value is less than the preset power threshold, send a second prompt message indicating that the remaining power is insufficient.

[0177] In some embodiments, the navigation device for the underwater environment also includes a water quality assessment module, which is used to obtain water quality data collected by a water quality sensor; use the water quality data to perform an environmental assessment to obtain an environmental assessment result; when the environmental assessment result is that the assessment fails, control the underwater electronic device to return.

[0178] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer readable storage medium. The processor of the underwater electronic device reads the computer executable instruction from the computer readable storage medium, and the processor executes the computer executable instruction, so that the underwater electronic device executes the navigation method of the underwater environment described in the embodiment of the present application.

[0179] An embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the underwater environment navigation method provided by the embodiment of the present application.

[0180] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. Computer executable instructions may be in the form of programs, software, software modules, scripts, or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0181] As examples, computer executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as one or more scripts in a Hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). Computer executable instructions may be deployed to be executed on one underwater electronic device, or on multiple underwater electronic devices located at one location, or on multiple underwater electronic devices distributed at multiple locations and interconnected by a communication network.

[0182] To summarize, through the embodiments of the present application, underwater electronic equipment can perform path planning and obstacle avoidance in real time according to the underwater environment, reduce human intervention, reduce the danger of manual underwater operations, and enhance the autonomy of the underwater electronic equipment navigation system so as to adapt to the complex and changeable underwater environment and improve work efficiency.

[0183] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A navigation method for an underwater environment, characterized in that: The method comprises: Acquiring underwater environmental data collected in real time, wherein the underwater environmental data at least includes image data, sonar data and inertial navigation data; Preprocessing the underwater environment data using a pretrained machine learning model to obtain an environment perception result, wherein the environment perception result includes location information and category information of underwater obstacles, and a first current location and posture information of the underwater electronic device; Building an underwater map based on the location information and category information of the underwater obstacles; Acquire a target position of the underwater electronic device, perform path planning based on the first current position, the target position and the underwater map, and obtain a first navigation path; Acquire the size information and turning radius of the underwater electronic device, as well as the minimum path width, the minimum path length, and the minimum turning width in the first navigation path of the first navigation path, and determine that the feasibility assessment result of the first navigation path is passed when the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius; The underwater electronic device is controlled to move based on the first navigation path.

2. The method according to claim 1, characterized in that The pre-trained machine learning model is obtained by training with the underwater environment training data through the back propagation algorithm, and the underwater environment data is pre-processed with the pre-trained machine learning model to obtain the environment perception result, including: Performing alignment processing on the image data, the sonar data and the inertial navigation data to obtain fused data, wherein the fused data includes the aligned image data, the aligned sonar data and the aligned inertial navigation data; Based on the aligned image data, calling the pre-trained machine learning model to perform prediction processing to obtain category information of underwater obstacles; Based on the aligned sonar data, calling the pre-trained machine learning model to perform prediction processing to obtain the position information of the underwater obstacle; Based on the aligned inertial navigation data, calling the pre-trained machine learning model to perform prediction processing to obtain the attitude information and the first current position of the underwater electronic device; The position information, category information, posture information and one or more features of the first current position of the underwater obstacle are determined as an environmental perception result.

3. The method according to claim 1, characterized in that The step of constructing an underwater map based on the location information and category information of the underwater obstacles comprises: Acquiring initial environmental information, and generating an initial grid map based on the initial environmental information; Determining a first target grid corresponding to the underwater obstacle in the initial grid map based on the position information and category information of the underwater obstacle; the position information includes the distance and direction of the underwater obstacle relative to the underwater electronic device, and the category information is used to characterize whether the underwater obstacle is a living body or a non-living body; The state corresponding to the first target grid in the initial grid map is set to an occupied state to obtain an underwater map.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When the feasibility assessment result is that the assessment fails, obtaining a first subpath and a second subpath in the first navigation path, wherein the first subpath is a subpath that does not need to be optimized, and the second subpath is a subpath that needs to be optimized; Performing path planning based on the starting position of the second sub-path, the target position and the underwater map to obtain a third sub-path; Determine a feasibility assessment result of the third subpath based on the size information and the turning radius of the underwater electronic device, as well as the minimum path width, the minimum path length, and the minimum turning width in the third subpath of the third subpath; When the feasibility assessment result of the third sub-path is passed, combining the first sub-path and the third sub-path to obtain a second navigation path; The underwater electronic device is controlled to move based on the second navigation path.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Acquiring underwater environment data collected in real time during the movement of the underwater electronic device based on the first navigation path; Using the pre-trained machine learning model, predicting and processing the underwater environment data, determining that an unmarked obstacle is detected, and controlling the underwater electronic device to stop moving, wherein the unmarked obstacle is an obstacle that is not marked in the underwater map; Acquire the location information of the unmarked obstacle and the second current location of the underwater electronic device, and re-plan the path according to the underwater map to obtain a third navigation path; The underwater electronic device is controlled to move based on the third navigation path.

6. The method according to claim 5, characterized in that The re-planning of the path according to the underwater map to obtain a third navigation path includes: Based on the position information of the unmarked obstacle, determining a second target grid corresponding to the unmarked obstacle in the underwater map; Updating the state corresponding to the second target grid in the underwater map to an occupied state, to obtain the updated underwater map; Based on the second current position, the target position and the updated underwater map, the path planning is re-performed to obtain a third navigation path.

7. The method according to claim 6, characterized in that The method further comprises: When unmarked obstacles are detected, the statistical number of unmarked obstacles is updated; When it is determined that the underwater electronic device successfully avoids the unmarked obstacle, updating the number of successful avoidances; Determine a probability of successful avoidance based on the statistical number and the number of successful avoidances; When the probability of successful avoidance is less than a preset probability threshold, the resolution of the sensor used to collect underwater environmental data is increased, or the obstacle avoidance algorithm used by the underwater electronic device is updated.

8. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Before controlling the underwater electronic device to move, obtaining a first power value of the underwater electronic device; In the process of controlling the movement of the underwater electronic device, when it is determined that a power collection condition is met, obtaining a second power value of the underwater device; Determining a power consumption rate of the underwater electronic device based on the first power value and the second power value; When the power consumption rate is greater than a preset rate threshold, a first prompt message indicating that the power consumption is too fast is sent.

9. A navigation device for an underwater environment, characterized in that: The device comprises: A data acquisition module, used to acquire underwater environmental data collected in real time, wherein the underwater environmental data at least includes image data, sonar data and inertial navigation data; A map construction module, used to pre-process the underwater environment data using a pre-trained machine learning model to obtain an environment perception result, wherein the environment perception result includes location information and category information of underwater obstacles, and a first current location and posture information of underwater electronic equipment; and construct an underwater map based on the location information and category information of the underwater obstacles; A first path planning module, used to obtain a target position of the underwater electronic device, perform path planning based on the first current position, the target position and the underwater map, and obtain a first navigation path; a feasibility assessment module, configured to obtain the size information and the turning radius of the underwater electronic device, as well as the minimum path width, the minimum path length and the minimum turning width in the first navigation path of the first navigation path, and determine that the feasibility assessment result of the first navigation path is passed when the minimum path width is greater than the device width, the minimum path length is greater than the device length, and the minimum turning width is greater than the turning radius; The navigation control module is used to control the underwater electronic device to move based on the first navigation path.

10. An underwater electronic device, characterized in that: The underwater electronic equipment comprises: A memory for storing computer executable instructions; A processor, configured to implement the method according to any one of claims 1 to 8 when executing the computer executable instructions stored in the memory.

11. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images