A multi-sensor-based online positioning method and device for underwater robots
By using multi-sensor fusion technology, which combines acoustic, optical, electromagnetic, and environmental perception data, the problem of low positioning accuracy of a single sensor in complex underwater environments has been solved, achieving efficient underwater target positioning and detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NAVAL UNIV OF ENG PLA
- Filing Date
- 2025-03-05
- Publication Date
- 2026-06-09
Smart Images
Figure CN120101770B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of underwater target detection, and more specifically, to an online positioning method and apparatus for underwater robots based on multiple sensors. Background Technology
[0002] With the continuous development of marine development and underwater exploration technologies, the application scope of underwater robots is becoming increasingly wide, including fields such as marine resource exploration, underwater archaeology, environmental monitoring, and military reconnaissance. Current underwater robots include tethered remotely operated vehicles (ROVs), untethered autonomous underwater vehicles (AUVs), and hybrid types. Tethered ROVs connect to a mother ship via a cable, transmitting data and power in real time. They are further divided into self-propelled, towed, and seabed-crawling types, suitable for long-duration, precision operations (such as pipeline maintenance and sample collection). Untethered AUVs primarily rely on internal power and programs for autonomous operation, with core features including untethered operation, autonomous navigation, and long-range exploration capabilities. Hybrid types combine both to adapt to different environments. Accurately locating both the robot's own position and the target position is crucial for efficient underwater operations, especially in environments with complex terrain, high turbidity, and high salinity.
[0003] However, existing technologies still have many shortcomings in underwater positioning.
[0004] For example, existing underwater positioning technologies mostly rely on a single sensor, such as sonar or optical camera. Acoustic positioning systems are susceptible to water temperature stratification, salinity changes, and water turbidity, leading to signal attenuation and increased reflection errors. Optical positioning systems are limited by the underwater light propagation characteristics and are easily affected by turbidity, light intensity, and water color, resulting in blurred images and reduced recognition accuracy. Therefore, it is difficult for a single sensor to achieve high-precision positioning in complex underwater environments.
[0005] There is currently no good solution to the above problems. Summary of the Invention
[0006] This invention provides an online positioning method and apparatus for underwater robots based on multiple sensors, which at least solves the problem of low positioning accuracy in related technologies.
[0007] According to an embodiment of the present invention, a method for online positioning of an underwater robot based on multiple sensors is provided, comprising:
[0008] Acquire sensor data, wherein the sensor data includes acoustic data, optical data, electromagnetic data, and environmental perception data;
[0009] Based on the acoustic data, a first suspicious area is determined, and the first suspicious area is then subjected to detail focusing and identification processing using optical data.
[0010] Based on the detail-focused identification processing results, the first electromagnetic data and the first environmental data of the first suspicious area are matched from the sensing data, and the environmental perception data includes the first environmental data;
[0011] The location information of the underwater target is determined based on the electromagnetic data and the environmental perception data.
[0012] In an exemplary embodiment, determining a first suspicious region based on the acoustic data and performing detail focusing and identification processing on the first suspicious region using optical data includes:
[0013] The noise is dynamically adjusted based on the acoustic data, and density clustering and isolated point removal are performed on the acoustic cloud points in the acoustic data based on the detection threshold to obtain the center point coordinates and edge coordinates of the first suspicious area.
[0014] The coordinates of the center point and the coordinates of the edge of the region are converted into optical camera coordinates to obtain the coordinates of the target center point and the coordinates of the target region edge.
[0015] The image data contained in the optical data is restored according to the preset underwater optical transmission model to obtain the first image;
[0016] The first image is subjected to detail focusing recognition processing based on the coordinates of the target center point and the edge coordinates of the target region to obtain the detail focusing recognition processing result.
[0017] In one exemplary embodiment, prior to acquiring the sensing data, the method further includes:
[0018] Minimum variance distortionless response beamforming is used to focus energy onto the first suspected region;
[0019] The transmission waveform parameters are optimized using a Bayesian probability model to dynamically match the water turbidity of the first suspected area.
[0020] The array gain distribution of the acoustic data is dynamically adjusted based on the water turbidity, wherein the detection threshold is obtained based on the array gain distribution.
[0021] In an exemplary embodiment, determining the location information of the underwater target based on the electromagnetic data and the environmental sensing data includes:
[0022] Based on the first electromagnetic data, determine the electric field gradient change information of the first suspicious area;
[0023] Based on the electric field gradient change information and the first environmental data, the object characteristics of the first usable region are determined;
[0024] If the object features meet the first condition, the pre-obtained coordinates of the target center point and the coordinates of the target region edge are fused using the extended Kalman filter algorithm to obtain the location information.
[0025] In an exemplary embodiment, after matching first electromagnetic data and first environmental data of the first suspicious region from the sensing data based on the detail focus identification processing result, wherein the environmental sensing data includes the first environmental data, the method further includes:
[0026] Based on the environmental perception data, the weights of the sensor data and the electromagnetic data are dynamically adjusted and allocated.
[0027] According to another embodiment of the present invention, an apparatus for online positioning of an underwater robot based on multiple sensors is provided, comprising:
[0028] The data acquisition module is used to acquire sensor data, which includes acoustic data, optical data, electromagnetic data, and environmental perception data.
[0029] The first processing module is used to determine a first suspicious area based on the acoustic data, and to perform detail focusing and identification processing on the first suspicious area using optical data;
[0030] The matching module is used to match the first electromagnetic data and the first environmental data of the first suspicious area from the sensing data based on the detail focusing identification processing result, wherein the environmental sensing data includes the first environmental data;
[0031] The location determination module is used to determine the location information of the underwater target based on the electromagnetic data and the environmental perception data.
[0032] In an exemplary embodiment, determining a first suspicious region based on the acoustic data and performing detail focusing and identification processing on the first suspicious region using optical data includes:
[0033] The noise is dynamically adjusted based on the acoustic data, and density clustering and isolated point removal are performed on the acoustic cloud points in the acoustic data based on the detection threshold to obtain the center point coordinates and edge coordinates of the first suspicious area.
[0034] The coordinates of the center point and the coordinates of the edge of the region are converted into optical camera coordinates to obtain the coordinates of the target center point and the coordinates of the target region edge.
[0035] The image data contained in the optical data is restored according to the preset underwater optical transmission model to obtain the first image;
[0036] The first image is subjected to detail focusing recognition processing based on the coordinates of the target center point and the edge coordinates of the target region to obtain the detail focusing recognition processing result.
[0037] In one exemplary embodiment, the apparatus further includes:
[0038] A beamforming module is used to focus energy onto the first suspected region by minimum variance distortionless response beamforming before acquiring the sensing data.
[0039] The dynamic matching module is used to optimize the transmission waveform parameters through a Bayesian probability model and dynamically match the water turbidity of the first suspected area.
[0040] A dynamic adjustment module is used to dynamically adjust the array gain distribution of the acoustic data based on the water turbidity, wherein the detection threshold is obtained based on the array gain distribution.
[0041] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.
[0042] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.
[0043] By integrating multiple sensors to locate and detect underwater targets from multiple dimensions, this invention can solve the problem of low positioning accuracy of underwater targets and improve positioning accuracy. Attached Figure Description
[0044] Figure 1 This is a flowchart of an online positioning method for an underwater robot based on multiple sensors according to an embodiment of the present invention;
[0045] Figure 2 This is a structural block diagram of an underwater robot online positioning method device based on multiple sensors according to an embodiment of the present invention. Detailed Implementation
[0046] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0047] In the following description, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0048] Furthermore, in this application, directional terms such as "upper," "lower," "left," and "right" may be defined relative to the orientation of the components shown in the accompanying drawings. It should be understood that these directional terms can be relative concepts, used for relative description and clarification, and may change accordingly depending on the orientation of the components in the accompanying drawings.
[0049] In this application, unless otherwise expressly specified and limited, the term "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral part; it can be a direct connection or an indirect connection through an intermediate medium. Furthermore, the term "coupled" can refer to an electrical connection that enables signal transmission.
[0050] As used herein, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from the given value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the given quantity (i.e., the limitations of the measurement system).
[0051] Existing underwater target localization methods typically employ a single sensor (sonar or visual processing) for detection, which proves ineffective in environments with high turbidity and significant electromagnetic interference. This embodiment presents a multi-sensor-based online localization method for underwater robots, avoiding the aforementioned problems and thereby improving the accuracy of underwater target detection.
[0052] Figure 1 This is a flowchart of an online positioning method for an underwater robot based on multiple sensors according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:
[0053] Step S11: Acquire sensing data, wherein the sensing data includes acoustic data, optical data, electromagnetic data, and environmental perception data;
[0054] In this embodiment, compared with the prior art, this application also incorporates electromagnetic data and environmental perception data into the underwater target detection, thereby effectively adapting to complex underwater scenarios and improving detection accuracy.
[0055] The acoustic data includes depth sounding data (measuring the time difference between sound wave transmission and reception using a sonar system), echo signals (using active sonar to transmit sound waves and receive the echoes reflected from the target), Doppler frequency shift (detecting the target's speed by changing the sound wave frequency), and scattering signals (scattered sound waves from suspended particles or bubbles in turbid water, used to assess water quality (e.g., scattering intensity fed back by a turbidity sensor)). Optical data includes underwater images of target surface details acquired through blue-green light (532nm) cameras or laser scanning, spectral information in different bands, laser stripe projection data analyzing the target's three-dimensional contour through laser stripe deformation, and photon count data corresponding to extremely weak light signals captured using single-photon detectors (e.g., SNSPD). Electromagnetic data includes changes in magnetic field gradient, electric field intensity, electromagnetic spectrum characteristics, and low-frequency electric field data. Environmental sensing data includes water parameters (turbidity, salinity, temperature, etc.), water flow velocity and direction, noise levels, and seabed sediment data.
[0056] Step S12: Based on the acoustic data, determine the first suspicious area, and perform detail focusing and identification processing on the first suspicious area using optical data;
[0057] In this embodiment, acoustic data is first used to determine the areas where target objects (underwater robots, underwater targets, etc.) may exist. Then, optical data is used to further identify the details of these areas, thereby making a preliminary judgment on the areas where target objects may exist.
[0058] Specifically, an 8-channel ring transducer array (center frequency 300kHz, bandwidth ±100kHz) is first deployed to support beamforming and multi-target tracking, and an integrated turbidity sensor (NTU measurement range 0-200) is used to provide real-time feedback to the transmit power controller. Then, a 532nm wavelength laser (output power 1W) is used in conjunction with a galvanometer to achieve 0.1° precision pointing control for data scanning of the target area. During acoustic data acquisition, to achieve anti-turbidity signal processing, a linear frequency modulated signal (50-500kHz) is used, combined with Costas coding to suppress multipath interference. Simultaneously, minimum variance distortionless response beamforming is employed to focus energy onto the suspected area, and the array gain distribution is dynamically adjusted based on turbidity feedback (NTU value) and background noise. The detection threshold is dynamically adjusted based on the acoustic level, and areas with a signal-to-noise ratio >12dB (the first suspicious area) are marked. Then, based on the pre-built correlation, the coordinates of the suspicious area are converted into an optical table, and combined with data from fiber optic gyroscopes (INS) and Doppler logs (DVL), coordinate offsets caused by device movement are corrected in real time. After that, the laser centerline of the laser sensor is extracted, the three-dimensional contour of the target surface is calculated, and the reflective features of the suspicious area are identified through phase consistency analysis, thereby obtaining the detailed focusing identification results. By seamlessly connecting acoustic anti-turbidity coarse positioning with optical high-precision identification, the target detection efficiency in complex environments can be effectively improved. At the same time, based on real-time parameter adjustment of environmental parameters (NTU value, motion attitude), it can adapt to the needs of all scenarios from nearshore turbid waters to deep-sea environments.
[0059] It should be noted that during this process, acoustic and optical data need to be cross-validated repeatedly. For example, if the optical recognition confidence level is less than 70%, a second acoustic scan is triggered (increasing the transmission power by 10dB); or if the acoustic rescan still fails to confirm the target, the magnetoelectric sensor is activated to assist in detection. When multiple suspicious areas appear simultaneously, targets with high optical recognition confidence and strong acoustic echo intensity are prioritized. If the optical sensor fails to detect the target for three consecutive frames, the mode is switched to a broad-spectrum imaging mode and the search range is expanded (±30°). If the target still cannot be locked, the mode is reverted to the acoustic global scan mode, and so on.
[0060] Step S13: Based on the detail focusing recognition processing result, match the first electromagnetic data and the first environmental data of the first suspicious area from the sensing data, wherein the environmental perception data includes the first environmental data;
[0061] In this embodiment, after obtaining the acoustic and optical details, the corresponding electromagnetic data is further determined, and the electromagnetic data is further adjusted based on the environmental data, thereby more accurately determining the location of the target object.
[0062] Specifically, first, the optical coordinate system (x i,y i ,z i The data is converted to an electromagnetic coordinate system, and then a target spatiotemporal window (±0.1m spatial range, ±10ms time window) is extracted from the electromagnetic data stream to obtain the magnetic field gradient. The system uses electric field pulse signals and calculates the difference between the magnetic field strength of the target area and the background mean to determine the magnetic field difference (e.g., a difference greater than 50 nT triggers a metal target alarm). It also detects 1-10 Hz bioelectric signals using short-time Fourier transform (STFT) to distinguish the underwater robot from surrounding organisms. Subsequently, it extracts environmental data such as turbidity (NTU), salinity (PSU), and temperature (°C) within the target's spatiotemporal window and calculates correction coefficients for environmental factors on the electromagnetic signals.
[0063]
[0064] In the formula, k Env The correction factor is NTU for turbidity, S for salinity, T for temperature, α = 0.2, β = 0.1, δ = 0.4. Of course, if the salinity is >40 PSU or the turbidity is >100 NTU, the weight of electromagnetic data is reduced by 50%, and optical data is used preferentially.
[0065] Specifically, if the electromagnetic signal is overwhelmed by strong interference, the historical electromagnetic fingerprint database will be used for matching (similarity > 80% is considered valid).
[0066] Step S14: Determine the location information of the underwater target based on the electromagnetic data and the environmental perception data.
[0067] In this embodiment, after determining the electromagnetic data and correcting it with environmental perception data, the data is fused using the extended Kalman algorithm to output the precise location information of the underwater target.
[0068] For low-frequency electric field detection, it is necessary to deploy high-sensitivity charge sensing electrodes (such as copper / iron materials, with an input impedance >100GΩ) to capture the axial frequency electric field (1-10Hz) or magnetic field gradient (0.1nT resolution) generated by the underwater target. The charge sensing chip converts the signal into a voltage signal and filters it to reduce noise. The water depth of the data acquisition module is determined by a flexible cable to realize the spatial distribution measurement of the electric field of the underwater target.
[0069] By following the steps above, and by integrating multiple sensors to locate and detect underwater targets from multiple dimensions, the problem of low positioning accuracy of underwater targets is solved, and the positioning accuracy is improved.
[0070] The entities that perform the above steps can be base stations, terminals, etc., but are not limited to these.
[0071] In an optional embodiment, determining a first suspicious region based on the acoustic data and performing detail focusing and identification processing on the first suspicious region using optical data includes:
[0072] Step S121: Dynamically adjust the noise based on the acoustic data, and perform density clustering and outlier removal on the acoustic cloud points in the acoustic data based on the detection threshold to obtain the center point coordinates and edge coordinates of the first suspicious area.
[0073] Step S122: Convert the center point coordinates and the region edge coordinates into optical camera coordinates to obtain the target center point coordinates and the target region edge coordinates.
[0074] Step S123: The image data contained in the optical data is restored according to the preset underwater optical transmission model to obtain the first image;
[0075] Step S124: Perform detail focus recognition processing on the first image based on the coordinates of the target center point and the coordinates of the target region edge to obtain the detail focus recognition processing result.
[0076] In this embodiment, the background noise intensity (such as noise power spectral density) is monitored in real time, and the detection threshold is dynamically adjusted according to the background noise level. Areas with a signal-to-noise ratio >12dB are marked as suspicious areas. Of course, the threshold can also be dynamically updated by calculating the local signal-to-noise ratio (SNR) and combining it with Kalman filtering to predict noise changes. This is not limited here.
[0077] Subsequently, the DBSCAN equal-density clustering algorithm was used to cluster acoustic cloud points (such as sonar point clouds), identifying high-density regions as potential targets, and determining the center point coordinates (x, y) of the first suspicious region. c ,y c ,z c ) and region edge coordinates (x ce ,y ce ,z ce Among them, key parameters such as neighborhood radius (eps, assumed to be 5m) and minimum number of points (minPts, assumed to be 5) need to be optimized according to sonar resolution and target size. For example, for small targets, eps can be reduced to capture details, while removing isolated points in the cluster with fewer than the threshold is to reduce noise interference.
[0078] The coordinates of the center point and the edge of the region in the acoustic sensor coordinate system are transformed into the optical camera coordinate system. Assuming the relative positions and orientations of the acoustic sensor and the optical camera are known, the transformation can be accomplished using a coordinate transformation matrix:
[0079]
[0080] In the formula, R is the rotation matrix and T is the translation vector. For optical coordinates, Using acoustic coordinates, the center point coordinates and the edge coordinates of the region are then substituted into the calculation to obtain the target center point coordinates and the target region edge coordinates in optical coordinates.
[0081] Then, based on the underwater optical transmission model I(x)=J(x)e -βz +B(1-e -βz The transmittance t(x) is estimated a priori through the dark channel to restore the clear image J(x) (i.e. the first image mentioned above). During the restoration process, it is necessary to compensate for the image distortion caused by water absorption and scattering.
[0082] Next, based on the coordinates of the target center point and the edge coordinates of the target region, the target region is cropped from the first image. The cropped image is then preprocessed, including grayscale conversion, noise reduction (such as Gaussian filtering), and contrast enhancement, to improve image quality. Then, image processing algorithms (such as edge detection and feature extraction) are used to perform detail focusing recognition on the cropped image. For example, the Canny edge detection algorithm is used to extract the contour information of the target, and the extracted edge information is analyzed to identify the shape, texture, and other features of the target. At the same time, based on a preset feature library (such as the known shape and texture pattern of the target), the type and confidence level of the target are determined. The type, texture features, and confidence level of the target are then output as the result of detail focusing recognition to the subsequent processing module for further target confirmation and localization.
[0083] In an optional embodiment, prior to acquiring the sensing data, the method further includes:
[0084] Step S101: Focus energy onto the first suspected region by minimum variance distortionless response beamforming;
[0085] Step S102: Optimize the transmission waveform parameters using a Bayesian probability model to dynamically match the water turbidity of the first suspected area;
[0086] Step S103: Adjust the array gain distribution of the acoustic data dynamically based on the water turbidity, wherein the detection threshold is obtained based on the array gain distribution.
[0087] In this embodiment, assuming the acoustic sensor array is a uniform linear array or a ring array, its output signal can be expressed as:
[0088]
[0089] In the formula, The beam weight vector, Let be the input signal vector of the sensor array. express The conjugate transpose; subsequently, the beam weight vector It can be calculated using the following formula:
[0090]
[0091] In the formula, Let be the covariance matrix of the input signal. θ is the guide vector for the target direction, and θ is the target direction angle.
[0092] Then, based on the preset target direction or the estimated location of the suspicious area, the corresponding guide vector is calculated. And using the optimized beam weight w opt The acoustic sensor array is weighted to direct the beam toward the first suspicious area, thereby increasing the signal strength in that area.
[0093] A Bayesian probabilistic model is used to describe the relationship between transmitted waveform parameters and water turbidity. Assume the transmitted waveform parameters are... If the turbidity of the water body is T, then the Bayesian model can be expressed as:
[0094]
[0095] In the formula, Let be the conditional probability, representing the transmitted waveform parameters given a specific turbidity T in the water body. The probability of; Let be the likelihood function, representing the likelihood given the transmitted waveform parameters. The probability of water turbidity T under the following conditions; Let P(T) be the prior probability, representing the prior distribution of the transmitted waveform parameters; P(T) is the normalization constant.
[0096] Based on the Bayesian probability model, the optimal emission waveform parameter popt is calculated for the current water turbidity T. For example, the optimal parameter can be selected by maximizing the posterior probability P(p|T).
[0097]
[0098] Then, adjust the sonar system's transmission signal based on the optimized transmission waveform parameters, such as adjusting the signal frequency, bandwidth, or modulation method, to adapt to the current water environment.
[0099] It should be noted that dynamically adjusting the gain distribution of the acoustic sensor array includes:
[0100]
[0101] Where G(T) is the adjusted array gain distribution. This is an adjustment function.
[0102] In an optional embodiment, determining the location information of the underwater target based on the electromagnetic data and the environmental sensing data includes:
[0103] Step S141: Determine the electric field gradient change information of the first suspicious region based on the first electromagnetic data;
[0104] Step S142: Determine the object characteristics of the first usable region based on the electric field gradient change information and the first environmental data;
[0105] Step S143: If the object features meet the first condition, the pre-obtained target center point coordinates and target area edge coordinates are fused using the extended Kalman filter algorithm to obtain the position information.
[0106] In this embodiment, recording electric and magnetic fields over time facilitates the establishment of correlations with acoustic and optical data.
[0107] Specifically, the measured values of electric field strength (E) and magnetic field strength (B) are expressed as follows:
[0108] E(t) = [E x (t),E y (t),E z (t)] (Formula 8)
[0109] B(t) = [B x (t),B y (t),B z (t)] (Formula 9)
[0110] In the formula, E x (t),E y (t),E z (t) and B x (t),B y (t),B z (t) represents the components of the electric field and magnetic field on the three coordinate axes, respectively.
[0111] Simultaneously, spatial gradient calculations are performed on the electric field intensity data to obtain information on electric field gradient changes; the electric field gradient can be calculated using the finite difference method.
[0112]
[0113] The specific calculation formula is as follows:
[0114]
[0115] Where Δx, Δy, and Δz represent the spatial resolution of the sensor.
[0116] Next, calculate the magnitude of the electric field gradient:
[0117]
[0118] This analysis examines the characteristics of the electric field gradient changes, such as the peak position and direction of gradient change. Based on the corrected electric field gradient information, the characteristics of objects in suspicious areas are then determined. For example, if the difference between the magnetic field strength and the background magnetic field exceeds 50 nT, it is identified as a metallic target; if a 1-10 Hz bioelectric signal is detected, it is identified as an underwater organism; if the peak position of the electric field gradient matches the target position in the optical data, further confirmation is performed, such as matching the extracted features with a pre-defined target feature library to determine the target type and confidence level. For example, the degree of matching can be evaluated by calculating feature similarity (such as Euclidean distance or cosine similarity).
[0119] The correction formula is as follows:
[0120]
[0121] The process of fusing the pre-obtained target center point coordinates and target region edge coordinates using the extended Kalman filter algorithm includes:
[0122] Define state vector This includes the target's position coordinates and velocity vector:
[0123]
[0124] And initialize the state estimate. Covariance Matrix For example, suppose the initial position is the coordinates of the target's center point in the optical data, and the initial velocity is zero.
[0125]
[0126] Where, σ x ,σ y ,σ z Given the initial uncertainty of the location, The initial uncertainty of the velocity.
[0127] Assuming the target's motion is uniform linear motion, its discrete-time motion model is as follows:
[0128]
[0129] in, Here is the state transition matrix:
[0130]
[0131] w k-1 The process noise is assumed to be zero-mean Gaussian white noise.
[0132] By combining the target location information from electromagnetic and optical data, an observation model is defined:
[0133]
[0134] in, The observation vector includes the coordinates of the target center point and the coordinates of the region edge; The observation matrix is used to map the state vector to the observation space v. k To observe the noise, we assume it to be zero-mean Gaussian white noise.
[0135] Subsequently, position prediction is performed based on the extended Kalman filter:
[0136]
[0137] Where Q is the process noise covariance matrix, k-1 is the data from the previous time step, and k is the data from the current time step.
[0138] In an optional embodiment, after matching the first electromagnetic data and the first environmental data of the first suspicious area from the sensing data based on the detail focus identification processing result, wherein the environmental perception data includes the first environmental data, the method further includes:
[0139] Based on the environmental perception data, the weights of the sensor data and the electromagnetic data are dynamically adjusted and allocated.
[0140] In this embodiment, dynamically adjusting the weights can adapt to different usage environments, thereby further improving detection accuracy.
[0141] Specifically, when turbidity NTU > 100, the weight of optical data is reduced and the weight of acoustic data is increased; when salinity PSU > 40, the weight of electromagnetic data is reduced and the weight of optical data is increased; when background noise exceeds 60 dB, the weight of acoustic data is reduced and the weight of electromagnetic data is increased.
[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0143] This embodiment also provides an online positioning device for underwater robots based on multiple sensors. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0144] Figure 2 This is a structural block diagram of an online positioning device for an underwater robot based on multiple sensors according to an embodiment of the present invention, such as... Figure 2 As shown, the device includes:
[0145] The data acquisition module 21 is used to acquire sensor data, wherein the sensor data includes acoustic data, optical data, electromagnetic data and environmental perception data.
[0146] The first processing module 22 is used to determine a first suspicious area based on the acoustic data, and to perform detail focusing and identification processing on the first suspicious area using optical data;
[0147] Matching module 23 is used to match the first electromagnetic data and the first environmental data of the first suspicious area from the sensing data based on the detail focusing recognition processing result, wherein the environmental sensing data includes the first environmental data;
[0148] The location determination module 24 is used to determine the location information of the underwater target based on the electromagnetic data and the environmental perception data.
[0149] In an optional embodiment, determining a first suspicious region based on the acoustic data and performing detail focusing and identification processing on the first suspicious region using optical data includes:
[0150] The noise is dynamically adjusted based on the acoustic data, and density clustering and isolated point removal are performed on the acoustic cloud points in the acoustic data based on the detection threshold to obtain the center point coordinates and edge coordinates of the first suspicious area.
[0151] The coordinates of the center point and the coordinates of the edge of the region are converted into optical camera coordinates to obtain the coordinates of the target center point and the coordinates of the target region edge.
[0152] The image data contained in the optical data is restored according to the preset underwater optical transmission model to obtain the first image;
[0153] The first image is subjected to detail focusing recognition processing based on the coordinates of the target center point and the edge coordinates of the target region to obtain the detail focusing recognition processing result.
[0154] In an optional embodiment, the apparatus further includes:
[0155] A beamforming module is used to focus energy onto the first suspected region by minimum variance distortionless response beamforming before acquiring the sensing data.
[0156] The dynamic matching module is used to optimize the transmission waveform parameters through a Bayesian probability model and dynamically match the water turbidity of the first suspected area.
[0157] A dynamic adjustment module is used to dynamically adjust the array gain distribution of the acoustic data based on the water turbidity, wherein the detection threshold is obtained based on the array gain distribution.
[0158] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.
[0159] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.
[0160] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.
[0161] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.
[0162] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.
[0163] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0164] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0165] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0166] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0167] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0168] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A multi-sensor-based online positioning method for underwater robots, characterized in that, include: Acquire sensor data, wherein the sensor data includes acoustic data, optical data, electromagnetic data, and environmental perception data; Based on the acoustic data, a first suspicious area is determined, and the first suspicious area is then subjected to detail focusing and identification processing using optical data. Based on the detail-focused identification processing results, the first electromagnetic data and the first environmental data of the first suspicious area are matched from the sensing data. The environmental sensing data includes the first environmental data; wherein, the first electromagnetic data is obtained after correction based on the first environmental data. ； ； In the formula, For electromagnetic data, The corrected first electromagnetic data, The correction coefficients are: NTU, S, and T. NTU represents the turbidity in the first environmental data; S represents the salinity in the first environmental data; and T represents the temperature in the first environmental data. α, β, and δ are the corresponding weights. The location information of the underwater target is determined based on the first electromagnetic data and the first environmental data. Prior to acquiring the sensor data, the method further includes: Minimum variance distortionless response beamforming is used to focus energy onto the first suspected region; The transmission waveform parameters are optimized using a Bayesian probability model to dynamically match the water turbidity of the first suspected area. The array gain distribution of the acoustic data is dynamically adjusted based on the water turbidity, and the detection threshold is determined based on the array gain distribution.
2. The method according to claim 1, characterized in that, The process of determining a first suspicious region based on the acoustic data and performing detail focusing and identification processing on the first suspicious region using optical data includes: The noise is dynamically adjusted based on the acoustic data, and density clustering and isolated point removal are performed on the acoustic cloud points in the acoustic data based on the detection threshold to obtain the center point coordinates and edge coordinates of the first suspicious area. The coordinates of the center point and the coordinates of the edge of the region are converted into optical camera coordinates to obtain the coordinates of the target center point and the coordinates of the target region edge. The image data contained in the optical data is restored according to the preset underwater optical transmission model to obtain the first image; The first image is subjected to detail focusing recognition processing based on the coordinates of the target center point and the edge coordinates of the target region to obtain the detail focusing recognition processing result.
3. The method according to claim 1, characterized in that, Determining the location information of the underwater target based on the electromagnetic data and the environmental sensing data includes: Based on the first electromagnetic data, determine the electric field gradient change information of the first suspicious area; Based on the electric field gradient change information and the first environmental data, the object characteristics of the first suspicious area are determined; If the object features meet the first condition, the pre-obtained coordinates of the target center point and the coordinates of the target region edge are fused using the extended Kalman filter algorithm to obtain the location information.
4. The method according to claim 1, characterized in that, After matching the first electromagnetic data and first environmental data of the first suspicious area from the sensing data based on the detail focus identification processing result, wherein the environmental sensing data includes the first environmental data, the method further includes: Based on the environmental perception data, the weights of the sensor data and the electromagnetic data are dynamically adjusted and allocated.
5. An online positioning device for underwater robots based on multiple sensors, characterized in that, include: The data acquisition module is used to acquire sensor data, which includes acoustic data, optical data, electromagnetic data, and environmental perception data. The first processing module is used to determine a first suspicious area based on the acoustic data, and to perform detail focusing and identification processing on the first suspicious area using optical data; The matching module is used to match the first electromagnetic data and the first environmental data of the first suspicious area from the sensing data based on the detail focusing recognition processing result. The environmental sensing data includes the first environmental data; wherein the first electromagnetic data is obtained after correction based on the first environmental data. ； ； In the formula, For electromagnetic data, The corrected first electromagnetic data, The correction coefficients are: NTU, S, and T. NTU represents the turbidity in the first environmental data; S represents the salinity in the first environmental data; and T represents the temperature in the first environmental data. α, β, and δ are the corresponding weights. The location determination module is used to determine the location information of the underwater target based on the first electromagnetic data and the first environmental data. The device further includes: A beamforming module is used to focus energy onto the first suspected region by minimum variance distortionless response beamforming before acquiring the sensing data. The dynamic matching module is used to optimize the transmission waveform parameters through a Bayesian probability model and dynamically match the water turbidity of the first suspected area. The dynamic adjustment module is used to dynamically adjust the array gain distribution of the acoustic data based on the turbidity of the water body, and to determine the detection threshold based on the array gain distribution.
6. The apparatus according to claim 5, characterized in that, The process of determining a first suspicious region based on the acoustic data and performing detail focusing and identification processing on the first suspicious region using optical data includes: The noise is dynamically adjusted based on the acoustic data, and density clustering and isolated point removal are performed on the acoustic cloud points in the acoustic data based on the detection threshold to obtain the center point coordinates and edge coordinates of the first suspicious area. The coordinates of the center point and the coordinates of the edge of the region are converted into optical camera coordinates to obtain the coordinates of the target center point and the coordinates of the target region edge. The image data contained in the optical data is restored according to the preset underwater optical transmission model to obtain the first image; The first image is subjected to detail focusing recognition processing based on the coordinates of the target center point and the edge coordinates of the target region to obtain the detail focusing recognition processing result.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to perform the method described in any one of claims 1 to 4 when executed.
8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method described in any one of claims 1 to 4.