Design method of in-situ fine detection framework for deep sea based on acousto-optic remote sensing and VR technology

Abstract

This invention discloses a design method for a deep-sea in-situ fine-scale detection framework based on acoustic-optical remote sensing and VR technology. The main steps include: (1) studying the structure of the photon-counting lidar scanning system and the dynamic geometric relationship between the mother ship and the submersible, and constructing a calculation model for the lidar bathymetry point coordinates in the WGS84 spatial rectangular coordinate system; (2) considering the submersible's attitude, and based on the ray-tracking model and the geometric relationship between the mother ship and the submersible, constructing a calculation model for the multibeam bathymetry point coordinates in the WGS84 spatial rectangular coordinate system; (3) fusing the lidar and multibeam bathymetry points to construct a seabed topographic map; (4) proposing a development scheme and detection method for a deep-sea functional imaging spectrometer; and (5) constructing a VR environment for the deep-sea target survey area using a three-dimensional seabed topography model, rendering technology, and binocular stereo imaging technology. Through these steps, deep-sea in-situ fine-scale detection based on acoustic-optical remote sensing technology and the construction of a VR environment for the survey area can be achieved.

Description

Technical Field

[0001] This invention relates to a design method for a deep-sea in-situ fine exploration framework based on acoustic-optical remote sensing and VR technology, and particularly to a design method for a deep-sea in-situ fine exploration framework based on acoustic-optical remote sensing and VR technology for large depths. Background Technology

[0002] my country possesses enormous potential for marine resource exploration and development, with its deep-sea areas harboring abundant strategic resources such as oil and gas, minerals, and biological resources. However, the current overall proven rate of my country's marine resources is only 12.1%, far below the world average, which places higher demands on marine exploration technologies. In recent years, my country's pace of marine resource and energy development has accelerated significantly, and vigorously developing refined deep-sea resource and energy exploration technologies and equipment has become a major national strategic need.

[0003] Due to space limitations, this review only summarizes typical research findings related to the research on deep-sea 3D lidar, multi-functional combined spectrometers, deep-sea submersibles, and real-time 3D imaging and positioning technologies for deep-sea target areas.

[0004] 1. Current Status of Foreign Deep-Sea Optical Detection Instruments and Equipment Technology

[0005] (1) Deep-sea 3D lidar

[0006] Underwater 3D laser imaging technology mainly includes range gating technology, synchronous scanning technology, stripe tube camera technology, continuous light amplitude modulation technology, structured light imaging technology, and single photon detection technology.

[0007] 1) Range gating technique. The application of range gating technique in underwater imaging began in the 1960s. [1] While it can effectively reduce the impact of water backscattering on imaging detection, range gating techniques can only provide two-dimensional image results. Until 2019, researchers at the Technical University of Denmark combined time-of-flight technology with gating techniques to achieve three-dimensional imaging of dynamic underwater targets in real seawater. [2] 2) Synchronous scanning technology. Synchronous scanning technology improves the signal-to-noise ratio of imaging detection by reducing the impact of backscattering on the received signal by isolating the transmitted and received beams. In 1996, the U.S. Naval Surface Warfare Center researched a laser line-scanning multispectral imaging system using synchronous scanning technology for imaging underwater coral reef environments. [3] 3) Streak camera technology. The emergence of streak cameras has brought new possibilities to underwater 3D imaging. In 1999, McLean et al. applied streak cameras to a lidar system, mounted on an airborne platform, and combined with an underwater electro-optical discrimination system to achieve underwater 3D imaging with a resolution better than 1 inch. [4]This system can be used to detect underwater targets such as mines. 4) Structured light imaging. Structured light imaging uses a narrow laser beam to project onto the target object, and the beam direction is off-axis from the camera direction to reduce the effect of backscattering. Dalgleish et al. applied the structured light projection method to the reconstruction of underwater targets and made a series of improvements. [5] 5) Combination of multibeam sonar and scanning laser beams. The combination of traditional sonar and optical technologies can also be used for three-dimensional structural imaging of underwater targets. In 2018, researchers at the Technical University of Gdansk in Poland studied a method for reconstructing underwater targets using a combination of multibeam sonar and scanning laser beams, obtaining three-dimensional images of an underwater shipwreck. [6] 6) Application of single-photon imaging technology in underwater 3D imaging. In 2019, researchers at Heriot-Watt University in Edinburgh used picosecond resolution silicon single-photon avalanche diode single-photon detector array technology to achieve 3D imaging of moving underwater targets in turbid waters. In a dark environment, they achieved a target detection range of 6.7 attenuation lengths in a pool experiment. [7] .

[0008] (2) Deep-sea multi-functional imaging spectrometer

[0009] The MBARI (Marine Research Institute) in the United States is a pioneer in applying laser Raman spectroscopy to ocean exploration. In 2004, Peter and colleagues at the institute developed the world's first deep-sea laser Raman spectroscopy detection system—DORISS (Deep Ocean Raman In-Situ Spectrometer). [8] The DORISS system has been used multiple times to conduct deep-sea experiments with the Tiburon ROV and Ventana ROV, achieving remarkable research results in key scientific issues such as natural gas hydrate detection, hydrothermal activity detection, the role of deep-sea environmental changes in global climate change and carbon cycle, and sediment pore water.

[0010] In 2007, the second-generation DORISS II system was successfully developed. [9] The DORISS II system integrates the spectrometer, laser, and control and communication systems within a titanium alloy pressure tank, enabling it to operate at depths of up to 4000 meters on the seabed. The DORISS II system offers significantly improved reliability compared to the DORISS system, while also experiencing substantial reductions in weight and power consumption.

[0011] The French Ifremer Institute for Oceanography developed an underwater Raman spectroscopy system capable of reaching depths of up to 6,000 meters in 2005, using a small Raman spectrometer from Jobin Yvon.

[10] The system has completed shallow-sea performance testing in the Baltic Sea.

[0012] (3) Deep-sea submersible

[0013] Submersibles typically fall into three categories: unmanned autonomous underwater vehicles (AUVs), unmanned remotely operated vehicles (ROVs), and manned submersibles (HOVs). The technical challenges increase with increasing diving depth. For HOVs, as early as January 1960, the Trieste deep-sea submersible successfully reached the deepest point of the Mariana Trench, becoming the first human to reach the Challenger Deep. Second-generation deep-sea manned submersibles from abroad include the American Alvin, the Russian Mir, the French Nautilus, and the Japanese Shinkai 6000, with a maximum diving depth of 6000 meters. In 2012, the Deepsea Challenger single-person manned submersible returned to the Mariana Trench.

[11] HOT Corporation of the United States has completed the development and testing of the Deep Flight Challenger prototype, a full-ocean-depth manned submersible.

[12] Regarding ROVs, Japan built the full-ocean-depth "Kaiko" as early as 1995, but this submersible was lost in 2003. In 2009, the "Poseidon" composite unmanned submersible developed by the Woods Hole Oceanographic Institution in the United States successfully dived to the Mariana Trench, reaching a depth of 10,902 meters, but the "Poseidon" was lost in 2014. For AUVs, the traditional maximum depth is 6,000 meters.

[13] Until May 2020, the Russian unmanned deep-sea submersible "Vityaz" descended to the bottom of the Mariana Trench and recorded a depth of 10,028 meters.

[0014] (4) Real-time three-dimensional imaging study of deep-sea target areas

[0015] 1) Research on 3D spatial reconstruction algorithm for multi-site scanning of deep-sea 3D lidar. From 1996 to 1999, field tests of the STIL system in marine and coastal environments demonstrated that STIL can provide high-resolution 3D imaging for various applications such as bathymetry and precise target localization and identification.

[14] Between 2009 and 2011, dozens of ALMDS systems were deployed on the U.S. Navy's Littoral Combat Ships (LCS) and Navy MH-60S helicopters.

[15] The above test results verify the STIL system's ability to achieve high frame rate, high resolution, and high-precision 3D imaging. Brown categorizes image registration structures into four types: feature space selection, search strategy selection, search space determination, and similarity criterion measurement.

[16] Faugeras et al. first studied the problem of arbitrary shape matching in 3D data and proposed a method for least-squares minimization based on corresponding 3D point sets using unit quadruples.

[17] Among free-joining algorithms, the iterative nearest-point method proposed by Besl et al. is the most representative.

[18] .

[0016] 2) Research on high-precision 3D self-localization algorithm for deep-sea submersibles using multiple sensors. Navigation is one of the major technical challenges faced by underwater vehicles. Currently, the autonomous navigation of foreign underwater vehicles mainly relies on a combination of strapdown inertial navigation systems and Doppler logs, supplemented by corrections from the Global Positioning System, underwater acoustic positioning systems, and magnetic compasses.

[19] Since most dynamic systems are nonlinear, standard Kalman filtering cannot accurately estimate their performance. SLAM methods based on Extended Kalman Filter (EKF) achieve optimal estimation of the SLAM problem by linearizing the nonlinear motion and measurement models. However, the EKF-SLAM method is not suitable for complex environments with strong nonlinearity and dense environmental feature points.

[20] Fast SLAM can effectively overcome nonlinear problems.

[21] In 2009, Arasaratnam and Haykin proposed the CKF filtering algorithm.

[22] Based on the volume criterion for probabilistic deduction, CKF has significant advantages in terms of numerical accuracy and filtering stability.

[0017] 2. Current Status of Domestic Deep-Sea Optical Detection Instruments and Equipment Technology

[0018] (1) Deep-sea 3D lidar

[0019] Domestic institutions such as the Shanghai Institute of Optics and Fine Mechanics of the Chinese Academy of Sciences, Harbin Institute of Technology, Beijing Institute of Technology, Huazhong University of Science and Technology, and Ocean University of China have conducted relevant systematic and experimental research.

[0020] Since 2002, the Shanghai Institute of Optics and Fine Mechanics has developed an airborne laser depth sounding system, pioneering three-dimensional mapping of seabed topography in China. After three generations of performance upgrades and modifications, the product's performance has reached the forefront in China and is comparable to similar foreign products. In 2017, the Mapper5000 product was finalized, and in 11 flight tests in the South China Sea, it obtained three-dimensional topographic data of islands and reefs in the South China Sea.

[23] Since 2005, Professor Sun Jianfeng's research group at Harbin Institute of Technology has been conducting research on stripe tube laser imaging radar technology in China, developing STIL with independent intellectual property rights, and carrying out underwater target detection experiments in the Yellow Sea.

[24] In 2008, Ocean University of China proposed a real-time display method for 3D information of underwater target detection based on laser digital scanning grids. This method can suppress backscattering noise in a wide field of view and obtain relative depth information between different detection points.

[25] In 2010, Li Hailan of Beijing Institute of Technology successfully created a 3D image of multiple wooden planks placed inside a water pipe. However, the imaging process required frame-by-frame reconstruction, which was time-consuming.

[26] In 2016, Huazhong University of Science and Technology proposed a range-gated 3D imaging method for underwater targets at known distances, and successfully performed 3D imaging of an underwater target at a distance of 15 meters.

[27] In 2018, Tian Zhaoshuo of Harbin Institute of Technology designed a UAV-borne lidar 3D imaging system based on triangulation normal scanning and performed scanning in shallow coastal waters.

[28] .

[0021] (2) Deep-sea multi-functional imaging spectrometer

[0022] Research on in-situ underwater spectral detection in China started relatively late. Zheng Rong'er and colleagues at Ocean University of China developed China's first deep-sea self-contained laser Raman spectroscopy detection (DOCARS) system.

[29] In 2010, Du Zengfeng et al. obtained the world's first dual-wavelength excited in-situ Raman spectrum of a self-contained sample at a depth of 4003m, verifying the DOCARS system's ability to quantitatively detect common acid radicals. Zhang Xin et al. from the Institute of Oceanology, Chinese Academy of Sciences, developed China's first probe-type deep-sea laser Raman spectroscopy (RiP) system, and in 2016, they conducted the first in-situ quantitative detection and analysis of hydrothermal fluids with temperatures as high as 290℃.

[30] The team led by Li Can at the Dalian Institute of Chemical Physics, Chinese Academy of Sciences, has developed the world's first deep-sea Raman spectrometer using ultraviolet lasers as the excitation source.

[31] In 2017, a 7000m sea trial was conducted in the Mariana Trench. In 2018, Ocean University of China launched a small underwater Raman spectroscopy system.

[32] It has been successfully tested and applied in deep-sea cold seeps and hydrothermal vent areas. In 2019, Zhang Xin and others discovered for the first time an inverted lake formed by numerous "mushroom-shaped" hydrothermal chimney structures in a deep-sea hydrothermal vent area.

[33] .

[0023] Due to the diversity of materials in the deep-sea environment, existing single-wavelength light sources or simple Raman and fluorescence detection are no longer sufficient to meet the needs of marine exploration. Furthermore, the complexity of the deep-sea environment means that deploying detectors to predetermined locations or controlling equipment with robotic arms results in low positioning accuracy, affecting the actual detection capabilities of the instruments.

[0024] (3) Deep-sea submersible

[0025] In June 2012, the Jiaolong manned submersible achieved a dive of 7,062 meters in the Mariana Trench in the Pacific Ocean, marking China as the fifth country to master the technology of manned deep-sea diving to depths of 6,000 meters.

[0026] During the 13th Five-Year Plan period, my country vigorously promoted the research and development of full-ocean-depth manned and unmanned submersibles. It is expected that 2020 will mark a peak in the achievements of full-ocean-depth submersibles, which will also promote the development and leapfrog development of my country's manned / unmanned deep-sea submersible technology.

[0027] In 2016, with funding from the Shanghai Science and Technology Commission's "Science and Technology Innovation Action Project," the Deep-Sea Center of Shanghai Ocean University completed the development of three full-ocean-depth landers and one full-ocean-depth unmanned submersible. The three landers reached the Challenger Deep in the Mariana Trench, with a maximum diving depth of 10,890 meters, while the unmanned submersible reached 6,300 meters. Simultaneously, the "Tianya" and "Haijiao" landers, developed by the Institute of Deep-Sea Science and Engineering, Chinese Academy of Sciences, reached the deepest point in the Mariana Trench. Also in 2016, the "Qianlong II" 4,500-meter-class AUV, jointly developed by the Shenyang Institute of Automation, Chinese Academy of Sciences, and the Second Institute of Oceanography, successfully explored hydrothermal activity areas on the Southwest Indian Ridge, obtaining data and photographs on the topography, polymetallic sulfides, and marine life of the test area. In 2016, the "Haidou" unmanned submersible, developed by the Shenyang Institute of Automation, Chinese Academy of Sciences, reached the deepest point in the Mariana Trench but was unfortunately lost during a test voyage. In June 2020, the "Haidou-1" full-ocean-depth autonomous remotely operated vehicle (ROV), developed by the Shenyang Institute of Automation, Chinese Academy of Sciences, successfully completed its first 10,000-meter sea trial and experimental application test in the Mariana Trench, reaching a maximum diving depth of 10,907 meters, breaking my country's record for the deepest dive by a submersible. Currently, the 702nd Research Institute of China Shipbuilding Industry Corporation is developing a full-ocean-depth manned submersible, while Harbin Engineering University is developing a full-ocean-depth unmanned autonomous submersible. In Shanghai, Shanghai Jiao Tong University is developing a full-ocean-depth unmanned submersible, and the Deep-Sea Center of Shanghai Ocean University is developing a second-generation full-ocean-depth operational unmanned submersible.

[0028] (4) Research on real-time three-dimensional imaging and positioning technology for deep-sea target areas

[0029] 1) Research on 3D spatial reconstruction algorithm for multi-site scanning of deep-sea 3D lidar. Regarding high-precision imaging modeling of lidar systems, Zhao Mingbo et al. established a model of the interaction process between lidar signals and target scenes based on the response function of the laser beam projection point to the lidar signal.

[34] Wang Tianjiao et al. improved the radar equations of a flash focal plane laser imaging radar.

[35] He Jin et al., taking into account various factors such as system structure, atmospheric attenuation, background noise, and target scattering characteristics, established a simulation model and conducted a detailed analysis of the imaging process and resolution.

[36] In the field of lidar image stitching, in 2004, Zhao Xiangyang et al. proposed a fully automatic and robust stitching algorithm.

[37] The algorithm achieved good results in experiments. In 2009, Wei Xueli et al. proposed an image stitching algorithm that uses maximum mutual information as the matching metric.

[38] In 2013, He Bin et al. proposed a high real-time F-SIFT stitching algorithm.

[39] It maintains the characteristics of fast computation without sacrificing the accuracy of the SIFT algorithm.

[0030] 2) Research on high-precision 3D self-localization algorithm for deep-sea submersibles using multiple sensors. Gu Dongqing et al. proposed the UKF method to solve the alignment problem of strapdown inertial navigation systems (INS) during motion, addressing the initial alignment problem under large azimuth misalignment angles.

[40] Cheng Yali proposed an EKF navigation and localization method based on line features and elaborated on it in detail.

[41] Zhang Jie processed the forward-looking sonar information stream to obtain a feature map of the underwater environment, and used the EKF navigation and positioning algorithm to simulate the autonomous localization and navigation of UUVs.

[42] To address the challenge of precise navigation for AUVs due to the unique underwater environment, Zhang Tao et al. designed an AUV underwater navigation and positioning system based on a tight combination of SINS / LBL.

[43] Xu Xiaosu et al. proposed an underwater navigation and positioning method that combines terrain and environmental features.

[44] .

[0031] 3. Development Trends of Deep-Sea Exploration Technology

[0032] (1) Systematization. Advances in deep-sea exploration technology have made large-scale, high-precision, and quasi-synchronous global deep-sea exploration possible. The systematization of deep-sea exploration technology helps to acquire multidisciplinary, multi-scale, three-dimensional, and long-term deep-sea exploration data. Based on massive amounts of data, new theories, models, and methods can be adopted to promote scientific research in fields such as deep-sea dynamic environment, deep-sea geophysical field, and deep-sea engineering geology.

[0033] (2) Collaboration. Deep-sea exploration methods are limited and extremely costly. Collaborative operations are the development direction of the next generation of deep-sea exploration technology. By combining emerging technologies such as artificial intelligence, environmental perception, and communication control, deploying a large number of deep-sea vehicles, represented by untethered autonomous underwater vehicles, in specific sea areas is expected to achieve large-scale and multi-platform network operations and improve collaborative exploration capabilities.

[0034] (3) Intelligentization. In recent years, intelligent technologies, represented by intelligent sensing, augmented virtual reality, and deep learning, have received much attention and are developing rapidly, integrating into and changing human life and production. In the field of deep-sea exploration, artificial intelligence technology can be used to identify and extract exploration targets, diagnose and control faults in exploration equipment, and efficiently perceive the deep-sea environment, thus promoting the comprehensive intelligentization of deep-sea exploration technology.

[0035] References:

[0036] [1]Heckman Jr P.J.Underwater Range Gated Photography[C].In UnderwaterPhoto Optics I,Year:60-68.

[0037] [2]Mariani P.,Quincoces I.,Haugholt K.H.,et al.Range-Gated ImagingSystem for Underwater Monitoring in Ocean Environment[J].Sustainability,2019,11(1).

[0038] [3]Strand M.P.,Coles B.W.,Nevis A.J.,et al.Laser line-scanfluorescence and multispectral imaging of coral reef environments[C].In OceanOptics XIII,Year:790-795.

[0039] [4]McLean J.W.High-resolution 3D underwater imaging[C].In Airborneand In-Water Underwater Imaging,Year:10-19.

[0040] [5]Narasimhan S.G.,Nayar S.K.,Sun B.,et al.Structured light inscattering media[C].In Tenth IEEE International Conference on Computer Vision(ICCV'05)Volume 1,Year:420-427.

[0041] [6]Kulawiak M.,Lubniewski Z.3D OBJECT SHAPE RECONSTRUCTION FROMUNDERWATER MULTIBEAM DATA AND OVER GROUND LIDAR SCANNING[J].Polish MaritimeResearch,2018,25(2):47-56.

[0042] [7]Maccarone A.,Della Rocca F.M.,McCarthy A.,et al.Three-dimensionalimaging of stationary and moving targets in turbid underwater environmentsusing a single-photon detector array[J].Optics Express,2019,27(20):28437-28456.

[0043] [8]Brewer P.G.,Malby G.,Pasteris J.D.,et al.(2004)Development of alaser Raman spectrometer for deep-ocean science[J].Deep Sea Research Part I,2004,51(5):0-753.

[0044] [9]White S.(2006)Laser Raman Spectroscopy as a Tool for In SituMineralogical Analyses on the Seafloor.OCEANS 2006.10.1109 / OCEANS.2006.307048.

[0045]

[10] Lehaitre M.,Charlou J.L.,Donval J.P.(2005)Raman Spectroscopy:newperspectives for in situ shallow or deep ocean exporation,GeophysicalResearch Abstracts,2005,7.

[0046]

[11] Fletcher B.,Bowen A.,Yoerger D.R.et al.(2009)Journey to thechallenger deep:50years later with the Nereus hybrid remotely operatedvehicle[J].Marine Technology Society Journal,43(5):65-76.

[0047]

[12] Hawkes G. (2012) Available on line: http: / / www.deepflight.com / project / deepflight - challenger [EB / OL].

[0048]

[13] Kudo K. (2008) Overseas trends in the development of human occupied deep submersibles and a proposal for Japan’s way to take [J], Quarterly Review, No.26: 104 - 123.

[0049]

[14] Redman B.C., Griffis A.J., and Schibley E.B. (2000) Streak tube imaging lidar (STIL) for 3 - D imaging of terrestrial targets. Tucson: Arete Associates.

[0050]

[15] Liu Shangfu, Hu Hui (2017). Overseas trends in the development of lidar. Ship Electronic Engineering, 37(6): 1 - 4.

[0051]

[16] Brown, L.G. (1992) A survey of image registration techniques. ACM Computing Surveys, 24(4): 325 - 376.

[0052]

[17] Faugeras O.D., Hebert M. (1986) The Presentation, recognition and locating of 3D objects. Robotics Research, 5(3): 27 - 52.

[0053]

[18] Besl P.J., McKay N.D. (1992). A method of registration of 3D shapes. IEEE Trans. On Pattern Analysis and Machine Intelligence, 14(2): 239 - 256.

[0054]

[19] Morgado M.,Oliveira P.,Silvestre C.,and JFVasconcelos.(2014).Embedded Vehicle Dynamics Aiding for USBL / INS Underwater NavigationSystem.IEEE Transactions on Control Systems Technology 22(1):322-330.

[0055]

[20] Havangi R.(2016)Robust SLAM:SLAM base on square root unscentedKalman filter.Nonlinear dynamics 83(1):767-779.

[0056]

[21] Arulampalam,Maskell S.,Gordon N.,Clapp T.(2002).A Tutorial onParticle Filters for Online Nonlinear / Non-Gaussian Bayesian Tracking.IEEETransactions on Signal Processing 50(2):174-188.

[0057]

[22] Arasaratnam I.,Haykin S.,Hurd TR(2010)Cubature Kalman Filtering for Continuous-Discrete Systems: Theory and Simulations.IEEE Transactions onSignal Processing,58(10):4977-4993.

[0058]

[23] He Yan, Hu Shanjiang et al. (2018) Research progress on domestic airborne dual-frequency lidar detection technology [J]. Progress in Lasers and Optoelectronics. Vol.08, 082801.

[0059]

[24] Sun Jianfeng, Gao Jian, et al. (2010) Research progress on underwater detection imaging of stripe tube laser imaging radar [J]. Infrared and Laser Engineering, 39(5):811-814.

[0060]

[25] Zheng B.,Hou J.,Wang G.(2008)An approach of 3D information real-time display for detection of underwater target based on laser digitalscanning grid[C].OCEANS 2008.Piscataway,NJ,USA:IEEE,978-1-4244-2619-5.

[0061]

[26] Li Hailan, Wang Xia, Jin Weiqi et al. (2010) A three-dimensional reconstruction method based on multi-frame underwater range-gated images [J]. Acta Photonica Sinica, 30(12):3464-3470.

[0062]

[27] Huang Ziheng, Li Wei, Yang Kecheng et al. (2016) Underwater laser range-gated three-dimensional imaging method [J]. Laser & Infrared, 46(11):1315-1319.

[0063]

[28] Wu Yuhao, Tian Zhaoshuo, et al. (2020) Study on three-dimensional imaging of shallow waters by laser line scanning from UAV[J]. Laser & Infrared, 50(2):143-147.

[0064]

[29] Du Z., Li Y., Chen J., et al. (2015) Feasibility investigation on deepocean compact autonomous Raman spectrometer developed for in-situ detection of acid radical ions[J]. Chinese journal of oceanology and limnology, 2015, 33(2): 545-550.

[0065]

[30] Zhang X., Du Z., Luan Z., et al. (2017) In situ Raman Raman Detectionof Gas Hydrates Exposed on the Seafloor of the South China Sea[J]. Geochemistry Geophysics Geosystems, 2017.

[0066]

[31] Fan F., Feng Z., Li C.(2010)UV Raman spectroscopic study on thesynthesis mechanism and assembly of molecular sieves[J].Chemical SocietyReviews, 39(12):p.1-9.

[0067]

[32] Liu Qingsheng, Guo Jinjia, Yang Dewang et al. (2018) Small high-sensitivity underwater Raman spectroscopy system [J]. Optics and Precision Engineering.

[0068]

[33] Li L., Zhang

[0069]

[34] Zhao Mingbo, He Jun, Fu Qiang (2012) Modeling, simulation and analysis of full waveform lidar echo signal. Acta Optica Sinica, 32(6):238-251.

[0070]

[35] Wang Tianjiao, Sun Jianfeng, Wang Xuefeng (2012) Improvement and simulation of equations for flash focal plane laser imaging radar. Infrared and Laser Engineering, 41(3):626-631.

[0071]

[36] He Jin, Zhang Qun, Yang Xiaoyou et al. (2012) Modeling and imaging simulation of inverse synthetic aperture imaging lidar system. Journal of System Simulation, 24(3):632-637.

[0072]

[37] Zhao Xiangyang, Du Limin (2004) A fully automatic and robust image stitching and fusion algorithm. Journal of Image and Graphics, 9(4):417-422.

[0073]

[38] Wei Xueli, Zhang Hua, Ma Yanjie (2009) Image stitching optimization algorithm based on maximum mutual information. Optoelectronics Laser, 20(10):1399-1402.

[0074]

[39] He Bin, Tao Dan, Peng Bo. (2013) High real-time F-SIFT image stitching algorithm. Infrared and Laser Engineering, 42(52):440-444.

[0075]

[40] Gu Dongqing, Qin Yongyuan. (2006) UKF design for alignment of marine strapdown inertial navigation system during motion. Systems Engineering and Electronics, 28(8):1218-1220.

[0076]

[41] Cheng Yali. (2007) Research on Simultaneous Localization and Mapping Algorithm Based on Extended Kalman Filter, Ocean University of China.

[0077]

[42] Zhang Jie. (2008) Research on simultaneous localization and mapping technology of underwater robots based on sonar, Ocean University of China.

[0078]

[43] Zhang Tao, Shi Hongfei, Xu Xiaosu. (2015) AUV underwater navigation and positioning technology based on SINS / LBL tight combination. Journal of Chinese Inertial Technology, (4): 86-90.

[0079]

[44] Xu Xiaosu, Tang Junjun, Li Peijuan, Zhang Tao, Yue Zengyang. (2015) An underwater navigation and positioning method combining terrain and environmental features. Journal of Chinese Inertial Technology, 23(5):590-596.

[0080]

[45] Zhai Guojun, Huang Motao, Ouyang Yongzhong. Key technologies in the development of airborne laser depth sounding system [J]. Marine Surveying, 2014, 34(3):73-76.

[0081]

[46] Wang Alian, Yu Jianyuan, Zeng Pei. System calibration of laser Raman spectrometer [J]. Spectroscopy and Spectral Analysis, 1993, 13(5):37-45.

[0082]

[47] He Ning. Research on DSP-based marine detection laser fluorescence spectrometer [D]. Harbin Institute of Technology, 2016.

[0083]

[48] ​​Ding Guoqiang. Research on key issues of transfer alignment technology in inertial navigation system [D]. Harbin Engineering University, 2010.

[0084]

[49] Zou Haichao. Research on Doppler speed measurement and its display control system software design [D]. Harbin Engineering University, 2008.

[0085]

[50] Niu Qingzheng. Design and implementation of detection device for ultra-short baseline positioning system [D]. Harbin: Harbin Engineering University, 2017.

[0086]

[51] Lai Xianghua, Pan Guofu, Fu Xiaoming, et al. Application of single-beam bathymetry in submarine pipeline inspection [J]. Ocean Engineering, 2007, 25(4):66-72.

[0087]

[52] He Linbang, Zhao Jianhu, Zhang Hongmei, et al. A method for accurate tracking of multi-beam acoustic rays considering attitude angle [J]. Journal of Harbin Engineering University, 2015, (1): 46-50.

[0088]

[53] You Xiong. Battlefield environment simulation based on virtual reality technology [J]. Acta Geodaetica et Cartographica Sinica, 2002, 31(1):7-11. Summary of the Invention

[0089] The technical problem this invention aims to solve is the low resolution of traditional contact-based detection of cold seeps and target area scanning. It designs a deep-sea in-situ fine-scale detection technology framework based on acoustic-optical remote sensing and VR technology. A calculation model of lidar and multibeam bathymetry point cloud coordinates in the WGS84 spatial rectangular coordinate system is constructed and fused to improve the resolution of seabed topography inversion. A highly realistic seabed survey area surrounding environment is constructed using 3D seabed virtual reality technology. Simultaneously, a multifunctional deep-sea spectroscopic imager is developed to conduct close-range, non-contact monitoring of leaking gases such as methane in cold seep areas, aiming to obtain the gas leakage concentration per unit time and provide technical support for global greenhouse effect research.

[0090] To achieve the above objectives, the present invention provides a deep-sea in-situ fine detection framework design method based on acoustic-optical remote sensing and VR technology, comprising the following steps:

[0091] (1) Study the structure of the photon counting lidar scanning system, and study the dynamic geometric relationship between the mother ship's ultra-short baseline and the submersible beacon. Construct a calculation model of the lidar underwater depth sounding point coordinates in the WGS84 spatial rectangular coordinate system, and realize the splicing of adjacent strip lidar point clouds.

[0092] (2) Taking into account the changes in the attitude of the submersible, based on the acoustic tracking model, the geometric relationship between the mother ship's ultra-short baseline and the submersible's beacon, a calculation model for the coordinates of the multibeam sonar sounding point in the WGS84 spatial rectangular coordinate system is constructed.

[0093] (3) It is beneficial to normalize the laser radar and multibeam scattering intensity data by Z-score, and to fuse the laser radar sounding points and multibeam sonar sounding points under the near-identical rule to construct the fused seabed topography map.

[0094] (4) A development scheme and detection method for a deep-sea functional imaging spectrometer were proposed;

[0095] (5) By establishing a model of the three-dimensional seabed topography, using rendering technology and binocular stereo imaging technology, a VR environment for the deep-sea target survey area is constructed.

[0096] In one embodiment of the present invention, a calculation model for the coordinates of underwater depth sounding points of a lidar system in the WGS84 spatial rectangular coordinate system is constructed, and the stitching of adjacent strip lidar point clouds is realized. This mainly includes the following steps:

[0097] (1) Establishment of the laser radar scanning reference coordinate system and its transition coordinate system;

[0098] (2) Calculation of the normal vector of the reflector in the laser radar scanning reference coordinate system;

[0099] (3) Calculation of the nadir angle and azimuth angle of the reflected light in the laser radar scanning reference coordinate system;

[0100] (4) Calculation of underwater depth sounding point coordinates in the lidar scanning reference coordinate system;

[0101] (5) Calculation of coordinates of laser underwater depth sounding points in the coordinate system of the submersible beacon array;

[0102] (6) Establishment of the geometric relationship between the submersible beacon array and the mother ship's ultra-short baseline center;

[0103] (7) Calculation of the coordinates of the submersible beacon array in the WGS84 spatial rectangular coordinate system;

[0104] (8) The laser underwater sounding point is repositioned in the WGS84 spatial rectangular coordinate system;

[0105] (9) Adjacent strip laser point cloud splicing based on the Plücker coordinate description rule of straight line.

[0106] In one embodiment of the present invention, the calculation model for the coordinates of multibeam sonar sounding points in the WGS84 spatial rectangular coordinate system mainly includes the following steps:

[0107] (1) Taking into account the changes in the attitude of the submersible, calculate the initial incident angle of the multibeam sonar beam;

[0108] (2) Establish a sound ray tracking model;

[0109] (3) Calculate the coordinates of the beam footprint in the transducer coordinate system;

[0110] (4) The beam footprint coordinates are repositioned into the submersible beacon array coordinate system;

[0111] (5) Establish the geometric relationship between the submersible beacon array and the mother ship's ultra-short baseline;

[0112] (6) Relocate the beam footprint coordinates to the WGS84 spatial rectangular coordinate system.

[0113] In one embodiment of the present invention, the fusion processing of lidar bathymetry points and multibeam sonar bathymetry points to construct a seabed topographic map after data fusion mainly includes the following steps:

[0114] (1) It is beneficial for Z-score to normalize lidar and multibeam scattering intensity data;

[0115] (2) Conversion between Z-score and gray value;

[0116] (3) Merging of point clouds of lidar and multibeam sonar near the same point;

[0117] (4) Geographic coding;

[0118] (5) Image resampling.

[0119] In one embodiment of the present invention, the development scheme and detection technology of the deep-sea functional imaging spectrometer mainly include the following steps:

[0120] (1) The development plan of the imaging spectrometer was determined from five aspects: parameter simulation analysis, overall scheme design, unit scheme design, prototype development and experimental verification.

[0121] (2) Conduct research on four aspects: detection technology using three-wavelength Raman and fluorescence, binocular vision imaging technology, high-precision in-situ measurement and positioning technology, and probe separation technology.

[0122] (3) Conduct research on the detection and identification technology of Raman and fluorescence spectroscopy multiplexing for deep-sea materials;

[0123] (4) Conduct research on deep-sea high-precision imaging positioning and spectral measurement error calibration technology.

[0124] In one embodiment of the present invention, constructing a VR environment for a deep-sea target survey area mainly includes the following steps:

[0125] (1) Based on the dynamic segmentation and merging mechanism of incomplete binary tree, a TIN seabed topography model of the entire survey area is established.

[0126] (2) Multicolor gradient rendering technology for seabed terrain based on HSL color model to render the terrain at different depths of the seabed.

[0127] (3) A binocular stereo imaging method is proposed;

[0128] (4) Construction of a virtual environment that integrates a three-dimensional seabed terrain model and a binocular stereo imaging model of the target.

[0129] The technical problems to be solved by this invention mainly include the following aspects:

[0130] (1) Construct a calculation model for the coordinates of underwater depth sounding points of lidar in the WGS84 spatial rectangular coordinate system;

[0131] (2) Construct a calculation model for the coordinates of multibeam sonar sounding points in the WGS84 spatial rectangular coordinate system;

[0132] (3) A method for fusing lidar sounding points and multibeam sonar sounding points is proposed to construct a seabed topographic map after data fusion.

[0133] (4) Propose a development scheme for a deep-sea functional imaging spectrometer and its detection technology methods;

[0134] (5) Utilize the three-dimensional topography of the seabed to build a model, rendering technology and binocular stereo imaging technology to construct a VR environment for the deep-sea target survey area.

[0135] The beneficial effects of this utility model through the above technical solution are:

[0136] (1) By studying the scanning structure of photon counting lidar, a calculation model of the underwater depth sounding point coordinates of lidar under the WGS84 spatial rectangular coordinate system is constructed. By fusing the lidar point cloud and the multibeam sonar point cloud, the resolution of the seabed topography map can be greatly improved.

[0137] (2) Develop a deep-sea functional spectral imager that can effectively improve the sensitivity of target detection and effectively reduce the noise of the surrounding environment of the target.

[0138] (3) Using the three-dimensional topography of the seabed to build a model, rendering technology and binocular stereo imaging technology to construct a VR environment for the deep-sea survey area can increase the visual sense of deep-sea cold seep detection. Attached Figure Description

[0139] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0140] Figure 1 This is a schematic diagram of the elliptical scanning system of the lidar of the present invention;

[0141] Figure 2 These are the two Cartesian coordinate systems of the normal direction vector of the reflector in this invention;

[0142] Figure 3 This refers to the geometric angle of the reflected light rays in the sensor coordinate system according to the present invention;

[0143] Figure 4 This is a schematic diagram of the geometric structure of the emitted laser calculated from the change of normal in this invention;

[0144] Figure 5 This is a schematic diagram of the underwater laser radar projection point of the present invention.

[0145] Figure 6 This is a schematic diagram of the ultra-short baseline underwater acoustic positioning principle of the present invention;

[0146] Figure 7 The spatial helical motion represented by the Plücker linear representation of the present invention;

[0147] Figure 8 This invention relates to the image resampling and scanning filling method;

[0148] Figure 9 This is a technical roadmap for the deep-sea multifunctional combined imaging spectrometer of the present invention;

[0149] Figure 10 This is a flowchart illustrating the construction and visualization of the deep-sea seabed virtual environment dataset for the present invention.

[0150] Figure 11 This is a flowchart of a framework design method for deep-sea in-situ fine exploration based on acoustic-optical remote sensing and VR technology. Detailed Implementation

[0151] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0152] First, this invention relates to the following technical terms:

[0153] Ocean lidar depth sounding system

[0154] A marine lidar depth sounding system is a complex active depth sounding system that integrates multiple technologies, including laser ranging, attitude determination, computer processing, environmental parameter correction, and measurement data processing.

[45] .

[0155] Laser Raman spectrometer

[0156] A laser Raman spectrometer is a comprehensive measurement system integrating laser spectroscopy, precision mechanics, and microelectronics. Its ultimate result is the spectrum of the intensity of scattered light with a specific polarization state in a given direction, distributed as a function of frequency.

[46] .

[0157] Fluorescence spectrometer

[0158] A fluorescence spectrometer, also known as a fluorescence spectrophotometer, is an instrument for qualitative and quantitative analysis. Through detection using a fluorescence spectrometer, one can obtain information such as the excitation spectrum, emission spectrum, quantum yield, fluorescence intensity, fluorescence lifetime, Stokes shift, fluorescence polarization and depolarization characteristics, and fluorescence quenching.

[47] .

[0159] Inertial navigation system

[0160] An inertial navigation system (INS) is an autonomous navigation system that does not rely on external information and does not radiate energy to the outside.

[48] Its working environment includes not only the air and ground, but also underwater. The basic working principle of inertial navigation is based on Newton's laws of motion. By measuring the acceleration of the carrier in the inertial reference frame, integrating it over time, and transforming it into the navigation coordinate system, information such as velocity, yaw angle, and position in the navigation coordinate system can be obtained.

[0161] Doppler log

[0162] A Doppler log is a precision instrument for measuring speed and accumulating distance, based on the Doppler effect of sound waves in water.

[49] This device uses the Doppler frequency shift between the emitted sound waves and the received reflected waves from the seabed to measure the ship's speed and cumulative distance relative to the seabed. Doppler logs are limited by water depth; beyond several hundred meters, they can only use water particles in the water layer as a reflector. At this depth, they can measure the flow velocity and direction of different water layers, but they cannot measure the ship's speed relative to the seabed.

[0163] Ultra-short baseline underwater acoustic positioning system

[0164] The ultra-short baseline positioning system consists of a transmitting transducer, a transponder, and a receiving array. The transmitting transducer and receiving array are mounted on a ship, while the transponder is fixed to the underwater hull. The transmitting transducer emits an acoustic pulse, which the transponder receives and then transmits back. The receiving array receives this pulse, measures the phase difference in the X and Y directions, and calculates the distance R from the underwater device to the array based on the arrival time of the sound waves. This allows the system to calculate the underwater detector's position on a planar coordinate system and its depth.

[50] .

[0165] Multibeam sonar depth sounding technology

[0166] A transducer emits a short-pulse sound wave (beam) with a specific spatial direction into the water. The sound wave propagates through the water, and upon encountering the seabed, it undergoes reflection, transmission, and scattering. The reflected echo is received by the transducer. Knowing the time interval t between the transducer's emission and reception of the echo, and the average speed V of sound in the water, the one-way travel distance Z of the sound wave, i.e., the distance from the transducer to the seabed, can be calculated.

[51] .

[0167] Sound tracking method

[0168] Sound ray tracking is a method for calculating the beam footprint (projection point) relative to the ship's coordinate system, based on sound speed profiles.

[52] Sound ray tracking typically employs a layer-addition method, which divides two adjacent sound velocity sampling points within a sound velocity profile into a layer. The sound velocity variation within a layer can be assumed to be constant (zero gradient) or constant gradient. The former is simpler in terms of calculation concept and process, while the latter is relatively more complex. In the latter sound ray tracking calculation, the sound velocity variation function uses the Harmonic average sound velocity.

[0169] Virtual Reality (VR) Technology

[0170] Virtual reality (VR), also known as virtual environment, virtual world, or artificial environment, refers to a technology that uses computers to generate a virtual world that allows participants to directly experience visual, auditory, and tactile sensations and interact with it.

[53] .

[0171] The design method for a deep-sea in-situ fine exploration framework based on acoustic-optical remote sensing and VR technology, as described in this invention, mainly includes the following steps:

[0172] (1) Study the structure of the photon counting lidar scanning system, and study the dynamic geometric relationship between the mother ship's ultra-short baseline and the submersible beacon. Construct a calculation model of the lidar underwater depth sounding point coordinates in the WGS84 spatial rectangular coordinate system, and realize the splicing of adjacent strip lidar point clouds.

[0173] (2) Taking into account the changes in the attitude of the submersible, based on the acoustic tracking model, the geometric relationship between the mother ship's ultra-short baseline and the submersible's beacon, a calculation model for the coordinates of the multibeam sonar sounding point in the WGS84 spatial rectangular coordinate system is constructed.

[0174] (3) It is beneficial to normalize the laser radar and multibeam scattering intensity data by Z-score, and to fuse the laser radar sounding points and multibeam sonar sounding points under the near-identical rule to construct the fused seabed topography map.

[0175] (4) Propose a development plan for a deep-sea functional imaging spectrometer and study its multiplexing detection technology;

[0176] (5) By establishing a model of the three-dimensional seabed topography, using rendering technology and binocular stereo imaging technology, a VR environment for the deep-sea target survey area is constructed.

[0177] See Figures 1 to 11 As shown, the specific embodiments of the present invention will now be described in detail below:

[0178] (1) Overall technical solution

[0179] First, the structure of the photon-counting lidar scanning system is studied. Based on the transmission relationship between the mother ship's ultra-short baseline and the submersible beacon, a calculation model for the underwater bathymetry coordinates of the lidar in the WGS84 spatial rectangular coordinate system is constructed, enabling the stitching of adjacent strip lidar point clouds. Second, considering the submersible's attitude changes, a calculation model for the coordinates of the multibeam sonar bathymetry coordinates in the WGS84 spatial rectangular coordinate system is constructed based on the ray tracking model and the geometric relationship between the mother ship's ultra-short baseline and the submersible beacon. Third, the lidar and multibeam scattering intensity data are normalized using Z-scores, and the lidar and multibeam sonar bathymetry points are fused under near-identical rules to construct a fused seabed topographic map. Then, a development scheme and detection method for a deep-sea functional imaging spectrometer are proposed. Finally, a VR environment for the deep-sea target survey area is constructed using a three-dimensional seabed topography model, rendering technology, and binocular stereo imaging technology.

[0180] (2) Calculation of the coordinates of the scanning foot points of the lidar in the WGS84 spatial rectangular coordinate system

[0181] 1) Structure of the LiDAR Elliptical Scanning System

[0182] The marine lidar described in this article uses a conventional elliptical scanning structure. Figure 1 As shown, a prism that can rotate around a rotation axis is used as a reflector to control the direction of the emitted laser beam. The emitted laser is reflected by the prism and points towards the sea surface. The angle between the normal direction of the prism and the rotation axis is 7.5°. When the prism surface rotates around the rotation axis, the laser traces a path on the sea surface with an incident angle of approximately 15°. Since the incident angle is not always equal to 15° during one scan (depending on the normal direction), the final laser point trajectory on the sea surface when the aircraft is hovering is an approximately elliptical oval shape. Therefore, this scanning structure is also called an oval scanning structure.

[0183] 2) LiDAR scanning reference coordinate system

[0184] Definition of the lidar scanning reference coordinate system: with the center point of the reflector as the origin O, X... s The axis points in the negative direction of the emitted laser beam, Y. s The axis points in the direction of flight, Z s Axis and X s Y s Construct a right-handed coordinate system with the axes pointing vertically upwards. The incident laser and the motor axis are in the same plane (X). s Z s (surface), laser incident horizontally (along X) s (In the negative direction of the axis), the incident point of the laser beam on the mirror is the center of the mirror. For ease of understanding, as... Figure 2As shown, the original X s Y s Z s coordinate system around Y s Rotating the axis counterclockwise by 45° yields a new coordinate system X. s 'Y s ′Z s ′, at this time Z s The rotation direction of the axis and the motor coincides. The normal of the reflector is in the X direction. s Z s Projection of the surface and Z s Angle between axes In Y s Z s Projection of the surface and Z s Angle between axes

[0185] like Figure 3 As shown, the reflected light rays in X s Z s Y s Z s The angles between the projection of the plane and the Z-axis are φ. x φ y Its nadir angle is φ. Because the laser beam travels along the X... s Incident along the negative direction of the axis, with the normal at Y. s Z s Projection of the surface and Z s Angle between axes Equal to the reflected laser in Y s Z s Projection of the surface and Z s The included angle φ of the axis y (Because the incident laser line, the mirror normal, and the reflected laser line are coplanar, and the incident laser line is perpendicular to Y) s Z s According to the theorem that if one plane passes through a perpendicular line to another plane, the two planes are orthogonal, therefore, in Y... s Z s On a plane, the angle of rotation of the normal is synchronized with the angle of rotation of the reflected ray (i.e., the normal rotates by an angle θ, and the reflected ray also rotates by an angle θ). However, in the X... s Z s On a plane, when the mirror rotates (i.e., the normal) by an angle θ, the reflected ray rotates by an angle 2θ. When the normal angle changes, the included angle... And so it changed, from φ is easy to solve x And then calculate the nadir angle φ and azimuth angle of the beam. Therefore, the change in the angle of the normal is the key.

[0186] 3) Direction vector of the mirror normal

[0187] exist Figure 2 In the middle, the normal of the mirror is in X s 'Y s ′Z s The normal vector of the coordinate system (F) x′ ,F y′ ,F z′ ):

[0188]

[0189] Then by going around Y s Rotating the coordinate axis 45° clockwise will yield the X coordinate. s Y s Z s The normal vector of the mirror in the coordinate system (F) x ,F y ,F z ):

[0190]

[0191] 4) The relevant angles of the reflected light rays in the laser scanning reference coordinate system

[0192] Depend on Figure 2 , Figure 4 According to geometric relationships, Therefore:

[0193] φ x (θ)=2arctan(F x / |F z |)-90° (3)

[0194] Depend on Figure 2 According to geometric relationships, and so:

[0195] φ y (θ)=arctan(F y / |F z |) (4)

[0196] Depend on Figure 3 The geometric relationship yields the nadir angle φ and the azimuth angle. for:

[0197]

[0198]

[0199] 5) Coordinates of underwater sounding points in the lidar scanning reference coordinate system

[0200] like Figure 5As shown, the lidar on the submersible emits laser light that passes through only one medium (water). The center of the laser is denoted as S, the underwater depth sounding point is P1, the distance from the center of the reflector to the seabed is H, the slant range of the laser beam underwater is L1, and the azimuth angle is... The coordinates of the laser projection point on the seabed are:

[0201] x s =Htan(φ) x (7)

[0202] y s =Htan(φ) y (8)

[0203] z s =-H (9)

[0204] 6) Coordinates of the underwater depth sounding points of the lidar in the coordinate system of the submersible beacon array.

[0205]

[0206] In equation (10), (X) s ,Y s Z s R1(yaw,pitch,roll) represents the coordinates of the underwater depth sounding point of the lidar in the coordinate system of the submersible beacon array; R1(yaw,pitch,roll) is the rotation matrix for transforming the body coordinate system to the local navigation coordinate system. It includes two parts: the eccentricity difference between the center of the laser scanning reference coordinate system and the center of the submersible IMU body coordinate system, and the eccentricity difference between the center of the beacon array and the center of the IMU body coordinate system. The offset angle of the laser scanning reference coordinate system relative to the IMU body coordinate system.

[0207] 7) Coordinates of the submersible beacon array in the WGS84 Cartesian coordinate system

[0208] The coordinates of the submersible beacon array in the WGS84 Cartesian coordinate system were calculated using the coordinates of four transducers at different locations on the surface mothership and the ship's attitude. The origin was defined as the intersection of the diagonals of the four transducer locations, with the bow direction of the mothership as the x-axis, the starboard direction as the y-axis, and the vertical downward direction as the z-axis. Figure 6 As shown, let the coordinates of the submersible beacon transponder be T(T x T y T z There are 4 mothership hydrophones located at the vertices of a rectangle with side lengths of 2a and 2b, and their coordinates are H1(a,-b,0), H2(a,b,0), H3(-a,-b,0), and H4(-a,b,0).

[0209] Without considering the curvature of the vocal tract, the localization equation can be obtained from geometric relationships:

[0210]

[0211]

[0212]

[0213]

[0214] Eliminating z, we get

[0215]

[0216]

[0217]

[0218]

[0219] The solution is:

[0220]

[0221]

[0222] From any of equations (11) to (14), z can be solved, thus obtaining four possible depth values:

[0223]

[0224]

[0225]

[0226]

[0227] The mean depth can be obtained by averaging the four values, i.e.

[0228]

[0229] The slant R from each hydrophone to the seabed transponder i It can be obtained through the following formula:

[0230]

[0231] T0 is the round-trip time between the ship's central intercom and the submersible's beacon transponder, t i This refers to the signal transmission time from the question-and-answer device to the transponder and back to each hydrophone.

[0232] In practice, the center of the mother ship's transducer array coordinate system and the IMU center are located at different positions, and their three axes have a certain degree of skewness. These factors need to be considered in the repositioning calculation of the submersible beacon array. The conversion formula for repositioning the submersible beacon array coordinates to the WGS84 Cartesian coordinate system is as follows:

[0233]

[0234] And (X) GPS ,Y GPS Z GPS )for:

[0235]

[0236] In equations (27) and (28), (X) R-wgs84 ,Y R-wgs84 Z R-wgs84 (X) represents the coordinates of the submersible beacon array in the WGS84 Cartesian coordinate system; GPS ,Y GPS Z GPS The coordinates of the center of the GPS antenna on the mother ship in the WGS84 Cartesian coordinate system; R(yaw,pitch,roll) is the rotation matrix for transforming the body coordinate system to the local navigation coordinate system; It includes two parts: the eccentricity difference between the center of the mother ship transducer array coordinate system and the center of the IMU body coordinate system, and the eccentricity difference between the center of the GPS antenna and the center of the IMU body coordinate system. The offset angle of the mother ship transducer array coordinate system relative to the IMU body coordinate system.

[0237] After repositioning calculation, the coordinates of the submersible beacon array center in the WGS84 spatial rectangular coordinate system can be dynamically obtained.

[0238] 8) Coordinates of the underwater depth sounding points of the lidar in the WGS84 rectangular coordinate system

[0239] After calculating the coordinates of the underwater sounding point of the lidar in the coordinate system of the submersible beacon array and the coordinate system of the beacon array in the WGS84 coordinate system, the coordinates of the underwater sounding point of the lidar in the WGS84 rectangular coordinate system can be obtained, which are:

[0240]

[0241] 9) Stitching of lidar point clouds

[0242] First, a point cloud of a specific strip is identified as the reference, called the reference strip point cloud. The strips to be stitched together are called the LiDAR point clouds of the strips to be stitched. The essence of LiDAR point cloud stitching is to plan the point clouds acquired from different strips into a unified coordinate system, which involves solving for the rotation and translation parameters between the coordinate system of the strip to be stitched and the coordinate system of the reference strip. This process generally treats the point clouds as rigid body motion, solving for the rigid body motion parameters through corresponding features between the point clouds.

[0243] Based on the LiDAR point cloud stitching principle under linear constraints, the corresponding linear features of the point cloud to be stitched and the reference point cloud are described by the Plücker coordinates of the lines. Then, the algebraic relationship between the two overlapping Plücker lines is described by relevant formulas, and the collinearity condition equation is established to realize the calculation of stitching parameters.

[0244] like Figure 7 As shown, l1 and l2 are the same straight lines on the station cloud to be spliced ​​and the reference station cloud, respectively. The geometric meaning of the Plücker coordinate transformation of these two straight lines in three-dimensional space, described by dual quaternions, is: l1 along the vector The direction is translated by a distance d to obtain l1′, and l1′ is around Rotation by an angle θ yields l2. Wherein, the vector... O1 is the common perpendicular of l1 and l2, pointing from o1 to o2. S1 and S2 are mutually parallel planes that contain l1 and l2 respectively. Figure 7 It can be seen from this that l1′ and l1 are parallel to each other, and l1′ and l2 are parallel to each other. Figure 7 The plane S2 shown is coplanar. The Plücker equation for this spatial transformation is:

[0245]

[0246] In the formula, and Let be the unit dual vector of lines l1 and l2, equal to the normalized Plücker coordinates. To describe the dual quaternion of this helical motion, for The inverse of . Since the unit dual vector is essentially a dual quaternion, for ease of calculation, we will use . In matrix form, it is:

[0247]

[0248] According to the condition that the lines coincide, the corresponding Plücker coordinates should be equal, therefore six equations can be listed:

[0249]

[0250] To estimate the unknowns using the least squares algorithm, the collinearity condition equations first need to be linearized according to the first partial derivatives of the unknowns. Equation (32) is then applied to... Expanding the Taylor formula down to the first-order term and discarding the second-order and higher-order terms, we obtain the linearized formula as follows:

[0251]

[0252] The approximation of the linearization process means that the collinearity condition equations F1 to F6 are not strictly true; therefore, the corresponding error equations are:

[0253]

[0254] In the formula, i represents the number of equations, ranging from 1 to 6, and x... j correspond The correction number for each element.

[0255] Equation (34) can be written in matrix form as follows:

[0256] V = AX + F (35)

[0257] In the formula,

[0258]

[0259] X = [dq1 dq2 dq3 dq4 dq] 01 dq 02 dq 03 dq 04 ] T

[0260] F = [F 10 -F1 F 20 -F2 F 30 -F3 F 40 -F4 F 50 -F5 F 60 -F6] T

[0261] Given the following two conditions: the real part and the dual part of the quaternion are orthogonal, and the modulus of the real part quaternion is 1:

[0262]

[0263] Linearizing the above equation, we obtain the matrix form of the constraint equations as follows:

[0264] BX+W=0 (37)

[0265] In the formula,

[0266]

[0267]

[0268] The overall adjustment formula for the LiDAR point cloud stitching model under Plücker linear constraints is:

[0269]

[0270] In equation (38), V and F are 6n-dimensional vectors, and A is a 6n×8-dimensional matrix. According to the least squares principle, the solution to this model is:

[0271] Y = -N -1 W Y (39)

[0272] In the formula,

[0273]

[0274] Because the coefficient matrices A and B of the Plücker collinearity condition equation are taken from the first-order terms of the Taylor series expansion during linearization, and the least squares estimation algorithm requires an initial value, the solution process is an iterative one. In each iteration, the approximate value of the unknown is added to the correction calculated in the previous iteration to obtain a new approximate value. This process is repeated to find the new correction for the unknown, and this approximation continues until the correction is less than a certain limit. The final dual quaternion is the LiDAR point cloud stitching parameter.

[0275]

[0276] (3) Calculation of coordinates of multibeam sounding points in WGS84 spatial rectangular coordinate system

[0277] 1) Coordinates of the multibeam echo sounder point in the transducer coordinate system

[0278] According to the literature by He Linbang et al., the initial incident angle and azimuth angle of the beam are as follows:

[0279]

[0280]

[0281] Here, θ i θ' is the actual incident angle of the beam. i Let α be the initial beam assignment angle, β be the angle of rotation about the OX axis, and β be the angle of rotation about the OY axis. The azimuth angle of the beam.

[0282] After calculating the initial incident angle of the beam, the coordinates of the beam footprint in the ship's coordinate system can be calculated using the constant gradient sound velocity tracking method. Assume the Harmonic average sound velocity of the sound ray propagating in the i-th layer is... The horizontal displacement Δy of the sound rays propagating within this layer i and time t i for:

[0283]

[0284]

[0285] In layered sound ray tracking, in addition to calculating the vertical and horizontal displacements and propagation time of the entire layer, it is also necessary to calculate the vertical and horizontal displacements of the remaining layers based on the remaining propagation time. Assume that the sound ray terminates at point r within the i-th layer during propagation, at which point the remaining time t... r Equal to the one-way travel time t of the beam all Subtracting the accumulated propagation time of the i-th layer and above, the vertical displacement Δz of the sound ray in the remaining layers is obtained. r and horizontal displacement Δy r for:

[0286]

[0287] The total vertical displacement z and horizontal displacement y of the sound ray propagation are:

[0288]

[0289] Based on the obtained horizontal and vertical displacements, and combined with the beam azimuth angle, the coordinates (X, Y, φ) of the beam footprint in the transducer coordinate system can be obtained. m ,Y m Z m ):

[0290]

[0291] 2) Coordinates of the beam footprint in the submersible beacon array coordinate system

[0292] After obtaining the coordinates of the beam footprint in the transducer coordinate system, the beam footprint coordinates are then reallocated to the submersible beacon array coordinate system through reallocation calculation.

[0293]

[0294] (X m-r ,Y m-r Z m-r R1(yaw, pitch, roll) represents the coordinates of the multibeam sonar beam footprint in the submersible beacon array coordinate system; R2(yaw, pitch, roll) is the rotation matrix for transforming the body coordinate system to the local navigation coordinate system. It includes two parts: the eccentricity difference between the center of the transducer coordinate system and the center of the IMU body coordinate system, and the eccentricity difference between the center of the submersible beacon array and the center of the IMU body coordinate system. The offset angle of the transducer coordinate system relative to the IMU body coordinate system.

[0295] 3) Coordinates of the beam footprint in the WGS84 Cartesian coordinate system

[0296] After calculating the coordinates of the beam footprint in the submersible beacon array coordinate system and the beacon array in the WGS84 Cartesian coordinate system, the coordinates of the beam footprint in the WGS Cartesian coordinate system can be obtained, which are:

[0297]

[0298] (4) Fusion of lidar and multibeam sonar data

[0299] 1) Facilitates data normalization using Z-scores

[0300] Because the backscattering intensity of lidar and multibeam sonar is affected by their respective emission mechanisms, the ranges of their backscattering intensity differ. Therefore, their backscattering intensity can be normalized using Z-scores before grayscale conversion.

[0301] Z-scores are a common mathematical and statistical method used to generalize parameters from multiple different ranges of variation to a common range of variation. For any set of scattering intensity sequences B (mean μ, standard deviation σ), the Z-score for any scattering intensity value b in B is:

[0302]

[0303] 2) Conversion between Z-score and grayscale value

[0304] Before mapping the seabed topography, the Z-score needs to be converted to grayscale values. The conversion formula is as follows:

[0305]

[0306] Where Z is the Z-fraction of the backscattering intensity, Z min Z is the smallest score in the sequence. max This represents the largest score in the sequence, where I is the grayscale value after linear quantization.

[0307] 3) Merging of near-identical point lidar and multibeam sonar point clouds

[0308] After obtaining the data of lidar sounding points and multibeam sonar sounding points, since their coordinates have been returned to the WGS84 spatial rectangular coordinate system, their sounding points form a full-coverage strip measurement. When two sounding points of different types meet the conditions of equation (52), it is considered that these two points are approximately the same point. Then the coordinates of the sounding points need to be reassigned, as shown in equation (53). Similarly, the gray value also needs to be reassigned, as shown in equation (54).

[0309]

[0310]

[0311]

[0312] 4) Geographic Coding

[0313] After linearly quantizing the scattering intensity data, it is necessary to calculate the specific pixel position of each sampling point in the image. Assuming the pixel resolution is res, the formula for calculating the position of the sampling point in the image is:

[0314]

[0315] Where (X) i Y i ) represent the pixel positions of the i-th sampling point in the image, and (x) represent the pixel positions of the i-th sampling point in the image. i y i ) are the geographical coordinates of the i-th sampling point, and (x) are the geographical coordinates of the i-th sampling point. min y min Then, it represents the minimum value of the geographic coordinates of the overall sampling points.

[0316] 5) Image resampling

[0317] Image resampling mainly addresses the gap problem caused by insufficient sampling rate in the track direction. Considering the imaging characteristics of lidar and multibeam sonar images, an image resampling method based on scan filling is presented here.

[0318] The basic principle of the scan-fill method is that for any closed region, each row of pixels in the region is scanned sequentially from top to bottom using horizontal scan lines. A series of intersection points generated by each scan line and the boundary are calculated. These intersection points are sorted according to the horizontal axis. The sorted intersection points are then taken out in pairs as the left and right boundary points. All pixels within the left and right boundary points are marked as fill points. When the entire region has been scanned, the region filling is completed.

[0319] like Figure 8As shown, A1 and A2 are two adjacent echoes on the same scan line, and B3 and B4 are two adjacent points on adjacent scan lines. A1, A2, B3, and B4 form a closed connected region. Using the scan-fill method, all pixels inside the region can be calibrated. The pixel value of each pixel can be obtained by the inverse distance weighting method, as shown in equation (56).

[0320]

[0321] (5) Development and detection of deep-sea multifunctional imaging spectrometer

[0322] 1) Development of a multi-functional imaging spectrometer for deep-sea applications

[0323] Deep-sea multi-functional imaging spectrometer technology approach, such as Figure 9 As shown, the development process involves five steps: parameter simulation analysis, overall scheme design, unit scheme design, prototype development, and experimental verification.

[0324] The main technical research of deep-sea multifunctional combined imaging spectrometers can be divided into the following four aspects:

[0325] ① Research on a three-wavelength Raman and fluorescence combined detection method. Since the spectral signals from in-situ detection using deep-sea Raman or fluorescence spectroscopy are very weak and affected by seawater background and system noise, the extracted target spectral signals have certain errors. Accuracy can be improved by combining and verifying spectral data from multiple wavelengths. Integrating multiple wavelength lasers increases system complexity. The combined system needs to address structural and functional challenges such as the structure of the Raman and fluorescence spectroscopy shared optical path measurement system, the layout and optimization of the system's optomechanical structure, and the mutual interference effects of multiple laser beams.

[0326] To achieve high-precision detection of Raman and fluorescence spectra, the instrument uses spectral data excited at three wavelengths: 266 nm, 532 nm, and 785 nm, which are then used in conjunction for calculation and verification. The 266 nm wavelength is shorter and focuses on fluorescence detection; the 785 nm wavelength is longer and excites weaker fluorescence, focusing on Raman detection; while the 532 nm wavelength balances both Raman and fluorescence detection and, being located within the seawater transmission window, has strong penetration capabilities. The combined system requires addressing structural and functional challenges, including the structure of the Raman and fluorescence spectroscopy common-path measurement system, the layout and optimization of the system's optomechanical structure, and the mutual interference effects of multiple laser beams.

[0327] ② Binocular vision imaging method. To achieve high-precision in-situ detection of marine materials by a multi-functional imaging spectrometer, a binocular vision unit was designed to achieve high-precision target positioning. The binocular vision unit is rigidly connected to the detection part of the multi-functional imaging spectrometer, and is used to measure the high-precision quantitative relationship between the underwater three-dimensional spatial relative position of the multi-functional imaging spectrometer probe and the detected seabed material in real time in the deep sea, ensuring the realization of high-precision in-situ detection. This involved solving technical issues related to illumination, the position of the two CCD cameras, optical parameters, and installation.

[0328] ③ High-precision in-situ measurement and positioning method. Precise optomechanical matching and calibration of the binocular vision unit and the spectral measurement unit are required to achieve accurate in-situ positioning. This necessitates addressing the field-of-view matching of the two optical units, structural design and workflow, as well as calibration and testing methods.

[0329] ④ Probe separation technology. Generally, to ensure the stability of the optical system and reduce attenuation, the optical system is placed on the same working platform as much as possible. However, due to the weight limitations of the robotic arm, the instrument needs to be divided into a control cabin and a probe section. Therefore, deep-sea combined systems need to solve the problems of parameter matching and beam quality parameter matching when the optical path system is not on the same reference platform, the correction of the influence of the optical path on the light amplitude and spectral distribution, the resolution of background interference and mutual interference between multiple wavelengths, and the challenges of system optical path alignment and optomechanical system stability.

[0330] 2) Deep-sea material Raman and fluorescence spectroscopy multiplexing detection and identification technology

[0331] The system uses a single spectrometer to collect the spectrum of the target. Therefore, the spectral data collected by the spectrometer includes both Raman and fluorescence spectra. Based on the different characteristics of the two spectra, an identification algorithm is used to identify the Raman and fluorescence spectra. Then, the substances are identified using Raman inversion algorithm and fluorescence inversion algorithm, respectively.

[0332] (1) Based on the characteristics of Raman spectroscopy and fluorescence spectroscopy, the two spectra can be identified by a trend discrimination algorithm.

[0333] (2) The collected Raman spectra are subjected to baseline correction, characteristic peak search, and characteristic information extraction in sequence, and finally the spectra after feature extraction are analyzed.

[0334] (3) The collected fluorescence spectra are sequentially subjected to convex point extraction, convex point clustering, convex point selection and region selection to obtain the required data.

[0335] 3) Deep-sea high-precision imaging positioning and spectral measurement error calibration technology

[0336] ① Pool Simulation Experiment. To test the system's functionality and formulate the workflow for marine experiments, a pool simulation experiment was first conducted. The simulated target in the pool was tested using binocular visual positioning and in-situ spectral detection to obtain the system's performance parameters and the underwater operation workflow. Based on the obtained information, system improvement work was carried out, providing basic information and preparation for the next stage of deep-sea operations.

[0337] ② Detection parameters of instrument window deformation in deep-sea environment. The instrument probe with built-in camera is placed in a pressure tank, which simulates the pressure environment of the deep sea. The camera takes pictures of a standard plate through the window glass inside the tank. The geometric deformation position and amount of the window glass are determined by the captured images of the standard plate, providing window deformation information for deep-sea operations and quantitative data for the calibration and correction of the final acquired spectral information.

[0338] ③ Optical path simulation and algorithm correction. Based on the position and geometric deformation of the detection window in the deep-sea environment, the paths of light rays of different wavelengths after passing through the detection window are simulated to determine the impact of changes in the light path on the instrument's imaging positioning and spectral measurement. Combined with the instrument's optical path design parameters, corresponding correction algorithms are added to the instrument's binocular vision positioning and spectral data processing algorithms. The corrected instrument is then placed in a pressure vessel for a simulation experiment to verify the effectiveness and accuracy of the instrument correction method.

[0339] (6) VR environment construction technology for deep-sea target survey areas

[0340] To address the need for rapid seafloor topography modeling using bathymetry data from target survey areas, a dynamic partitioning and merging mechanism based on incomplete binary trees is designed. This mechanism first divides the target survey area data points into blocks according to the survey area's boundaries, constructs sub-triangulations for each sub-block, and then merges these sub-triangulations in reverse order of block division to quickly establish a TIN seafloor topography model for the entire survey area. To address the issue of ill-conditioned triangles (such as intersecting triangles and linear triangles) that may arise during sub-network merging, leading to reduced algorithm robustness, a bidirectional stitching algorithm with joint vector product testing is proposed. This algorithm completely avoids ill-conditioned triangles and significantly reduces the number of subsequent local optimizations by ensuring that the newly generated initial triangles approach their optimal shapes during sub-network stitching. This improves the speed and robustness of sub-network merging, ultimately achieving rapid and robust seafloor digital topography modeling and visualization.

[0341] The amount of water depth data points obtained by fusing multibeam and lidar systems is enormous, while the time complexity of the Dealunay triangulation algorithm is generally O(N log N), and in the worst case, it reaches O(N log N). 2Where N is the number of data points, it is clearly not advisable to directly construct a Delaunay triangulation network to build a TIN digital terrain model from a large volume of bathymetry data. To improve the speed of seabed digital terrain modeling based on irregular triangulation networks from large volume of bathymetry data, a dynamic segmentation and merging mechanism for survey area data points based on binary trees is adopted. The survey area data points are segmented according to the survey area, and a sub-triangulation network is constructed for each sub-data block. Then, the sub-triangulation networks are merged in reverse order of segmentation to finally obtain the TIN seabed terrain model of the entire survey area, achieving realistic 3D seabed terrain visualization. Figure 10 A flowchart for constructing and visualizing a virtual seabed environment dataset for a deep-sea survey area.

[0342] Therefore, the present invention solves the following technical problems:

[0343] (1) A calculation model for the coordinates of underwater depth sounding points of lidar in the WGS84 spatial rectangular coordinate system was constructed;

[0344] (2) A calculation model for the coordinates of multibeam sonar sounding points in the WGS84 spatial rectangular coordinate system was constructed;

[0345] (3) A method for fusing lidar data with multibeam sonar data at approximately the same point in the overlapping region is proposed.

[0346] (4) A method for detecting and identifying deep-sea materials by combining Raman and fluorescence spectroscopy and a development plan for a prototype of a multifunctional imaging spectrometer for deep-sea materials were proposed.

[0347] (5) Key technical methods for constructing VR environment in deep-sea target survey areas were proposed.

[0348] In addition, the technical features of the present invention are as follows:

[0349] (1) Calculation models of the coordinates of lidar and multibeam sonar sounding points in the WGS84 spatial rectangular coordinate system were constructed respectively.

[0350] (2) To address the problem of fusing approximately identical lidar and multibeam sonar data in overlapping regions, a corresponding fusion processing method is proposed;

[0351] (3) A method for identifying cold seeps by combining Raman and fluorescence spectroscopy of deep-sea materials was proposed;

[0352] (4) Key technical methods for constructing VR environment of topography and geomorphology of deep-sea target survey area were proposed.

Claims

1. A method for designing a deep-sea in-situ fine-scale exploration framework based on acoustic-optical remote sensing and VR technology, characterized in that, Includes the following steps: (1) Study the structure of the photon counting lidar scanning system, and study the dynamic geometric relationship between the mother ship's ultra-short baseline and the submersible beacon. Construct a calculation model of the lidar underwater depth sounding point coordinates in the WGS84 spatial rectangular coordinate system, and realize the splicing of adjacent strip lidar point clouds. (2) Taking into account the changes in the attitude of the submersible, based on the acoustic tracking model, the geometric relationship between the mother ship's ultra-short baseline and the submersible's beacon, a calculation model for the coordinates of the multibeam sonar sounding point in the WGS84 spatial rectangular coordinate system is constructed. (3) It is beneficial to normalize the laser radar and multibeam scattering intensity data by Z-score, and to fuse the laser radar sounding points and multibeam sonar sounding points under the near-identical rule to construct the fused seabed topography map. (4) A development scheme and detection method for a deep-sea functional imaging spectrometer were proposed; (5) Construct a VR environment for the deep-sea target survey area by using three-dimensional seabed topography modeling, rendering technology and binocular stereo imaging technology; The stitching of adjacent strip laser point clouds in step (1) includes the following steps: 1) Structural analysis of the lidar scanning system; 2) Establishment of the laser scanning reference coordinate system and its transition coordinate system; 3) Establishing the relationship between the reflected ray and the normal vector of the mirror in the laser scanning reference coordinate system; 4) Calculation of underwater depth sounding point coordinates in the laser scanning reference coordinate system; 5) Calculation of coordinates of laser underwater sounding points in the submersible beacon array coordinate system; 6) Establish the geometric relationship between the submersible beacon array and the mother ship's ultra-short baseline center; 7) Calculation of the coordinates of the submersible beacon array in the WGS84 spatial rectangular coordinate system; 8) The laser underwater sounding points are repositioned in the WGS84 spatial rectangular coordinate system; 9) Adjacent strip laser point cloud stitching based on the linear Plücker coordinate description rule; The development scheme and detection technology of the deep-sea functional imaging spectrometer described in step (4) include the following steps: 1) The development plan for the imaging spectrometer was determined from five aspects: parameter simulation analysis, overall scheme design, unit scheme design, prototype development, and experimental verification. 2) Conduct research in four areas: detection technology using three-wavelength Raman and fluorescence combined, binocular vision imaging technology, high-precision in-situ measurement and positioning technology, and probe separation technology; 3) Conduct research on the multi-stage detection and identification technology of Raman and fluorescence spectroscopy for deep-sea materials; 4) Conduct research on error calibration technology for high-precision imaging positioning and spectral measurement in the deep sea; The construction of the VR environment for the deep-sea target survey area described in step (5) includes the following steps: 1) Based on the dynamic segmentation and merging mechanism of an incomplete binary tree, a TIN seabed topography model of the entire survey area is established; 2) Multi-color gradient rendering technology for seabed terrain based on HSL color model, which renders the terrain at different depths of the seabed; 3) A binocular stereo imaging method is proposed; 4) Construction of a virtual environment that integrates a three-dimensional seabed terrain model and a binocular stereo imaging model of the target.

2. The deep-sea in-situ fine exploration framework design method based on acoustic-optical remote sensing and VR technology as described in claim 1, characterized in that, The calculation model for constructing the coordinates of multibeam sonar sounding points in the WGS84 spatial rectangular coordinate system, as described in step (2), includes the following steps: 1) Taking into account the changes in the submersible's attitude, calculate the initial incident angle of the multibeam sonar beam; 2) Establish a ray tracking model; 3) Calculate the coordinates of the beam footprint in the transducer coordinate system; 4) The beam footprint coordinates are repositioned into the submersible beacon array coordinate system; 5) Establish the geometric relationship between the submersible beacon array and the mother ship's ultra-short baseline; 6) Reorient the beam footprint coordinates to the WGS84 spatial rectangular coordinate system.

3. The deep-sea in-situ fine exploration framework design method based on acoustic-optical remote sensing and VR technology as described in claim 1, characterized in that, The laser radar sounding points and multibeam sonar sounding points mentioned in step (3) are fused to construct a fused seabed topographic map, including the following steps: 1) It is beneficial for Z-score to normalize lidar and multi-beam scattering intensity data; 2) Conversion between Z-score and grayscale value; 3) Merging of point clouds from lidar and multibeam sonar at near-identical points; 4) Geographic coding; 5) Image resampling.

Images