Underwater robot based on optical imaging and multicarrier modulation communication technology

Abstract

The invention relates to the technical field of underwater robots, in particular to an underwater robot based on optical imaging and multicarrier modulation communication technology, which comprises an optical imaging module, a multicarrier modulation communication module, a communication module and a control module, the multi-carrier signal modulation module is used for modulating optical image data into multi-carrier signals and realizing remote transmission, and the data processing module is used for carrying out preprocessing, coding and decoding coordination and physical and underwater acoustic engineering interdisciplinary cross fusion on the image data and combining innovative ideas and technologies in different fields; the underwater optical vision imaging technology and the acoustic multi-carrier modulation technology are applied to the automatic underwater robot, the image capturing accuracy and the wireless real-time communication stability of the automatic underwater robot are improved, a camera basic imaging model is optimized through significance analysis of underwater imaging, meanwhile, the laser scanning technology is tried to be applied, and the image capturing accuracy is improved. And the fixed-point image capturing capability with higher precision is obtained.

Description

Technical Field

[0001] The invention relates to an underwater robot based on optical imaging and multi-carrier modulation communication technology, belonging to the technical field of underwater robots. Background Art

[0002] Underwater robots (UAVs), with their ability to assist or replace human operators in underwater monitoring and operations in complex environments, play a vital role in marine engineering. Currently, there is an increasing demand for autonomous UAVs equipped with detection equipment to conduct high-precision seabed detection and identify small targets. Underwater environmental perception is essential for the safe navigation of UAVs and serves as a guide for their operations. However, due to the unknowns and complexity of the underwater environment, autonomous UAV operations face numerous challenges. Limited by the state of technology in environmental perception and path planning, the current intelligence level of UAVs is relatively low, making it difficult to meet the diverse demands of practical application scenarios. Therefore, improving the intelligence and operational accuracy of UAVs has become a major research topic that needs to be tackled urgently.

[0003] In underwater operations, the detection and positioning of targets are fundamental and directly impact the quality of the work. Currently, underwater perception sensors primarily consist of sonar and visual sensors. Sonar technology offers significant advantages in detecting large-scale environments, but in smaller-scale scenarios, it is susceptible to environmental interference, resulting in blurred images and reduced detection accuracy. Visual sensors, on the other hand, are suitable for smaller environments and offer high resolution and close-range accuracy. However, for longer-range detection, they face challenges such as a limited field of view and significantly reduced imaging accuracy.

[0004] For operational underwater robots, the identification, detection, tracking, and positioning of underwater targets are core research areas in vision systems. Underwater optical imaging is a key means for humans to obtain ocean information. Improving the accuracy and stability of underwater robots' long-range visual imaging is a pressing technical challenge. Furthermore, reducing optical image loss during information transmission and ensuring efficient and stable transmission to the control console is also a key technical challenge.

[0005] Based on the above status quo, this study aims to deeply integrate underwater acoustic multi-carrier modulation technology with underwater optical imaging technology. By optimizing the relevant performance of existing underwater robots, it is possible to effectively improve the ability of underwater robots to obtain environmental image information at a long distance using visual imaging technology, as well as the accuracy and stability of processing and transmitting information using acoustic technology, thereby providing technical support for the widespread application of underwater robots in the fields of marine resource development and environmental exploration. Summary of the Invention

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions: an underwater robot based on optical imaging and multi-carrier modulation communication technology, comprising:

[0007] Optical imaging module, used to collect optical images of underwater environments and perform target detection;

[0008] Multi-carrier modulation communication module, used to modulate optical image data into multi-carrier signals and realize long-distance transmission;

[0009] The data processing module is used to coordinate the preprocessing, encoding and decoding of image data.

[0010] Furthermore, the optical imaging module is based on the Jaffe-McGlamery imaging model and combines the Lambert-Beer law to establish a light attenuation compensation algorithm, and the formula E(d)=E0e -β(λ)d Compensate for the absorption and scattering of light by water, where β(λ)=α(λ)+γ(λ) is the total attenuation coefficient, including the scattering coefficient α(λ) and the absorption coefficient γ(λ).

[0011] Furthermore, the optical imaging module includes:

[0012] underwater cameras for capturing underwater images;

[0013] Laser light source for enhanced local lighting;

[0014] The visual saliency detection unit uses the Itti model to extract color, brightness, and direction features in the image, generate a comprehensive saliency map to segment the target area, and improve the accuracy of target detection.

[0015] Furthermore, the visual saliency detection unit optimizes the saliency map by adjusting the weight parameters of color, brightness, and direction features and combining depth information to achieve positioning of underwater targets within the range of 0-1200mm with a positioning error of ≤5%.

[0016] Furthermore, the multi-carrier modulation communication module adopts multiple-input multiple-output orthogonal frequency division multiplexing technology, and the signal processing process includes:

[0017] After serial-to-parallel conversion and channel coding, the input image data is converted into a time domain signal through IFFT and a cyclic prefix is ​​added;

[0018] The transmitting transducer transmits the signal to the underwater acoustic channel, and the receiving end recovers the original data through A / D conversion, FFT demodulation and particle swarm optimization algorithm decoding.

[0019] Furthermore, the multi-carrier modulation communication module adopts orthogonal amplitude modulation, reduces the signal bit error rate through Gray mapping coding, and optimizes the carrier spacing in combination with the multipath effect model to achieve a bit error rate ≤ 10-4 Remote data transmission.

[0020] Furthermore, the data processing module constructs an underwater environment optical property database to store the attenuation coefficients and scattering properties of light of different wavelengths, which is used to perform color correction and contrast enhancement on the collected images to reduce blue-green tone distortion.

[0021] Furthermore, by integrating the close-range, high-precision detection capability of optical imaging with the long-range stability of multi-carrier modulation communication, it is suitable for marine resource exploration, underwater target monitoring, and autonomous operation scenarios in complex environments.

[0022] Technical effects and advantages of the present invention:

[0023] 1. The interdisciplinary integration of physics and underwater acoustic engineering combines innovative ideas and technologies from different fields, and applies underwater optical visual imaging technology and acoustic multi-carrier modulation technology to automated underwater robots, achieving improved image capture accuracy and wireless real-time communication stability for automated underwater robots.

[0024] 2. Optimize the basic camera imaging model through saliency analysis of underwater imaging, and try to use laser scanning technology to obtain more accurate fixed-point image capture capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is the relationship between the underwater environment attenuation coefficient and the wavelength of light in the present invention.

[0026] Figure 2 This is the Jaffe-McGlamery underwater optical imaging model of the present invention.

[0027] Figure 3 This is the visual saliency detection of the present invention.

[0028] Figure 4 This is a significant result of the present invention.

[0029] Figure 5 This is the basic communication system based on OFDM in the present invention.

[0030] Figure 6 This is the MQAM constellation diagram of the present invention.

[0031] Figure 7 Schematic diagram of the QAM modulation principle of the present invention.

[0032] Figure 8 Schematic diagram of the QAM demodulation principle of the present invention.

[0033] Figure 9 This is a schematic diagram of the arrangement of the water pool experimental equipment of the present invention.

[0034] Figure 10 This is the water pool experimental system structure of the present invention.

[0035] Figure 11 This is a simplified simulation diagram of the underwater structure close-range measurement experiment of the present invention.

[0036] Figure 12 This is a close-up view of the underwater structure of the present invention. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] Underwater optical imaging independent system, underwater optical characteristics analysis, according to the Lambert-Beer law, when light propagates in water, its intensity gradually decays with the increase of propagation distance, and decays at an exponential rate, the relationship is: E(d) = E0e -β(λ)d Where E0 is the light intensity at the reference position, E(d) is the light intensity at a distance d from the reference position, β(λ) is the total attenuation coefficient of light, including absorption and scattering, which is related to the wavelength and medium of the light, λ is the wavelength of the light, and its relationship with the frequency v and speed of light c is c=λ*v.

[0039] For underwater environments, the total attenuation coefficient β(λ) consists of two parts: the scattering coefficient α(λ) and the absorption coefficient γ(λ). The relationship is: β(λ) = α(λ) + γ(λ). In the visible light range, the relationship between the wavelength of light and each coefficient is as follows: Figure 1 shown.

[0040] The underwater environment scatters light in two ways: scattering caused by the water and scattering caused by the medium in the water. Scattering can change the propagation direction of light, resulting in reduced clarity and blurring of underwater images. According to Rayleigh's law, the relationship between the scattering coefficient α (λ) of light and wavelength is as follows: The coefficient of λ related to wavelength is determined according to the relationship between the wavelength of light and the size of the medium.

[0041] Analysis of underwater optical imaging models,Jaffe-McGlamery model is a classic underwater imaging model, such as Figure 2As shown in the figure, in this underwater imaging model, the underwater imaging process includes the complete transmission of light from the water surface to the camera's imaging plane. The light intensity on the water surface is Eair, the reflected light intensity from the underwater object is Eobject, and the light intensity at the underwater camera's imaging plane is Etotal. The light Eair on the water surface reaches the object after traveling a distance D. During this process, the underwater environment absorbs and scatters the light. After traveling a distance d(x), the object's reflected light Eobject propagates to the camera's imaging plane. This process is also affected by absorption and scattering. At the same time, the object's reflected light Eobject also propagates to the camera's imaging plane at a small angle. Furthermore, some of the light in the water changes direction after being scattered by the water and the medium, ultimately reaching the camera's imaging plane.

[0042] Therefore, the imaging of the camera imaging plane can be regarded as a combination of three components, as shown in Eq. total =E d +E fsl +E bs As shown in the figure. The first part is the direct component Ed. When underwater light propagates to an object, it is reflected on the surface of the object. The reflected light then continues to propagate and eventually reaches the imaging plane of the camera. The second part is the forward scattered component Efs. This is because the reflected light from the object is scattered at a small angle and eventually forms an image on the camera plane. The third part is the backscattered component Ebs. When light propagates to non-target objects in the water, it is also scattered. Some of the scattered light propagates and eventually reaches the imaging plane of the camera.

[0043] In the above underwater imaging model, it is assumed that the light source is a point light source, and the light propagates outward in a spherical shape. When the light propagates a distance of Rs and reaches the object (x′, y′), the light is reflected. The intensity of the light reflected from the object Eobject can be calculated by the incident light intensity and the reflection coefficient, which is the product of the two. The light reflected from the object continues to propagate, and the direct component of (x, y) at a distance of Rc from the object is Ed(x, y), which is calculated according to the theory of geometric optics as follows:

[0044]

[0045] E d (x,y): direct component at (x, y);

[0046] E object (x′, y′): the intensity of reflected light from the object (x', y');

[0047] M(x′,y′): reflectivity of the object (x', y');

[0048] T I : camera transmittance;

[0049] TI : focal length of the camera;

[0050] θ: The angle between the reflected light and the reflecting plane.

[0051] Light reflected from objects also undergoes small-angle scattering, known as forward scattering, which can cause image blur. Small-angle scattering can be calculated using a point spread function (PSF). This method demonstrates that forward scattering differs from backscattering in that it arises from multiple angles and cannot be solved using small-angle scattering methods. The solution is to segment the space into volume elements, calculate each component using a volume spread function, and finally superimpose the results to obtain the backscattering result. This allows the prediction of a simplified relationship between camera imaging and scene illumination intensity in the Jaffe-McGlamery underwater imaging model. (Forward scattering has a smaller impact than backscattering, so it can be ignored in the mathematical model.) This data is used to assess the accuracy and stability of the established underwater optical imaging model.

[0052] Analysis of visual saliency methods: The visual saliency mechanism simulates the human visual attention mechanism, ignoring irrelevant areas in the image and focusing on the target of interest. Visual saliency detection is to highlight the prominent target in the visual scene, and then obtain a binary mask image of the target after segmentation, such as Figure 3 .

[0053] Since underwater scenes are usually simple, with the background mostly consisting of water or seabed sand, the target in underwater target detection is generally quite different from the background. Therefore, the visual saliency method can be used to detect targets in simple underwater scene images. The classic Itti model can be used here. This model is a bottom-up approach, and the features extracted are color, brightness, and direction. Applying the Itti model to underwater scene images produces the following results: Figure 4 As shown, a is the original image, b is the color part, c is the brightness part, d is the direction part, and e is the comprehensive saliency map.

[0054] The characteristics of underwater vision are analyzed based on the characteristics of underwater organisms. Their vision can be analyzed from the following perspectives: sensitivity to light brightness, sensitivity to color, sensitivity to depth, as well as their mathematical relationships and sensitivity to contrast.

[0055] The classic Itti model extracts brightness, color, and direction feature maps. The factors it considers are consistent with the factors affecting underwater visual saliency. Based on the Itti model, we consider combining the characteristics of the underwater visual system to perform preliminary detection to obtain a saliency map, and then optimize the saliency map by combining it with depth information.

[0056] Initial underwater target detection and segmentation: To obtain a more reasonable underwater image saliency map, we consider minimizing the influence of individual visual features on the overall saliency map. Based on the Itti saliency model, adjustments are made to the model by relatively reducing or increasing the weights of different features for fitting. This is used to refine and optimize the saliency map, improve its quality, and generate a comprehensive saliency map. Underwater targets are detected in the saliency map optimized with depth information. Segmentation is then used to extract underwater targets from the image. Salient target detection results are analyzed and evaluated, and a reliable dataset is selected for comparative testing using different saliency algorithms.

[0057] The basic camera imaging model was analyzed. Using a binocular camera for positioning is an effective solution for small-scale scene positioning. The underwater refractive camera model was then calibrated. The camera's intrinsic and extrinsic parameters were calibrated using the Zhang Zhengyou calibration method. For the outer shell parameters, an error model was established between the corresponding points on the imaging plane solved according to the model and the actual imaging plane points, and a particle swarm optimization algorithm was used to solve it. Points in the underwater scene were then located based on analysis of underwater binocular vision.

[0058] Multi-carrier modulation technology and optimized orthogonal frequency division multiplexing (OFDM) for the underwater robot's acoustics enable the underwater robot to stably and remotely transmit image data to a receiving monitor after completing its underwater image capture mission. To achieve this goal, the underwater robot for this project was optimized, and a research route and experimental plan were designed. This section provides an overview of multi-carrier technology and the principles and approaches for signal processing.

[0059] An OFDM symbol includes a composite signal of multiple modulated subcarriers, where each subcarrier can be modulated by a phase shift keying (PSK) or quadrature amplitude modulation (QAM) symbol.

[0060] The signal processing flow is as follows: the input bit sequence is converted from serial to parallel, followed by channel coding, and then the corresponding modulation mapping is completed according to the modulation method used to form a modulation information sequence, which is subjected to IFFT to calculate the time domain sampling sequence of the OFDM modulated signal. Add a cyclic prefix CP (the cyclic prefix can enable the OFDM system to completely eliminate the inter-symbol interference and inter-carrier interference caused by the multipath propagation of the signal, and then perform D / A conversion to obtain the time domain waveform of the OFDM modulated signal. The receiving end first performs A / D conversion on the received signal, removes the cyclic prefix (CP), and obtains the sampling sequence of the OFDM modulated signal. The sampling sequence is subjected to FFT to obtain the original modulation information sequence, which is then demodulated and demapped, and then channel decoded to obtain the output bit sequence. The basic composition of the OFDM-based communication system is as follows: Figure 5 .

[0061] After capturing underwater images, the underwater robot needs to transmit the data remotely. We will utilize acoustics to handle this task. The following are the key technologies we are considering. Using amplitude or phase alone to carry information does not fully utilize the signal platform. We have concluded that using quadrature amplitude modulation signals for multi-carrier modulation is ideal and will achieve the desired results.

[0062] In multi-level amplitude modulation, the vector endpoints are distributed along a single axis; in multi-level phase modulation, the vector endpoints are distributed along a circle. As the number of bases increases, the minimum distance between vector endpoints in the signal space decreases, and the area of ​​the decision space also decreases, which increases the difficulty of demodulating the received signal. If the vector endpoints are re-distributed reasonably to fully utilize the entire plane, it is possible to increase the number of signal vector endpoints without reducing the minimum distance, such as Figure 6 As shown in Figure 2, quadrature amplitude modulation is a modulation method that is generated based on this requirement and has a higher frequency bandwidth utilization rate.

[0063] The general expression of quadrature amplitude modulation is:

[0064] y(i)=A m cosω c t+B m sinω c t 0≤t <T s

[0065] The above formula consists of two mutually orthogonal carriers, each of which is modulated by a set of discrete amplitudes, so this modulation method is called orthogonal amplitude modulation.

[0066] Where T s is the symbol width, the amplitude A in QAM m and B m It can be expressed as:

[0067]

[0068] Where A is a fixed amplitude and (dm,em) is determined by the input data. (dm,em) determines the coordinates of the modulated QAM signal in signal space.

[0069] The modulation and coherent demodulation block diagram of QAM is as follows Figure 7 、 8 shown.

[0070] A water tank experiment was designed to investigate the effectiveness of various modulation and channel coding techniques in underwater acoustic communication under the aforementioned OFDM framework. Image data captured by an underwater robot was used as the digital signal source. Gray-map coding modulation was then applied and stored as a waveform file. This signal was transmitted via a sound card, amplified by a power amplifier, and then transmitted by a transmitting transducer. The receiving hydrophone then collected the signal through the sound card and stored it as a waveform file. This file was then processed, including demodulation and decoding, and finally synthesized into the original image format. The received original image and the processed synthesized image were compared and analyzed in terms of both quantitative bit error rate and qualitative image quality to determine the effectiveness of the communication. The experiment was conducted using a computer and a sound card for transmission and reception.

[0071] See the schematic diagram of the placement of the pool and experimental equipment and the experimental system structure. Figure 9 、 Figure 10 .

[0072] For the assembly and operation of underwater robots, we developed an intelligent underwater robot detection system. We used advanced underwater robots as a platform, equipped with optical cameras, sound cards, power amplifiers, radiation transducers and other sensors to conduct close observation and measurement of underwater structures. The experimental simulation diagram is as follows: Figure 11 and 12 .

[0073] This application primarily cross-disciplinaryly integrates physics and underwater acoustics, combining innovative ideas and technologies from different fields to achieve a multi-faceted effect, creating new possibilities for traditional automated underwater robots. By applying underwater optical visual imaging and acoustic multi-carrier modulation techniques to automated underwater robots (AUVs), the application improves image capture accuracy and the stability of wireless real-time communications within AUVs.

[0074] Secondly, it aims to optimize and enhance underwater imaging technology suitable for automated underwater robots. By analyzing the saliency of underwater imaging to optimize the basic camera imaging model, and by exploring the use of laser scanning technology, it aims to achieve even more precise, fixed-point image capture capabilities. Underwater imaging is a key research area in underwater optics and marine optics, and a crucial tool for understanding, developing, utilizing, and protecting the ocean. It offers advantages such as intuitive target detection, high imaging resolution, and high information content.

[0075] Furthermore, this application aims to improve the efficiency of process communication in underwater robots by using a novel underwater acoustic communication model based on the more robust multi-carrier modulation technology of orthogonal frequency division multiplexing (OFDM). This model effectively converts optical and acoustic signals, achieves robust underwater acoustic communication for automated underwater robots, and reduces optical information transmission losses.

[0076] Ultimately, the AUV, the outcome of this project, will achieve improved image capture accuracy and wireless real-time communication stability for automated underwater robots.

[0077] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0078] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. An underwater robot based on optical imaging and multi-carrier modulation communication technology, characterized in that: include: Optical imaging module, used to collect optical images of underwater environments and perform target detection; Multi-carrier modulation communication module, used to modulate optical image data into multi-carrier signals and realize long-distance transmission; The data processing module is used to coordinate the preprocessing, encoding and decoding of image data.

2. The underwater robot based on optical imaging and multi-carrier modulation communication technology according to claim 1, characterized in that: The optical imaging module is based on the Jaffe-McGlamery imaging model and combines the Lambert-Beer law to establish a light attenuation compensation algorithm. The formula E(d)=E0e −β(λ)d Compensate for the absorption and scattering of light by water, where β(λ)=α(λ)+γ(λ) is the total attenuation coefficient, including the scattering coefficient α(λ) and the absorption coefficient γ(λ).

3. The underwater robot based on optical imaging and multi-carrier modulation communication technology according to claim 1, characterized in that: The optical imaging module includes: underwater cameras for capturing underwater images; Laser light source for enhanced local lighting; The visual saliency detection unit uses the Itti model to extract color, brightness, and direction features in the image, generate a comprehensive saliency map to segment the target area, and improve the accuracy of target detection.

4. The underwater robot based on optical imaging and multi-carrier modulation communication technology according to claim 3, characterized in that: The visual saliency detection unit adjusts the weight parameters of color, brightness, and direction features and optimizes the saliency map in combination with depth information to achieve positioning of underwater targets within the range of 0-1200mm, with a positioning error of ≤5%.

5. The underwater robot based on optical imaging and multi-carrier modulation communication technology according to claim 1, characterized in that: The multi-carrier modulation communication module adopts multiple-input multiple-output orthogonal frequency division multiplexing technology, and the signal processing process includes: After serial-to-parallel conversion and channel coding, the input image data is converted into a time domain signal through IFFT and a cyclic prefix is ​​added; The transmitting transducer transmits the signal to the underwater acoustic channel, and the receiving end recovers the original data through A / D conversion, FFT demodulation and particle swarm optimization algorithm decoding.

6. The underwater robot based on optical imaging and multi-carrier modulation communication technology according to claim 5, characterized in that: The multi-carrier modulation communication module adopts orthogonal amplitude modulation, reduces the signal bit error rate through Gray mapping coding, and optimizes the carrier spacing in combination with the multipath effect model to achieve a bit error rate of ≤10 -4 Remote data transmission.

7. The underwater robot based on optical imaging and multi-carrier modulation communication technology according to claim 1, characterized in that: The data processing module builds an underwater environment optical property database to store the attenuation coefficients and scattering properties of light of different wavelengths, which is used to perform color correction and contrast enhancement on the collected images and reduce blue-green tone distortion.

8. An underwater robot based on optical imaging and multi-carrier modulation communication technology according to any one of claims 1 to 7, characterized in that: By integrating the close-range, high-precision detection capabilities of optical imaging with the long-range stability of multi-carrier modulation communications, it is suitable for marine resource exploration, underwater target monitoring, and autonomous operation scenarios in complex environments.

Images