A joint simulation method of imaging sonar and binocular camera based on multi-beamforming

Abstract

The present invention discloses a joint simulation method of imaging sonar and binocular camera based on multi-beam forming, which relates to the field of images. An underwater video target tracking system based on imaging sonar and binocular camera is designed and constructed; a binocular camera model is established in Gazebo and a world model file with a moving target is imported. The target motion is observed by publishing topics using Rviz, a three-dimensional visualization platform that displays multiple data under the ROS system; a beam-level time series sonar simulation method is used to generate real coherent image speckle and a correct point spread function under the Gazebo framework, and to provide real-time sensor-level beam intensity data and scene rendering; and the measurement values ​​of underwater video targets in the coordinate systems of underwater binocular cameras, imaging sonars, etc. are calculated. The present invention can generate qualitatively realistic images and provide the data required by underwater autonomous systems. It has broad application prospects in deep-sea operations, space robots and other fields.

Description

Technical Field

[0001] The present invention relates to the field of images, and in particular to a joint simulation method of an imaging sonar and a binocular camera based on multi-beam forming. Background Art

[0002] Currently, my country has significant development needs in the field of underwater operation and control equipment. Among these, target motion state estimation for underwater unmanned operations is one of the key technical bottlenecks. Imaging sonar, a key instrument for ocean exploration, is an electronic device that utilizes the propagation characteristics of sound waves underwater to complete underwater exploration and communication tasks through electroacoustic conversion and information processing. Imaging sonar has the characteristics of strong wavelength detection, target identification capabilities, and high concealment. However, it also has many shortcomings, such as being susceptible to multipath effects, reverberation interference, ocean noise, self-noise, target reflection characteristics, or radiated noise intensity, resulting in positioning errors. In recent years, with the popularization of artificial intelligence in various fields, underwater operation and control equipment has achieved further development.

[0003] Therefore, researchers in this field have dedicated themselves to developing a method for co-simulating imaging sonar and binocular cameras based on multi-beamforming. Taking into account the environmental characteristics of deep-sea environments and the imaging characteristics of imaging sonar, they designed a system and method for co-simulating imaging sonar and binocular cameras based on multi-beamforming. This system boasts stable performance, high accuracy, and a close resemblance to real-world scenarios, advancing the development of unmanned underwater simulation and control technology. Summary of the Invention

[0004] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is to overcome the characteristics of actual underwater experiments that are greatly affected by environmental factors, consume a lot of manpower and material resources, and are inconvenient to implement the experiments. An underwater video target simulation tracking system is proposed that fully utilizes the real-time characteristics of multi-beam forming imaging sonar and the optical characteristics of binocular cameras.

[0005] To achieve the above objectives, the present invention provides a method for joint simulation of imaging sonar and binocular camera based on multi-beamforming, comprising the following steps:

[0006] Step 1: Design an underwater video target system based on the Gazebo platform in the ROS system using a binocular camera and imaging sonar, and configure the environment.

[0007] Step 2: Create a world model with a moving target. Use the binocular camera component provided in Gazebo, import the world model in the launch file (a collection of nodes and parameters to be launched), and use Rviz (a ROS visualization tool) to observe the movement of the model in the left and right eyes of the camera by obtaining topics.

[0008] Step 3: Using a beam-level time-series sonar simulation method, we generate realistic coherent image speckle and a correct point spread function within the Gazebo framework, and provide real-time sensor-level beam intensity data and scene rendering.

[0009] Step 4: Based on the imaging characteristics of the multi-sensor, the measurement value of the underwater video target in the underwater multi-sensor simulation tracking system in the imaging sonar coordinate system is calculated.

[0010] Furthermore, the step 1 includes the following steps:

[0011] Step 1.1: Design an underwater video target tracking simulation system based on imaging sonar and binocular camera;

[0012] Step 1.2: Complete the configuration of the simulation environment in Gazebo.

[0013] Furthermore, the step 1.1 includes system mechanism design, sensor plug-in design, and target model design.

[0014] Furthermore, the step 2 includes the following steps:

[0015] Step 2.1. Create and compile the plug-in source file for controlling the target motion.

[0016] Step 2.2: Create a world model file, set the parameters of the motion target, and import the plugin to control the motion.

[0017] Step 2.3: Start the launch file for the binocular camera, apply it to the world model, and use Rviz (ROS visualization tool) to select the left and right target topics of the binocular camera to obtain the target image.

[0018] Furthermore, the plug-in source file in step 2.1 is a .cc source file.

[0019] Furthermore, the step 2.1 includes the following steps:

[0020] Step 2.1.1: Instantiate the motion target and set the motion model name and time parameters;

[0021] Step 2.1.2: Set the specific parameters of the target movement and use the time of reaching the pose to set the speed;

[0022] Step 2.1.3. Set up the trigger to update the program in response to events and register the plugin in the Gazebo simulator.

[0023] Furthermore, the step 3 includes the following steps:

[0024] Step 3.1. Use the camera plug-in in Gazebo to obtain the underwater scene, and obtain the two-dimensional field of view parameter set based on the ray-based beam model in the sonar rendering scene;

[0025] Step 3.2, calculate the point scattering model for each ray data;

[0026] Step 3.3, calculate the sum of rays for each beam;

[0027] Step 3.4: Calculate the beam pattern effect of the beam, and perform windowing and FFT (Fast Fourier Transform) to obtain the range intensity sonar data of each beam.

[0028] Furthermore, the step 4 includes the following steps:

[0029] Step 4.1, calculating the measurement value of the underwater video target in the coordinate system of the underwater multi-sensor detection and tracking system;

[0030] Step 4.2: Calculate the measurement value of the underwater video target in the imaging sonar coordinate system.

[0031] Furthermore, the binocular camera element is a cube.

[0032] Furthermore, the topics published by the left and right eyes of the binocular camera are "left" and "right".

[0033] In a preferred embodiment of the present invention, the present invention provides a system and method for joint simulation of imaging sonar and binocular camera based on multi-beamforming, comprising the following steps:

[0034] (1) Design a multi-beamforming-based imaging sonar and binocular camera simulation system and method to complete the configuration of the required environment;

[0035] (2) Build a world model with a moving target. Use the binocular camera component provided by Gazebo, import the world model into the launch file, and use Rviz (ROS visualization tool) to observe the movement of the model in the left and right eyes of the camera by obtaining topics.

[0036] The binocular camera component is designed as an orange cube. The topics published by the left and right eyes of the binocular camera are "left" and "right" respectively, and the moving target is set to move at a constant speed.

[0037] (3) Using a beam-level time-series sonar simulation method, we generate realistic coherent image speckle and correct point spread functions within the Gazebo framework, and provide real-time sensor-level beam intensity data and scene rendering;

[0038] (4) Based on the imaging characteristics of multiple sensors, the measurement value of the underwater video target in the underwater multi-sensor simulation tracking system in the imaging sonar coordinate system is calculated;

[0039] Step (1) further comprises the following steps:

[0040] First, we designed an underwater video target tracking simulation system based on multi-beamforming imaging sonar and binocular camera, including system structure design, sensor plug-in design, and target model design.

[0041] Then, complete the configuration of the required simulation environment in Gazebo;

[0042] Step (2) further comprises the following steps:

[0043] First, set up environment variables and compile the plug-in source file for controlling the target motion. The plug-in file for controlling the target motion is a .cc source file. The steps include:

[0044] a. Instantiate the motion target and set parameters such as motion model name and time;

[0045] b. Set the specific parameters of the target movement and use the time to reach a certain posture to set the speed;

[0046] c. Set up the trigger to update the program in response to events and register this plugin in the Gazebo simulator;

[0047] Secondly, create a world model file, set the geometric parameters of the motion target, and import the plug-in that controls the motion;

[0048] Then, start the launch file that provides the binocular camera, apply the established world model to it, and use Rviz (ROS visualization tool) to select the corresponding topics of the left and right eyes of the binocular camera to obtain the target image.

[0049] According to the multi-beamforming-based imaging sonar and binocular camera simulation system and method described in the embodiment of the present invention, step (3) further includes the following steps:

[0050] First, the camera plug-in in Gazebo is used to obtain the underwater scene. In the sonar rendering scene, a set of two-dimensional field of view parameters is obtained based on the ray-based beam model.

[0051] The steps for obtaining the field of view parameter set are as follows:

[0052] a. Using a ray-based beam model, the beam is defined as the vector z from the sensor frame origin to the first intersection with the visual object in the scene i , calculate the incident angle α i for

[0053]

[0054] in, and are the ray and normal unit vectors respectively;

[0055] b. Assume that dθ of each ray i and dφ i The angle changes are very small, and the beam scene projection surface is calculated

[0056] The product (the area projected by a single ray onto the visual object) dA is:

[0057]

[0058] where dθ i is the infinitesimal element of the azimuth angle of the ray in the sensor frame, dφ i is the infinitesimal value of the ray elevation angle;

[0059] c. Get the parameters in the sonar field of view, including the target distance r i , ray angle θ i and φ i , ray region projection area dA, ray incident angle α i , target normal direction unit vector n i and target reflectivity μ i ;

[0060] Secondly, the point scattering model of each ray data is calculated;

[0061] The synthetic time series is simulated using a point-based scattering model that uses discrete scatterers distributed over a surface defined by a discretized model mesh. Each surface mesh element intersected by a ray is defined as a scatterer; the number of rays is equal to the number of scatterers. This method is valid as long as the ray-intersection surface mesh is discretized with an average ray spacing that is smaller than the system's resolution area. If this criterion is not met, the number of rays in the beam should be increased before using this ray-based scattering model.

[0062] Then, the sum of the rays for each beam is calculated;

[0063] The steps for finding the sum are:

[0064] Assuming the sonar horizontal beam pattern is a uniform ideal beam pattern within the beamwidth, simulate each beam, generate a time series for the beam fan whose direction corresponds to the beamforming direction of a specific FLS (Forward Looking Sonar) or MBES (Multibeam Echo Sounder), and define the beam pattern of the array in polar coordinates, such as Figure 4 shown.

[0065] Mark the beamwidth θ on the main lobe bω =-3dB, defining the half beamwidth θ ω for:

[0066]

[0067] The sidelobe effect is simulated by performing a correction where each beam is a weighted sum of all the other beams. The weights are the directivity pattern of the array, with the main response axis pointing in a specific direction, e.g.

[0068]

[0069] Among them, ω i,j is the weight of the i-th beam steered in the j-th direction, θ i and θ j are the horizontal angles corresponding to the i-th and j-th beams respectively;

[0070] For a continuous linear array of length L and radiated energy λ, the beam pattern is a function of uniform aperture. The radiation pressure is modeled as a normalized sinc function. For high-frequency sonar, assuming L>>λ and using a small parameter approximation for the internal sinusoidal function, the final beam pattern B(θ) is obtained, which represents the response pressure ratio of the array at an angle θ relative to the main axis:

[0071]

[0072] Apply this to each beam and finally get the sum;

[0073] Finally, perform Windowing and FFT (Fast Fourier Transform) to obtain the range intensity sonar data of each beam;

[0074] According to the multi-beamforming-based imaging sonar and binocular camera simulation system and method described in the embodiment of the present invention, step (4) further includes the following steps:

[0075] a. Calculate the measurement value of the underwater video target in the coordinate system of the underwater multi-sensor detection and tracking system;

[0076] b. Calculate the measurement value of the underwater video target in the imaging sonar coordinate system.

[0077] Compared with the prior art, the present invention has the following obvious substantial features and significant advantages:

[0078] Actual sonar simulation requires access to the real underwater world, and existing simulation frameworks for underwater robotic manipulation lack a physics-based interaction model between sound and scenes. Multi-beamforming imaging sonar can address this shortcoming and more efficiently generate qualitatively realistic images. This invention utilizes imaging sonar and binocular camera components in Gazebo. For the binocular camera, a movable target is established to observe the target through the left and right eyes. For the imaging sonar, a beam-level time series simulation method is used to provide raw signals that can be used by the underwater autonomous system. The acoustic interaction between the target and the environment is calculated using a point-based scattering model, improving the authenticity of the final generated image and providing sonar images and physical time series signal data. This invention significantly improves the realism of the generated images in simulation experiments and plays a key role in the development, testing, and evaluation of underwater manipulation strategies that use sonar as a perception component.

[0079] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 1 is a schematic diagram of an imaging sonar and binocular camera simulation system and method based on multi-beamforming according to a preferred embodiment of the present invention;

[0081] Figure 2 is a scene graph of a binocular camera element and a moving target in Gazebo in a preferred embodiment of the present invention;

[0082] Figure 3 is an overall flow chart of an imaging sonar simulation process according to a preferred embodiment of the present invention;

[0083] Figure 4 FIG1 is a schematic diagram of a beam pattern of a half-power beam width of a preferred embodiment of the present invention;

[0084] Figure 5 This is a simulation result diagram of a binocular camera of a preferred embodiment of the present invention observing a moving target with its left and right eyes respectively;

[0085] Figure 6 This is a result diagram obtained by simulating a multi-beam echo sounder on a cylinder in a square sonar tank according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0086] The following describes several preferred embodiments of the present invention with reference to the accompanying drawings to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0087] In the drawings, components with identical structures are denoted by the same reference numerals, and components with similar structures or functions are denoted by similar reference numerals. The size and thickness of each component shown in the drawings are arbitrary and are not limited by the present invention. For clarity, the thickness of components in some places in the drawings is appropriately exaggerated.

[0088] The present invention relates to an underwater video target simulation platform in the imaging field, and in particular to an imaging sonar and binocular camera simulation system and method based on multi-beamforming for simulating imaging sonar and binocular cameras in underwater environments. The system has practical significance, such as the observation and imaging of cooperative targets in the deep sea, and the real-time status perception of non-cooperative targets.

[0089] A multi-beamforming-based imaging sonar and binocular camera joint simulation system and method, specifically comprising the following steps: (1) designing and constructing an underwater video target tracking system based on imaging sonar and binocular cameras, and completing the configuration of sensor plug-ins; (2) establishing a binocular camera model in Gazebo and importing a world model file with a moving target, and observing the target motion using Rviz (a visualization tool of ROS) by publishing topics; (3) using a beam-level time series sonar simulation method to generate realistic coherent image speckle and a correct point spread function in the Gazebo framework, and providing real-time sensor-level beam intensity data and scene rendering; (4) calculating the measurement values ​​of underwater video targets in the coordinate systems of underwater binocular cameras, imaging sonars, etc. based on multi-sensor imaging characteristics. The simulation system proposed in the present invention can run at a speed close to or faster than that of binocular cameras and sonar equipment in the real world. Although the simulation of the underwater environment cannot replace underwater testing, it is a key capability for developing manipulation strategies in complex and changing marine environments. Furthermore, by comparing real-world binocular camera images with sonar images of the same scene, we show that our method can generate qualitatively realistic images and provide the data required for underwater autonomous systems. This plays a key role in developing, testing, and evaluating underwater manipulation strategies that use sonar as a perception component, and has broad application prospects in deep-sea operations, space robotics, and other fields.

[0090] like Figure 1 As shown, a multi-beamforming-based imaging sonar and binocular camera joint simulation system and method includes the following steps:

[0091] (1) Build a world model with a moving target. Use the binocular camera component provided by Gazebo, import the world model in the launch file, and use Rviz (ROS visualization tool) to observe the movement of the model in the left and right eyes of the camera by obtaining topics.

[0092] like Figure 2As shown in the figure, the moving target used in the present invention is set as a 0.3*0.3*1.5 rectangular parallelepiped. The motion trajectory is set to move 0.5 distance toward the origin every five seconds starting from the 21st second of the simulation time until it reaches the origin of the binocular camera. The motion mode is set to loop. The orange square represents the binocular camera element, which is located at the origin of the ground coordinate system, with both left and right eyes facing the direction of the moving target.

[0093] (2) Using a beam-level time-series sonar simulation method, we generate realistic coherent image speckle and correct point spread functions within the Gazebo framework, and provide real-time sensor-level beam intensity data and scene rendering;

[0094] The overall procedure of the imaging sonar simulation process is as follows: Figure 3 As shown, specifically including:

[0095] A Gazebo-based camera plugin renders sonar field of view data from the scene using the actual sonar azimuth and elevation to generate a 3D point cloud of the target object. The point cloud resolution is set to match the number of beams in the sonar in azimuth and the number of rays in elevation.

[0096] b. Use discrete rays to model the beam: For N rays, set the subscripts of each ray to i = {1, 2, ..., N}, and for N B beams, and each beam is subscripted as j = {1, 2, ..., N B The following information is generated for each ray within a single beam:

[0097] The distance r from the origin of the sonar reference frame to the first intersection of the ray and the target in the field of view i

[0098] The difference α between the ray vector z and the normal vector n of the target surface i (angle of incidence)

[0099] The target reflectivity μ provided by the Gazebo 3D simulation framework at each cycle i

[0100] c. Obtain a set of two-dimensional field of view parameters based on the ray-based beam model in the sonar rendering scene:

[0101] Adopt a ray-based beam model, defining the beam as the vector z from the sensor frame origin to the first intersection with a visual object in the scene i , calculate the incident angle α i for

[0102]

[0103] in, and are the ray and normal unit vectors respectively;

[0104] Assume that dθ of each ray i and dφ i The angle changes are very small, and the beam scene projection area (the area projected by a single light ray onto the visual object) dA is calculated as:

[0105]

[0106] where dθ i is the infinitesimal element of the azimuth angle of the ray in the sensor frame, dφ i is the infinitesimal value of the ray elevation angle;

[0107] Calculate dS as a function of the angle of incidence using the area of ​​the ray projected onto the surface i :

[0108]

[0109] Get the parameters in the sonar field of view, including the target distance r i , ray angle θ i and φ i , ray region projection area dA, ray incident angle α i , target normal direction unit vector n i and target reflectivity μ i ;

[0110] d. Calculate the point scattering model for each ray data, including calculating Gaussian noise, amplitude, and total spectrum of the received signal:

[0111] The scattering intensity Ir can be obtained as a function of the incident angle i With the incident intensity I Ii When the target has a certain surface roughness or is contaminated by biological substances, the intensity ratio will change, replacing the incident angle function dS i , the amplitude of the point scatterer can be calculated using formula (10):

[0112]

[0113]

[0114] where μ is a parameter that controls the total reflectivity of the scattered field;

[0115] The total spectrum P of the signal received from each beam j (f) is calculated as:

[0116]

[0117] Where S(f) is the transmission spectrum of the source, N is the number of scatterers, and a n is the complex scatterer amplitude, and f is the sound frequency. j (f) Calculated as a combination of the physical model of the echo level and the complex random scaling factor of the speckle noise resolved in the frequency domain, with the acoustic wave number k ω =2πf / c, where c is the median speed of sound. The attenuation includes the imaginary part α of the wave number rad / m Generate a time series for each nominal beam angle, where the directivity of beam j is represented by D(θ,Φ) as a function of the vertical angle θ and the horizontal angle φ between the sensor and the scatterer;

[0118] The source spectrum is a user-defined input and remains constant for each ray. It is modeled in the frequency domain by S(f) using a Gaussian model. The center frequency and bandwidth parameters control the position and width of the spectrum, respectively:

[0119]

[0120] Calculate the scattering amplitude α for each ray i , where the variable ξ xi and ξ yi From independent Gaussian random variables, random variable ξ i Indexed by i to indicate the ray index:

[0121]

[0122] e. Calculate the sum of the rays for each beam:

[0123] To mitigate the inefficiency of simulating the time series of each element of the receive array and performing beamforming, each beam is simulated assuming a uniform ideal beam pattern across the beamwidth.

[0124] like Figure 4 As shown, the array's beam pattern is defined in polar coordinates, where the acoustic intensity is the distance along the radial axis and the angle is the angle relative to the transducer axis. The beam pattern is considered to have a central main lobe with smaller side lobes radiating outward from the main axis. Calculating the sum of the rays primarily involves calculating the beam pattern effect and the received spectrum. The main steps in calculating the beam pattern effect are:

[0125] Mark the beamwidth θ on the main lobe bω =-3dB, defining the half beamwidth θ ω for:

[0126]

[0127] The sidelobe effect is simulated by performing a correction where each beam is a weighted sum of all the other beams. The weights are the directivity pattern of the array, with the main response axis pointing in a specific direction, e.g.

[0128]

[0129] Among them, the weight ω i,j is the value of the beam pattern sampled at a specific angle. Using a directivity pattern corresponding to a uniform linear array, where θ i and θ j are the horizontal angles corresponding to the i-th and j-th beams respectively:

[0130] ω i,j =B(θ i -θ j ) (16)

[0131] For a continuous linear array of length L and radiated energy λ, the beam pattern is a uniform aperture function. The radiation pressure is modeled as a normalized sinc function:

[0132]

[0133] Where u is the electrical angle, expressed as

[0134] For high-frequency sonar, assuming L>>λ and using a small parameter approximation for the internal sine function, we end up with the beam pattern B(θ), which represents the pressure ratio of the array response at an angle θ relative to the main axis:

[0135]

[0136] f. Perform Windowing and FFT (Fast Fourier Transform) to obtain the distance intensity sonar data of each beam, which is divided into calculating the corrected receiving intensity and sound pressure level.

[0137] The following combination Figure 5 and Figure 6 The binocular camera effect and imaging sonar effect of this simulation system are analyzed respectively.

[0138] use Figure 5 The results of the observation of the moving target by the left and right eyes of the binocular camera are shown. It can be seen that the cross-sectional shape of the target can be clearly seen in the left and right eyes, and the image is clear. Figure 6 It can be seen that this imaging sonar simulation system can produce real coherent image speckles, and the imaging results are close to the real images.

[0139] From the statistical results of the above overall simulation effects and objective indicators, it can be seen that the imaging sonar and binocular camera simulation system and method based on multi-beamforming of the present invention has good imaging performance, obvious display of target features, and relatively accurate calculation of data, providing a very effective technical means for the simulation field of underwater target detection and tracking.

[0140] The preferred embodiments of the present invention have been described in detail above. It should be understood that numerous modifications and variations based on the concepts of the present invention are possible without inventive effort by those skilled in the art. Therefore, any technical solution that can be derived by one skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A joint simulation method of imaging sonar and binocular camera based on multi-beamforming, characterized in that: The following steps are involved: Step 1: Design an underwater video target system based on the Gazebo platform in the ROS system using a binocular camera and imaging sonar, and configure the environment. Step 2: Create a world model with a moving target. Use the binocular camera component provided by Gazebo, import the world model into the launch file, and use the ROS visualization tool - Rviz to observe the movement of the model in the left and right eyes of the camera by obtaining topics. Step 3: Using a beam-level time-series sonar simulation method, we generate realistic coherent image speckle and a correct point spread function within the Gazebo framework, and provide real-time sensor-level beam intensity data and scene rendering. Step 4: Based on the imaging characteristics of the multi-sensor, the measurement value of the underwater video target in the underwater multi-sensor simulation tracking system in the imaging sonar coordinate system is calculated.

2. The method for joint simulation of imaging sonar and binocular camera based on multi-beamforming according to claim 1, wherein: The step 1 comprises the following steps: Step 1.1: Design an underwater video target tracking simulation system based on imaging sonar and binocular camera; Step 1.2: Complete the configuration of the simulation environment in Gazebo.

3. The method for joint simulation of imaging sonar and binocular camera based on multi-beamforming according to claim 2, wherein: The step 1.1 includes system structure design, sensor plug-in design, and target model design.

4. The method for joint simulation of imaging sonar and binocular camera based on multi-beamforming according to claim 1, wherein: The step 2 comprises the following steps: Step 2.

1. Create and compile the plug-in source file for controlling the target motion. Step 2.2: Create a world model file, set the parameters of the motion target, and import the plugin to control the motion. Step 2.3: Start the launch file for the binocular camera, apply it to the world model, and use the ROS visualization tool - Rviz to select the left and right target topics of the binocular camera to obtain the target image.

5. The method for joint simulation of imaging sonar and binocular camera based on multi-beamforming according to claim 4, characterized in that: The plug-in source file in step 2.1 is a .cc source file.

6. The method for joint simulation of imaging sonar and binocular camera based on multi-beamforming according to claim 4, characterized in that: The step 2.1 includes the following steps: Step 2.1.1: Instantiate the motion target and set the motion model name and time parameters; Step 2.1.2: Set the specific parameters of the target movement and use the time of reaching the pose to set the speed; Step 2.1.

3. Set up the trigger to update the program in response to events and register the plugin in the Gazebo simulator.

7. The method for joint simulation of imaging sonar and binocular camera based on multi-beamforming according to claim 1, wherein: The step 3 comprises the following steps: Step 3.

1. Use the camera plug-in in Gazebo to obtain the underwater scene, and obtain the two-dimensional field of view parameter set based on the ray-based beam model in the sonar rendering scene; Step 3.2, calculate the point scattering model for each ray data; Step 3.3, calculate the sum of rays for each beam; Step 3.4: Calculate the beam pattern effect of the beam, and perform windowing and fast Fourier transform to obtain the range intensity sonar data of each beam.

8. The method for joint simulation of imaging sonar and binocular camera based on multi-beamforming according to claim 1, wherein: The step 4 comprises the following steps: Step 4.1, calculating the measurement value of the underwater video target in the coordinate system of the underwater multi-sensor detection and tracking system; Step 4.2: Calculate the measurement value of the underwater video target in the imaging sonar coordinate system.

9. The method for joint simulation of imaging sonar and binocular camera based on multi-beamforming according to claim 1, wherein: The binocular camera element is a cube.

10. The method for joint simulation of imaging sonar and binocular camera based on multi-beamforming according to claim 1, wherein: The topics published by the left and right eyes of the binocular camera are "left" and "right".

Images