A method for generating simulated sonar image data based on Unity3D

Abstract

The present invention provides a method for generating simulated sonar image data based on Unity3D, including: establishing a three-dimensional model of seabed terrain and a three-dimensional model of the target object based on seabed terrain point cloud information and the target object; importing the established three-dimensional model into Unity3D to build a simulated seabed scene; modeling the sound waves emitted by the sonar as a group of equidistant ray clusters to form a forward-looking three-dimensional rotating body area and a side-scanning two-dimensional fan-shaped area of ​​a given position and direction; modeling the reflection of the sound wave at the target object as a collision between the ray and the three-dimensional model of the target object, and calculating the intensity of the sound wave simulated at the same distance for each ray in the ray cluster according to the echo time sequence after the collision, and the result is a frame of sonar simulation image data. The present invention is convenient for simulating and generating multi-angle and multi-directional simulated side-scan sonar and forward-looking sonar images of complex seabed scenes, and the imaging principle is close to that of real sonar, with high accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method for generating simulated sonar image data based on Unity3D. Background Art

[0002] Sonar sensors are widely used in underwater environment detection due to their wide scanning range and ability to observe objects in dark or highly turbid environments. Research based on sonar image data, such as underwater target detection methods based on deep learning, often requires a large number of sonar image samples. However, due to the high price of sonar sensors and the complex experimental conditions of underwater environments, it takes a lot of time and resources to obtain real sonar images, which is difficult. Sonar image simulation can be used to synthesize sonar images under various settings, greatly reducing the difficulty of acquisition.

[0003] Using generative adversarial networks and stylized transfer technology to synthesize sonar images is a commonly used method for generating sonar simulation images. However, the simulated images obtained by this method differ from the original images in terms of content and features, and their accuracy cannot be guaranteed. At the same time, this method also requires a large amount of real sonar data to train the network to achieve good results, and does not fundamentally solve the problem of insufficient data samples. In addition, this method cannot simulate additional information such as the position and angle of the submarine contained in the real sonar data, and the obtained data is difficult to apply to research such as image stitching.

[0004] At present, the methods of using three-dimensional scenes to generate sonar images are mostly simplified scenes that combine simple geometric targets and plane terrains, which have a low similarity to real seabed scenes. The simulated sonar images are only separate forward-looking sonars or side-scan sonars, and the sonar scanning area is also fixed. The resulting simulated images are single and differ greatly from the real data, and have limited effect on expanding the data set. In addition, these methods cannot simulate the scanning method of real submersibles underwater, and cannot obtain continuous simulation data.

[0005] Therefore, how to provide a method for generating image data of multi-angle and multi-directional simulated side-scan sonar and forward-looking sonar images for complex seabed scenes that is easy to simulate and has high accuracy is an urgent problem that technicians in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a method for generating simulated sonar image data based on Unity3D, which is convenient for simulating and generating multi-angle and multi-directional simulated side-scan sonar and forward-looking sonar images of complex seabed scenes. The imaging principle is close to that of real sonar and has high accuracy.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A method for generating simulated sonar image data based on Unity3D, comprising:

[0009] S1. Building a three-dimensional model of the seabed terrain and a three-dimensional model of the target object based on the seabed terrain point cloud information and the target object;

[0010] S2, import the 3D model established in S1 into Unity3D to build a simulated seabed scene;

[0011] S3, modeling the sound waves emitted by the sonar as a set of equally spaced ray clusters, forming a forward-looking three-dimensional rotating body area and a side-scanning two-dimensional fan-shaped area at a given position and direction;

[0012] S4. Model the reflection of sound waves at the target as the collision between rays and the three-dimensional model of the target. According to the order of echo time after the collision, at fixed intervals, calculate the simulated sound wave intensity at the same distance for each ray in the ray cluster. The process ends when the distance reaches the maximum radius of the sonar area. The result is a frame of sonar simulation image data.

[0013] Preferably, S2 includes setting different materials for different targets in Unity3D.

[0014] Preferably, S2 includes: steps of deleting and placing targets in a simulated seabed scene based on a C# script.

[0015] Preferably, the sonar in S3 includes a forward-looking sonar:

[0016] The forward-looking sonar is modeled as a three-dimensional rotating body region on the xoy plane, which is a sector-shaped region rotated around the y-axis at a fixed angle, and the sector-shaped region has a preset angle with the x-axis.

[0017] Preferably, the sonar in S3 includes a side-scan sonar:

[0018] The side scan sonar is modeled as two two-dimensional fan-shaped areas on the yoz plane, and the two-dimensional fan-shaped areas have a preset angle with the -z axis.

[0019] Preferably, S3 includes: a step of adjusting the central angle, radius, and preset angle of the sonar area based on a C# script.

[0020] Preferably, before S4, the method further includes: adding a submarine model to the scene, binding the sonar area to the relative position of the submarine, controlling the movement of the submarine based on a C# script, and modifying the relative position and angle of the binding between the sonar and the submarine model.

[0021] Preferably, after S4, the step further includes: adding salt and pepper noise to the sonar simulation image data.

[0022] Preferably, after S4, the step further includes: saving the position and angle information of the submersible and the sonar simulation data of the current frame simultaneously.

[0023] It can be seen from the above technical solution that, compared with the prior art, the beneficial effects of the present invention include:

[0024] The present invention can conveniently simulate the real submersible scanning method, and generate multi-angle and multi-directional simulated side-scan sonar and forward-looking sonar images of complex seabed scenes according to user needs. The imaging principle is close to that of real sonar, ensuring data accuracy. At the same time, the method can generate virtual data without a large amount of real sonar data as samples. In addition, the method can provide relevant information such as the simulated position and angle corresponding to the image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative work.

[0026] Figure 1 A flow chart of a method for generating simulated sonar image data based on Unity3D provided in an embodiment of the present invention;

[0027] Figure 2 It is a forward-looking sonar modeling simulation diagram provided by an embodiment of the present invention;

[0028] Figure 3 It is a side scan sonar modeling simulation diagram provided by an embodiment of the present invention;

[0029] Figure 4 The embodiments of the present invention provide point cloud information of real seabed terrain and target object images;

[0030] Figure 5 The three-dimensional model of the seabed terrain and the three-dimensional model diagram of the target object are established according to the real seabed terrain point cloud information and the target object provided by the embodiment of the present invention;

[0031] Figure 6 This is a simulated sonar image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] like Figure 1 As shown, the present invention provides a method for generating simulated sonar image data based on Unity3D provided in this embodiment, including:

[0034] S1. Based on the seabed topography point cloud information and the target object, a three-dimensional model of the seabed topography and a three-dimensional model of the target object are established.

[0035] In one embodiment of the process of executing this step, 3dmax software is used to import the real seabed terrain point cloud information to establish a three-dimensional model of the seabed terrain. 3dmax software is used to establish three-dimensional models of target objects such as sunken ships, airplanes, pipelines, and tires.

[0036] S2. Import the 3D model created in S1 into Unity3D to build a simulated seabed scene.

[0037] In one embodiment of the process of performing this step, different materials are set for different targets in Unity3D. Different materials will affect the echo attenuation degree of the sound waves emitted by the sonar.

[0038] In one embodiment of the process of executing this step, the objects are arbitrarily deleted and placed in the scene according to user needs based on C# script.

[0039] S3. The sound waves emitted by the sonar are modeled as a set of equally spaced ray clusters, forming a forward-looking three-dimensional rotating body area and a side-scanning two-dimensional fan-shaped area at a given position and direction.

[0040] In one embodiment of the process of performing this step, Figure 2 As shown, the sonar includes a forward-looking sonar: the forward-looking sonar is modeled as a three-dimensional rotating body area on the xoy plane, which is rotated around the y-axis by a fixed angle, and the sector has an angle of 20 degrees with the x-axis.

[0041] In one embodiment of the process of performing this step, Figure 3 As shown, the sonar includes a side scan sonar: the side scan sonar is modeled as two two-dimensional fan-shaped areas on the yoz plane, and the two-dimensional fan-shaped areas have a preset angle with the -z axis.

[0042] In the above two specific embodiments, the center angle, radius and x-axis angle of the forward-looking sonar area and the center angle, radius and -z-axis angle of the side-scan sonar area are adjusted according to user needs based on the C# script.

[0043] S4. The reflection of the sound wave at the target object is modeled as the collision between the ray and the three-dimensional model of the target object. The intensity of the sound wave is attenuated according to the distance of the collision point, the material of the target object, and the normal of the collision surface. According to the echo time after the collision, at fixed intervals, the intensity of the sound wave simulated at the same distance is calculated for each ray in the ray cluster. When the distance reaches the maximum radius of the sonar area, it ends. The result is a frame of sonar simulation image data.

[0044] During the execution of this step, the pixels from the center to both sides of the sonar image can be understood as the echo intensity from the near to the far position of the submarine. From the emission point to the maximum scanning distance, a fixed resolution is calculated once every interval, and a horizontal pixel is obtained on the sonar image. The interval distance can be adjusted according to the required resolution and maximum scanning distance.

[0045] In one embodiment, the step before S4 also includes: adding a submarine model to the scene, binding the sonar area to the relative position of the submarine, controlling the movement of the submarine based on a C# script, and modifying the relative position and angle of the sonar bound to the submarine model. The submarine can scan the sonar area while moving, so as to simulate the working mode of the forward-looking sonar and side-scanning sonar of a real submarine at the same time.

[0046] In one embodiment, S5, adding salt and pepper noise to the sonar simulation image data to simulate the noise impact in sonar transmission.

[0047] S6. The additional information such as the position and angle information of the submersible is saved simultaneously with the sonar simulation data of the current frame.

[0048] like Figure 4 As shown, it is the real seabed topography point cloud information and target image. Figure 5 In order to establish a three-dimensional seabed terrain model and a three-dimensional target object model based on the real seabed terrain point cloud information and the target object, the simulated targets are tires and square boxes. Figure 6 The figure shows the simulated sonar image data obtained by simulation based on the method of this embodiment, which can scan the real scene information over a large range and with high precision to obtain a simulated sonar image with high accuracy.

[0049] The principle of generating simulation data in this embodiment is close to real sonar, which ensures data accuracy. At the same time, a large amount of real sonar data is not required as samples. In addition, the simulated position and angle corresponding to the image data can be given, which is additional information contained in the real sonar data. The simulation system can easily simulate the real submersible scanning method and can scan the scene from multiple angles and directions. The seabed scene imports real seabed terrain information, with complex targets and can be adjusted according to user needs.

[0050] The above is a detailed introduction to the method for generating simulated sonar image data based on Unity3D provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

[0051] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating simulated sonar image data based on Unity3D, characterized in that: include: S1. Building a three-dimensional model of the seabed terrain and a three-dimensional model of the target object based on the seabed terrain point cloud information and the target object; S2, import the 3D model established in S1 into Unity3D to build a simulated seabed scene; S3, modeling the sound waves emitted by the sonar as a group of equally spaced ray clusters to form a forward-looking three-dimensional rotating body area and a side-scanning two-dimensional fan-shaped area at a given position and direction; the sonar includes a forward-looking sonar and a side-scanning sonar: The forward-looking sonar is modeled as a three-dimensional rotating body area on the xoy plane that rotates around the y axis at a fixed angle, and the fan-shaped area has a preset angle with the x-axis; the side-scan sonar is modeled as two two-dimensional fan-shaped areas on the yoz plane, and the two-dimensional fan-shaped areas have a preset angle with the -z axis; S4. Model the reflection of sound waves at the target as the collision between rays and the three-dimensional model of the target. According to the order of echo time after the collision, at fixed intervals, calculate the simulated sound wave intensity at the same distance for each ray in the ray cluster. The process ends when the distance reaches the maximum radius of the sonar area. The result is a frame of sonar simulation image data.

2. The method for generating simulated sonar image data based on Unity3D according to claim 1, characterized in that: The S2 includes setting different materials for different targets in Unity3D.

3. The method for generating simulated sonar image data based on Unity3D according to claim 1, characterized in that: The S2 includes: steps of deleting and placing targets in the simulated seabed scene based on a C# script.

4. The method for generating simulated sonar image data based on Unity3D according to claim 1, characterized in that: The S3 includes: adjusting the central angle, radius, and preset angle of the sonar area based on a C# script.

5. The method for generating simulated sonar image data based on Unity3D according to claim 1, characterized in that: The S4 also includes: adding a submarine model to the scene, binding the sonar area to the relative position of the submarine, controlling the movement of the submarine based on a C# script and modifying the relative position and angle of the sonar and submarine model.

6. The method for generating simulated sonar image data based on Unity3D according to claim 1, characterized in that: After S4, the method further includes: adding salt and pepper noise to the sonar simulation image data.

7. The method for generating simulated sonar image data based on Unity3D according to claim 1, characterized in that: The step after S4 also includes: simultaneously saving the position and angle information of the submersible and the sonar simulation data of the current frame.

Images