Marine unknown scene image synthesis method

Abstract

The invention discloses a marine unknown scene image synthesis method. The method comprises the following steps: S1, processing a marine environment background picture; s2, synthesis of target material picture processing is carried out; s3, according to the distance ranges of different types of ships, the distance of the ship target is configured, and a horizontal field angle and a vertical field angle are set; s4, calculating the converted pixel distance, wherein the data content provided by the pixel distance calculation comprises parameters given by the ship and unmanned aerial vehicle parameters; s5, calculating the pixel height after conversion; s6, acquiring the center point position of the target; and S7, synthesizing the target into the ocean background. Through a target and image synthesis method, image synthesis of a new scene is realized, and simulation of an unknown scene can be quickly realized; and the data set is used as a training and verification data set of an intelligent detection and recognition algorithm of the marine target and environment, so that the problem of lack of marine task scene data can be solved with low cost and high efficiency.

Description

Technical Field

[0001] The invention relates to the field of marine virtual environment generation, and in particular to a method for synthesizing images of marine unknown scenes. Background Art

[0002] The rapid development of unmanned intelligent technology at sea has put forward huge new demands on the refined simulation capabilities of complex marine environments. It is of great significance to accelerate the development and application of unmanned marine systems. There are problems such as low efficiency, high cost and long cycle in real boat testing at sea and testing methods based on real data. Virtual testing and training of mission scenarios of unmanned marine systems has become one of the important solutions. However, the construction theories and methods of virtual environments for traditional unmanned marine systems are relatively poor in scene modeling accuracy and sensor simulation accuracy. As a result, the virtual environment can only reflect the real situation logically and qualitatively. The training and testing efficiency of intelligent algorithms or intelligent systems is reduced.

[0003] Although the industry has carried out some research work on marine environment simulation, the sensor simulation accuracy is generally low, which makes it difficult to meet the training and verification needs of intelligent perception algorithms. Among them, a set of unmanned boat intelligent driving simulators was developed based on the Unity platform, which performed high-precision modeling of environmental weather, waves, visible light and radar modules. Although the modeling accuracy has been improved, it is difficult to cover up the problems of simple models and environmental distortion. A set of unmanned boat simulation platforms developed based on Unity uses CFD simulation technology to accurately model water flow and wind field environments, but ignores the problems of scene construction and sensor modeling.

[0004] Therefore, this scheme proposes a method for synthesizing images of unknown marine scenes to solve the above-mentioned problems. Summary of the invention

[0005] The purpose of the present invention is to provide a method for synthesizing images of unknown marine scenes to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a method for synthesizing an image of an unknown marine scene, the method comprising the following steps:

[0007] S1, firstly processes the ocean environment background image;

[0008] S2, synthesizing the target material image processing;

[0009] S3, according to the distance range of different types of ships, configure the distance of the ship target and set the horizontal and vertical field of view angles;

[0010] S4, calculating the converted pixel distance, wherein the data content provided by the pixel distance calculation includes the parameters given by the ship and the parameters of the drone;

[0011] S5, calculating the converted pixel height, wherein the data content provided by the pixel height calculation includes the parameters given by the ship and the parameters of the drone;

[0012] S6, obtaining the center point position of the target;

[0013] S7, synthesize the target into the ocean background.

[0014] Preferably, the processing content of the ocean environment background picture in S1 is to put the ocean environment background picture into a fixed folder, and name and rename the folder. The named name can be confirmed according to the classification of the folder.

[0015] Preferably, the S2 synthetic target material picture processing first creates a target general folder, then creates different folders for different types of ships in it, and finally puts the fitted target picture into the corresponding folder.

[0016] Preferably, S3 randomly configures a specific distance for the hull according to the distance ranges of different types of ship targets in the configuration file, and sets the horizontal field of view angle and the vertical field of view angle of the hull in the camera.

[0017] Preferably, the S4 pixel distance is obtained in terms of ships according to given parameters by the following formula:

[0018]

[0019] Among them, BGInfo[h] represents the installation height of the camera, dis represents the actual distance between the target object and the camera, represents the pixel range of the sea level in the image, Indicates the target's field of view angle in the vertical direction.

[0020] Preferably, the S4 pixel distance is obtained in the drone according to the given parameters by the following formula:

[0021]

[0022] Among them, BGInfo[h] represents the camera height, and altitude represents the altitude of the drone.

[0023] Preferably, the S5 pixel height is obtained in terms of the ship according to the given parameters by the following formula:

[0024]

[0025] Among them, real_height represents the actual height of the target, and dir_angle represents the angle of the target in the horizontal direction.

[0026] Preferably, the S5 pixel height is obtained in the drone according to the given parameters by the following formula:

[0027]

[0028] Among them, real_height represents the actual height of the target.

[0029] Preferably, the calculation formula for the center point position of S6 is:

[0030] center=round(((1 / 2)-(1 / 2)*tan(40°)·tan(-hor°))·BGInfo[length])(5);

[0031] Among them, BGInfo[length] represents the length of the background image.

[0032] Preferably, the S7 target synthesis determines the layer level of the target according to the distance of the target, and generates a corresponding mask image and box label txt file at the same time as the synthetic image is generated. The texture of the target is adjusted through the synthetic image and the mask image using an image coordination network to make it meet the environmental conditions of the background image.

[0033] Technical effects and advantages of the present invention:

[0034] The present invention realizes image synthesis of new scenes through the target and image synthesis method, which can quickly realize the simulation of unknown scenes; and uses it as a training and verification data set for the intelligent algorithm for detecting and identifying marine targets and environments, which can solve the problem of lack of marine mission scene data at low cost and high efficiency; and the method of the present invention provides a nurturing environment for the intelligent training and capability verification of marine unmanned systems, and accelerates the iterative upgrade of the intelligence level of marine unmanned systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The present invention is an operational flow chart of the synthesis method. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] The present invention provides Figure 1 The method for synthesizing an image of an unknown marine scene shown in the figure comprises the following steps:

[0038] S1, firstly processes the ocean environment background image;

[0039] Specifically, the processing content of the ocean environment background picture in S1 is to put the ocean environment background picture into a fixed folder, and name and rename the folder. The named name can be confirmed according to the classification of the folder.

[0040] It should be noted that the marine environment background picture is placed in a fixed folder, which is named "back1" here. Multiple folders can also be named according to classification, including "back2" and "back3"; the background picture is renamed.

[0041] It should be noted that a target folder is created, in which different folders are created for different types of ships, for example: "fishing", "UAV" or "combat"; the fitted target images are placed in the corresponding folders.

[0042] S2, synthesizing the target material image processing;

[0043] Specifically, S2 synthesizes the target material image processing, first creates a target general folder, then creates different folders for different types of ships in it, and finally puts the fitted target image into the corresponding folder.

[0044] S3, according to the distance range of different types of ships, configure the distance of the ship target and set the horizontal and vertical field of view angles;

[0045] Specifically, S3 randomly configures a specific distance for the hull according to the distance ranges of different types of ship targets in the configuration file, and sets the horizontal and vertical field of view angles of the hull in the camera.

[0046] It should be noted that when processing the distance range and field of view configuration of ship targets, the specific distance is randomly generated according to the different types of ships, and the horizontal and vertical field of view angles in the camera are set, including:

[0047] Define ship types and their distance ranges, and list different types of ships and their distance ranges according to the configuration file;

[0048] Randomly generate distance, randomly generate a specific distance value according to the distance range;

[0049] Set the field of view to define the appropriate horizontal and vertical field of view for each ship type;

[0050] Output the results and output the generated distance and field of view angle.

[0051] Among them, the ship type and distance range represent the minimum and maximum distances defined for different types of ships; the field of view represents the horizontal and vertical field of view defined for each type of ship; random.choice is used to randomly select the ship type, and random.uniform is used to generate a random distance within a given range; finally, the selected ship type, random distance, and corresponding field of view are output.

[0052] S4, calculating the converted pixel distance, wherein the data content provided by the pixel distance calculation includes the parameters given by the ship and the parameters of the drone;

[0053] Specifically, the S4 pixel distance in terms of ships is obtained according to the given parameters by the following formula:

[0054]

[0055] Among them, BGInfo[h] represents the installation height of the camera, dis represents the actual distance between the target object and the camera, represents the pixel range of the sea level in the image, Indicates the field of view angle of the target in the vertical direction, and then obtains the converted pixel distance; according to the width of the picture, obtains the y value position of the lower boundary of the hull in the background.

[0056] The S4 pixel distance is calculated based on the given parameters of the drone using the following formula:

[0057]

[0058] Among them, BGInfo[h] represents the camera height, altitude represents the altitude of the drone, and the position of the lower boundary in the image is obtained like a ship.

[0059] S5, calculating the converted pixel height, wherein the data content provided by the pixel height calculation includes the parameters given by the ship and the parameters of the drone;

[0060] Specifically, the S5 pixel height is obtained from the following formula based on the given parameters in terms of ships:

[0061]

[0062] Among them, real_height represents the actual height of the target, dir_angle represents the angle of the target in the horizontal direction, and then the converted pixel height is obtained; the y value position of the upper boundary of the hull in the background is obtained according to the image width.

[0063] The S5 pixel height is calculated based on the given parameters of the drone using the following formula:

[0064]

[0065] Among them, real_height represents the actual height of the target. Like a ship, the position of the upper boundary in the picture is obtained, and finally the center point position is obtained. Based on the obtained upper boundary, lower boundary and center point, the original hull picture is resized to the background picture.

[0066] S6, obtaining the center point position of the target;

[0067] Specifically, the calculation formula for the center point position of S6 is:

[0068] center=round(((1 / 2)-(1 / 2)*tan(40°)·tan(-hor°))·BGInfo[length])(5);

[0069] Among them, BGInfo[length] indicates the length of the background image. According to the obtained upper boundary, lower boundary and center point, the original hull image is resized to the background image.

[0070] S7, synthesize the target into the ocean background.

[0071] Specifically, S7 target synthesis determines the layer level of the target (i.e., which one is above and which one is below) according to the distance of the target, and generates the corresponding mask image and box label txt file at the same time as the synthetic image is generated. The texture of the target is adjusted through the synthetic image and mask image using the image coordination network to make it meet the environmental conditions of the background image.

[0072] It should be noted that the synthesis of the target into the ocean background includes the following steps: layer level determination of the target, generation of the composite image and mask image, creation of box labels, and texture adjustment using an image coordination network;

[0073] 1. Determine the distance and layer level of the target including:

[0074] Target distance measurement, according to the relative distance between the target and the observer, the layer level of each target is determined, where the closer targets should be in the upper layer and the farther targets should be in the lower layer;

[0075] Tier sorting: Assign a tier to each target, sort by distance, and generate a layer list.

[0076] 2. The generation of synthetic images and mask images includes:

[0077] The generation of composite images, using image synthesis technology to combine the target image with the background image, ensuring that the target is at the correct level, for example, using image editing software (including Photoshop or OpenCV) to synthesize layer by layer;

[0078] Generation of mask images: A binary mask image is generated for each target. The target area in the mask image is white and the background area is black. The mask is generated through image segmentation technology (including threshold processing and edge detection).

[0079] 3. The creation of box labels includes:

[0080] Label format, using text file format (YOLO format) to record the location and category of each target;

[0081] Format example,<class_id><x_center><y_center> <width> <height>, where the coordinates and dimensions need to be normalized to the image size.

[0082] Fourth, texture adjustment using image coordination network includes:

[0083] Image coordination network, using pre-trained image coordination networks (including CycleGAN, pix2pix), taking the composite image and mask image as input, and performing texture adjustment. After inputting the composite image and mask image, the network will adjust the texture of the target according to the environmental conditions of the background image to make it more natural and coordinated;

[0084] Training and fine-tuning: Fine-tune the network to suit specific contexts and object types.

[0085] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.< / height> < / width>

Claims

1. A method for synthesizing an image of an unknown marine scene, characterized in that: The synthesis method comprises the following steps: S1, firstly processes the ocean environment background image; S2, synthesizing the target material image processing; S3, according to the distance range of different types of ships, configure the distance of the ship target and set the horizontal and vertical field of view angles; S4, calculating the converted pixel distance, wherein the data content provided by the pixel distance calculation includes the parameters given by the ship and the parameters of the drone; S5, calculating the converted pixel height, wherein the data content provided by the pixel height calculation includes the parameters given by the ship and the parameters of the drone; S6, obtaining the center point position of the target; S7, synthesize the target into the ocean background.

2. The method for synthesizing an image of an unknown marine scene according to claim 1, characterized in that: The processing content of the ocean environment background picture in S1 is to put the ocean environment background picture into a fixed folder, and name and rename the folder. The named name can be confirmed according to the classification of the folder.

3. The method for synthesizing an image of an unknown marine scene according to claim 1, characterized in that: The S2 synthetic target material image processing first creates a target general folder, then creates different folders for different types of ships in it, and finally puts the fitted target image into the corresponding folder.

4. The method for synthesizing an image of an unknown marine scene according to claim 1, characterized in that: The S3 randomly configures a specific distance for the hull according to the distance ranges of different types of ship targets in the configuration file, and sets the horizontal field of view angle and the vertical field of view angle of the hull in the camera.

5. The method for synthesizing an image of an unknown marine scene according to claim 1, characterized in that: The S4 pixel distance is obtained in terms of the ship according to the given parameters by the following formula: Among them, BGInfo[h] represents the installation height of the camera, dis represents the actual distance between the target object and the camera, BGInfo[csea] represents the pixel range of the sea level in the image, and BGInfo[angle ver] represents the field of view angle of the target in the vertical direction.

6. The method for synthesizing an image of an unknown marine scene according to claim 5, characterized in that: The S4 pixel distance is obtained in the drone according to the given parameters by the following formula: Among them, BGInfo[h] represents the camera height, and altitude represents the altitude of the drone.

7. The method for synthesizing an image of an unknown marine scene according to claim 1, characterized in that: The S5 pixel height is obtained in terms of the ship according to the given parameters by the following formula: Among them, real height represents the actual height of the target, and dir angle represents the angle of the target in the horizontal direction.

8. The method for synthesizing an image of an unknown marine scene according to claim 7, characterized in that: The S5 pixel height is obtained in the drone according to the given parameters by the following formula: Among them, real height represents the actual height of the target.

9. The method for synthesizing an image of an unknown marine scene according to claim 1, characterized in that: The calculation formula of the center point position of S6 is: center=round(((1 / 2)-(1 / 2)*tan(40°)·tan(-hor°))·BGInfo[length]) (5); Among them, BGInfo[length] represents the length of the background image.

10. The method for synthesizing an image of an unknown marine scene according to claim 1, characterized in that: The S7 target synthesis determines the layer level of the target according to the distance of the target, and generates the corresponding mask image and box label txt file at the same time as the synthesis image is generated. The texture of the target is adjusted through the synthesis image and mask image using the image coordination network to make it meet the environmental conditions of the background image.

Images