Data augmentation method for acoustic optical image based on image local features

Abstract

The invention discloses a data augmentation method for an acoustic optical image based on image local features, and relates to the technical field of underwater image feature enhancement, and the method comprises the following steps: S1, data preparation: selecting acoustic and optical images under different depths and different water quality conditions for an experiment; s2, extracting local features, and carrying out edge detection and texture analysis on the prepared image; and S3, data augmentation: fusing the acoustic image and the optical image and locally and specially augmenting the acoustic image and the optical image. According to the method, through edge detection, texture analysis and combination of a fusion algorithm and a deep learning algorithm, fine extraction and enhancement of local features of an underwater image are realized, and meanwhile, the advantages of an acoustic image and an optical image are combined; the deep learning algorithm and the image processing technology are utilized to realize fine extraction and enhancement of the local features of the underwater image, so that the definition and quality of the image are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image feature enhancement, and in particular to a data augmentation method for acoustic optical images based on local features of the images. Background Art

[0002] With the urgent need for underwater resource development, high-quality and clear underwater images have become an important guarantee for a series of underwater operations such as marine resource exploration and marine safety monitoring. However, the underwater environment is complex, and images are often affected by light attenuation, scattering and absorption, resulting in color distortion, blurred details and low contrast. In addition, the underwater imaging mechanism is different from that on land. The propagation attenuation and scattering characteristics of light in water lead to poor underwater image quality. The current data augmentation methods for acoustic optical images based on local image features generally use one of the following methods: histogram equalization, contrast-limited adaptive histogram equalization, defogging algorithm and multi-scale reflection elimination algorithm. However, these methods mainly rely on optical information and fail to make full use of the acoustic information of the underwater environment, resulting in poor image clarity and quality. Summary of the invention

[0003] The present invention provides a data augmentation method for acoustic optical images based on local features of the images, which solves the problems in the background technology.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A method for data augmentation of an acoustic optical image based on local features of the image comprises the following steps:

[0006] S1. Data preparation: Select acoustic and optical images at different depths and water quality conditions for experiments;

[0007] S2, local feature extraction, edge detection, texture analysis, scale space construction and key point detection of the prepared image;

[0008] S3, data augmentation, which fuses the acoustic image and the optical image and enhances their local features. After extracting the local features of the image, geometric transformation, color gamut transformation and clarity transformation are performed;

[0009] S4, model training and optimization, bringing the augmented data into the training model for training and optimization;

[0010] S5. Experimental Results and Analysis,Through the local feature enhancement processing of the deep learning,algorithm, the image quality and recognition accuracy are further improved.

[0011] As a further description of the above technical solution:

[0012] The edge detection in the local feature extraction adopts the CNN algorithm. The CNN algorithm extracts the features of the image through convolution operation. In edge detection, the convolution layer of CNN is used to capture the edge information in the image. The formula is as follows:

[0013] y(i,j)=(x*ω)(i,j)=∑ m ∑ n x(i+m,j+n)ω(m,n).

[0014] As a further description of the above technical solution:

[0015] The texture analysis in the local feature extraction uses a texture analysis method based on a gray level co-occurrence matrix to perform texture analysis on underwater images. The scale space is constructed in the local feature extraction to generate images of different scales through Gaussian blurring and downsampling. The key points, i.e., stable feature points in images of different scales, are detected in the local feature extraction, and the directional gradient histogram of the key points is calculated to generate a feature descriptor.

[0016] As a further description of the above technical solution:

[0017] The fusion of acoustic images and optical images in the data augmentation includes image registration and image fusion, and the local feature enhancement in the data augmentation also includes feature extraction network, feature fusion and loss function design.

[0018] As a further description of the above technical solution:

[0019] The geometric transformations in the data augmentation include rotation transformation, flip transformation and scaling transformation. The rotation transformation is to rotate the image around the center point by a certain angle to generate a new image. The rotation angle is randomly selected between [-15, 15] degrees. The formula for the rotation transformation is:

[0020] I′(x, y)=I(x′, y′)

[0021] Among them, x′=(x-ω / 2)cosθ+(yh / 2)sinθ+w / 2;

[0022] y′=(yh / 2)cosθ-(x-ω / 2)sinθ+h / 2,

[0023] Where I(x,y) represents the original image, I′(x,y) represents the rotated image, ω and h represent the width and height of the image respectively, and θ represents the rotation angle;

[0024] The flip transformation is to flip the image horizontally or vertically to generate a new image. The formula for the flip transformation is:

[0025] I′(x,y)=I(wx-1,y) (horizontal flip);

[0026] I′(x,y)=I(x,hy-1)(vertical flip);

[0027] The scaling transformation is to scale the image to generate a new image. The scaling ratio is randomly selected between [0.8, 1.2]. The scaling transformation formula is:

[0028] I′(x,y)=I(x′,y′);

[0029] Where x′ = x·scale;

[0030] y′=y·scale;

[0031] Among them, scale represents the scaling ratio.

[0032] As a further description of the above technical solution:

[0033] The model training and optimization include model selection, data augmentation, distributed training and optimization algorithms.

[0034] As a further description of the above technical solution:

[0035] The steps of the CNN algorithm include input data, data enhancement and normalization.

[0036] As a further description of the above technical solution:

[0037] The color gamut transformation in the data augmentation includes brightness transformation, contrast transformation and saturation transformation. The brightness transformation is to adjust the brightness of the image to generate a new image. The brightness variation range is randomly selected between [0.5, 1.5]. The formula for brightness transformation is:

[0038] I′(x,y)=α·I(x,y);

[0039] Among them, α represents the brightness change coefficient;

[0040] The contrast transformation is to adjust the contrast of the image to generate a new image. The contrast variation range is randomly selected between [0.5, 1.5]. The contrast transformation can be achieved by linear transformation. The formula is:

[0041] I′(x,y)=β·(I(x,y)-μ)+μ;

[0042] Among them, β represents the contrast variation coefficient, and μ represents the mean of the image;

[0043] The saturation transformation adjusts the saturation of the image to generate a new image. The saturation change range is randomly selected between [0.5, 1.5]. The saturation transformation can be performed in the HSV color space. The formula is:

[0044] S'(x,y)=γ·S(x,y);

[0045] Among them, S(x, y) represents the saturation of the original image, S'(x, y) represents the adjusted saturation, and γ represents the saturation change coefficient.

[0046] As a further description of the above technical solution:

[0047] In the data augmentation, the definition transformation performs sharpening and blurring processing on the image to generate a new image. The sharpening can be achieved by a high-contrast edge filter, and the blurring can be achieved by a Gaussian filter.

[0048] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0049] In the present invention, the method realizes the fine extraction and enhancement of local features of underwater images through the combination of edge detection, texture analysis and fusion algorithm with deep learning algorithm. At the same time, it combines the advantages of acoustic images and optical images, and uses deep learning algorithm and image processing technology to realize the fine extraction and enhancement of local features of underwater images, thereby improving the clarity and quality of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention is a flow chart of a method for data augmentation of acoustic optical images based on local features of the image. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Reference Figure 1 , a data augmentation method for acoustic optical images based on local features of the image, comprising the following steps:

[0053] S1. Data preparation: select acoustic and optical images at different depths and water quality conditions for experiments; S2. Local feature extraction: edge detection, texture analysis, scale space construction and key point detection of the prepared images; S3. Data augmentation: fuse the acoustic image and the optical image and enhance their local features, and perform geometric transformation, color gamut transformation and clarity transformation after extracting the local features of the image; S4. Model training and optimization: bring the augmented data into the training model for training and optimization; S5. Experimental results and analysis: through deep learning algorithms The local feature enhancement processing further improves the image quality and recognition accuracy. In data preparation, acoustic images are obtained through underwater sonar equipment, which mainly reflect the shape and position information of underwater targets; optical images are taken by underwater cameras to capture detailed information such as the target's color and texture. In addition, data preprocessing is required when preparing data. Underwater images are preprocessed, including denoising, grayscale and normalization. Denoising can use median filtering or Gaussian filtering, grayscale converts color images into grayscale images, and normalization adjusts image pixel values ​​to the range of 0-1.

[0054] Furthermore, the edge detection in the local feature extraction adopts the CNN algorithm, and the CNN algorithm extracts the features of the image through convolution operation. In edge detection, the convolution layer of CNN is used to capture the edge information in the image. The formula is as follows:

[0055] y(i,j)=(x*ω)(i,j)=∑ m ∑ n x(i+m,j+n)ω(m,n),

[0056] Among them, x is the input image, ω is the convolution kernel (filter), and y is the output image after convolution. After the convolution operation, a pooling operation is required. There are many pooling operations usually used for downsampling, such as maximum pooling and average pooling. Take maximum pooling as an example:

[0057] y=(i,j)=max m，n∈R x(i+m,j=n)

[0058] Where R is the pooling window.

[0059] Furthermore, in the local feature extraction, the texture analysis adopts a texture analysis method based on gray level co-occurrence matrix to perform texture analysis on underwater images. In the local feature extraction, a scale space is constructed to generate images of different scales through Gaussian blurring and downsampling. In the local feature extraction, key points, i.e. stable feature points in images of different scales, are detected and the directional gradient histogram of the key points is calculated to generate feature descriptors, including gray level quantization: the gray level of the image is quantized to reduce the computational complexity; GLCM construction: the gray level symbiosis relationship between any two points in the image is calculated to construct GLCM; texture feature extraction: contrast, energy, homogeneity, correlation and other texture features are extracted from GLCM. Texture analysis can also adopt a pyramid decomposition method to perform texture analysis on sub-images. The pyramid decomposition methods include Gaussian pyramid and Laplace pyramid. Gaussian pyramid: the image is decomposed into low-frequency information of different scales through Gaussian smoothing and downsampling; Laplace pyramid: high-frequency information of different scales is obtained through Gaussian smoothing, downsampling, upsampling and Gaussian blurring.

[0060] Furthermore, the fusion of acoustic image and optical image in the data augmentation includes image registration and image fusion, and the local feature enhancement in the data augmentation also includes feature extraction network, feature fusion and loss function design, image registration: adopting feature point-based image registration algorithm to accurately align the acoustic image and the optical image; image fusion: adopting weighted average method or maximum method and other methods to fuse the acoustic image and the optical image to generate a fused image, feature extraction network: constructing a deep convolutional neural network to extract local features of the image; feature fusion: fusing the extracted local features with the original image to generate an enhanced image; loss function design: designing a composite loss function including edge loss, texture loss and reconstruction loss to guide model training and optimization.

[0061] Furthermore, the geometric transformation in the data augmentation includes rotation transformation, flip transformation and scaling transformation. The rotation transformation is to rotate the image around the center point by a certain angle to generate a new image. The rotation angle is randomly selected between [-15, 15] degrees. The formula of the rotation transformation is:

[0062] I′(x, y)=I(x′, y′)

[0063] Among them, x′=(x-ω / 2)cosθ+(yh / 2)sinθ+w / 2;

[0064] y′=(yh / 2)cosθ-(x-ω / 2)sinθ+h / 2,

[0065] Where I(x,y) represents the original image, I′(x,y) represents the rotated image, ω and h represent the width and height of the image respectively, and θ represents the rotation angle;

[0066] The flip transformation is to flip the image horizontally or vertically to generate a new image. The formula for the flip transformation is:

[0067] I′(x,y)=I(wx-1,y) (horizontal flip);

[0068] I′(x,y)=I(x,hy-1)(vertical flip);

[0069] The scaling transformation is to scale the image to generate a new image. The scaling ratio is randomly selected between [0.8, 1.2]. The scaling transformation formula is:

[0070] I′(x,y)=I(x′,y′);

[0071] Where x′=x2scale;

[0072] y′=y·scale;

[0073] Among them, scale represents the scaling ratio.

[0074] Furthermore, the model training and optimization include model selection, data augmentation, distributed training and optimization algorithms. The model selection selects convolutional neural network (CNN) as the feature extraction and classification model. CNN has powerful feature extraction capabilities and can automatically learn deep-level features in images. Data augmentation technology is used to expand the data set to avoid model overfitting. Data augmentation technology includes operations such as rotation, flipping, and scaling. For underwater images, noise, blur, etc. can also be added to simulate the impact of underwater environment. Since underwater image data sets are usually large, distributed training strategies are used to improve computing efficiency and speed. Distributed training includes two methods: data parallelism and model parallelism. Data parallelism divides the training data into several batches and synchronously trains the same model copies on different computing nodes. Model parallelism distributes different parts of the model to different nodes for calculation. Optimization algorithm: The Adam optimization algorithm is used for model training. The Adam algorithm combines the advantages of the momentum method and the RMSProp algorithm, and can converge quickly and find the global optimal solution. At the same time, the learning rate scheduling strategy is used to dynamically adjust the learning rate to avoid skipping the optimal point too quickly in the early stage of training or slow convergence in the later stage of training.

[0075] Furthermore, the steps of the CNN algorithm include input data, data enhancement and normalization. The input data is usually image data, which can be a grayscale image (single channel) or a color image (multi-channel, such as RGB three channels). Data enhancement: in order to improve the generalization ability of the model, the input data can be enhanced, such as rotation, scaling, translation, flipping, etc. Normalization: normalize the pixel values ​​of the input data to the same range, such as between 0 and 1, to eliminate the brightness differences between different images.

[0076] Furthermore, the color gamut transformation in the data augmentation includes brightness transformation, contrast transformation and saturation transformation. The brightness transformation is to adjust the brightness of the image to generate a new image. The brightness variation range is randomly selected between [0.5, 1.5]. The formula for brightness transformation is:

[0077] I′(x,y)=α·I(x,y);

[0078] Among them, α represents the brightness change coefficient;

[0079] The contrast transformation is to adjust the contrast of the image to generate a new image. The contrast variation range is randomly selected between [0.5, 1.5]. The contrast transformation can be achieved by linear transformation. The formula is:

[0080] I′(x,y)=β·(I(x,y)-μ)+μ;

[0081] Among them, β represents the contrast variation coefficient, and μ represents the mean of the image;

[0082] The saturation transformation adjusts the saturation of the image to generate a new image. The saturation change range is randomly selected between [0.5, 1.5]. The saturation transformation can be performed in the HSV color space. The formula is:

[0083] S'(x,y)=γ·S(x,y);

[0084] Among them, S(x, y) represents the saturation of the original image, S'(x, y) represents the adjusted saturation, and γ represents the saturation change coefficient.

[0085] Furthermore, in the data augmentation, the definition transformation performs sharpening and blurring processing on the image to generate a new image. The sharpening can be achieved by a high-contrast edge filter, and the blurring can be achieved by a Gaussian filter.

[0086] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for data augmentation of acoustic optical images based on local features of the image, characterized in that: The steps include: S1. Data preparation: Select acoustic and optical images at different depths and water quality conditions for experiments; S2, local feature extraction, edge detection, texture analysis, scale space construction and key point detection of the prepared image; S3, data augmentation, which fuses the acoustic image and the optical image and enhances their local features. After extracting the local features of the image, geometric transformation, color gamut transformation and clarity transformation are performed; S4, model training and optimization, bringing the augmented data into the training model for training and optimization; S5. Experimental Results and Analysis,Through the local feature enhancement processing of the deep learning,algorithm, the image quality and recognition accuracy are further improved.

2. The method for data augmentation of acoustic optical images based on local features of an image according to claim 1, characterized in that: The edge detection in the local feature extraction adopts the CNN algorithm. The CNN algorithm extracts the features of the image through convolution operation. In edge detection, the convolution layer of CNN is used to capture the edge information in the image. The formula is as follows: y(i,j)=(x*ω)(i,j)=∑ m ∑ n x(i+m,j+n)ω(m,n)。 3. The method for data augmentation of acoustic optical images based on local features of an image according to claim 1, characterized in that: The texture analysis in the local feature extraction uses a texture analysis method based on a gray level co-occurrence matrix to perform texture analysis on underwater images. The scale space is constructed in the local feature extraction to generate images of different scales through Gaussian blurring and downsampling. The key points, i.e., stable feature points in images of different scales, are detected in the local feature extraction, and the directional gradient histogram of the key points is calculated to generate a feature descriptor.

4. The method for data augmentation of acoustic optical images based on local features of an image according to claim 1, characterized in that: The fusion of acoustic images and optical images in the data augmentation includes image registration and image fusion, and the local feature enhancement in the data augmentation also includes feature extraction network, feature fusion and loss function design.

5. The method for data augmentation of acoustic optical images based on local features of an image according to claim 1, characterized in that: The geometric transformations in the data augmentation include rotation transformation, flip transformation and scaling transformation. The rotation transformation is to rotate the image around the center point by a certain angle to generate a new image. The rotation angle is randomly selected between [-15, 15] degrees. The formula for the rotation transformation is: I′(x, y)=I(x′, y′) Among them, x′=(x-ω / 2)cosθ+(yh / 2)sinθ+w / 2; y′=(yh / 2)cosθ-(x-ω / 2)sinθ+h / 2, Where I(x,y) represents the original image, I′(x,y) represents the rotated image, ω and h represent the width and height of the image respectively, and θ represents the rotation angle; The flip transformation is to flip the image horizontally or vertically to generate a new image. The formula for the flip transformation is: I′(x,y)=I(wx-1,y) (horizontal flip); I′(x,y)=I(x,hy-1)(vertical flip); The scaling transformation is to scale the image to generate a new image. The scaling ratio is randomly selected between [0.8, 1.2]. The scaling transformation formula is: I′(x,y)=I(x′,y′); Where x′ = x·scale; y′=y·scale; Among them, scale represents the scaling ratio.

6. The method for data augmentation of acoustic optical images based on local features of an image according to claim 1, characterized in that: The model training and optimization include model selection, data augmentation, distributed training and optimization algorithms.

7. The method for data augmentation of acoustic optical images based on local features of an image according to claim 1, characterized in that: The steps of the CNN algorithm include input data, data enhancement and normalization.

8. The method for data augmentation of acoustic optical images based on local features of images according to claim 1, characterized in that: The color gamut transformation in the data augmentation includes brightness transformation, contrast transformation and saturation transformation. The brightness transformation is to adjust the brightness of the image to generate a new image. The brightness variation range is randomly selected between [0.5, 1.5]. The formula for brightness transformation is: I′(x,y)=α·I(x,y); Among them, α represents the brightness change coefficient; The contrast transformation is to adjust the contrast of the image to generate a new image. The contrast variation range is randomly selected between [0.5, 1.5]. The contrast transformation can be achieved by linear transformation. The formula is: I′(x,y)=β·(I(x,y)-μ)+μ; Among them, β represents the contrast variation coefficient, and μ represents the mean value of the image; The saturation transformation adjusts the saturation of the image to generate a new image. The saturation change range is randomly selected between [0.5, 1.5]. The saturation transformation can be performed in the HSV color space. The formula is: S'(x,y)=γ·S(x,y); Among them, S(x, y) represents the saturation of the original image, S'(x, y) represents the adjusted saturation, and γ represents the saturation change coefficient.

9. The method for data augmentation of acoustic optical images based on local features of an image according to claim 1, characterized in that: In the data augmentation, the definition transformation performs sharpening and blurring processing on the image to generate a new image. The sharpening can be achieved by a high-contrast edge filter, and the blurring can be achieved by a Gaussian filter.

Images