Acoustic intensity and image coupling method for side-scan sonar target recognition

Abstract

The present invention relates to the field of seabed detection technology, in particular to a method for coupling sound intensity and image based on side-scan sonar target recognition. The method comprises the following steps: S1, using a sound intensity feature extraction network to perform feature analysis and extraction on the sound intensity domain, and using an image feature extraction network to perform feature analysis and extraction on the image domain; S2, using channel feature representation to find the potential relationship between the two domains, the sound intensity domain includes a sound intensity channel and an information fusion channel, the image domain includes an image channel and an information fusion channel, the two types of channels in the same domain share the same preliminary feature extraction, the information fusion channels in the two domains are fused to find a common representation, and the sound intensity channel and the image channel extract their own feature representations; S3, outputting sound intensity recognition, image recognition and coupled recognition. It can identify a variety of marine target types that are difficult to distinguish, couple the sound intensity with the image, and improve the marine target recognition effect.

Description

Technical Field

[0001] The invention relates to the technical field of seabed detection, and in particular to a sound intensity and image coupling method based on side-scan sonar target recognition. Background Art

[0002] Side-scan sonar has the advantages of high detection efficiency, wide scanning range, and unlimited depth of ocean investigation. It is widely used as the main sensor for marine equipment and is an important tool for exploring deep and distant seas. Marine equipment equipped with side-scan sonar plays an increasingly important role in civil and military fields. Its online side-scan sonar information recognition method enables marine equipment to have autonomous capabilities and can be driven autonomously to carry out marine missions without human intervention.

[0003] Traditional side-scan sonar information recognition is mostly focused on image processing, but the pixels of the image are converted from the original sound intensity information, and some effective information will be lost during the conversion process. Therefore, image processing lacks bottom-end sound intensity information. Sound intensity information can only reflect the fluctuation phenomenon of sound wave intensity. If the side-scan sonar detects an interference object similar to a target, its sound intensity fluctuation cannot reflect the difference between the target and the interference object. At this time, the imaging can be identified and distinguished based on information such as texture features. Single image recognition or sound signal recognition has the disadvantages of low recognition rate and missing effective information. If the sound intensity can be coupled with the image, the advantages can be complementary and the accuracy of ocean detection and recognition can be improved. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned defects of the prior art and propose a sound intensity and image coupling method based on side-scan sonar target recognition, which can identify a variety of marine target types that are difficult to distinguish, couple the sound intensity with the image, and improve the marine target recognition effect.

[0005] The technical solution of the present invention is: a method for coupling sound intensity and image based on side-scan sonar target recognition, which includes the following steps:

[0006] S1, using the sound intensity feature extraction network to perform feature analysis and extraction on the sound intensity domain, and using the image feature extraction network to perform feature analysis and extraction on the image domain;

[0007] S2. Use channel feature representation to find the potential relationship between the two domains. The sound intensity domain contains the sound intensity channel and the information fusion channel, and the image domain contains the image channel and the information fusion channel. The two types of channels in the same domain share the same preliminary feature extraction. The information fusion channels in the two domains are fused to find a common representation, and the sound intensity channel and the image channel extract their own feature representations.

[0008] S3, output sound intensity recognition, image recognition and coupling recognition.

[0009] In the present invention, in step S1,

[0010] During the detection process of the side scan sonar sensor, when a target object with sound intensity information inconsistent with the background information is detected, the sound intensity information will fluctuate;

[0011] According to the relationship between sound intensity and pixels, the sound intensity is used as the input of the sound intensity domain, and the pixels are used as the input of the image domain.

[0012] In step S1,

[0013] The sound intensity feature extraction network is used to perform feature analysis and extraction to obtain the sound intensity information S; the image feature extraction network is used to perform feature analysis and extraction to obtain the acoustic image information I;

[0014] Let the training set be , the data form of S is , the data format of I is . Among them, h in the dataset S is the channel of the sound intensity information; p is the sampling point of the sound intensity information; c is the number of channels of the input acoustic image; h' in the dataset I represents the width of the image, which is equal to the sound intensity information channel h; w is the length of the image;

[0015] The input S and I data are first subjected to a downsampling convolution operation:

[0016] ,

[0017] ,

[0018] The sound intensity feature extraction network is used and image feature extraction network Perform feature analysis and extraction.

[0019] Sound intensity feature extraction network The output is as follows:

[0020] ;

[0021] Sound intensity feature extraction network The specific expansion steps are as follows:

[0022] Reshape S to get B, C, D:

[0023] ,

[0024] Perform matrix multiplication and softmax processing on B and C to get E:

[0025] ,

[0026] The final enhanced result is G:

[0027] ,

[0028] Finally, after the upsampling convolution operation:

[0029] .

[0030] Image feature extraction network The output is as follows:

[0031] ;

[0032] Image feature extraction network The specific expansion steps are as follows:

[0033] I will pass and Different feature maps obtained by convolution kernel processing and , and add them together to get M:

[0034] ,

[0035] Pooling M:

[0036] ,

[0037] Among them, H and W are the length and width of the feature map M respectively;

[0038] Then do the following:

[0039] ,

[0040] ,

[0041] in, For normalization, is a full connection operation, and Z is the full connection result; is softmax processing, A is the weight;

[0042] The feature map and Multiply them by weight A and add them together to get Y:

[0043] ,

[0044] Y is the representation of the image domains of different scales obtained;

[0045] Finally, after the upsampling convolution operation:

[0046] .

[0047] In step S2,

[0048] The sound intensity channel is expressed as follows:

[0049] ,

[0050] Image channels are represented as follows:

[0051] ,

[0052] in, and The convolution kernel is Convolution operation; and Represent the parameters of the sound intensity domain and image domain respectively;

[0053] The information fusion channel is expressed as follows:

[0054] ,

[0055] ,

[0056] in, The convolution kernel is The convolution operation, is the fusion parameter of the two domains.

[0057] In step S3,

[0058] The sound intensity channel Information fusion channel Data combined with output sound intensity domain recognition :

[0059] ;

[0060] Image Channel Information fusion channel Data combined with output image domain recognition :

[0061] ;

[0062] right and Pooling is performed separately to obtain and , then combine and output coupling identification O:

[0063] ,

[0064] Among them, H' and W' represent and The dimensions of the space are length and width;

[0065] The recognition result output P is , , O jointly determine:

[0066] ,

[0067] in, It indicates the accuracy of the three recognitions.

[0068] When P is greater than the threshold value of 0.5, it is determined that the target type is identified; otherwise, the target is not identified.

[0069] The beneficial effects of the present invention are:

[0070] (1) Coupling sound intensity with images weakens the dependence on single sound intensity recognition or image recognition, avoids the impact of weak sound intensity recognition perception and image recognition information loss, achieves complementary advantages, and helps solve the current problems of poor results in marine target recognition;

[0071] (2) Efficient identification can be performed when the target type is uncertain. For example, reefs and offshore engineering foundations with almost identical sound intensity information can be identified by coupling with image recognition through information such as texture and shape, or reefs and metal objects that are similar in imaging, such as pre-placed weapons. When image recognition is difficult to distinguish, coupled identification is performed through sound intensity information. Therefore, the method described in this application can identify a variety of marine target types that are difficult to distinguish, such as mines, reefs, underwater cables, fishing nets, etc., to assist marine equipment in performing operations such as effective obstacle avoidance and efficient strikes. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a schematic diagram of the principle of the present invention;

[0073] Figure 2 It is a schematic diagram of the principles of the sound intensity feature extraction network and the image feature extraction network. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0075] In the following description, specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in a variety of other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0076] The present invention discloses a method for coupling sound intensity and image based on side-scan sonar target recognition. During the detection process of the side-scan sonar sensor, when a target object whose sound intensity information is inconsistent with the background information is detected, the sound intensity information will fluctuate. At the same time, according to the relationship between sound intensity and image, the information of the two is used as input in the sound intensity domain and the image domain respectively.

[0077] like Figure 1 As shown in the figure, first, the sound intensity feature extraction network and image feature extraction network are used for feature analysis and extraction respectively. Since the sound intensity information and image information are represented in different forms, the sound intensity information focuses on feature attention, and the acoustic image focuses on feature enhancement. Secondly, the channel feature representation is used to find the potential relationship between the two domains. The sound intensity domain contains the sound intensity channel and the information fusion channel, and the image domain contains the image channel and the information fusion channel. The two types of channels in the same domain share the same preliminary feature extraction. The information fusion channels in the two domains are fused to find a common representation, and the sound intensity channel and the image channel extract their own feature representations. Finally, the sound intensity recognition, image recognition and coupled recognition are output.

[0078] The method described in the present invention comprises the following specific steps.

[0079] The first step is to extract the sound intensity information and acoustic image features.

[0080] For the sound intensity information S, the sound intensity feature extraction network is used for feature extraction; for the acoustic image information I, the image feature extraction network is used for feature extraction.

[0081] Let the training set be , the data form of S is , the data format of I is Among them, h in the dataset S is the channel of the sound intensity information; p is the sampling point of the sound intensity information; c is the number of channels of the input acoustic image; h' in the dataset I represents the width of the image, which is equal to the sound intensity information channel h; and w is the length of the image.

[0082] The input S and I data are first subjected to a downsampling convolution operation:

[0083] ,

[0084] ,

[0085] The sound intensity feature extraction network is used and image feature extraction network Perform feature analysis and extraction. Figure 2 shown.

[0086] Sound intensity feature extraction network The overall output is as follows:

[0087] .

[0088] The specific expansion steps are described as follows.

[0089] Reshape S to get B, C, D:

[0090] ,

[0091] Perform matrix multiplication and softmax processing on B and C to get E:

[0092] ,

[0093] The final enhanced result is G:

[0094] ,

[0095] This result not only contains shallow features but also retains more detailed information, improving the feature attention of sound intensity information.

[0096] Finally, after the upsampling convolution operation:

[0097] .

[0098] Image feature extraction network The overall output is as follows:

[0099] .

[0100] The specific expansion steps are as follows:

[0101] I will pass and Different feature maps obtained by convolution kernel processing and , and add them together to get M:

[0102] ,

[0103] Pooling M:

[0104] ,

[0105] Among them, H and W are the length and width of the feature map M respectively.

[0106] Then do the following:

[0107] ,

[0108] ,

[0109] in, For normalization, is a full connection operation, and Z is the full connection result; is softmax processing, and A is the weight.

[0110] The feature map and Multiply them by weight A and add them together to get Y:

[0111] ,

[0112] Y is the representation of the image domains of different scales.

[0113] Finally, after the upsampling convolution operation:

[0114] .

[0115] The second step is channel feature representation.

[0116] The feature extraction of sound intensity information and acoustic images needs to be projected into different channels for feature representation of different channels.

[0117] The sound intensity channel is expressed as follows:

[0118] ,

[0119] Image channels are represented as follows:

[0120] ,

[0121] in, and The convolution kernel is Convolution operation; and Represent the parameters of the sound intensity domain and image domain respectively.

[0122] The information fusion channel is expressed as follows:

[0123] ,

[0124] ,

[0125] in, The convolution kernel is The convolution operation, is the fusion parameter of the two domains.

[0126] The third step is to identify the output.

[0127] Combine the sound intensity channel with the information fusion channel data to output the sound intensity domain recognition :

[0128] .

[0129] Combine image channel and information fusion channel data to output image domain recognition :

[0130] .

[0131] right and Pooling is performed separately to obtain and , then combine and output coupling identification O:

[0132] ,

[0133] Among them, H' and W' represent and The dimensions of the space are length and width.

[0134] The recognition result output P is , , O jointly determine:

[0135] ,

[0136] in, It indicates the accuracy of the three recognitions.

[0137] When P is greater than the threshold of 0.5, the target type is identified; otherwise, the target is not identified. This judgment weakens the dependence on single sound intensity recognition or image recognition. The design of a coupled recognition method and the joint judgment of the two avoids the impact of weak sound intensity recognition perception and image recognition information loss, and increases the accuracy of recognition.

[0138] The above is a detailed introduction to the sound intensity and image coupling method based on side scan sonar target identification provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined in this article can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in this article, but will conform to the widest range consistent with the principles and novel features disclosed in this article.

Claims

1. A method for coupling sound intensity and image based on side-scan sonar target recognition, characterized in that: The following steps are involved: S1, using the sound intensity feature extraction network to perform feature analysis and extraction on the sound intensity domain, and using the image feature extraction network to perform feature analysis and extraction on the image domain; S2. Use channel feature representation to find the potential relationship between the two domains. The sound intensity domain contains the sound intensity channel and the information fusion channel, and the image domain contains the image channel and the information fusion channel. The two types of channels in the same domain share the same preliminary feature extraction. The information fusion channels in the two domains are fused to find a common representation, and the sound intensity channel and the image channel extract their own feature representations. S3, output sound intensity recognition, image recognition and coupling recognition; In step S1, a sound intensity feature extraction network is used to perform feature analysis and extraction to obtain sound intensity information S; an image feature extraction network is used to perform feature analysis and extraction to obtain acoustic image information I; Assume that the training set is {S, I}, the data form of S is h×p, and the data form of I is c×h'×w, where h in the data set S belongs to the channel of the sound intensity information; p is the sampling point of the sound intensity information; c is the number of channels of the input acoustic image; h' in the data set I represents the width of the image, which is equal to the sound intensity information channel h; w is the length of the image; The input S and I data are first subjected to a downsampling convolution operation: S = conv(S), I = conv(I), The sound intensity feature extraction network N S (·) and image feature extraction network N I (·) Perform feature analysis and extraction; Sound intensity feature extraction network N S The output of (·) is as follows: Sound intensity feature extraction network N S The specific expansion steps of (·) are as follows: Reshape S to get B, C, D: B=C=D=reshape(S), Perform matrix multiplication and softmax processing on B and C to get E: The final enhanced result is G: Finally, after the upsampling convolution operation: Image feature extraction network N I The output of (·) is as follows: Image feature extraction network N I The specific expansion steps of (·) are as follows: Process I with 3×3 and 5×5 convolution kernels to obtain different feature maps M1 and M2, and add them together to get M: M=M1+M2, Pooling M: Among them, H and W are the length and width of the feature map M respectively; Then do the following: Z = δ(η(P)), A=φ(Z), Among them, η(·) is the normalization process, δ(·) is the full connection operation, and Z is the full connection result; φ(·) is the softmax process, and A is the weight; Multiply the feature maps M1 and M2 by the weight A respectively and then add them together to get Y: Y=Y1+Y2=AM1+AM2, Y is the representation of the image domains of different scales obtained; Finally, after the upsampling convolution operation: In step S2, the sound intensity channel is represented as follows: Image channels are represented as follows: Among them, C S (·) and C I (·) is the convolution operation with a convolution kernel of 1×1; θ S and θ I Represent the parameters of the sound intensity domain and image domain respectively; The information fusion channel is expressed as follows: Among them, F (S,I) (·) is the convolution operation with a convolution kernel of 1×1, θ F is the fusion parameter of the two domains; in step S3, The sound intensity channel S C With the information fusion channel S F Data combined with output sound intensity domain identification S FC : Set image channel I C With the information fusion channel S F Data combined with output image domain recognition I FC : For S FC and I FC Pooling is performed separately to obtain S with a common spatial size FC 'and I FC ', then combine and output coupling identification O: Among them, H' and W' represent S FC 'and I FC 'The spatial dimensions are length and width; The recognition result output P' is S FC ,I FC , O jointly determine: Among them, PA(·) represents the accuracy of the three recognitions.

2. The method for coupling sound intensity and image based on side scan sonar target recognition according to claim 1, characterized in that: In step S1, During the detection process of the side scan sonar sensor, when a target object with sound intensity information inconsistent with the background information is detected, the sound intensity information will fluctuate; According to the relationship between sound intensity and pixels, the sound intensity is used as the input of the sound intensity domain, and the pixels are used as the input of the image domain.

3. The method for coupling sound intensity and image based on side scan sonar target recognition according to claim 1, characterized in that: When P' is greater than the threshold value of 0.5, it is determined that the target type is identified; Otherwise, the target object is not recognized.

Images