Underwater target recognition method based on active deep learning

Abstract

The present invention discloses an underwater target recognition method based on active deep learning, comprising the following steps: 1. Preprocessing the original data of sonar signals and dividing them into a candidate set and a test set; 2. Selecting a part of data from the candidate set divided in step 1, and moving it from the candidate set to the training set after annotation as initial training data; 3. Using the data in the training set to train a CNN model, and recording the model performance when the training is completed; 4. In the remaining candidate sets, according to the active learning strategy, selecting a certain amount of data, and moving it to the training set after annotation; 5. Repeating steps 3 and 4 until all the data in the candidate set are added to the training set; 6. After the model training is completed, according to the model performance recorded in each step, the relationship between the model performance and the amount of training data is obtained. The present invention has the effect of significantly reducing the data annotation cost and improving the data annotation efficiency in related tasks.

Description

Technical Field

[0001] The invention relates to an underwater target recognition method based on active deep learning, and belongs to the intersecting field of digital signal processing and oceanography. Background Art

[0002] Sonar echoes can be used to quickly and conveniently determine the location and type of underwater objects, and can also be used to communicate with other ships. Sonar systems are now widely used by military and civilian ships. In the military field, ships and submarines equipped with sonars can quickly and effectively identify enemy ships and improve naval combat capabilities in conjunction with underwater target detection technology; in the civilian field, sonars are widely used in deep-sea fishing, archaeological operations, shipwreck salvage, and other work to improve work efficiency.

[0003] In practical applications, due to the complexity of the marine environment, changes in operating conditions, and the presence of various noises, it is very challenging to improve the performance and efficiency of recognition algorithms. Traditional classification and recognition algorithms mostly use mathematical modeling or artificial feature construction methods. Such methods have weak generalization performance in complex marine environments, and their data processing efficiency is difficult to meet requirements. With the rapid development of deep learning technology, the application of neural networks in underwater target recognition has also achieved good results. In the task of identifying and classifying sonar echoes and sonar images, compared with traditional recognition algorithms, it not only has higher accuracy and better generalization performance, but also greatly improves data processing efficiency.

[0004] In the field of underwater target recognition, it is often difficult to obtain enough high-quality labeled training data. The labeling of sonar signal data usually requires a lot of professional knowledge and a lot of training experience, and the time cost of data labeling is also high, so the cost of manual labeling of sonar data is very high. Problems such as high data labeling costs or low data labeling efficiency often become bottlenecks in the development of related research.

[0005] Because of the above problems, active learning technology has been introduced into the field of underwater target recognition. Active learning technology refers to a method in which, in the process of machine learning, a part of the unlabeled data set is iteratively selected through a specific sampling strategy to be labeled by experts, and then the labeled data is added to the training set for training and updating the model. Due to the existence of sampling strategies in active learning, the data selected for experts to label are data with high uncertainty and large amount of information, thus greatly improving the performance of the current model. Summary of the invention

[0006] In view of the problems of high data annotation cost and low efficiency in the field of underwater target recognition, the present invention proposes an underwater target recognition method based on active deep learning. By combining the active learning sample selection strategy with the training of the deep learning model, the data that can greatly improve the model performance is selected from the unlabeled data, and this part of the data is labeled first and used for model training, which can quickly improve the performance of the model, thereby reducing the data annotation workload of experts in related fields and improving data annotation efficiency.

[0007] The above purpose is achieved through the following technical methods:

[0008] An underwater target recognition method based on active deep learning, the method comprising the following steps:

[0009] Step 1: Preprocess the original data of underwater acoustic signals and divide them into a candidate set and a test set, which are mutually exclusive. Establish a training set that is an empty set in the initial state.

[0010] Step 2: Select a portion of data from the candidate set divided in step 1, obtain the annotations and move it from the candidate set to the training set as the initial training data.

[0011] Step 3: Train the CNN model using the data in the training set and record the model performance when training is complete.

[0012] Step 4: In the remaining candidate sets, according to the active learning strategy, select a certain amount of data, obtain annotations and move them to the training set.

[0013] Step 5: Repeat steps 3 and 4 until all the data in the candidate set are added to the training set.

[0014] Step 6: After model training is completed, based on the model performance recorded at each step, the changing relationship between model performance and the increase in the amount of training data is obtained.

[0015] In the above-mentioned underwater target recognition method based on active deep learning, the raw data of the underwater acoustic signal is preprocessed in step 1, and the data preprocessing method includes: pre-emphasizing the high-frequency part of the acoustic signal, dividing the longer signal into shorter segments, and adding windows to make the edge of the frame smoothly transition to 0.

[0016] In the above-mentioned underwater target recognition method based on active deep learning, the step 1 of dividing the water acoustic signal data into mutually exclusive candidate sets and test sets is as follows: 20% of all pre-processed water acoustic signal data are randomly selected as the test set, and the rest are used as the candidate set. The data in the test set are annotated as a criterion for judging the performance of the CNN model.

[0017] In the above-mentioned underwater target recognition method based on active deep learning, the CNN model is trained using the data in the training set in step 3. The structure of the CNN model is shown in Table 1, which includes five feature extraction modules. The first three modules are a structure of one convolution layer + one pooling layer, and the last two modules are a structure of two convolution layers + one pooling layer. Then three fully connected layers are connected, among which the first fully connected layer uses Dropout to randomly discard the output of a part of the neurons to reduce overfitting; the last fully connected layer uses Softmax classification output. In addition, the experiment in this paper processes one-dimensional sonar echo data, so the feature map, convolution kernel and pooling kernel in the network are adjusted to a one-dimensional structure.

[0018]

[0019]

[0020] Table 1 Convolutional neural network structure

[0021] In the above-mentioned underwater target recognition method based on active deep learning, a certain amount of data is selected according to the active learning strategy in step 4, wherein the active learning sample selection strategy includes: "uncertainty standard" active learning strategy, and "uncertainty + diversity standard" active learning strategy.

[0022] In the underwater target recognition method based on active deep learning, the specific steps of the "uncertainty standard" active learning sample selection strategy are to select the most ambiguous samples judged by the model from the unlabeled samples, that is, the samples with the lowest confidence of the model in the category to which it belongs. For each sample x, the probability of the two most likely predictions output by the classifier is used to calculate its uncertainty, that is, unc(x):

[0023]

[0024] In the formula, p(y1|x) and p(y2|x) represent the probabilities of the two most likely and second most likely categories of sample x in the classifier output, respectively. The larger the value of unc(x), the higher the confidence of the model in the most likely sample, and the less likely the classifier will misclassify the sample. Therefore, the active learner selects the sample with the smallest unc(x) value as the sample with the greatest uncertainty, sends it for annotation and adds it to the training set.

[0025] In the underwater target recognition method based on active deep learning, the specific steps of the "uncertainty + diversity standard" active learning sample selection strategy are to use distance measurement to judge the diversity of samples on the basis of the "uncertainty standard", and then consider uncertainty and diversity comprehensively to select data with higher uncertainty and greater sample diversity as the selection result. Use Gaussian kernel distance to measure the distance between two samples xi and x j The similarity between:

[0026]

[0027] Wherein, σ is the Gaussian kernel parameter, which is set to 1 in the present invention.

[0028] When calculating the similarity of sample x to a set Z consisting of samples, the distance between sample x and the sample in set Z with the greatest similarity to it is expressed as:

[0029]

[0030] The smaller the sim(x,Z) value is, the smaller the similarity between sample x and set Z is, that is, the smaller the redundancy between sample x and the existing elements in set Z is, so the diversity of the selected samples is greater.

[0031] Using the active learning strategy of “uncertainty + diversity criterion” to select n samples for training, we first use the “uncertainty criterion” to select m samples with larger uncertainty to form a set M = {x1, x2, …, x m}, and then select the n best samples from M using the method that comprehensively considers uncertainty and diversity as shown in the following formula:

[0032] score(x)=λunc(x)+(1-λ)sim(x,M)

[0033] Wherein, λ is a weighing parameter used to measure the importance of uncertainty and diversity, and is set to 0.8 in the present invention. The sample with the smallest score(x) finally calculated is the sample with large uncertainty and diversity, and has the greatest training value.

[0034] Beneficial effects:

[0035] 1. The present invention constructs a framework that integrates active learning strategies into the deep learning model training process in an iterative manner. That is, instead of using active learning strategies to select part of the data at one time and observe the performance of the model after training with this part of the data, data screening is performed less frequently and multiple times, and the performance changes of the model are observed when the training data increases. The advantages brought by such an integrated approach include: the experimental results do not depend on the performance of the model in the initial state; and the results of each data selection and model training can be recorded for easy comparative analysis.

[0036] 2. Compared with the learning method that does not select samples, the two active learning sample selection strategies of the present invention can significantly reduce the cost of data annotation. Only less data needs to be annotated to make the performance of the deep learning model reach the effect of training with all data. Specifically, when using the "uncertainty standard" active learning strategy, only 49% of the training data is needed to converge the model and achieve the same 97% validation set accuracy as when all data is used for training. When using the "uncertainty + diversity standard" active learning strategy, the amount of data required to achieve model convergence is basically the same as the "uncertainty standard", but in the early stage of model training, the effect of accelerating the performance improvement speed is more obvious, and the accuracy rate of 95% can be achieved faster, and the training process is smoother. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the overall framework of the present invention, wherein the ellipse represents data or output, and the rectangle represents the framework or module.

[0038] Figure 2 This is a schematic diagram of the CNN structure adopted in the present invention.

[0039] Figure 3 Schematic diagram of the algorithm flow of the present invention.

[0040] Figure 4 The trends of the accuracy of the three sample selection strategies as the amount of training data changes.

[0041] Figure 5 The trend of the test set loss for all categories for the three sample selection strategies as the amount of training data changes. DETAILED DESCRIPTION

[0042] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0043] Figure 1 The overall framework of the underwater target recognition method based on active deep learning proposed by the present invention is shown. In the schematic diagram, the ellipse represents data or output, and the rectangle represents the framework or module. This framework mainly includes three modules: data preprocessing module, active learning module, and deep learning module. Each module is described in detail below.

[0044] 1. Data Preprocessing Module

[0045] The data preprocessing module performs preprocessing operations on the original data to generate a candidate set, that is, unlabeled data. The preprocessing operation is divided into three steps: pre-emphasis, framing, and windowing. The processing details are as follows:

[0046] The purpose of pre-emphasis is to improve the problem of uneven distribution of high and low frequencies of underwater sound signals. The energy of underwater sound signals in the low frequency part is large, but insufficient in the high frequency part. The sound signal generally contains more useful information in the high frequency part, so it is necessary to appropriately enhance its high frequency part, which is generally achieved by adding a first-order FIR high-pass digital filter to the original signal:

[0047] H(z)=1-az -1

[0048] Convert this to the difference equation form:

[0049] y(n)=x(n)-ax(n-1)

[0050] The value of a in the above formula is usually between 0.9 and 1, and is set to 0.95 in the present invention.

[0051] Underwater acoustic signals have time-varying characteristics, but can remain relatively stable within a shorter time range. Therefore, a section of acoustic signals in a short period of time can be regarded as a quasi-steady-state process. When analyzing and processing the acoustic signal, it is necessary to divide it into frames of equal length, that is, frame processing, and the length of each frame is called the frame length. After a longer acoustic signal data is framed, each frame is processed and analyzed one by one, which is a commonly used method in acoustic signal processing. If the acoustic signal is cut directly, the data information in a short period of time between the two frames will be lost, and the continuity of the frame will be destroyed. Therefore, when framing, two adjacent frames are usually overlapped, and the length of the overlapping part is called frame shift. The present invention uses the API provided by the Librosa library to perform frame processing on the underwater acoustic signal, taking a frame length of 2048 and a frame shift of 512.

[0052] After the acoustic signal is divided into frames, there is a sudden change in the signal value at the edge of each frame. In order to make the edge of the frame transition smoothly to 0, it is usually necessary to perform windowing on each frame, that is, multiply the original signal with a fixed-length window function and then output it. The most common window function is the Hann window, which is defined as:

[0053]

[0054] In the present invention, the API provided by the Librosa library is used to window the acoustic signal, using a Hann window, with the window length being equal to the frame length, i.e., 2048.

[0055] In the experimental process of the present invention, the raw data after preprocessing is shown in Table 2, which is divided into 5 categories:

[0056]

[0057] Table 2 Data set data distribution information

[0058] After the above data preprocessing steps, a total of 35,702 data samples were generated. 20% of them, about 7,142 samples, were randomly selected and manually labeled as the test set for the deep learning model. The remaining 28,560 unlabeled samples were kept as a candidate set.

[0059] 2. Active Learning Module

[0060] The active learning module selects samples with higher training value from the candidate set, obtains the annotations of these samples, and moves them from the candidate set to the training set. The input of the active learning module, in addition to the samples of the candidate set, also includes the output of the deep learning module. The reason is that in the active learning sample selection strategy, the quantitative calculation of the training value of the candidate set samples requires the participation of the deep learning module, including prediction or extraction of effective information. The active learning sample selection strategy in the present invention includes: "uncertainty standard" active learning strategy, and "uncertainty + diversity standard" active learning strategy.

[0061] The specific steps of the "uncertainty criterion" active learning sample selection strategy are to select the most ambiguous samples from the unlabeled samples, that is, the samples with the lowest confidence in the model's category. For each sample x, the probability of the two most likely predictions output by the classifier is used to calculate its uncertainty, that is, unc(x):

[0062]

[0063] In the formula, p(y1|x) and p(y2|x) represent the probabilities of the two most likely and second most likely categories of sample x in the classifier output, respectively. The larger the value of unc(x), the higher the confidence of the model in the most likely sample, and the less likely the classifier will misclassify the sample. Therefore, the active learner selects the sample with the smallest unc(x) value as the sample with the greatest uncertainty, sends it for annotation and adds it to the training set.

[0064] The specific steps of the "uncertainty + diversity standard" active learning sample selection strategy are to use distance measurement to judge the diversity of samples based on the "uncertainty standard", and then consider uncertainty and diversity comprehensively to select data with higher uncertainty and greater sample diversity as the selection result. Use Gaussian kernel distance to measure the distance between two samples x i and x j The similarity between:

[0065]

[0066] Wherein, σ is the Gaussian kernel parameter, which is set to 1 in the present invention.

[0067] When calculating the similarity of sample x to a set Z consisting of samples, the distance between sample x and the sample in set Z with the greatest similarity to it is expressed as:

[0068]

[0069] The smaller the sim(x,Z) value is, the smaller the similarity between sample x and set Z is, that is, the smaller the redundancy between sample x and the existing elements in set Z is, so the diversity of the selected samples is greater.

[0070] Using the active learning strategy of “uncertainty + diversity criterion” to select n samples for training, we first use the “uncertainty criterion” to select m samples with larger uncertainty to form a set M = {x1, x2, …, x m}, and then select the n best samples from M using the method that comprehensively considers uncertainty and diversity as shown in the following formula:

[0071] score(x)=λunc(x)+(1-λ)sim(x,M)

[0072] Wherein, λ is a weighing parameter used to measure the importance of uncertainty and diversity, and is set to 0.8 in the present invention. The sample with the smallest score(x) finally calculated is the sample with large uncertainty and diversity, and has the greatest training value.

[0073] 3. Deep Learning Module

[0074] The deep learning module is trained using samples in the training set, and after the training, the samples in the test set are used for performance evaluation, and the performance evaluation results are output. The deep learning module of the present invention uses a convolutional neural network. The structure of the model is shown in Table 1. It includes five feature extraction modules. The first three modules are a structure of a convolution layer + a pooling layer, and the last two modules are a structure of two convolution layers + a pooling layer. After feature extraction, the output is expanded into a vector, and then connected to three fully connected layers. Among them, the first fully connected layer uses Dropout to randomly discard the output of a part of the neurons to reduce overfitting; the last fully connected layer uses Softmax classification output. In addition, the experiment in this paper deals with one-dimensional sonar echo data, so the feature map, convolution kernel and pooling kernel in the network are adjusted to a one-dimensional structure.

[0075] Figure 2 This is a schematic diagram of the CNN structure used in the present invention, which includes the size information of the feature map output by each layer.

[0076]

[0077] Table 1 Convolutional neural network structure

[0078] Figure 3 The following is a schematic diagram of the algorithm flow of the present invention. The specific process of the present invention is as follows:

[0079] 1. Preprocess the original data of underwater acoustic signals and divide them into candidate sets and test sets, which are mutually exclusive. Establish a training set that is an empty set in the initial state.

[0080] 2. Select a portion of data from the candidate set divided in step 1, obtain the annotations and move it from the candidate set to the training set as the initial training data.

[0081] 3. Train the CNN model using the data in the training set and record the model performance when training is complete.

[0082] 4. In the remaining candidate sets, according to the active learning strategy, select a certain amount of data, obtain the annotations and move them to the training set.

[0083] 5. Repeat steps 3 and 4 until all the data in the alternative set are added to the training set.

[0084] 6. After the model training is completed, based on the model performance recorded at each step, the changing relationship between the model performance and the increase in the amount of training data is obtained.

[0085] During the experiment of the present invention, the performance of the deep learning module was compared with the amount of training data when the active learning module used the following three sample selection strategies:

[0086] 1) Random sample selection strategy

[0087] 2) “Uncertainty Criteria” Sample Selection Strategy

[0088] 3) “Uncertainty + Diversity Standard” Sample Selection Strategy

[0089] First, 1560 samples are randomly selected from the 28560 candidate set samples to form the initial training set. The model is trained using the initial training set to obtain the initial model, and the performance of the model in the initial state is recorded. Subsequently, according to the sample selection strategy of the active learning module, 1000 samples are selected from the remaining 27000 samples in the candidate set for annotation each time, and after moving them to the training set, training is performed based on the initial model. After saving the new model performance, the iteration is restarted until the candidate set is empty.

[0090] The experiment conducted 5 complete experiments on the two active learning sample selection strategies and the random sample selection strategy of the present invention, and took the average value as the final experimental result. The trend of the accuracy of the three sample selection strategies with the amount of training data is as follows: Figure 4As shown in the figure, the trend of the test set loss of all categories for the three sample selection strategies changes with the amount of training data is as follows Figure 5 As shown in the figure, random represents the random sample selection strategy, uncertainty represents the "uncertainty standard" sample selection strategy, and uncertainty+diversity represents the "uncertainty+diversity standard" sample selection strategy.

[0091] As the amount of training data increases, the frameworks using the three data selection strategies gradually converge the performance of the deep learning model. However, for the framework using the two active learning strategies, the model convergence speed is significantly faster than that of the framework using the random selection strategy, which proves the effectiveness of the active learning strategy. When using the "uncertainty standard" active learning strategy, only 49% of the training data is needed to make the model converge, achieving the same 97% test set accuracy as when all the data is used for training. For the random selection strategy, 84% of the data is needed to make the model converge. Comparing the "uncertainty standard" and random selection methods, if the goal is model convergence, the data annotation workload can be reduced by 35%. For the training data set of 27,000 samples used in the experiment in this paper, reducing the training data requirement by 35% can reduce the annotation of more than 9,400 samples, greatly reducing the data annotation cost. Compared with the "uncertainty standard", the degree of improvement of the model performance by the "uncertainty + diversity standard" active learning strategy is more obvious in the early stage of training. The framework using this method can reach 95% accuracy faster, and the amount of data required to reach model convergence is basically the same as the framework using the "uncertainty standard", which is about 49% of all training data.

[0092] In addition, in the initial stage of model training, the advantages of active learning strategies are most obvious, which is reflected in the steeper accuracy improvement curve and loss reduction curve, which quickly widens the gap with the random selection method. The reason is that at the beginning of the process, the model using active learning obtains data with higher information content for training, which can improve model performance faster and make the model converge quickly. Using the "uncertainty + diversity standard" framework, the model can reduce the loss of the test set faster and more steadily. The effect is also more obvious in the early stage of training, and there will be no jitter in the first few steps.

[0093] The present invention proposes an underwater target recognition method based on active deep learning. There are many ways to implement the solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention. The parts that are clear in this embodiment can be implemented by existing technologies.

Claims

1. An underwater target recognition method based on active deep learning, characterized in that: The method comprises the following steps: Step 1: Preprocess the original data of the underwater acoustic signal and divide it into a candidate set and a test set. The candidate set and the test set are mutually exclusive, and a training set that is an empty set in the initial state is established; Step 2: Select a portion of data from the candidate set divided in step 1, obtain annotations and move it from the candidate set to the training set as the initial training data; Step 3: Train the CNN model using the data in the training set and record the model performance when the training is completed; Step 4: In the remaining candidate sets, according to the active learning strategy, select a certain amount of data, obtain annotations and move them to the training set; Step 5: Repeat steps 3 and 4 until all the data in the candidate set are added to the training set; Step 6: After the model training is completed, the relationship between the model performance and the increase in the amount of training data is obtained based on the model performance recorded at each step; The CNN model described in step 3 is trained using the data in the training set, wherein the structure of the CNN model is shown in the following table, which includes five feature extraction modules, the first three modules are a structure of one convolution layer + one pooling layer, and the last two modules are a structure of two convolution layers + one pooling layer; after feature extraction, the output is expanded into a vector, and then connected to three fully connected layers, wherein the first fully connected layer uses Dropout to randomly discard the output of a part of the neurons to reduce overfitting; the last fully connected layer uses Softmax classification output; in addition, one-dimensional sonar echo data is processed, so the feature map, convolution kernel and pooling kernel in the network are adjusted to a one-dimensional structure; In step 4, a certain amount of data is selected according to the active learning strategy, wherein the active learning sample selection strategy includes: "uncertainty standard" active learning strategy and "uncertainty + diversity standard" active learning strategy; the specific steps of the "uncertainty standard" active learning sample selection strategy are to select the most ambiguous sample judged by the model from the unlabeled samples, that is, the sample with the lowest confidence of the model in the category to which it belongs; for each sample x, the probability of the two most likely predictions output by the classifier is used to calculate its uncertainty, that is, unc(x): In the formula, p(y1|x) and p(y2|x) represent the probabilities of the two most likely and second most likely categories of sample x in the classifier output respectively; the larger the value of unc(x), the higher the confidence of the model in the most likely sample, and the less likely the classifier is to misclassify the sample; therefore, the active learner selects the sample with the smallest unc(x) value as the sample with the greatest uncertainty, sends it for annotation and adds it to the training set; The specific steps of the "uncertainty + diversity standard" active learning sample selection strategy are: based on the "uncertainty standard", using distance measurement to judge the diversity of samples, and then considering uncertainty and diversity comprehensively, selecting data with higher uncertainty and greater sample diversity as the selection result; using Gaussian kernel distance to measure the distance between two samples x i and x j The similarity between: Where, σ is the Gaussian kernel parameter, which is set to 1; When calculating the similarity of sample x to a set Z consisting of samples, the distance between sample x and the sample in set Z with the greatest similarity to it is expressed as: The smaller the sim(x,Z) value is, the smaller the similarity between sample x and set Z is, that is, the smaller the redundancy between sample x and the existing elements in set Z is, so the diversity of the selected samples is greater; Using the active learning strategy of "uncertainty + diversity criterion" to select n samples for training, we first use the "uncertainty criterion" to select m samples with larger uncertainty to form a set M = {x1, x2, ..., x m }, and then select n optimal samples from M using the method that comprehensively considers uncertainty and diversity as shown in the following formula: score(x)=λunc(x)+(1-λ)sim(x,M) Among them, λ is a trade-off parameter used to measure the importance of uncertainty and diversity, which is set to 0.8; the sample with the smallest score (x) is the sample with large uncertainty and diversity, and its training value is the greatest.

2. The underwater target recognition method based on active deep learning according to claim 1 is characterized in that: The raw data of the underwater acoustic signal is preprocessed in step 1, and the data preprocessing method includes: pre-emphasizing the high-frequency part of the acoustic signal, dividing the longer signal into shorter segments, and adding windows to make the edge of the frame smoothly transition to 0.

3. The underwater target recognition method based on active deep learning according to claim 1 is characterized in that: The step 1 is to divide the water acoustic signal data into mutually exclusive candidate sets and test sets. The specific method is: randomly select 20% of all preprocessed water acoustic signal data as the test set, and the rest as the candidate set, and annotate the data in the test set as a criterion for judging the performance of the CNN model.

Images