Deep learning low-speed small target identification method and system based on accumulated time-frequency spectrum

Abstract

The invention provides a deep learning low-speed small target identification method and system based on a cumulative time-frequency spectrum, and the method comprises the steps: carrying out the short-time Fourier transform of a target fixed window beam domain echo data segment, obtaining a target single-frame time-frequency spectrum image, carrying out the cumulative fusion of a plurality of generated single-frame time-frequency spectrum images, and generating a multi-frame cumulative time-frequency spectrum image; the multiple frames of accumulated time-frequency spectrum images containing multiple kinds of physical information are sent to a deep learning model for training; the three-channel multi-frame accumulation time-frequency spectrum image directly uses a large optical image pre-training weight to carry out transfer learning fine tuning, so that the model starts training on a better initialization weight, and the model is further prevented from being converged to a local minimum value. According to the method, multi-frame accumulation is carried out on the basis of extracting the target single-frame time-frequency spectrum image, more physical information can be provided, the distinguishability of input features of the deep learning model is improved, the deep learning model is enabled to further extract high-dimensional abstract features with higher discriminability, and the underwater low-speed small target recognition accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of underwater small target recognition technology, and more specifically, to a deep learning method and system for slow small target recognition based on cumulative time spectrum. Background Technology

[0002] The detection and identification of slow-moving small underwater targets, such as divers and unmanned underwater vehicles, is one of the core challenges in the fields of maritime security and underwater defense. These targets are characterized by their small size, complex structure, and weak acoustic features, posing a significant challenge to existing target identification systems. Active sonar is the primary means of detecting such targets, but its echo signals typically exhibit low signal-to-noise ratios, weak target features, and susceptibility to interference from complex marine environments, leading to a significant performance degradation of traditional identification methods. Deep learning models, due to their powerful automatic feature extraction and recognition capabilities, have been introduced into the field of underwater acoustic target identification. However, directly inputting raw sonar echo data or conventional time-spectrum images obtained from short-time Fourier transforms into the network has obvious limitations. In single-frame echo data or time-spectrum images, target energy is dispersed, and effective information is drowned out by strong background noise and reverberation. This fails to fully utilize the relatively stable features exhibited by the target over a longer time period, making it impossible to form highly discriminative patterns. Consequently, deep learning models struggle to learn stable and discriminative feature representations and are prone to overfitting to noisy patterns. Therefore, the current goal is to effectively accumulate and enhance the weak acoustic features of slow, small targets under low signal-to-noise ratio conditions, provide high-quality, high-discrimination input information for deep learning models, and improve the target recognition accuracy of deep learning models. This requires the development of deep learning slow, small target recognition methods based on accumulated time-spectrum.

[0003] Patent application CN109100710A discloses an underwater target recognition method based on a convolutional neural network, comprising: 1 simulating underwater acoustic target radiation noise; 2 acquiring underwater target tracking beams; 3 acquiring time-frequency maps of the target tracking beams, segmenting all time-frequency maps according to a fixed duration and dividing them into training samples and test samples; 4 performing data augmentation, scaling, and cropping on the samples; 5 inputting labeled training samples into a constructed convolutional neural network for supervised learning to obtain parameters of each layer of the convolutional neural network; 6 initializing the network using the parameters of each layer to obtain a convolutional neural network with underwater target recognition function; 7 acquiring the radiation noise of the target under test using a towed array, converting it into a time-frequency map and segmenting it, inputting the segmented sub-maps as test samples into the convolutional neural network to obtain the recognition result of each sub-map, and taking the target with the most identified targets as the final recognition result. However, this patent cannot completely solve the existing technical problems, nor can it meet the needs of this invention. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide a deep learning method and system for identifying slow small targets based on cumulative time-frequency spectrum.

[0005] The deep learning-based slow small target recognition method based on cumulative time spectrum provided by the present invention includes: Step 1: Use active sonar to transmit linear frequency modulated signals and receive array element domain echo data. Perform beamforming preprocessing on the array element domain echo data to obtain beam domain echo data. Based on the target distance and azimuth, extract a fixed window of beam domain echo data segment from the beam domain echo data. Step 2: Perform time-frequency analysis on the beam domain echo data segment to generate a single-frame time-frequency image of the target. The time-frequency analysis includes windowing the data segment and performing a short-time Fourier transform to obtain the time-frequency representation of the data segment, and normalizing and pseudo-color mapping the amplitude spectrum of the time-frequency representation. Step 3: Accumulate and fuse the generated single-frame time-spectrum images to generate a multi-frame accumulated time-spectrum image. The accumulation and fusion includes pixel-level alignment and summation averaging of each frame of the single-frame time-spectrum image based on the distance and azimuth translation of the target in a continuous period. Step 4: Construct a deep convolutional neural network model, which includes multiple convolutional blocks and fully connected layers connected in sequence. Each convolutional block contains a convolutional layer, a ReLU activation layer, and a pooling layer. The convolutional layer is used to extract features from the input image, and the pooling layer is used to downsample the feature map. Step 5: Input the generated multi-frame cumulative time-spectral image into the deep convolutional neural network model for training. The training process adopts a transfer learning strategy based on pre-trained weights from a large optical image dataset, and iteratively updates the parameters of the deep convolutional neural network model by minimizing the loss function. Step 6: Input the multi-frame cumulative time-spectral image of the test set into the trained deep convolutional neural network model to obtain the prediction probability of the model for each type of target, and calculate the recognition accuracy of the model based on the prediction probability; Step 7: Prepare a single-frame time-domain echo image dataset, a single-frame time-spectrum image dataset, and a multi-frame cumulative time-spectrum image dataset, respectively; using the same deep convolutional neural network model structure and training strategy, train three comparative models based on the single-frame time-domain echo image dataset, the single-frame time-spectrum image dataset, and the multi-frame cumulative time-spectrum image dataset, respectively; evaluate and compare the recognition accuracy and overall recognition accuracy of the three comparative models for different targets on the same test set.

[0006] Preferably, the step of performing time-frequency analysis on the beam domain echo data segment to generate a single-frame time-spectrum image of the target specifically involves: Perform a short-time Fourier transform on the beam domain echo data segment b(n) to generate a time-frequency spectrum. The calculation formula is as follows:

[0007] Where n is the period index, k is the frequency index, w(m) is a Hamming window of length L, and N is the number of FFT points. This represents the Fourier transform value at the k-th frequency point in the n-th period; It is the index of each point in the data segment; It is a data segment; It is an imaginary number; The Hamming window w(m) has a window length L of 256, an overlap number of 128, and an FFT number N of 512; generating the single-frame time-spectral image includes: Taking the modulus of the two-dimensional complex matrix X(n,k) obtained by the short-time Fourier transform, we obtain the amplitude spectrum |X(n,k)| of the time spectrum; The amplitude spectrum |X(n,k)| is normalized and mapped to the pixel value range of [0, 255]. The mapped pixel value range is converted into a color image by applying pseudo-color transformation to obtain the single-frame time-spectrum image, where the rows of the image correspond to frequencies and the columns correspond to distances.

[0008] Preferably, the step of accumulating and fusing the generated multiple single-frame time-spectrum images to generate a multi-frame accumulated time-spectrum image specifically involves: The time-spectral images of a single frame over M consecutive periods are accumulated according to a multi-frame accumulation formula, which is as follows:

[0009] Where M is the cumulative number of frames, and n is the period index. and The distance and orientation translations are caused by the movement of the target. This represents the pixel index of the time-spectrum image; This represents the spectrum image during the i-th period; The cumulative frame count M is 5, and the cumulative fusion process includes: Record the distance and orientation information of the target in the nth cycle; Calculate the distance translation of the (n+1), (n+2), (n+3), and (n+4)th periods relative to the nth period. and azimuth translation ; The time-spectrum image of a single frame in the nth period is compared with the time-spectrum images of the single frames in the (n+1), (n+2), (n+3), and (n+4) periods after being shifted according to the shift amount, and the pixel values ​​are accumulated and summed at the corresponding pixel positions. The cumulative summation of the image pixel values ​​is averaged to obtain a 5-frame cumulative time spectrum image.

[0010] Preferably, the training process of the deep convolutional neural network model specifically includes: The multi-frame cumulative time-spectral image dataset is divided into a training set, a validation set, and a test set; The model is trained using the training set, and the model performance is evaluated using the validation set during training. The loss function is the cross-entropy loss function, the optimizer is the Adam optimizer, the initial learning rate is set to 1e-4, and the number of training rounds is 24. The transfer learning strategy specifically involves initializing the convolutional layer weights of the deep convolutional neural network model using pre-trained weights from the ImageNet dataset. During the training process, the input multi-frame cumulative time-spectral image is preprocessed, and the preprocessing includes: The size of the input image is uniformly normalized to 256 pixels × 256 pixels; The image is randomly flipped horizontally or vertically with a probability of 0.5 as a data augmentation technique.

[0011] Preferably, the step of calculating the recognition accuracy of the model based on the predicted probability specifically involves: The output of the fully connected layer of the deep convolutional neural network model is connected to the Softmax layer. The Softmax layer outputs a probability vector, which represents the probability that the input image belongs to each target category. The category corresponding to the highest probability value in the probability vector is taken as the recognition result of the model; The recognition accuracy Acc is calculated based on the model's predictions on the test set and the true labels, using the following formula:

[0012] Wherein, TP represents the number of cases where the model predicts the class as positive and the true label is also positive; TN represents the number of cases where the model predicts the class as negative and the true label is also negative; FP represents the number of cases where the model predicts the class as positive but the true label is negative; and FN represents the number of cases where the model predicts the class as negative but the true label is positive.

[0013] The deep learning-based slow small target recognition system based on cumulative time spectrum provided by the present invention includes: Module M1: Uses active sonar to transmit linear frequency modulated signals and receive array element domain echo data, performs beamforming preprocessing on the array element domain echo data to obtain beam domain echo data, and extracts a fixed window of beam domain echo data segment from the beam domain echo data according to the target distance and azimuth. Module M2: Performs time-frequency analysis on the beam domain echo data segment to generate a single-frame time-frequency image of the target. The time-frequency analysis includes windowing the data segment and performing a short-time Fourier transform to obtain the time-frequency representation of the data segment, and normalizing and pseudo-color mapping the amplitude spectrum of the time-frequency representation. Module M3: Accumulates and fuses the generated single-frame time-spectrum images to generate a multi-frame accumulated time-spectrum image. The accumulation and fusion includes pixel-level alignment and summation averaging of each frame of the single-frame time-spectrum image based on the distance and azimuth translation of the target in a continuous period. Module M4: Constructs a deep convolutional neural network model, which includes multiple convolutional blocks and fully connected layers connected in sequence. Each convolutional block contains a convolutional layer, a ReLU activation layer, and a pooling layer. The convolutional layer is used to extract features from the input image, and the pooling layer is used to downsample the feature map. Module M5: The generated multi-frame cumulative time-spectral image is input into the deep convolutional neural network model for training. The training process adopts a transfer learning strategy based on pre-trained weights from a large optical image dataset, and the parameters of the deep convolutional neural network model are iteratively updated by minimizing the loss function. Module M6: Input the multi-frame cumulative time-spectral image of the test set into the trained deep convolutional neural network model to obtain the prediction probability of the model for various targets, and calculate the recognition accuracy of the model based on the prediction probability; Module M7: Prepare single-frame time-domain echo image datasets, single-frame time-spectrum image datasets, and multi-frame cumulative time-spectrum image datasets respectively; using the same deep convolutional neural network model structure and training strategy, train three comparative models based on the single-frame time-domain echo image dataset, the single-frame time-spectrum image dataset, and the multi-frame cumulative time-spectrum image dataset respectively; evaluate and compare the recognition accuracy and total recognition accuracy of the three comparative models for different targets on the same test set.

[0014] Preferably, the step of performing time-frequency analysis on the beam domain echo data segment to generate a single-frame time-spectrum image of the target specifically involves: Perform a short-time Fourier transform on the beam domain echo data segment b(n) to generate a time-frequency spectrum. The calculation formula is as follows:

[0015] Where n is the period index, k is the frequency index, w(m) is a Hamming window of length L, and N is the number of FFT points. This represents the Fourier transform value at the k-th frequency point in the n-th period; It is the index of each point in the data segment; It is a data segment; It is an imaginary number; The Hamming window w(m) has a window length L of 256, an overlap number of 128, and an FFT number N of 512; generating the single-frame time-spectral image includes: Taking the modulus of the two-dimensional complex matrix X(n,k) obtained by the short-time Fourier transform, we obtain the amplitude spectrum |X(n,k)| of the time spectrum; The amplitude spectrum |X(n,k)| is normalized and mapped to the pixel value range of [0, 255]. The mapped pixel value range is converted into a color image by applying pseudo-color transformation to obtain the single-frame time-spectrum image, where the rows of the image correspond to frequencies and the columns correspond to distances.

[0016] Preferably, the step of accumulating and fusing the generated multiple single-frame time-spectrum images to generate a multi-frame accumulated time-spectrum image specifically involves: The time-spectral images of a single frame over M consecutive periods are accumulated according to a multi-frame accumulation formula, which is as follows:

[0017] Where M is the cumulative number of frames, and n is the period index. and The distance and orientation translations are caused by the movement of the target. This represents the pixel index of the time-spectrum image; This represents the spectrum image during the i-th period; The cumulative frame count M is 5, and the cumulative fusion process includes: Record the distance and orientation information of the target in the nth cycle; Calculate the distance translation of the (n+1), (n+2), (n+3), and (n+4)th periods relative to the nth period. and azimuth translation ; The time-spectrum image of a single frame in the nth period is compared with the time-spectrum images of the single frames in the (n+1), (n+2), (n+3), and (n+4) periods after being shifted according to the shift amount, and the pixel values ​​are accumulated and summed at the corresponding pixel positions. The cumulative summation of the image pixel values ​​is averaged to obtain a 5-frame cumulative time spectrum image.

[0018] Preferably, the training process of the deep convolutional neural network model specifically includes: The multi-frame cumulative time-spectral image dataset is divided into a training set, a validation set, and a test set; The model is trained using the training set, and the model performance is evaluated using the validation set during training. The loss function is the cross-entropy loss function, the optimizer is the Adam optimizer, the initial learning rate is set to 1e-4, and the number of training rounds is 24. The transfer learning strategy specifically involves initializing the convolutional layer weights of the deep convolutional neural network model using pre-trained weights from the ImageNet dataset. During the training process, the input multi-frame cumulative time-spectral image is preprocessed, and the preprocessing includes: The size of the input image is uniformly normalized to 256 pixels × 256 pixels; The image is randomly flipped horizontally or vertically with a probability of 0.5 as a data augmentation technique.

[0019] Preferably, the step of calculating the recognition accuracy of the model based on the predicted probability specifically involves: The output of the fully connected layer of the deep convolutional neural network model is connected to the Softmax layer. The Softmax layer outputs a probability vector, which represents the probability that the input image belongs to each target category. The category corresponding to the highest probability value in the probability vector is taken as the recognition result of the model; The recognition accuracy Acc is calculated based on the model's predictions on the test set and the true labels, using the following formula:

[0020] Wherein, TP represents the number of cases where the model predicts the class as positive and the true label is also positive; TN represents the number of cases where the model predicts the class as negative and the true label is also negative; FP represents the number of cases where the model predicts the class as positive but the true label is negative; and FN represents the number of cases where the model predicts the class as negative but the true label is positive.

[0021] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a deep learning method for identifying slow, small targets based on accumulated time-spectrum data. It combines time-spectrum image representation of slow, small underwater targets with deep learning. By accumulating data across multiple frames, the motion information between target frames is represented in the time-spectrum image, improving the discriminability of the input to the deep learning model. This allows the deep learning model to automatically extract more discriminative features, thereby improving the accuracy of target recognition. Attached Figure Description

[0022] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of the slow small target recognition process based on time-frequency spectrum deep learning described in this invention; Figures 2a-2c These are the spectral images of single frames for targets 1 to 3 of the present invention, respectively. Figures 3a-3cThese are the cumulative spectral images of multiple frames for targets 1 to 3 of the present invention, respectively. Figure 4 This is a structural diagram of the deep learning recognition model based on multi-frame temporal spectrum images described in this invention. Detailed Implementation

[0023] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0024] Example To address the issue of low feature discriminability among slow-moving small underwater targets under low signal-to-noise ratio conditions, this invention proposes a deep learning-based method for identifying slow-moving small targets based on cumulative time-spectrum data. A short-time Fourier transform is performed on the target's fixed-window beam domain echo data segment to obtain a single-frame time-spectrum image of the target. Multiple generated single-frame time-spectrum images are accumulated and fused to generate a multi-frame cumulative time-spectrum image, enabling the image to simultaneously contain both target time-spectrum information and motion information between multiple frames. This multi-frame cumulative time-spectrum image, containing various physical information, is fed into a deep learning model for training, allowing the model to learn more discriminative features. The 3-channel multi-frame cumulative time-spectrum image is directly fine-tuned using pre-trained weights from a large optical image through transfer learning, allowing the model to begin training with better weight initialization and further preventing the model from converging to local minima. The purpose of this invention is to provide a deep learning method for identifying slow, small targets based on accumulated time-spectrum data. By extracting a single-frame time-spectrum image of the target and accumulating it across multiple frames, this method provides more physical information compared to single-frame time-domain echo images and single-frame time-spectrum images. This improves the discriminability of the input features of the deep learning model, enabling the deep learning model to further extract more discriminative high-dimensional abstract features, thereby improving the accuracy of underwater slow, small target identification.

[0025] like Figure 1 The specific implementation process is as follows: Step 1: Active sonar echo data acquisition and preprocessing. Active sonar transmits linear frequency modulated signals to receive echo data in the array element domain. After beamforming preprocessing, beam domain echo data is obtained. Based on the target distance and azimuth, a fixed window size is used to extract the beam domain echo data as the raw data segment to be processed.

[0026] Step 2: Generation of a single-frame time-frequency spectrum image. The original data segment b(n) obtained in Step 1 is subjected to time-frequency analysis to generate a time-frequency spectrum image. This invention preferably uses short-time Fourier transform for this purpose, and the calculation formula is as follows:

[0027] Where n is the period index, k is the frequency index, w(m) is a Hamming window of length L, and N is the number of FFT points. This represents the Fourier transform value at the k-th frequency point in the n-th period; It is the index of each point in the data segment; It is a data segment; It is an imaginary number; Step 3: Multi-frame time-spectrum image accumulation. The multiple single-frame time-spectrum images generated in Step 2 are accumulated and fused to generate a multi-frame accumulated time-spectrum image I containing target motion information. The multi-frame accumulation formula is as follows:

[0028] Where M is the cumulative number of frames, and n is the period index. and The distance and orientation translations are caused by the movement of the target. This represents the pixel index of the time-spectrum image; This represents the spectral image during the i-th period.

[0029] Step 4: Deep Learning Recognition Model Construction and Training. A deep convolutional neural network model is constructed. The multi-frame cumulative time-of-flight spectrogram images generated in Step 3 are used for training the deep learning model. Preprocessing is performed on the multi-frame cumulative time-of-flight spectrogram images as input, and a transfer learning fine-tuning strategy is employed during training.

[0030] Step 5: Model Recognition Performance Evaluation and Comparison. The recognition accuracy of deep learning models trained on single-frame time-domain echo image datasets, single-frame time-spectrum image datasets, and multi-frame cumulative time-spectrum image datasets is compared. The results are shown in Table 1.

[0031] This invention also provides a deep learning-based slow small target recognition system based on cumulative time-frequency spectrum, comprising: module M1: transmitting a linear frequency modulated signal using an active sonar and receiving array element domain echo data, performing beamforming preprocessing on the array element domain echo data to obtain beam domain echo data, and extracting a fixed window of beam domain echo data segments from the beam domain echo data according to the target distance and azimuth; module M2: performing time-frequency analysis on the beam domain echo data segments to generate a single-frame time-frequency image of the target, wherein the time-frequency analysis includes windowing the data segments and performing short-time Fourier transform. The data segment is transformed using a leaf transform to obtain its time-frequency representation, and the amplitude spectrum of the time-frequency representation is normalized and pseudo-color mapped. Module M3: The multiple generated single-frame time-frequency images are accumulated and fused to generate a multi-frame accumulated time-frequency image. The accumulation and fusion includes pixel-level alignment and summation averaging of each frame of the single-frame time-frequency image based on the distance and azimuth translation of the target within a continuous period. Module M4: A deep convolutional neural network model is constructed. The model includes multiple convolutional blocks and fully connected layers connected in sequence. Each convolutional block contains a convolutional layer and a ReLU layer. The deep convolutional neural network (DNN) consists of an activation layer and a pooling layer. The convolutional layer extracts features from the input image, and the pooling layer downsamples the feature map. Module M5 inputs the generated multi-frame cumulative time-spectral images into the deep convolutional neural network model for training. The training process employs a transfer learning strategy based on pre-trained weights from a large optical image dataset, iteratively updating the parameters of the deep convolutional neural network model by minimizing the loss function. Module M6 inputs the multi-frame cumulative time-spectral images from the test set into the trained deep convolutional neural network model to obtain the model's prediction probabilities for various targets. The model's recognition accuracy is calculated based on the prediction probabilities. Module M7 prepares a single-frame temporal echo image dataset, a single-frame time-spectral image dataset, and a multi-frame cumulative time-spectral image dataset. Using the same deep convolutional neural network model structure and training strategy, three comparative models are trained based on the single-frame temporal echo image dataset, the single-frame time-spectral image dataset, and the multi-frame cumulative time-spectral image dataset, respectively. The recognition accuracy and overall recognition accuracy of the three comparative models for different targets are evaluated and compared on the same test set.

[0032] The step of performing time-frequency analysis on the beam domain echo data segment to generate a single-frame time-frequency image of the target specifically involves: Perform a short-time Fourier transform on the beam domain echo data segment b(n) to generate a time-frequency spectrum. The calculation formula is as follows:

[0033] Where n is the period index, k is the frequency index, w(m) is a Hamming window of length L, and N is the number of FFT points. This represents the Fourier transform value at the k-th frequency point in the n-th period; It is the index of each point in the data segment; It is a data segment; It is an imaginary number; The Hamming window w(m) has a window length L of 256, an overlap number of 128, and an FFT number N of 512; generating the single-frame time-spectral image includes: Taking the modulus of the two-dimensional complex matrix X(n,k) obtained by the short-time Fourier transform, we obtain the amplitude spectrum |X(n,k)| of the time spectrum; The amplitude spectrum |X(n,k)| is normalized and mapped to the pixel value range of [0, 255]. The mapped pixel value range is converted into a color image by applying pseudo-color transformation to obtain the single-frame time-spectrum image, where the rows of the image correspond to frequencies and the columns correspond to distances.

[0034] The step of accumulating and fusing the generated multiple single-frame time-spectrum images to generate a multi-frame accumulated time-spectrum image specifically involves: The time-spectral images of a single frame over M consecutive periods are accumulated according to a multi-frame accumulation formula, which is as follows:

[0035] Where M is the cumulative number of frames, and n is the period index. and The distance and orientation translations are caused by the movement of the target. This represents the pixel index of the time-spectrum image; This represents the spectrum image during the i-th period; The cumulative frame count M is 5, and the cumulative fusion process includes: Record the distance and orientation information of the target in the nth cycle; Calculate the distance translation of the (n+1), (n+2), (n+3), and (n+4)th periods relative to the nth period. and azimuth translation ; The time-spectrum image of a single frame in the nth period is compared with the time-spectrum images of the single frames in the (n+1), (n+2), (n+3), and (n+4) periods after being shifted according to the shift amount, and the pixel values ​​are accumulated and summed at the corresponding pixel positions. The cumulative summation of the image pixel values ​​is averaged to obtain a 5-frame cumulative time spectrum image.

[0036] The training process of the deep convolutional neural network model specifically includes: The multi-frame cumulative time-spectral image dataset is divided into a training set, a validation set, and a test set; The model is trained using the training set, and the model performance is evaluated using the validation set during training. The loss function is the cross-entropy loss function, the optimizer is the Adam optimizer, the initial learning rate is set to 1e-4, and the number of training rounds is 24. The transfer learning strategy specifically involves initializing the convolutional layer weights of the deep convolutional neural network model using pre-trained weights from the ImageNet dataset. During the training process, the input multi-frame cumulative time-spectral image is preprocessed, and the preprocessing includes: The size of the input image is uniformly normalized to 256 pixels × 256 pixels; The image is randomly flipped horizontally or vertically with a probability of 0.5 as a data augmentation technique.

[0037] The specific steps for calculating the recognition accuracy of the model based on the predicted probability are as follows: The output of the fully connected layer of the deep convolutional neural network model is connected to the Softmax layer. The Softmax layer outputs a probability vector, which represents the probability that the input image belongs to each target category. The category corresponding to the highest probability value in the probability vector is taken as the recognition result of the model; The recognition accuracy Acc is calculated based on the model's predictions on the test set and the true labels, using the following formula:

[0038] Wherein, TP represents the number of cases where the model predicts the class as positive and the true label is also positive; TN represents the number of cases where the model predicts the class as negative and the true label is also negative; FP represents the number of cases where the model predicts the class as positive but the true label is negative; and FN represents the number of cases where the model predicts the class as negative but the true label is positive.

[0039] Example 2 This invention proposes a deep learning-based method for identifying slow, small targets based on cumulative time-frequency spectra, comprising: (1) Active sonar echo data acquisition and preprocessing Active sonar is used to detect the monitored waters, transmitting linear frequency modulated signals and receiving echo data in the element domain. Beamforming preprocessing is performed on the raw element domain echo data to obtain beam domain echo data. Based on the target's range and azimuth, the target's position in the beam domain echo data is determined. A fixed window size is used, centered on the target's beam, to extract the beam domain echo data within that window as the raw data segment to be processed. The same raw data segment extraction is performed on multiple periods of beam domain echo data for each type of target, forming a raw data segment dataset for each target type.

[0040] (2) Generation of single-frame time spectrum image The original data segment b(n) obtained in step 1 is subjected to time-frequency analysis to generate a time-frequency spectrum. This invention preferably uses short-time Fourier transform for this purpose, and the calculation formula is as follows:

[0041] Where n is the period index and k is the frequency index. Let N be a Hamming window of length L, and N be the number of FFT points. First, perform a short-time Fourier transform on the original data segment b(n), using a Hamming window with a length of 256, an overlap of 128, and an FFT point count of 512, to obtain a two-dimensional complex matrix. The magnitude spectrum of the time spectrum is obtained by taking the modulus value. Secondly, regarding the amplitude spectrum... Normalization was performed and mapped to the pixel value range of [0, 255]. A pseudo-color transformation was then applied to obtain the single-frame spectral image. The single-frame spectral images of targets 1 to 3 are shown below. Figures 2a-2c As shown, the rows in this image correspond to frequency, the columns correspond to distance, and the pixel values ​​correspond to echo energy intensity.

[0042] (3) Multi-frame time spectrum image accumulation The multiple single-frame time-spectrum images generated in step 2 are accumulated and fused to generate a multi-frame accumulated time-spectrum image I containing target motion information. The multi-frame accumulation formula is as follows:

[0043] Where M is the cumulative number of frames, and n is the period index. and This refers to the distance and azimuth translation caused by the target's movement. First, record the target's distance and azimuth information for the nth period. Assuming the cumulative frame count M is 5, calculate the distance and azimuth translations relative to the nth period for periods (n+1, n+2, n+3, n+4). Summate the corresponding pixel values ​​of the time-spectrum image for the nth period and the time-spectrum images after translation for periods (n+1, n+2, n+3, n+4) to obtain 5 cumulative time-spectrum images. The cumulative time-spectrum images for targets 1 to 3 are shown below. Figures 3a-3c As shown.

[0044] (4) Construction and training of deep learning recognition model like Figure 4The diagram illustrates the construction of a deep convolutional neural network model. The model consists of multiple convolutional blocks and fully connected layers. Each convolutional block contains a convolutional layer, a ReLU layer, and a pooling layer, used to extract high-dimensional abstract features from the multi-frame cumulative time-of-flight spectrogram images generated in step 3. The dataset of multi-frame cumulative time-of-flight spectrogram images generated in step 3 is divided into training, validation, and test sets. During training, the deep learning recognition model is trained using the training set, and the model's recognition accuracy is evaluated using the validation set. Cross-entropy is used as the loss function, the optimizer is Adam, the initial learning rate is set to 1e-4, the training epochs are 24, and ImageNet pre-trained weights are used to fine-tune the network through transfer learning. The optimization objective is to minimize the loss function, and the model parameters are iteratively updated. During training, the multi-frame cumulative time-of-flight spectrogram images are preprocessed, including but not limited to: size normalization. Pixels are randomly flipped for data augmentation with a probability of 0.5.

[0045] (5) Evaluation and comparison of model recognition performance The multi-frame cumulative time-spectrum image test set obtained in step 4 is input into the trained deep learning recognition model. The model performs forward propagation calculations, and after passing through the Softmax layer, it outputs a probability vector containing each class of target. The recognition result is determined by the class corresponding to the highest probability value. Finally, the recognition accuracy is calculated using the following formula:

[0046] Wherein, TP represents the number of cases where the model predicts the class as positive and the true label is also positive; TN represents the number of cases where the model predicts the class as negative and the true label is also negative; FP represents the number of cases where the model predicts the class as positive but the true label is negative (false alarm); and FN represents the number of cases where the model predicts the class as negative but the true label is positive (false negative).

[0047] Furthermore, this embodiment compares the recognition accuracy of deep learning models trained on single-frame time-domain echo image datasets, single-frame time-spectrum image datasets, and multi-frame cumulative time-spectrum image datasets. The results are shown in Table 1.

[0048] Table 1 Comparison of recognition accuracy of deep learning models trained on different data types

[0049] Experiments show that the deep learning recognition model trained using a single-frame time-domain echo image performs poorly in recognizing target 1 compared to a single-frame time-domain echo image, while the recognition accuracy of other targets is improved. Furthermore, the deep learning recognition model trained using multiple frames of cumulative time-domain echo images greatly improves the recognition accuracy of target 1, increasing it from 64.63% to 93.10%, and also achieves the highest overall recognition accuracy.

[0050] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0051] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A deep learning-based method for identifying slow, small targets based on cumulative time-frequency spectrum, characterized in that, include: Step 1: Use active sonar to transmit linear frequency modulated signals and receive array element domain echo data. Perform beamforming preprocessing on the array element domain echo data to obtain beam domain echo data. Based on the target distance and azimuth, extract a fixed window of beam domain echo data segment from the beam domain echo data. Step 2: Perform time-frequency analysis on the beam domain echo data segment to generate a single-frame time-frequency image of the target. The time-frequency analysis includes windowing the data segment and performing a short-time Fourier transform to obtain the time-frequency representation of the data segment, and normalizing and pseudo-color mapping the amplitude spectrum of the time-frequency representation. Step 3: Accumulate and fuse the generated single-frame time-spectrum images to generate a multi-frame accumulated time-spectrum image. The accumulation and fusion includes pixel-level alignment and summation averaging of each frame of the single-frame time-spectrum image based on the distance and azimuth translation of the target in a continuous period. Step 4: Construct a deep convolutional neural network model, which includes multiple convolutional blocks and fully connected layers connected in sequence. Each convolutional block contains a convolutional layer, a ReLU activation layer, and a pooling layer. The convolutional layer is used to extract features from the input image, and the pooling layer is used to downsample the feature map. Step 5: Input the generated multi-frame cumulative time-spectral image into the deep convolutional neural network model for training. The training process adopts a transfer learning strategy based on pre-trained weights from a large optical image dataset, and iteratively updates the parameters of the deep convolutional neural network model by minimizing the loss function. Step 6: Input the multi-frame cumulative time-spectral image of the test set into the trained deep convolutional neural network model to obtain the prediction probability of the model for each type of target, and calculate the recognition accuracy of the model based on the prediction probability; Step 7: Prepare a single-frame time-domain echo image dataset, a single-frame time-spectrum image dataset, and a multi-frame cumulative time-spectrum image dataset, respectively; using the same deep convolutional neural network model structure and training strategy, train three comparative models based on the single-frame time-domain echo image dataset, the single-frame time-spectrum image dataset, and the multi-frame cumulative time-spectrum image dataset, respectively; evaluate and compare the recognition accuracy and overall recognition accuracy of the three comparative models for different targets on the same test set.

2. The deep learning-based slow small target recognition method based on cumulative time spectrum according to claim 1, characterized in that, The step of performing time-frequency analysis on the beam domain echo data segment to generate a single-frame time-frequency image of the target specifically involves: Perform a short-time Fourier transform on the beam domain echo data segment b(n) to generate a time-frequency spectrum. The calculation formula is as follows: Where n is the period index, k is the frequency index, w(m) is a Hamming window of length L, and N is the number of FFT points. This represents the Fourier transform value at the k-th frequency point in the n-th period; It is the index of each point in the data segment; It is a data segment; It is an imaginary number; The Hamming window w(m) has a window length L of 256, an overlap number of 128, and an FFT number N of 512; generating the single-frame time-spectral image includes: Taking the modulus of the two-dimensional complex matrix X(n,k) obtained by the short-time Fourier transform, we obtain the amplitude spectrum |X(n,k)| of the time spectrum; The amplitude spectrum |X(n,k)| is normalized and mapped to the pixel value range of [0, 255]. The mapped pixel value range is converted into a color image by applying pseudo-color transformation to obtain the single-frame time-spectrum image, where the rows of the image correspond to frequencies and the columns correspond to distances.

3. The deep learning-based slow small target recognition method based on cumulative time spectrum according to claim 2, characterized in that, The step of accumulating and fusing the generated multiple single-frame time-spectrum images to generate a multi-frame accumulated time-spectrum image specifically involves: The time-spectral images of a single frame over M consecutive periods are accumulated according to a multi-frame accumulation formula, which is as follows: Where M is the cumulative number of frames, and n is the period index. and The distance and orientation translations are caused by the movement of the target. This represents the pixel index of the time-spectrum image; This represents the spectrum image during the i-th period; The cumulative frame count M is 5, and the cumulative fusion process includes: Record the distance and orientation information of the target in the nth cycle; Calculate the distance translation of the (n+1), (n+2), (n+3), and (n+4)th periods relative to the nth period. and azimuth translation ; The time-spectrum image of a single frame in the nth period is compared with the time-spectrum images of the single frames in the (n+1), (n+2), (n+3), and (n+4) periods after being shifted according to the shift amount, and the pixel values ​​are accumulated and summed at the corresponding pixel positions. The cumulative summation of the image pixel values ​​is averaged to obtain a 5-frame cumulative time spectrum image.

4. The deep learning-based slow small target recognition method based on cumulative time spectrum according to claim 1, characterized in that, The training process of the deep convolutional neural network model specifically includes: The multi-frame cumulative time-spectral image dataset is divided into a training set, a validation set, and a test set; The model is trained using the training set, and the model performance is evaluated using the validation set during training. The loss function is the cross-entropy loss function, the optimizer is the Adam optimizer, the initial learning rate is set to 1e-4, and the number of training rounds is 24. The transfer learning strategy specifically involves initializing the convolutional layer weights of the deep convolutional neural network model using pre-trained weights from the ImageNet dataset. During the training process, the input multi-frame cumulative time-spectral image is preprocessed, and the preprocessing includes: The size of the input image is uniformly normalized to 256 pixels × 256 pixels; The image is randomly flipped horizontally or vertically with a probability of 0.5 as a data augmentation technique.

5. The deep learning-based slow small target recognition method based on cumulative time spectrum according to claim 1, characterized in that, The specific steps for calculating the recognition accuracy of the model based on the predicted probability are as follows: The output of the fully connected layer of the deep convolutional neural network model is connected to the Softmax layer. The Softmax layer outputs a probability vector, which represents the probability that the input image belongs to each target category. The category corresponding to the highest probability value in the probability vector is taken as the recognition result of the model; The recognition accuracy Acc is calculated based on the model's predictions on the test set and the true labels, using the following formula: Wherein, TP represents the number of cases where the model predicts the class as positive and the true label is also positive; TN represents the number of cases where the model predicts the class as negative and the true label is also negative; FP represents the number of cases where the model predicts the class as positive but the true label is negative; and FN represents the number of cases where the model predicts the class as negative but the true label is positive.

6. A deep learning-based slow small target recognition system based on cumulative time-frequency spectrum, characterized in that, include: Module M1: Uses active sonar to transmit linear frequency modulated signals and receive array element domain echo data, performs beamforming preprocessing on the array element domain echo data to obtain beam domain echo data, and extracts a fixed window of beam domain echo data segment from the beam domain echo data according to the target distance and azimuth. Module M2: Performs time-frequency analysis on the beam domain echo data segment to generate a single-frame time-frequency image of the target. The time-frequency analysis includes windowing the data segment and performing a short-time Fourier transform to obtain the time-frequency representation of the data segment, and normalizing and pseudo-color mapping the amplitude spectrum of the time-frequency representation. Module M3: Accumulates and fuses the generated single-frame time-spectrum images to generate a multi-frame accumulated time-spectrum image. The accumulation and fusion includes pixel-level alignment and summation averaging of each frame of the single-frame time-spectrum image based on the distance and azimuth translation of the target in a continuous period. Module M4: Constructs a deep convolutional neural network model, which includes multiple convolutional blocks and fully connected layers connected in sequence. Each convolutional block contains a convolutional layer, a ReLU activation layer, and a pooling layer. The convolutional layer is used to extract features from the input image, and the pooling layer is used to downsample the feature map. Module M5: The generated multi-frame cumulative time-spectral image is input into the deep convolutional neural network model for training. The training process adopts a transfer learning strategy based on pre-trained weights from a large optical image dataset, and the parameters of the deep convolutional neural network model are iteratively updated by minimizing the loss function. Module M6: Input the multi-frame cumulative time-spectral image of the test set into the trained deep convolutional neural network model to obtain the prediction probability of the model for various targets, and calculate the recognition accuracy of the model based on the prediction probability; Module M7: Prepare single-frame time-domain echo image datasets, single-frame time-spectrum image datasets, and multi-frame cumulative time-spectrum image datasets respectively; using the same deep convolutional neural network model structure and training strategy, train three comparative models based on the single-frame time-domain echo image dataset, the single-frame time-spectrum image dataset, and the multi-frame cumulative time-spectrum image dataset respectively; evaluate and compare the recognition accuracy and total recognition accuracy of the three comparative models for different targets on the same test set.

7. The deep learning slow small target recognition system based on cumulative time spectrum according to claim 6, characterized in that, The step of performing time-frequency analysis on the beam domain echo data segment to generate a single-frame time-frequency image of the target specifically involves: Perform a short-time Fourier transform on the beam domain echo data segment b(n) to generate a time-frequency spectrum. The calculation formula is as follows: Where n is the period index, k is the frequency index, w(m) is a Hamming window of length L, and N is the number of FFT points. This represents the Fourier transform value at the k-th frequency point in the n-th period; It is the index of each point in the data segment; It is a data segment; It is an imaginary number; The Hamming window w(m) has a window length L of 256, an overlap number of 128, and an FFT number N of 512; generating the single-frame time-spectral image includes: Taking the modulus of the two-dimensional complex matrix X(n,k) obtained by the short-time Fourier transform, we obtain the amplitude spectrum |X(n,k)| of the time spectrum; The amplitude spectrum |X(n,k)| is normalized and mapped to the pixel value range of [0, 255]. The mapped pixel value range is converted into a color image by applying pseudo-color transformation to obtain the single-frame time-spectrum image, where the rows of the image correspond to frequencies and the columns correspond to distances.

8. The deep learning slow small target recognition system based on cumulative time spectrum according to claim 7, characterized in that, The step of accumulating and fusing the generated multiple single-frame time-spectrum images to generate a multi-frame accumulated time-spectrum image specifically involves: The time-spectral images of a single frame over M consecutive periods are accumulated according to a multi-frame accumulation formula, which is as follows: Where M is the cumulative number of frames, and n is the period index. and The distance and orientation translations are caused by the movement of the target. This represents the pixel index of the time-spectrum image; This represents the spectrum image during the i-th period; The cumulative frame count M is 5, and the cumulative fusion process includes: Record the distance and orientation information of the target in the nth cycle; Calculate the distance translation of the (n+1), (n+2), (n+3), and (n+4)th periods relative to the nth period. and azimuth translation ; The time-spectrum image of a single frame in the nth period is compared with the time-spectrum images of the single frames in the (n+1), (n+2), (n+3), and (n+4) periods after being shifted according to the shift amount, and the pixel values ​​are accumulated and summed at the corresponding pixel positions. The cumulative summation of the image pixel values ​​is averaged to obtain a 5-frame cumulative time spectrum image.

9. The deep learning slow small target recognition system based on cumulative time spectrum according to claim 6, characterized in that, The training process of the deep convolutional neural network model specifically includes: The multi-frame cumulative time-spectral image dataset is divided into a training set, a validation set, and a test set; The model is trained using the training set, and the model performance is evaluated using the validation set during training. The loss function is the cross-entropy loss function, the optimizer is the Adam optimizer, the initial learning rate is set to 1e-4, and the number of training rounds is 24. The transfer learning strategy specifically involves initializing the convolutional layer weights of the deep convolutional neural network model using pre-trained weights from the ImageNet dataset. During the training process, the input multi-frame cumulative time-spectral image is preprocessed, and the preprocessing includes: The size of the input image is uniformly normalized to 256 pixels × 256 pixels; The image is randomly flipped horizontally or vertically with a probability of 0.5 as a data augmentation technique.

10. The deep learning slow small target recognition system based on cumulative time spectrum according to claim 6, characterized in that, The specific steps for calculating the recognition accuracy of the model based on the predicted probability are as follows: The output of the fully connected layer of the deep convolutional neural network model is connected to the Softmax layer. The Softmax layer outputs a probability vector, which represents the probability that the input image belongs to each target category. The category corresponding to the highest probability value in the probability vector is taken as the recognition result of the model; The recognition accuracy Acc is calculated based on the model's predictions on the test set and the true labels, using the following formula: Wherein, TP represents the number of cases where the model predicts the class as positive and the true label is also positive; TN represents the number of cases where the model predicts the class as negative and the true label is also negative; FP represents the number of cases where the model predicts the class as positive but the true label is negative; and FN represents the number of cases where the model predicts the class as negative but the true label is positive.

Images