Underwater sound signal enhancement method based on generative adversarial network, electronic equipment and computer readable storage medium

Abstract

The invention provides an underwater sound signal enhancement method based on a generative adversarial network, and the method comprises the steps: S1, collecting an underwater noise-containing signal of a target water area through sound signal transmitting equipment and receiving equipment, carrying out the preprocessing and noise reduction of the noise-containing signal through an LMS adaptive filtering method, and making an underwater sound signal data set; s2, training an adaptive generative adversarial network CP-GAN special for underwater sound signal enhancement by using the underwater sound signal data set; and S3, receiving a sound signal transmitted by the transmitting equipment, preprocessing the sound signal through an LMS adaptive filtering method, inputting the sound signal into the CP-GAN network to enhance the quality of the sound signal, and transmitting the enhanced sound signal back to the receiving equipment. The invention further provides electronic equipment and a computer readable storage medium. According to the invention, sound signals interfered by underwater noise and weakened by underwater media are enhanced, and the effect of infinitely approaching original sound signals is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater acoustic signal processing, and in particular to an underwater acoustic signal enhancement method based on a generative adversarial network, an electronic device and a computer-readable storage medium. Background Art

[0002] With the continuous development of science and technology, underwater acoustic signal enhancement technology has become increasingly important in the fields of marine engineering, underwater acoustic communications, sonar detection, etc. However, due to the complexity and limitations of the underwater environment, underwater acoustic signals are often affected by various noise interferences and water medium weakening, which reduces the quality of acoustic signals and makes them difficult to obtain and analyze effectively. In traditional underwater acoustic signal enhancement methods, filtering, noise reduction, and compensation techniques are often used to improve signal quality, but these methods require more physical equipment support and have certain limitations. Not only that, according to Shannon's theorem, every time a certain distance of transmission medium is passed, the information will have a certain probability of distortion, which is also an important reason why traditional methods often cannot completely restore the quality of acoustic signals or remove noise. Therefore, a more advanced and more effective method is needed to solve the problem of underwater acoustic signal enhancement. Summary of the invention

[0003] In view of the shortcomings in the prior art, the present invention provides an underwater acoustic signal enhancement method based on a generative adversarial network, which aims to enhance the acoustic signal interfered by underwater noise and weakened by underwater media, so as to achieve an effect infinitely close to the original acoustic signal.

[0004] According to an embodiment of the present invention, a method for enhancing underwater acoustic signals based on a generative adversarial network includes:

[0005] S1 uses acoustic signal transmitting equipment and receiving equipment to collect underwater noisy signals in the target waters, and uses LMS adaptive filtering method to pre-process and reduce noise of the noisy signals to produce an underwater acoustic signal dataset;

[0006] S2 uses the underwater acoustic signal dataset to train an adaptive generative adversarial network CP-GAN specifically for underwater acoustic signal enhancement;

[0007] S3 receives the acoustic signal from the transmitting device, and after preprocessing by the LMS adaptive filtering method, it is input into the CP-GAN network to enhance the acoustic signal quality, and the enhanced acoustic signal is transmitted back to the receiving device.

[0008] Furthermore, collecting underwater noise signals in the target water area in S1 includes the following steps:

[0009] a1 denotes the local acoustic signal collected for the nth time as X(n), and the noisy signal obtained after the local acoustic signal X(n) is transmitted to the receiving device through the acoustic signal transmitting device is x(n);

[0010] a2 transmits the acoustic signal X(n) to the underwater acoustic signal receiving device through the land acoustic signal transmitting device, and obtains the acoustic signal x(n) with noise; the local acoustic signal is recorded as X(n+1), and the acoustic signal X(n+1) is transmitted to the land acoustic signal receiving device through the underwater acoustic signal transmitting device, and the acoustic signal x(n+1) with noise is obtained;

[0011] a3 Repeat sub-steps a1 and a2 until the amount of data reaches more than 100,000, and store the recorded data.

[0012] Furthermore, the preparation of the underwater acoustic signal dataset in S1 includes the following steps:

[0013] b1 construct an LMS adaptive filter in the acoustic signal receiving device;

[0014] b2 Input the noisy signal x(n) into the LMS adaptive filter for processing to obtain the filtered signal y(n), where the local signal X(n) collected for the nth time in step a1 is the target real sound signal, y(n) is the noisy signal after filtering, and y(n) and X(n) are the nth data of the sound signal data set;

[0015] b3 Repeat step b2 until all collected data are processed and a complete acoustic signal data set is obtained.

[0016] Further, S2 includes the following steps:

[0017] c1 improves the generator loss function of the generative adversarial network GAN as follows:

[0018] Loss_CGAN(G)=-E_y[log D(G(y),y)]+∥G(y)-x∥,

[0019] Where y represents the conditional signal input to the generator G as a condition, G(y) represents the false sound signal generated by the generator G for y, x represents the real sound signal in the training data, -E_y[logD(G(y),y)] represents the negative logarithmic expectation of the discrimination probability obtained by inputting the false sound signal G(y) generated by the generator G and the conditional signal y into the discriminator D, ∥G(y)-x∥ is the reconstruction loss term of the generator, which represents the difference between the generated false sound signal G(y) and the real sound signal x;

[0020] C2 uses a multi-granularity discriminator based on dynamic context information to discriminate segments of enhanced acoustic signals:

[0021] The global discriminator discriminates the output of the generator, outputs the probability that the generated sound signal is a real sound signal, and judges the effect of sound signal recovery as a whole. Its mathematical expression is as follows:

[0022] D_real=σ(C(x,y)-C(G(y),y)),

[0023] Among them, σ represents the sigmoid function, which maps the input value to a probability value between 0 and 1. C(x,y) represents the output of the discriminator for the real sound signal x and the conditional signal y, that is, the confidence or probability of the discriminator judging whether the input is a real sample. C(G(y),y) represents the output of the discriminator for the false sound signal G(y) and the conditional signal y generated by the generator, that is, the confidence or probability of the discriminator judging whether the input is a real sample. The greater the difference between the output C(x,y) and the output C(G(y),y), the more accurately the discriminator can distinguish between real samples and generated samples.

[0024] The local discriminator randomly selects 10 different sound signal segments for discrimination, and judges the degree of recovery of the generated sound signal from the local point of view. The mathematical description is as follows:

[0025] D_fake=σ(C(G(y),y)-C(x,y)),

[0026] Among them, the meanings of σ, C(x,y), C(G(y),y)G(y) are consistent with the mathematical description in the global discriminator. However, the goal of the local discriminator is to encourage the discriminator to accurately identify the generated false sound signal as a generated sample by minimizing the output difference of the generated samples.

[0027] c3 inputs the obtained data set into the improved generative adversarial network CP-GAN in steps c1 and c2 for training to obtain an underwater acoustic signal enhancement model for the target waters.

[0028] Furthermore, S3 includes the following steps:

[0029] d1 receives the acoustic signal X from the transmitting device in real time, and pre-processes the acoustic signal using the LMS adaptive filtering method to obtain the acoustic signal x with some noise eliminated;

[0030] d2 inputs the acoustic signal x into the trained CP-GAN acoustic signal enhancement model to enhance the acoustic signal quality and obtain the enhanced acoustic signal y;

[0031] d3 transmits the sound signal y back to the sound signal receiving device.

[0032] According to an embodiment, an electronic device is also provided, which includes: one or more processors; a memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the above-mentioned underwater acoustic signal enhancement method based on a generative adversarial network.

[0033] According to an embodiment, a computer-readable storage medium is also provided, which includes: storing a computer program that can be loaded by a processor and execute the above-mentioned underwater acoustic signal enhancement method based on a generative adversarial network.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] GAN is an adversarial model composed of a generator and a discriminator. The generator generates realistic samples, while the discriminator tries to distinguish between the generated samples and the real samples. By training the generator and the discriminator, GAN can learn how to generate high-quality and real sound signals. In underwater sound signal enhancement, the generator can receive noisy or water-weakened sound signals as input and generate enhanced sound signals as output. The discriminator is used to distinguish the difference between the generated sound signals and the real sound signals. By repeatedly training the generator and the discriminator, the GAN network can gradually optimize the generator's ability to generate underwater sound signals that are closer to reality and clearer. In addition, the generative adversarial network also has a certain degree of robustness, can adapt to changes in different underwater environments and noise scenes, and restore sound signals weakened by water media. It has broad prospects and potential in applications such as improving the quality of underwater sound signals, improving underwater communications and sonar detection.

[0036] The underwater acoustic signal enhancement method based on the generative adversarial network of the present invention does not rely on too many physical devices, but through the continuous game between the acoustic signal generator and the acoustic signal discriminator in the GAN network, until the generator generates an acoustic signal that the discriminator cannot judge whether it is true or false. This method of generating enhanced acoustic signals, which is different from the traditional physical signal enhancement method, can effectively eliminate noise and restore the acoustic signal weakened by the water medium, so as to achieve the purpose of being infinitely close to the real acoustic signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the structure of an LMS adaptive filter according to an embodiment of the present invention;

[0038] Figure 2 Schematic diagram of the overall structure of the CP-GAN network of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0040] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0041] In an exemplary embodiment, this embodiment provides an underwater acoustic signal enhancement method based on a generative adversarial network, which includes:

[0042] S1 uses acoustic signal transmitting equipment and receiving equipment to collect underwater noisy signals in the target waters, and uses LMS adaptive filtering method to pre-process and reduce noise of the noisy signals to produce an underwater acoustic signal dataset;

[0043] Furthermore, collecting underwater noise signals in the target water area in S1 includes the following steps:

[0044] a1 denotes the local acoustic signal collected for the nth time as X(n), and the noisy signal obtained after the local acoustic signal X(n) is transmitted to the receiving device through the acoustic signal transmitting device is x(n);

[0045] a2 transmits the acoustic signal X(n) to the underwater acoustic signal receiving device through the land acoustic signal transmitting device, and obtains the acoustic signal x(n) with noise; the local acoustic signal is recorded as X(n+1), and the acoustic signal X(n+1) is transmitted to the land acoustic signal receiving device through the underwater acoustic signal transmitting device, and the acoustic signal x(n+1) with noise is obtained;

[0046] a3 Repeat sub-steps a1 and a2 until the amount of data reaches more than 100,000, and store the recorded data.

[0047] Furthermore, the preparation of the underwater acoustic signal dataset in S1 includes the following steps:

[0048] b1 constructs an LMS adaptive filter in the acoustic signal receiving device. The LMS adaptive filter consists of two parts: a FIR filter and an LMS adaptive algorithm. The tap coefficients of the FIR filter are adjusted in real time by the LMS algorithm, and the output acoustic signal of the FIR filter is finally converged. This design can better eliminate the noise in the underwater acoustic signal and restore the original acoustic signal as much as possible. The structure of the LMS adaptive filter is as follows: Figure 1As shown in the figure, x(n) is the noisy signal sequence obtained by the nth acquisition in step (S1), y(n) is the output sound signal after the LMS adaptive filter processes x(n), d(n) is the desired sound signal, ω(n) is the filter coefficient, and the FIR filter is essentially a standard multiplication and addition operation set. The LMS algorithm is based on the minimum mean square error criterion. On the basis of the gradient method, by improving the calculation method of the estimated value of the mean square error gradient, the gradient of the square of a single error sample is taken as the estimated value of the mean square error, and the Wiener-Hoff equation is converted into a set of recursive formulas, namely:

[0049]

[0050] Where x(n) is the noisy signal sequence obtained by the nth acquisition in step (S1), y(n) is the output sound signal after the LMS adaptive filter processes x(n), d(n) is the desired sound signal, ω(n) is the filter coefficient, ε(n) is the error signal, and μ is the step size factor when the weighted vector is updated;

[0051] b2 Input the noisy signal x(n) into the LMS adaptive filter for processing to obtain the filtered signal y(n), where the local signal X(n) collected for the nth time in step a1 is the target real sound signal, y(n) is the noisy signal after filtering, and y(n) and X(n) are the nth data of the sound signal data set;

[0052] b3 Repeat step b2 until all collected data are processed and a complete acoustic signal data set is obtained.

[0053] S2 uses the underwater acoustic signal dataset to train an adaptive generative adversarial network CP-GAN specifically for underwater acoustic signal enhancement.

[0054] Furthermore, S2 is characterized by improving the generative adversarial network (GAN) originally applied to the field of image generation, including the following steps:

[0055] c1 improves the generator loss function of the generative adversarial network GAN to better target the field of acoustic signal processing, enhance the acoustic signal that has been weakened and degraded by the water medium, and generate an acoustic signal similar to the real acoustic signal. The mathematical description of the improved generator loss function is as follows:

[0056] Loss_CGAN(G)=-E_y[logD(G(y),y)]+∥G(y)-x∥,

[0057] Where y represents the conditional signal input to the generator G as a condition, G(y) represents the false sound signal generated by the generator G for y, x represents the real sound signal in the training data, -E_y[logD(G(y),y)] represents the negative logarithmic expectation of the discrimination probability obtained by inputting the false sound signal G(y) generated by the generator G and the conditional signal y into the discriminator D, ∥G(y)-x∥ is the reconstruction loss term of the generator, which represents the difference between the generated false sound signal G(y) and the real sound signal x; the goal of the generator is to maximize the probability that the discriminator cannot accurately distinguish the generated false sound signal G(y) from the real sound signal y, minimize the difference between the generated false sound signal and the real sound signal, and make the generated sound closer to the real sound signal;

[0058] C2 uses a multi-granularity discriminator with dynamic context information to effectively discriminate segments of enhanced acoustic signals:

[0059] The existing GAN-based acoustic signal enhancement model only designs a discriminator to discriminate the acoustic signal generated by the generator. However, the acoustic signal is an unstable signal. Especially after being affected by the water medium, the overall amplitude is low, the signal quality is reduced, and there is even strong noise in some positions, and weak noise or no noise in other positions. Therefore, a dual discriminator, a global discriminator and a local discriminator, is used to judge the degree of restoration of the acoustic signal by combining the overall acoustic signal and the local fragments. The global discriminator discriminates the output of the generator, outputs the probability that the generated acoustic signal is a real acoustic signal, and judges the effect of acoustic signal restoration as a whole. Its mathematical expression is as follows:

[0060] D_real=σ(C(x,y)-C(G(y),y)),

[0061] Among them, σ represents the sigmoid function, which maps the input value to a probability value between 0 and 1. C(x,y) represents the output of the discriminator for the real sound signal x and the conditional signal y, that is, the confidence or probability of the discriminator judging whether the input is a real sample. C(G(y),y) represents the output of the discriminator for the false sound signal G(y) and the conditional signal y generated by the generator, that is, the confidence or probability of the discriminator judging whether the input is a real sample. The greater the difference between the output C(x,y) and the output C(G(y),y), the more accurately the discriminator can distinguish between real samples and generated samples.

[0062] The local discriminator randomly selects 10 different sound signal segments for discrimination, and judges the degree of recovery of the generated sound signal from the local point of view. The mathematical description is as follows:

[0063] D_fake=σ(C(G(y),y)-C(x,y)),

[0064] Among them, the meanings of σ, C(x,y), C(G(y),y)G(y) are consistent with the mathematical description in the global discriminator. However, the goal of the local discriminator is to encourage the discriminator to accurately identify the generated false sound signal as a generated sample by minimizing the output difference of the generated samples.

[0065] like Figure 2 As shown in the figure, CP-GAN contains a densely connected feature pyramid generator and a dynamic context granularity discriminator. The noisy signal is first input into the generator composed of dense blocks and feature pyramid modules to generate a denoised signal. Then the generated denoised signal and the original acoustic signal will be used as the input of the global discriminator. The generated local segments of the denoised signal and the local segments of the original acoustic signal will be used as the input of the local discriminator to judge the quality of the enhanced acoustic signal in a hierarchical manner.

[0066] c3 inputs the obtained data set into the improved generative adversarial network CP-GAN in steps c1 and c2 for training to obtain an underwater acoustic signal enhancement model for the target waters.

[0067] S3 receives the acoustic signal from the transmitting device, and after preprocessing by the LMS adaptive filtering method, it is input into the CP-GAN network to enhance the acoustic signal quality, and the enhanced acoustic signal is transmitted back to the receiving device.

[0068] Furthermore, S3 includes the following steps:

[0069] d1 receives the acoustic signal X from the transmitting device in real time, and pre-processes the acoustic signal using the LMS adaptive filtering method to obtain the acoustic signal x with some noise eliminated;

[0070] d2 inputs the acoustic signal x into the trained CP-GAN acoustic signal enhancement model to enhance the acoustic signal quality and obtain the enhanced acoustic signal y;

[0071] d3 transmits the sound signal y back to the sound signal receiving device.

[0072] In another embodiment, an electronic device is also provided, which includes: one or more processors; a memory; and one or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the one or more processors, and the one or more applications are configured to execute the above-mentioned underwater acoustic signal enhancement method based on a generative adversarial network.

[0073] In another embodiment, a computer-readable storage medium is provided, which includes: a computer program stored therein that can be loaded by a processor and execute the above-mentioned underwater acoustic signal enhancement method based on a generative adversarial network.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for underwater acoustic signal enhancement based on generative adversarial network, characterized in that: include: S1 uses acoustic signal transmitting equipment and receiving equipment to collect underwater noisy signals in the target waters, and uses LMS adaptive filtering method to pre-process and reduce noise of the noisy signals to produce an underwater acoustic signal dataset; S2 uses the underwater acoustic signal dataset to train an adaptive generative adversarial network CP-GAN specifically for underwater acoustic signal enhancement; S3 receives the acoustic signal from the transmitting device, and after preprocessing by the LMS adaptive filtering method, it is input into the CP-GAN network to enhance the acoustic signal quality, and the enhanced acoustic signal is transmitted back to the receiving device.

2. The underwater acoustic signal enhancement method based on a generative adversarial network according to claim 1, characterized in that: The acquisition of underwater noise signals in the target water area in S1 includes the following steps: a1 denotes the local acoustic signal collected for the nth time as X(n), and the noisy signal obtained after the local acoustic signal X(n) is transmitted to the receiving device through the acoustic signal transmitting device is x(n); a2 transmits the acoustic signal X(n) to the underwater acoustic signal receiving device through the land acoustic signal transmitting device, and obtains the acoustic signal x(n) with noise; the local acoustic signal is recorded as X(n+1), and the acoustic signal X(n+1) is transmitted to the land acoustic signal receiving device through the underwater acoustic signal transmitting device, and the acoustic signal x(n+1) with noise is obtained; a3 Repeat sub-steps a1 and a2 until the amount of data reaches more than 100,000, and store the recorded data.

3. The underwater acoustic signal enhancement method based on generative adversarial network according to claim 2, characterized in that: The steps of making underwater acoustic signal dataset in S1 are as follows: b1 construct an LMS adaptive filter in the acoustic signal receiving device; b2 Input the noisy signal x(n) into the LMS adaptive filter for processing to obtain the filtered signal y(n), where the local signal X(n) collected for the nth time in step a1 is the target real sound signal, y(n) is the noisy signal after filtering, and y(n) and X(n) are the nth data of the sound signal data set; b3 Repeat step b2 until all collected data are processed and a complete acoustic signal data set is obtained.

4. The underwater acoustic signal enhancement method based on a generative adversarial network according to claim 1, characterized in that: S2 includes the following steps: c1 improves the generator loss function of the generative adversarial network GAN as follows: Loss_CGAN(G)=-E_y[log D(G(y),y)]+∥G(y)-x∥, Where y represents the conditional signal input to the generator G as a condition, G(y) represents the false sound signal generated by the generator G for y, x represents the real sound signal in the training data, -E_y[log D(G(y),y)] represents the negative logarithmic expectation of the discrimination probability obtained by inputting the false sound signal G(y) generated by the generator G and the conditional signal y into the discriminator D, ∥G(y)-x∥ is the reconstruction loss term of the generator, which represents the difference between the generated false sound signal G(y) and the real sound signal x; C2 uses a multi-granularity discriminator based on dynamic context information to discriminate segments of enhanced acoustic signals: The global discriminator discriminates the output of the generator, outputs the probability that the generated sound signal is a real sound signal, and judges the effect of sound signal recovery as a whole. Its mathematical expression is as follows: D_real=σ(C(x,y)-C(G(y),y)), Among them, σ represents the sigmoid function, which maps the input value to a probability value between 0 and 1. C(x,y) represents the output of the discriminator for the real sound signal x and the conditional signal y, that is, the confidence or probability of the discriminator judging whether the input is a real sample. C(G(y),y) represents the output of the discriminator for the false sound signal G(y) and the conditional signal y generated by the generator, that is, the confidence or probability of the discriminator judging whether the input is a real sample. The greater the difference between the output C(x,y) and the output C(G(y),y), the more accurately the discriminator can distinguish between real samples and generated samples. The local discriminator randomly selects 10 different sound signal segments for discrimination, and judges the degree of recovery of the generated sound signal from the local point of view. The mathematical description is as follows: D_fake=σ(C(G(y),y)-C(x,y)), Among them, the meanings of σ, C(x,y), C(G(y),y)G(y) are consistent with the mathematical description in the global discriminator. However, the goal of the local discriminator is to encourage the discriminator to accurately identify the generated false sound signal as a generated sample by minimizing the output difference of the generated samples. c3 inputs the obtained data set into the improved generative adversarial network CP-GAN in steps c1 and c2 for training to obtain an underwater acoustic signal enhancement model for the target waters.

5. The underwater acoustic signal enhancement method based on generative adversarial network according to claim 1, characterized in that: S3 includes the following steps: d1 receives the acoustic signal X from the transmitting device in real time, and pre-processes the acoustic signal using the LMS adaptive filtering method to obtain the acoustic signal x with some noise eliminated; d2 inputs the acoustic signal x into the trained CP-GAN acoustic signal enhancement model to enhance the acoustic signal quality and obtain the enhanced acoustic signal y; d3 transmits the sound signal y back to the sound signal receiving device.

6. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in a memory and configured to be executed by one or more processors, and the one or more applications are configured to execute the underwater acoustic signal enhancement method based on a generative adversarial network according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: include: The computer program is stored which can be loaded by a processor and execute the underwater acoustic signal enhancement method based on a generative adversarial network as claimed in any one of claims 1 to 5.

Images