Signal processing model training method, signal processing method, device and equipment

Abstract

The invention relates to a signal processing model training method, a signal processing method, a signal processing device and signal processing equipment, and relates to the technical field of signal processing. According to the method, through cooperative work of an encoder and a decoder, effective information in a logarithmic time-frequency spectrum can be better mined, and on this basis, encoding features and decoding features are processed through a cross attention mechanism to obtain attention features, so that the encoding features and the decoding features can fully interact with each other, and the accuracy of the encoding features and the decoding features is improved. The more critical information of each other is concerned, so that the signal processing model can more efficiently capture the association between different feature levels of the sound signal, and the perceptual ability of the target features is improved; on the basis, decoding features and attention features are fused to obtain a prediction matrix, and feature information in different processing stages is further integrated, so that the prediction matrix can more comprehensively and accurately reflect whether target features exist in sound signals, and the performance of a signal processing model is improved.

Description

Technical Field

[0001] This disclosure relates to the field of signal processing technology, and in particular to a training method for a signal processing model, a signal processing method, an apparatus, and a device. Background Technology

[0002] Acoustic monitoring can be applied in various fields, such as marine exploration and equipment condition monitoring. For example, acoustic monitoring can be used to identify the presence of target features in acoustic signals to determine whether equipment is damaged.

[0003] In related technologies, when identifying target features in sound signals, the presence of noise in the sound signals and the relatively weak and variable frequency of the target features make it impossible to accurately identify the target features when manually analyzing the sound signals to identify them. Summary of the Invention

[0004] This disclosure provides a training method for a signal processing model, a signal processing method, an apparatus, and a device. The technical solution of this disclosure is as follows.

[0005] According to one aspect of the embodiments of this disclosure, a method for training a signal processing model is provided, the method comprising: Multiple sets of training samples are obtained. Each set of training samples includes a first logarithmic time spectrum of an acoustic signal and a label matrix of the first logarithmic time spectrum. The acoustic signal includes target features. The element values ​​of multiple target positions in the label matrix are target values. The multiple target positions are used to indicate multiple coordinate positions on the frequency band trajectory in the first logarithmic time spectrum. The frequency band trajectory is used to indicate the target features. For each training sample, the encoder of the signal processing model processes the first logarithmic time-spectrum of the training sample to obtain a first encoded feature. The decoder of the signal processing model processes the first encoded feature to obtain a first decoded feature. The attention module of the signal processing model performs cross-attention processing on the first encoded feature and the first decoded feature to obtain a first attention feature. The signal processing model fuses the first decoded feature and the first attention feature to obtain a first prediction matrix. The element values ​​at each position in the first prediction matrix are used to determine whether the target feature exists in the acoustic signal. The signal processing model is trained based on the label matrix and the first prediction matrix of each of the multiple sets of training samples.

[0006] In some embodiments, the first encoded feature includes sub-features for each of multiple channels, and the first attention feature is obtained by performing cross-attention processing on the first encoded feature and the first decoded feature through the attention module of the signal processing model, including: The attention module processes the sub-features of each of the multiple channels to obtain the weights of each of the multiple channels. The attention module performs cross-attention processing on the first encoded feature and the first decoded feature to obtain a second attention feature, which includes the sub-attention features of each of the multiple channels. The attention module weights the sub-attention features of each of the multiple channels based on their respective weights to obtain the first attention feature.

[0007] In some embodiments, the step of performing cross-attention processing on the first encoded feature and the first decoded feature through the attention module to obtain the second attention feature includes: The first encoded feature is processed using the query data in the attention module to obtain the query feature; the first decoded feature is processed using the key data in the attention module to obtain the key feature; and the first decoded feature is processed using the value data in the attention module to obtain the value feature. The second attention feature is obtained by processing the query feature, the key feature, and the value feature through the attention module.

[0008] In some embodiments, the process of processing the query feature, the key feature, and the value feature through the attention module to obtain the second attention feature includes: The attention module is used to divide the query feature, the key feature, and the value feature into their respective feature sequences, each of which includes multiple sub-features arranged in order. For each feature group, based on the sub-query features, sub-key features, and sub-value features in the feature group, the corresponding sub-attention features are determined. Each feature group includes sub-features at the same position in multiple feature sequences. The second attention feature is obtained by concatenating the sub-attention features of each feature group.

[0009] In some embodiments, the signal processing model includes multiple network layers, each network layer including an encoder, a decoder, and an attention module. The first coded feature, the first decoded feature, and the first attention feature in the same network layer have the same size, and the multiple network layers correspond to different sizes. The total number of the multiple network layers is M. The input of the encoder of the first network layer is the first logarithmic time spectrum. The input of the encoder of the j-th network layer is the first coded feature output by the encoder of the (j-1)-th network layer. The input of the decoder of the M-th network layer is the first coded feature output by the encoder of the M-th network layer. The input of the decoder of the j-th network layer is the fused feature output by the decoder of the (j+1)-th network layer. The fused feature is obtained based on the first decoded feature and the first attention feature output by the (j+1)-th network layer. M is a positive integer greater than 1, and j is a positive integer greater than 1 and less than or equal to M.

[0010] In some embodiments, each group of training samples further includes a second logarithmic time spectrum, wherein the second logarithmic time spectrum is the first logarithmic time spectrum of image edge binarization, and the method further includes: For each training sample, the encoder processes the second logarithmic time spectrum of the training sample to obtain a second encoded feature. The decoder processes the second encoded feature to obtain a second decoded feature. The attention module performs cross-attention processing on the second encoded feature and the second decoded feature to obtain a third attention feature. The signal processing model fuses the second decoded feature and the third attention feature to obtain a second prediction matrix. The step of training the signal processing model based on the label matrices and first prediction matrices of the multiple sets of training samples includes: The signal processing model is trained based on the label matrix, first prediction matrix, and second prediction matrix of each of the multiple sets of training samples.

[0011] In some embodiments, training the signal processing model based on the label matrix, the first prediction matrix, and the second prediction matrix of each of the multiple sets of training samples includes: For each set of training samples, the first prediction matrix and the second prediction matrix are fused to obtain the third prediction matrix; Based on the label matrix and the third prediction matrix, a loss value is determined, and the signal processing model is trained based on the loss value. The loss value is used to indicate the difference information between the third prediction matrix and the label matrix.

[0012] In some embodiments, training the signal processing model based on the label matrix, the first prediction matrix, and the second prediction matrix of each of the multiple sets of training samples includes: For each training sample, a first loss value is determined based on the label matrix and the first prediction matrix. The first loss value is used to indicate the difference information between the first prediction matrix and the label matrix. Based on the label matrix and the second prediction matrix, a second loss value is determined, which is used to indicate the difference information between the second prediction matrix and the label matrix; The signal processing model is trained based on the first loss value and the second loss value.

[0013] In some embodiments, each group of training samples further includes a second logarithmic time spectrum, wherein the second logarithmic time spectrum is the first logarithmic time spectrum of image edge binarization, and the method further includes: For each set of training samples, the first logarithmic time spectrum and the second logarithmic time spectrum in the training samples are concatenated to obtain the concatenated logarithmic time spectrum, which is then input into the signal processing model. The signal processing model's encoder processes the spliced ​​logarithmic time-spectrum of the training samples to obtain a third encoded feature. The signal processing model's decoder processes the third encoded feature to obtain a third decoded feature. The signal processing model's attention module performs cross-attention processing on the third encoded feature and the third decoded feature to obtain a fourth attention feature. The signal processing model fuses the third decoded feature and the fourth attention feature to obtain a fourth prediction matrix. The element values ​​at each position in the fourth prediction matrix are used to determine whether the target feature exists in the acoustic signal. The signal processing model is trained based on the label matrix and fourth prediction matrix of each of the multiple sets of training samples.

[0014] In some embodiments, obtaining multiple sets of training samples includes: Acquire a target acoustic signal of a preset duration, wherein the target acoustic signal includes the target features; Multiple segments of sound signal are extracted from the target sound signal based on a preset time window; A tag matrix is ​​determined based on multiple coordinate positions on the frequency band trajectory of each sound signal, wherein the frequency band trajectory is used to indicate the target features; The multiple sets of training samples are obtained based on the first logarithmic time spectrum and label matrix of each of the multiple acoustic signals.

[0015] According to another aspect of the embodiments of this disclosure, a signal processing method is provided, the method comprising: Obtain the logarithmic time spectrum of the target acoustic signal; The target encoded features are obtained by processing the logarithmic time-spectrum using the encoder of the signal processing model. The target encoded features are then processed by the decoder of the signal processing model to obtain target decoded features. The target encoded features and the target decoded features are then cross-attention processed by the attention module of the signal processing model to obtain target attention features. Finally, the target decoded features and the target attention features are fused by the signal processing model to obtain a target prediction matrix. The element values ​​at each position in the target prediction matrix are used to determine whether the target features exist in the acoustic signal. The signal processing model is a signal processing model trained through any of the above embodiments.

[0016] In some embodiments, the method further includes at least one of the following: The target acoustic signal is an acoustic signal generated during the operation of the equipment. The target feature is used to indicate the target fault. If the target feature is determined to exist in the target acoustic signal based on the element values ​​at each position in the target prediction matrix, then it is determined that the target fault is detected in the equipment. The target acoustic signal is an acoustic signal generated on the seabed, and the target feature is used to indicate a school of fish. If the target feature is determined to exist in the target acoustic signal based on the element values ​​at each position in the target prediction matrix, then it is determined that a school of fish has been detected on the seabed.

[0017] According to another aspect of the embodiments of this disclosure, a training apparatus for a signal processing model is provided, the apparatus comprising: The acquisition module is used to acquire multiple sets of training samples. Each set of training samples includes a first logarithmic time spectrum of an acoustic signal and a label matrix of the first logarithmic time spectrum. The acoustic signal includes target features. The element values ​​of multiple target positions in the label matrix are target values. The multiple target positions are used to indicate multiple coordinate positions on the frequency band trajectory in the first logarithmic time spectrum. The frequency band trajectory is used to indicate the target features. The processing module is used to process the first logarithmic time-spectrum of the training samples in each training sample by the encoder of the signal processing model to obtain a first encoded feature, process the first encoded feature by the decoder of the signal processing model to obtain a first decoded feature, perform cross-attention processing on the first encoded feature and the first decoded feature by the attention module of the signal processing model to obtain a first attention feature, and fuse the first decoded feature and the first attention feature by the signal processing model to obtain a first prediction matrix. The element values ​​at each position in the first prediction matrix are used to determine whether the target feature exists in the acoustic signal. The training module is used to train the signal processing model based on the label matrix and the first prediction matrix of each of the multiple sets of training samples.

[0018] In some embodiments, the first encoded feature includes sub-features for each of multiple channels, and the processing module is configured to: The attention module processes the sub-features of each of the multiple channels to obtain the weights of each of the multiple channels. The attention module performs cross-attention processing on the first encoded feature and the first decoded feature to obtain a second attention feature, which includes the sub-attention features of each of the multiple channels. The attention module weights the sub-attention features of each of the multiple channels based on their respective weights to obtain the first attention feature.

[0019] In some embodiments, the processing module is configured to: The first encoded feature is processed using the query data in the attention module to obtain the query feature; the first decoded feature is processed using the key data in the attention module to obtain the key feature; and the first decoded feature is processed using the value data in the attention module to obtain the value feature. The second attention feature is obtained by processing the query feature, the key feature, and the value feature through the attention module.

[0020] In some embodiments, the processing module is configured to: The attention module is used to divide the query feature, the key feature, and the value feature into their respective feature sequences, each of which includes multiple sub-features arranged in order. For each feature group, based on the sub-query features, sub-key features, and sub-value features in the feature group, the corresponding sub-attention features are determined. Each feature group includes sub-features at the same position in multiple feature sequences. The second attention feature is obtained by concatenating the sub-attention features of each feature group.

[0021] In some embodiments, the signal processing model includes multiple network layers, each network layer including an encoder, a decoder, and an attention module. The first coded feature, the first decoded feature, and the first attention feature in the same network layer have the same size, and the multiple network layers correspond to different sizes. The total number of the multiple network layers is M. The input of the encoder of the first network layer is the first logarithmic time spectrum. The input of the encoder of the j-th network layer is the first coded feature output by the encoder of the (j-1)-th network layer. The input of the decoder of the M-th network layer is the first coded feature output by the encoder of the M-th network layer. The input of the decoder of the j-th network layer is the fused feature output by the decoder of the (j+1)-th network layer. The fused feature is obtained based on the first decoded feature and the first attention feature output by the (j+1)-th network layer. M is a positive integer greater than 1, and j is a positive integer greater than 1 and less than or equal to M.

[0022] In some embodiments, each group of training samples further includes a second logarithmic time spectrum, wherein the second logarithmic time spectrum is the first logarithmic time spectrum of image edge binarization, and the processing module is further configured to: For each training sample, the encoder processes the second logarithmic time spectrum of the training sample to obtain a second encoded feature. The decoder processes the second encoded feature to obtain a second decoded feature. The attention module performs cross-attention processing on the second encoded feature and the second decoded feature to obtain a third attention feature. The signal processing model fuses the second decoded feature and the third attention feature to obtain a second prediction matrix. The training module is also used for: The signal processing model is trained based on the label matrix, first prediction matrix, and second prediction matrix of each of the multiple sets of training samples.

[0023] In some embodiments, the processing module is configured to: For each set of training samples, the first prediction matrix and the second prediction matrix are fused to obtain the third prediction matrix; Based on the label matrix and the third prediction matrix, a loss value is determined, and the signal processing model is trained based on the loss value. The loss value is used to indicate the difference information between the third prediction matrix and the label matrix.

[0024] In some embodiments, the training module is configured to: For each training sample, a first loss value is determined based on the label matrix and the first prediction matrix. The first loss value is used to indicate the difference information between the first prediction matrix and the label matrix. Based on the label matrix and the second prediction matrix, a second loss value is determined, which is used to indicate the difference information between the second prediction matrix and the label matrix; The signal processing model is trained based on the first loss value and the second loss value.

[0025] In some embodiments, each set of training samples further includes a second logarithmic time spectrum, the second logarithmic time spectrum being the first logarithmic time spectrum of image edge binarization, and the apparatus further includes a stitching module for: For each set of training samples, the first logarithmic time spectrum and the second logarithmic time spectrum in the training samples are concatenated to obtain the concatenated logarithmic time spectrum, which is then input into the signal processing model. The processing module is further configured to process the spliced ​​logarithmic time-spectrum of the training samples through the encoder of the signal processing model to obtain a third coded feature, process the third coded feature through the decoder of the signal processing model to obtain a third decoded feature, perform cross-attention processing on the third coded feature and the third decoded feature through the attention module of the signal processing model to obtain a fourth attention feature, and fuse the third decoded feature and the fourth attention feature through the signal processing model to obtain a fourth prediction matrix. The element values ​​at each position in the fourth prediction matrix are used to determine whether the target feature exists in the acoustic signal. The training module is used to train the signal processing model based on the label matrix and the fourth prediction matrix of each of the multiple sets of training samples.

[0026] In some embodiments, the acquisition module is configured to: Acquire a target acoustic signal of a preset duration, wherein the target acoustic signal includes the target features; Multiple segments of sound signal are extracted from the target sound signal based on a preset time window; A tag matrix is ​​determined based on multiple coordinate positions on the frequency band trajectory of each sound signal, wherein the frequency band trajectory is used to indicate the target features; The multiple sets of training samples are obtained based on the first logarithmic time spectrum and label matrix of each of the multiple acoustic signals.

[0027] According to another aspect of the embodiments of this disclosure, a training apparatus for a signal processing model is provided, the apparatus comprising: The acquisition module is used to acquire the logarithmic time spectrum of the target acoustic signal; The processing module is used to process the logarithmic time spectrum through the encoder of the signal processing model to obtain target encoded features, process the target encoded features through the decoder of the signal processing model to obtain target decoded features, perform cross-attention processing on the target encoded features and the target decoded features through the attention module of the signal processing model to obtain target attention features, and fuse the target decoded features and the target attention features through the signal processing model to obtain a target prediction matrix. The element values ​​at each position in the target prediction matrix are used to determine whether the target features exist in the acoustic signal. The signal processing model is a signal processing model trained through any of the above embodiments.

[0028] In some embodiments, the apparatus further includes a determining module for performing at least one of the following: The target acoustic signal is an acoustic signal generated during the operation of the equipment. The target feature is used to indicate the target fault. If the target feature is determined to exist in the target acoustic signal based on the element values ​​at each position in the target prediction matrix, then it is determined that the target fault is detected in the equipment. The target acoustic signal is an acoustic signal generated on the seabed, and the target feature is used to indicate a school of fish. If the target feature is determined to exist in the target acoustic signal based on the element values ​​at each position in the target prediction matrix, then it is determined that a school of fish has been detected on the seabed.

[0029] According to another aspect of the embodiments of this disclosure, an electronic device is provided, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the training method or signal processing method of the above-described signal processing model.

[0030] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the training method or signal processing method of the above-described signal processing model.

[0031] According to another aspect of the present disclosure, a computer program product is provided, the computer program product including a computer program that, when executed by a processor, implements the training method or signal processing method of the above-described signal processing model.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0033] This application provides a training method for a signal processing model. The method uses the logarithmic time-frequency spectrum of an acoustic signal including target features and the corresponding label matrix as training samples. The logarithmic time-frequency spectrum is input into the signal processing model, and encoded and decoded features are obtained through an encoder and a decoder. Since the encoder can extract more representative and abstract feature representations from the logarithmic time-frequency spectrum, and the decoder can reconstruct feature representations related to the original features by decoding the encoded features, the collaborative work of the encoder and decoder can better extract effective information from the logarithmic time-frequency spectrum. Based on this, the encoded and decoded features are processed through a cross-attention mechanism to obtain attention features, enabling full interaction between the encoded and decoded features and focusing on each other's more critical information. This allows the signal processing model to more efficiently capture the correlation between different feature levels of the acoustic signal, improving its ability to perceive target features. Furthermore, the decoded features and attention features are fused to obtain a prediction matrix, further integrating feature information from different processing stages. This allows the prediction matrix to more comprehensively and accurately reflect the presence of target features in the acoustic signal, improving the performance of the signal processing model. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0035] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application.

[0036] Figure 2 This is a flowchart of a signal processing model training method provided in an embodiment of this application.

[0037] Figure 3 This is a schematic diagram of an acoustic signal and its logarithmic time spectrum provided in an embodiment of this application.

[0038] Figure 4 This is a flowchart of a signal processing model training method provided in an embodiment of this application.

[0039] Figure 5 This is a flowchart of an acoustic signal processing method provided in an embodiment of this application.

[0040] Figure 6 This is a schematic diagram of a cross-head attention mechanism provided in an embodiment of this application.

[0041] Figure 7 This is a flowchart of a signal processing model provided in an embodiment of this application.

[0042] Figure 8This is a schematic diagram of a first logarithmic time spectrum and a second logarithmic time spectrum provided in an embodiment of this application.

[0043] Figure 9 This is a flowchart of a signal processing model provided in an embodiment of this application.

[0044] Figure 10 This is a flowchart illustrating the training and use of a signal processing model provided in an embodiment of this application.

[0045] Figure 11 This is a flowchart illustrating the training and use of a signal processing model provided in an embodiment of this application.

[0046] Figure 12 This is a flowchart of a signal processing method provided in an embodiment of this application.

[0047] Figure 13 This is a block diagram of a training device for a signal processing model provided in an embodiment of this application.

[0048] Figure 14 This is a block diagram of a signal processing device provided in an embodiment of this application.

[0049] Figure 15 This is a block diagram of a terminal provided in an embodiment of this application.

[0050] Figure 16 This is a block diagram of a server provided in an embodiment of this application. Detailed Implementation

[0051] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0052] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0053] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the sound signals involved in this disclosure were all obtained with full authorization.

[0054] The signal processing model training method provided in this application can be executed by a computer device, which can be at least one of a terminal and a server. The following is a schematic diagram illustrating the implementation environment of the signal processing model training method provided in this application. See also... Figure 1 , Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application. The implementation environment includes: terminal 101 and server 102.

[0055] In some embodiments, terminal 101 may run a client application with a target application that provides a function for monitoring based on sound signals. This application does not limit the implementation form of the target application; for example, it may be an application that requires downloading and installation, a mini-program that does not require installation, a web application, etc.

[0056] In some embodiments, terminal 101 can be installed in the scene to be monitored to acquire acoustic signals in the scene. Optionally, during the model training phase, terminal 101 transmits the acquired acoustic signals including target features to server 102. Server 102 processes the acoustic signals to obtain multiple sets of training samples and trains a signal processing model based on these training samples. After training the signal processing model, the trained model is deployed to the server. During the model application phase, terminal 101 transmits the acquired acoustic signals to server 102. The server processes the acoustic signals using the deployed signal processing model to identify whether the acoustic signals include target features. Alternatively, terminal 101 itself may have a trained signal processing model deployed on it. Terminal 101 processes the acoustic signals using the deployed signal processing model to identify whether the acoustic signals include target features.

[0057] In the embodiments of this application, the monitoring scenario can be set as needed. For example, if the monitoring scenario is a scenario for monitoring whether the equipment is faulty, the target feature is the whistle sound generated by the equipment fault; or if the monitoring scenario is a scenario for locating schools of fish on the seabed, the target feature is the whistle sound generated by the schools of fish on the seabed.

[0058] Terminal 101 can be a computer device such as an acoustic sensor with signal processing and output display functions, a mobile phone, a tablet computer, a multimedia playback device, a PC (Personal Computer), a wearable device, a VR (Virtual Reality) device, an AR (Augmented Reality) device, or a MR (Mixed Reality) device. Server 102 can be a standalone physical server, a server cluster consisting of multiple physical servers, or a distributed file system. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 102 and terminal 101 are directly or indirectly connected via wired or wireless communication.

[0059] Figure 2 This is a flowchart of a signal processing model training method provided in an embodiment of this application, such as... Figure 2 As shown, this method is executed by a terminal or server, and includes the following steps.

[0060] 201. Obtain multiple sets of training samples. Each set of training samples includes the first logarithmic time spectrum of the acoustic signal and the label matrix of the first logarithmic time spectrum. The acoustic signal includes target features. The element values ​​of multiple target positions in the label matrix are target values. Multiple target positions are used to indicate multiple coordinate positions on the frequency band trajectory in the first logarithmic time spectrum. The frequency band trajectory is used to indicate target features.

[0061] The duration of the acoustic signal in each training sample can be set as needed. Each training sample includes target features, and these features can be the same or different across different samples, allowing the trained model to recognize multiple target features in the acoustic signal. For example, some training samples may include whistles caused by equipment malfunctions, while others may include whistles produced by schools of fish on the seabed.

[0062] Optionally, the first logarithmic time spectrum is a logarithmic time spectrum, where the horizontal axis represents time and the vertical axis represents frequency. The logarithmic time spectrum is a two-dimensional image, where the pixel brightness at each coordinate position represents the signal energy intensity at that coordinate position; that is, for each coordinate position, the higher the energy value, the higher the brightness. When a target feature exists in the acoustic signal, the energy of this target feature is mostly locked within a narrow frequency range, resulting in high energy. Therefore, this forms a frequency band trajectory in the logarithmic time spectrum, which indicates the target feature. Optionally, the frequency band trajectory is a bright band. Since higher brightness corresponds to higher energy values, the energy values ​​at multiple coordinate positions on the frequency band trajectory may exceed a certain energy threshold.

[0063] It should be noted that the target features exhibit frequency variation in the time-frequency domain, meaning the signal frequency is fixed but changes over time. Therefore, the frequency band trajectory appears as an irregular curve (bright band). For example, the whistling sound produced by a broken wind turbine blade exhibits a sine wave frequency band trajectory in the logarithmic time spectrum, while the whistling sound produced by a school of fish on the seabed exhibits a multiple harmonic curve frequency band trajectory in the logarithmic time spectrum.

[0064] For example, see Figure 3 , Figure 3 This is a schematic diagram of an acoustic signal and its logarithmic time spectrum provided in an embodiment of this application. The upper part of the left-hand diagram shows an acoustic signal including the whistling sound produced by a broken wind turbine blade, and the lower part shows the logarithmic time spectrum of this acoustic signal. The frequency band trajectory in the logarithmic time spectrum also indicates the whistling sound produced by the broken wind turbine blade. The upper part of the right-hand diagram shows an acoustic signal including the whistling sound produced by a school of fish on the seabed, and the lower part shows the logarithmic time spectrum of this acoustic signal. The frequency band trajectory in the logarithmic time spectrum also indicates the whistling sound produced by a school of fish on the seabed.

[0065] Optionally, the first logarithmic time spectrum is a matrix, and the label matrix has the same size as the first logarithmic time spectrum, or the size of the label matrix and the first logarithmic time spectrum are in a preset multiple relationship, which is not specifically limited here.

[0066] Taking an example where the size of the tag matrix is ​​the same as the size of the first logarithmic time-frequency spectrum, multiple positions in the tag matrix correspond one-to-one with multiple positions in the first logarithmic time-frequency spectrum. Multiple coordinate positions on the frequency band trajectory in the first logarithmic time-frequency spectrum are mapped to the tag matrix to determine multiple target positions corresponding to these coordinate positions. The element values ​​of these multiple target positions are then defined as target values. Target values ​​can be set as needed, such as a target value of 1. The element values ​​at positions in the tag matrix other than the target positions differ from the target values; for example, the element values ​​at these positions are 0.

[0067] 202. For each training sample, the encoder of the signal processing model processes the first logarithmic time spectrum of the training sample to obtain the first encoded feature. The decoder of the signal processing model processes the first encoded feature to obtain the first decoded feature. The attention module of the signal processing model performs cross-attention processing on the first encoded feature and the first decoded feature to obtain the first attention feature. The signal processing model fuses the first decoded feature and the first attention feature to obtain the first prediction matrix. The element values ​​at each position in the first prediction matrix are used to determine whether the target feature exists in the acoustic signal.

[0068] Both the first and second coding features can be matrices, and both can include matrices with multiple channels, where a matrix with one channel represents a feature of one dimension.

[0069] The first encoded feature is an abstract representation output by the encoder after processing the first logarithmic time spectrum. The encoder is used to compress and reduce the dimensionality of the input first logarithmic time spectrum. For example, the first logarithmic time spectrum contains a lot of redundant information. The encoder transforms the high-dimensional first logarithmic time spectrum into a low-dimensional encoded feature through operations such as convolution and pooling. This retains the key information in the first logarithmic time spectrum while removing redundant information.

[0070] The decoder is used to restore the abstract encoded features. Through operations such as deconvolution and upsampling, the decoder restores the low-dimensional encoded features to features with a spectral dimension similar to or a specific dimension of the first logarithm of the input, and supplements detailed information.

[0071] 203. Train the signal processing model based on the label matrix and the first prediction matrix of each of the multiple training samples.

[0072] Optionally, with the goal of minimizing the difference between the label matrix and the first prediction matrix, the signal processing model is trained iteratively so that the matrix predicted by the signal processing model gradually approaches the label matrix.

[0073] The signal processing model is iteratively trained based on the label matrices and first prediction matrices of multiple training samples until an iteration stopping condition is met. In each iteration, one or more training samples can be used. Optionally, the iteration stopping condition includes at least one of the following: the number of iterations reaches a preset number, the first prediction matrix converges, or the loss value between the first prediction matrix and the label matrix is ​​less than a preset value.

[0074] This application provides a training method for a signal processing model. The method uses the logarithmic time-frequency spectrum of an acoustic signal including target features and the corresponding label matrix as training samples. The logarithmic time-frequency spectrum is input into the signal processing model, and encoded and decoded features are obtained through an encoder and a decoder. Since the encoder can extract more representative and abstract feature representations from the logarithmic time-frequency spectrum, and the decoder can reconstruct feature representations related to the original features by decoding the encoded features, the collaborative work of the encoder and decoder can better extract effective information from the logarithmic time-frequency spectrum. Based on this, the encoded and decoded features are processed through a cross-attention mechanism to obtain attention features, enabling full interaction between the encoded and decoded features and focusing on each other's more critical information. This allows the signal processing model to more efficiently capture the correlation between different feature levels of the acoustic signal, improving its ability to perceive target features. Furthermore, the decoded features and attention features are fused to obtain a prediction matrix, further integrating feature information from different processing stages. This allows the prediction matrix to more comprehensively and accurately reflect the presence of target features in the acoustic signal, improving the performance of the signal processing model.

[0075] The above Figure 2 The diagram shown is merely the basic flow of this application. The following section, based on a specific implementation method, further elaborates on the solution provided in this application. See also... Figure 4 , Figure 4 This is a flowchart of a training method for a signal processing model provided in an embodiment of this application. The method is executed by a terminal and includes at least one of the following steps.

[0076] 401. Obtain multiple sets of training samples. Each set of training samples includes the first logarithmic time spectrum of the acoustic signal and the label matrix of the first logarithmic time spectrum. The acoustic signal includes target features. The element values ​​of multiple target positions in the label matrix are target values. The multiple target positions are used to indicate multiple coordinate positions on the frequency band trajectory in the first logarithmic time spectrum. The frequency band trajectory is used to indicate target features.

[0077] In some embodiments, the acoustic signals in multiple training samples come from the same long acoustic signal. The process of acquiring the multiple training samples includes the following steps: acquiring a target acoustic signal of a preset duration, wherein the target acoustic signal includes target features; extracting multiple segments of acoustic signals from the target acoustic signal based on a preset time window; determining a label matrix based on multiple coordinate positions on the frequency band trajectory in the first logarithmic time spectrum of each segment of acoustic signal; and obtaining multiple training samples based on the first logarithmic time spectrum and label matrix of each segment of acoustic signal.

[0078] Specifically, based on multiple coordinate positions on the frequency band trajectory in the first logarithmic time spectrum, multiple target positions corresponding to these multiple coordinate positions are determined in the tag matrix, and the element values ​​of the multiple target positions are assigned as target values.

[0079] In this embodiment, the example of multiple training samples originating from the same long acoustic signal is used for illustration. In other embodiments, multiple target acoustic signals of preset durations can be acquired, and these multiple target acoustic signals include different target features. Multiple training samples are obtained based on each target acoustic signal, and a signal processing model is trained based on these multiple training samples of each target acoustic signal.

[0080] In the above embodiments, the fixed duration of multiple sound signals is used as an example. In other embodiments, the duration of multiple sound signals may be different, that is, multiple sound signals are extracted according to different time windows, thereby enhancing the diversity of training samples.

[0081] In some embodiments, noise removal can also be performed on the target acoustic signal, which can be achieved by filtering or downsampling the target acoustic signal. Optionally, whether to downsample the target acoustic signal is determined based on the frequency band range where the target features are located in the target acoustic signal.

[0082] Optionally, the target acoustic signal is first filtered using a digital filter to remove noise other than the target features. If the difference between the frequency band of the noise in the target acoustic signal and the frequency band of the target features is greater than a preset difference, the filtered acoustic signal is then downsampled.

[0083] In some embodiments, the target acoustic signal is also normalized to eliminate dimensions. Optionally, normalization is performed using the following formula (1).

[0084] (1) in, Indicates the target acoustic signal. This indicates taking the average. Indicates the standard deviation. This represents the target acoustic signal after normalization.

[0085] In some embodiments, after capturing multiple segments of the target acoustic signal based on a preset time window, a short-time Fourier transform or wavelet transform is performed on each segment to obtain the time spectrum of the acoustic signal. Optionally, a logarithmic dB transform is performed on the time spectrum to obtain a first logarithmic time spectrum, or, after performing a logarithmic dB transform on the first time spectrum, the obtained time spectrum is normalized again to obtain the first logarithmic time spectrum.

[0086] Among them, performing logarithmic dB transformation on the time spectrum can reduce the difference between the original features and noise of the acoustic signal in the logarithmic time spectrum, as well as amplify and enhance the weak frequency conversion features in the original time spectrum, thereby enhancing the features of the target.

[0087] For example, see Figure 5 , Figure 5 This is a flowchart of an audio signal processing method provided in an embodiment of this application. Multiple segments of the target audio signal are extracted according to a preset time window to obtain the time spectrum of each segment. After determining the logarithmic time spectrum corresponding to the time spectrum, the first logarithmic time spectrum of each segment is obtained.

[0088] 402. For each training sample, the encoder of the signal processing model processes the first logarithmic time spectrum of the training sample to obtain the first encoded feature, and the decoder of the signal processing model processes the first encoded feature to obtain the first decoded feature.

[0089] 403. The attention module of the signal processing model processes the sub-features of each channel of the first encoded feature to obtain the weights of each channel.

[0090] In some embodiments, the attention module performs global average pooling (GAP) on the sub-features of each of the multiple channels to obtain the weights of each channel. Global average pooling aggregates information on the spatial range of each channel, capturing the statistical characteristics of each channel across the entire space. Therefore, the weight of each channel can also be defined as a spatial feature weight.

[0091] Each channel's sub-feature is a two-dimensional feature map (matrix), whose height and width can be defined as follows: and The weight of each channel can be obtained by the following formula (2).

[0092] (2) in, Indicates channel The weight, Indicates channel In the sub-features OK The formula represents the average of the feature values ​​at multiple positions in a sub-feature, calculated by summing the feature values ​​at those positions. All are positive integers greater than 0. Indicates the height of the sub-feature, This represents the width of the sub-feature. Since the sub-features are represented in matrix form, then... That is, the number of rows representing the sub-feature. That is, the number of columns representing the sub-features.

[0093] 404. The first encoded feature and the first decoded feature are cross-attention processed by the attention module to obtain the second attention feature, which includes the sub-attention features of each of the multiple channels.

[0094] In some embodiments, the process of performing cross-attention processing on the first encoded feature and the first decoded feature through the attention module to obtain the second attention feature includes the following steps: processing the first encoded feature with query data in the attention module to obtain a query feature; processing the first decoded feature with key data in the attention module to obtain a key feature; processing the first decoded feature with value data in the attention module to obtain a value feature; and processing the query feature, key feature, and value feature through the attention module to obtain the second attention feature.

[0095] Optionally, the query features, key features, and value features are obtained using the following formula (3): ; ; (3) in, Indicates querying data. The size is , Indicates query characteristics; Represents key data, The size is , Indicates key features; Represents value data, The size is , Indicates value characteristics; Indicates the first coding feature, Expressing the first decoding feature, They are positive integers greater than 0.

[0096] Optionally, the attention module obtains the second attention feature using the following formula (4).

[0097] (4) in, This represents the second attention feature. Indicates transpose. Indicates the dimension of the queried data. This indicates normalization.

[0098] In this embodiment, the encoded features output by the encoder contain global information of the first logarithmic time spectrum of the input, while the decoded features output by the decoder focus more on local detail information. By fusing the two through a cross-attention mechanism, the resulting attention features retain the global information advantage of the encoded features while ensuring the detail accuracy and task relevance of the decoded features. This allows the decoded features to make full use of the global information in the encoded features, thereby improving the accuracy of the generated results.

[0099] In some embodiments, the second attention feature is obtained through a multi-head attention mechanism. The process of processing the query feature, key feature, and value feature through the attention module to obtain the second attention feature includes the following steps: dividing the query feature, key feature, and value feature into their respective feature sequences through the attention module, each feature sequence including multiple sub-features arranged in order; for each feature group, determining the corresponding sub-attention feature based on the sub-query feature, sub-key feature, and sub-value feature in the feature group, each feature group including sub-features at the same position in multiple feature sequences; concatenating the sub-attention features of each feature group to obtain the second attention feature.

[0100] Each feature group corresponds to one attention head, and the number of feature groups is defined as follows: , that is The attention head, then for the first attention head, The subquery features, subkey features, and subvalue features of the head are shown in the following formula (5): ; ; (5) In one iteration of model training, the first logarithmic time-spectrum input signal processing model of multiple sets of training samples can be processed at once, thus obtaining multiple first encoded features. This represents the number of these multiple first-coded features, where n is an integer greater than 0.

[0101] Optionally, the sub-query features, sub-key features, and sub-value features in each feature group are used to obtain the corresponding sub-attention features of the feature group through the above formula (4).

[0102] In this embodiment, different attention heads can focus on different subspaces or different dimensions of feature correlation information. By capturing these different dimensions of correlation in parallel with multiple attention heads, the relationship between encoded and decoded features can be explored more comprehensively and richly, avoiding the one-sided information capture that may exist with a single attention head. Therefore, concatenating these sub-attention features can integrate multiple different feature interaction information, greatly enhancing the signal processing model's ability to express the fusion of encoded and decoded features. This allows the signal processing model to learn more complex and refined feature correlations, thereby improving the performance of the signal processing model.

[0103] 405. Using the attention module, the sub-attention features of multiple channels are weighted based on their respective weights to obtain the first attention feature.

[0104] In some embodiments, the first attention feature is obtained for each channel by the following formula (6).

[0105] (6) in, express A vector composed of the weights of each channel. The second attention feature includes Sub-features of each channel, Let c represent the first attention feature, where c is an integer greater than 0.

[0106] For example, see Figure 6 , Figure 6 This is a schematic diagram of a cross-multi-head attention mechanism provided in an embodiment of this application. Cross-attention processing is performed based on the first encoded features and the first decoded features of multiple channels to obtain a second attention feature that includes sub-attention features of each of the multiple channels. The weights of each of the multiple channels are obtained based on the first encoded features, then weighted to obtain the first attention feature, which is then fused with the first decoded features of the multiple channels.

[0107] In this embodiment, steps 403-405 described above implement the process of performing cross-attention processing on the first encoded feature and the first decoded feature using the attention module of the signal processing model to obtain the first attention feature. It should be noted that this implementation is only one optional method for implementing this process; other methods can also be used, which will not be elaborated upon here.

[0108] In this embodiment, the weight of each channel is calculated using global average pooling. Global average pooling aggregates information across the spatial range of each channel, capturing the statistical characteristics of each channel across the entire space. Therefore, the weights obtained from this aggregation are used to weight the features of each channel, enabling the signal processing model to more fully consider the global contribution of spatial information from each channel when fusing features, thus enhancing its ability to model spatially correlated patterns. Furthermore, by calculating the weight of each channel using global average pooling, it can evaluate each channel of the first encoded feature, identifying those channels containing more critical information. Weighting is then used to enhance the feature representation of channels containing more critical information while suppressing the influence of relatively minor channels.

[0109] 406. The first prediction matrix is ​​obtained by fusing the first decoding feature and the first attention feature through the signal processing model.

[0110] The fusion method of the first decoding feature and the first attention feature can be set as needed. For example, the first decoding feature and the first attention feature can be concatenated to obtain the first prediction matrix. Optionally, after concatenating the first decoding feature and the first attention feature, the concatenated features are normalized using the softmax activation function to obtain a probability prediction matrix, where the element values ​​at multiple positions in the probability prediction matrix are probability values. Optionally, the probability prediction matrix can be directly determined as the first prediction matrix; or the element values ​​at positions in the probability prediction matrix where the probability value is greater than a preset value can be replaced with target values, and the element values ​​at positions in the probability prediction matrix where the probability value is not greater than the preset value can be replaced with non-target values ​​to obtain the first prediction matrix.

[0111] In the above embodiments, the signal processing model is illustrated using a single network layer comprising an encoder, a decoder, and an attention module. In other embodiments, the signal processing model comprises multiple network layers, each comprising an encoder, a decoder, and an attention module. Within the same network layer, the first encoded feature, the first decoded feature, and the first attention feature have the same size, but the dimensions differ across the multiple network layers. The total number of network layers is M. The input to the encoder of the first network layer is the first logarithmic time spectrum. The input to the encoder of the j-th network layer is the first encoded feature output by the encoder of the (j-1)-th network layer. The input to the decoder of the M-th network layer is the first encoded feature output by the encoder of the M-th network layer. The input to the decoder of the j-th network layer is the fused feature output by the decoder of the (j+1)-th network layer, which is obtained based on the first decoded feature and the first attention feature output by the (j+1)-th network layer. M is a positive integer greater than 1, and j is a positive integer greater than 1 and less than or equal to M.

[0112] In this context, the first encoding feature, the first decoding feature, and the first attention feature having the same size in the same network layer means that these features have the same height, width, and number of channels.

[0113] Optionally, in the multiple dimensions corresponding to the multi-layer network, the height and width both tend to decrease, while the number of channels tends to increase. For example, the width and height of the first network layer are 572 and the number of channels is 64, the width and height of the second network layer are 284 and the number of channels is 128, and the width and height of the third network layer are 140 and the number of channels is 256.

[0114] Optionally, the first logarithmic time spectrum is processed sequentially by multiple encoders in the multi-layer network layer to obtain multiple first coding features output by the encoders in the multi-layer network layer. Then, multiple decoders in the multi-layer network layer iterate based on these multiple first coding features to obtain the first decoding features.

[0115] For example, see Figure 7 , Figure 7 This is a schematic diagram of a multi-layer network provided in an embodiment of this application. The encoder input in the first layer is the first logarithmic time spectrum, and the encoder inputs in other layers are the outputs of the previous encoders. Multiple first encoded features are obtained by passing through multiple network layers. The output of the encoder in the last layer (Mth layer) is used as the input to the decoder in the last layer to obtain the first decoded feature output by the decoder. For the decoder in the j-th network layer, its input is the fused feature output by the (j+1)-th network layer. The decoder in the j-th network layer processes this fused feature to obtain the first decoded feature. Then, based on the attention module of the j-th network layer, it performs cross-attention processing on the first encoded feature and the first decoded feature output by the network layer to obtain the first attention feature. The first attention feature and the first decoded feature output by the network layer are fused to obtain the fused feature output by the j-th network layer. This fused feature is subsequently used as the input of the decoder in the (j-1)-th network layer. For the decoder of multiple network layers, the above steps are repeated in the order from the M-th layer to the 1-th layer until the 1-th network layer is reached. Based on the first attention feature and the first decoded feature output by the 1-th network layer, the fused feature output by the 1-th network layer is obtained. Based on this fused feature, the first prediction matrix is ​​obtained.

[0116] It's important to note that the shallow encoder layers of a network are used to extract low-level features such as edges and textures. As the number of layers increases, the network can combine and abstract these low-level features, gradually extracting high-level features and improving feature representation capabilities. The decoder is used to recover the required output information from the abstract encoded features output by the encoder. Multi-layer processing can progressively refine the features, gradually recovering detailed information from the coarse high-level representation. Furthermore, the multi-layer structure allows for multiple feature interactions between the encoder and decoder. Skip connections between the encoder and decoder enable feature transfer across multiple layers, allowing the decoder to utilize features from different levels of the encoder, achieving a combination of high-level representation and low-level details, thus improving the performance of the signal processing model.

[0117] 407. Train the signal processing model based on the label matrix and the first prediction matrix of each of the multiple training samples.

[0118] In some embodiments, for each set of training samples, a loss value between the label matrix and the first prediction matrix is ​​determined, and the parameters of the signal processing model are adjusted based on the loss value.

[0119] Optionally, the signal processing model is trained iteratively based on multiple sets of training samples. If each iteration uses one set of training samples, the parameters are adjusted using the loss value of that set of training samples. If each iteration uses at least two sets of training samples, the parameters are adjusted using the combined loss value of those at least two sets of training samples, where the combined loss value can be the mean or sum of multiple loss values.

[0120] In some embodiments, in the first prediction matrix, each element at a position is a probability value indicating the probability that the element at that position is a target value. In other embodiments, each element at a position in the first prediction matrix is ​​either a target value or a non-target value. Optionally, since there are multiple positions in the label matrix, the first prediction matrix also has multiple corresponding positions. For each position in the first prediction matrix, the loss value for that position is determined based on the element value at that position and the element value at that position in the label matrix. Then, based on the loss values ​​of multiple positions, the loss value of the first prediction matrix is ​​obtained. Optionally, the average of the loss values ​​of multiple positions, the sum of the loss values ​​of multiple positions, or the weighted sum of the loss values ​​of multiple positions can be used as the loss value of the first prediction matrix; no specific limitation is made here.

[0121] Optionally, the loss value can be determined using a focus loss function. The focus loss function addresses the class imbalance problem in object detection tasks. It dynamically adjusts the contribution of training samples of different difficulties and classes to the loss, thereby mitigating the negative impact of class imbalance on model training and allowing the model to focus more on difficult-to-distinguish samples and minority class samples. The example uses the probability value of each element in the first prediction matrix. The focus loss function is shown in formula (7) below: (7) in, Indicates the loss value; This represents the balance factor, which has a value range of [0,1]. It is used to adjust the weights of positive and negative training samples to weight the training samples. It can be set according to the reciprocal of the class frequency or adjusted as a hyperparameter through cross-validation. No specific limitation is made here. This represents the probability that the element at position t in the first prediction matrix is ​​the target value. If the target value in the label matrix is ​​1, That is, it represents the probability that the element at that position has a value of 1; This represents the modulation factor, which is usually taken as... ≥0 is used to reduce the loss weight of easily classified training samples. This is used when training samples are correctly classified (e.g., the predicted probability of a positive training sample is close to 1). It will approach 0, thus significantly reducing the loss weight of the training sample; when the training sample is misclassified (e.g., the prediction probability of a positive training sample is close to 0). As the value approaches 1, the loss weight remains almost unchanged.

[0122] In some embodiments, in order to ensure that the output features are not distorted and have a certain degree of generalization, a reconstruction loss is also set to constrain the output features of the decoder. This loss is shown in the following formula (8): (8) in, This represents the first logarithmic time spectrum after normalization using the softmax activation function. This represents the output of the last decoder. Representing the metric space middle The distance between y and y Denotes an integral infinitesimal element. and Represented in metric space Two probability distributions on, Represents all joint distributions A set that satisfies The edge distributions are respectively and ; It represents the infimum, which is the minimum possible value. Used to measure the distributional difference between the encoder's input and the decoder's output; express The mean square error between y and y The weighted term represents the mean squared error.

[0123] For example, see Figure 8 , Figure 8 This is a flowchart of a signal processing model provided in an embodiment of this application. The first logarithmic time-frequency spectrum is input into the signal processing model. After processing by the encoder, decoder, and attention module in a multi-layer network, the channel weights are obtained based on the first encoded features output by the encoder in the first layer. First attention features are obtained based on the first encoded features and first decoded features output by the encoder in the first layer. These first attention features and first decoded features are then fused. The fused features are processed by the convolutional layer of the output module to output a first prediction matrix. Optionally, the encoder in each network layer includes a convolutional layer, which may include a ConvLSTM (Convolutional Long Short-Term Memory) network. The decoder in each network layer includes a convolutional layer and an upsampled two-dimensional convolutional layer (UPSampling2D).

[0124] In the above embodiments, the example given is that each training sample group includes one logarithmic time spectrum. In other embodiments, each training sample group may also include multiple logarithmic time spectra. Optionally, each training sample group may also include a second logarithmic time spectrum, which is the first logarithmic time spectrum after binarization of image edges. For example, if it is the first logarithmic time spectrum after binarization by Sobel operator edge detection, the second logarithmic time spectrum is further processed by a signal processing model. This process includes the following steps: for each training sample group, the second logarithmic time spectrum in the training sample is processed by an encoder to obtain a second encoded feature; the second encoded feature is processed by a decoder to obtain a second decoded feature; the second encoded feature and the second decoded feature are cross-attention processed by an attention module to obtain a third attention feature; and the second decoded feature and the third attention feature are fused by a signal processing model to obtain a second prediction matrix.

[0125] The specific process of the signal processing model for the second logarithmic time spectrum processing is the same as that for the first logarithmic time spectrum processing, and will not be repeated here.

[0126] The second logarithmic time-frequency spectrum is obtained by edge detection on the first logarithmic time-frequency spectrum. Optionally, the Sobel operator is applied to the first logarithmic time-frequency spectrum to obtain the second logarithmic time-frequency spectrum. Alternatively, the Prewitt operator is applied to the first logarithmic time-frequency spectrum to obtain the second logarithmic time-frequency spectrum; the specific processing method is not limited here. For example, see [link to relevant documentation]. Figure 9 , Figure 9 This is a schematic diagram of a first logarithmic time spectrum and a second logarithmic time spectrum provided in an embodiment of this application.

[0127] Accordingly, the process of training a signal processing model based on the label matrix and first prediction matrix of each of the multiple sets of training samples includes the following steps: training a signal processing model based on the label matrix, first prediction matrix and second prediction matrix of each of the multiple sets of training samples.

[0128] In some embodiments, the process of training a signal processing model based on the label matrix, the first prediction matrix, and the second prediction matrix of multiple sets of training samples includes the following two implementation methods.

[0129] (1) For each training sample, the first prediction matrix and the second prediction matrix are fused to obtain the third prediction matrix; based on the label matrix and the third prediction matrix, the loss value is determined, and the signal processing model is trained based on the loss value. The loss value is used to indicate the difference information between the third prediction matrix and the label matrix.

[0130] Optionally, the mean of the first and second prediction matrices is used to determine the third prediction matrix; or, the first and second prediction matrices are weighted and summed to obtain the third prediction matrix. Optionally, the fusion of the first and second prediction matrices is achieved through the convolutional layer of the output module in the signal processing model, thereby outputting a third prediction matrix by the signal processing model.

[0131] Optionally, the difference information between the third prediction matrix and the label matrix can be determined by at least one of the above formulas (7) and (8), the specific process of which will not be elaborated here. Accordingly, for formula (8) This represents the desired feature between the first logarithmic time spectrum and the second logarithmic time spectrum. The desired feature is obtained by normalizing the average of the two logarithmic time spectra using the softmax activation function.

[0132] In this embodiment, since the two prediction matrices are the outputs of the signal processing model after processing the two logarithmic time-frequency spectra, the model parameters are adjusted based on the fusion matrix of the two prediction matrices. This enables the signal processing model to learn both the indirect correlation between the global frequency domain of the logarithmic time-frequency spectrum and the target features, and the direct correlation between the local structure of the logarithmic time-frequency spectrum and the target features. Moreover, only one loss value needs to be calculated, which improves efficiency while enhancing the effectiveness of model learning.

[0133] (2) For each training sample, a first loss value is determined based on the label matrix and the first prediction matrix. The first loss value is used to indicate the difference information between the first prediction matrix and the label matrix. A second loss value is determined based on the label matrix and the second prediction matrix. The second loss value is used to indicate the difference information between the second prediction matrix and the label matrix. The signal processing model is trained based on the first loss value and the second loss value.

[0134] Both loss values ​​can be obtained by at least one of the above formulas (7) and (8), and the specific process will not be repeated here.

[0135] In this embodiment, since the two prediction matrices are the outputs of the signal processing model after processing the two logarithmic time-frequency spectra, the loss values ​​are determined based on the two prediction matrices respectively. The loss value based on the first prediction matrix enables the signal processing model to learn the indirect relationship between the global frequency domain of the logarithmic time-frequency spectrum and the target features, while the loss value based on the second prediction matrix enables the signal processing model to learn the direct relationship between the local structure of the logarithmic time-frequency spectrum and the target features, thereby improving the comprehensiveness of the model's learning and thus improving the model's generalization ability.

[0136] For example, see Figure 10 , Figure 10 This is a flowchart of a signal processing model provided in an embodiment of this application. The first and second logarithmic time-frequency spectra are both input into the signal processing model. For each logarithmic time-frequency spectra, after processing by encoders, decoders, and attention modules in a multi-layer network, the channel weights are obtained based on the first encoded features output by the encoder in the first layer. A first attention feature is obtained based on the first encoded and first decoded features output by the first layer. Then, the first attention feature and the first decoded feature are fused. The fused feature is processed by the convolutional layer of the output module to output a prediction matrix. It should be noted that since the input data are two logarithmic time-frequency spectra, separate prediction matrices for each of the two logarithmic time-frequency spectra are obtained. Optionally, the two prediction matrices are processed by the convolutional layer of the output module to obtain a single prediction matrix, which is then processed by the softmax activation function to obtain a prediction matrix where the element values ​​are either target values ​​or non-target values.

[0137] In this embodiment, edge detection is performed on the first logarithmic time-frequency spectrum to obtain a second logarithmic time-frequency spectrum binarized from the image edges. Since the logarithmic time-frequency spectrum binarized from the image edges focuses on drastically changing structural edges, this characteristic can guide the signal processing model to pay more attention to the structured components in the acoustic signal, rather than irregular background noise, thereby reducing the interference of irrelevant information in the logarithmic time-frequency spectrum. In this way, by leveraging the potential correlation between the frequency domain features of the logarithmic time-frequency spectrum and the sound elements, the robustness of the signal processing model in recognizing specific sound elements is improved. Furthermore, by processing the first and second logarithmic time-frequency spectra separately by the signal processing model, prediction matrices for each of the two logarithmic time-frequency spectra are obtained. This complements the preservation of global features and the focus on key features, allowing the signal processing model to learn the frequency domain patterns of the acoustic signal more comprehensively and robustly, thus improving the robustness of the signal processing model.

[0138] In the above embodiments, the processing of the first logarithmic time spectrum and the second logarithmic time spectrum by the signal processing model is used as an example. In other embodiments, the following implementation method is also included: For each set of training samples, the first logarithmic time spectrum and the second logarithmic time spectrum in the training samples are concatenated to obtain the concatenated logarithmic time spectrum, which is then input into the signal processing model; the encoder of the signal processing model processes the concatenated logarithmic time spectrum in the training samples to obtain the third encoded feature; the decoder of the signal processing model processes the third encoded feature to obtain the third decoded feature; the attention module of the signal processing model performs cross-attention processing on the third encoded feature and the third decoded feature to obtain the fourth attention feature; the signal processing model fuses the third decoded feature and the fourth attention feature to obtain the fourth prediction matrix; the element values ​​at each position in the fourth prediction matrix are used to determine whether the target feature exists in the acoustic signal; the signal processing model is trained based on the label matrix of each set of training samples and the fourth prediction matrix.

[0139] The training process of the signal processing model based on the concatenated logarithmic time spectrum is the same as the training process of the signal processing model based on the first logarithmic time spectrum, and the specific process will not be repeated here.

[0140] The concatenated logarithmic time-frequency spectrum includes both a first and a second logarithmic time-frequency spectrum, effectively combining them into multi-channel data, which is then used as input to the signal processing model. The first logarithmic time-frequency spectrum retains the fine details of the original acoustic signal's amplitude and frequency distribution, while the second logarithmic time-frequency spectrum highlights the contour information of the frequency band trajectory through edge detection. This combination forms a multi-channel feature representation, achieving feature complementarity and preserving both detailed information and enhanced structural features. Edge binarization suppresses background noise interference and highlights the frequency band trajectory contour of the target feature. For weak target features in complex acoustic environments, edge information can assist the model in locating key frequency bands, thus improving the target feature's recognizability. Furthermore, the multi-channel input provides redundant feature representation for the model. When the original logarithmic time-frequency spectrum is affected by noise, edge features can serve as supplementary features, reducing the model's dependence on a single feature and enhancing its robustness. Additionally, edge features help the attention module more accurately locate frequency band trajectory features, guiding the attention mechanism's focus. This feature fusion strategy, achieved through multi-channel input, allows the model to simultaneously learn the original energy distribution and structural contour features of the acoustic signal, thereby enabling more stable identification of the frequency band trajectory of target features in complex scenarios.

[0141] In some embodiments, the example of using multiple sets of training samples for model training is described. In other embodiments, after obtaining multiple sets of training samples, these samples are divided into two sets: a training set and a test set. The signal processing model is trained using the training samples in the training set, and the trained signal processing model is tested using the training samples in the test set. If the test results meet the expected requirements, the trained signal processing model is directly applied to the processing of acoustic signals. If the test results do not meet the expected requirements, the parameters of the signal processing model are tuned until the test results meet the expected requirements.

[0142] For example, see Figure 11 , Figure 11This is a flowchart illustrating the training and use of a signal processing model provided in this application embodiment. After acquiring an acoustic signal, it is filtered or downsampled, then normalized. Multiple segments of the acoustic signal are extracted according to a preset time window. For each segment, a short-time Fourier transform and logarithmic dB transform are applied to obtain the first logarithmic time spectrum. This, combined with a label matrix, yields a dataset containing multiple training samples. The dataset is divided into a training set and a test set. The signal processing model is trained using the test set. If the loss does not converge, iterative training continues until convergence is achieved, resulting in a pre-trained signal processing model. If the test results of the pre-trained model do not meet expectations, the parameters of the signal processing model are optimized. If the test results meet expectations, the signal processing model is applied to acoustic signal processing. The optimized signal processing model is deployed to a terminal. When an acoustic signal is input to the terminal, it is transmitted to the signal processing module. The signal processing module processes the logarithmic time spectrum of the acoustic signal using the signal processing model, and the terminal outputs the processing result.

[0143] This application provides a training method for a signal processing model. The method uses the logarithmic time-frequency spectrum of an acoustic signal including target features and the corresponding label matrix as training samples. The logarithmic time-frequency spectrum is input into the signal processing model, and encoded and decoded features are obtained through an encoder and a decoder. Since the encoder can extract more representative and abstract feature representations from the logarithmic time-frequency spectrum, and the decoder can reconstruct feature representations related to the original features by decoding the encoded features, the collaborative work of the encoder and decoder can better extract effective information from the logarithmic time-frequency spectrum. Based on this, the encoded and decoded features are processed through a cross-attention mechanism to obtain attention features, enabling full interaction between the encoded and decoded features and focusing on each other's more critical information. This allows the signal processing model to more efficiently capture the correlation between different feature levels of the acoustic signal, improving its ability to perceive target features. Furthermore, the decoded features and attention features are fused to obtain a prediction matrix, further integrating feature information from different processing stages. This allows the prediction matrix to more comprehensively and accurately reflect the presence of target features in the acoustic signal, improving the performance of the signal processing model.

[0144] In this embodiment, signal processing is performed using the signal processing model trained through any of the above embodiments. The signal processing process is described in [reference needed]. Figure 12 , Figure 12 This is a flowchart of a signal processing method provided in an embodiment of this application, which includes at least one of the following steps.

[0145] 1201. Obtain the logarithmic time spectrum of the target acoustic signal.

[0146] 1202. The target coding features are obtained by processing the logarithmic time spectrum through the encoder of the signal processing model. The target coding features are obtained by processing the target decoding features through the decoder of the signal processing model. The target coding features and target decoding features are cross-attention processed by the attention module of the signal processing model to obtain target attention features. The target decoding features and target attention features are fused by the signal processing model to obtain the target prediction matrix. The element values ​​at each position in the target prediction matrix are used to determine whether the target features exist in the acoustic signal.

[0147] In the case of obtaining one logarithmic time-frequency spectrum of the target acoustic signal, the processing procedure of the signal processing model for the logarithmic time-frequency spectrum of the target acoustic signal is the same as the processing procedure for the first logarithmic time-frequency spectrum in the above embodiment, and will not be repeated here. In the case of obtaining two logarithmic time-frequency spectra of the target acoustic signal, the two logarithmic time-frequency spectra correspond to the first logarithmic time-frequency spectrum and the second logarithmic time-frequency spectrum in the above embodiment, respectively. The processing procedure of the signal processing model for the two logarithmic time-frequency spectra of the target acoustic signal is the same as the processing procedure for the first logarithmic time-frequency spectrum and the second logarithmic time-frequency spectrum in the above embodiment; that is, they can be processed separately or concatenated before processing. The specific process will not be repeated here.

[0148] In this embodiment, the logarithmic time-frequency spectrum is input into the signal processing model. Encoded and decoded features are obtained through an encoder and a decoder. Since the encoder can extract more representative and abstract feature representations from the logarithmic time-frequency spectrum, and the decoder can reconstruct feature representations related to the original features by decoding the encoded features, the collaborative work of the encoder and decoder can better extract effective information from the logarithmic time-frequency spectrum. Based on this, the encoded and decoded features are processed through a cross-attention mechanism to obtain attention features, enabling full interaction between the encoded and decoded features and focusing on each other's more critical information. This allows the signal processing model to more efficiently capture the correlation between different feature levels of the acoustic signal, improving its ability to perceive target features. Furthermore, the decoded and attention features are fused to obtain a prediction matrix, further integrating feature information from different processing stages. This allows the target prediction matrix to more comprehensively and accurately reflect the presence of target features in the acoustic signal, improving the performance of the signal processing model.

[0149] The target acoustic signal in this application embodiment can be an acoustic signal collected in different scenarios, including at least one of the following implementation methods: the target acoustic signal is an acoustic signal generated during the operation of the device, and the target feature is used to indicate the target fault. If the target acoustic signal is determined to have a target feature based on the element values ​​at each position in the target prediction matrix, then it is determined that the device has a target fault; the target acoustic signal is an acoustic signal generated on the seabed, and the target feature is used to indicate the fish school. If the target acoustic signal is determined to have a target feature based on the element values ​​at each position in the target prediction matrix, then it is determined that the seabed has a fish school.

[0150] In this embodiment, the logarithmic time-frequency spectrum of the target acoustic signal is processed by a signal processing model to obtain a target prediction matrix. The target prediction matrix is ​​then matched with a desired matrix. If the matching degree between the target prediction matrix and the desired matrix reaches a preset value, it is determined that a target feature has appeared in the target acoustic signal. The desired matrix is ​​then the matrix corresponding to the acoustic signal containing the target feature. Optionally, multiple positions in the target prediction matrix and the desired matrix correspond one-to-one, and the combined difference of the element values ​​at multiple positions is determined as the matching degree between the target prediction matrix and the desired matrix.

[0151] The target acoustic signal is the acoustic signal generated during the operation of the equipment. The target feature is used to indicate the target fault. That is, the method is applied to the fault detection scenario of the equipment. Through the signal processing model, the target prediction matrix corresponding to the target acoustic signal can be obtained. Based on the target prediction matrix, it can be determined whether there is a target feature corresponding to the target fault in the target acoustic signal, and then it can be determined whether the equipment has the target fault.

[0152] The target fault can be set as needed, such as mechanical wear, loose parts, damaged parts, circuit abnormalities, etc. Equipment faults often form specific frequency band trajectories in the time spectrum of the sound signal. By performing detailed analysis of the logarithmic time spectrum using a large model and combining it with an attention mechanism to focus on key frequency band trajectories, fault characteristics can be accurately identified, reducing misjudgments caused by background noise.

[0153] In this method, the target acoustic signal is an acoustic signal generated on the seabed, and the target features are used to indicate the fish school. That is, this method is applied to the seabed fish school location scenario. Through the signal processing model, the target prediction matrix corresponding to the target acoustic signal can be obtained. Based on the target prediction matrix, it can be determined whether there are target features corresponding to the target fault in the target acoustic signal, and thus determine whether there is a fish school on the seabed.

[0154] Among them, the target features are used to indicate fish schools, and can be target features corresponding to fish schools of different species and different marine environments. The specific features can be set according to needs.

[0155] Among these, seabed acoustic signals are mixed with various noises such as ocean currents, ships, and marine life, often masking the acoustic characteristics of fish school activities. The signal processing model, through logarithmic time-spectrum analysis, highlights the frequency band trajectory of fish school acoustic signals and then focuses on effective features through an attention mechanism. This allows it to distinguish fish school signals from non-fish school signals in complex backgrounds. Compared to traditional manual listening or simple spectrum analysis, the signal processing model can automatically process large amounts of seabed acoustic signals, quickly output target prediction results, and improve signal processing efficiency. It is widely applicable to scenarios requiring continuous, real-time analysis, such as fisheries resource exploration and ecological monitoring.

[0156] Figure 13 This is a block diagram of a training device for a signal processing model provided in an embodiment of this application. (Refer to...) Figure 13 The device includes: The acquisition module 1301 is used to acquire multiple sets of training samples. Each set of training samples includes the first logarithmic time spectrum of the acoustic signal and the label matrix of the first logarithmic time spectrum. The acoustic signal includes target features. The element values ​​of multiple target positions in the label matrix are target values. The multiple target positions are used to indicate multiple coordinate positions on the frequency band trajectory in the first logarithmic time spectrum. The frequency band trajectory is used to indicate the target features. The processing module 1302 is used to process the first logarithmic time spectrum of the training samples in each training sample by the encoder of the signal processing model to obtain the first encoded feature, process the first encoded feature by the decoder of the signal processing model to obtain the first decoded feature, perform cross-attention processing on the first encoded feature and the first decoded feature by the attention module of the signal processing model to obtain the first attention feature, and fuse the first decoded feature and the first attention feature by the signal processing model to obtain the first prediction matrix. The element values ​​at each position in the first prediction matrix are used to determine whether the target feature exists in the acoustic signal. Training module 1303 is used to train a signal processing model based on the label matrix and first prediction matrix of multiple training samples.

[0157] In some embodiments, the first encoded feature includes sub-features for each of multiple channels, and the processing module 1302 is configured to: The attention module processes the sub-features of each channel separately to obtain the weights of each channel. The first encoded feature and the first decoded feature are cross-attention processed by the attention module to obtain the second attention feature, which includes the sub-attention features of each of the multiple channels. The attention module weights the sub-attention features of multiple channels based on their respective weights to obtain the first attention feature.

[0158] In some embodiments, the processing module 1302 is configured to: The query feature is obtained by processing the query data in the attention module, the first encoded feature is obtained by processing the key data in the attention module, the first decoded feature is obtained by processing the key feature, and the value feature is obtained by processing the value data in the attention module. The second attention feature is obtained by processing the query features, key features, and value features through the attention module.

[0159] In some embodiments, the processing module 1302 is configured to: The attention module is used to divide the query features, key features, and value features into their respective feature sequences. Each feature sequence includes multiple sub-features arranged in order. For each feature group, based on the sub-query features, sub-key features, and sub-value features in the feature group, the corresponding sub-attention features are determined. Each feature group includes sub-features at the same position in multiple feature sequences. By concatenating the sub-attention features of each feature group, the second attention feature is obtained.

[0160] In some embodiments, the signal processing model includes multiple network layers, each network layer including an encoder, a decoder, and an attention module. The first coded feature, the first decoded feature, and the first attention feature in the same network layer have the same size, and the multiple network layers correspond to different sizes. The total number of multiple network layers is M. The input of the encoder of the first network layer is the first logarithmic time spectrum. The input of the encoder of the j-th network layer is the first coded feature output by the encoder of the (j-1)-th network layer. The input of the decoder of the M-th network layer is the first coded feature output by the encoder of the M-th network layer. The input of the decoder of the j-th network layer is the fused feature output by the decoder of the (j+1)-th network layer. The fused feature is obtained based on the first decoded feature and the first attention feature output by the (j+1)-th network layer. M is a positive integer greater than 1, and j is a positive integer greater than 1 and less than or equal to M.

[0161] In some embodiments, each set of training samples further includes a second logarithmic time spectrum, which is the first logarithmic time spectrum of the image edge binarization. The processing module 1302 is further configured to: For each training sample, the second logarithmic time spectrum in the training sample is processed by the encoder to obtain the second encoded feature. The second encoded feature is processed by the decoder to obtain the second decoded feature. The second encoded feature and the second decoded feature are cross-attention processed by the attention module to obtain the third attention feature. The second decoded feature and the third attention feature are fused by the signal processing model to obtain the second prediction matrix. Training module 1303 is also used for: The signal processing model is trained based on the label matrix, first prediction matrix and second prediction matrix of multiple training samples.

[0162] In some embodiments, the processing module 1302 is configured to: For each set of training samples, the first prediction matrix and the second prediction matrix are fused to obtain the third prediction matrix; Based on the label matrix and the third prediction matrix, a loss value is determined, and a signal processing model is trained based on the loss value. The loss value is used to indicate the difference information between the third prediction matrix and the label matrix.

[0163] In some embodiments, the training module 1303 is used for: For each training sample, a first loss value is determined based on the label matrix and the first prediction matrix. The first loss value is used to indicate the difference information between the first prediction matrix and the label matrix. Based on the label matrix and the second prediction matrix, a second loss value is determined. The second loss value is used to indicate the difference information between the second prediction matrix and the label matrix. The signal processing model is trained based on the first loss value and the second loss value.

[0164] In some embodiments, each set of training samples further includes a second logarithmic time spectrum, which is the first logarithmic time spectrum of the image edge binarization. The apparatus further includes a stitching module for: For each set of training samples, the first logarithmic time spectrum and the second logarithmic time spectrum in the training samples are concatenated to obtain the concatenated logarithmic time spectrum, which is then input into the signal processing model. The processing module 1302 is also used to process the spliced ​​logarithmic time spectrum of the training samples through the encoder of the signal processing model to obtain the third coded feature, process the third coded feature through the decoder of the signal processing model to obtain the third decoded feature, perform cross-attention processing on the third coded feature and the third decoded feature through the attention module of the signal processing model to obtain the fourth attention feature, and fuse the third decoded feature and the fourth attention feature through the signal processing model to obtain the fourth prediction matrix. The element values ​​at each position in the fourth prediction matrix are used to determine whether the target feature exists in the acoustic signal. Training module 1303 is used to train a signal processing model based on the label matrix and fourth prediction matrix of multiple training samples.

[0165] In some embodiments, the acquisition module 1301 is used for: Acquire a target acoustic signal of a preset duration, the target acoustic signal including target features; Multiple sound signals are extracted from the target sound signal based on a preset time window; Based on multiple coordinate positions on the frequency band trajectory of each sound signal, a label matrix is ​​determined, and the frequency band trajectory is used to indicate target features; Multiple training samples are obtained based on the first logarithmic time spectrum and label matrix of each of the multiple acoustic signals.

[0166] Figure 14 This is a block diagram of a signal processing apparatus provided in an embodiment of this application. (Refer to...) Figure 14 The device includes: The acquisition module 1401 is used to acquire the logarithmic time spectrum of the target acoustic signal; The processing module 1402 is used to process the logarithmic time spectrum through the encoder of the signal processing model to obtain target encoded features, process the target encoded features through the decoder of the signal processing model to obtain target decoded features, perform cross-attention processing on the target encoded features and target decoded features through the attention module of the signal processing model to obtain target attention features, and fuse the target decoded features and target attention features through the signal processing model to obtain a target prediction matrix. The element values ​​at each position in the target prediction matrix are used to determine whether target features exist in the acoustic signal. The signal processing model is a signal processing model trained through any of the above embodiments.

[0167] In some embodiments, the apparatus further includes a determining module for performing at least one of the following: The target acoustic signal is the acoustic signal generated during the operation of the equipment. The target features are used to indicate the target fault. If the target features are determined to exist in the target acoustic signal based on the element values ​​at each position in the target prediction matrix, then it is determined that a target fault has been detected in the equipment. The target acoustic signal is an acoustic signal generated on the seabed. The target features are used to indicate the fish school. If the target features are determined to exist in the target acoustic signal based on the element values ​​at each position in the target prediction matrix, then it is determined that a fish school has been detected on the seabed.

[0168] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0169] In some embodiments, the electronic device is provided as a terminal. Figure 15A structural block diagram of a terminal 1500 provided in an exemplary embodiment of this disclosure is shown. The terminal 1500 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 1500 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0170] Typically, terminal 1500 includes a processor 1501 and a memory 1502.

[0171] Processor 1501 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1501 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1501 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0172] The memory 1502 may include one or more computer-readable storage media, which may be non-transitory. The memory 1502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1502 are used to store at least one program code, which is executed by the processor 1501 to implement the training method or signal processing method of the signal processing model provided in the method embodiments of this disclosure.

[0173] In some embodiments, the terminal 1500 may also optionally include a peripheral device interface 1503 and at least one peripheral device. The processor 1501, memory 1502, and peripheral device interface 1503 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: radio frequency circuitry 1504, display screen 1505, camera assembly 1506, audio circuitry 1507, and power supply 1508.

[0174] Peripheral device interface 1503 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1501 and memory 1502. In some embodiments, processor 1501, memory 1502 and peripheral device interface 1503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1501, memory 1502 and peripheral device interface 1503 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0175] The radio frequency (RF) circuit 1504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1504 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1504 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1504 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1504 may also include circuitry related to NFC (Near Field Communication), which is not limited in this disclosure.

[0176] Display screen 1505 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1505 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1501 for processing. In this case, display screen 1505 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1505, which serves as the front panel of terminal 1500; in other embodiments, there may be at least two display screens, respectively disposed on different surfaces of terminal 1500 or in a folded design; in still other embodiments, display screen 1505 may be a flexible display screen, disposed on a curved or folded surface of terminal 1500. Furthermore, display screen 1505 may also be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1505 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0177] The camera assembly 1506 is used to acquire images or videos. Optionally, the camera assembly 1506 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1506 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0178] The audio circuit 1507 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1501 for processing, or input to the radio frequency circuit 1504 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 1500. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1501 or the radio frequency circuit 1504 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1507 may also include a headphone jack.

[0179] Power supply 1508 is used to power the various components in terminal 1500. Power supply 1508 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1508 includes a rechargeable battery, the rechargeable battery can support wired or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0180] Those skilled in the art will understand that Figure 15 The structure shown does not constitute a limitation on terminal 1500 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0181] Figure 16 This is a schematic diagram of a server structure according to an embodiment of this application. The server 1600 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 1601 and one or more memories 1602. The memories 1602 are used to store executable program code, and the processors 1601 are configured to execute the executable program code to implement the training method or signal processing method of the signal processing model provided in the various method embodiments above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated here.

[0182] In an exemplary embodiment, a computer-readable storage medium is also provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the training method for the signal processing model described above. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0183] In an exemplary embodiment, a computer program product is also provided, the computer program product including a computer program that, when executed by a processor, implements the training method or signal processing method of the above-described signal processing model.

[0184] In some embodiments, the computer program product involved in this disclosure may be deployed on an electronic device for execution, or on multiple electronic devices located in one location, or on multiple electronic devices distributed in multiple locations and interconnected through a communication network. The multiple electronic devices distributed in multiple locations and interconnected through a communication network may constitute a blockchain system.

[0185] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims. All the above-described optional technical solutions can be combined in any way to form optional embodiments of this application, and will not be elaborated upon here.

[0186] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A training method for a signal processing model, characterized in that, The method includes: Multiple sets of training samples are obtained. Each set of training samples includes a first logarithmic time spectrum of an acoustic signal and a label matrix of the first logarithmic time spectrum. The acoustic signal includes target features. The element values ​​of multiple target positions in the label matrix are target values. The multiple target positions are used to indicate multiple coordinate positions on the frequency band trajectory in the first logarithmic time spectrum. The frequency band trajectory is used to indicate the target features. For each training sample, the encoder of the signal processing model processes the first logarithmic time-spectrum of the training sample to obtain a first encoded feature. The decoder of the signal processing model processes the first encoded feature to obtain a first decoded feature. The attention module of the signal processing model performs cross-attention processing on the first encoded feature and the first decoded feature to obtain a first attention feature. The signal processing model fuses the first decoded feature and the first attention feature to obtain a first prediction matrix. The element values ​​at each position in the first prediction matrix are used to determine whether the target feature exists in the acoustic signal. The signal processing model is trained based on the label matrix and the first prediction matrix of each of the multiple sets of training samples.

2. The method according to claim 1, characterized in that, The first encoded feature includes sub-features for each of multiple channels. The first attention feature is obtained by performing cross-attention processing on the first encoded feature and the first decoded feature through the attention module of the signal processing model, including: The attention module processes the sub-features of each of the multiple channels to obtain the weights of each of the multiple channels. The attention module performs cross-attention processing on the first encoded feature and the first decoded feature to obtain a second attention feature, which includes the sub-attention features of each of the multiple channels. The attention module weights the sub-attention features of each of the multiple channels based on their respective weights to obtain the first attention feature.

3. The method according to claim 2, characterized in that, The step of performing cross-attention processing on the first encoded feature and the first decoded feature through the attention module to obtain the second attention feature includes: The first encoded feature is processed using the query data in the attention module to obtain the query feature; the first decoded feature is processed using the key data in the attention module to obtain the key feature; and the first decoded feature is processed using the value data in the attention module to obtain the value feature. The second attention feature is obtained by processing the query feature, the key feature, and the value feature through the attention module.

4. The method according to claim 3, characterized in that, The process of processing the query feature, the key feature, and the value feature through the attention module to obtain the second attention feature includes: The attention module is used to divide the query feature, the key feature, and the value feature into their respective feature sequences, each of which includes multiple sub-features arranged in order. For each feature group, based on the sub-query features, sub-key features, and sub-value features in the feature group, the corresponding sub-attention features are determined. Each feature group includes sub-features at the same position in multiple feature sequences. The second attention feature is obtained by concatenating the sub-attention features of each feature group.

5. The method according to claim 1, characterized in that, The signal processing model includes multiple network layers. Each network layer includes an encoder, a decoder, and an attention module. The first coded feature, the first decoded feature, and the first attention feature in the same network layer have the same size, and the multiple network layers correspond to different sizes. The total number of the multiple network layers is M. The input of the encoder of the first network layer is the first logarithmic time spectrum. The input of the encoder of the j-th network layer is the first coded feature output by the encoder of the (j-1)-th network layer. The input of the decoder of the M-th network layer is the first coded feature output by the encoder of the M-th network layer. The input of the decoder of the j-th network layer is the fused feature output by the decoder of the (j+1)-th network layer. The fused feature is obtained based on the first decoded feature and the first attention feature output by the (j+1)-th network layer. M is a positive integer greater than 1, and j is a positive integer greater than 1 and less than or equal to M.

6. The method according to claim 1, characterized in that, Each training sample also includes a second logarithmic time spectrum, which is the first logarithmic time spectrum of the image edge binarization. The method further includes: For each training sample, the encoder processes the second logarithmic time spectrum of the training sample to obtain a second encoded feature. The decoder processes the second encoded feature to obtain a second decoded feature. The attention module performs cross-attention processing on the second encoded feature and the second decoded feature to obtain a third attention feature. The signal processing model fuses the second decoded feature and the third attention feature to obtain a second prediction matrix. The step of training the signal processing model based on the label matrices and first prediction matrices of the multiple sets of training samples includes: The signal processing model is trained based on the label matrix, first prediction matrix, and second prediction matrix of each of the multiple sets of training samples.

7. The method according to claim 6, characterized in that, The step of training the signal processing model based on the label matrix, first prediction matrix, and second prediction matrix of each of the multiple sets of training samples includes: For each set of training samples, the first prediction matrix and the second prediction matrix are fused to obtain the third prediction matrix; Based on the label matrix and the third prediction matrix, a loss value is determined, and the signal processing model is trained based on the loss value. The loss value is used to indicate the difference information between the third prediction matrix and the label matrix.

8. The method according to claim 6, characterized in that, The step of training the signal processing model based on the label matrix, first prediction matrix, and second prediction matrix of each of the multiple sets of training samples includes: For each training sample, a first loss value is determined based on the label matrix and the first prediction matrix. The first loss value is used to indicate the difference information between the first prediction matrix and the label matrix. Based on the label matrix and the second prediction matrix, a second loss value is determined, which is used to indicate the difference information between the second prediction matrix and the label matrix; The signal processing model is trained based on the first loss value and the second loss value.

9. The method according to claim 1, characterized in that, Each training sample also includes a second logarithmic time spectrum, which is the first logarithmic time spectrum of the image edge binarization. The method further includes: For each set of training samples, the first logarithmic time spectrum and the second logarithmic time spectrum in the training samples are concatenated to obtain the concatenated logarithmic time spectrum, which is then input into the signal processing model. The signal processing model's encoder processes the spliced ​​logarithmic time-spectrum of the training samples to obtain a third encoded feature. The signal processing model's decoder processes the third encoded feature to obtain a third decoded feature. The signal processing model's attention module performs cross-attention processing on the third encoded feature and the third decoded feature to obtain a fourth attention feature. The signal processing model fuses the third decoded feature and the fourth attention feature to obtain a fourth prediction matrix. The element values ​​at each position in the fourth prediction matrix are used to determine whether the target feature exists in the acoustic signal. The signal processing model is trained based on the label matrix and fourth prediction matrix of each of the multiple sets of training samples.

10. The method according to claim 1, characterized in that, The acquisition of multiple sets of training samples includes: Acquire a target acoustic signal of a preset duration, wherein the target acoustic signal includes the target features; Multiple segments of sound signal are extracted from the target sound signal based on a preset time window; A tag matrix is ​​determined based on multiple coordinate positions on the frequency band trajectory of each sound signal, wherein the frequency band trajectory is used to indicate the target features; The multiple sets of training samples are obtained based on the first logarithmic time spectrum and label matrix of each of the multiple acoustic signals.

11. A signal processing method, characterized in that, The method includes: Obtain the logarithmic time spectrum of the target acoustic signal; The target encoded features are obtained by processing the logarithmic time-spectrum using the encoder of the signal processing model. The target encoded features are then processed by the decoder of the signal processing model to obtain target decoded features. The target encoded features and the target decoded features are then cross-attention processed by the attention module of the signal processing model to obtain target attention features. Finally, the target decoded features and the target attention features are fused by the signal processing model to obtain a target prediction matrix. The element values ​​at each position in the target prediction matrix are used to determine whether target features exist in the acoustic signal. The signal processing model is a signal processing model trained by any one of claims 1-10.

12. The method according to claim 11, characterized in that, The method further includes at least one of the following: The target acoustic signal is an acoustic signal generated during the operation of the equipment. The target feature is used to indicate the target fault. If the target feature is determined to exist in the target acoustic signal based on the element values ​​at each position in the target prediction matrix, then it is determined that the target fault is detected in the equipment. The target acoustic signal is an acoustic signal generated on the seabed, and the target feature is used to indicate a school of fish. If the target feature is determined to exist in the target acoustic signal based on the element values ​​at each position in the target prediction matrix, then it is determined that a school of fish has been detected on the seabed.

13. A training device for a signal processing model, characterized in that, The device includes: The acquisition module is used to acquire multiple sets of training samples. Each set of training samples includes a first logarithmic time spectrum of an acoustic signal and a label matrix of the first logarithmic time spectrum. The acoustic signal includes target features. The element values ​​of multiple target positions in the label matrix are target values. The multiple target positions are used to indicate multiple coordinate positions on the frequency band trajectory in the first logarithmic time spectrum. The frequency band trajectory is used to indicate the target features. The processing module is used to process the first logarithmic time-spectrum of the training samples in each training sample by the encoder of the signal processing model to obtain a first encoded feature, process the first encoded feature by the decoder of the signal processing model to obtain a first decoded feature, perform cross-attention processing on the first encoded feature and the first decoded feature by the attention module of the signal processing model to obtain a first attention feature, and fuse the first decoded feature and the first attention feature by the signal processing model to obtain a first prediction matrix. The element values ​​at each position in the first prediction matrix are used to determine whether the target feature exists in the acoustic signal. The training module is used to train the signal processing model based on the label matrix and the first prediction matrix of each of the multiple sets of training samples.

14. A signal processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the logarithmic time spectrum of the target acoustic signal; The processing module is configured to process the logarithmic time-spectrum using the encoder of the signal processing model to obtain target encoded features, process the target encoded features using the decoder of the signal processing model to obtain target decoded features, perform cross-attention processing on the target encoded features and the target decoded features using the attention module of the signal processing model to obtain target attention features, and fuse the target decoded features and the target attention features using the signal processing model to obtain a target prediction matrix. The element values ​​at each position in the target prediction matrix are used to determine whether target features exist in the acoustic signal. The signal processing model is a signal processing model trained by any one of claims 1-10.

15. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the training method of the signal processing model as claimed in any one of claims 1 to 10 or the signal processing method as claimed in any one of claims 11 to 12.

16. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the training method of the signal processing model according to any one of claims 1 to 10 or the signal processing method according to any one of claims 11 to 12.

17. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the training method of the signal processing model according to any one of claims 1 to 10 or the signal processing method according to any one of claims 11 to 12.

Images