Marine miniaturized network cabin voice recording method and device

Abstract

The invention discloses a marine miniaturized network cabin voice recording method and a marine miniaturized network cabin voice recording device, which relate to the technical field of shipborne recording equipment, can realize a strong-pointing pickup function, effectively eliminate environmental noise, enhance voice and intelligently judge a voice instruction of a specific person (such as a commander) needing to be extracted through voiceprint recognition. A directional pickup function is realized by using a beam forming technology based on a microphone array, and enhancement is performed aiming at the position of a specific commander in a cabin, so that interference caused by environmental noise can be effectively reduced. And detecting whether the picked sound contains human voice by using a human voice detection algorithm, if the picked sound contains the human voice, further enhancing the picked sound by using a voice enhancement algorithm, otherwise, if the picked sound does not contain the human voice, estimating the environmental noise. In addition, a commander instruction extraction function based on voiceprint recognition is also realized, and the command voice instruction of the commander is automatically extracted according to pre-configured commander identity information.

Description

Technical Field

[0001] The invention relates to the technical field of shipboard recording equipment, and in particular to a method and device for recording human voice in a miniaturized network cabin for a ship. Background Art

[0002] In the field of navigation, the current ship operation simulation training process often requires a review of the training process. Playing back the recordings during the training process is very important for investigation and evidence collection, especially for the recording and playback of the crew's voices and speech. Traditionally, the recording equipment on the ship uses a microphone to pick up the user's voice. The biggest disadvantage of this method is that it is limited by the performance of the single-channel signal enhancement algorithm and cannot effectively shield the environmental noise on the ship (engine, wind, waves, etc.), which will affect the effectiveness of the playback investigation and evidence collection.

[0003] Existing shortcomings: The current shipboard recording equipment only uses a single microphone and can only record the sound on the scene, but cannot effectively remove various harsh noises in the ship's environment, such as engine sound, wind sound, wave sound, etc., and cannot intelligently pick up the voice commands of specific people (such as commanders, etc.) for review and verification. Summary of the invention

[0004] In view of this, the present invention provides a method and device for recording human voice in a miniaturized network cabin on a ship. By using a distributed sound pickup terminal based on a microphone array, in combination with an array signal processing algorithm, through source orientation (DOA) and beamforming, the function of strong directional sound pickup is realized, and environmental noise is effectively eliminated, the human voice is enhanced, and through voiceprint recognition, the voice commands of specific people (such as commanders) that need to be extracted are intelligently judged, thereby ensuring the user experience during playback training.

[0005] To achieve the above object, the technical solution of the present invention is a method for recording human voice in a miniaturized network cabin for a ship, comprising the following steps:

[0006] Step (1) Pick up the original sound signal through the microphone array

[0007] Step (2) captures sound segments of the original sound signal, selects the sound segments in sequence for processing, and uses a voice recognition algorithm to determine whether there is a human voice in the sound segment currently being processed. If there is a human voice, step (3) is executed. If there is no human voice, the sound segment currently being processed is sent to the environmental noise library for storage.

[0008] Step (3) identifies whether a preset specific person appears. If the specific person appears, execute step (4). If the specific person does not appear, execute step (5) and use a conventional beamforming algorithm to enhance the sound in a specific direction through the beamforming algorithm.

[0009] Step (4) Calculate the sound source direction through a multi-sound source directional algorithm, then perform beamforming on each sound source direction to obtain a directionally enhanced sound signal, then use each enhanced sound signal to perform voiceprint recognition, and according to the voiceprint recognition score, take the direction with the highest score for a predetermined specific person as the target direction, and take the enhanced output signal of the target direction as the voice signal of the specific person; Execute step (6)

[0010] Step (5) first uses the direction of arrival estimation DOA algorithm to obtain the direction of the sound source, and then uses the beamforming algorithm to enhance the original sound signal picked up by the microphone array with the direction of the sound source as the microphone array direction to obtain the enhanced sound signal; execute step (6).

[0011] Step (6) selects one of the multiple sound signals picked up by the microphone array as the first channel, and uses the enhanced sound signal obtained in step (5) or the voice of the specific person obtained in step (4) as the second channel to form a two-channel sound signal, which is then packaged according to the UDP protocol to form a UDP data packet.

[0012] Step (7) sends the data packet to the backend server via the network.

[0013] Furthermore, the microphone array is an acoustic sensor array composed of more than two microphones.

[0014] Furthermore, the voice recognition algorithm uses the TCN network:

[0015] The TCN network is divided into TCN module 1, TCN module 2 and TCN module 3. TCN modules 1-3 have the same structure, which are all composed of hole convolution, BN layer, ReLU layer and Dropout layer connected in sequence. The hole convolution is formed by superposition of causal convolution on both sides, and the convolution kernel size is 2. The BN layer, ReLU layer and Dropout layer are conventional batch normalization layer, ReLU activation function layer and discard layer respectively; it also includes: a 1x1 convolution layer is directly connected from the input to the ReLU activation function layer of the module output to form a residual connection to avoid network degradation.

[0016] Furthermore, step 4 includes the following specific steps:

[0017] S401: A sound source localization method based on time delay estimation. The method first estimates the time difference between the sound reaching the sound sensors at different positions by receiving the signal from the sound sensor, then constructs multiple hyperboloids using the time delay value and the position information of the sound sensor, and obtains the position estimate of the sound source by calculating the intersection of the multiple hyperboloids.

[0018] S402: For the directions of multiple sound sources, beamforming enhancement is performed respectively, and then the obtained output y(t) is used for voiceprint recognition scoring in S403.

[0019] S403: Perform voiceprint recognition on each enhanced sound signal. The voiceprint recognition adopts TCN network. The voiceprint recognition network calculates the likelihood value of the network output for the predetermined person model, selects the direction corresponding to the largest value as the position direction of the predetermined person, and selects the enhanced output in the corresponding direction as the signal to be transmitted.

[0020] Another embodiment of the present invention also provides a miniaturized marine network cabin voice recording device, including a microphone array, a voice recognition module, a voiceprint recognition module, a voiceprint likelihood beamforming module, a conventional beamforming module and a UDP packet module.

[0021] The microphone array is used to pick up the original sound signal and send it to the voice recognition module.

[0022] The voice recognition module captures sound fragments of the original sound signal, selects sound fragments in sequence for processing, and uses the voice recognition algorithm to determine whether there is a human voice in the sound fragment currently being processed. If there is a human voice, the sound fragment currently being processed is sent to the voiceprint recognition module. If there is no human voice, the sound fragment currently being processed is sent to the environmental noise library for storage.

[0023] The voiceprint recognition module is used to identify whether a preset specific person appears. If a specific person appears, the currently processed sound segment is sent to the voiceprint likelihood beamforming module. If no specific person appears, the conventional beamforming module is used.

[0024] The voiceprint likelihood beamforming module calculates the direction of the sound source through a multi-sound source directional algorithm, and then performs beamforming on each sound source direction to obtain a directionally enhanced sound signal. The enhanced sound signals are then used to perform voiceprint recognition. According to the voiceprint recognition score, the direction with the highest score for a predetermined specific person is taken as the target direction, and the enhanced output signal of the target direction is taken as the voice signal of the specific person and sent to the UDP packet module.

[0025] The conventional beamforming module first uses the direction of arrival estimation DOA algorithm to obtain the direction of the sound source, and then uses the beamforming algorithm to enhance the currently processed sound segment based on the direction of the sound source as the microphone array direction, obtains the enhanced sound signal, and sends it to the UDP packet module.

[0026] The UDP packet module selects one of the multiple sound signals picked up by the microphone array as the first channel, and uses the enhanced sound signal obtained by the conventional beamforming module or the voice of a specific person obtained by the voiceprint likelihood beamforming module as the second channel to form a two-channel sound signal, which is then packetized according to the UDP protocol to form a UDP data packet; the data packet is then sent to the background server through the network.

[0027] Furthermore, the microphone array is an acoustic sensor array composed of more than two microphones.

[0028] Furthermore, the human voice recognition module uses the TCN network to determine whether there is a human voice in the currently processed sound segment:

[0029] The TCN network is divided into TCN module 1, TCN module 2 and TCN module 3. TCN modules 1-3 have the same structure, which are all composed of hole convolution, BN layer, ReLU layer and Dropout layer connected in sequence. The hole convolution is formed by superposition of causal convolution on both sides, and the convolution kernel size is 2. The BN layer, ReLU layer and Dropout layer are conventional batch normalization layer, ReLU activation function layer and discard layer respectively; it also includes: a 1x1 convolution layer is directly connected from the input to the ReLU activation function layer of the module output to form a residual connection to avoid network degradation.

[0030] Furthermore, the voiceprint likelihood beamforming module specifically performs the following steps:

[0031] A sound source localization method based on time delay estimation first estimates the time difference between the sound reaching the sound sensors at different locations through the received signals from the sound sensors, then constructs multiple hyperboloids using the time delay value and the location information of the sound sensors, and obtains the location estimate of the sound source by calculating the intersection points of multiple hyperboloids.

[0032] For the directions of multiple sound sources, beamforming enhancement is performed separately, and then the output y(t) obtained is used for voiceprint recognition scoring.

[0033] Voiceprint recognition is performed on each enhanced sound signal. The voiceprint recognition adopts the TCN network. The voiceprint recognition network calculates the likelihood value of the network output for the predetermined person model, selects the direction corresponding to the largest value as the position direction of the predetermined person, and selects the enhanced output in the corresponding direction as the signal to be transmitted.

[0034] Beneficial effects:

[0035] The present invention provides a method and device for recording human voice in a miniaturized network cabin on a ship, which uses beamforming technology based on a microphone array to achieve the function of directional sound pickup, and enhances the specific position of the commander in the cabin, thereby effectively reducing the interference caused by environmental noise. A human voice detection algorithm is used to detect whether the picked up sound contains human voice. If it contains human voice, a speech enhancement algorithm is used for further enhancement. Otherwise, if it does not contain human voice, it is used for environmental noise estimation. The present invention also implements a commander's command extraction function based on voiceprint recognition, and automatically extracts the commander's command voice instructions based on the pre-configured commander's identity information. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flow chart of a method for recording human voice in a miniaturized network cabin for use on a ship provided by an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the TCN network structure in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the sound source orientation method;

[0039] Figure 4 Schematic diagram of the ideal acoustic environment model for time delay estimation;

[0040] Figure 5 Schematic diagram of GCC delay estimation method;

[0041] Figure 6 Schematic diagram of linear array beamforming structure. DETAILED DESCRIPTION

[0042] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0043] Embodiment 1:

[0044] The present invention provides a method for recording human voice in a miniaturized network cabin on a ship, the process of which is as follows: Figure 1 As shown, the following steps are included:

[0045] Step (1) picks up the original sound signal through a microphone array; the microphone array used in the present invention is an acoustic sensor array composed of more than two microphones.

[0046] Step (2) captures sound segments of the original sound signal, selects the sound segments in sequence for processing, and uses a voice recognition algorithm to determine whether there is a human voice in the sound segment currently being processed. If there is a human voice, step (3) is executed. If there is no human voice, the sound segment currently being processed is sent to the environmental noise library for storage.

[0047] In the embodiment of the present invention, the human voice recognition algorithm adopts a TCN network, and its network structure is as follows: Figure 2 shown.

[0048] The TCN network is divided into TCN module 1, TCN module 2 and TCN module 3. TCN modules 1-3 have the same structure, which are all connected in sequence by dilated convolution, BN layer, ReLU layer and Dropout layer. The dilated convolution is formed by superposition of causal convolution on both sides, and the convolution kernel size is 2. The BN layer, ReLU layer and Dropout layer are conventional batch normalization layer, ReLU activation function layer and dropout layer respectively; it also includes: the 1x1 convolution layer is directly connected from the input to the ReLU activation function layer of the module output to form a residual connection to avoid network degradation;

[0049] The mean square error loss function is used as the loss function of the TCN network. Assume that the number of nodes in the input layer, hidden layer, and output layer of the neural network are n, i, and m respectively, the weight of the hidden layer is w, the threshold is v, the weight of the output layer is u, and the threshold is γ. Then the input α of the jth node in the hidden layer is j The calculation formula is as follows:

[0050]

[0051] w lj is the connection weight between the lth node of the input layer and the jth node of the hidden layer, x l is the input corresponding to the lth node of the input layer;

[0052] The input β of the tth node in its output layer is t The calculation formula is as follows:

[0053]

[0054] where b j represents the output of the jth node in the hidden layer;

[0055] Set the final output of the TCN network as follows:

[0056]

[0057] They are the outputs of the 1st to mth nodes of the output layer respectively;

[0058] Then the output of the tth node in the output layer is:

[0059]

[0060] Where f is the activation function; γ t is the threshold corresponding to the tth node in the output layer;

[0061] After calculation using the mean square error loss function, the mean square error E obtained is:

[0062]

[0063] When using the TCN network for prediction, the error between the predicted value and the target value is required to be minimized. Therefore, the above calculation process is reduced in the direction of the negative gradient. The calculation formula for this reduction process is as follows:

[0064]

[0065] u jt The weight between the jth node in the hidden layer and the tth node in the output layer; where μ is the given learning rate, and the gradient value on the right side of the equal sign is calculated by the following formula:

[0066]

[0067] And by β t The calculation formula is:

[0068]

[0069] Assuming that the activation function used in the output layer is the Sigmod function, the activation function is derived to obtain:

[0070] f=f(1-f) (9)

[0071] The above TCN network output and the calculation formula of mean square error are obtained:

[0072]

[0073] Finally, the parameter u jt The update formula is:

[0074] Δu jt =μe t b j (11)

[0075] Similarly, the parameters γ are calculated separately t 、w lj 、v j The update formulas are as follows:

[0076] Δγ t =-μe t (12)

[0077] Δw lj =μg j x l (13)

[0078] Δvj =-μg j (14)

[0079] Δγ t For γ t The updated value of Δw lj w lj The update to Δv j is the threshold v of the jth node in the hidden layer j The updated value of x l is the input of the lth node in the input layer

[0080] In the above formula

[0081]

[0082] f ′ (*) is the derivative of the activation function f;

[0083] in

[0084]

[0085] Finally, we get:

[0086]

[0087] The training process is multiple cycles of the above-mentioned error back propagation algorithm. When the error is less than a predetermined value, the cycle ends and the training process is completed. After the training is completed, the trained TCN obtained can be used for voice recognition.

[0088] Step (3) identifying whether a preset specific person appears. If the specific person appears, executing step (4); if the specific person does not appear, executing step (5) using a conventional beamforming algorithm to enhance sound in a specific direction through the beamforming algorithm;

[0089] Step (4) Calculate the sound source direction through a multi-sound source directional algorithm, then perform beamforming on each sound source direction to obtain a directionally enhanced sound signal, then use each enhanced sound signal to perform voiceprint recognition, and according to the voiceprint recognition score, take the direction with the highest score for a predetermined specific person as the target direction, and take the enhanced output signal of the target direction as the voice signal of the specific person; Execute step (6)

[0090] Step (4) includes the following specific steps:

[0091] S401: a sound source localization method based on time delay estimation, wherein the method first estimates the time difference between the sound reaching the sound sensors at different positions by receiving the signal from the sound sensor, then constructs multiple hyperboloids using the time delay value and the position information of the sound sensor, and obtains the position estimation of the sound source by calculating the intersection points of the multiple hyperboloids; Figure 3 This is the principle diagram of the sound source directional method.

[0092] (1) Ideal environment model

[0093] In an ideal acoustic environment, ignoring the attenuation of signal propagation, the signals received by the two acoustic sensors are expressed as

[0094]

[0095] Where s(n) and s(n-τ) are the sound signals of the two acoustic sensors, b 1 (n) and b 2 (n) is the background noise, and τ is the relative delay between the signals received by the two microphones. Figure 4 Ideal acoustic environment model for time delay estimation

[0096] (2) Delay Estimation

[0097] The cross-spectral function of the signal received by the microphone is So the generalized cross-correlation function is X 1 (f) x 1 (n), x 2 (n) The corresponding spectrum

[0098]

[0099] f is the frequency, where φ(f) is the weighting function, is the generalized cross-spectral function, that is

[0100]

[0101] So the delay is estimated to be

[0102]

[0103] In practical applications, the choice of φ(f) is very important. In the GCC method, φ(f) = 1. In the noise-free case, it can be seen from equation (20) that X i (f) can be expressed as

[0104]

[0105] α i is the frequency domain transform coefficient of signal i, S(f) is the frequency domain signal of s(n), then

[0106]

[0107] Ψ cc (f) is the generalized cross-correlation function, Ψ ccThe value of (f) depends on the sound source signal, so the delay estimation can be obtained. The specific process is as follows Figure 5 shown.

[0108] Obviously, the TDOA information is contained in the phase of equation (22). Therefore, pre-whitening the input signal can sharpen the peak of the cross spectrum. For example, when taking When there is no noise, the cross spectrum It can be expressed as

[0109]

[0110] Table 1: Commonly used generalized cross-correlation weighting functions

[0111]

[0112] at this time Depends only on τ 12 , so the PHAT method can achieve better performance than GCC. For different noise and reverberation conditions, different weighting functions φ(f) can be selected to make Has a sharper spectral peak;

[0113] In the sound source localization method based on time delay estimation, the second step is to locate the sound source using the time delay information and array position information. The sound source localization problem is described as follows: given the topological structure of the acoustic sensor array, the TDOA information between different acoustic sensor pairs and the acoustic sensor array position information are used to jointly determine the direction of the sound source;

[0114] S402: For the directions of multiple sound sources, beamforming enhancement is performed respectively, and then the obtained output y(t) is used for voiceprint recognition scoring in S403.

[0115] Figure 6 The figure shows a simple linear array beamforming structure. M microphones perform spatial sampling on the sound wave, and the system output y(t) at time t is the spatial sample x m (t), m = 0, 1, ..., M-1 instantaneous linear combination, that is:

[0116]

[0117] The * indicates complex conjugation. Represents the sampling gain of the mth microphone for spatial sampling

[0118] Narrowband beamformers are only effective for sinusoidal or narrowband signals, that is, signals with bandwidth narrow enough that the signals received at both ends of the array are still correlated. Assume that the input signal is an impulse complex plane wave e jwt , its angular frequency is w, and its angle of arrival is Assuming that the phase of the first array element receiving signal is 0, the first array element receiving signal is x 0 (t) = e jwt , the mth array element receives the signal m=1,2,…,M-1, where τ m is the propagation delay of the wave from the first microphone to the mth microphone, which is a function of the arrival angle θ; the output of the beamformer is:

[0119]

[0120] where τ 0 =0; the response of the beamformer is:

[0121]

[0122] The vector W contains the complex conjugate coefficients of the M sensors, and H represents the conjugate transpose, that is,

[0123] W=[w 0 w 1 …w M-1 ] T

[0124] T represents transpose, and the vector d(ω,θ) is the array response vector, also called the focusing vector or pointing vector, that is,

[0125] d(ω,θ)=[1e^(-jwτ_1)…e^(-jwτ_(M-1))] T

[0126] Assume that the array element spacing d = λ / 2, λ is the wavelength of sound, then Then the narrowband beamformer response is written as

[0127]

[0128] The frequency response of a finite length impulse response FIR filter with the same coefficients is:

[0129]

[0130] Where Ω∈[-π,π] is the normalized frequency; and Therefore πsinθ∈[-π,π];

[0131] A necessary input of beamforming is the enhancement direction angle θ. If there are multiple sound sources, there can be multiple enhancement directions. When there are multiple enhancement directions, beamforming enhancement is performed respectively, and then the obtained output y(t) is executed in S403.

[0132] S403: Perform voiceprint recognition on each enhanced sound signal. The voiceprint recognition adopts TCN network to calculate the likelihood value of the network output for the predetermined person model, select the direction corresponding to the largest value as the position direction of the predetermined person, and select the enhanced output in the corresponding direction as the signal to be transmitted.

[0133] Step (5) first uses the direction of arrival estimation DOA algorithm to obtain the direction of the sound source, and then uses the beamforming algorithm to enhance the original sound signal picked up by the microphone array with the direction of the sound source as the microphone array direction to obtain an enhanced sound signal; execute step (6);

[0134] Step (6) selects one of the multiple sound signals picked up by the microphone array as the first channel, and uses the enhanced sound signal obtained in step (5) or the voice of the specific person obtained in step (4) as the second channel to form a two-channel sound signal, and then encapsulates it according to the UDP protocol to form a UDP data packet;

[0135] Step (7) sends the data packet to the backend server via the network.

[0136] Embodiment 2:

[0137] A miniaturized marine network cabin voice recording device, characterized in that it comprises a microphone array, a voice recognition module, a voiceprint recognition module, a voiceprint likelihood beamforming module, a conventional beamforming module and a UDP packet module;

[0138] The microphone array is used to pick up the original sound signal and send it to the voice recognition module.

[0139] The voice recognition module captures sound fragments of the original sound signal, selects sound fragments in sequence for processing, and uses the voice recognition algorithm to determine whether there is a human voice in the sound fragment currently being processed. If there is a human voice, the sound fragment currently being processed is sent to the voiceprint recognition module. If there is no human voice, the sound fragment currently being processed is sent to the environmental noise library for storage.

[0140] The voiceprint recognition module is used to identify whether a preset specific person appears. If a specific person appears, the currently processed sound segment is sent to the voiceprint likelihood beamforming module. If no specific person appears, the conventional beamforming module is used.

[0141] The voiceprint likelihood beamforming module calculates the direction of the sound source through a multi-sound source directional algorithm, and then performs beamforming on each sound source direction to obtain a directionally enhanced sound signal. The enhanced sound signals are then used to perform voiceprint recognition. According to the voiceprint recognition score, the direction with the highest score for a predetermined specific person is taken as the target direction, and the enhanced output signal of the target direction is taken as the voice signal of the specific person and sent to the UDP packet module.

[0142] The conventional beamforming module first uses the direction of arrival estimation DOA algorithm to obtain the direction of the sound source, and then uses the beamforming algorithm to enhance the currently processed sound segment based on the direction of the sound source as the microphone array direction, obtains the enhanced sound signal, and sends it to the UDP packet module.

[0143] The UDP packet module selects one of the multiple sound signals picked up by the microphone array as the first channel, and uses the enhanced sound signal obtained by the conventional beamforming module or the voice of a specific person obtained by the voiceprint likelihood beamforming module as the second channel to form a two-channel sound signal, which is then packetized according to the UDP protocol to form a UDP data packet; the data packet is then sent to the background server through the network.

[0144] In the embodiment of the present invention, the microphone array is an acoustic sensor array composed of more than two microphones.

[0145] In the embodiment of the present invention, the human voice recognition module uses the TCN network to determine whether there is a human voice in the currently processed sound segment:

[0146] The TCN network is divided into TCN module 1, TCN module 2 and TCN module 3. TCN modules 1-3 have the same structure, which are all composed of hole convolution, BN layer, ReLU layer and Dropout layer connected in sequence. The hole convolution is formed by superposition of causal convolution on both sides, and the convolution kernel size is 2. The BN layer, ReLU layer and Dropout layer are conventional batch normalization layer, ReLU activation function layer and discard layer respectively; it also includes: a 1x1 convolution layer is directly connected from the input to the ReLU activation function layer of the module output to form a residual connection to avoid network degradation.

[0147] In the embodiment of the present invention, the voiceprint likelihood beamforming module specifically performs the following steps:

[0148] A sound source localization method based on time delay estimation first estimates the time difference between the sound reaching the sound sensors at different locations through the received signals from the sound sensors, then constructs multiple hyperboloids using the time delay value and the location information of the sound sensors, and obtains the location estimate of the sound source by calculating the intersection points of multiple hyperboloids.

[0149] For the directions of multiple sound sources, beamforming enhancement is performed separately, and then the output y(t) obtained is used for voiceprint recognition scoring.

[0150] Voiceprint recognition is performed on each enhanced sound signal. The voiceprint recognition adopts the TCN network. The voiceprint recognition network calculates the likelihood value of the network output for the predetermined person model, selects the direction corresponding to the largest value as the position direction of the predetermined person, and selects the enhanced output in the corresponding direction as the signal to be transmitted.

[0151] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for recording human voice in a miniaturized network cabin for a ship, characterized in that: The steps include: Step (1) Pick up the original sound signal through the microphone array Step (2) intercepts sound segments of the original sound signal, selects sound segments in sequence for processing, and determines whether there is a human voice in the sound segment currently being processed by a human voice recognition algorithm. If there is a human voice, step (3) is executed. If there is no human voice, the sound segment currently being processed is sent to an environmental noise library for storage; Step (3) identifying whether a preset specific person appears. If the specific person appears, executing step (4); if the specific person does not appear, executing step (5) using a conventional beamforming algorithm to enhance sound in a specific direction through the beamforming algorithm; Step (4) Calculate the direction of the sound source by using a multi-sound source directional algorithm, then perform beamforming on each sound source direction to obtain a directionally enhanced sound signal, then use each enhanced sound signal to perform voiceprint recognition respectively, and according to the voiceprint recognition score, take the direction with the highest score for a predetermined specific person as the target direction, and take the enhanced output signal of the target direction as the voice signal of the specific person; execute step (6) Step (5) First use the direction of arrival estimation DOA algorithm to obtain the direction of the sound source, then use the beamforming algorithm with the sound source direction as the microphone array direction to enhance the original sound signal picked up by the microphone array to obtain an enhanced sound signal; execute step (6); Step (6) selects one of the multiple sound signals picked up by the microphone array as the first channel, and uses the enhanced sound signal obtained in step (5) or the voice of the specific person obtained in step (4) as the second channel to form a two-channel sound signal, and then encapsulates it according to the UDP protocol to form a UDP data packet; Step (7) sends the data packet to the backend server via the network.

2. A method for recording human voice in a miniaturized network cabin for a ship as claimed in claim 1, characterized in that: The microphone array is an acoustic sensor array composed of more than two microphones.

3. A method for recording human voice in a miniaturized network cabin for a ship as claimed in claim 1, characterized in that: The human voice recognition algorithm adopts TCN network: The TCN network is divided into TCN module 1, TCN module 2 and TCN module 3. TCN modules 1-3 have the same structure, which are all composed of hole convolution, BN layer, ReLU layer and Dropout layer connected in sequence. The hole convolution is formed by superposition of causal convolution on both sides, and the convolution kernel size is 2. The BN layer, ReLU layer and Dropout layer are conventional batch normalization layer, ReLU activation function layer and discard layer respectively; it also includes: a 1x1 convolution layer is directly connected from the input to the ReLU activation function layer of the module output to form a residual connection to avoid network degradation.

4. A method for recording human voice in a miniaturized network cabin for a ship as claimed in claim 1, characterized in that: The step 4 comprises the following specific steps: S401: a sound source localization method based on time delay estimation, wherein the method first estimates the time difference between the sound reaching the sound sensors at different positions by receiving the signal from the sound sensor, then constructs multiple hyperboloids using the time delay value and the position information of the sound sensor, and obtains the position estimation of the sound source by calculating the intersection points of the multiple hyperboloids; S402: For the directions of multiple sound sources, beamforming enhancement is performed respectively, and the obtained output y(t) is used for voiceprint recognition scoring in S403; S403: Perform voiceprint recognition on each enhanced sound signal. The voiceprint recognition adopts TCN network. The voiceprint recognition network calculates the likelihood value of the network output for the predetermined person model, selects the direction corresponding to the largest value as the position direction of the predetermined person, and selects the enhanced output in the corresponding direction as the signal to be transmitted.

5. A miniaturized network cabin voice recording device for ships, characterized in that: It includes microphone array, human voice recognition module, voiceprint recognition module, voiceprint likelihood beamforming module, conventional beamforming module and UDP packet module; The microphone array is used to pick up the original sound signal and send it to the voice recognition module; The human voice recognition module intercepts the sound segments of the original sound signal, selects the sound segments in sequence for processing, and determines whether there is a human voice in the sound segment currently being processed through the human voice recognition algorithm. If there is a human voice, the sound segment currently being processed is sent to the voiceprint recognition module; if there is no human voice, the sound segment currently being processed is sent to the environmental noise library for storage; The voiceprint recognition module is used to identify whether a preset specific person appears. If a specific person appears, the currently processed sound segment is sent to the voiceprint likelihood beamforming module. If no specific person appears, the conventional beamforming module is used. The voiceprint likelihood beamforming module calculates the sound source direction through a multi-sound source directional algorithm, and then performs beamforming on each sound source direction to obtain a directionally enhanced sound signal, and then uses each enhanced sound signal to perform voiceprint recognition respectively. According to the voiceprint recognition score, the direction with the highest score for a predetermined specific person is taken as the target direction, and the enhanced output signal of the target direction is taken as the voice signal of the specific person and sent to the UDP packet module; The conventional beamforming module first uses the direction of arrival estimation DOA algorithm to obtain the direction of the sound source, and then uses the beamforming algorithm to enhance the currently processed sound segment with the direction of the sound source as the microphone array direction, obtains the enhanced sound signal, and sends it to the UDP packet module; The UDP packet module selects one of the multiple sound signals picked up by the microphone array as the first channel, and uses the enhanced sound signal obtained by the conventional beamforming module or the voice of the specific person obtained by the voiceprint likelihood beamforming module as the second channel to form a two-channel sound signal, which is then packetized according to the UDP protocol to form a UDP data packet; and then the data packet is sent to the background server through the network.

6. A miniaturized network cabin voice recording device for ships as claimed in claim 5, characterized in that: The microphone array is an acoustic sensor array composed of more than two microphones.

7. A miniaturized network cabin voice recording device for ships as claimed in claim 5, characterized in that: The human voice recognition module uses the TCN network to determine whether there is a human voice in the currently processed sound segment: The TCN network is divided into TCN module 1, TCN module 2 and TCN module 3. TCN modules 1-3 have the same structure, which are all composed of hole convolution, BN layer, ReLU layer and Dropout layer connected in sequence. The hole convolution is formed by superposition of causal convolution on both sides, and the convolution kernel size is 2. The BN layer, ReLU layer and Dropout layer are conventional batch normalization layer, ReLU activation function layer and discard layer respectively; it also includes: a 1x1 convolution layer is directly connected from the input to the ReLU activation function layer of the module output to form a residual connection to avoid network degradation.

8. A miniaturized network cabin voice recording device for ships as claimed in claim 5, characterized in that: The voiceprint likelihood beamforming module specifically performs the following steps: A sound source localization method based on time delay estimation. This method first estimates the time difference between the sound reaching the sound sensors at different locations through the received signal of the sound sensor, then constructs multiple hyperboloids using the time delay value and the location information of the sound sensor, and obtains the location estimate of the sound source by calculating the intersection of multiple hyperboloids. For the directions of multiple sound sources, beamforming enhancement is performed separately, and the obtained output y(t) is used for voiceprint recognition scoring; Voiceprint recognition is performed on each enhanced sound signal. The voiceprint recognition adopts the TCN network. The voiceprint recognition network calculates the likelihood value of the network output for the predetermined person model, selects the direction corresponding to the largest value as the position direction of the predetermined person, and selects the enhanced output in the corresponding direction as the signal to be transmitted.

Images