Underwater real-time communication method, system and device based on speech recognition

Abstract

This disclosure provides an underwater real-time communication method, system, and apparatus based on speech recognition, including: in an offline state, constructing a dataset of mouth modulator-distorted speech signals-text, and based on pre-trained... ASR The model, for the ASR The parameters in the model are adjusted to obtain a target recognition model. In online mode, underwater voice data is acquired at the transmitting end, and the target recognition model is used to identify distorted voice, converting it into processed data in text form. The processed data in text form is encoded into an underwater acoustic signal and sent to the receiving end. The receiving end reconstructs the text information through underwater acoustic signal parsing and presents the text information. This disclosure effectively overcomes many shortcomings such as voice distortion caused by the limited vocal range of divers, underwater transmission distortion, high bandwidth consumption of analog voice, large communication delay, high cost of full-face mask equipment, and the inability to freely communicate using preset text.

Description

Technical Field

[0001] This disclosure relates to the field of underwater communication, and in particular to an underwater real-time communication method, system and device based on speech recognition. Background Technology

[0002] In recreational diving, underwater photography, and underwater operations, real-time information exchange between divers, and between divers and surface support personnel or underwater photographers, is crucial for ensuring diving safety and improving operational efficiency and experience. Because the underwater environment does not allow for direct air-conducted communication, conventional voice communication equipment is unsuitable; currently, underwater personnel primarily rely on dedicated underwater communication devices for communication.

[0003] In common half-mask diving scenarios, divers typically breathe underwater using a mouthpiece regulator. This method directly restricts the normal range of motion of the lips, teeth, and tongue, leading to significant distortion in underwater vocalization. The resulting unclear and indistinct pronunciation is difficult for the human ear or conventional audio equipment to accurately recognize. Furthermore, underwater acoustic channels suffer from multipath effects, water absorption, bubble noise, and environmental interference. Direct transmission of analog voice signals is prone to signal attenuation, distortion, and reverberation. Ensuring accurate voice reproduction requires high communication bandwidth, but current underwater acoustic communication bandwidth is limited. This forces compromises such as segmented transmission and reduced bitrates, further increasing transmission latency and information loss, severely impacting communication fluency and accuracy.

[0004] Some existing underwater communication solutions use bone conduction combined with ultrasonic voice transmission in an attempt to improve underwater sound reception. However, they still cannot avoid the pronunciation distortion caused by mouthpieces. At the same time, the direct transmission of analog voice signals is still limited by the characteristics of the underwater acoustic channel. The voice signal obtained by the receiving end is of low clarity. Divers need to spend a lot of attention to identify information and may even need to repeat themselves multiple times to complete simple communication, resulting in low overall communication efficiency.

[0005] Another common solution is the full-face underwater communication device, which uses a microphone built into the full-face mask to pick up sound and uses an acoustic module to transmit and play voice messages. While this type of device improves sound reception to some extent, it requires specialized full-face mask equipment, demands specialized user skills, has a high learning curve, and is expensive, bulky, and lacks aesthetic appeal and portability, making it unsuitable for the lightweight and low-cost requirements of ordinary recreational diving. Furthermore, this type of solution also primarily transmits analog voice signals, continuously occupying the communication channel, typically only supporting a one-way, turn-based speaking mode, resulting in high transmission latency and an inability to achieve smooth, two-way real-time interaction.

[0006] Besides voice communication, some existing diving wearable devices support simple underwater communication via preset text messages, allowing divers to select and send a small amount of fixed content. While this method reduces bandwidth usage, it can only transmit preset information and cannot support free expression of arbitrary content. This makes it difficult to meet the real-time communication needs in underwater portrait photography, the complex information expression in emergencies, and flexible communication among diving partners, greatly limiting its flexibility and applicability.

[0007] In summary, current underwater communication technologies generally fail to meet practical needs such as adapting to the common diving habits of people with mouthpieces, accurately recognizing distorted underwater speech, efficient transmission with low bandwidth usage, low-latency real-time interaction, and low-cost, easily accessible equipment. They have significant limitations in terms of ease of use, communication efficiency, equipment cost, and scenario adaptability, and cannot provide divers with a stable, smooth, and free real-time underwater communication experience.

[0008] Therefore, a real-time underwater communication method with high transmission efficiency and low latency is urgently needed for development. Summary of the Invention

[0009] The purpose of this disclosure is to provide an underwater real-time communication method, system, and apparatus based on speech recognition, which aims to solve at least one technical problem in the prior art.

[0010] The technical solution disclosed herein is: A real-time underwater communication method based on speech recognition includes: In offline mode, a dataset of mouth modulator-distorted speech signals and text is constructed, and based on pre-trained... Century The model, for the Century The parameters in the model are adjusted to obtain the target recognition model; When online, underwater voice data is acquired at the transmitting end, and the distorted voice is identified using the target recognition model and converted into processed data in text form; The processed data in text format is encoded into underwater acoustic signals and sent to the receiving end. The receiving end obtains the text information by analyzing and reconstructing the underwater acoustic signal, and then presents the text information.

[0011] The pre-trained Century The model, for the Century The parameters in the model are adjusted to obtain the target recognition model, including: The first method involves using a water tank to simulate a real-world scene to collect distorted speech produced by a mouthpiece, and matching it with corresponding text annotations to construct a distorted speech-text paired dataset. The speech data collected using this method is then overlaid with real underwater scene signals to complete the dataset augmentation process. The second method involves using underwater real-world data collected by divers to capture distorted speech produced by mouthpieces, and matching it with corresponding text annotations to construct a distorted speech-text paired dataset. The distorted speech-text pairing datasets obtained by any of the above methods are all based on pre-training. Century The model weights are adjusted to obtain a target recognition model adapted to underwater distortion speech recognition scenarios.

[0012] Based on pre-training Century The model adjusts its weights, including: Freeze the Century The core model parameters; In the Century Two sets of low-rank matrix product structures are added to each attention module of the encoder and decoder of the model, and the low-rank matrices are used to adjust the model weights during training.

[0013] The processed data is transmitted to the receiving end via underwater acoustic signals, including: The processed data is obtained locally at the sending end; The processed data is converted into a binary data stream using a character encoding format. The binary data stream is modulated with underwater acoustic signals to generate underwater acoustic carrier signals of different frequency bands; The sending end can manually select the communication target ID to achieve point-to-point transmission, or use broadcast mode to achieve broadcast transmission; The modulated underwater acoustic signal is emitted outward through the transducer; The receiving end receives the underwater acoustic signal, demodulates the underwater acoustic signal, and restores the processed data.

[0014] The receiving end demodulates the underwater acoustic signal and restores the processed data, including: After the underwater acoustic signal is demodulated by the underwater acoustic communication system, the receiving end receives the underwater acoustic signal and then... FFT Decoding is performed to obtain different frequency band data representing the data; Convert any frequency band data into a binary data stream, deserialize the data stream, and decode to restore the processed data.

[0015] The information presentation includes: via head-up display HUD The text is displayed in a certain way.

[0016] The information presentation includes: The processed data is synthesized into voice information locally at the receiving end, and then played back as voice at the receiving end.

[0017] An underwater real-time communication system based on speech recognition, based on the aforementioned underwater real-time communication method based on speech recognition, includes: The acquisition module is used to acquire voice data from the speaker and the transmitter. ID information; The recognition module interacts with the acquisition module to acquire the voice data and convert the voice data into processed data. The sending module interacts with the identification module to transmit the processed data and ID The information is sent in the form of sound waves; The receiving module communicates with the sending module via underwater acoustic waves to acquire the processed data. The parsing module interacts with the receiving module to reconstruct the processed data and obtain text information; The output module interacts with the parsing module to present the text information. The presentation of the text information includes: HUD Display and / or voice broadcast.

[0018] An underwater real-time communication device based on voice recognition, comprising: Underwater sealed observation assembly for underwater wear; The circuit assembly is mounted on the underwater sealed observation assembly and is based on the aforementioned underwater real-time communication system based on voice recognition. An output component, disposed on the underwater sealed observation component and electrically connected to the circuit component, is used for transmitting signals via... HUD The text information is output in the form of display and / or voice broadcast.

[0019] The underwater sealed observation component is a mask or goggles.

[0020] The beneficial effects of this disclosure include at least the following: This disclosure employs an underwater target recognition model specifically trained for underwater mouthpiece acoustic distortion and environmental noise to perform local speech recognition, converts the speech into text data, and then transmits it. HUDThe display module enables underwater visual information output. Compared with existing underwater communication technologies, it effectively overcomes numerous shortcomings, such as speech distortion caused by limited vocal range, transmission distortion due to multipath effects and noise interference in underwater acoustic channels, high bandwidth consumption of analog voice, large communication latency, high cost of full-face masks, high learning curve, and the inability to freely communicate using pre-set text. It significantly improves the accuracy and information restoration of underwater distorted speech recognition while drastically reducing bandwidth consumption and transmission latency in underwater acoustic communication. This enables smooth and efficient real-time two-way interaction, and it is compatible with conventional half-face masks and ordinary regulators. No professional operating skills are required, the equipment cost is lower, and it is safer and more aesthetically pleasing to wear. It also supports free communication of any content, fully meeting the diverse needs of underwater photography, complex situation description, and daily diving communication. HUD The visual display method further transforms the traditional auditory information reception into intuitive and efficient visual reception, significantly improving the stability, smoothness, convenience, and applicability of underwater communication, and providing more reliable and easier-to-use real-time communication support for recreational diving and underwater operations. Attached Figure Description

[0021] Figure 1 This is a flowchart of the underwater real-time communication method based on speech recognition as described in this disclosure; Figure 2 The flowchart shows how the model is fine-tuned to adapt to the aforementioned data distribution shift, thereby enabling the recognition of underwater distorted speech signals. Figure 3 This is a system block diagram of the underwater real-time communication system based on speech recognition as described in this disclosure; Figure 4 This is one embodiment of the underwater real-time communication device based on voice recognition described in this disclosure.

[0022] exist Figure 1 to Figure 4 middle: 100. Acquisition Module; 200. Identification Module; 300. Transmission Module; 400. Reception Module; 500. Analysis Module; 600. Output Module; 1. Underwater Sealed Observation Component; 2. Circuit Component; 3. Output Component. Detailed Implementation

[0023] The present disclosure will now be described in detail with reference to the accompanying drawings. Specific Implementation Example 1: This disclosure provides an embodiment: like Figure 1 An underwater real-time communication method based on speech recognition includes: in an offline state, constructing a dataset of mouth modulator-distorted speech signals-text, and based on pre-trained... Century The model, for the CenturyThe parameters in the model are adjusted to obtain the target recognition model. In the online state, underwater voice data is acquired at the transmitting end, and the target recognition model is used to identify distorted voice and convert it into processed data in text form. The processed data in text form is encoded into underwater acoustic signals and sent to the receiving end. The receiving end obtains the text information by parsing the underwater acoustic signals and presents the text information.

[0025] In this embodiment, as Figure 2 Regarding the above Century The process of adjusting the parameters in the model to obtain the target recognition model is presented in this embodiment in two different ways. The distorted speech-text pairing datasets obtained by either method are based on pre-training. Century The model weights are adjusted to obtain a target recognition model adapted to underwater distortion speech recognition scenarios.

[0026] The first method involves using a water tank to simulate a real-world scene, collecting distorted speech produced by a mouthpiece, and matching it with corresponding text annotations to construct a distorted speech-text paired dataset. Real underwater scene signals are then superimposed onto the collected speech data to complete the dataset augmentation process. This method simulates real-world scenarios and has the advantages of being fast and low-cost.

[0027] The second method involves using underwater real-world data collected by divers to capture distorted speech produced by a mouthpiece, and then matching it with corresponding text annotations to construct a distorted speech-text paired dataset. This method collects real-world data and is more accurate, but it is more expensive.

[0028] In performing pre-training Century During the process of adjusting the model weights, this embodiment freezes the aforementioned... Century The backbone model parameters; in the Century Two sets of low-rank matrix product structures are added to each attention module of the encoder and decoder of the model, and the low-rank matrices are used to adjust the model weights during training.

[0029] During the process of transmitting the processed data to the receiving end via underwater acoustic signals, to ensure the reliability of signal transmission, after the transmitting end locally acquires the processed data, it converts the processed data into a binary data stream using a character encoding format; and then modulates the binary data stream with underwater acoustic signals to generate underwater acoustic carrier signals of different frequency bands; the transmitting end manually selects the communication target. ID Point-to-point transmission or broadcast transmission is achieved; then, a modulated underwater acoustic signal is transmitted outward through a transducer; finally, the receiving end demodulates the underwater acoustic signal to restore the processed data. The character encoding format is a rule for mapping characters to binary data, such as... UTF or GBK wait.

[0030] At the receiving end, after receiving the underwater acoustic signal via demodulation by the underwater acoustic communication system, the receiving end transmits the signal through... FFT Decode the data to obtain different frequency band data representing the data; convert any frequency band data into a binary data stream, deserialize the data stream, and decode to restore the processed data.

[0031] In this embodiment, the information presentation is preferably performed via a head-up display. HUD The text can be displayed in a certain way, or the processed data can be synthesized into voice information locally on the receiving end and played back on the receiving end; the two methods can be used separately or simultaneously to adapt to different application scenarios.

[0032] It needs to be clarified that: Century The model is an automatic speech recognition model; FFT This is the Fast Fourier Transform algorithm.

[0033] In this embodiment, the steps for constructing the distorted speech-text dataset are as follows: S 1. Construct a corpus of commonly used texts for recreational diving: covering diving technical instructions such as "begin ascent", "prepare for a 5-meter 3-minute stop", and "my air pressure is 100". there is "Don't get past me," "Come closer to me," "Approach the cliff"; in photography scenes, "Hold your breath and look at the camera," "Swim over"; in biological observation scenes, "Come here, there's a white-tip reef shark," etc. S 2. Construct a general text corpus: Extract a portion of the corpus from common everyday expressions to ensure the model's ability to recognize and generalize from open corpora. This corpus can be extracted from open-source datasets. S 3. Collect distorted speech: Recruit multiple data collectors of different genders and accents; with a common diving regulator in their mouths, read out the steps sentence by sentence at a normal speaking speed in an experimental water tank. S 1 & Steps S The audio for the text mentioned in section 2 was captured using the microphone on the mask; S 4. Data cleaning: Data collectors compare the data with the text, remove misread speech data, and construct accurate matching samples of distorted speech and text.

[0034] Specific usage examples: The following scenario occurs during recreational diving: A group of two divers, by the divers A To the divers B Send a message.

[0035] S 1. Voice recording: Diver A While using the regulator, speak what you want to say, such as "We are prepared to stay for five meters and three minutes," and the speech will be captured through the microphone. S 2. Model Recognition: The audio is input into the finely tuned speech recognition model. The model encoder encodes the speech into a high-dimensional feature representation, which is then decoded by the decoder in an autoregressive manner to generate a text sequence word by word; the output is the recognition result, "We are prepared to stay for five meters and three minutes" text; S 3. Text Encoding. Encode the text into a byte stream and add communication protocol information; if the byte stream here is in Chinese, use... GBK Encoding (Chinese is 2) bytes If it is in English, use UTF -8 encoding.

[0036] S 4. Underwater acoustic modulation transmission; S 5. Demodulation at the receiving end: Restored to the text "We are prepared to stay for five meters and three minutes"; S 6. Information Output: This will include information on "divers" A "We are prepared to stay for three minutes and five meters." HUD Projection on the diver B The speech can be transmitted via a wearable device, such as a face mirror; or via a bone conduction speaker after reconstructing the speech signal using a local text-to-speech model. The speech transmission data process in this embodiment includes: converting text into a phoneme sequence; obtaining a high-dimensional feature representation using the phoneme sequence; and generating a speech signal in an autoregressive manner. Mel Spectrum; based on Mel Spectrum, reconstructing the time-domain speech waveform.

[0037] Comparison of the amount of data required by the method described in this embodiment with existing technologies: divers A Taking an 11-character message as an example; using the method described in this embodiment, 11 characters GBK The total encoding is 22 bytes; if traditional analog voice signal transmission is used, approximately 3 seconds of voice would use 8 bytes. kbps Encoding, data volume approximately 8000 x 3 / 8 = 3000 bytes; therefore, the data volume of the prior art is approximately 136 times that of the method described in this embodiment; therefore, the method described in this disclosure can significantly reduce the bandwidth occupation and transmission delay of underwater acoustic communication, and achieve smooth and efficient real-time two-way interaction. Specific Implementation Example 2: This disclosure provides another embodiment: like Figure 3 An underwater real-time communication system based on speech recognition, based on the underwater real-time communication method based on speech recognition described in Specific Embodiment 1, includes: a data acquisition module 100, a recognition module 200, a sending module 300, a receiving module 400, a parsing module 500, and an output module 600; wherein, the data acquisition module 100 is used to acquire the speech data of the speaker and the data of the sending end. ID Information; the recognition module 200 interacts with the acquisition module 100 to acquire the voice data and convert the voice data into processed data; the sending module 300 interacts with the recognition module 200 to transmit the text information and... ID The information is transmitted in the form of sound waves; the receiving module 400 communicates with the transmitting module 300 via underwater acoustic waves to acquire the text information; the parsing module 500 interacts with the receiving module 400 to reconstruct the processed data and obtain the text information; the output module 600 interacts with the parsing module 500 to present the text information; the presentation of the text information can be as follows: HUD Display and / or voice broadcast. Specific Implementation Example 3: This disclosure also provides an embodiment: like Figure 4 An underwater real-time communication device based on voice recognition includes: an underwater sealed observation component 1, a circuit component 2, and an output component 3; wherein, the underwater sealed observation component 1 is worn underwater and can be shaped as needed, preferably in the form of glasses; the circuit component 2 is disposed on the underwater sealed observation component 1 and is based on the underwater real-time communication system based on voice recognition described in specific embodiment 2; the output component 3 is disposed on the underwater sealed observation component 1 and is used to communicate via... HUD The text information is output in the form of display and / or voice broadcast.

[0040] The circuit component 2 includes, for example, an antenna 201 and a wireless module 202 for signal transmission / reception, and a main control unit 203 for voice signal recognition; the main control unit 203 integrates an underwater real-time communication method based on voice recognition as described in specific embodiment 1; the output component 3 includes a speaker or bone conduction speaker or other audio device for playing sound.

[0041] The working process is as follows: The diver, wearing an underwater sealed observation component 1 shaped like goggles, submerges underwater. The device is in a sealed and waterproof state, ensuring the stable operation of the internal circuit component 2 and the output component 3. The diver, with a breathing regulator in their mouth, emits distorted underwater speech. The acquisition module in the circuit component 2 collects the distorted speech data in real time and transmits the speech data to the main control unit 203. The main control unit 203 calls a pre-trained and fine-tuned target recognition model to recognize and process the underwater distorted speech data, converting the speech signal into processed data, and simultaneously acquiring the data from the transmitting end. ID Information. The main control unit 203 combines text information with... ID Information is transmitted to wireless module 202, which converts the processed data into a binary data stream using underwater acoustic communication encoding, employing different frequency bands to represent different frequencies. lice The encoding is completed at each bit. The encoded signal is then transmitted as an underwater acoustic signal via the transducer driven by antenna 201, allowing for target selection. ID Point-to-point transmission or global broadcast transmission. The receiving device receives underwater acoustic signals through antenna 201 and wireless module 202, and then... FFT Demodulation and decoding acquire the corresponding frequency band data, convert the frequency band data into a binary data stream and deserialize it, then the parsing module restores the data to obtain text information; the parsed text information is transmitted to output component 3, and then... HUD The head-up display is projected onto the lens, and / or the synthesized speech from the text-to-speech model is broadcast through a speaker, completing a single information interaction. The receiving end can initiate a voice response according to the same process, cyclically executing the process of voice acquisition, recognition and conversion to text, underwater acoustic encoding and transmission, signal reception and parsing, and information output, to achieve low-latency, high-accuracy, two-way real-time underwater communication between divers.

[0042] The above disclosures only cover a few specific implementation scenarios. However, this disclosure is not limited to these, and any variations that can be conceived by those skilled in the art should fall within the protection scope of this disclosure. The serial numbers in this disclosure are for descriptive purposes only and do not represent the superiority or inferiority of the implementation scenarios.

Claims

1. A real-time underwater communication method based on speech recognition, characterized in that, include: In offline mode, a dataset of mouth modulator-distorted speech signals and text is constructed, and based on pre-trained... ASR The model, for the ASR The parameters in the model are adjusted to obtain the target recognition model; When online, underwater voice data is acquired at the transmitting end, and the distorted voice is identified using the target recognition model and converted into processed data in text form; The processed data in text format is encoded into underwater acoustic signals and sent to the receiving end. The receiving end obtains the text information by analyzing and reconstructing the underwater acoustic signal, and then presents the text information.

2. The underwater real-time communication method based on speech recognition according to claim 1, characterized in that, The pre-trained ASR The model, for the ASR The parameters in the model are adjusted to obtain the target recognition model, including: The first method involves using a water tank to simulate a real-world scene to collect distorted speech produced by a mouthpiece, and matching it with corresponding text annotations to construct a distorted speech-text paired dataset. The speech data collected using this method is then overlaid with real underwater scene signals to complete the dataset augmentation process. The second method involves using underwater real-world data collected by divers to capture distorted speech produced by mouthpieces, and matching it with corresponding text annotations to construct a distorted speech-text paired dataset. The distorted speech-text pairing datasets obtained by any of the above methods are all based on pre-training. ASR The model weights are adjusted to obtain a target recognition model adapted to underwater distortion speech recognition scenarios.

3. The underwater real-time communication method based on speech recognition according to claim 2, characterized in that, Based on pre-training ASR The model adjusts its weights, including: Freeze the ASR The core model parameters; In the ASR Two sets of low-rank matrix product structures are added to each attention module of the encoder and decoder of the model, and the low-rank matrices are used to adjust the model weights during training.

4. The underwater real-time communication method based on speech recognition according to claim 1, characterized in that, The processed data is transmitted to the receiving end via underwater acoustic signals, including: The processed data is obtained locally at the sending end; The processed data is converted into a binary data stream using a character encoding format. The binary data stream is modulated with underwater acoustic signals to generate underwater acoustic carrier signals of different frequency bands; The sending end can manually select the communication target ID to achieve point-to-point transmission, or use broadcast mode to achieve broadcast transmission; The modulated underwater acoustic signal is emitted outward through the transducer; The receiving end receives the underwater acoustic signal.

5. The underwater real-time communication method based on speech recognition according to claim 4, characterized in that, After receiving the underwater acoustic signal, the receiving end includes the following processing steps: After the underwater acoustic signal is demodulated by the underwater acoustic communication system, the receiving end receives the underwater acoustic signal and then... FFT Decoding is performed to obtain different frequency band data representing the data; Convert any frequency band data into a binary data stream, deserialize the data stream, and decode to restore the processed data.

6. The underwater real-time communication method based on speech recognition according to claim 1, characterized in that, The information presentation includes: via head-up display HUD The text is displayed in a certain way.

7. The underwater real-time communication method based on speech recognition according to claim 1, characterized in that, The information presentation includes: The processed data is synthesized into voice information locally at the receiving end, and then played back as voice at the receiving end.

8. An underwater real-time communication system based on speech recognition, comprising the underwater real-time communication method based on speech recognition as described in any one of claims 1 to 7, characterized in that, include: The acquisition module is used to acquire voice data from the speaker and the transmitter. ID information; The recognition module interacts with the acquisition module to acquire the voice data and convert the voice data into processed data. The sending module interacts with the identification module to transmit the processed data and ID The information is sent in the form of sound waves; The receiving module communicates with the sending module via underwater acoustic waves to acquire the processed data. The parsing module interacts with the receiving module to reconstruct the processed data and obtain text information; The output module interacts with the parsing module to present the text information. The presentation of the text information includes: HUD Display and / or voice broadcast.

9. An underwater real-time communication device based on speech recognition, characterized in that, include: Underwater sealed observation assembly for underwater wear; A circuit assembly is disposed on the underwater sealed observation assembly and is based on the underwater real-time communication system based on voice recognition as described in claim 8; An output component, disposed on the underwater sealed observation component and electrically connected to the circuit component, is used for transmitting signals via... HUD The text information is output in the form of display and / or voice broadcast.

10. The underwater real-time communication device based on speech recognition according to claim 9, characterized in that: The underwater sealed observation component is a mask or goggles.

Images