Artificial intelligence-based automatic marine rescue signal identification system

Abstract

An automatic marine rescue signal identification system according to an embodiment of the present disclosure is connected to a ship terminal through a network and capable of identifying a speaker of a rescue signal. The automatic marine rescue signal identification system comprises: a sound receiving unit for receiving sound information including voice and sounds generated from a ship from the ship terminal; a sound transmission unit for digitizing the sound information received from the sound receiving unit to generate and transmit sound data; a rescue signal identification unit for extracting voice data corresponding to a human voice from the sound data received from the sound transmission unit, and for identifying a rescue signal from the extracted voice data; and a speaker identification unit for identifying a speaker of the rescue signal if the rescue signal identification unit identifies the rescue signal from the voice data.

Description

AI-based automatic identification system for marine rescue signals

[0001] The present disclosure relates to an artificial intelligence-based automatic identification system and method for marine rescue signals, and more particularly, to a system and method for automatically identifying rescue signals by utilizing radio wave classification when a rescue signal is input to a receiver.

[0002] Unless otherwise indicated herein, the materials described in this section are not prior art to the claims of this application, and their inclusion in this section is not intended to be admitted as prior art.

[0003] Rapid recognition of maritime distress signals by rescue agencies is paramount during the rescue process. As part of the international community's efforts to improve distress signal listening, the International Maritime Organization (IMO), a UN agency, designated VHF 16 as a distress and safety listening channel through the International Telecommunication Union (ITU) and mandated that all rescue agencies worldwide listen for distress signals 24 / 7. In South Korea, the Korea Coast Guard monitors distress signals 24 hours a day.

[0004] Traditionally, when a vessel is in distress at sea, a distress signal is transmitted via a transmitter onboard the vessel, which is then relayed to nearby vessels. The Coast Guard then recognizes the distress signal and initiates rescue operations. The distress signal serves as the initial signal to initiate rescue operations, and accurately listening to it is crucial for securing the golden hour for rescue.

[0005] With over 100,000 vessels registered near South Korea, and neighboring countries like China and Japan boasting some of the world's highest maritime traffic, a wide variety of frequencies and channels exist. However, current distress signal listening methods can only accommodate a single channel. Furthermore, analog distress signal listening methods, which require only three people to cover the entire waters daily, have limitations. Furthermore, in an environment where listening is required 24 / 7 without breaks, fatigue can lead to decreased concentration, leaving users exposed to distress signal blind spots. Furthermore, wireless communication suffers from the drawbacks of this distress signal listening method, with reception range and sensitivity rapidly declining due to weather conditions caused by natural disasters or the aging of equipment and speakers.

[0006] The disclosed content aims to provide a marine rescue signal automatic identification system capable of automatically identifying a ship in distress by extracting voice data based on acoustic information generated from the ship and identifying the rescue signal and the speaker of the rescue signal, in order to solve the aforementioned problems.

[0007] A marine rescue signal automatic identification system capable of identifying a speaker of a rescue signal by being connected to a ship terminal and a network according to an embodiment of the present disclosure may include: an acoustic reception unit that receives acoustic information including voices and sounds generated in a ship from a ship terminal; an acoustic transmission unit that digitizes the acoustic information received from the acoustic reception unit to generate and transmit acoustic data; a rescue signal identification unit that extracts voice data corresponding to a human voice from the acoustic data received from the acoustic transmission unit and identifies a rescue signal from the extracted voice data; and a speaker identification unit that identifies a speaker of the rescue signal when the rescue signal identification unit identifies the rescue signal from the voice data.

[0008] As an embodiment, the marine rescue signal automatic identification system of the present disclosure further includes an artificial intelligence unit based on a large-scale language model, wherein the sound transmission unit transmits the voice data to the artificial intelligence unit, the artificial intelligence unit converts the transmitted voice data into text to generate text data, and if the generated text data includes any one of the keywords of 'mayday', 'help me', 'fire', and 'sinking', and if it is a rescue signal in context, the system transmits rescue signal identification information indicating that the rescue signal is included to the rescue signal identification unit, and the rescue signal identification unit can identify the rescue signal by receiving the rescue signal identification information from the artificial intelligence unit.

[0009] As an embodiment, the marine rescue signal automatic identification system of the present disclosure further includes a deep learning unit that learns standard voice data for standard sentences of a person in charge, wherein the person in charge includes all persons boarding a ship, including both the captain and crew members, and the standard voice data is digitized data of voice information recorded when the person in charge reads the standard sentences; and when the rescue signal identification unit identifies the rescue signal from the voice data, the deep learning unit receives the voice data including the rescue signal from the rescue signal identification unit, compares the voice data with the standard voice data to identify a speaker of the voice data including the rescue signal, and then transmits speaker identification information including information on the speaker of the voice data to the speaker identification unit, and the speaker identification unit can identify the speaker of the rescue signal by receiving the speaker identification information from the deep learning unit.

[0010] As an example, the marine rescue signal automatic identification system of the present disclosure may further include a vessel information receiving unit that receives a vessel ID and vessel location information from the vessel terminal using a portion of the bandwidth of the network.

[0011] The marine rescue signal automatic identification system according to the embodiment of the present disclosure can quickly respond to a ship's distress by automatically identifying the rescue signal and the speaker of the rescue signal, thereby minimizing damage to life and property.

[0012] The marine rescue signal automatic identification system according to an embodiment of the present disclosure can quickly respond to a ship's distress without having to inquire about the ship's location information from the person in charge of the distressed ship by receiving the ship ID and ship's location information from the ship terminal using a portion of the network bandwidth, thereby minimizing damage to life and property.

[0013] The effects of the present invention are not limited to the effects described above, and should be understood to include all effects that can be inferred from the detailed description of the present invention or the composition of the invention described in the claims.

[0014] FIG. 1 is a configuration diagram of an automatic marine rescue signal identification system according to an embodiment of the present disclosure.

[0015] FIG. 2 is a configuration diagram of a deep learning unit of an automatic marine rescue signal identification system according to an embodiment of the present disclosure.

[0016] FIG. 3 is a conceptual diagram of a process for converting standard voice data and voice data into frequency domain components performed in a preprocessing unit according to an embodiment of the present disclosure.

[0017] Figure 4 is a hierarchical diagram of a deep learning unit according to an embodiment of the present disclosure.

[0018] FIG. 5 is a drawing explaining the operation of a vessel information receiving unit of a marine rescue signal automatic identification system according to an embodiment of the present disclosure.

[0019] Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0020] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0021] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0022]

[0023] Hereinafter, a marine rescue signal automatic identification system (100) according to an embodiment of the present disclosure will be described in detail with reference to the drawings.

[0024] FIG. 1 is a block diagram of a marine rescue signal automatic identification system (100) according to an embodiment of the present disclosure. The marine rescue signal automatic identification system (100) communicates with a ship terminal (10) via a network, and also communicates with a maritime security information situation center server (20) via a network. The marine rescue signal automatic identification system (100) of the present disclosure may be a single computing device including a CPU and a memory, or a combination thereof. The ship terminal (10) may be a VHF radio terminal, a mobile communication terminal, or a satellite communication terminal, and is not limited thereto as long as it is a terminal capable of voice communication by connecting to a network. The maritime security information situation center server (20) may be a single computing device including a CPU and a memory, or a combination thereof. The maritime security information situation center server (20) may receive a report including a rescue signal and information about the speaker of the rescue signal from the marine rescue signal automatic identification system (100) of the present disclosure, and may enable measures to be taken in response to a ship's distress. As can be seen in FIG. 1, the marine rescue signal automatic identification system (100) of the present disclosure may include an acoustic receiver (110), an acoustic transmitter (120), a rescue signal identification unit (130), and a speaker identification unit (140). The aforementioned components of the marine rescue signal automatic identification system (100) of the present disclosure may be implemented in the form of software or hardware in the marine rescue signal automatic identification system (100).

[0025] The marine rescue signal automatic identification system (100) of the present disclosure may include a communication module for communicating with a ship terminal (10). The communication module may be a VHF wireless communication module, a mobile communication module, or a satellite communication module corresponding to the ship terminal (10).

[0026] The acoustic receiver (110) can receive acoustic information including voices and sounds generated on the ship from the ship terminal (10). Here, voice refers to a human voice that contains meaning or emotion emitted through the throat, and sound refers to all sounds that can be generated on the ship other than a human voice. When sound including voices and sounds is generated on the ship, the ship terminal (10) can transmit the acoustic information to the communication module of the marine rescue signal automatic identification system (100) via a network, and the communication module can transmit the transmitted acoustic information to the acoustic receiver (110).

[0027] The sound transmission unit (120) can digitize the sound information received from the sound reception unit (110) to generate sound data and transmit it to the structural signal identification unit (130) described later. Here, digitization means quantizing the amplitude of the sound information in the time domain and converting it into data. The purpose of digitizing the sound information in the sound transmission unit (120) to generate sound data is to enable the structural signal identification unit (130) to easily extract voice data from the sound data.

[0028] The structural signal identification unit (130) can extract voice data corresponding to a human voice from the sound data received from the sound transmission unit (120) and identify a structural signal from the extracted voice data.

[0029] The method for extracting voice data from audio data is as follows.

[0030] The rescue signal identification unit (130) can store various types of sounds that may occur on a ship in advance. Examples of various types of sounds that may occur on a ship include the noise of objects falling on the ship, the sound of footsteps of ship personnel moving busily in preparation for ship distress such as fire or sinking, etc. The rescue signal identification unit (130) can learn acoustic data for various types of sounds that may occur on a ship and derive characteristic points of the sounds of each type of sound. Before deriving characteristic points of the sounds of each type of sound, the rescue signal identification unit (130) can perform a Fourier transform to convert the acoustic data from the time domain to the frequency domain. In other words, the rescue signal identification unit (130) can learn acoustic data converted to the frequency domain for the sounds of each type of sound and derive characteristic points of the sounds of each type of sound.

[0031] The structural signal identification unit (130) receives the sound data from the sound transmission unit (120) as input, derives the characteristic points of the received sound data to determine whether the sound having characteristic points matching the characteristic points of the sounds of several stored cases is included in the received sound data, and then compares the derived characteristic points of the received sound data with the characteristic points of the sounds of several stored cases. Similarly, the structural signal identification unit (130) may perform a Fourier transform to convert the received sound data from the time domain to the frequency domain before deriving the characteristic points of the received sound data. That is, the structural signal identification unit (130) may derive the characteristic points of the sound data converted to the frequency domain.

[0032] The structural signal identification unit (130) compares the characteristic points of the acoustic data converted into the frequency domain with the characteristic points of several cases of sounds that may occur on the stored ship, and if it is determined that there is a sound of one or more stored cases among the several stored cases that have characteristic points that match the characteristic points of the acoustic data converted into the frequency domain, the reverse waveform of the sound of one or more stored cases that have characteristic points that match the characteristic points of the acoustic data is summed to the acoustic data, thereby removing sounds other than human voices occurring on the ship from the acoustic data converted into the frequency domain, thereby extracting voice data converted into the frequency domain. The voice data converted into the extracted frequency domain can be extracted by inversely converting the voice data converted into the time domain, that is, through the inverse Fourier transform. The process of extracting the voice data can be performed within the structural signal identification unit (130) or in the deep learning unit (160) described below.

[0033] The rescue signal identification unit (130) can identify a rescue signal from the extracted voice data. The extracted voice data can be converted into text using a voice recognition technique, and the rescue signal can be identified by determining whether the rescue signal is present in the converted text. The rescue signal identification unit (130) can identify a rescue signal by determining whether the converted text contains at least one keyword among “mayday,” “help me,” “fire,” and “sinking” as a rescue signal (including words in each language corresponding to the aforementioned keywords).

[0034] The marine rescue signal automatic identification system (100) of the present disclosure may further include an artificial intelligence unit (150) based on a large language model (LLM). The artificial intelligence unit (150) based on a large language model may be an external generative artificial intelligence server or generative artificial intelligence software installed within the marine rescue signal automatic identification system (100). The marine rescue signal automatic identification system (100) of the present disclosure may be linked to the artificial intelligence unit (150) based on a large language model through an API (Application Programming Interface). Since the artificial intelligence unit (150) based on a large language model can grasp the context of the text, it has the advantage of preventing an error in identifying a rescue signal simply because it contains keywords as a rescue signal.

[0035] The rescue signal identification unit (130) can transmit the extracted voice data to the artificial intelligence unit (150). The artificial intelligence unit (150) converts the received voice data into text to generate text data, and determines whether the generated text data contains at least one keyword (including words of each language corresponding to the aforementioned keywords) as a rescue signal among 'mayday', 'help me', 'fire', and 'sinking', and at the same time determines whether the text data is a rescue signal in context. The artificial intelligence unit (150) transmits rescue signal identification information indicating that the received voice data contains a rescue signal to the rescue signal identification unit (130), and the rescue signal identification unit (130) can identify the rescue signal by receiving the rescue signal identification information from the artificial intelligence unit (150).

[0036] FIG. 2 is a configuration diagram of a deep learning unit (160) of an automatic marine rescue signal identification system (100) according to an embodiment of the present disclosure, FIG. 3 is a conceptual diagram of a process of converting and processing standard voice data and voice data into frequency domain components performed in a preprocessing unit (162) according to an embodiment of the present disclosure, and FIG. 4 is a hierarchical configuration diagram of a deep learning unit (160) according to an embodiment of the present disclosure.

[0037] The speaker identification unit (140) performs the function of identifying the speaker of the rescue signal when the rescue signal identification unit (130) identifies the rescue signal from voice data. Since the marine rescue signal automatic identification system (100) of the present disclosure automatically identifies the speaker of the rescue signal, it has the advantage of being able to quickly respond to vessel distress by specifying the vessel in distress without having to ask the speaker of the rescue signal what kind of vessel is in distress. The method by which the speaker identification unit (140) identifies the speaker of the rescue signal from voice data is as follows.

[0038] The marine rescue signal automatic identification system (100) of the present disclosure may further include a deep learning unit (160). The deep learning unit (160) may learn standard voice data for standard sentences of a person in charge. Here, the person in charge includes all persons boarding the ship, including the captain and crew members, and the standard voice data is digitized data of voice information recorded when the person in charge reads the standard sentence. It is preferable that the standard sentence include at least one keyword among 'mayday', 'help me', 'fire', and 'sinking' corresponding to the rescue signal. It is even more preferable that the standard sentence include the keyword 'over' or 'abnormal'. This is because when the ship terminal (10) is a VHF radio terminal, the ship terminal (10) communicates with the marine rescue signal automatic identification system (100) of the present disclosure in a PTT (Push-To-Talk) manner, so there is a high possibility that the keyword 'over' or 'abnormal' will be included in the voice data.

[0039] When the rescue signal identification unit (130) identifies a rescue signal from voice data, the deep learning unit (160) can receive voice data including the rescue signal from the rescue signal identification unit (130), compare the voice data with standard voice data, identify the speaker of the voice corresponding to the voice data including the rescue signal, and then transmit speaker identification information including information on the speaker of the voice data to the speaker identification unit (140). The speaker identification unit (140) can identify the speaker of the rescue signal by receiving the speaker identification information from the deep learning unit (160). The above-described process will be described in more detail as follows.

[0040] As can be seen in FIG. 2, the deep learning unit (160) may include a preprocessing unit (162), a learning unit (164), and an output unit (166).

[0041] The preprocessing unit (162) performs a function of converting standard speech data and speech data including a structure signal into components in the frequency domain. Referring to FIG. 3, the speech data including the standard speech data and the structure signal are data in the time domain, and the speech data including the standard speech data and the structure signal are converted into standard speech data and speech data converted into components in the frequency domain through Fourier transform by the preprocessing unit (162). The reason for performing Fourier transform is that it is easier to extract feature points of standard speech data and speech data converted into components in the frequency domain than to extract feature points of standard speech data and speech data in the components of the time domain.

[0042] The learning unit (164) can extract feature points of the standard speech data converted into the frequency domain by the preprocessing unit (162). As described above, the standard speech data is digitized data of voice information recorded when a person in charge reads a standard sentence. The learning unit (164) can store the standard speech data for each person in charge, and extract and store feature points of the standard speech data converted into the frequency domain for each person in charge. As an example, the learning unit (164) can extract and store feature points of the standard speech data for each keyword as a structural signal. In other words, feature points of the standard speech data corresponding to the first keyword, the second keyword, etc. of the first person in charge can be extracted and stored, and feature points of the standard speech data corresponding to the first keyword, the second keyword, etc. of the second person in charge can be extracted and stored. Here, the feature points may be in the form of an amplitude graph for the frequency of the standard speech data and the speech data.

[0043] The output unit (166) can identify the speaker of the voice data including the structural signal by extracting the feature points of the voice data including the structural signal converted into the frequency domain by the preprocessing unit (162) and comparing them with the feature points of the standard voice data extracted by the learning unit (164). For example, if the voice data includes a first keyword, the speaker of the voice data can be identified more quickly by comparing the feature points of the portion of the voice data corresponding to the first keyword with the feature points of the standard voice data corresponding to the first keyword. The speaker identification unit (140) can identify the speaker of the structural signal by receiving speaker identification information including information on the specified speaker of the voice data from the output unit (166).

[0044] Referring to FIG. 4, the deep learning unit (160) of the marine rescue signal automatic identification system (100) of the present disclosure is composed of an input layer, a hidden layer, and an output layer. The input layer is a layer into which standard speech data and speech data are input, the hidden layer is a layer that learns the standard speech data to extract feature points of the standard speech data and extract feature points of the input speech data, and the output layer is a layer that identifies a speaker of the standard speech data that has feature points that match specific points of the speech data. The deep learning unit (160) learns the standard speech data as a learning data set to extract feature points of the standard speech data, and uses the speech data received from the rescue signal identification unit (130) as an input data set to extract feature points of the received speech data, and then compares the feature points with the feature points of the standard speech data to specify a speaker of the standard speech data that has feature points that match, thereby identifying the speaker of the speech data.

[0045] The artificial neural network model used in the deep learning unit (160) of the marine structure signal automatic identification system (100) of the present disclosure may be an artificial neural network (ANN), a recurrent neural network (RNN), a convolution neural network (CNN), a long short-term memory (LSTM), or a combination thereof. However, the present invention is not limited thereto.

[0046] FIG. 5 is a diagram illustrating the operation of a vessel information receiving unit (170) of a marine rescue signal automatic identification system (100) according to an embodiment of the present disclosure. As can be seen in FIG. 5, the marine rescue signal automatic identification system (100) according to an embodiment of the present disclosure may further include a vessel information receiving unit (170).

[0047] The vessel information receiving unit (170) can receive information about the vessel on which the rescue signal sender is on board from the vessel terminal (10). Referring to FIG. 5, the network through which the vessel terminal (10) and the marine rescue signal automatic identification system (100) of the present disclosure communicate may be a wireless network. Wireless networks generally have an allowable bandwidth (a usable frequency range). The marine rescue signal automatic identification system (100) of the present disclosure may allocate a portion of the allowable bandwidth and set it as a bandwidth for data transmission. Through the allocated portion of the allowable bandwidth, the vessel information receiving unit (170) can receive information about the vessel on which the rescue signal sender is on board from the vessel terminal (10). Here, the information about the vessel may include a vessel ID such as the vessel name and location information of the vessel. The vessel ID may further include a MAC address or IP address of the vessel terminal (10). The ship terminal (10) includes a GPS (Global Positioning System) module and can receive latitude and longitude information of the ship terminal (10)'s location from GPS satellites through the GPS module. The ship's location information may be latitude and longitude information of the ship terminal (10)'s location received from GPS satellites. The marine rescue signal automatic identification system (100) of the present disclosure receives the ship ID and ship's location information from the ship terminal (10) using a portion of the network bandwidth, thereby enabling a quick response to ship distress without having to inquire about the ship's location information from the person in charge of the ship in distress, thereby minimizing damage to life and property.

[0048] As described above, the marine rescue signal automatic identification system of the present disclosure can quickly respond to distress to a vessel by transmitting alarm data including the vessel ID and location information of the vessel that sent the rescue signal to the maritime security information situation center server (20) through the network when the rescue signal is identified and the speaker is identified.

[0049]

[0050] The disclosed content is merely an example, and various modifications and implementations can be made by a person skilled in the art without departing from the gist of the claims claimed in the patent, so the scope of protection of the disclosed content is not limited to the specific embodiments described above.

[0051]

[0052] (Explanation of symbols)

[0053] 10: Ship terminal

[0054] 20: Coast Guard Information Situation Center Server

[0055] 100: Marine rescue signal automatic identification system

[0056] 110: Sound receiver

[0057] 120: Sound transmission unit

[0058] 130: Structure signal identification unit

[0059] 140: Speaker identification section

[0060] 150: Artificial Intelligence Department

[0061] 160: Deep Learning Department

[0062] 162: Preprocessing unit

[0063] 164: Learning Department

[0064] 166: Output section

[0065] 170: Ship information receiving unit

Claims

1. In a marine rescue signal automatic identification system that is connected to a ship terminal and a network and can identify the speaker of a rescue signal, An acoustic receiving unit that receives acoustic information including voices and sounds generated on a ship from a ship terminal; An audio transmission unit that digitizes the audio information received from the audio reception unit to generate and transmit audio data; A structural signal identification unit that extracts voice data corresponding to a human voice from the sound data received from the sound transmission unit and identifies a structural signal from the extracted voice data; and A marine rescue signal automatic identification system including a speaker identification unit that identifies the speaker of the rescue signal when the rescue signal identification unit identifies the rescue signal from the voice data.

2. In claim 1, It further includes an artificial intelligence unit based on a large-scale language model; The above structural signal identification unit transmits the voice data to the above artificial intelligence unit, The artificial intelligence unit converts the received voice data into text to generate text data, and if the generated text data contains any one of the keywords 'mayday', 'help me', 'fire', and 'sinking', and if it is a rescue signal in context, it transmits rescue signal identification information indicating that a rescue signal is included to the rescue signal identification unit. A marine rescue signal automatic identification system characterized in that the rescue signal identification unit identifies the rescue signal by receiving the rescue signal identification information from the artificial intelligence unit.

3. In claim 1, A deep learning unit that learns standard voice data for a standard sentence of a person in charge - the person in charge includes all persons boarding a ship, including the captain and crew members, and the standard voice data is digitalized data of voice information recorded when the person in charge reads the standard sentence; When the structural signal identification unit identifies the structural signal from the voice data, the deep learning unit receives the voice data including the structural signal from the structural signal identification unit, compares the voice data with the standard voice data to identify the speaker of the voice data including the structural signal, and then transmits speaker identification information including information about the speaker of the voice data to the speaker identification unit. A marine rescue signal automatic identification system characterized in that the speaker identification unit identifies the speaker of the rescue signal by receiving the speaker identification information from the deep learning unit.

4. In claim 3, A marine rescue signal automatic identification system characterized in that the above standard sentence includes at least one keyword among 'mayday', 'help me', 'fire', 'sinking' and 'over'.

5. In claim 3, The above deep learning part A preprocessing unit that converts the above standard voice data and the above voice data including the structural signal into components of the frequency domain; A learning unit that extracts feature points of the standard voice data converted into the frequency domain in the above preprocessing unit; An automatic identification system for marine rescue signals, characterized by comprising: an output unit for extracting feature points of the voice data including the rescue signal converted to the frequency domain in the preprocessing unit and comparing the feature points with the feature points of the standard voice data extracted in the learning unit to identify the speaker of the voice data including the rescue signal; 6. In claim 1, A marine rescue signal automatic identification system, characterized in that it further includes a vessel information receiving unit that receives a vessel ID and vessel location information from the vessel terminal using a portion of the bandwidth of the network.

Images