System and method for controlling navigational attention based on maritime very high frequency(VHF) voice communication information

Abstract

The present invention relates to a maritime VHF voice communication information-based navigation attention concentration control system and method that recognizes navigation conditions based on maritime VHF voice communication information and dynamically controls the detection area and processing priority of ship sensors according to the recognized navigation conditions. The present invention receives VHF communication voice and converts it into text data, and extracts navigation semantic information including route information, object information, relative position information, and event type information from the text data. Furthermore, the present invention determines a valid navigation situation by comparing navigation state information, including the current position, speed, and direction of travel of a vessel, with the navigation semantic information, and generates a Region of Interest (ROI) based on the valid navigation situation. The present invention selectively controls at least one of a camera, RADAR, AIS, and ECDIS for an area corresponding to a generated region of interest, and can apply different artificial intelligence detection models depending on the object type. Furthermore, the present invention can dynamically reallocate computational resources according to risk level, track changes in the location, movement direction, and approach direction of a detected object, and generate warnings or autonomous navigation control inputs according to the collision risk level. Accordingly, the present invention can reduce computation for unnecessary areas, improve detection accuracy and response speed for high-risk areas, and enhance the situational awareness efficiency and navigation safety of autonomous vessels.

Description

Technology Field

[0001] The present invention relates to the field of maritime navigation support technology, and more specifically, to a navigation attention concentration control system and method based on maritime very high frequency (VHF) voice communication information that improves the situation awareness efficiency and navigation safety of an autonomous vessel or smart bridge system by recognizing navigation conditions based on maritime very high frequency (VHF) voice communication information and dynamically controlling the detection area, detection priority, and allocation of computational resources of sensors accordingly. Background Technology

[0002] Generally, autonomous vessels or navigation support systems are configured to recognize the surrounding maritime environment using various sensors, such as cameras, radar, Automatic Identification System (AIS), and Electronic Chart Display and Information System (ECDIS).

[0003] However, since most conventional autonomous navigation systems operate by uniformly processing the entire detection area, they use the same level of computational resources even in areas with a low probability of actual risk, which can lead to reduced computational efficiency and limited response speeds to emergency situations.

[0004] In particular, actual human navigators perform attention procedures by listening to information regarding specific routes, obstacles, work vessels, dredgers, or collision risk situations via very high frequency (VHF) voice communication, and then prioritizing visual observation of the relevant direction or specific area, or intensively checking radar, electronic chart display information systems (ECDIS), etc.

[0005] For example, when communications such as “Dredging vessel in operation to the right of Channel 3,” “Tugboat approaching ahead,” or “Concentration of fishing vessels to the port side” are received, the human navigator strengthens vigilance in that direction and prioritizes checking relevant sensor information.

[0006] However, conventional autonomous navigation systems have been unable to engineer and reflect these cognitive-based attention processes of human navigators, and there were limitations in utilizing very high frequency (VHF) voice communication information only as a simple recording or communication tool.

[0007] Furthermore, conventional technology lacks the capability to determine positional consistency or situational validity between objects mentioned in voice communication and actual sensor-detected objects, which could lead to the generation of unnecessary warnings or inefficient sensor processing issues. Prior art literature

[0008] Korean Registered Patent No. 10-1298925, “Multi-vessel Collision Risk Identification Support System” The problem to be solved

[0009] The present invention aims to solve the above-mentioned problems by providing a navigation attention concentration control system and method based on maritime very high frequency (VHF) voice communication information, which extracts route information, object information, relative position information, and danger event information from maritime very high frequency (VHF) voice communication information and determines a valid navigation situation by linking this information with the current navigation status of the vessel.

[0010] Furthermore, the present invention aims to provide a navigation attention concentration control system and method based on maritime very high frequency (VHF) voice communication information that can dynamically set a region of interest (ROI) for sensors such as cameras, radar, Automatic Identification System (AIS), and Electronic Chart Display Information System (ECDIS) according to a determined navigation situation, and adjust the detection priority for a specific direction or specific object.

[0011] Furthermore, the present invention aims to provide a navigation attention concentration control system and method based on maritime very high frequency (VHF) voice communication information that can improve detection efficiency and real-time response performance by selectively applying different artificial intelligence detection models according to object type and prioritizing the allocation of computational resources to high-risk regions of interest (ROI).

[0012] Furthermore, the present invention aims to provide a navigation attention control system and method based on maritime very high frequency (VHF) voice communication information that can improve the navigation safety and situation response capabilities of an autonomous vessel by engineeringly modeling the attention and situation awareness procedures of a human navigator and applying them to an autonomous navigation system. means of solving the problem

[0013] The maritime very high frequency (VHF) voice communication information-based navigation attention concentration control system according to the present invention is characterized by comprising: a very high frequency (VHF) voice input and processing module (110) that receives very high frequency (VHF) voice communication data and converts the very high frequency (VHF) voice communication data into text data; a navigation semantic interpretation module (120) that extracts navigation semantic information from the text data; a situation awareness and event matching module (130) that determines a valid navigation situation by comparing the navigation semantic information with the current navigation state; an attention concentration control module (140) that generates a region of interest (ROI) and a detection priority based on the determination result; a sensor interface and detection module (150) that controls a sensor to perform selective detection on an area corresponding to the region of interest (ROI); and a decision support and autonomous navigation linkage module (160) that analyzes the collision risk based on the distance between the detected object and the vessel, relative speed, and approach direction, and generates a warning or provides a control input to the autonomous navigation control system when the risk is greater than or equal to a set standard.

[0014] In addition, according to the present invention, the very high frequency (VHF) voice input and processing module (110) includes a voice receiving unit, a voice preprocessing unit, an automatic speech recognition (ASR) unit, an event separation unit, and a text data generation unit, wherein the voice preprocessing unit performs at least one of noise removal, voice active interval detection, and filtering, and the event separation unit separates continuously input voice communication data into event units or sentence units.

[0015] In addition, according to the present invention, the navigation semantic interpretation module (120) is characterized by being configured to extract route information, object information, relative position information, and event type information from the text data using a natural language processing-based semantic parsing model, and to normalize at least two expressions among starboard, right, and starboard into the same relative position information, or to normalize at least two expressions among port, left, and port into the same relative position information.

[0016] In addition, according to the present invention, the situation awareness and event matching module (130) collects the current position, speed, heading information and route status of the vessel, and matches the position and bearing information between the object mentioned in the voice communication and the actual detected object using Automatic Identification System (AIS) data or radar data, and is characterized by being configured to filter the event corresponding to the voice communication when the distance between the route mentioned in the voice communication and the current position of the vessel is greater than or equal to a set distance, when the relative position information mentioned in the voice communication and the bearing information of the actual detected object do not correspond to each other, or when a set time has elapsed from the time of reception of the voice communication.

[0017] In addition, according to the present invention, the attention control module (140) sets an angle range corresponding to the relative position information included in the navigation semantic information based on the ship's bow direction information as an area of ​​interest (ROI), calculates an attention weight using at least one of the risk level, object type, object approach speed, and event importance for each area of ​​interest (ROI), and sets a differential detection priority among a plurality of areas of interest (ROI) according to the attention weight.

[0018] In addition, according to the present invention, the attention focus control module (140) is characterized by being configured to dynamically recalculate the region of interest (ROI) according to the passage of time, changes in the position of the vessel, changes in bow direction information, changes in speed, or changes in the movement of surrounding objects, selectively activate different artificial intelligence detection models according to the object type, and prioritize the allocation of processing resources of a graphics processing unit or an artificial intelligence computing unit to the region of interest (ROI).

[0019] In addition, according to the present invention, the sensor interface and detection module (150) is characterized by being configured to control at least one of a camera, a radar, an automatic identification system (AIS), and an electronic chart display information system (ECDIS), and to perform high-resolution analysis or high-cycle frame processing on a camera image area corresponding to the region of interest (ROI), or to perform precise analysis on a radar bearing area corresponding to the region of interest (ROI).

[0020] In addition, according to the present invention, the decision support and autonomous navigation linkage module (160) calculates a collision risk using at least one of the distance between the detected object and the vessel, relative speed, approach direction, closest approach distance (CPA), and closest approach time (TCPA), and generates at least one of a visual warning, a voice warning, avoidance direction information, alternative route information, or autonomous navigation control input according to the collision risk.

[0021] In addition, the navigation attention concentration control method based on maritime very high frequency (VHF) voice communication information according to the present invention is characterized by comprising: a voice receiving step (S100) for receiving very high frequency (VHF) communication voice; a voice conversion step (S200) for converting the voice received in the voice receiving step (S100) into text data; a semantic information extraction step (S300) for extracting navigation semantic information including route information, object information, relative position information, and event type information from the text data generated in the voice conversion step (S200); a situation judgment step (S400) for determining a valid navigation situation by comparing navigation status information including current vessel position, speed, and direction of travel information with the navigation semantic information; an area of ​​interest generation step (S500) for generating a region of interest (ROI) based on the vessel's direction of travel based on the result of the situation judgment step (S400); and a sensor control step (S600) for controlling one or more sensors to selectively perform detection on an area corresponding to the region of interest (ROI) generated in the area of ​​interest generation step (S500).

[0022] In addition, according to the present invention, the voice conversion step (S200) is characterized by performing at least one of noise removal, voice active interval detection, and filtering on the received voice communication data, converting the voice communication data into text data using an automatic speech recognition (ASR) model that has learned specialized terminology in the maritime field, and separating and storing the continuously input very high frequency (VHF) communication voice in event units or sentence units.

[0023] In addition, according to the present invention, the semantic information extraction step (S300) is characterized by including the step of extracting the navigation semantic information using a natural language processing-based semantic parsing model and normalizing at least two expressions among starboard, right, and starboard, or at least two expressions among port, left, and port, into the same relative position information.

[0024] In addition, according to the present invention, the situation judgment step (S400) is characterized by including a step of matching location and bearing information between an object mentioned in a voice communication and an actual detected object using Automatic Identification System (AIS) data or radar data, and filtering an event corresponding to the voice communication when the distance between the route mentioned in the voice communication and the current position of the vessel is greater than or equal to a set distance, when the relative location information mentioned in the voice communication and the bearing information of the actual detected object do not correspond to each other, or when a set time has elapsed from the time of reception of the voice communication.

[0025] In addition, according to the present invention, the interest area generation step (S500) sets an angle range corresponding to the relative position information based on the ship's bow direction information as an interest area (ROI), calculates an attention weight using at least one of risk level, object type, object approach speed, and event importance, sets a differential detection priority among a plurality of interest areas (ROI) according to the attention weight, and is characterized in that the interest area (ROI) is dynamically recalculated according to the passage of time, ship movement, change in bow direction information, or change in movement of surrounding objects.

[0026] In addition, according to the present invention, the sensor control step (S600) is characterized by performing selective detection or highlighting processing on at least one of a camera, radar, Automatic Identification System (AIS), and Electronic Chart Display Information System (ECDIS), selectively applying different artificial intelligence detection models according to object type, dynamically reallocating computational resources according to risk level to increase the detection frequency or detection resolution for the region of interest (ROI), performing a precise analysis on the bearing zone corresponding to the region of interest (ROI) among the radar data, and tracking changes in the location, direction of movement, and direction of approach of the detected object.

[0027] In addition, according to the present invention, after the sensor control step (S600), the method further includes a decision support step (S700) which analyzes the collision risk based on the distance between the detected object and the vessel, relative speed, and approach direction, and generates a warning or provides input for speed adjustment, direction change, or evasive maneuver control to the autonomous navigation control system if the risk is greater than or equal to a set standard. Effects of the invention

[0028] According to the present invention, by extracting semantic information related to navigation conditions from very high frequency (VHF) voice communication information and linking it with the current navigation status, there is an effect of effectively determining risk events that are highly relevant to the actual navigation situation.

[0029] In addition, according to the present invention, a region of interest (ROI) corresponding to a direction or specific object with a high probability of risk can be dynamically set, and the detection priority for that area can be increased, thereby reducing computation on unnecessary areas and improving the overall computational efficiency of the system.

[0030] In addition, according to the present invention, different artificial intelligence detection models can be selectively activated depending on the object type, thereby having the effect of improving detection accuracy for specific objects such as dredgers, tugboats, fishing boats, or workboats.

[0031] In addition, according to the present invention, by prioritizing the allocation of processing resources of a graphics processing unit or an artificial intelligence computing unit to a region of interest (ROI), high-resolution analysis and high-cycle detection of the risk area can be performed, thereby improving real-time response performance.

[0032] In addition, according to the present invention, since the situational consistency can be verified by comparing the location and orientation information between an object mentioned in voice communication and an actual sensor-detected object, there is an effect of reducing the occurrence of unnecessary warnings or false detections.

[0033] In addition, according to the present invention, since a human navigator's cognitive-based attention procedure can be applied to an autonomous navigation system, there is an effect of improving navigation safety for autonomous vessels, smart bridge systems, port control systems, etc. Brief explanation of the drawing

[0034] FIG. 1 is a block diagram showing the overall configuration of a navigation attention concentration control system based on maritime very high frequency (VHF) voice communication information according to one embodiment of the present invention. FIG. 2 is a flowchart showing the overall data processing flow of a navigation attention concentration control system based on maritime very high frequency (VHF) voice communication information according to one embodiment of the present invention. FIG. 3 is a flowchart of a navigation attention concentration control method based on maritime very high frequency (VHF) voice communication information according to one embodiment of the present invention. Specific details for implementing the invention

[0035] Hereinafter, embodiments of the present invention will be described with reference to the attached drawings. First, it should be noted that identical components or parts in the drawings are indicated by the same reference numerals whenever possible. In describing the present invention, specific descriptions of related known functions or configurations are omitted to avoid obscuring the essence of the invention.

[0036] As used in this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "composed" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as meaning that some of the components or steps may be excluded, or that additional components or steps may be included. Furthermore, terms such as "...part," "module," etc., as used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software.

[0038] As illustrated in FIG. 1, the maritime very high frequency (VHF) voice communication information-based navigation attention concentration control system (100) of the present invention may include a very high frequency (VHF) voice input and processing module (110), a navigation semantic interpretation module (120), a situation awareness and event matching module (130), an attention concentration control module (140), a sensor interface and detection module (150), and a decision support and autonomous navigation linkage module (160).

[0039] The above very high frequency (VHF) voice input and processing module (110) can be configured to receive voice communication data from very high frequency (VHF) communication equipment installed on a ship and convert it into text data suitable for subsequent artificial intelligence processing.

[0040] The above navigation semantic interpretation module (120) may be configured to extract navigation semantic information including route information, object information, relative position information, and event type information from converted text data.

[0041] The above situation recognition and event matching module (130) can determine whether the event is a valid event related to the current navigation situation by comparing the extracted navigation semantic information with the current position, speed, heading, and route status of the vessel.

[0042] The above attention control module (140) can generate a region of interest (ROI) based on a navigation situation determined to be a valid event, and set a detection priority and an attention weight.

[0043] The sensor interface and detection module (150) can control one or more sensors, such as a camera, radar, automatic identification system (AIS), and electronic chart display information system (ECDIS), to perform region of interest (ROI)-based selective detection.

[0044] The above decision support and autonomous navigation linkage module (160) can analyze the collision risk based on the detection results and, if the risk is greater than or equal to a set standard, generate a warning or provide input to the autonomous navigation control system for speed adjustment, direction change, or avoidance maneuver control.

[0045] As illustrated in FIG. 2, the maritime very high frequency (VHF) voice communication information-based navigation attention concentration control system (100) of the present invention can be configured to have a continuous data flow from very high frequency (VHF) voice communication input to sensor control and decision support.

[0046] Specifically, when very high frequency (VHF) communication voice is received, the very high frequency (VHF) voice input and processing module (110) can preprocess the voice data and perform automatic speech recognition (ASR) to convert it into text data.

[0047] Afterward, the navigation meaning interpretation module (120) can extract route name, ship type, bearing expression, danger event, and navigation instruction information from the text data.

[0048] For example, when a voice communication is received saying “Caution: Dredger operation is underway to the right of Route 3,” the navigation semantic interpretation module (120) can extract “Route 3” as route information, “right” as relative position information, “drecker” as object information, and “caution” as event type information.

[0049] Next, the situation awareness and event matching module (130) can determine whether the communication event is related to the current position and course status of the vessel by linking with the electronic chart display information system (ECDIS), automatic identification system (AIS), radar, and navigation plan data.

[0050] If the above communication event is determined to be valid, the attention control module (140) can calculate an angle range corresponding to relative position information based on the heading of the vessel and set it as an area of ​​interest.

[0051] Subsequently, the sensor interface and detection module (150) can perform high-resolution image analysis, high-period radar (RADAR) analysis, or Automatic Identification System (AIS) object highlighting on the area of ​​interest.

[0052] Finally, the above decision support and autonomous navigation linkage module (160) can analyze the collision risk based on the distance between the detected object and the vessel, relative speed, and approach direction, and generate warning or avoidance control information if necessary.

[0053] In addition, the system (100) can respond to changes in real-time navigation conditions by updating the area of ​​interest, detection priority, and attention weight when a new very high frequency (VHF) voice communication is received, when the position, speed, or heading of the vessel changes, or when the movement status of surrounding objects changes.

[0054] The very high frequency (VHF) voice input and processing module (110) illustrated in FIG. 1 may include a voice receiving unit, a voice preprocessing unit, an automatic voice recognition unit, an event separation unit, and a text data generation unit.

[0055] The above voice receiver can receive real-time voice communication data from very high frequency (VHF) communication equipment installed on the ship.

[0056] The above voice preprocessing unit can perform preprocessing to remove or reduce sea noise, engine noise, wave noise, radio noise, and background noise included in the received voice data.

[0057] For example, the voice preprocessing unit can improve voice recognition accuracy by performing frequency filtering, spectrum subtraction, adaptive noise removal, or voice active interval detection (VAD).

[0058] The above automatic speech recognition unit can convert preprocessed speech data into text data and can utilize a domain-specific automatic speech recognition (ASR) model that has learned specialized terminology in the maritime field.

[0059] Accordingly, the above automatic voice recognition unit can recognize maritime-specific terms such as “dredging vessel,” “tugboat,” “fishing vessel,” “starboard,” “port,” “arrival,” “departure,” and “3rd route” with high accuracy.

[0060] The above event separation unit can separate continuously input voice data into event units or sentence units.

[0061] In addition, the above event separation unit may assign a priority processing flag to urgent communications, repetitive communications, or communications containing danger keywords.

[0062] The text data generation unit can generate and store recognized text data and event unit information in the form of structured data. At this time, the text data generation unit (115) can store the generation time, sending vessel information, call sign, channel information, and priority information together.

[0063] In addition, the text data generation unit can transmit the stored event unit text data to the subsequent navigation semantic interpretation module (120), and the event unit text data can be reused as a navigation log or artificial intelligence learning data.

[0064] The navigation semantic interpretation module (120) illustrated in FIG. 1 can analyze text data generated as a result of voice recognition to extract semantic information necessary for determining the navigation situation.

[0065] The above navigation semantic interpretation module (120) can identify route information, object information, relative position information, event type information, and risk information from communication content using a natural language processing-based semantic parsing model.

[0066] For example, when the text “3rd channel right dredger operation in progress” is entered, the navigation meaning interpretation module (120) can separate “3rd channel” into channel information, “right” into relative position information, “dredging vessel” into object information, and “operation in progress” into event type information.

[0067] In addition, the navigation meaning interpretation module (120) can normalize expressions having the same meaning, such as “starboard,” “right,” and “starboard,” into the same relative position information.

[0068] Likewise, “port,” “left,” and “port” can be normalized to the same relative position information.

[0069] In addition, the navigation meaning interpretation module (120) can classify expressions such as “caution,” “collision risk,” “deceleration wind,” “approaching,” and “operation” as danger events or navigation instruction information.

[0070] The above navigation semantic interpretation module (120) can assign risk and event importance to the extracted semantic information, and can increase the event importance for the same event that is received repeatedly.

[0071] Additionally, the navigation semantic interpretation module (120) can convert the extracted navigation semantic information into structured data and transmit it to the situation awareness and event matching module (130). For example, the structured data may include a route identifier, object type, relative position, risk keyword, time of event occurrence, and risk score.

[0072] The situation awareness and event matching module (130) illustrated in FIG. 1 can collect information on the current ship's position, speed, heading, route status, and surrounding objects in real time and compare it with navigation semantic information extracted from the navigation semantic interpretation module (120).

[0073] The above situation recognition and event matching module (130) can determine whether the current vessel is entering, approaching, or has already passed the route mentioned in the communication using the electronic chart display information system (ECDIS) and navigation plan data.

[0074] For example, even if a message “Caution: Right side of Route 3” is received in a very high frequency (VHF) communication, if the vessel is currently located in an area unrelated to Route 3 or has already passed through Route 3, the situation recognition and event matching module (130) can filter the communication event as an invalid event.

[0075] In addition, the above situation awareness and event matching module (130) can detect surrounding objects using Automatic Identification System (AIS) data or radar data, and match location and bearing information between the object mentioned in the communication and the actual detected object.

[0076] For example, if information such as “right-direction tug approach” is received in a communication, the situation awareness and event matching module (130) can search for an Automatic Identification System (AIS) object or a radar object in the right-direction bearing sector based on the current heading of the vessel.

[0077] In addition, the above situation recognition and event matching module (130) can verify situation consistency by comparing the relative position mentioned in the event with the bearing value of the actual detected object.

[0078] The above situation awareness and event matching module (130) can prevent unnecessary attention activation by filtering out events that are not relevant to the current navigation situation or are not temporally valid.

[0079] In addition, the above-mentioned situation recognition and event matching module (130) can calculate an event validity score using at least one of the time of event occurrence, the current direction of movement of the vessel, the distance from the corresponding route, the location of surrounding objects, and the direction of approach to objects. If the event validity score is greater than or equal to a set standard, the event can be determined to be a valid navigation situation.

[0080] The attention control module (140) illustrated in FIG. 1 can generate a region of interest based on the judgment result of the situation awareness and event matching module (130), and determine the detection priority and the method of allocating computational resources.

[0081] The above attention control module (140) can calculate an angle range corresponding to relative position information based on the current heading information of the vessel.

[0082] For example, if relative position information such as “right” or “starboard” is extracted, the attention control module (140) can set a certain angle range corresponding to the right direction relative to the heading as an area of ​​interest.

[0083] In addition, if an event called “forward attention” is extracted, the attention focus control module (140) can set the forward area based on the heading as the area of ​​interest.

[0084] The above attention control module (140) can calculate an attention weight based on the risk analysis results and assign a higher detection priority to areas of interest that have a high risk.

[0085] Specifically, the attention control module (140) can calculate an attention weight using at least one of the risk level, object type, object access speed, and event importance for each area of ​​interest, and can differentially set detection priorities among multiple areas of interest according to the calculated attention weight.

[0086] For example, a higher attention weight can be assigned to events containing expressions such as “collision risk,” “urgent avoidance,” or “deceleration required” than to simple informational communications.

[0087] In addition, the above attention control module (140) can selectively activate different artificial intelligence detection models depending on the object type.

[0088] For example, if a dredger is mentioned, the dredger detection model can be activated; if a tugboat is mentioned, the tugboat detection model can be activated; and if a fishing vessel is mentioned, the fishing vessel or fishing net related object detection model can be activated.

[0089] In addition, the above attention control module (140) can prioritize the allocation of processing resources of the GPU or AI computing device to the area of ​​interest.

[0090] Accordingly, high-resolution analysis or high-period detection can be performed on high-risk areas of interest, while only low-period detection or basic monitoring can be performed on low-risk areas.

[0091] In addition, the above attention control module (140) can dynamically recalculate the area of ​​interest periodically or at the time of an event according to the passage of time, changes in the position of the vessel, changes in heading, changes in speed, or changes in the movement of surrounding objects.

[0092] For example, when the ship turns and the heading changes, the absolute bearing of the area of ​​interest, which was previously set to the right, changes, so the attention control module (140) can recalculate the angle range of the area of ​​interest based on the changed heading.

[0093] The sensor interface and detection module (150) illustrated in FIG. 1 can be coupled with at least one of a camera, radar, automatic identification system (AIS), electronic chart display information system (ECDIS), infrared camera, LiDAR, or ultrasonic sensor.

[0094] The sensor interface and detection module (150) can perform selective detection of a specific area based on the area of ​​interest information set in the attention control module (140).

[0095] For example, in the case of camera images, high-resolution analysis, magnification analysis, or high-cycle frame processing can be performed on the region of interest within the entire image.

[0096] In the case of radar data, the sensor interface and detection module (150) can perform a precise analysis on a specific bearing sector corresponding to the area of ​​interest.

[0097] In the case of Automatic Identification System (AIS) data, the sensor interface and detection module (150) can prioritize displaying or tracking Automatic Identification System (AIS) objects located within or approaching the area of ​​interest.

[0098] In the case of Electronic Chart Display Information System (ECDIS) data, the sensor interface and detection module (150) can highlight routes, obstacles, work areas, or danger areas mentioned in the communication.

[0099] In addition, the sensor interface and detection module (150) can perform a tracking function for the detected object to continuously update the object's movement path, direction of movement, direction of approach, and relative speed.

[0100] For example, if it is determined that a work vessel within the area of ​​interest is approaching in the direction of the vessel, the sensor interface and detection module (150) can classify the object as a dangerous object and continuously track it.

[0101] Additionally, the sensor interface and detection module (150) can transmit detection results to the decision support and autonomous navigation linkage module (160), and the detection results may include object identification information, object location, object speed, object direction of travel, object tracking history, and sensor reliability.

[0102] The decision support and autonomous navigation linkage module (160) illustrated in FIG. 1 can analyze collision risk and navigation risk based on the detection results of the sensor interface and detection module (150).

[0103] The above decision support and autonomous navigation linkage module (160) can calculate the collision risk using at least one of the distance between the detected object and the current vessel, relative speed, approach direction, closest approach distance (CPA), and closest approach time (TCPA).

[0104] The above decision support and autonomous navigation linkage module (160) can generate a visual warning or a voice warning when the calculated risk level is greater than or equal to a set standard.

[0105] For example, a warning message such as “Caution: Dredger approaching to the starboard ahead” can be displayed on the bridge display, or a warning such as “Dangerous object approaching to the starboard ahead” can be output verbally.

[0106] In addition, the above decision support and autonomous navigation linkage module (160) can calculate an avoidance direction or an alternative route in situations where there is a high probability of collision.

[0107] The above decision support and autonomous navigation linkage module (160) can transmit the generated risk information or avoidance information to the autonomous navigation control system so that it can be used as input for speed adjustment, direction change, or avoidance maneuver control.

[0108] In addition, the above decision support and autonomous navigation linkage module (160) can store very high frequency (VHF) communication events, semantic interpretation results, interest area setting results, sensor detection results, risk analysis results and autonomous navigation linkage results as navigation logs.

[0109] The above navigation logs can be utilized as data for future accident analysis, operational evaluation, improvement of artificial intelligence models, and learning of risk patterns by route.

[0110] In addition, the above decision support and autonomous navigation linkage module (160) can determine the event to be prioritized for processing by comparing the risk level, approach time, and avoidance possibility of each risk event when multiple risk events occur simultaneously.

[0112] (Example)

[0113] For example, while an autonomous vessel is navigating along a port entry channel, it may receive voice information via very high frequency (VHF) communication stating, “Dredging work is underway to the right of Channel 3, so incoming vessels are advised to exercise caution.”

[0114] In this case, the very high frequency (VHF) voice input and processing module (110) can receive the voice, remove noise, and convert it into text data through an automatic speech recognition (ASR) model.

[0115] The navigation meaning interpretation module (120) can extract “3rd route” as route information, “right” as relative position information, “dredging vessel” as object information, and “working” and “caution” as event type information from the text data.

[0116] The situation awareness and event matching module (130) can determine whether the current vessel is actually approaching the third route by comparing the current vessel's position with the route information on the electronic chart display information system (ECDIS).

[0117] In addition, the situation awareness and event matching module (130) can use Automatic Identification System (AIS) and radar data to determine whether there is a dredger or an object presumed to be a dredger in the right direction relative to the current ship's heading.

[0118] If the above event is determined to be a valid event related to the current navigation situation, the attention control module (140) can set a certain angle range to the right of the heading as an area of ​​interest and assign a high attention weight to the area of ​​interest.

[0119] The sensor interface and detection module (150) can perform camera high-resolution analysis, radar bearing sector precision analysis, and Automatic Identification System (AIS) object priority tracking for the area of ​​interest.

[0120] The decision support and autonomous navigation linkage module (160) can calculate the collision risk by analyzing the distance, relative speed, and approach direction between the detected dredger and the current vessel.

[0121] When the risk level is above the set standard, the decision support and autonomous navigation linkage module (160) can provide a visual or voice warning to the navigator and provide a deceleration or evasive maneuver control input to the autonomous navigation control system.

[0123] As illustrated in FIG. 3, the navigation attention concentration control method based on maritime very high frequency (VHF) voice communication information according to the present invention may include a voice reception step (S100), a voice conversion step (S200), a semantic information extraction step (S300), a situation judgment step (S400), an interest region generation step (S500), a sensor control step (S600), and a decision support step (S700).

[0124] The voice reception step (S100) described above is a step of receiving very high frequency (VHF) communication voice. Specifically, in the voice reception step (S100), voice communication data related to navigation can be received in real time from VHF communication equipment installed on the vessel. The voice communication data can be received from a control agency, a port control channel, a nearby vessel, or a communication channel between vessels, and, if necessary, can be received by monitoring multiple VHF channels in parallel.

[0125] The voice conversion step (S200) described above is a step of converting the voice received in the voice reception step (S100) into text data. Specifically, the voice conversion step (S200) may perform at least one of noise removal, voice active segment detection, and filtering to remove or reduce at least one of maritime noise, engine noise, wave noise, radio noise, and background noise included in the received voice communication data. Subsequently, the voice communication data may be converted into text data using an Automatic Speech Recognition (ASR) model that has learned specialized terminology in the maritime field. Additionally, the voice conversion step (S200) may separate and store the continuously input Very High Frequency (VHF) communication voice in event units or sentence units.

[0126] The semantic information extraction step (S300) is a step of extracting navigational semantic information, including route information, object information, relative position information, and event type information, from the text data generated in the speech conversion step (S200). Specifically, in the semantic information extraction step (S300), a natural language processing-based semantic parsing model can be used to identify route names, vessel types, bearing expressions, danger events, and navigation instruction information included in the text data. For example, if text data such as “Dredging vessel operation on the right side of Route 3, please be careful” is generated, “Route 3” can be classified as route information, “right side” as relative position information, “dredging vessel” as object information, and “operation in progress” or “caution” as event type information. Additionally, in the semantic information extraction step (S300), at least two expressions among starboard, right side, and starboard can be normalized into the same relative position information, and port, left side, and port can also be normalized in the same way.

[0127] The above situation judgment step (S400) is a step of determining a valid navigation situation by comparing navigation status information, including information on the current position, speed, and direction of travel of the vessel, with the navigation semantic information. Specifically, in the above situation judgment step (S400), it can be determined whether the route, object, and relative position mentioned in voice communication are related to the actual navigation situation of the vessel using Electronic Chart Display Information System (ECDIS), navigation plan data, Automatic Identification System (AIS) data, or radar data. Additionally, in the above situation judgment step (S400), position and bearing information between the object mentioned in voice communication and the actual detected object can be matched using Automatic Identification System (AIS) data or radar data. For example, if information such as "tug approaching from the right" is received via VHF communication, Automatic Identification System (AIS) or radar objects located in the right bearing zone based on the vessel's current bow information can be searched, and the consistency of the situation can be verified by comparing the relative position mentioned in the event with the bearing value of the actually detected object. Additionally, unnecessary attention can be suppressed by filtering out events that are not relevant to the current navigation situation or are temporally invalid.

[0128] The above-mentioned region of interest generation step (S500) is a step of generating a region of interest (ROI) based on the ship's direction of travel, based on the result of the above-mentioned situation judgment step (S400). Specifically, in the above-mentioned region of interest generation step (S500), an angle range corresponding to the relative position information based on the ship's bow direction information can be set as the region of interest (ROI). For example, if relative position information such as "right" or "starboard" is extracted, a certain angle range in the right direction based on the bow direction information can be set as the region of interest (ROI), and if an event such as "forward caution" is extracted, a forward area based on the bow direction information can be set as the region of interest (ROI). In addition, in the above-mentioned region of interest generation step (S500), an attention concentration weight is calculated using at least one of risk level, object type, object approach speed, and event importance, and detection priorities among multiple regions of interest (ROI) can be differentially set according to the attention concentration weight. Furthermore, the above-mentioned region of interest (ROI) can be dynamically recalculated according to the passage of time, ship movement, changes in bow direction information, or changes in the movement of surrounding objects.

[0129] The sensor control step (S600) is a step of controlling one or more sensors to selectively perform detection on an area corresponding to the region of interest (ROI) generated in the region of interest generation step (S500). Specifically, in the sensor control step (S600), selective detection or highlighting processing can be performed on at least one of a camera, radar, Automatic Identification System (AIS), and Electronic Chart Display Information System (ECDIS). In addition, different artificial intelligence detection models can be selectively applied depending on the object type, and computational resources can be dynamically reallocated according to the risk level to increase the detection frequency or detection resolution for the region of interest (ROI). Furthermore, a precise analysis can be performed on the bearing zone corresponding to the region of interest (ROI) among the radar data, and the location, movement direction, and changes in approach direction of the detected object can be tracked.

[0130] The above decision support step (S700) is a step that can be performed after the above sensor control step (S600), and is a step of analyzing the collision risk based on the distance between the detected object and the vessel, relative speed, and approach direction. Specifically, in the above decision support step (S700), the collision risk can be calculated using at least one of the distance between the detected object and the vessel, relative speed, approach direction, closest approach distance (CPA), and closest approach time (TCPA). If the risk is greater than or equal to a set criterion, a visual warning or voice warning can be generated, and inputs for speed adjustment, direction change, or evasive maneuver control can be provided to the autonomous navigation control system.

[0131] For example, when an autonomous vessel is navigating along a port entry route and receives voice information via very high frequency (VHF) communication stating, “Dredging vessel operations are underway to the right of Route 3, so incoming vessels should be careful,” the voice is received in the voice reception step (S100), and the voice can be converted into text data in the voice conversion step (S200). Subsequently, in the semantic information extraction step (S300), “Route 3” can be extracted as route information, “right” as relative position information, “dredging vessel” as object information, and “operation underway” and “caution” as event type information. Subsequently, in the situation judgment step (S400), it can be determined whether the vessel is actually approaching Route 3 and whether a dredging vessel or an object presumed to be a dredging vessel exists in the right bearing area of ​​the vessel. If the above event is determined to be a valid navigation situation, in the region of interest generation step (S500), a certain angle range in the right direction based on the heading information is set as the region of interest (ROI), and a high attention weight may be assigned to the region of interest (ROI). Subsequently, in the sensor control step (S600), high-resolution camera analysis, precise radar bearing area analysis, and Automatic Identification System (AIS) object priority tracking for the region of interest (ROI) may be performed, and in the decision support step (S700), a warning or autonomous navigation control input may be generated based on the collision risk analysis results.

[0133] The terms and words used in the specification and claims described above should not be interpreted as being limited to their ordinary or dictionary meanings, and should be interpreted in a meaning and concept consistent with the technical spirit of the invention, based on the principle that the inventor can appropriately define the concept of the terms to best describe his invention.

[0134] Accordingly, the configurations illustrated in the drawings and embodiments described in this specification are merely one preferred embodiment of the present invention and do not represent all of the technical ideas of the present invention. Therefore, it should be understood that various equivalents and modifications that can replace them may exist at the time of filing this application. A person skilled in the art with ordinary knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions should be considered to fall within the scope of the following claims. Explanation of the symbols

[0136] 100: Navigation Attention Centralized Control System 110: Very High Frequency (VHF) Voice Input and Processing Module 120: Navigation Semantic Interpretation Module 130: Context Awareness and Event Matching Module 140: Attention Control Module 150: Sensor Interface and Detection Module 160: Decision Support and Autonomous Navigation Linkage Module S100: Voice reception stage S200: Voice conversion step S300: Semantic information extraction stage S400: Situation Assessment Stage S500: Region of Interest generation step S600: Sensor control stage S700: Decision Support Stage

Claims

Claim 1 A very high frequency (VHF) voice input and processing module (110) that receives very high frequency (VHF) voice communication data and converts the very high frequency (VHF) voice communication data into text data; a navigation semantic interpretation module (120) that extracts navigation semantic information from the text data; a situation awareness and event matching module (130) that determines a valid navigation situation by comparing the navigation semantic information with the current navigation status; an attention control module (140) that generates a region of interest (ROI) and detection priority based on the judgment result from the situation awareness and event matching module (130); and a sensor interface and detection module (150) that controls a sensor to perform selective detection on an area corresponding to the region of interest (ROI). A maritime very high frequency (VHF) voice communication information-based navigation attention concentration control system characterized by including a decision support and autonomous navigation linkage module (160) that analyzes the collision risk based on the distance between the object and the vessel detected by the sensor interface and detection module (150), relative speed, and approach direction, and generates a warning or provides control input to the autonomous navigation control system when the risk is greater than or equal to a set standard. Claim 2 In claim 1, the very high frequency (VHF) voice input and processing module (110) comprises a voice receiving unit, a voice preprocessing unit, an automatic speech recognition (ASR) unit, an event separation unit, and a text data generation unit, wherein the voice preprocessing unit performs at least one of noise removal, voice active interval detection, and filtering, and the event separation unit separates continuously input voice communication data into event units or sentence units, characterized in that it is a maritime very high frequency (VHF) voice communication information-based navigation attention concentration control system. Claim 3 In claim 1, the navigation semantic interpretation module (120) is configured to extract route information, object information, relative position information, and event type information from the text data using a natural language processing-based semantic parsing model, and to normalize at least two expressions among starboard, right side, and starboard or at least two expressions among port, left side, and port into the same relative position information, characterized by a maritime very high frequency (VHF) voice communication information-based navigation attention concentration control system. Claim 4 In claim 1, the situation awareness and event matching module (130) collects the current position, speed, heading information and route status of the vessel, and matches the position and bearing information between the object mentioned in the voice communication and the actual detected object using Automatic Identification System (AIS) data or radar data, and is configured to filter the event corresponding to the voice communication when the distance between the route mentioned in the voice communication and the current position of the vessel is greater than or equal to a set distance, or when the relative position information mentioned in the voice communication and the bearing information of the actual detected object do not correspond to each other, or when a set time has elapsed from the time of reception of the voice communication. This characterizes a maritime very high frequency (VHF) voice communication information-based navigation attention concentration control system. Claim 5 In claim 1, the attention concentration control module (140) sets an angle range corresponding to relative position information included in the navigation semantic information based on the ship's bow direction information as an area of ​​interest (ROI), calculates an attention concentration weight using at least one of risk level, object type, object approach speed, and event importance for each area of ​​interest (ROI), and sets a differential detection priority among multiple areas of interest (ROI) according to the attention concentration weight, thereby forming a navigation attention concentration control system based on maritime very high frequency (VHF) voice communication information. Claim 6 A maritime very high frequency (VHF) voice communication information-based navigation attention control system according to claim 1, wherein the attention control module (140) is configured to dynamically recalculate the region of interest (ROI) according to the passage of time, changes in the position of the vessel, changes in heading information, changes in speed, or changes in movement of surrounding objects, selectively activate different artificial intelligence detection models according to the object type, and prioritize the allocation of processing resources of a graphics processing unit or an artificial intelligence computing unit to the region of interest (ROI). Claim 7 A maritime very high frequency (VHF) voice communication information-based navigation attention concentration control system according to claim 1, wherein the sensor interface and detection module (150) is configured to control at least one of a camera, radar, automatic identification system (AIS), and electronic chart display information system (ECDIS), and to perform high-resolution analysis or high-cycle frame processing on a camera image area corresponding to the region of interest (ROI), or to perform precise analysis on a radar bearing area corresponding to the region of interest (ROI). Claim 8 In claim 1, the decision support and autonomous navigation linkage module (160) calculates a collision risk using at least one of the distance between the detected object and the vessel, relative speed, approach direction, closest approach distance (CPA), and closest approach time (TCPA), and generates at least one of a visual warning, a voice warning, avoidance direction information, alternative route information, or autonomous navigation control input according to the collision risk, characterized in that it is a maritime very high frequency (VHF) voice communication information-based navigation attention concentration control system. Claim 9 A method for controlling focused navigation attention based on maritime very high frequency (VHF) voice communication information, comprising: a voice receiving step (S100) for receiving very high frequency (VHF) communication voice; a voice conversion step (S200) for converting the voice received in the voice receiving step (S100) into text data; a semantic information extraction step (S300) for extracting navigation semantic information including route information, object information, relative position information, and event type information from the text data generated in the voice conversion step (S200); a situation judgment step (S400) for determining a valid navigation situation by comparing navigation status information including current vessel position, speed, and direction of travel information with the navigation semantic information; a region of interest generation step (S500) for generating a region of interest (ROI) based on the vessel's direction of travel based on the result of the situation judgment step (S400); and a sensor control step (S600) for controlling one or more sensors to selectively perform detection on an area corresponding to the region of interest (ROI) generated in the region of interest generation step (S500). Claim 10 A method for controlling navigation attention based on maritime VHF voice communication information, wherein, in claim 9, the voice conversion step (S200) performs at least one of noise removal, voice active interval detection, and filtering on the received voice communication data, converts the voice communication data into text data using an automatic speech recognition (ASR) model that has learned specialized terminology in the maritime field, and separates and stores the continuously input VHF voice communication in event units or sentence units. Claim 11 In claim 9, the semantic information extraction step (S300) is characterized by including the step of extracting the navigation semantic information using a natural language processing-based semantic parsing model and normalizing at least two expressions among starboard, right, and starboard or at least two expressions among port, left, and port into the same relative position information. Claim 12 In claim 9, the situation judgment step (S400) is characterized by including a step of matching position and bearing information between an object mentioned in a voice communication and an actual detected object using Automatic Identification System (AIS) data or radar data, and filtering an event corresponding to the voice communication when the distance between the route mentioned in the voice communication and the current position of the vessel is greater than or equal to a set distance, when the relative position information mentioned in the voice communication and the bearing information of the actual detected object do not correspond to each other, or when a set time has elapsed from the time of reception of the voice communication. Claim 13 In claim 9, the above-mentioned area of ​​interest generation step (S500) sets an angle range corresponding to the above-mentioned relative position information as an area of ​​interest (ROI) based on the vessel's bow direction information, calculates an attention weight using at least one of risk level, object type, object approach speed, and event importance, sets a differential detection priority among a plurality of areas of interest (ROI) according to the attention weight, and the area of ​​interest (ROI) is dynamically recalculated according to the passage of time, vessel movement, change in bow direction information, or change in movement of surrounding objects, characterized in a maritime very high frequency (VHF) voice communication information-based navigation attention control method. Claim 14 In claim 9, the sensor control step (S600) is characterized by performing selective detection or highlighting processing on at least one of a camera, radar, automatic identification system (AIS), and electronic chart display information system (ECDIS), selectively applying different artificial intelligence detection models according to object type, dynamically reallocating computational resources according to risk level to increase detection frequency or detection resolution for the region of interest (ROI), performing precise analysis on the bearing zone corresponding to the region of interest (ROI) among the radar data, and tracking changes in the location, direction of movement, and direction of approach of the detected object. Claim 15 A method for controlling navigation attention based on maritime very high frequency (VHF) voice communication information, characterized in that, in claim 9, after the sensor control step (S600), it further includes a decision support step (S700) that analyzes the collision risk based on the distance between the detected object and the vessel, relative speed, and approach direction, and generates a warning or provides input for speed adjustment, direction change, or evasive maneuver control to the autonomous navigation control system if the risk is greater than or equal to a set standard.

Images