Ship-assisted autonomous navigation system and method based on language model

Abstract

The invention provides a ship-assisted autonomous navigation system and method based on a language model, and the system comprises a language input module which is used for collecting an external analog sound signal and converting the external analog sound signal into a language signal; the language model module is connected with the language input module and is used for analyzing the language signal and generating a control instruction; the language output module is used for converting the language information into analog quantity sound signals and outputting the analog quantity sound signals; the navigation control module is connected with the language model module and used for adjusting the course, speed and rudder angle of the ship according to the control instruction; and the shore-based data center is connected with the language model module through the satellite communication module and provides external data support. According to the technical scheme, the ship autonomous navigation system is established based on the language model, real-time communication of navigation information between the ship and the ship / shore end and control over the ship are achieved, the autonomous navigation function is achieved in an auxiliary mode, and the core problems of cross-language communication, real-time response, rule compliance and the like are solved.

Description

Technical Field

[0001] This application belongs to the technical field of ships, and particularly relates to a ship auxiliary autonomous navigation system and method based on a language model. Background Art

[0002] Existing ship communications usually rely on manual operations, and voice interactions are carried out through devices such as medium frequency / high frequency (MF / HF), very high frequency (VHF), and satellite communications (such as VSAT, Iridium) to transmit navigation intentions, voyage information, and maneuvering instructions. However, the existing technology has the following defects:

[0003] Low efficiency: Unmanned ships need to transfer communications through shore-based duty officers. Affected by network latency, weather interference, equipment failures, etc., the response speed is slow.

[0004] Language barrier: Misunderstandings are likely to occur when operators with different mother tongues communicate, and insufficient proficiency in international common languages (such as English) may lead to omission of key information.

[0005] Lack of autonomy: Existing systems cannot directly parse voice commands and convert them into navigation control actions, and still require manual intervention in decision-making, making it difficult to meet the collision avoidance requirements in complex scenarios.

[0006] Weak data integration ability: There is a lack of real-time linkage between navigation control and external information (such as laws and regulations, port rules), resulting in insufficient decision-making basis.

[0007] Therefore, there is an urgent need for a ship navigation system that can achieve multi-language instant interaction and autonomous decision-making. Summary of the Invention

[0008] In view of the above-mentioned disadvantages of the existing technology, the purpose of this application is to provide a ship auxiliary autonomous navigation system and method based on a language model, which is used to solve problems such as instant communication of navigation information between the ship and other ships / shore terminals, so as to ensure smooth navigation of the ship.

[0009] In a first aspect, this application provides a ship auxiliary autonomous navigation system based on a language model, which at least includes the following modules:

[0010] A language input module, which is used to collect external analog sound signals and convert them into language signals;

[0011] A language model module, which is connected to the language input module and is used to parse the language signals and generate control instructions;

[0012] A language output module, which is used to convert the language information into analog sound signals and output them;

[0013] Navigation control module, connected to the language model module, for adjusting the ship's heading, speed and rudder angle according to control instructions;

[0014] Shore-based data center, connected to the language model module through a satellite communication module, providing external data support.

[0015] In an alternative embodiment, the language model module includes a multilingual database, supporting semantic parsing and generation in at least Chinese and English.

[0016] In an alternative embodiment, the language input module integrates a signal source recording function for distinguishing the input signal sources from MF / HF, VHF, VSAT or Iridium devices.

[0017] In an alternative embodiment, the language model module includes an authentication unit to verify the identities of the calling party and the called party through the ship name, call sign or IMO number, and trigger the subsequent semantic processing process after successful verification.

[0018] In an alternative embodiment, the shore-based data center provides flag state laws and regulations, port rules and meteorological data in real time for the language model module to call.

[0019] In an alternative embodiment, when the semantics related to collision avoidance actions are recognized, the navigation control module automatically generates rudder angle, speed and track adjustment instructions based on the COLREGs rules in a collision avoidance scenario, and simultaneously triggers an audible and visual alarm to the shore-based control center.

[0020] In an alternative embodiment, the language output module is built-in with a multilingual pronunciation database, supporting dynamic switching of the speech synthesis mode according to the language preference of the receiving party.

[0021] In an alternative embodiment, the language model module establishes a direct voice link between the on-board communication device and the shore-based control center through satellite communication.

[0022] In a second aspect, the present application provides a method for ship-assisted autonomous navigation, including the following steps:

[0023] Collect and process external voice signals through the language input module;

[0024] The language model module parses the semantics and generates instructions;

[0025] The navigation control module executes the instructions or calls shore-based data;

[0026] The voice output module feeds back the execution result to the external device.

[0027] In an alternative embodiment, the language model module preferentially executes the COLREGs rules in a collision avoidance scenario and records the adjustment parameters in the black box; if the instruction exceeds the scope of the rules, manual takeover is forcibly triggered.

[0028] Compared with the prior art, the technical solution provided by this application has the following beneficial effects:

[0029] The technical solution provided by this application establishes a ship autonomous navigation system based on a language model, which well realizes the instant communication of navigation information between the own ship and other ships / shore terminals and the control of the ship, assists in realizing the function of autonomous navigation, realizes the autonomous interaction and control of the ship through the language model, solves the core problems such as cross-language communication, instant response and rule compliance, significantly improves the navigation safety and operation efficiency of unmanned ships, and is applicable to various scenarios such as ocean transportation and intelligent ports. Description of the Drawings

[0030] Figure 1 It shows a system diagram of the ship assisted autonomous navigation provided by this application.

[0031] Description of the Reference Numerals:

[0032] 1. Language input module; 2. Language model module; 3. Language output module; 4. Navigation control module; 5. Shore-based data center; 6. Shore-based control center; 7. External communication module; 8. Satellite communication module. Specific Embodiments

[0033] The following specific examples illustrate the embodiments of this application. Those skilled in the art can easily understand the other advantages and principles of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application.

[0034] It should be noted that the structures, ratios, sizes, etc. shown in the drawings of this specification are only used to cooperate with the content disclosed in this specification for those skilled in this technology to understand and read, and are not used to limit the limited conditions for the implementation of this patent. Therefore, they do not have technical essential meanings. Any modification of the structure, change of the proportional relationship or adjustment of the size should still fall within the scope that can be covered by the technical content disclosed in this patent without affecting the effects that this patent can produce and the purposes that can be achieved. At the same time, the terms such as "upper", "lower", "left", "right", "middle" cited in this specification are only for the convenience of description and are not used to limit the scope for the implementation of this patent. The change or adjustment of their relative relationships should also be regarded as the scope for the implementation of this patent without substantial change in the technical content.

[0035] Example 1:

[0036] This embodiment provides a ship auxiliary autonomous navigation system based on a language model. When the ship auxiliary autonomous navigation system is activated, the system will continuously operate unless it is cut off manually due to power supply or system failure. The ship auxiliary autonomous navigation system includes the following modules:

[0037] A language input module 1, which is used to collect external analog sound signals and convert them into language signals;

[0038] A language model module 2, connected to the language input module 1, which is used to parse the language signals and generate control instructions;

[0039] A language output module 3, which is used to convert the language information into analog sound signals and output them;

[0040] A navigation control module 4, connected to the language model module 2, which is used to adjust the ship's course, speed and rudder angle according to the control instructions;

[0041] A shore-based data center 5, connected to the language model module 2 through a satellite communication module 8, which provides external data support.

[0042] By adopting the above technical solutions, the ship has the ability to communicate with the outside world autonomously and instantaneously in language and adjust the navigation state, achieving the purpose of avoiding collisions and improving navigation safety. At the same time, it can also enable the staff of the shore-based control center 6 to communicate with the outside world and control the ship.

[0043] Specifically, the analog sound signals called from other ships or shore-based calling stations to this ship are collected and converted into digital signals in real time by the language input module 1. The digital language signals are input to the language model module 2 for processing. The external voice signals from other ships or the shore are received through external communication modules 7 such as MF / HF and VHF, and are collected in real time by the microphone array of the language input module 1. The analog voice signals are converted into digital signals by an analog-to-digital converter (ADC). The sampling frequency is selected as 16 kHz and the quantization accuracy is 16 bits. Further, the language input module 1 includes an amplification function to perform dynamic gain amplification on weak signals, with a gain range of 0 to 30 dB, ensuring that the signal strength meets the recognition threshold to improve the recognition ability of weak sound signals. Further, the language input module 1 also has a filtering function. For example, the Kalman filtering algorithm is used to eliminate background noise such as sea waves and engine noise, and the signal-to-noise ratio is increased to more than 20 dB to improve the recognition rate. Further, the language input module 1 integrates a signal source recording function, inputs operating signals from external communication modules 7, MF / HF, VHF, VSAT, Iridium or other communication devices, records the sources of on-board communication devices, and distinguishes signal sources.

[0044] The language model module 2 identifies and processes the digital language signals from the language input module 1. It contains a database of multiple languages and supports semantic parsing and generation in at least Chinese and English.

[0045] The language model module 2 includes an authentication unit with the function of caller identity recognition. It verifies the identities of the caller and the callee in the language signal, including the ship name, call sign, and IMO number. After successful verification, it triggers the subsequent semantic processing process.

[0046] In specific implementation, the language model module 2 compares the collected language signals with the language database, converts the language signals into text, extracts feature words such as nouns, verbs, adverbs, and question marks in the text, and combines language processing strategies to identify and process the semantics.

[0047] If the recognized language request is related to the basic information of the ship itself and navigation and route information, such as ship name, call sign, ship type, IMO number, flag state, classification society, and navigation course and speed, etc., then after the language model module 2 reads the real-time information of the navigation control module 4, it outputs the text information to the language output module 3 according to the natural language organizational structure of the real-time information.

[0048] If the recognized language request contains information other than the ship itself, such as the name of the operator and the laws and regulations of the flag state of the navigation destination, etc., then the language model module 2 communicates through the satellite communication module 8 to read the data of the shore-based data center 5. The shore-based data center 5 can be a fleet monitoring center or a shore-based data center on the Internet. After reading, it outputs the shore-based information and the ship's own information to the language output module 3 according to the natural language organizational structure.

[0049] For the recognized semantics that contains the collision avoidance action information of both parties in a meeting situation, the language model module 2 will read the adjustment amounts of the navigation control module 4: rudder angle, propulsion speed, speed, track information, estimated time, and the adjustment basis, which is generally the articles of the collision avoidance rules, and output the text information to the language output module 3.

[0050] During the implementation process, if the identity of the other ship or the language request is beyond the code of conduct of the collision avoidance rules and requires the ship to take actions to adjust the navigation route, then the language model module 2 inputs the confirmed navigation parameters such as rudder angle, course, and speed to the navigation control module 4. The navigation control module 4 automatically generates rudder angle, speed, and track adjustment instructions based on the COLREGs rules to locally adjust the ship's route. At the same time, it emits a sound alarm and a prompt sound at the ship and the shore-based control center 6 to remind the duty officers to pay attention to the collision avoidance process.

[0051] The language output module 3 has a built-in multilingual pronunciation database and supports dynamic switching of speech synthesis modes according to the recipient's language preference. The language output module 3 compares the text information language signal with the code in the pronunciation database, transmits the matching pronunciation code to the built-in speaker, outputs the analog sound signal, and completes the call through the shipboard external communication module 7.

[0052] When the outside world requests manual communication, the language model module 2 will dial the preset manual telephone number through the default shipboard communication equipment such as VSAT. After the connection, the language model module 2 directly connects the digital signal of the language input module 1 to the shore-based control center 6 through the satellite communication of the satellite communication module 8, and establishes a direct communication between the external communication module 7 and the shore-based control center 6.

[0053] Embodiment 2:

[0054] This embodiment provides a method for assisting autonomous navigation of a ship, comprising the following steps:

[0055] Collect and process external voice signals through the language input module 1;

[0056] The language model module 2 analyzes semantics and generates instructions;

[0057] The navigation control module 4 executes instructions or calls shore-based data;

[0058] The voice output module feeds back the execution result to the external device.

[0059] In an optional implementation, the language model module 2 preferentially executes the COLREGs rules in a collision avoidance scenario and records the adjustment parameters to the black box; if the instruction exceeds the scope of the rules, manual takeover is forcibly triggered.

[0060] The method is described in detail below through a specific application scenario.

[0061] A large cargo ship (ship A) equipped with the autonomous navigation system of the present invention is crossing the Strait of Malacca. It encounters sudden strong winds and waves. At the same time, two nearby ships (ship B and ship C) issue a collision avoidance request:

[0062] Vessel B (call in English): "Vessel A, this is Vessel B. Strong currents detected ahead. Advise altering course to 220 degrees immediately."

[0063] Vessel C (Chinese call): "Vessel A, this is Vessel C. Unmarked obstacle detected to the right front. Recommend turning left 15 degrees and slowing down to 8 knots."

[0064] In addition, the shore-based data center 5 monitors the regional weather update, indicating that the wind force will increase to level 8 within the next 30 minutes.

[0065] The ship system executes the following assisted autonomous navigation process:

[0066] 1. Voice signal acquisition and preprocessing:

[0067] Language input module 1: Receives the English signal of ship B through the VHF device and the Chinese signal of ship C through the VSAT; the signal amplification function enhances the weak voice (the signal of ship B attenuates due to wind and waves); the Kalman filter eliminates engine noise and sea wave interference, and the signal-to-noise ratio is increased to 25 dB; records the signal sources as VHF (ship B) and VSAT (ship C).

[0068] 2. Semantic analysis and priority determination

[0069] Language model module 2:

[0070] Speech recognition: Converts English and Chinese voices into text respectively;

[0071] Identity authentication: Verifies the identities of ship B (call sign MMSI 123456) and ship C (call sign MMSI 654321), and enters the processing flow after confirmation of legality;

[0072] Semantic analysis:

[0073] Request from ship B: "Immediately turn 220 degrees" (keywords: currents, alter course, immediately);

[0074] Request from ship C: "Turn left 15 degrees and decelerate" (keywords: obstacle, turn left, decelerate);

[0075] Environmental data integration: Calls the shore-based data center 5 to obtain real-time weather data (wind force 8) and electronic chart obstacle information;

[0076] Priority ranking:

[0077] The request from ship B involves the course risk caused by strong currents, and the priority is "urgent";

[0078] The request from ship C is for avoiding static obstacles, and the priority is "high";

[0079] The system gives priority to responding to the instruction of ship B according to the COLREGs rules, but needs to coordinate the conflict between the two.

[0080] 3. Dynamic route adjustment and collision avoidance control

[0081] Navigation control module 4:

[0082] Generate comprehensive instructions:

[0083] First, execute the "Turn 220 degrees" for Ship B while reducing the speed from 12 knots to 10 knots (taking into account the deceleration request of Ship C);

[0084] When calculating the collision avoidance path, reserve a safety distance to avoid the obstacles reported by Ship C;

[0085] Output the rudder angle adjustment instruction (turn right 30 degrees) and the propeller power parameter to the actuator;

[0086] Monitor the track deviation in real time. If it exceeds the preset threshold, trigger fine-tuning.

[0087] 4. Multilingual feedback and alarm notification

[0088] Language output module 3:

[0089] Send an English voice message to Ship B: "Vessel B, course altered to 220degrees, speed reduced to 10knots."

[0090] Send a Chinese voice message to Ship C: "Ship C, has turned left 15 degrees and reduced speed to 8 knots. Obstacle avoidance completed."

[0091] Exception handling:

[0092] Detect that the wind force has increased to level 8, and the navigation control module 4 automatically enables the "wind resistance mode" (increase the rudder effect and lower the center of gravity);

[0093] Simultaneously send an alarm to the shore-based control center 6: "Wind force exceeds limit, emergency wind resistance mode has been activated, request for manual monitoring."

[0094] 5. Manual intervention and data recording

[0095] Shore-based control center 6: The duty officer can view the navigation parameters and environmental data of Ship A in real time through the satellite communication module 8; after confirming that the system decision complies with safety regulations, mark it as "no intervention required"; if the position update of the drift object is detected, manually push the new coordinates to the navigation control module 4 of Ship A.

[0096] In summary, the technical solutions of the AI-based through-hole design method and system provided by this application have high industrial utilization value because they effectively overcome various shortcomings in the prior art.

[0097] The above embodiments are only illustrative of the principles and effects of the present application and are not intended to limit the present application. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed in the present application should still be covered by the claims of the present application.

Claims

1. A ship-assisted autonomous navigation system based on a language model, characterized in that: Includes at least the following modules: A language input module (1) is used to collect external analog sound signals and convert them into language signals; A language model module (2), connected to the language input module (1), for parsing the language signal and generating a control instruction; A language output module (3), used for converting the language information into an analog sound signal and outputting the analog sound signal; A navigation control module (4), connected to the language model module (2), for adjusting the ship's heading, speed and rudder angle according to control instructions; The shore-based data center (5) is connected to the language model module (2) via a satellite communication module (8) to provide external data support.

2. The ship-assisted autonomous navigation system based on language model according to claim 1, characterized in that: The language model module (2) includes a multilingual database, and supports semantic analysis and generation of at least Chinese and English.

3. The ship-assisted autonomous navigation system based on language model according to claim 1, characterized in that: The language input module (1) integrates a signal source recording function, which is used to distinguish the source of the input signal from MF / HF, VHF, VSAT or satellite equipment.

4. The ship-assisted autonomous navigation system based on language model according to claim 1, characterized in that: The language model module (2) comprises an identity verification unit for verifying the identities of the caller and the called party by means of the ship name, call sign or IMO number, and triggering a subsequent semantic processing flow after the verification is passed.

5. The ship-assisted autonomous navigation system based on language model according to claim 1, characterized in that: The shore-based data center (5) provides flag state laws and regulations, port rules and meteorological data in real time for the language model module (2) to call.

6. The ship-assisted autonomous navigation system based on language model according to claim 1, characterized in that: When semantics involving collision avoidance actions are identified, the navigation control module (4) automatically generates rudder angle, speed and track adjustment instructions based on the COLREGs rules in the collision avoidance scenario, and simultaneously triggers an audible and visual alarm to the shore-based control center (6).

7. The language model-based ship-assisted autonomous navigation system according to claim 1, characterized in that: The language output module (3) has a built-in multilingual pronunciation database and supports dynamic switching of speech synthesis modes according to the language preference of the receiver.

8. The language model-based ship-assisted autonomous navigation system according to claim 1, characterized in that: The language model module (2) establishes a direct voice link between the external communication module (7) and the shore-based control center (6) via satellite communication (8).

9. A ship-assisted autonomous navigation method based on the system according to any one of claims 1 to 8, characterized in that: The following steps are involved: Collect and process external voice signals through the language input module; The language model module parses semantics and generates instructions; The navigation control module executes instructions or calls shore-based data; The voice output module feeds back the execution result to the external device.

10. The ship-assisted autonomous navigation method according to claim 9, characterized in that: The language model module (2) prioritizes the execution of COLREGs rules in collision avoidance scenarios and records the adjustment parameters in the black box; if the instruction exceeds the scope of the rules, manual takeover is forced.

Images