Navigation mark collision identification early warning system based on large language model

Abstract

The invention discloses a navigation mark collision recognition and early warning system based on a large language model, and relates to the technical field of navigation mark collision early warning. Comprising a sensor data acquisition system, a data analysis early warning system based on a large language model, a video acquisition system, a wireless communication module and a power supply module, the sensor data acquisition system collects and sorts vibration and audio signals of the surrounding environment of the navigation mark, the sorted signal data is transmitted to the data analysis early warning system based on a large language model through the wireless communication module to be processed and analyzed, whether a ship collides with the navigation mark or not is judged, and a judgment conclusion is transmitted to the video acquisition system; the video acquisition system shoots and records an image of the hit-and-run ship in real time, and submits the image of the hit-and-run ship to the cloud server and the terminal for checking and responsibility investigation; the power supply module supplies power to each system. The system is high in monitoring efficiency and accuracy, can monitor the dynamic state of the navigation mark in real time, timely maintains the navigation mark, and reduces the occurrence of secondary accidents.

Description

Technical Field

[0001] The invention belongs to the technical field of navigation mark collision warning, and more specifically, relates to a navigation mark collision recognition and warning system based on a large language model. Background Art

[0002] Inland waterway shipping is an important part of the modern integrated transportation system and occupies a pivotal position in the construction of the national transportation network. As an important aid to navigation to ensure the safe navigation of ships, navigation marks play an indispensable and key role in the rapid development of the inland waterway shipping industry. At present, there are more than 7,000 navigation marks deployed on the main channel of the Yangtze River alone to indicate the direction for ships to navigate. These navigation marks are key infrastructure to ensure waterway safety and promote the stability of economic and social development along the river. However, with the vigorous development of Yangtze River shipping, the number of ships continues to increase, the waterway is becoming increasingly crowded, and the challenges faced by the navigation mark system are increasing day by day. Frequent incidents of navigation mark malfunctions caused by ship collisions not only cause losses to waterway assets, but also bring safety hazards to ship navigation.

[0003] At present, with the development of communication technology, navigation mark management units use navigation mark telemetry and remote control systems and install sensors, GPS, distance detectors and other equipment on navigation marks to determine navigation mark drift and collision events. However, due to their own accuracy problems, it is difficult to detect navigation mark collision events in time, resulting in serious consequences. For example, the Chinese patent "High-precision positioning navigation mark" (CN201911416858.5) detects the distance between the hull and the navigation mark through an ultrasonic distance detector and feeds back to the channel terminal. Although it can improve positioning accuracy, the navigation mark is small in size and cannot carry too many devices. It is also easily affected by bad weather, and the device is easily damaged and difficult to maintain. There are also references to using sensors installed on navigation marks to warn ships, such as the Chinese patent "Navigation marks, navigation mark anti-collision warning device and method" (CN201911390741.4) which mentions a method based on a distance sensor to detect the distance between a navigation mark and its surrounding ships, and "A navigation mark anti-collision warning device" (CN202323061933.5) which mentions a method based on an infrared distance sensor to detect the distance between the ship and the navigation mark column. Although collision warnings can be achieved, due to the complex underwater conditions, the detection accuracy is greatly affected by vortices, wind and waves, etc., and the accuracy is relatively accurate only when the navigation mark is vertical, and its installation and maintenance are inconvenient. Therefore, it is urgent to develop a method to improve the anti-collision of navigation marks, to help the navigation mark telemetry and remote control system to achieve more accurate monitoring of navigation mark collision events and detection methods and warning devices for holding the ship responsible for the accident accountable. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a navigation mark collision recognition and warning system based on a large language model, which has high monitoring efficiency and monitoring accuracy, can monitor the navigation mark dynamics in real time, maintain the navigation mark in time, and reduce the occurrence of secondary accidents caused by the loss of navigation marks.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a navigation mark collision identification and warning system based on a large language model, including a sensor data acquisition system, a data analysis and warning system based on a large language model, a video acquisition system, a wireless communication module, and a power supply module; the sensor data acquisition system and the video acquisition system are both installed on the navigation mark; the sensor data acquisition system monitors and collects vibration and audio signals of the surrounding environment of the navigation mark in real time, and simply organizes the collected signal data, and transmits the organized signal data to the data analysis and warning system based on the large language model through the wireless communication module; the data analysis and warning system based on the large language model processes and analyzes the organized signal data to determine whether the ship collides with the navigation mark, and transmits the judgment conclusion to the video acquisition system through the wireless communication module; if the ship collides with the navigation mark, the video acquisition system captures and records the image of the ship in real time, and submits the image of the ship in the accident to the cloud server and terminal through the wireless communication module for viewing and accountability; the power supply module provides power to the wireless communication module, the video acquisition system, and the sensor data acquisition system through a transmission cable.

[0006] Preferably, the sensor data acquisition system comprises an acceleration sensor, an audio sensor, and an RK3588 chip unit, and the acceleration sensor and the audio sensor are electrically connected to the RK3588 chip unit via a transmission cable.

[0007] Preferably, the acceleration sensor monitors the water flow fluctuations around the navigation mark; the audio sensor monitors the sound of the ship's movement and the sound emitted by the ship at different positions from the navigation mark; the RK3588 chip unit organizes the data obtained by the acceleration sensor and the audio sensor, and transmits the organized data to the data analysis and early warning system based on the large language model through the wireless communication module.

[0008] Preferably, the data analysis and early warning system based on the large language model includes a cloud server, and the RK3588 chip unit serves as a field data processing unit, which transmits the sorted data to the cloud server through a wireless communication module. The large language model carried on the cloud server processes and analyzes the sorted data to determine whether a collision will occur between the ship and the navigation mark.

[0009] Preferably, the video collection system includes a camera, which is installed on the navigation mark. When the data analysis and early warning system based on the large language model draws a dangerous conclusion that there is a collision between the ship and the navigation mark, the camera is activated to run, and the camera records the process of the ship colliding with the navigation mark in a 360-degree circular manner, and transmits the identified ship image and collision video to the cloud server and terminal through the wireless communication module for viewing and accountability.

[0010] Preferably, the cloud server transmits the conclusion that the ship and the navigation mark have collided to the mobile terminal and the computer website through the communication module to issue an early warning, so that the navigation mark can be maintained in time.

[0011] The beneficial effects of the above technical solution are: the advantages of the present invention are that the detection device can be placed on the navigation mark for dynamic real-time monitoring, which improves the efficiency of monitoring; the acceleration sensor and audio sensor are used to monitor and collect the vibration and audio signals of the surrounding environment of the navigation mark, and the large language model is used to calculate the possibility of the ship colliding with the navigation mark, which improves the accuracy of monitoring; the communication module is used to upload the collision information to the terminal in real time, which is conducive to timely maintenance of the navigation mark and reducing the secondary accidents caused by the loss of the navigation mark. Using a camera to capture images close to the ship is conducive to the law enforcement department to retain evidence, which is of great help in investigating the illegal behavior of the ship that hits the navigation mark ship. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of a navigation aid collision identification and warning system; DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] like Figure 1 As shown, the navigation mark collision identification and warning system consists of a sensor data acquisition system, a data analysis and warning system based on a large language model, a video acquisition system, a power module, and a wireless communication module. The sensor data acquisition system is the basis of the present invention. It uses an acceleration sensor and an audio sensor to collect data, and then transmits the collected data to the cloud network after pre-processing by RK3588. The large language model is built on the cloud server, and the collected data can be deeply analyzed to obtain the result of whether the ship has collided with the navigation mark. Finally, the result of the collision is transmitted to the on-site video acquisition system, thereby activating the camera to shoot the ship in the accident.

[0015] 1. The sensor data acquisition system includes an acceleration sensor, an audio sensor, and an RK3588 chip unit. The sensor data acquisition system is installed on the navigation buoy to detect the surrounding environmental vibration and audio signals in real time. The acceleration sensor and the audio sensor are connected to the RK3588 chip unit through a transmission cable. The RK3588 chip unit can simply organize the collected data. The RK3588 chip unit is then connected to a wireless communication module through a transmission cable to upload the collected data. The power module can provide power for the sensor, RK3588, and wireless communication module through a transmission cable.

[0016] Among them, the acceleration sensor is a sensor that can be used to measure the spatial acceleration, that is, to measure how fast the speed of an object changes in space, and decompose the spatial acceleration along the three axes of X, Y, and Z. In the present invention, the acceleration sensor is used to detect the water flow fluctuation to form a three-dimensional waveform diagram, thereby proposing a ship-buoy collision detection method based on water flow fluctuation.

[0017] The audio sensor can be used to receive sound waves and convert the sound signal into an electrical signal, so as to estimate the distance of the sounding object. In the present invention, the audio sensor is used to detect the sound of the ship's travel and the sound emitted by the ship at different positions from the buoy, so as to further infer the ship-buoy collision accident.

[0018] The RK3588 chip unit is an application processor chip integrating multiple high-performance processing units, designed by Rockchip and adopting the ARM architecture. In the present invention, the domestic chip RK3588 is used as the core chip for data bearing and exchange.

[0019] 2. The data analysis and early warning system based on the large language model includes a cloud server. The RK3588 organizes the data collected by the sensor, and the communication module can transmit the organized data to the cloud network and receive the collision determination result. The cloud server can carry and build a large language model to analyze the collected data and judge whether a collision will occur between the ship and the buoy.

[0020] In the present invention, the detected ship-buoy collision result is uploaded to the RK3588 chip unit through the wireless communication module, and then the collision result that has occurred is transmitted to the mobile phone APP and the computer website for early warning, so that the ship-buoy can be maintained in time.

[0021] The present invention takes the large language model as the technical core. Compared with traditional machine learning, the large language model can automatically learn features from data, which is beneficial to dealing with complex problems; it has the characteristics of deep learning and strong generalization ability, can predict the unoccurred event scenarios, is easy to operate, and has higher accuracy.

[0022] Large language model:

[0023] (1) Scenario Description and Data Collection:

[0024] Scenario Description: Describe in detail various situations that may occur during ship navigation, including normal navigation, approaching a navigational buoy, water flow fluctuations, sound changes, etc. These descriptions will serve as the basis for natural language prompts.

[0025] Data Collection: Collect relevant data, such as the position of the ship, speed, position of the navigational buoy, changes in the surrounding water flow, sound information, etc. These data will be used to train the model.

[0026] (2) Model Training and Optimization:

[0027] ① Data Feature Analysis and Model Requirements:

[0028] Audio Data: Audio signals are usually time-series, containing information about the frequency and intensity of events such as collisions and approaching vessels. When a collision occurs, there will be a sudden change in the audio waveform, and the noise pattern may change significantly.

[0029] Accelerometer Data: The data from accelerometers is usually presented in the form of time-series data and may contain vibrations generated by ship contact, collisions, etc. It is necessary to consider its time-series and possible periodic characteristics.

[0030] ② Data Preprocessing: Select a suitable model to process the input natural language prompts.

[0031] Training Data Preparation: Use the generated natural language prompts as input data and the status of whether there is a collision as a label for training. At the same time, combine the surrounding water flow fluctuations and sound information as auxiliary features.

[0032] Audio Sensor Data: Select the Wav2Vec2 model to convert the audio signal into feature analysis and extract the time-series features of the audio.

[0033] Accelerometer Data: Use an analog-to-digital converter to convert the analog signal output by the accelerometer into a digital signal and extract the time-series features of the acceleration.

[0034] ③ Data Fusion

[0035] Audio Sensor Data: First, denoise the audio sensor data and segment the audio (segment it with a short time window, such as 1 or 0.5 seconds). Then perform feature extraction to identify possible collision signals, such as the feature of a sudden high-amplitude sound.

[0036] Accelerometer: First, denoise the accelerometer data (such as using a low-pass filter) and standardize the data. Then perform feature extraction to identify collision features, such as the change in acceleration within a certain time.

[0037] Data fusion: Since both the audio sensor data and the acceleration sensor data have been converted into digital signal forms, time calibration can be used to fuse the two data to ensure that the time of the acceleration data matches the time of the audio data.

[0038] ④ Model training fine-tuning and dataset setting

[0039] Dataset setting: In this project, the sensor data after time-corrected fusion and the status of the ship approaching the navigation mark are classified, and the classified dataset is classified according to 80% training set, 10% validation set, and 10% test set for each category.

[0040] Model fine-tuning: The present invention will use a predefined model and train it according to the data. First, the cross-validation method is used to evaluate the performance of the model, and the parameters of the model are adjusted according to the evaluation results. Then, all the training data is used to train the model, and adjustments are made according to the training results. Hyperparameters such as regularization, Dropout, learning rate, batch size, and optimizer can be used to find the most suitable configuration. Finally, the performance of the model is evaluated on the validation group, and the hyperparameters are adjusted to optimize metrics such as accuracy and recognition rate.

[0041] The present invention conducts training for corresponding scenarios based on the LLM model.

[0042] Training examples:

[0043] Perception: Relative speed of the navigation mark (longitudinal acceleration, lateral acceleration), longitudinal risk, lateral risk, audio signal (noise)

[0044] (1) Normal state - calm water surface

[0045] 1. Acceleration data:

[0046] - X-axis: 0.12 m / s 2

[0047] - Y-axis: 0.15 m / s 2

[0048] - Z-axis: 0.18 m / s 2

[0049] 2. Audio data:

[0050] - Noise level of sensor 1: 35 dB

[0051] - Noise level of sensor 2: 38 dB

[0052] Analysis result:

[0053] Status: Normal

[0054] Basis for judgment: All indicators are within the normal fluctuation range

[0055] Suggested operation: Continue with routine monitoring

[0056] (2) Normal state - water flow fluctuation

[0057] Real-time sensor data:

[0058] 1. Acceleration data:

[0059] - X-axis: 0.31 m / s 2

[0060] - Y-axis: 0.28 m / s 2

[0061] - Z-axis: 0.42 m / s 2

[0062] 2. Audio data:

[0063] - Noise level of sensor 1: 42 dB

[0064] - Noise level of sensor 2: 44 dB

[0065] Analysis result:

[0066] Status: Normal

[0067] Basis for judgment: Although there is water flow fluctuation, the indicators are still within the normal range

[0068] Suggested operation: Continue with routine monitoring

[0069] (3) Abnormal state - minor collision

[0070] Real-time sensor data:

[0071] 1. Acceleration data:

[0072] - X-axis: 1.75 m / s 2

[0073] - Y-axis: 1.82 m / s 2

[0074] - Z-axis: 0.95 m / s 2

[0075] 2. Audio data:

[0076] - Noise level of sensor 1: 58 dB

[0077] - Noise level of sensor 2: 62 dB

[0078] Analysis result:

[0079] Status: Minor collision Judgment basis: Both acceleration and noise level data exceed the normal range, meeting the characteristics of a minor collision Suggested operation:

[0080] 1. Activate the camera to record for 30 seconds

[0081] 2. Save the sensor data for 2 minutes before and after the collision

[0082] 3. Upload the data packet to the cloud

[0083] (4) Abnormal status - Severe collision

[0084] 1. Acceleration data:

[0085] - X-axis: 3.25 m / s 2

[0086] - Y-axis: 2.98 m / s 2

[0087] - Z-axis: 2.45 m / s 2

[0088] 2. Audio data:

[0089] - Noise level of sensor 1: 78 dB

[0090] - Noise level of sensor 2: 82 dB

[0091] Analysis result:

[0092] Status: Severe collision Judgment basis: Acceleration and noise level data significantly exceed the normal range, meeting the characteristics of a severe collision Suggested operation:

[0093] 1. Immediately activate the camera to record for 60 seconds

[0094] 2. Save the sensor data for 5 minutes before and after the collision

[0095] 3. Urgently upload the data packet to the cloud

[0096] 4. It is recommended to immediately go to the site for inspection

[0097] (4) Abnormal status - Scraping

[0098] Real-time sensor data:

[0099] 1. Acceleration data:

[0100] - X-axis: 0.85 m / s 2

[0101] - Y-axis: 0.92 m / s 2

[0102] - Z-axis: 0.68 m / s 2

[0103] 2. Audio data:

[0104] - Noise level of Sensor 1: 52 dB

[0105] - Noise level of Sensor 2: 54 dB

[0106] Analysis result:

[0107] Status: Slight rubbing. Judgment basis: The acceleration and noise level slightly exceed the normal range, conforming to the characteristics of rubbing

[0108] Suggested operation:

[0109] 1. Activate the camera to record for 20 seconds

[0110] 2. Save the sensor data for 1 minute before and after rubbing

[0111] 3. Upload the data packet to the cloud

[0112] ⑤ Deployment and real-time detection

[0113] After completing the model training, it can be deployed to the cloud server for real-time monitoring to detect the results of model training

[0114] (3) Training result (Natural language prompt generation):

[0115] Rule definition: According to the scenario description and data type, define a set of rules or templates for generating natural language prompts. The generated prompts in this project are:

[0116] If the ship is approaching the navigation buoy and the water flow fluctuates greatly, generate the prompt: "The ship is approaching the navigation buoy and the water flow fluctuation increases."

[0117] If the sound information shows an anomaly, generate the prompt: "An abnormal sound is detected. Please pay attention to safety."

[0118] Prompt examples:

[0119] "The ship is approaching the navigation buoy. Please pay close attention."

[0120] "The current water flow fluctuates greatly. Please pay attention to whether there are ships approaching around."

[0121] "Unusual sounds and vibrations are detected. It is recommended to check the surrounding environment and take pictures."

[0122] 3. The video acquisition system includes a camera. The camera is installed on the navigation mark. The camera can be used to take photos or videos for recording. After receiving the result of the collision event, the present invention uses a camera to record the process of the ship colliding with the navigation mark ship in a 360-degree circular manner as evidence of the illegal behavior of the ship causing the accident.

[0123] After the large language model is processed, a conclusion can be drawn as to whether there is a collision or not. The collision result is transmitted to the on-site RK3588 chip unit through the wireless communication module, triggering the on-site camera to start shooting. At the same time, the identified ship image and collision video can be transmitted to the cloud server so that law enforcement personnel can retain evidence to investigate the responsibility of the ship causing the accident.

[0124] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A navigation mark collision recognition and warning system based on a large language model, characterized in that: It includes a sensor data acquisition system, a data analysis and early warning system based on a large language model, a video acquisition system, a wireless communication module, and a power module; the sensor data acquisition system and the video acquisition system are both installed on the navigation mark; the sensor data acquisition system monitors and collects vibration and audio signals from the surrounding environment of the navigation mark in real time, and simply organizes the collected signal data, and transmits the organized signal data to the data analysis and early warning system based on a large language model through the wireless communication module; The data analysis and early warning system based on the large language model processes and analyzes the sorted signal data to determine whether the ship and the navigation mark have collided, and transmits the judgment conclusion to the video acquisition system through the wireless communication module; if the ship collides with the navigation mark, the video acquisition system will record the image of the ship in real time, and submit the image of the ship in the accident to the cloud server and terminal through the wireless communication module for viewing and accountability; The power module provides power to the wireless communication module, video acquisition system, and sensor data acquisition system through a transmission cable.

2. A navigation mark collision recognition and warning system based on a large language model according to claim 1, characterized in that: The sensor data acquisition system comprises an acceleration sensor, an audio sensor, and an RK3588 chip unit. The acceleration sensor and the audio sensor are electrically connected to the RK3588 chip unit via a transmission cable.

3. A navigation mark collision recognition and warning system based on a large language model according to claim 2, characterized in that: The acceleration sensor monitors the water flow fluctuations around the navigation mark; the audio sensor monitors the sound of the ship's movement and the sound emitted by the ship at different positions from the navigation mark; the RK3588 chip unit organizes the data obtained by the acceleration sensor and the audio sensor, and transmits the organized data to the data analysis and early warning system based on the large language model through the wireless communication module.

4. A navigation mark collision recognition and warning system based on a large language model according to claim 3, characterized in that: The data analysis and early warning system based on the large language model includes a cloud server. The RK3588 chip unit serves as a field data processing unit. The sorted data is transmitted to the cloud server through a wireless communication module. The large language model installed on the cloud server processes and analyzes the sorted data to determine whether a ship will collide with a navigation mark.

5. The navigation mark collision recognition and warning system based on a large language model according to claim 4 is characterized in that: The video collection system includes a camera, which is installed on the navigation mark. When the data analysis and early warning system based on the large language model draws a dangerous conclusion that the ship and the navigation mark collide, the camera is activated to run. The camera records the process of the ship colliding with the navigation mark in a 360-degree circular manner, and transmits the identified ship image and collision video to the cloud server and terminal through the wireless communication module for viewing and accountability.

6. The navigation mark collision recognition and warning system based on a large language model according to claim 4 is characterized in that: The cloud server transmits the conclusion that the ship has collided with the navigation mark to the mobile terminal and computer website through the communication module to issue an early warning, so that the navigation mark can be maintained in time.

Images