Inland ship behavior semantic detection method and device based on sliding window

Abstract

The application discloses a kind of based on sliding window's inland ship cross behavior semantic detection method and device, the method includes: obtaining the spatial coordinate data of waterway environment;AIS trajectory data of inland ship is obtained and preprocessed;Based on the spatial coordinate data of waterway environment and the AIS trajectory data after preprocessing, utilize sequence sliding window to traverse ship trajectory point, according to the 9 intersection model DE-91M of dimension expansion, calculate the spatial topological relation of ship trajectory point and trajectory section and waterway, obtain ship topological behavior semantic relation set;Topological behavior relationship order rule of inland ship cross behavior semantic is constructed, and the cross behavior of inland ship is detected in combination with ship topological behavior semantic relation set behavior.The application not only establishes, identifies and expands the expression ship cross behavior semantic from semantic level, but also can extract, calculate detection and visualization on ship data in real time on data level semantic behavior, and has obvious help to real-time supervision and auxiliary decision of water traffic.

Description

Technical Field

[0001] This invention belongs to the field of ship behavior recognition technology, specifically relating to a semantic detection method and device for the lateral movement behavior of inland waterway vessels based on a sliding window. Background Technology

[0002] In recent years, the fields of big data and artificial intelligence have gradually emerged, with ChatGPT appearing as a phenomenal representative in the AI ​​field. As a natural language processing tool driven by artificial intelligence technology, it can learn and understand human language to engage in conversation and interact based on the context of the chat, truly communicating like a human. It can even complete tasks such as writing emails, video scripts, copywriting, translation, coding, and writing papers. Meanwhile, artificial intelligence has already quietly risen in the field of transportation research. In the automotive transportation sector, for example, the driver assistance systems developed by companies like BYD, Tesla, and XPeng, including functions such as automatic parking, intelligent cruise control, braking, and lane changing, are gradually moving towards a trend of driving like a true human. In the field of water transportation, the analysis of ship behavior has always been a key research focus in order to achieve assisted driving of ships. In order to realize artificial intelligence like ChatGPT, knowledge-oriented semantic modeling of ship behavior has long been carried out. Following the semantic modeling of ship behavior, the identification, storage and even upgrading of ship semantic behavior to higher-order ship behavior semantics has become a bottleneck in the current development of the water transportation field. So how to detect the higher-order behavior semantics of ships when receiving AIS data in real time is a problem in the current research field of ship behavior semantics.

[0003] The semantics of vessel lateral movement is one of the most typical behavioral semantics among the higher-order traffic behavior semantics of vessels in inland waterways. Furthermore, abnormal lateral movement can reflect unexpected or illegal situations, such as vessel malfunctions or violations of navigation regulations. During vessel navigation, in complex geographical conditions and with high traffic density, operators need to analyze and process large amounts of data and events at all times to make relatively correct navigation decisions. However, complex traffic situations are most likely to occur in key inland waterways. Therefore, the identification of the semantics of inland waterway vessel lateral movement can accelerate the information processing process for vessel operators, playing a supporting role in decision-making. For maritime regulators, it is more helpful for monitoring inland waterway traffic. Therefore, it is necessary to solve this problem from a top-down, implementation-oriented perspective.

[0004] Current methods for identification based on behavioral semantic modeling have the following shortcomings: First, semantic modeling methods primarily model ship behavior in ideal scenarios. However, ship behavior identification at the data level is subject to data perturbations, which need to be addressed without affecting subsequent reasoning. Second, the identification of ship topological behavioral semantics mainly focuses on the ship itself and its relationship with environmental objects. However, the definition of ship traffic behavior semantics is not comprehensive enough, requiring refinement and improvement of each specific traffic behavior semantic to adapt to real-world navigation scenarios. Third, the identification and detection of ship traffic behavior semantics lacks real-time capability. Only by achieving real-time analysis of ship behavior semantics can it truly play its role in assisting decision-making and regulatory oversight.

[0005] In summary, from a data perspective, it is urgent to address issues such as building a framework in the time and space dimensions, resolving data disturbances without affecting the meaning of data required for subsequent knowledge reasoning, continuously improving and expanding the key feature meanings of the semantics of inland waterway vessel traversal behavior, establishing corresponding semantic calculation rules, setting up variable-scale sliding sequence windows that meet the semantic conditions for detecting the traversal behavior of inland waterway vessels, and realizing online detection of traffic behavior semantics. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention provides a method and apparatus for semantic detection of lateral movement behavior of inland waterway vessels based on a sliding window. Starting from the data level, it transforms real-time data into semantic knowledge, thereby solving the problem of semantic detection of lateral movement behavior of inland waterway vessels.

[0007] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0008] A semantic detection method for the lateral movement behavior of inland waterway vessels based on a sliding window, the method comprising the following steps:

[0009] Acquire spatial coordinate data of the waterway environment;

[0010] Acquire AIS trajectory data of inland waterway vessels and perform preprocessing;

[0011] Based on the spatial coordinate data of the acquired waterway environment and the preprocessed AIS trajectory data of inland waterway vessels, the vessel trajectory points are traversed using a sequence sliding window. The spatial topological relationship between the vessel trajectory points and trajectory segments and the waterway is calculated according to the dimension-extended 9-intersection model DE-91M, resulting in a set of semantic relationships of vessel topological behavior.

[0012] Construct a topological behavior relation order rule for the semantics of lateral movement of inland waterway vessels, and combine it with the set of semantic relations of vessel topological behavior to detect the lateral movement of inland waterway vessels.

[0013] Furthermore, the method also includes the following steps:

[0014] The detected trajectories of ships traversing inland waterways are visualized and their semantic information is annotated.

[0015] Furthermore, the AIS trajectory data of inland waterway vessels is processed into Kafka using streaming data processing, and the vessel AIS trajectory data stored in Kafka is consumed using Spark Streaming.

[0016] Furthermore, the streaming data processing method refers to storing the AIS trajectory data messages of inland waterway vessels precisely once in the same partition of the same topic, and using the vessel's MMSI code as the userid for subsequent consumption.

[0017] Furthermore, the AIS trajectory data of inland waterway vessels includes the vessel's mmsi code, latitude and longitude, time, heading, and speed.

[0018] Further preprocessing includes time series deduplication, outlier cleaning, angle denoising, three-point denoising, and post-denoising interpolation; specifically:

[0019] The ship's AIS trajectory data is sorted in timestamp order, and then deduplicated according to the timestamp. Next, outlier removal is performed based on the set minimum time interval for AIS data broadcast, maximum speed threshold, and maximum and minimum heading threshold. Then, three-point denoising and angle denoising are performed based on the turning rate threshold between three consecutive trajectory points, the acceleration difference threshold of the ship's speed, and the angle change and angle threshold of connecting three trajectory points. Finally, linear interpolation is performed on the AIS trajectory data.

[0020] Furthermore, the sliding window settings include:

[0021] The consecutive trajectory points judged within the sliding window must be consecutive and valid in the time series, and the order of the sliding window judgment is based on the time series of the AIS trajectory data.

[0022] The sliding window detects the semantics of the transverse navigation behavior of inland waterway vessels by locking onto the vessel trajectory points and trajectory segments within the target water area that have a corresponding spatial topological relationship with the waterway based on the acquired spatial coordinate data of the waterway environment and the vessel's AIS trajectory data.

[0023] The sliding window slides in a variable-scale manner in space and time based on the time and mileage of each traverse behavior that meets the calculation rules.

[0024] Furthermore, topological behavioral semantic relations refer to the different topological behavioral semantic relations that arise due to the different topological interaction relations between ship objects and waterway objects, and are divided into point-line relations, point-plane relations, line-line relations and line-plane relations;

[0025] Ships are abstracted as point objects, ship trajectories as line objects, channel boundaries as line objects, and channel regions as area objects. Using the dimension-extended 9-intersection model DE-91M, the spatial relationship matrix between ship objects and channel objects is calculated, yielding a set of semantic relationships related to ship topological behavior. Wherein:

[0026] In the point-line relationship, it is divided into two types: the ship is not on the line (PL1) and the ship is on the line (PL2).

[0027] In the point-area relationship, it is divided into three categories: the ship is within the channel area (PA1), the ship is outside the channel area (PA2), and the ship is on the channel boundary line (PA3).

[0028] In line-to-line relationships, there are three types: the ship crosses the channel boundary line (LL1), the ship's starting point and ending point are both on the channel boundary line (LL2), the ship's trajectory segment starts on the channel boundary line but ends on the channel boundary line (LL3), the ship's trajectory does not intersect with the channel boundary line (LL4), and the ship travels along the channel boundary line (LL5).

[0029] In line-plane relationships, there are four types: a ship crossing a channel area (LA1), a ship sailing alongside a channel area with part of its trajectory tangent to the channel boundary (LA2), a ship sailing outside a channel area (LA3), a ship entering or leaving a channel area (LA4), and a ship sailing within a channel area (LA5).

[0030] Furthermore, it is first determined whether the ship trajectory point i is within the target area (PA1) or on the boundary line of the target area (PA3). This is a prerequisite for performing semantic detection of the ship's lateral movement behavior within the target area.

[0031] The topological behavior order rules for the semantics of lateral navigation behavior of inland waterway vessels include:

[0032] (1) The ship's track segment i and subsequent track segment i+j generate cross-topology behavior relationships with the left and right boundary lines of the channel in turn (LL1).

[0033] (2) When the ship’s track segment i enters the channel area (LA4), the track points between track segment i and track segment i+j are all within the channel area (PA1), and when track segment i+j leaves the channel area (LA4).

[0034] (3) The vessel’s track points are in the following order: in one side of the channel (PA1), in the channel area, or multiple consecutive track points in the channel area (PA1) and in the other side of the channel (PA1);

[0035] (4) The ship's track segment i and the two side boundary lines of the channel simultaneously generate a cross-topology relationship (LL1);

[0036] (5) The vessel’s track points are directly in sequence in the area on one side of the channel (PA1) and in the area on the other side of the channel (PA1);

[0037] (6) The ship's trajectory point is in the channel area on one side (PA1) and then enters the channel area on the other side (LA4);

[0038] (7) The trajectory segment i (LL3) of the ship starts at one side boundary line of the channel and ends at another side boundary line of the channel, and the trajectory segment i crosses the other boundary line of the channel (LL1).

[0039] (8) The starting point of the ship's trajectory points is on the boundary line (PL2) on one side of the channel, and the ending point is on the boundary line (PL2) on the other side of the channel, and all trajectory points within the trajectory segment are within the channel area (PA1).

[0040] A sliding window-based semantic detection device for inland waterway vessel lateral behavior, used to implement the sliding window-based semantic detection method for lateral behavior of inland waterway vessels as described in any one of the above, includes a power supply, an AIS data interface, a user interface, a circuit board, a data processing module, a semantic tag library module, a topological behavior semantic relationship recognition module, a high-order semantic calculation module, and a server.

[0041] The power supply provides power, the circuit board houses the data processing module, semantic tag library module, topological behavior semantic relationship recognition module, and high-order semantic calculation module, the AIS data interface imports AIS trajectory data of inland waterway vessels, the data processing module preprocesses the AIS trajectory data of inland waterway vessels, the semantic tag library module stores and retrieves spatial coordinate data of the waterway environment, the topological behavior semantic relationship recognition module obtains the set of vessel topological behavior semantic relationships, the high-order semantic calculation module detects the lateral behavior of inland waterway vessels according to the topological behavior relationship order rules of the semantics of lateral behavior, the user interface outputs the semantics of lateral behavior of inland waterway vessels, and the server is used to call the lateral behavior of inland waterway vessels and visualize it.

[0042] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0043] This invention not only establishes, identifies, and expands the semantic expression of ship lateral movement behavior at the semantic level, but also extracts, calculates, detects, and visualizes the semantic behavior of ship data in real time at the data level, which is of great help to real-time monitoring and decision support for water traffic. Attached Figure Description

[0044] Figure 1Flowchart of the semantic detection method for lateral movement behavior of inland waterway vessels based on sliding window provided by the present invention;

[0045] Figure 2 This is a schematic diagram of the online AIS data processing flow provided by the present invention;

[0046] Figure 3 This is a semantic diagram illustrating the lateral movement behavior of inland waterway vessels provided by the present invention.

[0047] Figure 4 Flowchart of the semantic detection algorithm for lateral movement behavior of inland waterway vessels provided by this invention;

[0048] Figure 5 This is a schematic diagram of the device for online semantic calculation and detection of the lateral movement behavior of inland waterway vessels provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0050] Example 1

[0051] This embodiment presents an online semantic detection method for the lateral movement behavior of inland waterway vessels based on a sliding window. Figure 1 As shown, it includes the following steps:

[0052] Step 1: Taking the area near the Wuhan Yangtze River Bridge in China as a case study, we selected the Yangtze River electronic nautical chart and plotted the spatial coordinate data of the objects in the Wuhan Yangtze River Bridge area (hereinafter referred to as the bridge area) and the waterway environment (hereinafter referred to as the geographical environment). We also collected the streaming data of the AIS trajectory of inland waterway vessels (hereinafter referred to as vessels) in real time through the device.

[0053] Step 2: Store the AIS data to be processed into Kafka to begin online decoding and writing.

[0054] Step 3: Use Spark Streaming to consume the ship AIS data stored in Kafka, and perform preprocessing and subsequent operations on the AIS data.

[0055] Step 4: Combine spatial coordinate points to filter AIS trajectory data within the bridge area, and perform deduplication, outlier cleaning, noise reduction, and post-noise interpolation preprocessing on the original AIS trajectory data.

[0056] Step 5: Based on a sliding window under certain preset conditions, compare the continuous trajectory points under the sliding window with the coordinate points of the bridge area and waterway environment to obtain the topological behavior relationship, and store the obtained ship topological behavior semantic relationship set.

[0057] The semantics of lateral navigation behavior of inland waterway vessels are composed of topological behavioral semantics. Topological behavioral semantics refers to the different topological behavioral semantic relationships that arise from the different topological interactions between vessel objects and inland waterway navigation geographical environment objects. Based on the topological relationship between vessel objects and inland waterway navigation geographical environment objects, it can be divided into point-line relationship, point-plane relationship, line-line relationship, and line-plane relationship.

[0058] In this invention, ships are represented as point objects, the boundaries of bridge area objects and waterway objects within bridge areas in the navigation geographic area object are represented as line objects, and the bridge area objects and waterway objects in the navigation geographic area object are represented as surface objects.

[0059] Combining the 9-intersection model (DE-91M), and referring to the patent publication CN114550497A entitled "A Semantic Calculation Method and Device for Ship Behavior," the topological behavioral semantic relationships are extracted, including point-line relationships, point-plane relationships, line-line relationships, and line-plane relationships. In actual operation and calculation, this embodiment uses PyCharm as the development environment, and calculates the intersection matrix between points, lines, and planes by introducing the shaply library. Then, the intersection matrix is ​​transformed into the topological behavioral semantics of this invention:

[0060] In point-line relationships, it is divided into two categories: ships not on the line (PL1) and ships on the line (PL2).

[0061] In the point-area relationship, it is divided into the following categories: the ship is in the bridge area and inside the waterway (PA1), the ship is in the bridge area and outside the waterway (PA2), and the ship is on the boundary line of the bridge area and waterway area (PA3).

[0062] In line-to-line relationships, there are topological behaviors that are uniquely intersected by lines: (LL1) the ship crosses the channel boundary line; (LL2) the ship's starting point and ending point are both on the channel boundary; (LL3) the ship's trajectory segment starts on the channel boundary but ends on the channel boundary; (LL4) the ship's trajectory does not intersect with the channel boundary; and (LL5) the ship travels along the channel boundary.

[0063] In line-plane relationships, there are three types: a ship traversing a certain channel area (LA1), a ship sailing close to a certain channel area with part of its trajectory tangent to the channel boundary (LA2), a ship sailing outside the channel area (LA3), a ship entering or leaving a certain channel area (LA4), and a ship sailing within a certain channel area (LA5).

[0064] Step 6: Based on the preset semantic calculation engine for the lateral movement behavior of inland waterway vessels, calculate its topological behavioral semantic relation set, determine the higher-order semantics of the lateral movement behavior of inland waterway vessels, and store it in the database.

[0065] The semantics of a vessel's traversing behavior in inland waterways is not only composed of topological behavioral semantics, but also constitutes a higher-order behavioral semantics constructed from topological behavioral semantics in a specific order. Based on the topological behavioral semantic relationships extracted above, a set of topological behavioral semantic relationships is established. First, it is determined whether the vessel's trajectory point i is within the bridge area (PA1) or on the boundary line of the bridge area object (PA3). This is a prerequisite for performing traversing behavior semantic detection on all vessels within the specified area.

[0066] In this invention, the topological behavior relation order rules that satisfy the semantics of the lateral movement behavior of inland waterway vessels include:

[0067] The ship's track segment i and subsequent track segment i+j generate a transverse topological behavior relationship with the left and right boundary lines of the channel in turn (LL1);

[0068] The ship's track segment i enters the channel area (LA4), the track points between track segment i and track segment i+j are all within the channel area (PA1), and track segment i+j exits the channel area (LA4).

[0069] The vessel's track points are in the following order: in one side of the channel (PA1), in the channel area, or multiple consecutive track points in the channel area (PA1) and in the other side of the channel (PA1);

[0070] The ship's track segment i and the two boundary lines of the channel simultaneously generate a transverse topological behavior relationship (LL1);

[0071] The vessel's track points are directly and sequentially located in the area on one side of the channel (PA1) and the area on the other side of the channel (PA1);

[0072] The ship's trajectory point is in the area on one side of the channel (PA1), and then enters the area on the other side of the channel (LA4);

[0073] The trajectory segment i (LL3) of the ship starts at one side boundary line of the channel and ends at another side boundary line of the channel, and the trajectory segment i crosses the other boundary line of the channel (LL1).

[0074] The starting point of the ship's trajectory is on one side of the channel boundary line (PL2), and the ending point is on the other side of the channel boundary line (PL2), and all trajectory points within the trajectory segment are within the channel area (PA1).

[0075] Step 7: Visualize the detected ship trajectories in inland waterway and annotate their semantic information.

[0076] The semantic rules for the lateral movement of inland waterway vessels identified above are stored in a professional corpus. After calculating the higher-order semantic rules for the lateral movement of inland waterway vessels, the corresponding trajectory segments are stored and visualized. The Folium package is used to visualize the corresponding AIS data and spatial coordinates, and the semantic rules for the lateral movement are identified.

[0077] In this embodiment, the Yangtze River electronic nautical chart is selected to plot and obtain spatial coordinate data of the Wuhan Yangtze River Bridge area and waterway environment (hereinafter referred to as the geographical environment). The device collects AIS trajectory streaming data of inland waterway vessels (hereinafter referred to as vessels), decodes and obtains the vessel's MMSI code (Maritime Mobile Service Identifier), latitude and longitude, time, heading, and speed data, and stores them in Kafka. The required AIS trajectory data within the bridge area is filtered by combining the spatial coordinate data. Spark Streaming is used to consume the cached AIS trajectory data stored in Kafka. Then, the AIS trajectory data is preprocessed, including time series deduplication, outlier cleaning, angle denoising, three-point denoising, and post-denoising interpolation.

[0078] In this embodiment, the sliding window settings for detecting the semantics of lateral movement behavior of inland waterway vessels are as follows:

[0079] The consecutive trajectory points judged within the sliding window must also be consecutive and valid in the time series, and the order of the sliding window judgment is strictly in accordance with the time series of the AIS trajectory data;

[0080] The sliding window uses the acquired spatial coordinates and ship AIS data to lock ship trajectory points and lines within the waterway that have a corresponding topological relationship with the waterway to detect the semantics of inland waterway vessel lateral navigation behavior.

[0081] The sliding window slides in a variable-scale manner in both space and time based on the time and distance of each traverse behavior that meets the calculation rules. The larger the time and distance of the traverse behavior, the larger the scale of the sliding window.

[0082] The device includes a power supply, an AIS data interface, a user interface, a circuit board, a data processing module, a semantic tag library module, a topological behavior semantic relationship recognition module, a high-order semantic computing module, and a server. These three modules form a semantic computing engine for ship inland waterway traversing behavior.

[0083] The power supply is used for power supply, the circuit board is used to install various modules, the AIS data interface is used to import external AIS data, the data processing module performs the above-mentioned writing, consumption, and preprocessing on the input AIS data, the topological behavior semantic relationship recognition module calculates the spatial matrix based on the preprocessed ship AIS trajectory information and the spatial semantic information extracted from the semantic tag library according to the 9-cross model (DE-91M), and further converts it into ship topological behavior semantic relationships, the high-order semantic calculation module calculates the obtained ship topological behavior semantic relationship set according to the above semantic calculation rules, realizes the detection of inland waterway vessel lateral navigation behavior semantics, the user interface is used to output inland waterway vessel lateral navigation behavior semantics, and the server is used to call the stored high-order semantic behaviors and perform visualization.

[0084] Example 2

[0085] This embodiment presents an online semantic detection method for the lateral movement behavior of inland waterway vessels based on a sliding window. Figure 1 and Figure 2 As shown, it includes the following steps:

[0086] Based on the Yangtze River electronic nautical chart, spatial coordinate data of the bridge area and waterway environment of the Wuhan Yangtze River Bridge were plotted and obtained.

[0087] Based on the real-time AIS trajectory data of inland waterway vessels collected by the device, the data is processed using streaming data and stored in Kafka for consumption calculation.

[0088] The necessary data cleaning of inland waterway vessel AIS data is performed based on preprocessing methods such as time series deduplication, outlier cleaning, three-point denoising, angle denoising, and post-denoising interpolation.

[0089] Based on the characteristics of the topological behavior semantic relationship between ships and the geographic environment, and combined with the acquired spatial coordinates and AIS data, a suitable sliding sequence window is set to calculate the topological behavior semantic relationship constrained by the time series in turn, and a topological relationship behavior semantic set is formed.

[0090] Based on higher-order semantic calculation rules for inland waterway vessel traverse behavior, a semantic calculation engine for inland waterway vessel traverse behavior is set up. The engine calculates and detects higher-order semantics of inland waterway vessel traverse behavior through the semantic relation set of inland waterway vessel topology behavior, and stores the AIS data of inland waterway vessel traverse behavior.

[0091] Based on the existing AIS dataset of inland waterway vessel traverse behavior, the data is visualized and labeled.

[0092] Extracting spatial coordinates involves opening the Yangtze River Electronic Navigation Map (http: / / www.cjienc.cn), selecting the Hankou area of ​​Wuhan, and manually plotting the Yangtze River waterway shoreline, navigation channel, and navigation marks within a range of approximately 2.5 km upstream and downstream of the Wuhan Yangtze River Bridge. The obtained spatial coordinates are then stored in PyCharm as a semantic tag database, which can be easily accessed directly based on semantic information during semantic calculations.

[0093] Streaming data processing refers to storing AIS data messages (producers) precisely once in the same topic and partition when receiving AIS signals from ships on the Yangtze River using a device, and using the ship's mmsi as the userid for subsequent consumption by the server (consumers).

[0094] Preprocessing AIS data involves combining the geographical and traffic characteristics of the waterway. First, the ship AIS data is sorted according to timestamps. Then, duplicate data is removed based on timestamps. Next, outliers are removed based on the set minimum broadcast time interval, maximum speed threshold, and maximum and minimum heading thresholds. Then, three-point denoising and angle denoising are performed based on the turning rate (ROT) threshold between three consecutive trajectory points of the ship, the acceleration difference threshold of the speed, and the angle change and angle threshold of connecting three trajectory points. Finally, linear interpolation is performed on the AIS data.

[0095] The method of calculating the semantic relationship of ship topology behavior using a sequential sliding window refers to using the shaply library in the PyCharm development environment, based on the spatial relationship matrix in the DE-91M cross model, traversing the ship trajectory points using a sequential sliding window, calculating the spatial topology relationship between the ship trajectory points and segments and the environmental coordinate points, lines, and surfaces, and setting semantic labels for the corresponding semantic relationship of ship topology behavior.

[0096] The semantic engine for lateral navigation behavior of inland waterway vessels refers to constructing semantic calculation rules for lateral navigation behavior of inland waterway vessels by using the topological behavior relationship of vessels as the computational object and logical symbols as operators. When the semantic sequence of topological behavior of vessels in the sequence sliding window conforms to the above semantic calculation rules for lateral navigation behavior of inland waterway vessels, its AIS data is extracted, stored, and semantically tagged.

[0097] Semantic visualization of the lateral movement behavior of inland waterway vessels refers to introducing the Folium library into the PyCharm development environment to highlight and visualize the stored AIS data that conforms to the lateral movement behavior of vessels, and refreshing it every 1 second.

[0098] Example 3

[0099] The method for calculating the semantics of the lateral movement behavior of inland waterway vessels in this embodiment is as follows: Figure 1 andFigure 2 As shown, it includes the following steps:

[0100] Step S1: Access the ship's AIS trajectory data and write the dynamic data fields precisely to the same topic "data" and the same partition on the server for storage.

[0101] Step S2: Consume and process messages from the Kafka topic using Spark Streaming, process AIS data using the preprocessing described above, call the spatial semantic coordinates in the semantic tag database, and use a sequence sliding window to calculate and process the data. First, obtain the set of topological behavior semantic relations of ships, and then detect the high-order inland waterway ship lateral behavior semantics according to the calculation rules.

[0102] Step S3: Spark Streaming sends the processed data to the database to store it as semantic labels for the traversing behavior of inland waterway vessels and AIS trajectory data.

[0103] Step S4: Retrieve the lateral driving trajectory data from the database and use Folium to visualize the trajectory once per second in the PyCharm development environment.

[0104] In step S1, the access to the ship's AIS trajectory data is achieved by the device of this invention, which decodes, writes, consumes, and preprocesses the AIS data. AIS data is divided into static and dynamic data fields. This invention mainly uses the dynamic fields of AIS data. Table 1 is a description table of the dynamic field types of AIS data in this invention.

[0105] Table 1. Description of Dynamic Field Types in AIS Data

[0106]

[0107]

[0108] In step S2, preprocessing includes sorting the data in timestamp order, then deduplicating the trajectory data according to the timestamps, then removing outliers according to the set minimum time interval for AIS data broadcast, maximum speed threshold, and maximum and minimum heading thresholds, then performing three-point denoising and angle denoising according to the turning rate (ROT) threshold between three consecutive trajectory points of the ship, the acceleration difference threshold of the speed, and the angle change and angle threshold of connecting three trajectory points, and finally performing cubic spline interpolation on the AIS data.

[0109] Specifically, because the ASI technical standard stipulates that each minute is divided into 2250 time slots, each time slot can publish one message no longer than 256 bits. Messages longer than 256 bits need to have their time slots extended. Therefore, there is no explicit definition of the minimum time interval for AIS data broadcasting. Considering the practicality of receiving AIS data in general, this embodiment adopts 2 seconds as the minimum time interval for AIS data broadcasting to ensure data efficiency.

[0110] Specifically, according to regulations, the speed of a ship during normal navigation should comply with:

[0111] The maximum speed during flood season shall not exceed 30 km / h, and during dry season shall not exceed 25 km / h. This invention sets the maximum speed threshold to 16.2 knots during flood season and 13.5 knots during dry season. The maximum heading threshold is 360°, and the minimum heading threshold is 0°.

[0112] The acceleration difference in speed is the difference in acceleration between the preceding and following trajectory segments i and j in the three-point denoising process, and the formula is:

[0113]

[0114] In the above formula, diff ath The distance represents the difference in acceleration between the preceding and following trajectory segments. i and distance j t represents the distance between trajectory segments i and j, and t represents the time difference between the trajectory segments.

[0115] Turn rate represents the magnitude of the change in heading between preceding and following trajectory points. The formula for calculating turn rate is:

[0116] ROT = diff_course / t

[0117] In the above formula, ROT is the turning rate, diff_course is the heading difference, and t is the time difference of the trajectory segment.

[0118] This invention takes into account setting the rotation rate (ROT) threshold to 1.5 and the difference in acceleration between the preceding and following trajectory segments in the three-point denoising process to a threshold of 0.8.

[0119] Specifically, spatial semantic coordinates include the latitude and longitude coordinates of the vertices of the bridge area and the main areas of the waterway, as well as the coordinates of navigation marks, obtained through the Yangtze River electronic navigation map.

[0120] Specifically, the relationship between ship objects and geographic environment objects is mainly the topological interaction relationship between ship objects and the geographic environment, that is, the semantic relationship of ship topological behavior. Combined with... Figure 3 Let's take a look. Figure 3This invention provides a semantic diagram of the lateral navigation behavior of inland waterway vessels. Vessels are abstracted as point objects, vessel trajectories as line objects, and the geographical environment is abstracted into points, lines, and surfaces based on their different geometric attributes. Building upon the existing 9-cross model (DE-9IM) based on dimension expansion, a higher-order semantic diagram of the lateral navigation behavior of inland waterway vessels is constructed according to the spatial topological relationship between the corresponding vessels and the inland waterway.

[0121] The computational matrix for the dimension-extended 9-intersection model DE-91M is as follows:

[0122]

[0123] In the formula, A represents the interior of A, and A° represents the boundary of A. - This represents the outer region of A, and the same applies to B;

[0124] The 9-dimensional extended intersection model calculates the intersection between three regions of two spatial entities, yielding the DIM value. A DIM value of 0 indicates a point intersection, 1 indicates a line intersection, 2 indicates a surface intersection, and F indicates an empty set intersection. Each matrix result corresponds to a semantic relationship of ship topological behavior.

[0125] More specifically, the logical symbol ∩ is used to represent "and", which means that the expression is true only if both conditions on both sides are met, and the corresponding high-level semantic behavior is obtained.

[0126] An example of traversing behavior can be represented as:

[0127] traj i+j ,point i ,point i+1 ,point i+j ,point i+j+1

[0128] crossing={(traj i =Move)∩(R DE-9IM (traj i ,channel_line left )=LA1)

[0129] ∩(R DE-9IM (traj i ,channel_line right )=LA1)}

[0130] crossing={(R DE-9IM (point i,channel_area left )=PA1)

[0131] ∩(R DE-9IM (point i+1 ,channel_area right )=PA1)

[0132] ∩(R DE-9IM (point_area i ,channel)=PA2)

[0133] π(R DE-9IM (point i+1 ,channel_area)=PA2)}

[0134] crossing={(traj i =Move)π(traj i+j =Move)∩(R DE-9IM (traj i ,channel_line left )=LL1)

[0135] ∩(R DE-9IM (traj i+j ,channel_line right )=LL1)}

[0136] crossing={point

[0137] =point i+1 \point i+2 \...\point i+j ∩(R DE-9IM (traj i ,channel_area)=LA4)

[0138] ∩(R DE-9IM (point,channel_area)=PA1)

[0139] ∩(R DE-9IM (traj i+j ,channel_area)=LA4)}

[0140] crossing={point

[0141] =point i+1 \point i+2 \...\point i+j ∩(R DE-9IM(point i ,channel_area left )=PA1)

[0142] ∩(R DE-9IM (point,channel_area)=PA1)

[0143] ∩(R DE-9IM (point i+j+1 ,channel_area right )=PA1)}

[0144] crossing={traj i

[0145] =Move∩(R DE-9IM (poiht i ,channel_area left )=PA1)

[0146] ∩(R DE-9IM (traj i ,channel_area right )=LA4)}

[0147] crossing={traj i

[0148] =Move∩(R DE-9IM (traj i ,channel_line left )=LL3)

[0149] ∩(R DE-9IM (traj i ,channel_line right )=LL1)}

[0150] crossing={point

[0151] =point i+1 \poiht i+2 \...\poiht i+j π(R DE-9IM (poihti,channel_line left )=PL2)

[0152] ∩(R DE-9IM (poiht i+j ,channel_line right )=PL2)

[0153] ∩(R DE-9IM (point, channel_area)=PA1)}

[0154] In the formula, traj i The trajectory segment i, traj, represents the point at which the ship begins to traverse. i+j This represents the trajectory segment i+j after the i-th trajectory segment in which the ship completes its lateral movement, poiht i Point i represents the trajectory point i that was still outside the channel when the ship began to traverse. i+1 This represents the point i+1 on the ship's trajectory within the channel when it begins to traverse. i+j This represents the point i+j on the trajectory within the channel before the ship completes its lateral movement. i+j+1 This represents the trajectory point i+j+1 outside the channel when the ship completes its lateral movement. Move indicates that the ship is moving, not stationary.

[0155] Specifically, in combination Figure 4 Let's take a look. Figure 4 The flowchart of the semantic detection algorithm for lateral movement behavior of inland waterway vessels based on a sliding sequence window provided by this invention is as follows:

[0156] 1. Import ship AIS data and waterway spatial semantic data;

[0157] 2. Use the pandas library to extract the mmsi column from the ship AIS data and use pd.unique() to form a list of ship IDs;

[0158] 3. Traverse the list of IDs, extract the AIS dynamic data of each ship in turn, and perform preprocessing such as timestamp deduplication, outlier removal, angle denoising, three-point denoising, and linear interpolation.

[0159] 4. Establish a sequence sliding window, such that sequence i = 0, use a while loop to ensure that i will not be greater than the number of data rows of the ship, set j = i + 1, form a sequence window to extract continuous trajectory segments with different sequence lengths, use the shaply library to perform spatial topological mathematical calculations on the extracted trajectory points, segments and channel boundary lines and channel areas, transform the obtained spatial matrix into ship topological behavior semantics, and store it as a set of ship topological behavior semantic relations in a certain order.

[0160] 5. Import the semantic calculation rules for the lateral movement behavior of inland waterway vessels into the algorithm, filter the sequence set of semantic relationship of vessel topology behavior that conforms to the semantic calculation rules for the lateral movement behavior of inland waterway vessels, and store the dynamic AIS data of the vessels under the corresponding sequence in the lateral movement behavior semantic tag database.

[0161] Example 4

[0162] This invention also provides a semantic calculation device for the lateral movement behavior of inland waterway vessels based on a sequential sliding window, such as... Figure 5 As shown, the device includes a power supply, an AIS data interface, a user interface, a circuit board, a data processing module, a semantic tag library module, a topological behavior semantic relationship recognition module, a high-order semantic computing module, and a server. The three semantic modules together form a semantic computing engine for ship inland waterway traversing behavior.

[0163] The AIS data interface connects external AIS data to the device.

[0164] The circuit board is equipped with a data processing module, a semantic tag library module, a topological behavior semantic relationship recognition module, and a high-order semantic computing module.

[0165] The data processing module performs the aforementioned writing, consumption, and preprocessing tasks on the input AIS data, including data-perspective denoising, three-point denoising, time-series deduplication, and linear interpolation.

[0166] The semantic tag library module is used to store and retrieve the spatial semantic data information obtained from the above mapping, including storing the semantic data information of the transverse navigation behavior of inland waterway vessels detected subsequently.

[0167] The topological behavior semantic relationship recognition module calculates a spatial matrix based on the preprocessed ship AIS trajectory information and the spatial semantic information extracted from the semantic tag library using the 9-cross model (DE-91M), and further converts it into ship topological behavior semantic relationships.

[0168] The high-order semantic computation module performs calculations based on the above semantic computation rules, taking the obtained set of ship topological behavior semantic relations as input, to detect the semantics of inland waterway vessel traversing behavior and output high-order semantics of inland waterway vessel traversing behavior.

[0169] The power supply is used to power the various circuits or devices in the above-mentioned device.

[0170] The user interface is used to output the semantics of the lateral movement behavior of inland waterway vessels.

[0171] The server is used to invoke stored higher-order semantic behaviors and visualize them.

[0172] In summary, this invention discloses an online detection method and apparatus for the semantics of lateral navigation behavior of inland waterway vessels based on a sliding window. The method includes: taking the area near the Wuhan Yangtze River Bridge in China as a research example, selecting the Yangtze River electronic nautical chart, plotting and obtaining spatial coordinate data of objects in the bridge area and waterway environment of the Wuhan Yangtze River Bridge, collecting AIS trajectory data of inland waterway vessels in real time through an apparatus, using streaming data processing, storing the data in Kafka for consumption during real-time computation by Spark Streaming, filtering AIS trajectory data within the bridge area based on spatial coordinate points, and performing time-series deduplication, outlier cleaning, three-point denoising, angle denoising, and post-denoising interpolation preprocessing on the original AIS trajectory data; based on a sliding window under certain preset conditions, comparing the continuous trajectory points under the sliding window with the coordinate points of the bridge area and waterway environment to obtain a set of topological behavior relationships, and calculating and judging the set of topological behavior relationships based on a preset inland waterway vessel lateral navigation behavior semantics to obtain higher-order inland waterway vessel lateral navigation behavior semantics, which are then stored and visualized online. This invention not only establishes, identifies, and expands the semantic expression of ship lateral movement behavior at the semantic level, but also extracts, calculates, detects, and visualizes the semantic behavior of ship data in real time at the data level, which is of great help to real-time monitoring and decision support for water traffic.

[0173] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0174] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A semantic detection method for the lateral movement behavior of inland waterway vessels based on a sliding window, characterized in that, The method includes the following steps: Acquire spatial coordinate data of the waterway environment; Acquire AIS trajectory data of inland waterway vessels and perform preprocessing; Based on the spatial coordinate data of the acquired waterway environment and the preprocessed AIS trajectory data of inland waterway vessels, the vessel trajectory points are traversed using a sequence sliding window. The spatial topological relationship between the vessel trajectory points and trajectory segments and the waterway is calculated according to the dimension-extended 9-intersection model DE-91M, resulting in a set of semantic relationships of vessel topological behavior. Construct topological behavior relation order rules for the semantics of lateral movement of inland waterway vessels, and combine them with the set of semantic relations of vessel topological behavior to detect the lateral movement of inland waterway vessels; First, it is necessary to determine whether the ship trajectory point i is within the target area or on the boundary line of the target area. This is a prerequisite for semantic detection of the lateral movement behavior of ships within the target area. The topological behavior order rules for the semantics of lateral navigation behavior of inland waterway vessels include: (1) The ship's track segment i and subsequent track segment i+j successively generate cross-topological behavior relationships with the left and right boundary lines of the channel; (2) When the ship enters the channel area in track segment i, all track points between track segment i and track segment i+j are within the channel area, and when track segment i+j leaves the channel area. (3) The vessel’s track points are in the following order: in one side of the channel, in the channel area, or multiple consecutive track points in the channel area and in the other side of the channel; (4) The ship's track segment i and the two side boundary lines of the channel simultaneously generate a transverse topological behavior relationship; (5) The vessel’s track points are directly in sequence on one side of the channel and on the other side of the channel; (6) The ship's trajectory point is in the area on one side of the channel, and then enters the area on the other side of the channel; (7) The starting point of the vessel is on one side boundary line of the channel, the ending point is not on one side boundary line of the channel, and the trajectory segment i crosses the other boundary line of the channel. (8) The starting point of the ship's trajectory points is on one side of the channel boundary line, and the ending point is on the other side of the channel boundary line, and all trajectory points within the trajectory segment are within the channel area.

2. The semantic detection method for lateral movement behavior of inland waterway vessels based on a sliding window according to claim 1, characterized in that, The method also includes the following steps: The detected trajectories of ships traversing inland waterways are visualized and their semantic information is annotated.

3. The semantic detection method for lateral movement behavior of inland waterway vessels based on a sliding window according to claim 1, characterized in that, The AIS trajectory data of inland waterway vessels is processed into Kafka using a streaming data method, and the vessel AIS trajectory data stored in Kafka is consumed using Spark Streaming.

4. The semantic detection method for lateral movement behavior of inland waterway vessels based on a sliding window according to claim 3, characterized in that, The streaming data processing method refers to storing the AIS trajectory data messages of inland waterway vessels in the same partition of the same topic in one precise instance, and using the vessel's MMSI code as the userid for subsequent consumption.

5. The semantic detection method for lateral movement behavior of inland waterway vessels based on a sliding window according to claim 1, characterized in that, The AIS trajectory data of inland waterway vessels includes the vessel's mmsi code, latitude and longitude, time, heading, and speed.

6. The semantic detection method for lateral movement behavior of inland waterway vessels based on a sliding window according to claim 5, characterized in that, Preprocessing includes time series deduplication, outlier cleaning, angle denoising, three-point denoising, and post-denoising interpolation; specifically: The ship's AIS trajectory data is sorted in timestamp order, and then deduplicated according to the timestamp. Next, outlier removal is performed based on the set minimum time interval for AIS data broadcast, maximum speed threshold, and maximum and minimum heading threshold. Then, three-point denoising and angle denoising are performed based on the turning rate threshold between three consecutive trajectory points, the acceleration difference threshold of the ship's speed, and the angle change and angle threshold of connecting three trajectory points. Finally, linear interpolation is performed on the AIS trajectory data.

7. The semantic detection method for lateral movement behavior of inland waterway vessels based on a sliding window according to claim 1, characterized in that, The sliding window settings include: The consecutive trajectory points judged within the sliding window must be consecutive and valid in the time series, and the order of the sliding window judgment is based on the time series of the AIS trajectory data. The sliding window detects the semantics of the transverse navigation behavior of inland waterway vessels by locking onto the vessel trajectory points and trajectory segments within the target water area that have a corresponding spatial topological relationship with the waterway based on the acquired spatial coordinate data of the waterway environment and the vessel's AIS trajectory data. The sliding window slides in a variable-scale manner in space and time based on the time and mileage of each traverse behavior that meets the calculation rules.

8. The semantic detection method for lateral movement behavior of inland waterway vessels based on a sliding window according to claim 1, characterized in that, Topological behavioral semantic relations refer to the different topological behavioral semantic relations that arise from the different topological interaction relations between ship objects and waterway objects. They are divided into point-line relations, point-plane relations, line-line relations and line-plane relations. Ships are abstracted as point objects, ship trajectories as line objects, channel boundaries as line objects, and channel regions as area objects. Using the dimension-extended 9-intersection model DE-91M, the spatial relationship matrix between ship objects and channel objects is calculated, yielding a set of semantic relationships related to ship topological behavior. Wherein: In point-line relationships, there are two categories: ships not on the line and ships on the line. In the point-area relationship, it is divided into three categories: the ship is within the waterway area, the ship is outside the waterway area, and the ship is on the waterway boundary line. In line-to-line relationships, there are three types: a ship crossing the channel boundary line, the ship's starting and ending points being on the channel boundary line, the ship's trajectory segment starting on the channel boundary line but ending on the channel boundary line, the ship's trajectory not intersecting with the channel boundary line, and the ship traveling along the channel boundary line. In line-plane relationships, there are three types: a ship traversing a certain channel area, a ship sailing alongside a certain channel area with part of its trajectory tangent to the channel boundary, a ship sailing outside a channel area, a ship entering or leaving a certain channel area, and a ship sailing within a certain channel area.

9. A sliding window-based semantic detection device for inland waterway vessel traversing behavior, used to implement the sliding window-based semantic detection method for traversing behavior of inland waterway vessels according to any one of claims 1 to 8, characterized in that, It includes a power supply, AIS data interface, user interface, circuit board, data processing module, semantic tag library module, topological behavior semantic relationship recognition module, high-order semantic computing module, and server; The power supply provides power, the circuit board houses the data processing module, semantic tag library module, topological behavior semantic relationship recognition module, and high-order semantic calculation module, the AIS data interface imports AIS trajectory data of inland waterway vessels, the data processing module preprocesses the AIS trajectory data of inland waterway vessels, the semantic tag library module stores and retrieves spatial coordinate data of the waterway environment, the topological behavior semantic relationship recognition module obtains the set of topological behavior semantic relationships of vessels, the high-order semantic calculation module detects the lateral behavior of inland waterway vessels according to the topological behavior relationship order rules of the lateral behavior semantics of inland waterway vessels, the user interface outputs the semantics of the lateral behavior of inland waterway vessels, and the server is used to retrieve and visualize the lateral behavior trajectory of inland waterway vessels.

Images