Ship situation awareness enhancement method, electronic device and storage medium

Abstract

The present application discloses a method for enhancing ship situational awareness, an electronic device and a storage medium, and relates to the field of ship technology. The present application models the dynamic interaction between multiple ship attributes, effectively integrates multiple related factors, identifies complex behavioral interaction patterns between ships, and classifies and aggregates the characteristics of factors with different impact levels, and finally introduces a multi-source fusion module to embed and fuse the dynamic interaction characteristics between different ships, thereby effectively enhancing the situational awareness prediction capability of ships.

Description

Technical Field

[0001] The present application relates to the field of ship technology, and in particular to a method for enhancing ship situational awareness, an electronic device and a storage medium. Background Art

[0002] With the development of the global shipping industry, the safety of ships sailing at sea has received more and more attention. In the complex and ever-changing marine environment, ships face various potential risks and challenges. Although traditional ship navigation and monitoring systems can provide basic navigation information and environmental perception capabilities, they are often unsatisfactory when dealing with complex navigation situations. Traditional ship navigation systems mainly rely on sensors such as radar and automatic identification systems (AIS) to obtain navigation environment information around ships. With the increase in the number of ships operating at sea and the increase in the complexity of maritime traffic, various ships at sea have their own navigation behaviors. In addition to ensuring that their own navigation operations and itineraries are correct enough, ships must also ensure that their navigation is not affected by interference from other ships. Traditional methods mainly rely on the experience and observation of crew members, which are highly subjective and uncertain.

[0003] Artificial intelligence technology plays an increasingly important role in intelligent maritime navigation in the modern maritime industry. However, the accurate analysis of the behavior of ships in complex waters still needs to be improved, and the accurate prediction of ship status is still an important problem that needs to be solved in the shipping field.

[0004] There is currently no effective solution to the problem of poor ship situational awareness in related technologies. Summary of the invention

[0005] In this embodiment, a method for enhancing ship situation awareness, an electronic device and a storage medium are provided to solve the problem of poor ship situation awareness existing in the related art.

[0006] In a first aspect, the present invention provides a method for enhancing ship situation awareness, comprising:

[0007] Obtain the specifications and movement data of each ship, and use different ships as different nodes in the graph network;

[0008] Determining interrelationship characteristics between nodes of the first ship graph network based on the trajectory of the time before the current position of the ship;

[0009] Determine the mutual relationship characteristics between the nodes of the second ship graph network based on the positions of any two ships at any time;

[0010] Determine the relationship characteristics between the nodes of the third ship graph network based on the time between the ships to the nearest meeting point;

[0011] Determine the mutual relationship characteristics between the nodes of the fourth ship graph network based on the distances between the ships to the nearest encounter point;

[0012] Determine the interrelationship characteristics between the nodes of the fifth ship graph network based on different specification attributes of the ship;

[0013] Based on the graph convolution module and the attention module, the interrelationship characteristics between the second ship graph network nodes, the interrelationship characteristics between the third ship graph network nodes, and the interrelationship characteristics between the fourth ship graph network nodes are subjected to cohesive fusion processing to obtain a first impact level factor characteristic graph of ships;

[0014] Based on the graph convolution module, the mutual relationship characteristics between the first ship graph network nodes and the mutual relationship characteristics between the fifth ship graph network nodes are processed respectively to obtain the ship second impact level factor characteristic graph and the ship third impact level factor characteristic graph respectively;

[0015] Based on the multi-source fusion module, the first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship and the third impact level factor characteristic graph of the ship are fused to predict the ship situation.

[0016] In some of the embodiments, determining the interrelationship characteristics between the nodes of the first ship graph network based on the trajectory of the ship before the current position of the ship includes:

[0017] Step 1: Obtain a set of trajectories of the time before the current position in the ship's navigation area;

[0018] Step 2: traverse the trajectories in the trajectory set, determine the neighborhood with the visited current trajectory as the central axis, and if the trajectory in the trajectory set within the neighborhood of the current trajectory is not assigned to any group, assign the group label of the current trajectory to the trajectory in the trajectory set within the neighborhood of the current trajectory;

[0019] Step 3: determining whether the number of trajectories in the trajectory set within the neighborhood of the current trajectory is greater than a preset density threshold;

[0020] Step 4: If the number of trajectories in the trajectory set within the neighborhood of the current trajectory is greater than a preset density threshold, then the internal trajectories within the neighborhood of the current trajectory are visited in sequence until all the internal trajectories in the neighborhood are visited;

[0021] Step 5: If the number of trajectories in the trajectory set within the neighborhood of the current trajectory is less than a preset density threshold, determining whether the trajectory in the neighborhood set is located on one side of the neighborhood of the current trajectory;

[0022] Step 6: If the trajectory in the neighborhood set is located on one side of the neighborhood of the current trajectory, the current trajectory is used as the termination trajectory, and the internal trajectories in the neighborhood of the current trajectory are visited in sequence until all the internal trajectories in the neighborhood are visited; otherwise, the current trajectory is an interference trajectory;

[0023] Repeat steps 2 to 6 until all trajectories have group labels, remove interfering trajectories, and keep the trajectories formed by the terminal trajectory and the internal trajectory;

[0024] Step 7: Calculate the similarity between two adjacent frames in each trajectory under the group label;

[0025] Step 8: Calculate the average similarity between two adjacent frames in the same frame area according to the similarity obtained in step 8 and the total number of trajectory points, and save the average similarity in a similarity set;

[0026] Step 9: Take the average value of the two points with the smallest average similarity in the similarity set and merge them into the same point, replacing the original point in the trajectory set;

[0027] Repeat steps 7 to 9 until the number of trajectory points converges to the preset number of points;

[0028] The mutual relationship characteristics between the nodes of the first ship graph network are determined according to the average similarity in the similarity set.

[0029] In some embodiments, based on the graph convolution module and the attention module, the interrelationship characteristics between the second ship graph network nodes, the interrelationship characteristics between the third ship graph network nodes, and the interrelationship characteristics between the fourth ship graph network nodes are subjected to cohesive fusion processing to obtain a first impact level factor feature graph of ships, including:

[0030] The mutual relationship features between the second ship graph network nodes, the mutual relationship features between the third ship graph network nodes, and the mutual relationship features between the fourth ship graph network nodes are processed by a graph convolution module to obtain corresponding feature graphs;

[0031] The attention module first calculates the feature matrix of the corresponding feature map through the global pooling layer to extract the global features, then generates attention weights for the global features through the fully connected layer and the activation layer, and finally performs normalization and fusion processing to obtain the feature map of the first impact level factor of the ship.

[0032] In some of the embodiments, multiple channels in the attention module have corresponding activation values ​​generated by the activation layer, the multiple channels include deformed channels with abnormal activation value distribution and normal channels with normal activation value distribution, different quantization steps are used to quantize the corresponding activation values, and the quantization step size of the deformed channel is different from the quantization step size of the normal channel. The specific calculation formula is as follows:

[0033]

[0034] in, is the quantization step size of the normal channel, is the rounding function, is the maximum value of the activation values ​​corresponding to all normal channels, is the maximum value of the activation values ​​corresponding to all deformed channels, is the quantization step size of the deformed channel.

[0035] In some of the embodiments, the time between the ships to the nearest meeting point is calculated to determine the relationship characteristics between the nodes of the third ship graph network. The specific formula is as follows:

[0036] = ;

[0037]

[0038]

[0039] in, For time point, For ships With Ship The relative speed between represents the relative heading, , , as well as Representing ships With Ship The corresponding speed and direction, For ships The azimuth of is the time it takes for ships A and B to reach their closest meeting point, For ships With Ship The Euclidean distance between Characterize the relationship characteristics between the nodes of the third ship graph network.

[0040] In some of the embodiments, the distances between the ships to the nearest encounter point are calculated to determine the relationship characteristics between the nodes of the fourth ship graph network. The specific formula is as follows:

[0041]

[0042]

[0043] in, For time point, represents the relative heading, For ships The azimuth of For ships With Ship The Euclidean distance between them is the distance between ship a and ship b to the nearest meeting point. Characterize the relationship characteristics between the nodes of the fourth ship graph network.

[0044] In some of the embodiments, the first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship, and the third impact level factor characteristic graph of the ship are fused based on a multi-source fusion module to predict the ship situation, including:

[0045] The first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship and the third impact level factor characteristic graph of the ship are fused through a multi-source fusion module, and then the ship situation is predicted by the fusion of the CBAM module and the time series convolution module.

[0046] In a second aspect, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for enhancing ship situational awareness as described in the first aspect when executing the computer program.

[0047] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for enhancing ship situational awareness as described in the first aspect above.

[0048] Compared with the related art, the present invention provides a method for enhancing ship situation awareness, comprising:

[0049] Obtain the specifications and movement data of each ship, and use different ships as different nodes in the graph network;

[0050] Based on the trajectory of the time before the current position of the ship, the mutual relationship characteristics between the nodes of the first ship graph network are determined; based on the positions of any two ships at any time, the mutual relationship characteristics between the nodes of the second ship graph network are determined; based on the time between the ships to the nearest meeting point, the mutual relationship characteristics between the nodes of the fourth ship graph network are determined based on the distances between the ships to the nearest meeting point; based on the different specification attributes of the ships, the mutual relationship characteristics between the nodes of the second ship graph network, the mutual relationship characteristics between the nodes of the third ship graph network, and the mutual relationship characteristics between the nodes of the fourth ship graph network are cohesively fused based on the graph convolution module and the attention module to obtain the first impact level factor feature map of the ship; based on the graph convolution module, the mutual relationship characteristics between the nodes of the first ship graph network and the mutual relationship characteristics between the nodes of the fifth ship graph network are processed respectively to obtain the second impact level factor feature map of the ship and the third impact level factor feature map of the ship respectively; based on the multi-source fusion module, the first impact level factor feature map of the ship, the second impact level factor feature map of the ship, and the third impact level factor feature map of the ship are fused to predict the ship situation. This application models the dynamic interactions between multiple ship attributes, effectively integrates multiple related factors, identifies complex behavioral interaction patterns between ships, and classifies and aggregates the characteristics of factors with different impact levels. Finally, a multi-source fusion module is introduced to embed and fuse the dynamic interaction characteristics between different ships, effectively enhancing the situational awareness and prediction capabilities of ships.

[0051] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 This is a flow chart of the method for enhancing ship situational awareness of the present application;

[0054] Figure 2 Schematic diagram of different trajectories for this application;

[0055] Figure 3 It is a flow chart of the fusion processing of the multi-source fusion module, the CBAM module and the temporal convolution module;

[0056] Figure 4 This is the architecture diagram of the residual module. DETAILED DESCRIPTION

[0057] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0058] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0059] Ship situation awareness is one of the key technologies to ensure maritime navigation safety, which involves comprehensive perception and control of the ship's navigation status. By integrating multiple sensors, including lidar, automatic identification system (AIS), cameras, etc., better situation awareness can be achieved.

[0060] In order to ensure the safety of their own navigation, ships need to pay more attention to the factors that require timely grasp of the navigation conditions of other ships and make predictions before abnormal situations occur. This requires accurate and rapid response. As artificial intelligence technology plays an increasingly important role in ship situation awareness, the ship situation can be perceived by real-time analysis of other ships' data.

[0061] The present invention provides a method for enhancing ship situation awareness. Figure 1 is a flow chart of the method for enhancing ship situation awareness of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0062] Obtain the specifications and movement data of each ship, and use different ships as different nodes in the graph network;

[0063] In some embodiments, the ship's motion data can be obtained through an automatic identification system (AIS), providing information such as the ship's position, speed, heading, and distance.

[0064] The specifications of the ship can be the length, width, etc. of the ship. Of course, more restrictions can also be added, such as the stern curvature, bow, stern and other specification data.

[0065] Different ships are used to construct different nodes of the graph network, and different graph networks are constructed based on the relationships between different nodes.

[0066] The interrelationship characteristics between the nodes of the first ship graph network are determined based on the trajectory of the ship before the current position of the ship.

[0067] Determining the interrelationship characteristics between the nodes of the first ship graph network based on the trajectory of the time before the current position of the ship, including:

[0068] Step 1: Obtain a set of trajectories of the time before the current position in the ship's navigation area;

[0069] Step 2: traverse the trajectories in the trajectory set, determine the neighborhood with the visited current trajectory as the central axis, and if the trajectory in the trajectory set within the neighborhood of the current trajectory is not assigned to any group, assign the group label of the current trajectory to the trajectory in the trajectory set within the neighborhood of the current trajectory;

[0070] See the schematic Figure 2 Marks 2 and 3 in the figure represent the trajectories in the trajectory set within the neighborhood corresponding to the current trajectory, and the dotted box represents the neighborhood range corresponding to the current trajectory.

[0071] Step 3: determining whether the number of trajectories in the trajectory set within the neighborhood of the current trajectory is greater than a preset density threshold;

[0072] Step 4: If the number of trajectories in the trajectory set within the neighborhood of the current trajectory is greater than or equal to a preset density threshold, the current trajectory is taken as an internal trajectory and the internal trajectories within the neighborhood of the current trajectory are visited in sequence until all the internal trajectories in the neighborhood are visited;

[0073] See the schematic Figure 2 The mark 1 in represents the inner track.

[0074] Step 5: If the number of trajectories in the trajectory set within the neighborhood of the current trajectory is less than a preset density threshold, determining whether the trajectory in the neighborhood set is located on one side of the neighborhood of the current trajectory;

[0075] Step 6: If the trajectory in the neighborhood set is located on one side of the neighborhood of the current trajectory, the current trajectory is used as the termination trajectory, and the internal trajectories in the neighborhood of the current trajectory are visited in sequence until all the internal trajectories in the neighborhood are visited; otherwise, the current trajectory is an interference trajectory;

[0076] See the schematic Figure 2 The mark 4 in represents the termination trajectory, and the mark 5 represents the interference trajectory;

[0077] Repeat steps 2 to 6 until all trajectories have group labels, remove interfering trajectories, and keep the trajectories formed by the terminal trajectory and the internal trajectory;

[0078] See the schematic Figure 2 The exemplary example includes tracks under two sets of labels a and b, and of course, it can also include tracks under more sets of labels.

[0079] Step 7: Calculate the similarity between two adjacent frames in each trajectory under the group label ;

[0080] Optional, similarity Measured using the Euclidean metric.

[0081] Step 8: Calculate the average similarity between two adjacent frames in the same frame area based on the similarity obtained in step 8 and the total number of trajectory points , and save the average similarity in the similarity set;

[0082] in, , is the total number of trajectory points.

[0083] Step 9: Take the average value of the two points with the smallest average similarity in the similarity set and merge them into the same point, replacing the original point in the trajectory set;

[0084] Step 10, repeating steps 7 to 9 until the number of trajectory points converges to a preset number of points;

[0085] Optionally, the preset number of points is 5, and can also be set to other values ​​according to actual needs.

[0086] The mutual relationship characteristics between the nodes of the first ship graph network are determined according to the average similarity in the similarity set.

[0087] When using the Automatic Identification System (AIS) to collect ship data, due to the different collection frequencies and time points of the AIS system, the data points contained in the same length of the track are different, and the length of the track is also different. The above method can overcome the problem of track length and cluster the track more accurately. At the same time, due to the differences in navigation speed and collection time, the segmentation of the frame in the time domain cannot guarantee the uniformity of the track points. Therefore, the method of segmenting the frame in the spatial domain is used to ensure the uniformity of the points of the subsequent multi-track feature extraction data.

[0088] The similarity of the trajectories within the group and the average similarity of all trajectories between two adjacent trajectory points (i.e., between different frames) are used as the criteria for combining trajectory points and determining the characteristics of the mutual relationships between the nodes of the first ship graph network. This ensures that during the extraction of trajectory features, the trajectory feature extraction is no longer affected by the dense area of ​​trajectory points. The dense area of ​​the trajectories within the trajectory group can be comprehensively measured, and the trajectory points turning to the dense area can be better retained.

[0089] The mutual relationship characteristics between the nodes of the second ship graph network are determined based on the positions of any two ships at any time.

[0090] Considering the factor that the interaction between adjacent ships is stronger than that between distant ships, the mutual relationship characteristics between nodes of the second ship graph network are determined by calculating the Euclidean distance between two ships.

[0091] Specifically, the Euclidean distance between the ships can be calculated using the longitude and latitude parameters of each ship.

[0092] The mutual relationship characteristics between the nodes of the third ship graph network are determined based on the time between the ships to the nearest meeting point.

[0093] In some of the embodiments, the time between the ships to the nearest meeting point is calculated to determine the relationship characteristics between the nodes of the third ship graph network. The specific formula is as follows:

[0094] = ;

[0095]

[0096]

[0097] in, For time point, For ships With Ship The relative speed between represents the relative heading, , , as well as Representing ships With Ship The corresponding speed and direction, For ships The azimuth is the time it takes for ships A and B to reach their closest encounter point. For ships With Ship The Euclidean distance between Characterize the relationship characteristics between the nodes of the third ship graph network.

[0098] The mutual relationship characteristics between the nodes of the fourth ship graph network are determined based on the distances between the ships to the nearest meeting points.

[0099] In some of the embodiments, the distances between the ships to the nearest encounter point are calculated to determine the relationship characteristics between the nodes of the fourth ship graph network. The specific formula is as follows:

[0100]

[0101]

[0102] in, For time point, represents the relative heading, For ships The azimuth of For ships With Ship The Euclidean distance between is the distance between ship a and ship b to the nearest encounter point, Characterize the relationship characteristics between the nodes of the fourth ship graph network.

[0103] The relationship characteristics between the nodes of the fifth ship graph network are determined based on the different specification attributes of the ship.

[0104] In some of the embodiments, the relationship characteristics between the nodes of the fifth ship graph network are determined based on different specification attributes of the ship, and the specific formula is as follows:

[0105]

[0106] in, For time point, , Representing ships The length and width corresponding to the ship 𝑏, Characterize the interrelationship characteristics between the nodes of the fifth ship graph network.

[0107] Based on the graph convolution module and the attention module, the relationship characteristics between the second ship graph network nodes, the relationship characteristics between the third ship graph network nodes and the relationship characteristics between the fourth ship graph network nodes are cohesively fused to obtain the first impact level factor characteristic graph of ships.

[0108] In some embodiments, the graph convolution module can perform two-dimensional convolution operations on the interrelationship features between the second ship graph network nodes, the interrelationship features between the third ship graph network nodes, and the interrelationship features between the fourth ship graph network nodes. The attention module further aggregates features and identifies more critical information for subsequent fusion, and uses an adaptive global pooling layer to calculate the feature matrix to extract global features. The calculated global features generate attention weights through the fully connected layer and the activation layer, and then the softmax function is used to normalize the attention values ​​and obtain the final weights. Then the final weight is applied to fuse the corresponding feature matrix to obtain the feature map of the first impact level factor of the ship.

[0109] Based on the graph convolution module, the mutual relationship characteristics between the first ship graph network nodes and the mutual relationship characteristics between the fifth ship graph network nodes are processed respectively to obtain the ship second impact level factor characteristic graph and the ship third impact level factor characteristic graph respectively;

[0110] Based on the multi-source fusion module, the first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship and the third impact level factor characteristic graph of the ship are fused to predict the ship situation.

[0111] Among them, the specific architecture of the multi-source fusion module is not given as an example here, and multiple modal fusion can be realized. The multi-source fusion module is used to integrate the corresponding ship first impact level factor characteristic map, the ship second impact level factor characteristic map and the ship third impact level factor characteristic map under different modes, so as to capture deeper spatial dependencies and help improve prediction accuracy.

[0112] This application models the dynamic interactions between multiple ship attributes, effectively integrates multiple related factors, identifies complex behavioral interaction patterns between ships, and classifies and aggregates the characteristics of factors with different impact levels. Finally, a multi-source fusion module is introduced to embed and fuse the dynamic interaction characteristics between different ships, effectively enhancing the situational awareness and prediction capabilities of ships.

[0113] In some embodiments, based on the graph convolution module and the attention module, the interrelationship characteristics between the second ship graph network nodes, the interrelationship characteristics between the third ship graph network nodes, and the interrelationship characteristics between the fourth ship graph network nodes are subjected to cohesive fusion processing to obtain a first impact level factor feature graph of ships, including:

[0114] The mutual relationship features between the second ship graph network nodes, the mutual relationship features between the third ship graph network nodes, and the mutual relationship features between the fourth ship graph network nodes are processed by a graph convolution module to obtain corresponding feature graphs;

[0115] The attention module first calculates the feature matrix of the corresponding feature map through the global pooling layer to extract the global features, then generates attention weights for the global features through the fully connected layer and the activation layer, and finally performs normalization and fusion processing to obtain the feature map of the first impact level factor of the ship.

[0116] In some of the embodiments, multiple channels in the attention module have corresponding activation values ​​generated by the activation layer, the multiple channels include deformed channels with abnormal activation value distribution and normal channels with normal activation value distribution, different quantization steps are used to quantize the corresponding activation values, and the quantization step size of the deformed channel is different from the quantization step size of the normal channel. The specific calculation formula is as follows:

[0117]

[0118] in, is the quantization step size of the normal channel, is the rounding function, is the maximum value of the activation values ​​corresponding to all normal channels, is the maximum value of the activation values ​​corresponding to all deformed channels, is the quantization step size of the deformed channel.

[0119] In a multi-channel processing environment, some channels may have abnormal distribution of activation values, and these values ​​may stably tend to be extremely large or extremely small. If the same quantization step size is applied to all channels, a larger step size must be selected to accommodate extreme values. Although this can contain abnormal values, it will reduce the quantization accuracy of normal distribution channels. According to the specific characteristics of each channel, the quantization step size is dynamically adjusted to retain the uniqueness of abnormal distribution while ensuring the high accuracy of normal distribution channels. This method not only improves the ability to capture abnormal situations, but also maintains the quality of the overall data, achieving more refined and efficient quantization processing. In this way, the loss of quantization accuracy between different channels is effectively reduced, and the performance and reliability of the system are enhanced.

[0120] In some of the embodiments, the first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship, and the third impact level factor characteristic graph of the ship are fused based on a multi-source fusion module to predict the ship situation, including:

[0121] The first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship and the third impact level factor characteristic graph of the ship are fused through a multi-source fusion module, and then the ship situation is predicted by the fusion of the CBAM module and the time series convolution module.

[0122] By introducing a multi-source fusion module, it is used to integrate the corresponding ship first impact level factor characteristic diagram, the ship second impact level factor characteristic diagram and the ship third impact level factor characteristic diagram under different modes. The obtained fusion feature matrix is ​​used as the input of the subsequent ship situation prediction module, which helps to improve the prediction accuracy.

[0123] The multi-source fusion module is used to perform deep fusion processing on the first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship, and the third impact level factor characteristic graph of the ship. Then, the fusion processing flow of the CBAM module and the time series convolution module is as follows: Figure 3 As shown in the figure, multiple residual modules are connected in series to form a time series convolution module, which can capture and convey the relationship within the time series. Through multiplication and fusion calculation with the CBAM module, the ship situation can be predicted more accurately.

[0124] The specific architecture of the residual module is as follows Figure 4 As shown in the figure, the residual module is composed of two repeated modules and a two-dimensional convolution module. Each module in the two repeated modules includes an extended causal convolution module, a batch normalization module, an activation layer, and a Dropout layer. The introduction of adjustable extended parameters in the extended causal convolution module can increase the receptive field range within the convolution kernel, expand the effective area of ​​the kernel function, and help obtain a wider range of spatial information. The introduction of the batch normalization module can avoid overfitting caused by parameter distribution offset. This design can better coordinate the functions of each module, so that the ship's situation can be predicted more efficiently and reliably.

[0125] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0126] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0127] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0128] Optionally, in one embodiment, the processor may be configured to perform the following steps through a computer program:

[0129] Obtain the specifications and movement data of each ship, and use different ships as different nodes in the graph network;

[0130] Determining interrelationship characteristics between nodes of the first ship graph network based on the trajectory of the time before the current position of the ship;

[0131] Determine the mutual relationship characteristics between the nodes of the second ship graph network based on the positions of any two ships at any time;

[0132] Determine the relationship characteristics between the nodes of the third ship graph network based on the time between the ships to the nearest meeting point;

[0133] Determine the mutual relationship characteristics between the nodes of the fourth ship graph network based on the distances between the ships to the nearest encounter point;

[0134] Determine the interrelationship characteristics between the nodes of the fifth ship graph network based on different specification attributes of the ship;

[0135] Based on the graph convolution module and the attention module, the interrelationship characteristics between the second ship graph network nodes, the interrelationship characteristics between the third ship graph network nodes, and the interrelationship characteristics between the fourth ship graph network nodes are subjected to cohesive fusion processing to obtain a first impact level factor characteristic graph of ships;

[0136] Based on the graph convolution module, the mutual relationship characteristics between the first ship graph network nodes and the mutual relationship characteristics between the fifth ship graph network nodes are processed respectively to obtain the ship second impact level factor characteristic graph and the ship third impact level factor characteristic graph respectively;

[0137] Based on the multi-source fusion module, the first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship and the third impact level factor characteristic graph of the ship are fused to predict the ship situation.

[0138] It should be noted that the specific examples of the electronic device can refer to the examples described in the embodiments and optional implementations of the above method, and will not be repeated in this embodiment.

[0139] In addition, in combination with the ship situation awareness enhancement method provided in the present invention, a storage medium can also be provided in the present invention to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any of the ship situation awareness enhancement methods in the above embodiments is implemented.

[0140] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0141] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0142] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

Claims

1. A method for enhancing ship situational awareness, characterized in that: include: Obtain the specifications and movement data of each ship, and use different ships as different nodes in the graph network; Determining interrelationship characteristics between nodes of the first ship graph network based on the trajectory of the time before the current position of the ship; Determine the mutual relationship characteristics between the nodes of the second ship graph network based on the positions of any two ships at any time; Determine the relationship characteristics between the nodes of the third ship graph network based on the time between the ships to the nearest meeting point; Determine the mutual relationship characteristics between the nodes of the fourth ship graph network based on the distances between the ships to the nearest encounter point; Determine the interrelationship characteristics between the nodes of the fifth ship graph network based on different specification attributes of the ship; Based on the graph convolution module and the attention module, the interrelationship characteristics between the second ship graph network nodes, the interrelationship characteristics between the third ship graph network nodes, and the interrelationship characteristics between the fourth ship graph network nodes are subjected to cohesive fusion processing to obtain a first impact level factor characteristic graph of ships; Based on the graph convolution module, the mutual relationship characteristics between the first ship graph network nodes and the mutual relationship characteristics between the fifth ship graph network nodes are processed respectively to obtain the ship second impact level factor characteristic graph and the ship third impact level factor characteristic graph respectively; Based on the multi-source fusion module, the first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship and the third impact level factor characteristic graph of the ship are fused to predict the ship situation.

2. The method for enhancing ship situational awareness according to claim 1, characterized in that: Determining the interrelationship characteristics between the nodes of the first ship graph network based on the trajectory of the time before the current position of the ship, including: Step 1: Obtain a set of trajectories of the time before the current position in the ship's navigation area; Step 2: traverse the trajectories in the trajectory set, determine the neighborhood with the visited current trajectory as the central axis, and if the trajectory in the trajectory set within the neighborhood of the current trajectory is not assigned to any group, assign the group label of the current trajectory to the trajectory in the trajectory set within the neighborhood of the current trajectory; Step 3: determining whether the number of trajectories in the trajectory set within the neighborhood of the current trajectory is greater than a preset density threshold; Step 4: If the number of trajectories in the trajectory set within the neighborhood of the current trajectory is greater than a preset density threshold, then the internal trajectories within the neighborhood of the current trajectory are visited in sequence until all the internal trajectories in the neighborhood are visited; Step 5: If the number of trajectories in the trajectory set within the neighborhood of the current trajectory is less than a preset density threshold, determining whether the trajectory in the neighborhood set is located on one side of the neighborhood of the current trajectory; Step 6: If the trajectory in the neighborhood set is located on one side of the neighborhood of the current trajectory, the current trajectory is used as the termination trajectory, and the internal trajectories in the neighborhood of the current trajectory are visited in sequence until all the internal trajectories in the neighborhood are visited; otherwise, the current trajectory is an interference trajectory; Repeat steps 2 to 6 until all trajectories have group labels, remove interfering trajectories, and keep the trajectories formed by the terminal trajectory and the internal trajectory; Step 7: Calculate the similarity between two adjacent frames in each trajectory under the group label; Step 8: Calculate the average similarity between two adjacent frames in the same frame area according to the similarity obtained in step 8 and the total number of trajectory points, and save the average similarity in a similarity set; Step 9: Take the average value of the two points with the smallest average similarity in the similarity set and merge them into the same point, replacing the original point in the trajectory set; Repeat steps 7 to 9 until the number of trajectory points converges to the preset number of points; The mutual relationship characteristics between the nodes of the first ship graph network are determined according to the average similarity in the similarity set.

3. The method for enhancing ship situational awareness according to claim 1, characterized in that: Based on the graph convolution module and the attention module, the interrelationship features between the second ship graph network nodes, the interrelationship features between the third ship graph network nodes, and the interrelationship features between the fourth ship graph network nodes are subjected to cohesive fusion processing to obtain a first impact level factor feature graph of ships, including: The mutual relationship features between the second ship graph network nodes, the mutual relationship features between the third ship graph network nodes, and the mutual relationship features between the fourth ship graph network nodes are processed by a graph convolution module to obtain corresponding feature graphs; The attention module first calculates the feature matrix of the corresponding feature map through the global pooling layer to extract the global features, then generates attention weights for the global features through the fully connected layer and the activation layer, and finally performs normalization and fusion processing to obtain the feature map of the first impact level factor of the ship.

4. The method for enhancing ship situational awareness according to claim 3, characterized in that: The multiple channels in the attention module have corresponding activation values ​​generated by the activation layer, and the multiple channels include deformed channels with abnormal activation value distribution and normal channels with normal activation value distribution. Different quantization steps are used to quantize the corresponding activation values. The quantization step of the deformed channel is different from that of the normal channel. The specific calculation formula is as follows: in, is the quantization step size of the normal channel, is the rounding function, is the maximum value of the activation values ​​corresponding to all normal channels, is the maximum value of the activation values ​​corresponding to all deformed channels, is the quantization step size of the deformed channel.

5. The method for enhancing ship situational awareness according to claim 1, characterized in that: The time between the ships to the nearest meeting point is calculated to determine the relationship characteristics between the nodes of the third ship graph network. The specific formula is as follows: = ； in, For time point, For ships and vessels The relative speed between represents the relative heading, , , as well as Representing ships With Ship The corresponding speed and direction, For ships The azimuth of is the time it takes for ships A and B to reach their closest meeting point, For ships With Ship The Euclidean distance between Characterize the relationship characteristics between the nodes of the third ship graph network.

6. The method for enhancing ship situational awareness according to claim 5, characterized in that: The distances between the ships and the nearest encounter points are calculated to determine the relationship characteristics between the nodes of the fourth ship graph network. The specific formula is as follows: in, For time point, represents the relative heading, For ships The azimuth of For ships With Ship The Euclidean distance between is the distance between ship a and ship b to the nearest encounter point, Characterize the relationship characteristics between the nodes of the fourth ship graph network.

7. The method for enhancing ship situational awareness according to claim 1, characterized in that: Based on the multi-source fusion module, the first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship, and the third impact level factor characteristic graph of the ship are fused to predict the ship situation, including: The first impact level factor characteristic graph of the ship, the second impact level factor characteristic graph of the ship and the third impact level factor characteristic graph of the ship are fused through a multi-source fusion module, and then the ship situation is predicted by the fusion of the CBAM module and the time series convolution module.

8. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method for enhancing ship situational awareness according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for enhancing ship situational awareness according to any one of claims 1 to 7 are implemented.

Images