Route design method based on marine climate data and historical ship trajectory

Abstract

The invention relates to the technical field of ship navigation planning, in particular to a ship route design method based on marine climate data and historical ship trajectories. Through deep reinforcement learning, automatic learning and adaptation to complex marine meteorological conditions, intelligent route optimization is realized, through continuous learning and strategy updating, the route is dynamically adjusted according to real-time data, and the efficiency and flexibility of route optimization are improved; by introducing a Monte Carlo method and environment feedback weighted expectation and combining adaptive importance sampling, an expected operation value of a future state is estimated more accurately, states which have great influence on a route adjustment result are focused more intensively through adaptive sampling distribution, and the sample utilization rate is improved, so that the risk is assessed more accurately and the route is adjusted more accurately; through environment feedback weighting expectation, the system can dynamically adjust the route according to real-time meteorological data, and different weights are given to different future states, so that the accuracy and reliability of route design are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of ship navigation planning, and in particular to a route design method based on ocean climate data and historical ship tracks. Background Art

[0002] With the continuous growth of global trade, the importance of maritime transportation has become increasingly prominent. The modern shipping industry has put forward higher requirements for route design, including higher safety, lower operating costs and stronger dynamic adaptability.

[0003] At present, traditional route design methods are mostly based on static historical data and fixed navigation rules, which are difficult to respond to complex marine meteorological changes in real time; For example, when encountering sudden severe weather (such as typhoons, strong winds, and big waves), the ship may not be able to adjust its route in time, thus facing navigation risks; In addition, although the existing dynamic route optimization method introduces real-time meteorological data, the sampling efficiency in high-impact state areas is low, which may lead to insufficient sampling and affect the accuracy of the optimization results. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present invention provides a route design method based on ocean climate data and historical ship trajectories, which can effectively solve the problem of inaccurate ship route design in the prior art.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: The present invention provides a route design method based on ocean climate data and historical ship trajectories, which at least includes: Establish a database of ocean route points based on nautical books and materials; Combining historical trajectory data and weather data to optimize routes, the features include: Extract historical navigation routes , determine the coordinate area corresponding to the included routing points, and define historical meteorological data , calculate the risk value of the coordinate area and determine the initial navigation path ; Introducing deep reinforcement learning to optimize the initial navigation path ,in: During the optimization process, based on the state and actions To update the expected return, the mathematical expression is: in, Indicates expected return, represents the reward of the current step, Indicated in Select the maximum Q value of the optimal action, represents the discount factor, represents any action in the action space, Indicates the next state to be transferred to after executing the current action. represents the mathematical expectation, Indicates the expected operation value; Defining the first expected operating value based on Monte Carlo method and adaptive importance sampling , introduce the weighted expectation of environmental feedback to define the second expected operation value , to achieve the update of the expected operation value.

[0006] Furthermore, the Furthermore, historical navigation paths The method to determine is: Define ship historical trajectory data ; in, Represent the longitude and latitude of the trajectory point, Indicates the timestamp, Indicates the port of departure, Indicates the destination port; Statistics from the port of departure To destination port Historical navigation routes , and calculate the historical trajectory density , then: in, Indicates the number of historical ships that used this historical navigation path. Represents the total number of routes with the same origin and destination ports, extracting historical navigation paths above the density threshold .

[0007] Further, determine the initial navigation path The method is: Get historical navigation routes The coordinate area corresponding to the included routing point determines the historical meteorological data corresponding to the coordinate area in, Indicates wind speed, Indicates wave height, represents the seawater coverage, Indicates the historical typhoon paths; Calculate risk value based on wind speed, wave height and sea cover Therefore, if the risk value If it is higher than the risk threshold, the area is considered a high-risk area and is not considered for passage; otherwise, it is considered for passage; Generate optimized initial navigation path , then: in, represents the distance change of the optimized navigation path, Indicates the new optimized route, represents the fuel consumption of the optimized route, , Represent the corresponding weight coefficients respectively, and get the optimized initial navigation path .

[0008] Furthermore, fuel consumption Determined according to the following relationship: in, Indicates the distance of the flight segment and describes the routing point arrive The spherical distance between Indicates the speed. Indicates the specific fuel consumption rate, , Respectively represent the longitude and latitude coordinates of the routing points in the optimized route, Indicates the total number of routing points.

[0009] Furthermore, deep reinforcement learning optimizes the initial navigation path The method is: Define the reinforcement learning environment: state , Represents real-time weather data; action ; Reward Function , Indicates the change in sailing distance; Updates to expected returns; Calculate the final initial navigation path , then: in, represents the optimal policy function, represents the step size factor, represents the gradient of the optimal policy function.

[0010] Furthermore, the initial expected operating value is defined based on the Monte Carlo method , as follows: in, Represents multiple sampling times of future states, Indicates The instant reward obtained from the simulation, Indicates The next state of the simulation, Indicated in The Q value of performing the optimal action at .

[0011] Further, define a first expected operating value The method is: Adaptive importance sampling is introduced, and the sampling distribution is adaptively adjusted according to real-time meteorological data, then: Define a new sampling distribution : The updated first expected operating value is obtained by adaptive sampling , then: in, , Indicates The importance weights of the subsampling, and Respectively represent the immediate reward and next state obtained for each sampling.

[0012] Furthermore, the new sampling distribution is obtained according to the following relationship: in, represents the original sampling distribution, represents the importance weight function, Represents the normalization constant.

[0013] Furthermore, the importance weight function is obtained according to the following relationship: represents the adjustment parameter, represents the distance function; Furthermore, the second expected operating value The calculation is performed according to the following relationship: in, Represents the weight of environmental feedback.

[0014] Compared with the known prior art, the technical solution provided by the present invention has the following beneficial effects: Through deep reinforcement learning, the system can automatically learn and adapt to complex marine meteorological conditions to achieve intelligent route optimization. By continuously learning and updating strategies, the system can dynamically adjust routes based on real-time data to improve the efficiency and flexibility of route optimization. By introducing the Monte Carlo method and weighted expectation of environmental feedback, combined with adaptive importance sampling, the expected operation value of future states can be estimated more accurately. Adaptive sampling distribution can be used to focus more on states that have a greater impact on route adjustment results, thereby improving sample utilization and more accurately assessing risks and adjusting routes. At the same time, through environmental feedback weighted expectations, the route can be dynamically adjusted according to real-time meteorological data, and different weights can be given to different future states, thereby improving the accuracy and reliability of route design. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 It is the overall module block diagram of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] The present invention will be further described below in conjunction with the embodiments.

[0019] Example 1 (see Figure 1 ): The route design method based on ocean climate data and historical ship trajectories includes the following steps: S1. Establish a global port geographic information database, including: Building a global port collection , where each port Including latitude, longitude, port throughput, number of berths, etc., to determine the historical navigation paths between ports , forming a port path database .

[0020] S2. Build a database of ocean route points based on nautical books and materials, including: Extracting route points from navigation books , routing point Refers to, for example, recommended navigation points (recommended navigation paths marked on nautical charts), sheltered areas (typhoon or severe weather shelters), navigation mark locations (key lighthouses, buoys, lane separation points, etc.), and each route point Includes latitude and longitude, type (recommended waterway, sheltered area, navigation mark, etc.); Based on routing point Building a routing point database , determine the path between adjacent routing points and calculate the distance, combined with the port path database A complete initial navigation network is formed, which facilitates subsequent route optimization.

[0021] S3. Optimize routes based on historical trajectory data and weather data to ensure route safety and efficiency, including: Define ship historical trajectory data ; in, Represent the longitude and latitude of the trajectory point, Indicates the timestamp, Indicates the port of departure, Indicates the destination port; Statistics from the port of departure To destination port Historical navigation routes , and calculate the historical trajectory density , then: in, Indicates the number of historical ships that used this historical navigation path. Represents the total number of routes with the same origin and destination ports. Therefore, a density threshold is set to extract historical navigation paths above the density threshold. ; Further, obtain the historical navigation path The coordinate area (longitude and latitude) corresponding to the included routing points, determine the historical meteorological data corresponding to the coordinate area in, Indicates wind speed, Indicates wave height, represents the seawater coverage, Indicates the historical typhoon paths; By wind speed , high waves and seawater coverage Weighted sum to get risk value To measure the navigation risk of a certain area coordinate, thus, setting the risk threshold, if the risk value If it is higher than the risk threshold, the area is considered a high-risk area and is not considered for passage; otherwise, it is considered for passage; Secondly, it should be noted that if the coordinate area is affected by the historical typhoon path, the coordinate area will also be not considered passable.

[0022] Thus, the optimized initial navigation path can be generated , then: in, represents the distance change of the optimized navigation path, Indicates the new optimized route (historical navigation path The route is regenerated after bypassing the high-risk area). Indicates the fuel consumption of the optimized route (determined by distance, speed and fuel consumption rate, which will not be elaborated here). , Represent the corresponding weight coefficients respectively, thus obtaining the optimized initial navigation path , improving the safety and efficiency of ship navigation.

[0023] Among them, for fuel consumption Determined according to the following relationship: in, Indicates the distance of the flight segment (calculated according to the Haversine formula) and describes the routing point arrive The spherical distance between Indicates the speed. Indicates the specific fuel consumption rate, , Respectively represent the longitude and latitude coordinates of the routing points in the optimized route, Indicates the total number of routing points.

[0024] Furthermore, in order to ensure that the initial navigation path Able to dynamically adapt to real-time weather changes and introduce deep reinforcement learning to optimize the initial navigation path ,include: Define the reinforcement learning environment: state , so that the initial navigation path after optimization and real-time weather data (including wind speed, wave height and sea cover); action , allowing the speed and heading to be adjusted at each time step; Reward Function ; in, Indicates the change in sailing distance; Update the Q value, then: in, (Q function) means that in the state Next, select Action The expected return obtained, evaluating the pros and cons of each action, represents the reward of the current step, Indicated in Select the maximum Q value of the optimal action, represents the discount factor, represents any action in the action space, Indicates the next state to be transferred to after executing the current action. It represents mathematical expectation, which describes the calculation of the weighted average of all possible environmental feedback in a certain state. It evaluates the reward under the current state and action by combining the maximum Q value of the next state. Finally, it estimates the maximum total reward that can be obtained in the whole process through expectation, based on real-time meteorological data. Dynamically adjust the initial navigation path , Indicates the expected operation value.

[0025] Furthermore, in order to improve the initial navigation path In order to improve the accuracy of the navigation system, the Monte Carlo method and environmental feedback weighted expectation are introduced to comprehensively consider the meteorological factors in the complex dynamic environment, realize the update and optimization of the expected operation value, and ensure the initial navigation path. More adaptable and robust, then: in, represents the first expected operating value, represents the second expected operating value, , They represent the weight coefficients of model fusion respectively, and their sum is 1; According to the above, the expected operating value for adaptive sampling is The method is: in, represents the initial expected operating value obtained based on the Monte Carlo method, Represents multiple sampling times of future states, Indicates The instant reward obtained from the simulation, Indicates The next state of the simulation, Indicated in The Q value of the optimal action is then calculated more accurately by sampling multiple future states and rewards through the Monte Carlo method. , using the Monte Carlo method to solve the uncertainty of future states, so that the expected operation value More accurate calculations; It should be noted that in the above method, the Monte Carlo method may have the problem of insufficient efficiency in future state sampling, especially in the high-impact state area. Therefore, the introduction of adaptive importance sampling makes the sampling more concentrated on the states that have a greater impact on the route adjustment results. The sampling distribution will be adaptively adjusted according to the real-time meteorological data to improve the sample utilization rate. Then: Define a new sampling distribution : in, represents the original sampling distribution (i.e., the distribution based on historical data), Represents the importance weight function, which is used to measure the current real-time meteorological data With status Corresponding expected weather data The deviation between represents the normalization constant, ensuring that the new sampling distribution is normalized; In the above, for the importance weight function According to the following relationship: Represents the adjustment parameter to control the sampling bias. Represents the distance function, which is used to measure the current real-time meteorological data and status The deviation between the corresponding expected meteorological data reflects the similarity between the two meteorological data sets. Common distance metrics can be Euclidean distance, Manhattan distance or other indicators suitable for describing the difference in meteorological parameters. and If the difference is large, the importance weight function It will be relatively small, otherwise it will be large, which helps the algorithm pay more attention to the future state that better matches the current real-time conditions. Adaptive importance sampling can find a balance between diversity and targeting, reduce the estimation error caused by insufficient sampling, improve the effectiveness of sampling and the accuracy of expected value calculation, and thus make the Q value update more stable; Then, the first expected operating value obtained by adaptive sampling is , then: in, , Indicates The importance weights of the subsampling, and Represent the immediate reward and next state obtained by each sampling respectively. These sampling data are combined with adaptive importance sampling to more accurately estimate the original expected operation value. .

[0026] Based on the above method, considering the importance of meteorological data to route optimization, the weighted expectation of environmental feedback is performed to assign different weights to different future states and emphasize the impact of meteorological conditions on route decisions. Then, we have: in, represents the weight of environmental feedback; Therefore, the expected operation value of the future state is calculated through the Monte Carlo method to avoid overfitting problems, and the expectation is weighted by environmental feedback, and the route is dynamically adjusted according to real-time meteorological data to enhance adaptability; From this, the final initial navigation path is calculated , then: in, Represents the optimal policy function (the optimal policy function learned by deep reinforcement learning based on environmental feedback and real-time meteorological data), and obtains the current adjusted initial navigation path based on real-time meteorological data , represents the step size factor, which controls the dynamic degree of optimization. Represents the gradient of the optimal strategy function, describing the real-time meteorological data How to further adjust the route? The role of the gradient is to provide optimization direction. The route is adjusted according to the gradient to avoid adverse weather effects or improve route efficiency. Therefore, by introducing improvements such as adaptive importance sampling and environmental feedback weighted expectation, the expected operation value of future rewards can be estimated more accurately. This improvement will directly affect the update process of deep reinforcement learning, making the optimal policy function and Update more accurately, so that the final generated initial navigation path It has higher adaptability, robustness and safety when facing complex and dynamic marine meteorological conditions.

[0027] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. Route design method based on ocean climate data and historical ship trajectories, including: Establish a database of ocean route points based on nautical books and materials; Combining historical trajectory data and weather data to optimize routes, the features include: Extract historical navigation routes , determine the coordinate area corresponding to the included routing points, and define historical meteorological data , calculate the risk value of the coordinate area and determine the initial navigation path ; Introducing deep reinforcement learning to optimize the initial navigation path , determine the initial navigation path ,in: During the optimization process, based on the state and actions To update the expected return, the mathematical expression is: ； in, Indicates expected return, represents the reward of the current step, Indicated in Select the maximum Q value of the optimal action, represents the discount factor, represents any action in the action space, Indicates the next state to be transferred to after executing the current action. represents the mathematical expectation, Indicates the expected operation value; Defining the first expected operating value based on Monte Carlo method and adaptive importance sampling , introduce the weighted expectation of environmental feedback to define the second expected operation value , to achieve the update of the expected operation value.

2. The route design method based on ocean climate data and ship historical trajectory according to claim 1 is characterized in that: The historical navigation path The method for determining is: Define ship historical trajectory data ; in, Represent the longitude and latitude of the trajectory point, Indicates the timestamp, Indicates the port of departure, Indicates the destination port; Statistics from the port of departure To destination port Historical navigation routes , and calculate the historical trajectory density , then: ； in, Indicates the number of historical ships that used this historical navigation path. Represents the total number of routes with the same origin and destination ports, extracting historical navigation paths above the density threshold .

3. The route design method based on ocean climate data and ship historical trajectory according to claim 2 is characterized in that: Determine the initial navigation path The method is: Get historical navigation routes The coordinate area corresponding to the included routing point determines the historical meteorological data corresponding to the coordinate area ; in, Indicates wind speed, Indicates wave height, represents the seawater coverage, Indicates the historical typhoon paths; Calculate risk value based on wind speed, wave height and sea cover Therefore, if the risk value If it is higher than the risk threshold, the area is considered a high-risk area and is not considered for passage; otherwise, it is considered for passage; Generate optimized initial navigation path , then: ； in, represents the distance change of the optimized navigation path, Indicates the new optimized route, represents the fuel consumption of the optimized route, , Represent the corresponding weight coefficients respectively, and get the optimized initial navigation path .

4. The route design method based on ocean climate data and ship historical trajectory according to claim 3 is characterized in that: The fuel consumption Determined according to the following relationship: ； in, Indicates the distance of the flight segment and describes the routing point arrive The spherical distance between Indicates the speed. Indicates the specific fuel consumption rate, , Respectively represent the longitude and latitude coordinates of the routing points in the optimized route, Indicates the total number of routing points.

5. The route design method based on ocean climate data and ship historical trajectory according to claim 3 is characterized in that: The deep reinforcement learning optimizes the initial navigation path The method is: Define the reinforcement learning environment: state , Represents real-time weather data; action ; Reward Function , Indicates the change in sailing distance; Updates to expected returns; Calculate the final initial navigation path , then: ； in, represents the optimal policy function, represents the step size factor, represents the gradient of the optimal policy function.

6. The route design method based on ocean climate data and ship historical trajectory according to claim 5 is characterized in that: Defining the initial expected operating value based on the Monte Carlo method , as follows: ； in, Represents multiple sampling times of future states, Indicates The instant reward obtained from the simulation, Indicates The next state of the simulation, Indicated in The Q value of performing the optimal action at .

7. The route design method based on ocean climate data and ship historical trajectory according to claim 6 is characterized in that: Define the first expected operating value The method is: Adaptive importance sampling is introduced, and the sampling distribution is adaptively adjusted according to real-time meteorological data, then: Define a new sampling distribution : The updated first expected operating value is obtained by adaptive sampling , then: ； in, , Indicates The importance weights of the subsampling, and Respectively represent the immediate reward and next state obtained for each sampling.

8. The route design method based on ocean climate data and ship historical trajectory according to claim 7 is characterized in that: The new sampling distribution is obtained according to the following relationship: ； in, represents the original sampling distribution, represents the importance weight function, Represents the normalization constant.

9. The route design method based on ocean climate data and ship historical trajectory according to claim 8, characterized in that: The importance weight function is obtained according to the following relationship: ； represents the adjustment parameter, Represents the distance function.

10. The route design method based on ocean climate data and ship historical trajectory according to claim 6, characterized in that: The second expected operating value The calculation is performed according to the following relationship: ； in, Represents the weight of environmental feedback.

Images