VTS and unmanned ship edge sensing path planning method under communication instability

Abstract

The invention discloses a VTS and unmanned ship edge sensing path planning method under communication instability, and belongs to the technical field of unmanned ship path planning. Firstly, a track planning framework based on edge intelligent sensing is constructed, the real-time sensing capability of the unmanned ship in a complex environment is enhanced by utilizing cooperative communication and calculation among an edge server, a sensor terminal and the unmanned ship, and it is ensured that the unmanned ship can still automatically avoid obstacles and efficiently execute tasks when communication is poor or signals are interrupted. Secondly, an artificial potential field method based on depth deterministic strategy gradient algorithm optimization is provided, planning failure caused by a local minimum value in traditional route planning is effectively avoided, it is ensured that the unmanned ship can complete global optimal route planning in a complex electromagnetic environment, and the safety and efficiency of task execution are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of ship path planning, and in particular to a method for edge sensing path planning of a VTS and an unmanned ship under unstable communication. Background Art

[0002] With the rapid development of wireless communication technology and electronic equipment, the use of unmanned ships has become more and more popular, especially in the fields of ocean exploration, logistics and transportation, and military applications. However, with the emergence of a large number of frequency-using devices, the electromagnetic environment has become increasingly complex. This complexity places higher demands on the environmental perception and autonomous obstacle avoidance capabilities of unmanned ships, and new technologies are urgently needed to improve the operational safety and efficiency of unmanned ships in dynamic and uncertain environments.

[0003] Traditional trajectory planning methods often rely on centralized computing and known environmental models, and cannot effectively cope with real-time electromagnetic interference and changing environmental conditions. This makes unmanned ships prone to collisions and trajectory deviations when performing missions, thus affecting the success rate of the mission. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a VTS and unmanned ship edge perception path planning method under unstable communication. The present invention realizes efficient information transmission and real-time calculation through the collaborative work of the edge server and the sensor terminal. At the same time, the artificial potential field method optimized by the deep deterministic policy gradient algorithm is combined to avoid the common local minimum trap in track planning, thereby enhancing the track planning efficiency and intelligence level of the unmanned ship, and improving the perception ability and autonomous obstacle avoidance ability of the unmanned ship in a complex electromagnetic environment.

[0005] The technical means adopted by the present invention are as follows:

[0006] A method for edge sensing path planning of a VTS and an unmanned ship under unstable communication conditions comprises the following steps:

[0007] Step 1: Build an unmanned ship trajectory planning framework based on edge intelligent perception, which includes a base station, an unmanned ship and sensor equipment; the base station is equipped with an edge server; the sensor terminal is used to collect spectrum data in three-dimensional space and upload it to the edge server in real time, and the edge server is used to build a spectrum map based on the spectrum data; the unmanned ship maintains a communication connection with the base station during the process of traveling from the starting point to the target point, and receives the spectrum map sent by the base station;

[0008] Step 2: The unmanned ship obtains environmental perception data according to the spectrum map, and processes the environmental perception data using an artificial potential field method optimized with a deep deterministic policy gradient algorithm, thereby realizing the unmanned ship track planning.

[0009] Furthermore, the spectrum map received by the unmanned ship includes a global spectrum map and a local spectrum map, and the accuracy of the global spectrum map is lower than that of the local spectrum map.

[0010] Furthermore, the sensor equipment includes at least one of an unmanned ship, a VTS service center and a ship-borne receiver.

[0011] Furthermore, the edge server constructs a spectrum map, comprising the following steps:

[0012] Collecting spectrum data in three-dimensional space based on sensor equipment;

[0013] The Kriging interpolation algorithm is used to interpolate the unknown area based on the collected spectrum data to construct a spectrum map.

[0014] Furthermore, the unknown area is interpolated using the Kriging interpolation algorithm based on the collected spectrum data, including:

[0015] Based on the collected spectrum data, the Kriging interpolation algorithm is used to interpolate the unknown area to construct a high-precision spectrum map;

[0016] Before the unmanned ship sets off, low-precision spectrum data interpolation is performed based on the collected spectrum data to preliminarily determine the location of electromagnetic interference and spatial obstacles, and generate a compressed global spectrum map; when sudden interference or dynamic obstacles occur, the edge server distributes high-precision local spectrum maps to the unmanned ship in real time.

[0017] Furthermore, the data transmission rate between the base station and the unmanned ship is:

[0018]

[0019] Where N is the number of interference sources, W is the channel bandwidth, and Pow s is the transmission power of the base station, Pow i is the transmission power of the interference, σ is the Gaussian white noise power in the environment, h u,BS and h u,i are the channel gains between the unmanned ship and the base station, and between the unmanned ship and the interference source, respectively, and:

[0020]

[0021] Among them, g 0 is the received power at a reference distance of 1m with a transmission power of 1W, d(p,p BS ) and d(p,p i ) represent the Euclidean distances from the unmanned ship to the base station and the interference source respectively.

[0022] Furthermore, an artificial potential field method combined with deep deterministic policy gradient algorithm optimization is used to process environmental perception data, including:

[0023] Step 2.1: Assume that the speed of the unmanned ship is V and it sails from the current position p to the target point p g (x g ,y g ), d(p,p g ) represents the Euclidean distance from the current position to the target point, and the gravitational potential field of the target point to the unmanned ship is expressed as:

[0024]

[0025] Among them, δ is the attraction coefficient, and α is the correction coefficient in the gravitational field;

[0026] The attraction of the target point to the obstacle can be obtained from the gradient of the attractive potential field:

[0027]

[0028] The current position of the unmanned ship is affected by K repulsive potential fields. In order to prevent the unmanned ship from entering the interference, the safety radius coefficient β is introduced. The repulsive potential field of the kth interference on the unmanned ship can be expressed as:

[0029]

[0030] Among them, η k is the rejection coefficient of the interference k, r k It is the interference threat radius set according to the signal-to-interference ratio threshold of the received information of the unmanned ship. D is the range of action of the repulsive potential field. The repulsive force can be obtained by the negative gradient of the repulsive potential field:

[0031]

[0032] The resultant force vector of the unmanned ship at its current position is as follows:

[0033]

[0034] Among them, F att is the gravitational vector, F k,rep is the repulsive force vector of the kth interference on the unmanned ship;

[0035] Step 2.2: Trajectory planning is performed by combining the deep deterministic gradient algorithm in reinforcement learning with the artificial potential field method. The deep deterministic gradient algorithm in reinforcement learning uses four neural networks: the action reality network μ(s|θ μ ), action target network μ′(s|θ μ′ ), evaluate the real network Q(s|θ Q) and the evaluation target network Q′(s|θ Q′ ), action reality network parameters θ μ and evaluate the actual network parameters θ Q Obtained by random initialization, the initialization network parameters of the two target networks are the same as the actual network, and then the parameters are gradually updated, s is the state;

[0036] At time t, the observed state s is obtained according to the interference and training environment. t , define ΔL t,k is the distance vector from the unmanned ship to the interference k; The distance from the current position of the unmanned ship to the target point p g The distance necklace of R t is the data transmission rate of the observed unmanned ship at the current moment, then:

[0037]

[0038] The action realization network selects an action output a according to the state observed by the unmanned ship from the environment, [η 1 ,η k ,…,η K ] is the repulsive coefficient of each disturbance in the repulsive potential field:

[0039] a=[η 1 ,η k ,L,η K ]

[0040] According to the actions performed by the unmanned ship, an immediate reward r is obtained. t and the state s at the next moment t+1 ,but:

[0041]

[0042] Among them, d(p t ,p g ) is the current position pt to the target point p g The distance, d(p 0 ,p g ) is the starting point p 0 To the target point p g The distance

[0043] State transfer process t ,a t ,r t ,s t+1} is stored in the experience storage area B, and then the expected Q value is calculated using the action output of the action target network and the Q value of the evaluation network

[0044]

[0045] Will The mean square error of the output Q value of the evaluation network is used as the loss function to calculate the gradient of the evaluation real network, and the daily strategy is used to update the two real network parameters. The target network parameters are gradually updated according to the real network parameters, and finally the unmanned ship trajectory planning is realized.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] With the surge in the number of radio equipment in maritime missions, the electromagnetic environment in which unmanned ships are located has become increasingly complex. Frequent electromagnetic interference and communication interruptions have seriously affected their environmental cognition and autonomous obstacle avoidance capabilities. In recent years, the advancement of edge computing and intelligent perception technology has provided new solutions for unmanned ships. By performing real-time data processing and analysis at the edge layer, unmanned ships can obtain environmental information and make decisions faster. In this context, the present invention proposes a VTS (vessel traffic service) and unmanned ship edge perception path planning method suitable for unstable communication conditions. First, a trajectory planning framework based on edge intelligent perception is constructed. The collaborative communication and computing between edge servers, sensor terminals and unmanned ships are used to enhance the real-time perception ability of unmanned ships in complex environments, ensuring that they can still autonomously avoid obstacles and efficiently perform tasks when communication is poor or signal interruptions occur. Secondly, an artificial potential field method based on deep deterministic policy gradient (DDPG) algorithm optimization is proposed to effectively avoid planning failures caused by local minima in traditional trajectory planning, ensure that unmanned ships can complete global optimal trajectory planning in complex electromagnetic environments, and improve the safety and efficiency of task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 This is a schematic diagram of the unmanned ship trajectory planning framework based on edge intelligent perception of the present invention.

[0050] Figure 2 It is the global and local spectrum map of the present invention.

[0051] Figure 3 This is a flowchart of the execution of the artificial potential field method combined with the deep deterministic policy gradient algorithm optimization of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0053] The present invention provides a method for edge sensing path planning between a VTS and an unmanned ship under unstable communication, comprising the following steps:

[0054] Step 1: A trajectory planning framework for unmanned ships based on edge intelligent perception is proposed. This framework enhances the real-time perception of the unmanned ship's surrounding environment through collaborative communication and calculation between edge servers, sensor terminals and unmanned ships, especially for VTS (vessel traffic services) in the case of unstable communication or insufficient signal. Even in the case of poor signal, the unmanned ship can still autonomously process the surrounding environment information, improve the efficiency and accuracy of autonomous obstacle avoidance, and thus better adapt to complex and changing environmental conditions. Specifically, it includes the following steps:

[0055] Step 1.1: The present invention designs an edge network scenario, in which a base station with an edge server and several sensor devices with perception functions are deployed, such as unmanned ships, VTS service centers, ship-borne receivers, etc. The unmanned ship travels from the starting point to the target point, passing through the obstacle area and electromagnetic interference area, while maintaining a communication link with the base station. Therefore, the present invention proposes an edge intelligent perception unmanned ship trajectory planning framework, such as Figure 1 As shown. In a complex electromagnetic environment, unmanned boats, VTS service stations, ship-borne receivers and other sensor equipment with perception functions are deployed to collect spectrum data in three-dimensional space and upload them to the edge server. The edge server uses the Kriging interpolation algorithm to interpolate unknown areas based on the collected spectrum data to construct a high-precision spectrum map. However, in a complex electromagnetic environment, spatial spectrum resources and network available bandwidth are limited. If high-precision spectrum maps are transmitted directly, it will bring a great burden to the communication network bandwidth, causing high latency, and may cause the drone to be affected by strong sudden electromagnetic interference and lose communication. For this reason, we propose two ways to send low-precision global spectrum maps and high-precision local spectrum maps.

[0056] Before the unmanned ship departs, the edge server interpolates the low-precision spectrum data based on the collected spectrum data, preliminarily determines the location of electromagnetic interference and spatial obstacles, and sends the compressed global spectrum map to the unmanned ship. When sudden interference or dynamic obstacles occur, the edge server distributes high-precision local spectrum maps to the unmanned ship in real time, and the unmanned ship avoids obstacles in real time based on the threat level and boundaries of the interference or obstacle, such as Figure 2 As shown in the figure, since only low-precision interpolation is performed when constructing the global spectrum map, only the approximate location distribution of electromagnetic interference can be determined. The local spectrum map performs more precise interpolation and can provide the unmanned ship with information such as the radiation intensity, radiation range, and interference boundary of the interference source.

[0057] Step 1.2: Based on the trajectory planning framework in step 1.1, the present invention proposes a communication model between the base station and the unmanned ship. Here, the data transmission rate between the base station and the unmanned ship based on the free space loss model is as follows.

[0058]

[0059] Where W is the channel bandwidth, Pow s is the transmission power of the base station, Pow i is the transmission power of the interference, and σ is the Gaussian white noise power in the environment. u,BS and h u,i They are the channel gains between the unmanned ship and the base station, and between the unmanned ship and the interference source, which can be expressed as follows.

[0060]

[0061] Among them, g 0 is the received power at a reference distance of 1m with a transmission power of 1W, d(p,p BS ) and d(p,p i ) represent the Euclidean distances from the unmanned ship to the base station and the interference source respectively.

[0062] Step 2: An artificial potential field method combined with deep deterministic policy gradient algorithm optimization is proposed. This method is specially designed to deal with communication interruption problems in complex electromagnetic environments, prevent the unmanned ship from falling into local minima in path planning, and effectively improve the global optimality of track planning. When communication is interrupted or unstable, the system can still ensure the safety of the unmanned ship's track through autonomous planning functions, improve the stability and reliability of task execution, and thus ensure the efficiency and safety of the unmanned ship's task completion in complex environments. It specifically includes the following steps:

[0063] Step 2.1: This invention proposes an improved artificial potential field model. Assume that the speed of the unmanned ship is V, and it sails from the current position p to the target point p g (xg ,y g ). The gravitational potential field of the target point to the unmanned ship can be expressed as follows. Where δ is the attraction coefficient and α is the correction coefficient in the gravitational field.

[0064]

[0065] The attraction of the target point to the obstacle can be obtained from the gradient of the attractive potential field as shown below.

[0066]

[0067] The current position of the unmanned ship is affected by K repulsive potential fields. In order to prevent the unmanned ship from entering the interference, a safety radius coefficient β is introduced. The repulsive potential field of the kth interference on the unmanned ship can be expressed as follows.

[0068]

[0069] Among them, η k is the rejection coefficient of the interference k, r k It is the interference threat radius set according to the signal-to-interference ratio threshold of the received information of the unmanned ship, and D is the range of action of the repulsive potential field. The repulsive force can be obtained by the negative gradient of the repulsive potential field, as shown below.

[0070]

[0071] The resultant force vector of the unmanned ship at the current position is as follows. att is the gravitational vector, F k,rep is the repulsive force vector of the kth interference on the unmanned ship.

[0072]

[0073] Step 2.2: The present invention aims to solve the problem of effective obstacle avoidance in complex obstacle scenes and perform trajectory planning at the same time. However, simply using the artificial potential field method of step 2.1 proposed above will easily fall into the local minimum. Therefore, the present invention proposes a deep reinforcement learning method to optimize the repulsion coefficient of the repulsive potential field in the artificial potential field method, thereby optimizing the local minimum problem.

[0074] According to the above optimization objectives, the present invention combines the artificial potential field method with the deep deterministic gradient algorithm (DDPG) in reinforcement learning. A total of four neural networks are used in DDPG: the action reality network μ(s|θ μ ), action target network μ′(s|θ μ′ ), evaluate the real network Q(s|θ Q ) and the evaluation target network Q′(s|θ Q′ ). The parameters θ of the two real networks μand θ Q Obtained by random initialization, the initialization network parameters of the two target networks are the same as the actual network, and then the parameters are gradually updated, and s is the state.

[0075] At time t, the observed state s is obtained according to the interference and training environment. t , defined as follows. Where, ΔL t,k is the distance vector from the unmanned ship to the interference k; The distance from the current position of the unmanned ship to the target point p g The distance necklace of R t The data transmission rate of the observed unmanned ship at the current moment to prevent the unmanned ship from being disturbed and losing communication.

[0076]

[0077] The action realization network selects an action output a according to the state observed by the unmanned ship from the environment. 1 ,η k ,…,η K ] is the repulsion coefficient of each interference in the repulsive potential field.

[0078] a=[η 1 ,η k ,L,η K ] (10)

[0079] According to the actions performed by the unmanned ship, an immediate reward r is obtained. t and the state s at the next moment t+1 , as shown below. Among them, d(p t ,p g ) is the current position p t To the target point p g The distance, d(p 0 ,p g ) is the starting point p 0 To the target point p g distance.

[0080]

[0081] State transfer process t ,a t ,r t ,s t+1} is stored in the experience storage area B, and then the expected Q value is calculated using the action output of the action target network and the Q value of the evaluation network As shown below.

[0082]

[0083] Will The mean square error of the output Q value of the evaluation network is used as the loss function to calculate the gradient of the evaluation real network, and the daily strategy is used to update the two real network parameters. The target network parameters are gradually updated according to the real network parameters, and finally the unmanned ship trajectory planning is realized. The overall framework is as follows Figure 3 shown.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for edge sensing path planning between VTS and unmanned ship under unstable communication, characterized in that: The following steps are involved: Step 1: Build an unmanned ship trajectory planning framework based on edge intelligent perception, which includes a base station, an unmanned ship and sensor equipment; the base station is equipped with an edge server; the sensor terminal is used to collect spectrum data in three-dimensional space and upload it to the edge server in real time, and the edge server is used to build a spectrum map based on the spectrum data; the unmanned ship maintains a communication connection with the base station during the process of traveling from the starting point to the target point, and receives the spectrum map sent by the base station; Step 2: The unmanned ship obtains environmental perception data according to the spectrum map, and processes the environmental perception data using an artificial potential field method optimized with a deep deterministic policy gradient algorithm, thereby realizing the unmanned ship track planning.

2. According to the method of VTS and unmanned ship edge perception path planning under unstable communication in claim 1, it is characterized in that: The spectrum map received by the unmanned ship includes a global spectrum map and a local spectrum map, and the accuracy of the global spectrum map is lower than that of the local spectrum map.

3. According to the method of VTS and unmanned ship edge perception path planning under unstable communication in claim 1, it is characterized in that: The sensor equipment includes at least one of an unmanned ship, a VTS service center and a ship-borne receiver.

4. According to the method of VTS and unmanned ship edge perception path planning under unstable communication in claim 1, it is characterized in that: The edge server constructs a spectrum map, including the following steps: Collecting spectrum data in three-dimensional space based on sensor equipment; The Kriging interpolation algorithm is used to interpolate the unknown area based on the collected spectrum data to construct a spectrum map.

5. According to the method of VTS and unmanned ship edge perception path planning under unstable communication in claim 4, it is characterized in that: The Kriging interpolation algorithm is used to interpolate unknown areas based on the collected spectrum data, including: Based on the collected spectrum data, the Kriging interpolation algorithm is used to interpolate the unknown area to construct a high-precision spectrum map; Before the unmanned ship sets off, low-precision spectrum data interpolation is performed based on the collected spectrum data to preliminarily determine the location of electromagnetic interference and spatial obstacles, and generate a compressed global spectrum map; when sudden interference or dynamic obstacles occur, the edge server distributes high-precision local spectrum maps to the unmanned ship in real time.

6. According to the method of VTS and unmanned ship edge perception path planning under unstable communication in claim 1, it is characterized in that: The data transmission rate between the base station and the unmanned ship is: Where N is the number of interference sources, W is the channel bandwidth, and Pow s is the transmission power of the base station, Pow i is the transmission power of the interference, σ is the Gaussian white noise power in the environment, h u,BS and h u,i are the channel gains between the unmanned ship and the base station, and between the unmanned ship and the interference source, respectively, and: Where g0 is the received power at a reference distance of 1m with a transmission power of 1W, and d(p,p BS ) and d(p,p i ) represent the Euclidean distances from the unmanned ship to the base station and the interference source respectively.

7. The method for dynamic route optimization of multi-objective unmanned ships integrating edge perception according to claim 1 is characterized in that: The artificial potential field method combined with the deep deterministic policy gradient algorithm optimization is used to process the environmental perception data, including: Step 2.1: Assume that the speed of the unmanned ship is V and it sails from the current position p to the target point p g (x g ,y g ), d(p,p g ) represents the Euclidean distance from the current position to the target point, and the gravitational potential field of the target point to the unmanned ship is expressed as: Among them, δ is the attraction coefficient, and α is the correction coefficient in the gravitational field; The attraction of the target point to the obstacle can be obtained from the gradient of the attractive potential field: F att =▽U att =δd(p,p g )+α / d 2 (p,p g ) The current position of the unmanned ship is affected by K repulsive potential fields. In order to prevent the unmanned ship from entering the interference, the safety radius coefficient β is introduced. The repulsive potential field of the kth interference on the unmanned ship can be expressed as: Among them, η k is the rejection coefficient of the interference k, r k It is the interference threat radius set according to the signal-to-interference ratio threshold of the received information of the unmanned ship. D is the range of action of the repulsive potential field. The repulsive force can be obtained by the negative gradient of the repulsive potential field: The resultant force vector of the unmanned ship at its current position is as follows: Among them, F att is the gravitational vector, F k,rep is the repulsive force vector of the kth interference on the unmanned ship; Step 2.2: Trajectory planning is performed by combining the deep deterministic gradient algorithm in reinforcement learning with the artificial potential field method. The deep deterministic gradient algorithm in reinforcement learning uses four neural networks: the action reality network μ(s|θ μ ), action target network μ′(s|θ μ′ ), evaluate the real network Q(s|θ Q ) and the evaluation target network Q′(s|θ Q′ ), action reality network parameters θ μ and evaluate the actual network parameters θ Q Obtained by random initialization, the initialization network parameters of the two target networks are the same as the actual network, and then the parameters are gradually updated, s is the state; At time t, the observed state s is obtained according to the interference and training environment. t , define ΔL t,k is the distance vector from the unmanned ship to the interference k; The distance from the current position of the unmanned ship to the target point position p g The distance necklace of R t is the data transmission rate of the observed unmanned ship at the current moment, then: The action realization network selects an action output a according to the state observed by the unmanned ship from the environment, [η1,η k ,…,η K ] is the repulsive coefficient of each disturbance in the repulsive potential field: a=[η1,η k ,L,h K ] According to the actions performed by the unmanned ship, an immediate reward r is obtained. t and the state s at the next moment t+1 ,but: Among them, d(p t ,p g ) is the current position pt to the target point p g The distance, d(p0,p g ) is the starting point p0 to the target point p g The distance State transfer process t ,a t ,r t ,s t+1 } is stored in the experience storage area B, and then the expected Q value is calculated using the action output of the action target network and the Q value of the evaluation network Will The mean square error of the output Q value of the evaluation network is used as the loss function to calculate the gradient of the evaluation real network, and the daily strategy is used to update the two real network parameters. The target network parameters are gradually updated according to the real network parameters, and finally the unmanned ship trajectory planning is realized.

Images