A satellite internet-based data transmission scheduling method and device for a marine vessel

Abstract

The application provides a kind of based on satellite internet's marine ship data transmission scheduling method and device. The method assesses the initial weight of ship sensor data queue according to data delay and bandwidth requirement, and dynamically adjusts the weight based on abnormal data detection and abnormal target detection method;Design a multi-layer buffer mechanism, define the validity period of data packet, and design a packet loss decision mechanism based on relaxation decision diagram;According to satellite ephemeris data, predict the satellite service window time, define the potential output performance function of data, and model the ship multi-sensor data queue scheduling process as a hierarchical Markov decision process;Design a ship multi-sensor data queue scheduling algorithm based on improved D3QN, which prevents queue overflow, ensures queue transmission fairness, and maximizes long-term data output performance within the service window time. The application effectively solves the problem that the effectiveness and fairness of real-time data transmission cannot be guaranteed under the condition of frequent switching of satellite-sea link.

Description

Technical Field

[0001] This invention relates to the field of marine satellite Internet of Things (IoT) communication technology, and in particular to a method and apparatus for scheduling maritime ship data transmission based on satellite Internet. Background Technology

[0002] Oceans cover more than 70% of the Earth's surface, and maritime transport accounts for over 90% of global trade. This reality creates a new paradigm for the application of IoT technology in the marine field. Currently, marine data transmission mainly relies on satellite communication, shore-based communication, and maritime wireless communication. Compared to terrestrial mobile communication technology, the biggest advantage of satellite communication is its wide-area coverage without blind spots, regardless of terrain, topography, or distance. It's worth mentioning low-Earth orbit satellite internet, such as SpaceX Starlink, CloudSat, and SPECSI, which are playing an irreplaceable role in complex marine communication environments due to their low latency and low cost. Compared to traditional terrestrial communication networks, marine satellite IoT faces unique marine environments, bringing problems such as topological dynamics, communication resource complexity, network heterogeneity, and uncertainty, severely restricting the stability and continuity of data transmission.

[0003] As the primary means of transportation on the ocean, ships are indispensable components of the marine satellite Internet of Things (IoT). Therefore, researching optimization strategies for ship data transmission in the marine environment has become a crucial issue. Specifically, the continuous movement of ships and low-Earth orbit satellites within the marine satellite IoT, coupled with frequent switching of satellite-to-ocean links, makes it difficult to guarantee the effectiveness of real-time data transmission. In this complex scenario, the lifecycle of the links is affected by various dynamic factors. How to efficiently utilize communication resources within the effective service window, prioritize the transmission of latency-sensitive data, reduce data packet loss, and ensure fairness in the transmission of various types of sensor data is one of the core issues that urgently needs to be addressed in research on maritime ship data transmission scheduling.

[0004] In the complex and dynamic scenarios of marine satellite IoT, Deep Reinforcement Learning (DRL) offers a novel approach to solving data transmission scheduling problems. Traditional heuristic algorithms struggle to adapt to the multidimensional uncertainties brought about by frequent switching of satellite links, dynamic topology changes, and resource heterogeneity. DRL, however, demonstrates strong environmental adaptability by continuously learning optimal scheduling strategies through agent-environment interaction. Among these algorithms, the D3QN algorithm, an improvement on Deep Q-Network (DQN), separates action selection and action evaluation through Double Q-Learning, reducing Q-value overestimation and improving policy stability. By introducing a Dueling Network, it decomposes the Q-value into a state-value function and an action advantage function, enabling more efficient evaluation of action value, especially when the action space is large. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for scheduling maritime ship data transmission based on satellite internet, in order to eliminate or improve one or more defects existing in the prior art, and solve the problem that the effectiveness and fairness of real-time data transmission are difficult to guarantee under the condition of frequent switching of satellite and sea links in existing maritime ship data transmission. While preventing queue overflow and ensuring the fairness of queue transmission, it maximizes the long-term data output efficiency within the service window.

[0006] On one hand, this invention provides a method for scheduling maritime ship data transmission based on satellite internet. The method is characterized by being applied to a maritime ship system and executed on a satellite broadband terminal. The maritime ship is equipped with the satellite broadband terminal, and AIS, radar, hydrological, meteorological, and optical sensors are installed on the ship's mast. Each sensor is connected to the satellite broadband terminal, and the marine IoT sensing data is transmitted back to the ground based on the satellite internet. The method includes the following steps:

[0007] S1. The marine vessel system operates in discrete time. Based on the sensor category, the ship sensor data queue in the satellite broadband terminal data buffer is defined as follows: The ship sensor data queue is prioritized and assigned different weights. ;

[0008] S2. A method for managing ship multi-sensor data queues, which allocates buffers to each ship sensor data queue according to weights, defines the validity period of data packets, and discards data packets that exceed the length of the buffers allocated to each ship sensor data queue according to the principle of minimizing the validity period of data packets.

[0009] S3. Based on satellite ephemeris data, predict satellite position information and the service window time of the satellite and the satellite broadband terminal. Divide the service window time into several discrete time slots and define a data potential output efficiency function. The efficiency function is used to evaluate the utility value that can be obtained by transmitting data packet k in the ship sensor data queue n within time slot t. It is positively correlated with the queue weight and negatively correlated with the validity period of the data packet. The ship multi-sensor data queue scheduling process is modeled as a distributed Markov decision process. The process takes the ship sensor data queue state under discrete time slots as input and the decision to select the ship sensor data queue as output. The decision is driven by a reward function to maximize the long-term accumulated data potential output efficiency.

[0010] S4. Design a ship multi-sensor data queue scheduling algorithm based on the improved D3QN, which maximizes the long-term data output efficiency within the service window while preventing queue overflow and ensuring the fairness of queue transmission.

[0011] In some embodiments of the present invention, step S1, the method for prioritizing the ship sensor data queue includes the following steps:

[0012] The initial priority of the ship's sensor data queues is determined based on data latency and bandwidth requirements, and initial weights are assigned to the meteorological, hydrological, optical, AIS, and radar data transmission queues. ;

[0013]

[0014] Depending on the actual situation, the weights of the ship sensor data queue can be adjusted, including the following:

[0015] (1) Design an abnormal fluctuation detection method for meteorological / hydrological data based on K-means clustering. Perform cluster analysis on meteorological and hydrological data, and identify data points that are far from the cluster center as abnormal points, triggering the adjustment of the weight of meteorological and hydrological ship sensor data queues;

[0016] (2) Design an abnormal target detection method based on image edge detection. The Canny edge detection algorithm is used to detect the target edge in the image. If an abnormal target exists, the weight adjustment of the data queues of optical, AIS and radar ship sensors is triggered. The weight ratio is adjusted to optical > AIS > radar.

[0017] (3) Weight adjustment rules, , The adjusted weights The weights before adjustment. This refers to the proportion by which the weights are adjusted.

[0018] In some embodiments of the present invention, step S2 of the ship multi-sensor data queue management method includes the following steps:

[0019] A multi-layered buffer mechanism is designed, with the data buffer threshold of the satellite broadband terminal being [value missing]. Set a data queue buffer threshold for the ship sensor data queue according to the aforementioned weights. ;

[0020] Define the validity period of the arriving data packet. , The data packet deadline is carried when the data packet is generated, and it is handled according to the First Come First Service (FCFS) principle. Queue delay, Transmission delay includes sending delay and propagation delay. The satellite-to-ground link channel status is monitored, and the propagation delay is calculated. , Where c is the distance between the satellite broadband terminal and the satellite, and c is the speed of light;

[0021] The design of the packet loss decision mechanism involves using a top-down approach to select the data packet with the shortest validity period for packet loss when the length of the ship's sensor data queue exceeds the data queue buffer threshold, with the excess space size as a constraint. This is achieved by constructing a relaxed decision graph.

[0022] In some embodiments of the present invention, in step S3, the definition of the data potential output performance function is... , The weights of the ship sensor data queue n are: The validity period of data packet k in the ship sensor data queue n.

[0023] In some embodiments of the present invention, step S3, which involves predicting the service window time of the satellite and the satellite broadband terminal, includes the following steps:

[0024] Acquire satellite ephemeris data, predict satellite position coordinates, and transform the satellite position coordinates to the ECEF (Earth-Centered, Earth-Fixed) coordinate system. ;

[0025] Obtain the latitude and longitude of the location of the satellite broadband terminal installed on the ship, and convert the latitude and longitude of the satellite broadband terminal location to the ECEF coordinate system. ;

[0026] Calculate the elevation angle between the satellite and the satellite broadband terminal. Elevation angle condition: The elevation angle of the satellite relative to the ground terminal must be greater than the minimum elevation angle. Distance condition: The distance between the satellite and the ground terminal must be within the communication range. Serviceability is determined based on the above two conditions.

[0027] By iteratively calculating the satellite's position and elevation angle, the time points when the satellite enters and leaves the visible service area are determined, thereby obtaining the service window time. , , It is the initial time. , It is the time difference between when a satellite enters and leaves the visible service area.

[0028] In some embodiments of the present invention, step S3, which involves modeling the ship multi-sensor data queue scheduling process as a hierarchical Markov decision process, includes the following steps:

[0029] The design incorporates a hierarchical state space, including a high-level state space and a low-level state space. The high-level state space refers to the features related to the global scheduling target of the ship's multi-sensor data queue. This includes the overflow risk level of each ship sensor data queue. The potential output efficiency of the average header data packet of each ship sensor data queue Data transmission rate at time slot t The lower-level state space selects the local features of each ship sensor data queue set according to the higher-level strategy output by the higher-level state space. Includes the current length of the selected ship sensor data queue. Potential output performance of header data packets Historical transmission count and number of admissions ;

[0030] The design incorporates a hierarchical action space, including a high-level action space and a low-level action space. The high-level action space is used to select the global scheduling strategy for the ship's multi-sensor data queue. In emergency priority mode, only queues with an overflow risk level of warning or emergency are allowed to be processed. In performance-priority mode, select the queues with the highest potential output performance of the current header data packet, i.e. The underlying action space is used to select the specific sensor data queue to be transmitted under the constraints of the higher-level policy. ;

[0031] The design incorporates a hierarchical reward function, including a high-level reward function and a low-level reward function. The high-level reward function aims to maximize long-term cumulative efficiency while controlling queue overflow. The calculation method is as follows: The fairness penalty coefficient The degree of unfairness increases dynamically with the output of the queue; the underlying reward function calculates the data packet performance of the currently selected ship multi-sensor data queue and deducts the corresponding fairness penalty, i.e. ,in This is an indicator function that ensures local decisions are consistent with the global objective.

[0032] In some embodiments of the present invention, the execution algorithm of the hierarchical Markov decision process is the ship multi-sensor data queue scheduling algorithm based on the improved D3QN, which includes the following steps:

[0033] Initialize high-level main network parameters and underlying main network parameters ,in , For the branch weights of the value function, , Assign weights to the action advantage branch; initialize the high-level target network parameters respectively. and underlying target network parameters Clear the experience replay pool D, and set the probability of exploring higher levels. and the probability of exploring the bottom layer and attenuation rate Define the batch sample size B and the synchronization round L;

[0034] Initialize the satellite position and the satellite broadband terminal position, obtain the service window time TW, and determine the number and step size of scheduling time slots;

[0035] For each time slot t, the agent determines the current high-level state in the high-level state space. The current high-level action in the high-level action space is selected using an ε-greedy strategy. The underlying action space is generated according to the high-level selection strategy, and in the underlying state... Based on this, the Q-value of each ship sensor data queue is calculated using the underlying main network. The ship sensor data queue with the largest Q value is selected for transmission. ;

[0036] After executing the current underlying action in the underlying action space, the actual number of data transmitted is calculated based on the data transmission rate at time slot t, and the length of the ship sensor data queue is updated accordingly. ; Calculate the high-level rewards in real time and the underlying reward and transfer samples and Store it in the experience replay pool D;

[0037] Randomly sample batch B from the experience replay pool D and calculate the Q value of the high-level target. The underlying target Q value ,in This is the discount factor for the aforementioned high-level rewards. The high-level loss function is calculated using the discount factor of the underlying reward. Low-level loss function The high-level master network parameters and the low-level master network parameters are updated using the AdamW (Adaptive Moment Estimation with Weight Decay) optimizer; the high-level target network parameters are synchronized via soft updates every L steps. The underlying target network parameters ;

[0038] Repeat the above process until the maximum number of iterations is reached. Finally, under the constraints of preventing queue overflow and ensuring the fairness of queue transmission, generate the ship multi-sensor data transmission scheme with the highest long-term data output efficiency within the service window time.

[0039] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods mentioned above.

[0040] The beneficial effects of the present invention are at least as follows:

[0041] This invention provides a method and apparatus for scheduling maritime ship data transmission based on satellite internet. The method assesses the initial weights of ship sensor data queues based on data latency and bandwidth requirements, and dynamically adjusts the weights of corresponding sensor data queues using methods such as K-means clustering for detecting abnormal fluctuations in meteorological / hydrological data and designing anomaly target detection methods based on image edge detection. It designs a multi-layer buffer mechanism, defines data packet validity periods, and designs a packet loss decision mechanism based on a relaxation decision graph. Based on satellite ephemeris data, it predicts satellite service window times, defines a potential data output efficiency function, and models the ship multi-sensor data queue scheduling process as a hierarchical Markov decision process. It designs a ship multi-sensor data queue scheduling algorithm based on an improved D3QN, which maximizes long-term data output efficiency within the service window time while preventing queue overflow and ensuring fairness in queue transmission. The maritime ship data transmission scheduling method provided by this invention ensures the effectiveness of real-time data transmission under frequent switching of satellite-sea links, reduces data packet loss, and satisfies the fairness of multi-sensor data transmission.

[0042] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0043] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0045] Figure 1 This is a schematic diagram of a maritime ship data transmission scheduling scenario based on satellite internet, according to one embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram illustrating the steps of a satellite internet-based maritime ship data transmission scheduling method in one embodiment of the present invention.

[0047] Figure 3 This is a flowchart of a ship multi-sensor data queue scheduling algorithm based on improved D3QN in one embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0049] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0050] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0051] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0052] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0053] To address the challenges of ensuring the effectiveness and fairness of real-time data transmission under frequent switching of satellite links in existing maritime ship data transmission methods, and to maximize long-term data output efficiency within the service window while preventing queue overflow and ensuring fair queue transmission, this invention provides a satellite internet-based maritime ship data transmission scheduling method. This method is executed on a satellite broadband terminal. Maritime ships are equipped with satellite broadband terminals, and sensors such as AIS, radar, hydrological, meteorological, and optical sensors are installed on the ship's mast. Each sensor connects to the satellite broadband terminal, and the marine IoT sensing data is transmitted back to the ground via satellite internet. The specific method steps are illustrated in the diagram below. Figure 1 As shown, the specific scenario is as follows: Figure 2 As shown, the method specifically includes the following steps S1 to S4:

[0054] S1. The marine vessel system operates in discrete time slots, with a time slot length of 1 second. Based on the sensor category, the ship sensor data queue in the satellite broadband terminal data buffer is defined as follows: The ship sensor data queue is prioritized and assigned different weights. ;

[0055] S2. A method for managing ship multi-sensor data queues, which allocates buffers to each ship sensor data queue according to weights, defines the validity period of data packets, and discards data packets that exceed the length of the buffers allocated to each ship sensor data queue according to the principle of minimizing the validity period of data packets.

[0056] S3. Based on satellite ephemeris data, predict satellite position information and the service window time of the satellite and the satellite broadband terminal. Divide the service window time into several discrete time slots with a time slot length of 1s. Define a data potential output efficiency function. The efficiency function is used to evaluate the utility value that can be obtained by transmitting data packet k in the ship sensor data queue n within time slot t. It is positively correlated with the queue weight and negatively correlated with the validity period of the data packet. Model the ship multi-sensor data queue scheduling process as a distributed Markov decision process. The process takes the ship sensor data queue state under discrete time slots as input and the decision to select the ship sensor data queue as output. The decision is driven by a reward function to maximize the long-term accumulated data potential output efficiency.

[0057] S4. Design a ship multi-sensor data queue scheduling algorithm based on the improved D3QN, which maximizes the long-term data output efficiency within the service window while preventing queue overflow and ensuring the fairness of queue transmission.

[0058] In step S1, the satellite broadband terminal assesses the initial priority of the ship's sensor data queue based on data latency and bandwidth requirements during the initial operation. Meteorological, hydrological, and optical data are highly latency-sensitive, but optical data transmission requires a large bandwidth. AIS and radar data are less latency-sensitive. Therefore, the initial weighting of the ship's sensor data queue is set as follows: .

[0059] In some satellite broadband terminals, an anomaly fluctuation detection method based on K-means clustering for meteorological / hydrological data is run in real time. This method performs cluster analysis on meteorological and hydrological data. When anomaly data points far from the cluster center are detected, the weight of the meteorological and hydrological ship sensor data queue is increased. In some embodiments, this increase can be 20%. After all the anomaly data has been transmitted, the queue weight returns to its original weight. The satellite broadband terminal also runs an anomaly target detection method based on image edge detection in real time. This method uses the Canny edge detection algorithm to detect target edges in the image. If an anomaly target is found, the weight of the optical, AIS, and radar ship sensor data queues is increased, with the weight ratio adjusted to optical > AIS > radar. In some embodiments, this increase can be 30%, 20%, and 10%, respectively. After all the target data has been transmitted, the queue weight returns to its original weight. The weight adjustment rules are as follows. , The adjusted weights The weights before adjustment. This refers to the proportion by which the weights are adjusted.

[0060] In step S2, the ship multi-sensor data queue management method includes the following steps:

[0061] The satellite broadband terminal data buffer implements a multi-layer buffer mechanism, setting the data buffer threshold to [value missing]. Set a data queue buffer threshold for the ship sensor data queue according to the queue weight. The threshold of each data queue buffer will change dynamically with the queue weight.

[0062] Define the validity period of the arriving data packet. , The data packet deadline is carried when the data packet is generated and is calculated according to the first-come, first-served principle. Queue delay, Transmission delay includes sending delay and propagation delay. The satellite broadband terminal monitors the satellite-to-ground link channel status and calculates the propagation delay. , Where is the distance between the satellite broadband terminal and the satellite, c is the speed of light, and the validity period of the data packet measures the remaining valid time of the data packet. This indicator helps ensure that the data packet is transmitted within the valid time window and reduces latency.

[0063] In some embodiments, the satellite broadband terminal operates a packet loss decision mechanism based on a relaxation decision graph in real time, aiming to optimize the management of ship sensor data queues. The system monitors the queue length in real time. When the queue length exceeds a set buffer threshold, the packet loss decision mechanism based on the relaxation decision graph is triggered. Starting from the head of the queue, decisions are made for each data packet from top to bottom. The relaxation decision graph uses the validity period of each data packet as a decision node, gradually filtering out data packets with the shortest validity period to be discarded. The relaxation process of the graph simplifies the decision space, enabling packet loss decisions to be executed efficiently under complex constraints, minimizing the impact of packet loss on data transmission quality.

[0064] In step S3, the potential output performance function of the data is defined. , The weights of the ship sensor data queue n are given by the following: Let be the validity period of data packet k in the ship's sensor data queue n. The data potential output efficiency function takes into account both the sensor data queue priority and the data packet validity period. The higher the priority and the shorter the data packet validity period, the greater the data potential output efficiency.

[0065] In step S3, the ship's multi-sensor data queue scheduling process is modeled as a hierarchical Markov decision process, including the following steps:

[0066] The design incorporates a hierarchical state space, including a high-level state space and a low-level state space. The high-level state space refers to the characteristics related to the global scheduling target of the ship's multi-sensor data queue. This includes the spillover risk level of each ship's sensor data queue. Potential output efficiency of average header data packets in each ship's sensor data queue Data transmission rate at time slot t The lower-level state space is composed of local features of the data queues of each ship's sensor selected according to the higher-level strategy. Includes the current length of the selected ship sensor data queue. Potential output performance of header data packets Historical transmission count and number of admissions ;

[0067] The design incorporates a hierarchical action space, including a high-level action space and a low-level action space. The high-level action space is used to select the global scheduling strategy for the ship's multi-sensor data queue. In emergency priority mode, only queues with an overflow risk level of warning or emergency are allowed to be processed. In performance-priority mode, the queues with the highest potential output performance for the current header data packet are selected. The lower-level action space is used to select the specific sensor data queue to be transmitted under the constraints of higher-level policies. ;

[0068] The design incorporates a hierarchical reward function, including a high-level reward function and a low-level reward function. The high-level reward function aims to maximize long-term cumulative efficiency while controlling queue overflow. The calculation method is as follows: The fairness penalty coefficient The unfairness of the queue output increases dynamically; the underlying reward function calculates the data packet efficiency of the currently selected ship multi-sensor data queue and deducts the corresponding fairness penalty, i.e. ,in This is an indicator function that ensures local decisions are consistent with the global objective.

[0069] In some embodiments, Refers to the current time slot The Gini coefficient, used to measure the imbalance in queue output, is calculated using the following formula: , It is a queue In the time slot The historical number of transmissions.

[0070] In some embodiments, the data stored in each sensor data queue has the same attributes, size, and latency sensitivity. Therefore, the data packets in the same queue are arranged in order of validity period from smallest to largest. Conversely, the output gain of the data packets in each queue is arranged in order of data packet validity period from largest to smallest. Therefore, when selecting the data queue to be transmitted in each time slot, only the header data packets of the same queue need to be considered for scheduling. After selecting the data queue to be transmitted, the number of data packets to be transmitted in this time slot is determined according to the time slot size and the data packet size of the queue. In the next round of scheduling, the new header data packets are used as the criterion.

[0071] The execution algorithm for the hierarchical Markov decision process is a ship multi-sensor data queue scheduling algorithm based on an improved D3QN, which includes the following steps:

[0072] Initialize high-level main network parameters and underlying main network parameters ,in , For the branch weights of the value function, , Assign weights to the action advantage branch; initialize the high-level target network parameters respectively. and underlying target network parameters Clear the experience replay pool D, and set the probability of exploring higher levels. and the probability of exploring the bottom layer and attenuation rate Define the batch sample size B and the synchronization round L, for example, , , B=64, L=500;

[0073] Initialize satellite position and satellite broadband terminal position, obtain service window time TW, and determine the number and step size of scheduling time slots;

[0074] For each time slot t, the agent determines the current high-level state in the high-level state space. The current high-level action in the high-level action space is selected using an ε-greedy strategy. The underlying action space is generated according to the selected strategy at the higher level, in the underlying state. Based on this, the Q-values ​​of each ship's sensor data queue are calculated using the underlying main network. Select the ship sensor data queue with the largest Q value to perform the transmission. ;

[0075] After executing the current underlying action in the underlying action space, the actual number of data transmitted is calculated based on the data transmission rate at time slot t, and the length of the ship sensor data queue is updated accordingly. Calculate real-time high-level rewards and underlying rewards and transfer samples and Store in experience replay pool D;

[0076] Randomly sample batch B from the experience replay pool D and calculate the Q value of the high-level target. The underlying target Q value Calculate the high-level loss function Low-level loss function ,in This is the discount factor for the aforementioned high-level rewards. The discount factor for the underlying reward is used to update the higher-level main network parameters and the lower-level main network parameters through the AdamW optimizer; the higher-level target network parameters are synchronized through a soft update every L steps. Low-level target network parameters For example, , ;

[0077] Repeat the above process until the maximum number of iterations is reached. Finally, under the constraints of preventing queue overflow and ensuring the fairness of queue transmission, a ship multi-sensor data transmission scheme with the highest long-term data output efficiency within the service window time is generated.

[0078] In some embodiments, predicting the service window time of satellites and satellite broadband terminals includes the following steps:

[0079] Acquire satellite ephemeris data, predict satellite position coordinates, and transform the satellite position coordinates to the ECEF (Earth-Centered, Earth-Fixed) coordinate system. ;

[0080] Obtain the latitude and longitude of the satellite broadband terminal installed on the ship, and convert the latitude and longitude of the satellite broadband terminal to the ECEF coordinate system. ;

[0081] Calculate the elevation angle between the satellite and the satellite broadband terminal. Elevation angle condition: The elevation angle of the satellite relative to the ground terminal must be greater than the minimum elevation angle (usually 5°-10°). Distance condition: The distance between the satellite and the ground terminal must be within the communication range. Serviceability is determined based on the above two conditions.

[0082] By iteratively calculating the satellite's position and elevation angle, the time points when the satellite enters and leaves the visible service area are determined, thereby obtaining the service window time. , , It is the initial time. , It is the time difference between when a satellite enters and leaves the visible service area.

[0083] In some embodiments, hierarchical Markov modeling decomposes the complex ship multi-sensor data queue scheduling problem into two levels: global policy and local execution. This effectively reduces the decision-making complexity brought about by the high-dimensional state space and dynamic environment. The higher-level model focuses on long-term data output performance objectives, reducing the adverse effects of queue overflow, while the lower-level model focuses on short-term data output performance objectives, ensuring the timeliness and fairness of single-step transmission actions. This hierarchical structure not only adapts to the dynamic characteristics of frequent switching in the Starship Link, but also avoids conflicts in multi-objective optimization through objective decoupling, enabling the system to cope with sudden loads while maintaining global stability.

[0084] In some embodiments, the advantages of hierarchical modeling are further amplified by combining the improved D3QN algorithm. The improved D3QN separates state value and action advantage through a dual network architecture, accurately evaluating the long-term benefits of different level strategies. Simultaneously, it utilizes Double Q-Learning to reduce Q-value overestimation bias and improve decision robustness in link switching scenarios. The hierarchical experience replay mechanism enables the algorithm to learn the correlation between global strategies and local actions separately, accelerating convergence and enhancing generalization ability. This design, while preventing queue overflow and balancing multi-sensor data transmission, maximizes data output efficiency within the service window, providing an efficient and adaptive scheduling solution for marine satellite IoT.

[0085] The flowchart of the ship multi-sensor data queue scheduling algorithm based on the improved D3QN is as follows: Figure 3 As shown.

[0086] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for scheduling maritime ship data transmission based on satellite internet.

[0087] Corresponding to the above method, the present invention also provides an apparatus comprising a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus performs the steps of the method as described above.

[0088] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0089] In summary, this invention provides a method and apparatus for scheduling maritime ship data transmission based on satellite internet. The method assesses the initial weights of ship sensor data queues based on data latency and bandwidth requirements, and dynamically adjusts the weights of corresponding sensor data queues using methods such as K-means clustering for detecting abnormal fluctuations in meteorological / hydrological data and designing anomaly target monitoring methods based on image edge detection. It designs a multi-layer buffer mechanism, defines data packet validity periods, and designs a packet loss decision mechanism based on a relaxation decision graph. Based on satellite ephemeris data, it predicts satellite service window times, defines a potential data output efficiency function, and models the ship multi-sensor data queue scheduling process as a hierarchical Markov decision process. It designs a ship multi-sensor data queue scheduling algorithm based on an improved D3QN, which, while preventing queue overflow and ensuring fairness in queue transmission, maximizes long-term data output efficiency within the service window time. The maritime ship data transmission scheduling method provided by this invention ensures the effectiveness of real-time data transmission under frequent switching of satellite-to-sea links, reduces data packet loss, and satisfies the fairness of multi-sensor data transmission.

[0090] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0091] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0092] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for scheduling maritime ship data transmission based on satellite internet, characterized in that, The method is applied to a marine vessel system, which includes a satellite broadband terminal and five sensor categories, including AIS, radar, hydrological, meteorological, and optical sensors. Each sensor is connected to the satellite broadband terminal. The method includes the following steps: S1. The marine vessel system operates in discrete time slots. Based on the sensor category, the ship sensor data queue in the satellite broadband terminal data buffer is defined as follows: The ship sensor data queue is prioritized and assigned different weights. ; S2. A method for managing ship multi-sensor data queues, which allocates buffers to each ship sensor data queue according to weights, defines the validity period of data packets, and discards data packets that exceed the length of the buffers allocated to each ship sensor data queue according to the principle of minimizing the validity period of data packets. S3. Based on satellite ephemeris data, predict satellite position information and the service window time of the satellite and the satellite broadband terminal. Divide the service window time into several discrete time slots and define a data potential output efficiency function. The efficiency function is used to evaluate the utility value that can be obtained by transmitting data packet k in the ship sensor data queue n within time slot t. It is positively correlated with the queue weight and negatively correlated with the validity period of the data packet. The ship multi-sensor data queue scheduling process is modeled as a distributed Markov decision process. The process takes the ship sensor data queue state under discrete time slots as input and the decision to select the ship sensor data queue as output. The decision is driven by a reward function to maximize the long-term accumulated data potential output efficiency. S4. Design a ship multi-sensor data queue scheduling algorithm based on the improved D3QN, which maximizes the long-term data output efficiency within the service window while preventing queue overflow and ensuring the fairness of queue transmission.

2. The method for scheduling maritime ship data transmission based on satellite internet according to claim 1, characterized in that, In step S1, the method for prioritizing the ship sensor data queue includes the following steps: The initial priority of the ship's sensor data queues is determined based on data latency and bandwidth requirements, and initial weights are assigned to the meteorological, hydrological, optical, AIS, and radar data transmission queues. ; Depending on the actual situation, the weights of the ship sensor data queue can be adjusted, including the following: (1) Design an abnormal fluctuation detection method for meteorological / hydrological data based on K-means clustering. Perform cluster analysis on meteorological and hydrological data, and identify data points that are far from the cluster center as abnormal points, triggering the adjustment of the weight of meteorological and hydrological ship sensor data queues; (2) Design an abnormal target detection method based on image edge detection. The Canny edge detection algorithm is used to detect the target edge in the image. If an abnormal target exists, the weight adjustment of the data queues of optical, AIS and radar ship sensors is triggered. The weight ratio is adjusted to optical > AIS > radar. (3) Weighting adjustment rules, , The adjusted weights The weights before adjustment. This refers to the proportion by which the weights are adjusted.

3. The method for scheduling maritime ship data transmission based on satellite internet according to claim 1, characterized in that, In step S2, the ship multi-sensor data queue management method includes the following steps: A multi-layered buffer mechanism is designed, with the data buffer threshold of the satellite broadband terminal being [value missing]. Set a data queue buffer threshold for the ship sensor data queue according to the aforementioned weights. ; Define the validity period of the arriving data packet. , The data packet deadline is carried when the data packet is generated, and it is handled according to the first-come, first-served principle. Queue delay, Transmission delay includes sending delay and propagation delay. The satellite-to-ground link channel status is monitored, and the propagation delay is calculated. , Where c is the distance between the satellite broadband terminal and the satellite, and c is the speed of light; The design of the packet loss decision mechanism involves using a top-down approach to select the data packet with the shortest validity period for packet loss when the length of the ship's sensor data queue exceeds the data queue buffer threshold, with the excess space size as a constraint. This is achieved by constructing a relaxed decision graph.

4. The method for scheduling maritime ship data transmission based on satellite internet according to claim 1, characterized in that, In S3, the data potential output performance function is defined. , The weights of the ship sensor data queue n are: The validity period of data packet k in the ship sensor data queue n.

5. The method for scheduling maritime ship data transmission based on satellite internet according to claim 1, characterized in that, In step S3, modeling the ship's multi-sensor data queue scheduling process as a hierarchical Markov decision process includes the following steps: The design incorporates a hierarchical state space, including a high-level state space and a low-level state space. The high-level state space refers to the features related to the global scheduling target of the ship's multi-sensor data queue. This includes the overflow risk level of each ship sensor data queue. The potential output efficiency of the average header data packet of each ship sensor data queue Data transmission rate at time slot t The lower-level state space selects the local features of each ship sensor data queue set according to the higher-level strategy output by the higher-level state space. Includes the current length of the selected ship sensor data queue. Potential output performance of header data packets Historical transmission count and number of admissions ; The design incorporates a hierarchical action space, including a high-level action space and a low-level action space. The high-level action space is used to select the global scheduling strategy for the ship's multi-sensor data queue. In emergency priority mode, only queues with an overflow risk level of warning or emergency are allowed to be processed. In performance-priority mode, select the queues with the highest potential output performance of the current header data packet, i.e. The underlying action space is used to select the specific sensor data queue to be transmitted under the constraints of the higher-level policy. ; The design incorporates a hierarchical reward function, including a high-level reward function and a low-level reward function. The high-level reward function aims to maximize long-term cumulative efficiency while controlling queue overflow. The calculation method is as follows: Among them, the fairness penalty coefficient The unfairness level increases dynamically with the output of the queue; the underlying reward function calculates the data packet performance of the currently selected ship multi-sensor data queue and deducts the corresponding fairness penalty, i.e. ,in This is an indicator function that ensures local decisions are consistent with the global objective.

6. The hierarchical Markov decision process of the satellite internet-based maritime ship data transmission scheduling method according to claim 5, characterized in that, The execution algorithm of the hierarchical Markov decision process is the ship multi-sensor data queue scheduling algorithm based on the improved D3QN, which includes the following steps: Initialize high-level main network parameters and underlying main network parameters ,in , For the branch weights of the value function, , Assign weights to the action advantage branch; initialize the high-level target network parameters respectively. and underlying target network parameters Clear the experience replay pool D, and set the probability of exploring higher levels. and the probability of exploring the bottom layer and attenuation rate Define the batch sample size B and the synchronization round L; Initialize the satellite position and the satellite broadband terminal position, obtain the service window time TW, and determine the number and step size of scheduling time slots; For each time slot t, the agent determines the current high-level state in the high-level state space. The current high-level action in the high-level action space is selected using an ε-greedy strategy. The underlying action space is generated according to the high-level selection strategy, and in the underlying state... Based on this, the Q-value of each ship sensor data queue is calculated using the underlying main network. The ship sensor data queue with the largest Q value is selected for transmission. ; After executing the current underlying action in the underlying action space, the actual number of data transmitted is calculated based on the data transmission rate at time slot t, and the length of the ship sensor data queue is updated accordingly. ; Calculate the high-level rewards in real time and the underlying reward and transfer samples and Stored in the aforementioned experience replay pool D; Randomly sample batch B from the experience replay pool D and calculate the Q value of the high-level target. The underlying target Q value , in This is the discount factor for the aforementioned high-level rewards. The high-level loss function is calculated using the discount factor of the underlying reward. Low-level loss function The AdamW optimizer updates the higher-layer main network parameters and the lower-layer main network parameters; every L steps, the higher-layer target network parameters are synchronized via soft updates. The underlying target network parameters ; Repeat the above process until the maximum number of iterations is reached. Finally, under the constraints of preventing queue overflow and ensuring the fairness of queue transmission, generate the ship multi-sensor data transmission scheme with the highest long-term data output efficiency within the service window time.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images