Mesoscale vortex observation method based on AUG reinforcement learning

Through AUG reinforcement learning and Gaussian process regression model, high-precision, high-resolution and dynamic adaptive observation of mesoscale vortices are achieved, solving the problems of low spatial and temporal resolution and insufficient path planning in traditional methods, and significantly improving mesoscale Observation accuracy and efficiency of the three-dimensional structure of the vortex.

CN119066982APending Publication Date: 2024-12-03TIANJIN UNIV +1
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411291556.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-14
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing mesoscale vortex observation methods are difficult to meet the needs of high-precision, high-resolution and dynamic adaptive path planning, and cannot comprehensively and continuously obtain the spatial and temporal evolution process of mesoscale vortex, and the spatial and temporal resolution is low, resulting in three-dimensional temperature. There are blind spots in understanding salt structure.

Method used

Using the mesoscale vortex observation method based on AUG reinforcement learning, through autonomous underwater gliders, combined with Gaussian process regression model and value iteration algorithm of reinforcement learning, dynamic adaptive path planning and three-dimensional structure data acquisition are realized, and the observation path is adjusted in real time. To track the changes in the vortex.

Benefits of technology

It realizes efficient three-dimensional ocean temperature field gridization, precise path planning and dynamic adaptive sampling, improves the observation accuracy and efficiency of the three-dimensional structure of mesoscale vortex, and significantly improves the detailed information of the vortex center, boundary and temperature-salt structure. to adapt to complex dynamic marine environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066982A_ABST
    Figure CN119066982A_ABST
Patent Text Reader

Abstract

The invention provides a mesoscale vortex observation method based on AUG reinforcement learning, and the method mainly comprises the following steps: firstly, determining a target observation region, obtaining the vortex type, boundary and center position information of a mesoscale vortex, arranging a plurality of autonomous underwater gliders in the region according to a network layout, and setting related parameters and sensors; secondly, initializing a regional temperature field, meshing the regional temperature field, and selecting an initial course angle for each underwater glider; the underwater glider transmits data to the shore-based platform after executing profile observation, and reconstructs a temperature field through a Gaussian process regression model in combination with historical data; selecting a next course angle based on a value iterative algorithm of reinforcement learning; and observing and path planning are carried out circularly until the task is completed. Through collaborative observation and intelligent path planning, the precision and efficiency of mesoscale vortex observation are improved, and the method has a wide application prospect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of ocean observation, and in particular relates to a mesoscale eddy observation method based on AUG reinforcement learning. Background Art

[0002] Mesoscale eddies are widespread mesoscale ocean phenomena in the ocean, with a spatial scale of 50-500 kilometers and a life cycle ranging from a few days to hundreds of days. They are important carriers of the transport of heat, salinity, mass, plankton, dissolved oxygen and other biochemical substances in the global ocean, and have a significant impact on marine ecosystems and global climate change. However, due to the lack of systematic high-resolution observation data, our understanding of the three-dimensional temperature and salinity structure of mesoscale eddies is still incomplete. Although traditional observation methods can provide an overall overview, it is difficult to accurately capture the refined dynamic evolution of mesoscale eddies. Therefore, observation methods with higher temporal and spatial resolution have become a key need for scientific research.

[0003] Existing mesoscale eddy observation methods have multiple limitations and are difficult to meet the needs of scientific research for high-precision, high-resolution data. First, although satellite remote sensing can provide a wide range of ocean surface observation information, it mainly obtains two-dimensional surface data and cannot penetrate into the vertical structure of the ocean. The resolution is low and it is difficult to capture subtle changes in mesoscale eddies. Although large submersibles / buoys and ocean surface profile buoys can obtain more accurate temperature and salinity data at a certain depth, these observation devices are usually fixed in a certain position and cannot dynamically follow the movement of the vortex. The spatial coverage is limited and it is difficult to obtain the overall three-dimensional structure of the mesoscale eddy. In addition, the maintenance cost of equipment such as submersibles / buoys is high, the deployment is complex, and it is difficult to obtain updated data frequently.

[0004] Secondly, most of the existing observation methods do not take into account the dynamic characteristics of mesoscale vortices and cannot adjust the observation path in real time. For example, although underwater gliders can conduct long-term observations at multiple depths, the existing mission planning often presets fixed paths and lacks the ability to adjust the path online based on real-time observation data, and cannot respond to the movement and structural changes of vortices in a timely manner. Therefore, in the context of rapid changes in mesoscale vortices, traditional observation methods are difficult to adapt to such changes and cannot fully and continuously obtain their spatiotemporal evolution process. In addition, the spatiotemporal resolution of current technical means is still low, and it is impossible to accurately reflect the vortex characteristics at different depths and time points, resulting in blind spots in the understanding of the refined three-dimensional temperature-salinity structure of mesoscale vortices, and the observation results have great uncertainty.

[0005] Therefore, it is urgent to propose a new mesoscale eddy network observation method based on underwater gliders to promote the understanding of related research on ocean circulation, reveal the three-dimensional temperature and salinity structure of the reconstructed mesoscale eddies, especially their dynamic evolution process over time, and provide a better observation plan for my country to carry out high-resolution ocean observations. Summary of the invention

[0006] In response to the problems existing in the prior art, the present invention provides a mesoscale vortex observation method based on AUG reinforcement learning, which aims to solve the technical problems of high-precision, high-resolution, dynamic adaptive path planning and three-dimensional structure data acquisition in mesoscale vortex observation.

[0007] The present invention is implemented as follows: a mesoscale eddy observation method based on AUG reinforcement learning, characterized in that it includes the following steps:

[0008] Step A: Determine the target observation area and obtain relevant information about the mesoscale vortex in the area, including the vortex type, vortex boundary and vortex center position; deploy multiple autonomous underwater gliders in the target area according to a predetermined network layout formation; set the relevant operating parameters of each main underwater glider and configure the sensors used for the observation mission;

[0009] Step B: Initialize the temperature field of the target observation area and discretize it into grid points with a predetermined resolution; select the initial heading angle of each autonomous underwater glider within the feasible domain according to the movement direction of the mesoscale vortex and start the observation mission;

[0010] Step C: The autonomous underwater glider dives according to a predetermined heading angle to conduct profile observations; after completing a profile navigation, the collected data is transmitted to the shore-based platform; the data is combined with the historical profile data as training data to prepare for the Gaussian process regression model in the subsequent stage to reconstruct the temperature field of the target area;

[0011] Step D: Reconstruct the temperature field of the target area using the Gaussian process regression model and update the temperature estimate and its uncertainty at each grid point; select the heading angle of the next profile using the value iteration algorithm based on reinforcement learning;

[0012] Step E: The autonomous underwater glider performs observation and sampling according to steps A to D; determines whether to terminate the observation mission according to a predetermined mission completion judgment standard; if the observation mission is not completed, repeats steps B to D until the observation mission is completed.

[0013] In the above technical scheme, preferably, in step A, mesoscale eddies are identified based on satellite ocean surface height anomaly data of the target observation area, including determining the latitude and longitude range of the target sea area, obtaining near-real-time SLA data of the target sea area, and using the outermost closed contour method to identify eddies, and by identifying the outermost closed contour in the SLA data, the type, boundary and center position information of the eddy are determined.

[0014] In the above technical solution, preferably, in step B, the temperature field of the target observation area is discretized at a resolution of 0.001° based on the two-dimensional latitude and longitude directions to reduce the position error of the autonomous underwater glider during the mission observation; in the gridding of the three-dimensional sea area, the depth resolution in the vertical direction is increased, and the resolution in the depth direction is non-uniformly distributed, showing the characteristics of dense top and sparse bottom; the vertical depth range refers to the HYCOM forecast data, and within the depth range of 0 to 800m, the depth layers are: 0.0m, 2.0m, 4.0m, 5.0m, 7.0m, 8.0m, 9.0m, 10.0m, 11.0m, 12.0m, 13.0m, 14.0m, 15.0m, 16.0m, 17.0m, 18.0m, 19.0m, 20.0m, 21.0m, 23.0m, 24.0m, 25.0m, 26.0m, 27.0m, 28.0m, 29.0m, 30.0m, 31.0m, 36.0m, 37.0m, 38.0m, 39.0m, 40.0m, 41.0m, 42.0m, 43.0m, 44.0m, 45.0m, 46.0m, 47.0m, 48.0m, 49.0m, 50.0m, 51 ... .0m, 6.0m, 8.0m, 10.0m, 12.0m, 15.0m, 20.0m, 25.0m, 30.0m, 35.0m, 40.0m, 45.0m, 50.0m, 60.0m, 70.0m, 80.0m , 90.0m, 100.0m, 125.0m, 150.0m, 200.0m, 250.0m, 300.0m, 350.0m, 400.0m, 500.0m, 600.0m, 700.0m, 800.0m.

[0015] In the above technical solution, preferably, in step C, the autonomous underwater glider obtains temperature data at the trajectory points through the onboard CTD sensor; the position information of the water entry point and water exit point of each section is obtained through a satellite positioning system.

[0016] In the above technical solution, preferably, in the step D, the heading angle is selected based on a value iteration algorithm, which specifically includes the following steps:

[0017] Step D1: When the autonomous underwater glider completes a profile observation and reaches the water outlet, the data collected by the profile and all historical data are used as training data. These data are used to reconstruct the temperature field of the target observation area through the Gaussian process regression model, and the standard deviation corresponding to each grid point is calculated;

[0018] Step D2: Based on the strategy evaluation method in reinforcement learning, the value function corresponding to the current strategy is calculated, and the strategy is improved according to the value function to obtain a better heading angle selection strategy;

[0019] Step D3: Calculate the difference between the value function before and after the strategy improvement;

[0020] Step D4: If the difference is less than the predetermined threshold, adopt the strategy corresponding to the current state; otherwise, repeat steps D2 and D3 until the conditions are met.

[0021] In the above technical solution, preferably, in step D1, the predicted value μ obtained by running the Gaussian process regression model is * and variance Σ * For the following:

[0022]

[0023] In the above technical solution, preferably, the value of the value function at the k+1th time during the strategy evaluation in step D2 can be calculated from the value at the kth time and the corresponding reward:

[0024]

[0025] In the above technical solution, preferably, in step D2, based on the strategy evaluation method in reinforcement learning, once the value function changes, the strategy is improved immediately to accelerate convergence and obtain a better heading angle selection strategy.

[0026] In the above technical solution, preferably, when reinforcement learning is used in step D2, in order to make the state space approximation more accurate and speed up the calculation, the state space is improved to obtain the local state space S′, and the formula is:

[0027] S′={(x i y i )|(x i y i )∈S,(x i -x0) 2 +(y i -y0) 2 ≤(V*d) 2}.

[0028] Advantages and effects

[0029] The present invention proposes a mesoscale vortex observation method based on an autonomous underwater glider (AUG) and reinforcement learning. Through the collaborative observation of multiple AUGs, long-time series and wide-range adaptive data sampling at multiple depth levels is performed. During the task, the AUG reconstructs the temperature field of the area to be observed through the real-time acquired sampling data combined with the Gaussian process regression (GPR) model, and then based on the temperature field data, the value iteration algorithm in reinforcement learning is used to select the heading angle, realize the online adjustment and optimization of the underwater glider path, and drive the glider to complete the three-dimensional observation task of the mesoscale vortex. The following are the advantages and beneficial effects of this method:

[0030] 1. Efficient three-dimensional ocean temperature field gridding and precise path planning: By introducing gridding technology, the temperature field of the sea area to be observed is divided into fine three-dimensional grids, and discretized according to the specified resolution based on the position information of the AUG, so that the temperature field can be reconstructed according to the real-time data of each grid point during the path planning process. On this basis, combined with the Gaussian process regression model and the value iteration algorithm of reinforcement learning, the path planning of each voyage is made more intelligent and adaptive, and the heading angle can be dynamically adjusted to achieve efficient and comprehensive observation of the three-dimensional structure of the mesoscale vortex.

[0031] 2. Dynamic adaptive sampling and real-time online adjustment: The AUG in the method of the present invention can adjust the observation path online according to the real-time observation data transmitted back, ensuring that the observation task can continuously track the movement changes of the mesoscale vortex. This dynamic adaptive capability enables the glider to flexibly adjust the navigation trajectory according to the changes in the mesoscale vortex, ensuring that the acquired data covers the key areas of the entire vortex, and realizing real-time, dynamic, and high-resolution observation of the mesoscale vortex.

[0032] 3. Gaussian process regression improves the accuracy of observation data: By inputting the historical and current observation data collected by AUG into the Gaussian process regression model, the accuracy of temperature field reconstruction can be effectively improved. Especially when multiple AUGs are used for collaborative observation, the sampling data of each glider can be comprehensively utilized to achieve high-precision prediction and reconstruction of the temperature field in the area to be observed, thereby making path planning more accurate.

[0033] 4. Self-optimization path selection by reinforcement learning: The reinforcement learning algorithm in the present invention can gradually optimize the heading angle selection of AUG according to the evaluation results of different navigation strategies. By comparing the value function changes of the previous and next strategies through the value iteration algorithm, AUG can further improve its strategy after each navigation, so as to converge to the optimal path faster and maximize the observation efficiency and coverage.

[0034] 5. Realize intelligent observation of the three-dimensional structure of mesoscale vortices: Compared with traditional observation methods, the present invention can significantly improve the observation accuracy of the three-dimensional structure of mesoscale vortices, including detailed information on the vortex center, vortex boundary, and temperature-salinity structure. Through the collaborative operation of multiple AUGs, the evolution process of mesoscale vortices can be revealed in an all-round and three-dimensional manner, especially the observation of the laws of material transmission such as heat, salt, and plankton, which has extremely high scientific value.

[0035] 6. Promote the automation and flexibility of mesoscale eddy observation: This solution realizes the intelligence and automation of observation tasks through automated path planning and real-time data processing. Different from the traditional manually planned path, this solution can flexibly adjust the observation strategy according to real-time data, greatly improve the efficiency and flexibility of observation, adapt to the complex and dynamically changing ocean environment, and provide technical support for the study of ocean mesoscale eddies.

[0036] 7. Wide application prospects and scientific research value: This technical solution can not only be applied to typical observation tasks of mesoscale eddies, but can also be extended to the observation of other ocean phenomena, promoting the development of marine scientific research. By revealing the generation and evolution of mesoscale eddies and their impact on marine ecosystems, this solution provides powerful tools and support for global marine climate research and ecological monitoring.

[0037] In summary, the present invention significantly improves the observation accuracy, efficiency and intelligence level in the observation of ocean mesoscale eddies, can effectively solve the shortcomings of traditional observation methods, and has important scientific research and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the overall process of observation path planning of the method of the present invention;

[0039] Figure 2 A schematic diagram of an observation path in an implementation example of the method of the present invention;

[0040] Figure 3 A schematic diagram of data exchange in an implementation example of the method of the present invention;

[0041] Figure 4 It is the temperature field reconstructed from the data obtained by observation sampling along the planned path in the implementation example of the method of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0043] In order to solve the technical problems of high-precision, high-resolution, dynamic adaptive path planning and three-dimensional structure data acquisition in mesoscale vortex observation, the present invention provides a mesoscale vortex observation method based on AUG reinforcement learning. The present invention realizes high-precision, dynamic adaptive, intelligent three-dimensional structure observation of mesoscale vortices through collaborative observation of multiple autonomous underwater gliders and reinforcement learning algorithm, improves the efficiency, accuracy and flexibility of observation, and has broad application prospects and scientific research value. In order to further illustrate the structure of the present invention, the detailed description is as follows in conjunction with the attached drawings:

[0044] This embodiment proposes a mesoscale eddy observation method based on AUG reinforcement learning, which uses multiple AUGs to perform three-dimensional observation and data sampling of mesoscale eddies, including the following steps:

[0045] Step A: Determine the observation area, obtain relevant information such as the vortex type, vortex boundary and vortex center position of the mesoscale vortex to be observed, form a formation of multiple AUGs according to a certain network layout and deploy them in the area, set the relevant parameters of the underwater glider, and configure the sensors required for the observation mission.

[0046] The solution of the present invention is based on the networking and collaborative observation of multiple autonomous underwater gliders (AUGs). The AUGs are equipped with a variety of different types of sensors, including CTD, turbulence meter and ADCP. CTD (conductivity, temperature, depth sensor) is used to measure the salinity, temperature and depth of seawater; turbulence meter is used to detect the turbulence intensity and flow structure in the ocean; ADCP (acoustic Doppler current profiler) is used to measure the velocity and direction of the water body. Taking advantage of the long endurance, wide working range and repeated use of AUG, it is equipped with sensors such as CTD, and through the collaborative observation of multiple units, it performs long-term, wide-range adaptive data sampling and observation tasks of multiple depth layers in the mesoscale vortex. Based on the characteristics of the mesoscale vortex itself, after each AUG completes a profile observation, it combines the profile sampling data and historical data during the waiting stage on the water surface, and performs online path adjustment based on the reinforcement learning method, thereby realizing high-intelligence, high-efficiency, long-term three-dimensional structural observation of the mesoscale vortex. The overall planning solution flow chart is as follows Figure 1 shown.

[0047] In this embodiment, specifically, a preliminary observation area is first determined, and relevant information such as the vortex type, vortex boundary, and vortex center position of the mesoscale vortex to be observed is determined based on the satellite remote sensing altimeter data, and a circumscribed rectangle of the mesoscale vortex to be observed is made. Then, a mother ship is equipped with multiple AUGs to go to the designated area, and the AUGs are respectively deployed at four vertices. This embodiment is illustrated by taking the deployment of four AUGs as an example.

[0048] In step A, mesoscale eddies are identified based on satellite sea surface height anomaly (SLA) data of the target sea area: the latitude and longitude range of the target sea area is determined, the SLA value is obtained based on the near-real-time data set on CMEMS, and then the outermost closed contour method is used to identify the mesoscale eddies, that is, by finding the outermost closed contour of the SLA, the eddy type, eddy boundary, eddy center position and other related information of the eddy are determined. Specifically:

[0049] Step A1: Determine the latitude and longitude range of the target sea area, and obtain the sea level anomaly (SLA) based on the near-real-time dataset Global OceanGridded L 4Sea Surface Heights And Derived Variables Nrt on CMEMS;

[0050] Step A2: According to SLA, the mesoscale eddy is identified by using the outermost closed contour method, that is, in the geostrophic equilibrium state, the outermost closed contour of SLA is found to be approximately parallel to the streamline contour, thereby replacing the latter. While saving the amount of calculation, the observed signal changes are retained to the greatest extent, and the error caused by artificially determined thresholds is reduced, thereby determining the vortex type (-1 represents a cyclonic vortex, 1 represents an anticyclonic vortex), vortex boundary, vortex center position and other related information.

[0051] Step B: Initialize the temperature field of the area to be observed, discretize it into grid points of specified resolution, select the initial heading angle of each AUG in the feasible domain according to the direction of motion of the mesoscale eddy, and start the observation task. Specifically, carry out the three-dimensional temperature structure tracking observation of the mesoscale eddy based on reinforcement learning. The AUG of this implementation example is equipped with a CTD sensor and is set according to the parameters shown in Table 1.

[0052] Table 1

[0053]

[0054] After the parameters are set, start observing according to the planned path. The final path diagram is as follows: Figure 2 shown.

[0055] The specific steps are:

[0056] Initialize the area to be observed:

[0057] Generally, the temperature field of the entire area is expressed by the temperature values ​​of a set of discrete position points, which are obtained by discretizing the longitude and latitude of the area according to a certain resolution. In this embodiment, the two-dimensional ocean temperature field is divided according to 0.001° in the longitude and latitude directions. The purpose is to make it closer to the real ocean environment to reduce the position error of the underwater glider in adaptive sampling. The smaller the grid division, the more discrete points it contains, the greater the amount of calculation, but it will be more accurate; for the gridding of three-dimensional sea areas, it is only necessary to add a resolution in the depth direction. Considering that the temperature of shallow seawater in the ocean is unstable and the temperature of deep seawater is relatively stable, the resolution in the depth direction is usually non-uniform and has the characteristics of dense on top and sparse on the bottom. A series of depth layers are used to represent the resolution in the vertical direction. The vertical depth stratification of the HYCOM forecast data is based on the following two reasons: (a) the CTD sensor of the underwater glider will not be turned on at the ocean surface, so the temperature sampling data near the ocean surface is not accurate; (b) the target depth of the underwater glider is generally set at about 800-900m. The vertical depth range of the ocean temperature field mainly studied in this paper is between 50 and 800m, with a total of 17 standard layers - 0.0, 2.0, 4. 0,6.0,8.0,10.0,12.0,15.0,20.0,25.0,30.0,35.0,40.0,45.0,50.0,60.0,70.0,80.0,90.0,100.0,125.0,150.0,200.0,250.0,300.0,350.0,400.0,500.0,600.0,700.0,800.0.

[0058] Select the initial heading angle:

[0059] Since the horizontal structure of a real mesoscale vortex is always asymmetric, the coverage of the reconstructed vortex temperature field will be improved when sampling is performed along the direction of the vortex movement. The vortex in this embodiment moves from east to west, so the initial heading angles of UG1 and UG2 are selected to be eastward, and the initial heading angles of UG3 and UG4 are selected to be westward.

[0060] Step C: The underwater glider dives at a specified heading angle to conduct sampling and profile observation. After completing a profile navigation, the collected data is transmitted back to the shore-based platform, and it and the historical profile data are used as training data to prepare for the next stage of Gaussian process regression (GPR) model to reconstruct the temperature field of the area to be observed.

[0061] In this embodiment, after each underwater glider completes the sampling of a certain profile and is in the waiting stage on the water surface, it starts to transmit the temperature, salinity, depth and other data collected by the profile to the shore-based platform via satellite. The collected data is extracted from each depth layer according to the 17 discrete standard depth layers, including position, temperature, salinity, depth and other information. After processing the sampled data, the shore-based platform uses it for GPR model operation. The data exchange process is as follows: Figure 3 shown.

[0062] Step D: The GPR model is run to reconstruct the temperature field of the area to be observed and update the standard deviation at each grid point. The heading angle of the next profile is selected through the value iteration algorithm in reinforcement learning.

[0063] After processing the sampled data sent back by all underwater gliders, the shore-based platform uses the sampled data and historical data of the current profile of all underwater gliders as training data, reconstructs the temperature field of the measured area and calculates the standard deviation corresponding to each grid point. Taking the reconstruction of the temperature field 50m depth layer as an example, the reconstruction effect is as follows: Figure 4 As shown in the figure, (a) is the real temperature field and (b) is the reconstructed temperature field.

[0064] Based on the strategy evaluation in reinforcement learning, the value function corresponding to the current strategy is calculated, and the strategy is improved. A better strategy is obtained by using the value function. It is iterated according to the selected strategy, and the difference in the value function before and after the iteration is calculated. If the difference is less than the set threshold, the strategy corresponding to the current state is returned, otherwise the above steps are repeated until the conditions are met. Specifically:

[0065] Step D1: When the AUG completes a profile and reaches the water outlet, the data collected by the profile and all historical data are used as training data. The data is used to reconstruct the temperature field of the area to be measured through Gaussian process regression (GPR) and calculate the standard deviation corresponding to each grid point. The GPR model reconstructs the temperature field in the following way:

[0066] Input the training data into the GPR model, according to the formula The reconstructed temperature field can be obtained, and then according to The variance and standard deviation of each grid point can be obtained.

[0067] Step D2: Calculate the value function corresponding to the current strategy based on the strategy evaluation in reinforcement learning, and improve the strategy to obtain a better strategy using the value function. In this step, the following methods are used to obtain a better strategy:

[0068] pass The value of the k+1th value function can be calculated if v k+1 (s) = v k(s′), then stop training, otherwise according to the formula Find a better strategy.

[0069] Step D3: Iterate according to the selected strategy and calculate the difference in value function before and after iteration;

[0070] Step D4: If the difference in step D3 is less than the threshold, return the strategy corresponding to the current state, otherwise repeat steps D2 and D3 until the condition is met. In this step, the smaller the threshold, the more training iterations, but the longer the time spent. Considering all factors, it can be set to 1×10 -4 .

[0071] Reinforcement learning is specifically done in the following ways:

[0072] The basic theoretical framework of the reinforcement learning algorithm is the Markov decision process (MDP). We first use MDP to model the underwater glider observation process. MDP can be described by a 5-tuple (S, A, P, R, γ). The problem of underwater glider observing mesoscale vortices by selecting a series of heading angles can be modeled as a sequential decision problem of MDP. Then, the value iteration algorithm is used to solve the optimal strategy and complete the selection of the optimal heading angle.

[0073] Step E: AUG performs observation sampling according to steps AD, and determines whether to end the observation task according to the task end judgment standard. If the observation task is ended, the sampling data is output; otherwise, steps BD are repeated until the observation task is completed.

[0074] The task completion criteria are as follows:

[0075] (1) Since the GPR model gives the standard deviation corresponding to each location point when reconstructing the temperature field, the average standard deviation (ASD) can be used to judge the quality of the reconstructed temperature field obtained by adaptive sampling in actual ocean temperature field observations. The calculation formula of ASD is as follows:

[0076]

[0077] If ASD is used as the judgment criterion, an ASD value needs to be determined in advance as the threshold. However, for different temperature fields, when ASD is the same, the corresponding root mean square error (RMSE) may not be the same. Therefore, this criterion is more suitable for temperature fields with observation experience, and is set by referring to the changes in ASD values ​​of historical observation results.

[0078] (2) In the simulation comparison of the sampling schemes, since the ocean temperature field information is known, the RMSE value of the reconstructed temperature field and the real temperature field can be calculated. In order to compare the advantages and disadvantages of different sampling schemes, the number of observation profiles can be fixed to compare the RMSE, or the number of observation profiles can be fixed to compare the RMSE. The present invention regards the number of observation profiles as the observation cost. The more the number of observation profiles, the higher the cost. However, for real sea trials, this criterion can only be determined by empirical values ​​like (1), so it is not applicable.

[0079] (3) Referring to the traditional sampling scheme, when the underwater glider sampling track roughly covers the entire observation area, it is considered that the observation data can well reconstruct the target temperature field and the observation can be ended. This scheme is more suitable for the real ocean temperature field observation task.

[0080] In summary, in actual sea trials, we can decide whether to terminate the observation by observing the coverage of the sampling trajectory in the entire observation area.

[0081] The above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, a person skilled in the art should understand that the specific implementation mode of the present invention can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present invention, which should be included in the scope of the technical solution for protection of the present invention.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A mesoscale eddy observation method based on AUG reinforcement learning, It is characterized in that The following steps are involved: Step A: Determine the target observation area and obtain relevant information about the mesoscale vortex in the area, including the vortex type, vortex boundary and vortex center position; deploy multiple autonomous underwater gliders in the target area according to a predetermined network layout formation; Set relevant operating parameters of each host underwater glider and configure sensors for observation missions; Step B: Initialize the temperature field of the target observation area and discretize it into grid points with a predetermined resolution; select the initial heading angle of each autonomous underwater glider within the feasible domain according to the movement direction of the mesoscale vortex and start the observation mission; Step C: The autonomous underwater glider dives according to a predetermined heading angle to conduct profile observations; after completing a profile navigation, the collected data is transmitted to the shore-based platform; the data is combined with the historical profile data as training data to prepare for the Gaussian process regression model in the subsequent stage to reconstruct the temperature field of the target area; Step D: Reconstruct the temperature field of the target area using the Gaussian process regression model and update the temperature estimate and its uncertainty at each grid point; The heading angle of the next profile is selected based on the value iteration algorithm of reinforcement learning; Step E: The autonomous underwater glider performs observation and sampling according to steps A to D; and determines whether to terminate the observation mission according to a predetermined mission end judgment standard; If the observation task is not completed, repeat steps B to D until the observation task is completed.

2. The mesoscale eddy observation method based on AUG reinforcement learning according to claim 1, It is characterized in that In the step A, mesoscale eddies are identified based on satellite ocean surface height anomaly data of the target observation area, including determining the latitude and longitude range of the target sea area, obtaining near-real-time SLA data of the target sea area, and identifying eddies using the outermost closed contour method. The type, boundary and center position information of the eddies are determined by identifying the outermost closed contours in the SLA data.

3. The mesoscale eddy observation method based on AUG reinforcement learning according to claim 1, It is characterized in that In the step B, the temperature field of the target observation area is discretized at a resolution of 0.001° based on the two-dimensional latitude and longitude directions to reduce the position error of the autonomous underwater glider during the mission observation process; in the gridding of the three-dimensional sea area, the depth resolution in the vertical direction is increased, and the resolution in the depth direction is non-uniformly distributed, showing the characteristics of dense top and sparse bottom; the vertical depth range refers to the HYCOM forecast data, and within the depth range of 0 to 800m, the depth layers are: 0.0m, 2.0m, 4.0m, 6.0 m, 8.0m, 10.0m, 12.0m, 15.0m, 20.0m, 25.0m, 30.0m, 35.0m, 40.0m, 45.0m, 50.0m, 60.0m, 70.0m, 80.0m, 90 .0m, 100.0m, 125.0m, 150.0m, 200.0m, 250.0m, 300.0m, 350.0m, 400.0m, 500.0m, 600.0m, 700.0m, 800.0m.

4. The mesoscale eddy observation method based on AUG reinforcement learning according to claim 1, It is characterized in that In step C, the autonomous underwater glider obtains temperature data at the trajectory points through the CTD sensor carried by the vehicle; the position information of the water entry point and the water exit point of each section is obtained through the satellite positioning system.

5. The mesoscale eddy observation method based on AUG reinforcement learning according to claim 1, It is characterized in that In the step D, the heading angle is selected based on the value iteration algorithm, which specifically includes the following steps: Step D1: When the autonomous underwater glider completes a profile observation and reaches the water outlet, the data collected by the profile and all historical data are used as training data. These data are used to reconstruct the temperature field of the target observation area through the Gaussian process regression model, and the standard deviation corresponding to each grid point is calculated; Step D2: Based on the strategy evaluation method in reinforcement learning, the value function corresponding to the current strategy is calculated, and the strategy is improved according to the value function to obtain a better heading angle selection strategy; Step D3: Calculate the difference between the value function before and after the strategy improvement; Step D4: If the difference is less than the predetermined threshold, adopt the strategy corresponding to the current state; otherwise, repeat steps D2 and D3 until the conditions are met.

6. The mesoscale eddy observation method based on AUG reinforcement learning according to claim 5, It is characterized in that In step D1, the predicted value μ obtained by running the Gaussian process regression model * and variance Σ * For the following:

7. The mesoscale eddy observation method based on AUG reinforcement learning according to claim 5, It is characterized in that The value of the value function at the k+1th time during the strategy evaluation in step D2 can be calculated from the value at the kth time and the corresponding reward:

8. The mesoscale eddy observation method based on AUG reinforcement learning according to claim 5, It is characterized in that In step D2, based on the strategy evaluation method in reinforcement learning, once the value function changes, the strategy is improved immediately to accelerate convergence and obtain a better heading angle selection strategy.

9. The mesoscale eddy observation method based on AUG reinforcement learning according to claim 5, It is characterized in that When reinforcement learning is used in step D2, in order to make the state space approximation more accurate and speed up the calculation, the state space is improved to obtain the local state space S′, the formula is: S′={(x i y i )|(x i y i )∈S,(x i -x 0 ) 2 +(y i -y 0 ) 2 ≤(V*d) 2 }。

Citation Information

Cited By

  • Method for identifying turbulence influence temperature-salinity-depth data of portable underwater glider

    CN119474888A

  • A Method for Identifying Temperature-Salinity-Depth Data Affected by Turbulence in a Portable Underwater Glider

    CN119474888B

  • Method and system for observing sub-mesoscale structure of Yellow Sea cold water mass

    CN121498636A

  • Sea-ground-air-space integrated ocean intelligent collaborative three-dimensional networking observation method

    CN121655472A

  • Underwater glider trajectory tracking control method based on flow field reconstruction and electronic equipment

    CN122111076A