Path optimization method, system and equipment for underwater detection task and medium

Abstract

The invention discloses a path optimization method, system and device for an underwater detection task and a medium, and the method comprises the steps: obtaining underwater detection data when an operation unit carries out the underwater detection task, a local reward value self-generation model and a frozen prior global reward value grid network; inputting the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network; performing network reward value analysis processing according to the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network; and optimizing a task path of the underwater detection task according to the target reward value grid network to obtain an optimized task path. The method can effectively improve the path planning and obstacle avoidance capability of the operation unit in a complex water area. The invention relates to the technical field of path planning.

Description

Technical Field

[0001] The present invention relates to the field of path planning technology, and in particular to a path optimization method, system, equipment and medium for underwater detection tasks. Background Art

[0002] In underwater detection missions based on unmanned operating units, the complex underwater environment, unknown terrain, dynamic obstacles and other factors seriously threaten the safety of the operating units. During the detection process, the operating units adjust task allocation and paths in real time according to the detection results, which is an important means to ensure the safety and efficiency of the operating units.

[0003] At present, in underwater detection missions, relevant technologies usually adopt deep reinforcement learning technology, which trains operating units (such as underwater robots) based on the principle of continuous trial and error, so that the operating units can autonomously adjust their actions based on environmental observation results. This method often requires manual setting of reward items, and the path planning and obstacle avoidance capabilities of the operating units in complex waters are unsatisfactory.

[0004] Therefore, the problems existing in related technologies still need to be solved and optimized urgently. Summary of the Invention

[0005] The purpose of the present invention is to solve one of the technical problems existing in the related art to at least a certain extent.

[0006] To this end, an object of an embodiment of the present invention is to provide a path optimization method, system, device and medium for underwater detection tasks, wherein the method can effectively improve the path planning and obstacle avoidance capabilities of the operating unit in complex waters.

[0007] In order to achieve the above technical objectives, the technical solutions adopted in the embodiments of the present application include:

[0008] In a first aspect, an embodiment of the present application provides a path optimization method for an underwater detection mission, comprising:

[0009] Obtain underwater detection data when the operating unit performs underwater detection tasks, as well as a local reward value self-generation model and a frozen prior global reward value grid network;

[0010] Inputting the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network;

[0011] Performing network reward value analysis processing based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network;

[0012] The task path of the underwater detection task is optimized according to the target reward value grid network to obtain an optimized task path.

[0013] In addition, the method according to the above embodiment of the present application may also have the following additional technical features:

[0014] Furthermore, in one embodiment of the present application, obtaining a local reward value self-generation model and a frozen prior global reward value grid network includes:

[0015] Obtain detection trajectory datasets and detection behavior datasets, and generate adversarial reinforcement learning models;

[0016] Training the generative adversarial reinforcement learning model according to the detection trajectory dataset to obtain the frozen prior global reward value grid network;

[0017] The generative adversarial reinforcement learning model is pre-trained according to the detection behavior dataset to obtain the local reward value self-generation model.

[0018] Furthermore, in one embodiment of the present application, the training of the generative adversarial inverse reinforcement learning model based on the detection trajectory dataset to obtain the frozen prior global reward value grid network includes:

[0019] Obtaining an intermediate global reward value grid network and a current detection trajectory of the detection trajectory dataset;

[0020] Inputting the current detection trajectory into the generator of the generative adversarial reinforcement learning model to obtain an intermediate sampling trajectory;

[0021] Inputting the intermediate sampling trajectory and the current detection trajectory into the discriminator of the generative adversarial reinforcement learning model to obtain trajectory discrimination data;

[0022] Performing a trajectory reward value analysis on the trajectory discrimination data to obtain a trajectory reward value corresponding to the current track detection trajectory;

[0023] The intermediate global reward value grid network is updated with a reward value grid according to the trajectory reward value to obtain the frozen prior global reward value grid network.

[0024] Furthermore, in one embodiment of the present application, pre-training the generative adversarial reinforcement learning model based on the detection behavior dataset to obtain the local reward value self-generation model includes:

[0025] Obtaining the current detection behavior of the detection behavior dataset;

[0026] Inputting the current detection behavior into the current generator of the generative adversarial reinforcement learning model to obtain an intermediate sampling behavior;

[0027] Inputting the intermediate sampled behavior into the current discriminator of the generative adversarial reinforcement learning model to update the discriminator parameters, thereby obtaining behavior discrimination data and an updated discriminator;

[0028] updating the generator parameters of the current generator according to the behavior discrimination data to obtain an updated generator;

[0029] The local reward value self-generation model is obtained according to the updated generator and the updated discriminator.

[0030] Furthermore, in one embodiment of the present application, inputting the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network includes:

[0031] Obtaining latitude and longitude data, heading data, and speed data of the operating unit;

[0032] Performing state pair construction processing on the underwater detection data according to the latitude and longitude data, the heading data, and the speed data to obtain a state-action pair of the operating unit;

[0033] Inputting the state-action pair into the local reward value self-generation model to obtain a plurality of local reward values;

[0034] The posterior local reward value grid network is obtained according to all the local reward values.

[0035] Furthermore, in one embodiment of the present application, performing network reward value analysis processing based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network includes:

[0036] Obtaining the current position of the operation unit;

[0037] According to the current position, grid-aligning the frozen prior global reward value grid network and the posterior local reward value grid network to obtain aligned prior global reward value grid network and posterior local reward value grid network;

[0038] According to the aligned prior global reward value grid network and the posterior local reward value grid network, repeated grid reward value multiplication calculations are performed to obtain the target reward value grid network.

[0039] In a second aspect, an embodiment of the present application provides a path optimization system for an underwater detection mission, comprising:

[0040] A first processing unit is used to obtain underwater detection data when the operating unit performs underwater detection tasks, as well as a local reward value self-generation model and a frozen prior global reward value grid network;

[0041] A second processing unit is configured to input the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network;

[0042] a third processing unit, configured to perform network reward value analysis processing based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network;

[0043] The fourth processing unit is configured to optimize the task path of the underwater detection task according to the target reward value grid network to obtain an optimized task path.

[0044] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0045] at least one processor;

[0046] at least one memory for storing at least one program;

[0047] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0048] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used to implement the above method when executed by the processor.

[0049] In a fifth aspect, an embodiment of the present application further provides a computer program product, comprising a computer program, which implements the steps of the above method when the computer program is processed and executed.

[0050] The advantages and benefits of this application will be partially given in the following description, and partially become apparent from the following description, or learned through practice of this application:

[0051] The embodiment of the present application discloses a path optimization method, system, device and medium for an underwater detection task, wherein the method obtains underwater detection data when an operating unit performs an underwater detection task, as well as a local reward value self-generation model and a frozen prior global reward value grid network; the underwater detection data is input into the local reward value self-generation model to obtain a posterior local reward value grid network; network reward value analysis and processing are performed based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network; based on the target reward value grid network, the task path of the underwater detection task is optimized to obtain an optimized task path. The method determines the target reward value grid network based on the frozen prior global reward value grid network and the posterior local reward value grid network, and then uses the target reward value grid network to optimize the task path of the underwater detection task. It can avoid the interference caused by artificially set rewards and effectively improve the path planning and obstacle avoidance capabilities of the operating unit in complex waters. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present application or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly expressing some embodiments of the technical solutions of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 A schematic diagram of a flow chart of a path optimization method for an underwater detection task provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of a framework of a path optimization system for underwater detection tasks provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. For the step numbers in the following embodiments, they are provided only for the convenience of explanation and are not intended to limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0058] At present, in underwater detection tasks, relevant technologies usually adopt deep reinforcement learning technology, which trains operating units (such as underwater robots) based on the principle of continuous trial and error, so that the operating units can autonomously adjust their actions based on environmental observation results. However, since deep reinforcement learning training requires a lot of manual parameter adjustment and iterative updates, its training efficiency is low and the training cost is high; and this method requires manual setting of reward items, which will inevitably bring about interference from human factors, making the path planning and obstacle avoidance capabilities of the operating units in complex waters unsatisfactory.

[0059] It should be noted that the above-mentioned related technologies are only used to assist in understanding the technical solutions of this application and do not mean that they belong to the disclosed prior art.

[0060] In view of this, an embodiment of the present invention provides a path optimization method, system, device and medium for underwater detection tasks, wherein the method determines a target reward value grid network based on a frozen prior global reward value grid network and a posterior local reward value grid network, and then uses the target reward value grid network to optimize the task path of the underwater detection task. The frozen prior global reward value grid network specifically learns the global reward value of the environment in advance by generating a strategy for imitating the detection trajectory through adversarial reinforcement learning, so that it can autonomously generate reward values ​​and avoid interference from human factors, thereby helping to reduce the training cost of the model and improve the training efficiency of the model, and effectively improve the path planning and obstacle avoidance capabilities of the operating unit in complex waters.

[0061] Reference Figure 1 In an embodiment of the present application, a path optimization method for an underwater detection task includes:

[0062] Step 110: Acquire underwater detection data when the operating unit performs underwater detection tasks, as well as a local reward value self-generation model and a frozen prior global reward value grid network;

[0063] In an embodiment of the present application, the operating unit may be an underwater robot, and the underwater detection task may specifically be any one of underwater terrain detection (such as seabed topography and address detection), underwater resource detection (such as underwater mineral resource detection), etc.; the underwater detection data may be data collected by the operating unit in an underwater environment using its own sonar sensors, visual sensors, etc.

[0064] It can be understood that the frozen prior global reward value grid network includes several global grids, each grid corresponding to a trajectory reward value of a trajectory position; and the local reward value self-generation model is used to generate behavior reward values ​​corresponding to behaviors in underwater detection data.

[0065] In some embodiments, obtaining a local reward value self-generation model and a frozen prior global reward value grid network includes:

[0066] Obtain detection trajectory datasets and detection behavior datasets, and generate adversarial reinforcement learning models;

[0067] In an embodiment of the present application, the detection trajectory data set can be a collection of historical detection trajectory data of the operating unit, each historical detection trajectory data is used to represent a detection trajectory of the operating unit, and each detection trajectory includes a number of trajectory points and the longitude and latitude of each trajectory point; the detection behavior data set can be a collection of historical detection behavior data of the operating unit, and each historical detection behavior data is used to represent the state and action of the operating unit; the generative adversarial inverse reinforcement learning model can specifically be a generative adversarial network GAN.

[0068] Training the generative adversarial reinforcement learning model according to the detection trajectory dataset to obtain the frozen prior global reward value grid network;

[0069] Furthermore, the generative adversarial reinforcement learning model is trained based on the detection trajectory dataset to obtain the frozen prior global reward value grid network, including:

[0070] Obtaining an intermediate global reward value grid network and a current detection trajectory of the detection trajectory dataset;

[0071] Inputting the current detection trajectory into the generator of the generative adversarial reinforcement learning model to obtain an intermediate sampling trajectory;

[0072] Inputting the intermediate sampling trajectory and the current detection trajectory into the discriminator of the generative adversarial reinforcement learning model to obtain trajectory discrimination data;

[0073] Performing a trajectory reward value analysis on the trajectory discrimination data to obtain a trajectory reward value corresponding to the current track detection trajectory;

[0074] The intermediate global reward value grid network is updated with a reward value grid according to the trajectory reward value to obtain the frozen prior global reward value grid network.

[0075] In an embodiment of the present application, for a detection trajectory dataset, a generator that generates an adversarial inverse reinforcement learning model is used to generate intermediate sampled trajectories similar to the detection trajectory, while a discriminator that generates an adversarial inverse reinforcement learning model is used to determine whether the input trajectory belongs to the detection trajectory. Specifically, the parameters of the adversarial inverse reinforcement learning model generator and discriminator can be initialized first, and then the generator and discriminator can be updated in a cyclic iterative manner, while the intermediate global reward value grid network is updated at the same time, and the final intermediate global reward value grid network is determined as the frozen prior global reward value grid network.

[0076] It is understandable that if the current detection trajectory is the first detection trajectory in the detection trajectory dataset, the intermediate global reward value grid network can be a pre-set blank grid network; alternatively, if the current detection trajectory is the second or later detection trajectory in the detection trajectory dataset, the intermediate global reward value grid network can be the updated intermediate global reward value grid network in the previous loop iteration process.

[0077] For example, during a certain loop iteration, the current detection trajectory during the loop iteration can be input into the generator of the generative adversarial inverse reinforcement learning model to obtain the intermediate sampling trajectory output by the generator; the intermediate sampling trajectory is then mixed with the current detection trajectory and input into the current discriminator of the generative adversarial inverse reinforcement learning model to obtain the trajectory discrimination data output by the discriminator, and based on the trajectory discrimination data, the parameters of the discriminator are updated using the Adam gradient to obtain the updated discriminator, which is used as the discriminator for the next loop iteration. After obtaining the trajectory discrimination data, the trajectory reward value can be calculated, which can be expressed as:

[0078]

[0079] Among them, Q(τ i ) is the trajectory reward value during the i-th loop iteration; ln(·) is the logarithmic function with the natural base e as the base; In the i-th loop iteration process, the discriminator w i Output trajectory discrimination data τ i .

[0080] It should be noted that for each loop iteration process, after the operation unit obtains the trajectory reward value of the loop iteration process, it can add and update the trajectory reward value to the intermediate global reward value grid network. Specifically, the trajectory reward value can be added to the intermediate global reward value grid network, the reward value grid corresponding to the current position of the operation unit, so as to obtain the updated intermediate global reward value grid network in the current loop iteration process.

[0081] Then, when the discriminator cannot determine whether the input trajectory is a detection trajectory or the current loop iteration process is the last loop iteration process, the final intermediate global reward value grid network is determined as the frozen prior global reward value grid network; alternatively, when the discriminator can still determine whether the input trajectory is a detection trajectory and the current loop iteration process is not the last loop iteration process, the intermediate global reward value grid network updated in the current loop iteration process can be determined as the intermediate global reward value grid network used in the next loop iteration process, and the current detection trajectory is updated based on the next detection trajectory of the detection trajectory dataset, and then the step of obtaining the intermediate global reward value grid network and the current detection trajectory of the detection trajectory dataset is returned to.

[0082] The generative adversarial reinforcement learning model is pre-trained according to the detection behavior dataset to obtain the local reward value self-generation model.

[0083] Furthermore, the pre-training of the generative adversarial reinforcement learning model based on the detection behavior dataset to obtain the local reward value self-generation model includes:

[0084] Obtaining the current detection behavior of the detection behavior dataset;

[0085] Inputting the current detection behavior into the current generator of the generative adversarial reinforcement learning model to obtain an intermediate sampling behavior;

[0086] Inputting the intermediate sampled behavior into the current discriminator of the generative adversarial reinforcement learning model to update the discriminator parameters, thereby obtaining behavior discrimination data and an updated discriminator;

[0087] updating the generator parameters of the current generator according to the behavior discrimination data to obtain an updated generator;

[0088] The local reward value self-generation model is obtained according to the updated generator and the updated discriminator.

[0089] In an embodiment of the present application, for a detection behavior dataset, a generator that generates an adversarial inverse reinforcement learning model is used to generate intermediate sampled behaviors similar to the detection behavior, and a discriminator that generates an adversarial inverse reinforcement learning model is used to determine whether the input behavior belongs to the detection behavior. Specifically, the parameters of the adversarial inverse reinforcement learning model generator and the discriminator can be initialized first, and then the generator and the discriminator are updated in a loop-iterative manner. When the discriminator cannot determine whether the input trajectory is a detection behavior or the current loop iteration process is the last loop iteration process, the local reward value self-generation model is determined based on the final generator and discriminator.

[0090] It can be understood that if the current detection behavior is the first detection behavior in the detection behavior data set, the generator can be an initialized generator, and the discriminator can be an initialized discriminator; or, if the current detection behavior is the second or later detection behavior in the detection behavior data set, the corresponding generator can be the generator updated during the previous loop iteration, and the corresponding discriminator can be the discriminator updated during the previous loop iteration.

[0091] For example, in a certain loop iteration process, the current detection behavior in the loop iteration process can be input into the generator of the generative adversarial inverse reinforcement learning model to obtain the intermediate sampling behavior output by the generator; then the intermediate sampling behavior is mixed with the current detection behavior and input into the current discriminator of the generative adversarial inverse reinforcement learning model to obtain the behavior discrimination data output by the discriminator, and based on the behavior discrimination data, the parameters of the discriminator are updated using Adam gradient to obtain an updated discriminator, which is used as the discriminator used in the next loop iteration process. The updated discriminator can be specifically expressed as:

[0092]

[0093] in, is the updated discriminator during the i-th loop iteration; is the expected value function when the input behavior is the intermediate sampling behavior; is ln(D w The gradient of (s,a) at parameter w; D w (s,a) is the behavior discrimination data; is the expected value function of the input behavior as the detection behavior.

[0094] It should be noted that after obtaining the behavior discrimination data output by the discriminator, the behavior reward value can be determined based on the behavior discrimination data. The behavior reward value is similar to the trajectory reward value mentioned above and can be simply derived by analogy. Then, based on the behavior reward value, the TRPO (Trust Region Policy Optimization) rule algorithm is used to update the parameters of the generator to obtain the updated generator. The updated generator can be specifically expressed as:

[0095]

[0096] in, is the updated generator during the i-th loop iteration; is lnπ θ (s,a) is the gradient of Q(s,a) at the parameter θ; π θ(s,a) is the generator with parameter θ; Q(s,a) is the expected value of the behavior reward value in the i-th loop iteration process; λ is the control coefficient; H(π θ ) is the regularization term of the generator during the i-th loop iteration.

[0097] It is worth mentioning that after obtaining the updated generator and discriminator, if the discriminator can still determine whether the input trajectory is a detection trajectory and the current loop iteration process is not the last loop iteration process, then the current detection behavior can be updated based on the next detection behavior of the detection behavior data set, and the updated generator and discriminator can be determined as the generator and discriminator used in the next loop iteration process, and then return to execute the step of obtaining the current detection behavior of the detection behavior data set.

[0098] Step 120: Input the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network;

[0099] In an embodiment of the present application, underwater detection data collected by various sensors of the operating unit in the current state can be input into the local reward value self-generation model to obtain a posterior local reward value grid network.

[0100] In some embodiments, inputting the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network includes:

[0101] Obtaining latitude and longitude data, heading data, and speed data of the operating unit;

[0102] Performing state pair construction processing on the underwater detection data according to the latitude and longitude data, the heading data, and the speed data to obtain a state-action pair of the operating unit;

[0103] Inputting the state-action pair into the local reward value self-generation model to obtain a plurality of local reward values;

[0104] The posterior local reward value grid network is obtained according to all the local reward values.

[0105] In the embodiment of the present application, the latitude and longitude data, heading data, and speed data of the operating unit can be first determined based on the current position of the operating unit; then, the latitude and longitude data, heading data, and speed data are combined with the underwater detection data to obtain a state-action pair of the operating unit in the current state. The state-action pair can be expressed as:

[0106]

[0107] in, is a state-action pair, For status, is the action; x is the longitude in the longitude and latitude data, y is the latitude in the longitude and latitude data; f d is the underwater detection data; c is the heading data; v is the speed data.

[0108] It can be understood that after obtaining the state-action pair, the state-action pair can be input into the local reward value self-generation model to obtain the local reward value output by the local reward value self-generation model, and the local reward is added and updated to the reward value grid network where the current position of the work unit is located, so as to obtain the posterior local reward value grid network.

[0109] Step 130: Perform network reward value analysis based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network.

[0110] In an embodiment of the present application, the target reward value grid network may be determined based on the trajectory reward values ​​in the frozen prior global reward value grid network and the local reward values ​​in the posterior local reward value grid network.

[0111] In some embodiments, performing network reward value analysis processing based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network includes:

[0112] Obtaining the current position of the operation unit;

[0113] According to the current position, grid-aligning the frozen prior global reward value grid network and the posterior local reward value grid network to obtain aligned prior global reward value grid network and posterior local reward value grid network;

[0114] According to the aligned prior global reward value grid network and the posterior local reward value grid network, repeated grid reward value multiplication calculations are performed to obtain the target reward value grid network.

[0115] In an embodiment of the present application, first, the grid where the work unit is located in the posterior local reward value grid network can be recorded as the first central grid; then, based on the current position of the work unit, the grid where the work unit is located in the prior global reward value grid network is determined, and recorded as the second central grid; then, the first central grid of the posterior local reward value grid network is aligned with the second central grid in the prior global reward value grid network, and the reward values ​​in the two aligned grids are multiplied to obtain the final reward value grid network, which is recorded as the target reward value grid network.

[0116] Step 140: Optimize the task path of the underwater detection task according to the target reward value grid network to obtain an optimized task path.

[0117] In an embodiment of the present application, the task allocation and task path of the underwater detection task can be adjusted and optimized based on the reward value recorded in each grid of the target reward value grid network, so as to obtain an optimized task path. There are many specific implementation methods for adjusting and optimizing the task path based on the reward value, which will not be repeated here in this application.

[0118] A path optimization system for underwater detection tasks proposed according to an embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0119] Reference Figure 2 , a path optimization system for underwater detection tasks proposed in an embodiment of the present application includes:

[0120] The first processing unit 101 is used to obtain underwater detection data when the operating unit performs underwater detection tasks, as well as a local reward value self-generation model and a frozen prior global reward value grid network;

[0121] The second processing unit 102 is configured to input the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network;

[0122] The third processing unit 103 is configured to perform network reward value analysis processing based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network;

[0123] The fourth processing unit 104 is configured to optimize the task path of the underwater detection task according to the target reward value grid network to obtain an optimized task path.

[0124] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0125] Reference Figure 3 , an embodiment of the present application further provides an electronic device, including:

[0126] at least one processor 201;

[0127] At least one memory 202, configured to store at least one program;

[0128] When the at least one program is executed by the at least one processor 201 , the at least one processor 201 implements the above method embodiment.

[0129] Similarly, it can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0130] An embodiment of the present application further provides a computer-readable storage medium, in which a program executable by the processor 201 is stored. The program executable by the processor 201 is used to implement the above-mentioned method embodiment when executed by the processor 201.

[0131] Similarly, the contents of the above method embodiments are applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0132] An embodiment of the present application further provides a computer program product, including a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0133] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0134] In some optional embodiments, the functions / operations mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, the two boxes shown in succession may actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flow chart of the present application are provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logic flows presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0135] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present application. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present application as set forth in the claims using ordinary techniques without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0136] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0137] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0138] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0139] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0140] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0141] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0142] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present application, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A path optimization method for underwater detection tasks, characterized in that: include: Obtain underwater detection data when the operating unit performs underwater detection tasks, as well as a local reward value self-generation model and a frozen prior global reward value grid network; Inputting the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network; Performing network reward value analysis processing based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network; The task path of the underwater detection task is optimized according to the target reward value grid network to obtain an optimized task path.

2. The method according to claim 1, characterized in that Obtain the local reward value self-generation model and the frozen prior global reward value grid network, including: Obtain detection trajectory datasets and detection behavior datasets, and generate adversarial reinforcement learning models; Training the generative adversarial reinforcement learning model according to the detection trajectory dataset to obtain the frozen prior global reward value grid network; The generative adversarial reinforcement learning model is pre-trained according to the detection behavior dataset to obtain the local reward value self-generation model.

3. The method according to claim 2, characterized in that The step of training the generative adversarial inverse reinforcement learning model according to the detection trajectory dataset to obtain the frozen prior global reward value grid network includes: Obtaining an intermediate global reward value grid network and a current detection trajectory of the detection trajectory dataset; Inputting the current detection trajectory into the generator of the generative adversarial reinforcement learning model to obtain an intermediate sampling trajectory; Inputting the intermediate sampling trajectory and the current detection trajectory into the discriminator of the generative adversarial reinforcement learning model to obtain trajectory discrimination data; Performing a trajectory reward value analysis on the trajectory discrimination data to obtain a trajectory reward value corresponding to the current track detection trajectory; The intermediate global reward value grid network is updated with a reward value grid according to the trajectory reward value to obtain the frozen prior global reward value grid network.

4. The method according to claim 2, characterized in that Pre-training the generative adversarial reinforcement learning model based on the detection behavior dataset to obtain the local reward value self-generation model includes: Obtaining the current detection behavior of the detection behavior dataset; Inputting the current detection behavior into the current generator of the generative adversarial reinforcement learning model to obtain an intermediate sampling behavior; Inputting the intermediate sampled behavior into the current discriminator of the generative adversarial reinforcement learning model to update the discriminator parameters, thereby obtaining behavior discrimination data and an updated discriminator; updating the generator parameters of the current generator according to the behavior discrimination data to obtain an updated generator; The local reward value self-generation model is obtained according to the updated generator and the updated discriminator.

5. The method according to claim 1, wherein The step of inputting the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network includes: Obtaining latitude and longitude data, heading data, and speed data of the operating unit; Performing state pair construction processing on the underwater detection data according to the latitude and longitude data, the heading data, and the speed data to obtain a state-action pair of the operating unit; Inputting the state-action pair into the local reward value self-generation model to obtain a plurality of local reward values; The posterior local reward value grid network is obtained according to all the local reward values.

6. The method according to claim 1, characterized in that The method of performing network reward value analysis processing based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network includes: Obtaining the current position of the operation unit; According to the current position, grid-aligning the frozen prior global reward value grid network and the posterior local reward value grid network to obtain aligned prior global reward value grid network and posterior local reward value grid network; According to the aligned prior global reward value grid network and the posterior local reward value grid network, repeated grid reward value multiplication calculations are performed to obtain the target reward value grid network.

7. A path optimization system for underwater detection tasks, characterized in that: include: A first processing unit is used to obtain underwater detection data when the operating unit performs underwater detection tasks, as well as a local reward value self-generation model and a frozen prior global reward value grid network; A second processing unit is configured to input the underwater detection data into the local reward value self-generation model to obtain a posterior local reward value grid network; a third processing unit, configured to perform network reward value analysis processing based on the frozen prior global reward value grid network and the posterior local reward value grid network to obtain a target reward value grid network; The fourth processing unit is configured to optimize the task path of the underwater detection task according to the target reward value grid network to obtain an optimized task path.

8. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 6 when executed by the processor.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed, the steps of the method according to any one of claims 1 to 6 are implemented.

Images