System and method of automatic route generation applied reinforcement learning for coastal search operation of unmanned surface boat

Abstract

A reinforcement learning-applied automatic path generation system and method for coastal search operations of an unmanned surface vessel are disclosed. The system comprises: a search area setting device for setting a search area for coastal search of an unmanned surface vessel; a path setting device for automatically setting a path for searching the unmanned surface vessel in the search area set by the search area setting device; and an autonomous driving device for performing autonomous driving according to the path automatically set by the path setting device. According to the above-described reinforcement learning-applied automatic path generation system and method for coastal search operations of an unmanned surface vessel, by generating a path based on a shortest path algorithm and expanding the area based on reinforcement learning to search the search area without omission, the unmanned surface vessel can not only search the entire specific coastal search area without omission, but also drastically reduce the travel distance from the starting point to the destination and drastically reduce the number of duplicate visits to already searched places. Accordingly, since users do not need to manually specify routes or directly control the equipment during coastal search operations, wasted time and manpower can be reduced. Furthermore, by increasing the efficiency of movement routes, equipment operation time can be shortened, and fuel consumption and maintenance time can also be reduced.

Description

Technology Field

[0001] The present invention relates to an automatic path generation system and method, specifically to an automatic path generation system and method applying reinforcement learning, and more specifically to an automatic path generation system and method applying reinforcement learning for coastal search operations of unmanned surface vessels. Background Technology

[0002] Unmanned surface vessels are used in various border security operations and can also be utilized in coastal search operations to patrol specific areas and check for any anomalies.

[0003] Due to Korea's complex coastline, there is a problem in that users must manually issue patrol routes for unmanned surface vessels for every patrol. In areas with complex coastlines, generating and selecting multiple waypoints requires significant effort and time, leading not only to a waste of human resources but also to reduced operational efficiency and agility.

[0004] Therefore, to conduct search operations in coastal areas with complex coastline terrain, technology that automatically generates waypoints is required.

[0005] Most existing path generation algorithms generate paths based on criteria such as the shortest travel distance and lowest fuel consumption between a starting point and an ending point.

[0006] For reference, registered patent publication 10-1795935 discloses an algorithm that divides a specific area into cells to set up a region for collecting sensing information and generates a path for an aircraft. However, the algorithm in this document also focuses on generating a path based on the utility of the aircraft, namely, the least fuel consumption and the least travel distance.

[0007] The algorithm in this document visits only the cells on the optimal path without visiting all cells within the region, and because it is a flying vehicle, it is not affected by complex coastal terrain.

[0008] However, since unmanned surface vessels perform search missions in specific areas, an algorithm must be applied to visit all cells according to the complex coastal terrain of that region; there are various challenges to be addressed, such as optimizing fuel consumption, travel distance, and travel time while visiting all cells.

[0009] There is a problem in that it is difficult to efficiently carry out search operations for unmanned surface vessels using existing path generation algorithms. Prior art literature

[0010] Registered Patent Publication 10-1795935 Registered Patent Publication 10-2410738 The problem to be solved

[0011] The objective of the present invention is to provide an automatic path generation system applying reinforcement learning for unmanned surface vessel coastal search operations.

[0012] Another objective of the present invention is to provide an automatic path generation method applying reinforcement learning for unmanned surface vessel coastal search operations. means of solving the problem

[0013] A reinforcement learning-applied automatic path generation system for coastal search operations of an unmanned surface vessel according to the purpose of the present invention described above may be configured to include: a search area setting device for setting a search area for coastal search of an unmanned surface vessel; and a path setting device for automatically setting a path for searching the unmanned surface vessel in the search area set by the search area setting device.

[0014] Here, the system may be configured to further include an autonomous driving device that performs autonomous driving according to an automatically set path in the path setting device.

[0015] And the search area setting device may be configured to include: a map database in which a map is stored in advance; an automatic cell division module that automatically divides the map stored in advance in the map database into grid-shaped cells of a predetermined size; and a coastal terrain-based automatic node setting module that automatically sets a cell where the unmanned surface vessel can move as a node based on the coastal terrain on the map among the cells automatically divided by the automatic cell division module.

[0016] And the search area setting device may be configured to further include a search area setting module that sets a set of adjacent cells to be searched by the unmanned surface vessel on the map as a search area; and a departure / destination setting module that sets a departure point and a destination point among the nodes within the search area, respectively.

[0017] And the above-mentioned route setting device may be configured to further include an A* algorithm-based route calculation module that calculates an A* route between a departure point and a destination set in the departure point / destination setting module based on the A* algorithm.

[0018] And the path setting device may be configured to include: a search area image generation module that generates a search area image of a map in which nodes within the search area are displayed; a vector state information setting module that generates and sets vector state information of each node in real time according to the autonomous driving of the autonomous driving device; and a reinforcement learning-based path calculation module that performs reinforcement learning based on the path calculated by the A* algorithm-based path calculation module to calculate a reinforcement learning-based path and provides it to the autonomous driving device in real time, and performs reinforcement learning by applying the vector state information of each node set by the vector state information setting module to each node of the search area image generated by the search area image generation module in real time, and calculates a reinforcement learning-based path in real time.

[0019] And the path setting device may be configured to further include a path calculation control module that controls the A* algorithm-based path calculation module to calculate an A* path and controls the reinforcement learning-based path calculation module to calculate a reinforcement learning-based path based on the calculated A* path.

[0020] And the above path setting device may be configured to further include a visitable node verification module that verifies the visitable nodes of the unmanned surface vessel in real time using the vector state information of each node set in the vector state information setting module.

[0021] And the above path setting device may be configured to further include an A* path range change module that expands and changes the A* path range of the A* path to be calculated by the A* algorithm-based path calculation module in real time when there are no visitable nodes as a result of real-time verification by the visitable node verification module.

[0022] And the path calculation control module can control the A* algorithm-based path calculation module to recalculate the A* path within the A* path range that is expanded and changed in real time by the A* path range change module, and the reinforcement learning-based path calculation module to calculate the reinforcement learning-based path based on the recalculated A* path.

[0023] A method for automatically generating a path using reinforcement learning for a coastal search operation of an unmanned surface vessel according to another objective of the present invention described above may be configured to include: a search area setting device setting a search area for coastal search of the unmanned surface vessel; a path setting device automatically setting a path for search of the unmanned surface vessel in the search area set by the search area setting device; and an autonomous driving device performing autonomous driving according to the path automatically set by the path setting device.

[0024] Here, the search area setting device may be configured such that the step of setting a search area for coastal search of an unmanned surface vessel includes: a step in which an automatic cell division module automatically divides a map stored in advance in a map database into cells in the form of a grid of a predetermined size; a step in which a coastal terrain-based automatic node setting module automatically sets a cell that the unmanned surface vessel can move to as a node based on the coastal terrain on the map among the cells automatically divided by the automatic cell division module; a step in which a search area setting module sets a set of adjacent cells that the unmanned surface vessel must search on the map as a search area; and a step in which a departure point / arrival point setting module sets a departure point and an arrival point, respectively, among the nodes within the search area.

[0025] And the step of the path setting device automatically setting a path for the search of the unmanned surface vessel in the search area set by the search area setting device comprises: a step in which an A* algorithm-based path calculation module calculates an A* path between the origin and the destination set by the origin / destination setting module based on the A* algorithm; a step in which a search area image generation module generates a search area image of a map displaying nodes within the search area; a step in which a vector state information setting module generates and sets vector state information of each node in real time according to the autonomous driving of the autonomous driving device; a step in which a reinforcement learning-based path calculation module performs reinforcement learning by applying the vector state information of each node set by the vector state information setting module in real time to each node of the search area image generated by the search area image generation module based on the path calculated by the A* algorithm-based path calculation module, calculates a reinforcement learning-based path in real time, and provides the reinforcement learning-based path calculated in real time to the autonomous driving device in real time; and a step in which a visitable node verification module verifies the visitable nodes of the unmanned surface vessel in real time using the vector state information of each node set by the vector state information setting module. The A* path range change module may be configured to include a step of expanding and changing the A* path range in real time so that the A* algorithm-based path calculation module can calculate the A* path when there are no visitable nodes as a result of real-time verification by the visitable node verification module. Effects of the invention

[0026] According to the above-described automatic path generation system and method applying reinforcement learning for coastal search operations of unmanned surface vessels, by generating a path based on a shortest path algorithm and expanding the area based on reinforcement learning to search the search area without omission, the unmanned surface vessel can not only search the entire specific coastal search area without omission, but also drastically reduce the travel distance from the starting point to the destination and drastically reduce the number of duplicate visits to places that have already been searched.

[0027] Accordingly, since users do not need to manually specify routes or directly control the equipment during coastal search operations, wasted time and manpower can be reduced. Furthermore, by increasing the efficiency of movement routes, equipment operation time can be shortened, and fuel consumption and maintenance time can also be reduced. Brief explanation of the drawing

[0028] FIG. 1 is a block diagram of an automatic path generation system applying reinforcement learning for coastal search operations of an unmanned surface vessel according to one embodiment of the present invention. FIG. 2 is an example diagram of a cell division map according to one embodiment of the present invention. FIG. 3 is an example of a search area on a cell division map according to one embodiment of the present invention. FIG. 4 is a schematic diagram of a reinforcement learning-based path calculation process according to one embodiment of the present invention. Figure 5 is a graph showing the results of reinforcement learning according to one embodiment of the present invention. FIGS. 6 to 12 are exemplary diagrams showing the path of autonomous driving according to one embodiment of the present invention. FIG. 13 is a graph showing the degree of improvement before and after reinforcement learning according to one embodiment of the present invention. FIG. 14 is a flowchart of an automatic path generation method applying reinforcement learning for an unmanned surface vessel coastal search operation according to an embodiment of the present invention. Specific details for implementing the invention

[0029] The following description merely illustrates the principles of the present invention. Therefore, those skilled in the art may invent various devices that embody the principles of the present invention and are included within the concept and scope of the present invention, even though they are not explicitly described or illustrated in this specification. Furthermore, all conditional terms and embodiments listed in this specification are, in principle, explicitly intended only for the purpose of understanding the concept of the present invention and should be understood not as being limited to the embodiments and conditions specifically listed as such.

[0030] In addition, all detailed descriptions enumerating specific embodiments, as well as the principles, aspects, and embodiments of the present invention, should be understood as being intended to include structural and functional equivalents of such matters.

[0031] In addition, these equivalents should be understood to include not only currently known equivalents but also equivalents to be developed in the future, that is, all devices invented to perform the same function regardless of structure.

[0032] Accordingly, for example, the block diagrams in this specification should be understood as representing a conceptual view of an exemplary circuit embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudocode, etc., should be understood as representing various processes that can be substantially represented on a computer-readable medium and are performed by a computer or processor, regardless of whether the computer or processor is explicitly depicted.

[0033] The functions of various elements illustrated in the drawings, including functional blocks represented as processors or similar concepts, may be provided by the use of dedicated hardware as well as hardware capable of executing software in relation to appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

[0034] Furthermore, the explicit use of terms presented as processor, control, or similar concepts should not be interpreted as exclusively referring to hardware capable of executing software, but should be understood to implicitly include, without limitation, digital signal processor (DSP) hardware, ROM, RAM, and non-volatile memory for storing software. Other commonly known hardware may also be included.

[0035] In the claims of this specification, the component expressed as a means for performing the function described in the detailed description is intended to include all methods for performing the function, including, for example, a combination of circuit elements performing the function or software of any form including firmware / microcode, and is combined with a suitable circuit for executing said software to perform said function. Since the invention defined by these claims combines the functions provided by the various enumerated means and combines them in the manner required by the claims, any means capable of providing said function should be understood as equivalent to that identified from this specification.

[0036] The aforementioned objectives, features, and advantages will become clearer through the following detailed description in conjunction with the attached drawings, and accordingly, a person skilled in the art to which the present invention pertains will be able to easily implement the technical concept of the present invention. Furthermore, in describing the present invention, if it is determined that a detailed description of known technology related to the present invention may unnecessarily obscure the essence of the present invention, such detailed description will be omitted.

[0037] FIG. 1 is a block diagram of an automatic path generation system applying reinforcement learning for coastal search operations of an unmanned surface vessel according to an embodiment of the present invention. FIG. 2 is an example diagram of a cell division map according to an embodiment of the present invention, FIG. 3 is an example diagram of a search area on a cell division map according to an embodiment of the present invention, FIG. 4 is a schematic diagram of a reinforcement learning-based path calculation process according to an embodiment of the present invention, FIG. 5 is a graph showing reinforcement learning results according to an embodiment of the present invention, FIG. 6 to FIG. 12 are example diagrams showing a path of autonomous driving according to an embodiment of the present invention, and FIG. 13 is a graph showing the degree of improvement before and after reinforcement learning according to an embodiment of the present invention.

[0038] Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

[0039] First, referring to FIG. 1, a reinforcement learning applied automatic path generation system (100) for coastal search operations of an unmanned surface vessel according to one embodiment of the present invention may be configured to include a search area setting device (110), a path setting device (120), and an autonomous driving device (130).

[0040] The detailed composition is explained below.

[0041] The search area setting device (110) can be configured to set a search area for coastal search of an unmanned surface vessel.

[0042] The search area setting device (110) may be configured to include a map database (111), an automatic cell separation module (112), a coastal terrain-based automatic node setting module (113), a search area setting module (114), and a departure / arrival setting module (115).

[0043] The detailed composition is explained below.

[0044] The map database (111) can be configured so that maps are stored in advance.

[0045] The automatic cell separation module (112) can be configured to automatically separate a map stored in advance in the map database (111) into cells in the form of a grid of a predetermined size as shown in Fig. 2.

[0046] Figure 2 illustrates a map divided into cells of size 1 km x 1 km.

[0047] The coastal terrain-based automatic node setting module (113) can be configured to automatically set as nodes cells that an unmanned surface vessel can move to based on the coastal terrain on the map among the cells automatically separated by the automatic cell separation module (112). FIG. 3 illustrates an example where coastal cells that an unmanned surface vessel can move to are set as red nodes, and land or island cells that an unmanned surface vessel cannot move to are set as green nodes.

[0048] The search area setting module (114) can be configured to set a set of adjacent cells on a map that the unmanned surface vessel must search as a search area. In FIG. 3, the search area is indicated by a red border line.

[0049] The origin / destination setting module (115) can be configured to set the origin and destination respectively among the nodes within the search area.

[0050] The path setting device (120) can be configured to automatically set a path for searching the unmanned surface vessel in the search area set by the search area setting device (110).

[0051] The path setting device (120) may be configured to include an A* algorithm-based path calculation module (121), a search area image generation module (122), a vector state information setting module (123), a reinforcement learning-based path calculation module (124), a path calculation control module (125), a visitable node verification module (126), and an A* path range change module (127).

[0052] The detailed composition is explained below.

[0053] The A* algorithm-based path calculation module (121) can be configured to calculate an A* path between a starting point and a destination set in the starting point / destination setting module (115) based on the A* algorithm. Here, the A* algorithm is an algorithm that calculates the shortest path between a starting point and a destination. However, since the A* algorithm is an algorithm that calculates the shortest path, it may not be able to search all nodes in the search area without omission.

[0054] The search area image generation module (122) can be configured to generate a search area image of a map showing nodes within the search area.

[0055] The vector state information setting module (123) can be configured to generate and set vector state information for each node in real time according to the autonomous driving of the autonomous driving device (130).

[0056] Figure 4 shows the search area image and vector state information of each node.

[0057] Here, the vector state information of a node can consist of data such as the number of moves counted at each node as the unmanned surface vessel moves along the path, the number of nodes visited by the unmanned surface vessel, and the current location of the unmanned surface vessel. In this context, the number of moves counted at each node represents the number of duplicate visits to that node; a higher number of duplicate visits can be seen as a decrease in path efficiency. Additionally, regarding the number of nodes visited by the unmanned surface vessel, a higher number of such nodes results in a longer travel distance for the unmanned surface vessel, which can be seen as a decrease in path efficiency.

[0058] The reinforcement learning-based path calculation module (124) can be configured to perform reinforcement learning based on the path calculated by the A* algorithm-based path calculation module (121) to calculate a reinforcement learning-based path and provide it in real time to the autonomous driving device (400).

[0059] Specifically, as shown in FIG. 4, the reinforcement learning-based path calculation module (124) can be configured to perform reinforcement learning and calculate a reinforcement learning-based path in real time by applying the vector state information of each node set in the vector state information setting module (123) to each node of the search area image generated by the search area image generation module (122).

[0060] This reinforcement learning can be configured to reduce the travel distance of the unmanned surface vessel and minimize the number of revisits to visited nodes. Figure 5 illustrates the result of performing a total of 300,000 reinforcement learning iterations.

[0061] The path calculation control module (125) can control the A* algorithm-based path calculation module (121) to calculate an A* path and control the reinforcement learning-based path calculation module (124) to calculate a reinforcement learning-based path based on the calculated A* path.

[0062] The visitable node verification module (126) can be configured to verify the visitable nodes of the unmanned surface vessel in real time using the vector state information of each node set in the vector state information setting module (123).

[0063] The A* path range change module (127) can be configured to expand and change the A* path range of the A* path that the A* algorithm-based path calculation module (121) needs to calculate in real time when there are no visitable nodes as a result of real-time verification by the visitable node verification module (126).

[0064] The path calculation control module (125) can control the A* algorithm-based path calculation module (121) to recalculate the A* path within the A* path range that has been expanded and changed in real time by the A* path range change module (127), and the reinforcement learning-based path calculation module (124) to calculate the reinforcement learning-based path based on the recalculated A* path.

[0065] In other words, the system can be configured to calculate the shortest path based on the A* path by default, but to visit all surrounding nodes during the traversal and, if no more nodes are accessible, gradually expand the A* path range to calculate the A* path while visiting all nodes to perform the search. Consequently, it can be configured to reduce the path to its shortest possible length while thoroughly searching all nodes.

[0066] FIGS. 6 to 10 sequentially illustrate the above process through map images. FIG. 11 shows a search path by a conventional algorithm before learning, and FIG. 12 shows a search path by an algorithm after reinforcement learning.

[0067] Figure 13 shows that the number of moves, distance traveled, number of duplicate visits, and travel time of the search path after reinforcement learning were significantly reduced compared to the search path before reinforcement learning.

[0068] The autonomous driving device (130) can be configured to perform autonomous driving according to a path automatically set by the path setting device (120).

[0069] FIG. 14 is a flowchart of an automatic path generation method applying reinforcement learning for an unmanned surface vessel coastal search operation according to an embodiment of the present invention.

[0070] Referring to FIG. 14, first, the search area setting device (110) sets a search area for coastal search of an unmanned surface vessel (S110).

[0071] Step S110 is described in detail as follows. First, an automatic cell division module (112) automatically divides a map stored in advance in a map database (111) into cells in the form of a grid of a predetermined size (S111). Then, a coastal terrain-based automatic node setting module (113) automatically sets a cell that an unmanned surface vessel can move to as a node based on the coastal terrain on the map among the cells automatically divided by the automatic cell division module (112) (S112). Then, a search area setting module (114) sets a set of adjacent cells on the map that the unmanned surface vessel must search as a search area (S113). Then, a departure / arrival setting module (115) sets a departure point and an arrival point, respectively, among the nodes within the search area (S114).

[0072] Next, the path setting device (120) automatically sets a path for searching the unmanned surface vessel in the search area set by the search area setting device (110) (S120).

[0073] Step S120 is specifically as follows. First, an A* algorithm-based path calculation module (121) calculates an A* path between a starting point and a destination set by a starting point / destination setting module (115) based on the A* algorithm (S121). Then, a search area image generation module (122) generates a search area image of a map displaying nodes within the search area (S122). Then, a vector state information setting module (123) generates and sets vector state information for each node in real time according to the autonomous driving of the autonomous driving device (130) (S123). Then, the reinforcement learning-based path calculation module (124) performs reinforcement learning by applying the vector state information of each node set in the vector state information setting module (123) to each node of the search area image generated by the search area image generation module (122) based on the path calculated by the A* algorithm-based path calculation module (121) in real time, and calculates the reinforcement learning-based path in real time, and provides the real-time calculated reinforcement learning-based path to the autonomous driving device (130) in real time (S124). Then, the visitable node verification module (126) checks the visitable nodes of the unmanned surface vessel in real time using the vector state information of each node set in the vector state information setting module (123) (S125). Then, if the real-time verification result of the visitable node verification module (126) shows that there are no visitable nodes, the A* path range change module (127) expands and changes the A* path range in real time so that the A* algorithm-based path calculation module (121) can calculate the A* path (S126).

[0074] Next, the autonomous driving device (130) performs autonomous driving according to the path automatically set by the path setting device (120) (S130).

[0075] In addition, the operation method according to the various embodiments of the present invention described above may be implemented as a program and provided by being stored on various non-transitory computer-readable media. A non-transitory computer-readable medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short moment, such as a register, cache, or memory. Specifically, the various applications or programs described above may be provided by being stored on non-transitory computer-readable media such as CDs, DVDs, hard disks, Blu-ray disks, USBs, memory cards, ROMs, etc.

[0076] Furthermore, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modified embodiments should not be understood individually from the technical spirit or perspective of the present invention. Explanation of the symbols

[0077] 110: Search area setting device 111: Map database 112: Automatic cell separation module 113: Coastal Terrain-Based Automatic Node Setup Module 114: Search Area Setting Module 115: Origin / Destination Setting Module 120: Route setting device 121: A* Algorithm-based Path Calculation Module 122: Search Area Image Generation Module 123: Vector State Information Setting Module 124: Reinforcement Learning-based Path Generation Module 125: Path Calculation Control Module 126: Check available nodes module 127: A* Path Range Change Module 130: Autonomous driving device

Claims

Claim 1 A reinforcement learning applied automatic path generation system for coastal search operations of an unmanned surface vessel, comprising: a search area setting device that sets a search area for coastal search of an unmanned surface vessel based on environmental information and defines mission conditions; and a path setting device that automatically sets a path for searching of the unmanned surface vessel in the search area to satisfy the mission conditions, wherein the path setting device automatically sets a cell movable by the unmanned surface vessel as a node from the environmental information, calculates a reference path based on an A* algorithm, repeatedly updates the reference path using reinforcement learning based on vector state information including the current location and visit history collected during the operation of the unmanned surface vessel, and controls the system to search all nodes in the search area without omission by expanding the A* path range to recalculate the reference path and reapplying the result when it is confirmed that there are no visitable nodes. Claim 2 A reinforcement learning applied automatic path generation system for unmanned surface vessel coastal search operations, characterized in that, in claim 1, it is further configured to include an autonomous driving device that performs autonomous driving according to a path automatically set by the path setting device. Claim 3 In paragraph 2, the search area setting device is configured to include: a map database in which a map is stored in advance; an automatic cell division module that automatically divides the map stored in advance in the map database into grid-shaped cells of a predetermined size; and a coastal terrain-based automatic node setting module that automatically sets cells in which the unmanned surface vessel can move as nodes based on the coastal terrain on the map among the cells automatically divided by the automatic cell division module. This describes an automatic path generation system applying reinforcement learning for unmanned surface vessel coastal search operations. Claim 4 In paragraph 3, the search area setting device is configured to further include: a search area setting module that sets a set of adjacent cells to be searched by the unmanned surface vessel on the map as a search area; and a departure / destination setting module that sets a departure point and a destination point among the nodes within the search area, respectively, thereby forming an automatic path generation system applying reinforcement learning for unmanned surface vessel coastal search operations. Claim 5 A reinforcement learning applied automatic path generation system for unmanned surface vessel coastal search operations, characterized in that, in claim 4, the path setting device is configured to further include an A* algorithm-based path calculation module that calculates an A* path between a starting point and a destination set in the starting point / destination setting module based on the A* algorithm. Claim 6 In claim 5, the path setting device comprises: a search area image generation module that generates a search area image of a map displaying nodes within the search area; a vector state information setting module that generates and sets vector state information of each node in real time according to the autonomous driving of the autonomous driving device; and a reinforcement learning-based path calculation module that performs reinforcement learning based on a path calculated by the A* algorithm-based path calculation module to calculate a reinforcement learning-based path and provides it to the autonomous driving device in real time, and performs reinforcement learning by applying the vector state information of each node set by the vector state information setting module to each node of the search area image generated by the search area image generation module in real time, and calculates a reinforcement learning-based path in real time. This characterizes an automatic path generation system applying reinforcement learning for coastal search operations of an unmanned surface vessel. Claim 7 A reinforcement learning applied automatic path generation system for unmanned surface vessel coastal search operations, characterized in that, in claim 6, the path setting device further comprises a path calculation control module that controls the A* algorithm-based path calculation module to calculate an A* path and controls the reinforcement learning-based path calculation module to calculate a reinforcement learning-based path based on the calculated A* path. Claim 8 In claim 7, the reinforcement learning applied automatic path generation system for unmanned surface vessel coastal search operations is characterized in that the path setting device further comprises a visitable node verification module that verifies the visitable nodes of the unmanned surface vessel in real time using the vector state information of each node set in the vector state information setting module. Claim 9 In claim 8, the reinforcement learning applied automatic path generation system for unmanned surface vessel coastal search operations is characterized in that the path setting device is configured to further include an A* path range changing module that expands and changes the A* path range of the A* path to be calculated by the A* algorithm-based path calculation module in real time when there are no visitable nodes as a result of real-time verification by the visitable node verification module. Claim 10 In claim 9, the reinforcement learning applied automatic path generation system for unmanned surface vessel coastal search operations is characterized in that the path calculation control module controls the A* algorithm-based path calculation module to recalculate the A* path within the A* path range that is expanded and changed in real time by the A* path range change module, and the reinforcement learning-based path calculation module to calculate the reinforcement learning-based path based on the recalculated A* path. Claim 11 A method for automatically generating a path using reinforcement learning for an unmanned surface vessel coastal search operation, comprising: a step of a search area setting device setting a search area for coastal search of an unmanned surface vessel based on environmental information and defining mission conditions; a step of a path setting device automatically setting a path for searching of the unmanned surface vessel in the search area to satisfy the mission conditions; and a step of an autonomous driving device performing autonomous driving according to the path automatically set by the path setting device, wherein the path setting device automatically sets cells movable by the unmanned surface vessel as nodes from the environmental information, calculates a reference path based on an A* algorithm, repeatedly updates the reference path using reinforcement learning based on vector state information including current location and visit history collected during the operation of the unmanned surface vessel, and controls the search to ensure that all nodes in the search area are searched without omission by expanding the A* path range and recalculating the reference path and reapplying the result when it is confirmed that there are no visitable nodes. Claim 12 In claim 11, the search area setting device comprises the step of setting a search area for coastal search of an unmanned surface vessel, wherein the step includes: an automatic cell division module automatically dividing a map stored in advance in a map database into grid-shaped cells of a predetermined size; a coastal terrain-based automatic node setting module automatically setting a cell that the unmanned surface vessel can move to as a node based on the coastal terrain on the map among the cells automatically divided by the automatic cell division module; a search area setting module setting a set of adjacent cells that the unmanned surface vessel must search on the map as a search area; and a departure / arrival setting module setting a departure point and an arrival point among the nodes within the search area, respectively, thereby comprising a reinforcement learning applied automatic path generation method for coastal search operations of an unmanned surface vessel. Claim 13 In claim 12, the step of the path setting device automatically setting a path for the search of the unmanned surface vessel in the search area set by the search area setting device comprises: a step in which an A* algorithm-based path calculation module calculates an A* path between the origin and the destination set by the origin / destination setting module based on the A* algorithm; a step in which a search area image generation module generates a search area image of a map displaying nodes within the search area; a step in which a vector state information setting module generates and sets vector state information of each node in real time according to the autonomous driving of the autonomous driving device; a step in which a reinforcement learning-based path calculation module performs reinforcement learning by applying the vector state information of each node set by the vector state information setting module in real time to each node of the search area image generated by the search area image generation module based on the path calculated by the A* algorithm-based path calculation module, calculates a reinforcement learning-based path in real time, and provides the reinforcement learning-based path calculated in real time to the autonomous driving device in real time; a step in which a visitable node verification module verifies the visitable nodes of the unmanned surface vessel in real time using the vector state information of each node set by the vector state information setting module; A* path range A method for automatic path generation using reinforcement learning for coastal search operations of an unmanned surface vessel, characterized in that the modification module is configured to include a step of expanding and changing the A* path range in real time so that the A* algorithm-based path calculation module can calculate the A* path when there are no visitable nodes as a result of real-time verification by the visitable node verification module.

Images