Artificial intelligence-based ship route generation method, device and equipment and storage medium

Abstract

The invention belongs to the field of ship automation control, and provides a ship route generation method and device based on artificial intelligence, equipment and a storage medium. The method comprises the following steps: constructing a route generation model based on deep reinforcement learning training to generate a waypoint set, and connecting a plurality of waypoints in the waypoint set into a route; when a rudder angle control vector is generated by applying an action to a ship according to a course, based on a fuzzy rule, taking a course angle error, a speed and an acceleration as input, and performing fuzzy calculation to obtain a first rudder angle control increment; whether environment disturbance exists or not is determined based on a Kalman filter, when the environment disturbance exists, a second rudder angle control increment is obtained after calculation based on a fuzzy rule, and the first rudder angle control increment, the second rudder angle control increment and a rudder angle control vector are combined to control ship operation; the invention aims to realize optimal route planning and accurate path following of the ship in autonomous navigation through a reinforcement learning algorithm and fuzzy control, ensure navigation safety and reduce fuel consumption.

Description

Technical Field

[0001] The present invention relates to the field of ship automation control, and in particular to a ship route generation method, device, equipment and storage medium based on artificial intelligence. Background Art

[0002] With the development of autonomous ships and artificial intelligence technology, traditional manual route planning and heading control methods are difficult to meet the needs of modern autonomous navigation. Ships need to be able to autonomously generate the optimal route and accurately follow it while coping with complex marine environments. In the face of uncertainty in the marine environment, traditional path planning algorithms and control strategies have limitations and fail to take into account the safety, fuel efficiency and environmental disturbance effects of autonomous navigation. Summary of the invention

[0003] In view of the above technical problems, the present invention provides a ship route generation method, device, equipment and storage medium based on artificial intelligence, which aims to achieve optimal route planning and precise path following of ships in autonomous navigation through reinforcement learning algorithm and fuzzy control, ensure navigation safety and reduce fuel consumption, and take into account environmental disturbances to overcome the impact of environmental disturbances on the route.

[0004] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0005] According to one aspect of the present invention, a method for generating a ship route based on artificial intelligence is disclosed, the method comprising:

[0006] A route generation model is constructed based on deep reinforcement learning training to generate a waypoint set. The environment of the route generation model is based on a nautical chart between a destination and a departure place, and is combined with location data of an impassable area. Its actions include adjusting the speed and heading of the ship, and its state includes the current position, speed, and heading of the ship. The reward mechanism of the route generation model is set as follows: if the waypoint set generated at the t+1th time is less than the waypoint set generated at the tth time, or / and the actions at the tth time and the t+1th time in any waypoint set are the same, then the Q value is increased, and multiple waypoints in the waypoint set are connected into a route;

[0007] When the rudder angle control vector is generated by applying an action to the ship according to the route, a first rudder angle control increment is obtained after fuzzy calculation based on fuzzy rules with the heading angle error, speed and acceleration as inputs;

[0008] The prior state of the ship is estimated based on the Kalman filter, and the prior state is compared with the actual state actually measured by the sensor to determine whether there is an environmental disturbance. When there is an environmental disturbance, based on fuzzy rules, the error between the prior state and the actual state is used as input, and a second rudder angle control increment is obtained after fuzzy calculation. The first rudder angle control increment, the second rudder angle control increment and the rudder angle control vector are combined to control the operation of the ship.

[0009] Furthermore, the impassable areas include: shallow water areas, restricted areas, islands, dangerous areas, and land.

[0010] Further, after the waypoint set is generated based on the route generation model, the waypoint set is simplified based on the Douglas-Peucker algorithm, so that the waypoints in the waypoint set are reduced.

[0011] Furthermore, when the fuzzy calculation of the heading angle error, velocity and acceleration is performed, it includes:

[0012] Mapping heading angle error, velocity and acceleration into multiple fuzzy sets;

[0013] According to the preset fuzzy rule base, the applicability of each rule is calculated and different rudder angle control fuzzy quantities are given;

[0014] Performing weighted calculation on all the rudder angle control fuzzy quantities to obtain a first comprehensive fuzzy output;

[0015] The first integrated fuzzy output is defuzzified to obtain the first rudder angle control increment.

[0016] Furthermore, when performing fuzzy calculation on the error between the prior state and the actual state, it includes:

[0017] Mapping the error between the prior state and the actual state into a plurality of fuzzy sets;

[0018] According to the preset fuzzy rule base, the applicability of each rule is calculated and different rudder angle control fuzzy quantities are given;

[0019] Performing weighted calculation on all the rudder angle control fuzzy quantities to obtain a second comprehensive fuzzy output;

[0020] Defuzzifying the second comprehensive fuzzy output to obtain the impact force, rolling force and yaw moment of the environmental disturbance;

[0021] The impact force is converted into a thrust for controlling an actual ship, and the roll force and the yaw moment are converted into a rudder angle for controlling the actual ship, so as to obtain the second rudder angle control increment.

[0022] According to a second aspect of the present invention, there is provided a ship route generation device based on artificial intelligence, the device comprising:

[0023] A model generation module is used to construct a route generation model based on deep reinforcement learning training to generate a waypoint set. The environment of the route generation model is based on a nautical chart between the destination and the departure point, and is combined with location data of an impassable area. Its actions include adjusting the speed and heading of the ship, and its state includes the current position, speed, and heading of the ship. The reward mechanism of the route generation model is set as follows: if the waypoint set generated at the t+1th time is less than the waypoint set generated at the tth time, or / and the actions of the tth and t+1th times in any waypoint set are the same, then the Q value is increased, and multiple waypoints in the waypoint set are connected into a route;

[0024] A fuzzy speed control module is used to obtain a first rudder angle control increment after fuzzy calculation based on fuzzy rules, with heading angle error, speed and acceleration as inputs when generating a rudder angle control vector by applying an action to the ship according to the route;

[0025] The fuzzy error estimation module is used to estimate the prior state of the ship based on the Kalman filter, compare the prior state with the actual state actually measured by the sensor, determine whether there is an environmental disturbance, and when there is an environmental disturbance, based on fuzzy rules, use the error between the prior state and the actual state as input, obtain the second rudder angle control increment after fuzzy calculation, and combine the first rudder angle control increment, the second rudder angle control increment and the rudder angle control vector to control the operation of the ship.

[0026] According to a third aspect of the present disclosure, there is provided an artificial intelligence-based ship route generation device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor implements the artificial intelligence-based ship route generation method as described above;

[0027] According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the method for generating a ship route based on artificial intelligence as described above is implemented.

[0028] The technical solution disclosed in this disclosure has the following beneficial effects:

[0029] Use machine learning technology based on reinforcement learning algorithms to generate optimal routes. Deep reinforcement learning based on Q-learning algorithms is introduced. The optimal route is generated in a way that ensures ship safety while minimizing fuel consumption.

[0030] Based on fuzzy control to track the route, the ship controller will not cause large overshoot, and the ship will not deviate from the designated route. In addition, due to the smooth change of rudder angle, the energy efficiency of the ship is improved;

[0031] Based on the environmental disturbance estimated by fuzzy control and converting the environmental disturbance magnitude into the thrust and rudder angle for actual control of the ship, the designated route can be accurately followed even in the presence of environmental disturbances. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of a method for generating a ship route based on artificial intelligence in an embodiment of this specification;

[0033] Figure 2 This is a structural block diagram of a ship route generation device based on artificial intelligence in an embodiment of this specification;

[0034] Figure 3 A device for executing a method for generating a ship route based on artificial intelligence in an embodiment of this specification;

[0035] Figure 4 A computer-readable storage medium storing a method for generating a ship route based on artificial intelligence in an embodiment of this specification. DETAILED DESCRIPTION

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, systems, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0037] In addition, the accompanying drawings are only schematic illustrations of the present disclosure. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor systems and / or microcontroller systems.

[0038] like Figure 1 As shown, the embodiment of this specification provides a method for generating ship routes based on artificial intelligence, which is applied to a ship communication system, wherein the ship communication system includes a plurality of ships with integrated wireless modules and a server for coordinating ship communication spectrum, and the server may specifically be a base station. The method may specifically include the following steps S101 to S103:

[0039] In step S101, a route generation model is constructed based on deep reinforcement learning training to generate a waypoint set. The environment of the route generation model is based on a nautical chart between the destination and the departure point, and is combined with location data of impassable areas. Its actions include adjusting the speed and heading of the ship, and its state includes the current position, speed, and heading of the ship. The reward mechanism of the route generation model is set as follows: if the waypoint set generated for the t+1th time is less than the waypoint set generated for the tth time, or / and the actions of the tth and t+1th times in any waypoint set are the same, then the Q value is increased, and multiple waypoints in the waypoint set are connected into a route.

[0040] Among them, the basic environment is built based on the nautical chart, which contains the geographical information between the departure and destination. The nautical chart data contains water depth, impassable areas (such as shoals, reefs, land, etc.) and navigation rules (such as traffic separation zones). Impassable areas and specific areas that must be passed (such as pilot areas and specific navigation channels, green areas) are set in the environment. In the state information, the current position of the ship is the geographic coordinates, and the ship speed and heading are dynamic parameters. In the reward mechanism, path simplification, action consistency, area restrictions and penalties are included. In path simplification, if the number of waypoint sets generated for the t+1th time is less than that for the tth time, the model reward increases to encourage the reduction of unnecessary turning points; if the tth and t+1th actions are the same (for example, continuous selection to keep the direction unchanged), the reward is increased to avoid frequent turning. Finally, after the reward mechanism is adjusted, multiple waypoints are connected to form the ship's route to ensure that the ship can sail safely and efficiently from the departure point to the destination.

[0041] In step S102, when the rudder angle control vector is generated by applying an action to the ship according to the route, a fuzzy calculation is performed based on fuzzy rules with the heading angle error, speed and acceleration as inputs to obtain a first rudder angle control increment.

[0042] In order to achieve precise control of the ship and enable it to accurately follow the predetermined route, a control method based on fuzzy rules, namely the speed-type fuzzy PID controller, is proposed. The controller takes the heading angle error, speed and acceleration as input, and generates the increment of the ship's rudder angle through fuzzy logic calculation.

[0043] Specifically, the heading angle error is the difference between the current heading angle and the desired heading angle; the fuzzy rules are based on expert experience and control engineering knowledge. For example, if the heading angle error is positive and large, and the speed is positive and large, then the output first rudder angle control increment may be positive and large, which is used to determine the control action.

[0044] In step S103, the prior state of the ship is estimated based on the Kalman filter, and the prior state is compared with the actual state actually measured by the sensor to determine whether there is an environmental disturbance. When there is an environmental disturbance, based on fuzzy rules, the error between the prior state and the actual state is used as input, and a second rudder angle control increment is obtained after fuzzy calculation. The first rudder angle control increment, the second rudder angle control increment and the rudder angle control vector are combined to control the operation of the ship.

[0045] Among them, the Kalman filter is a recursive state estimation algorithm that uses a series of measurements to estimate the state of a dynamic system. In this case, the Kalman filter is used to estimate the prior state of the ship, that is, the prediction of the ship's state before considering the latest measurement information. Comparing the prior state estimated by the Kalman filter with the state actually measured by the sensor can help identify whether there is an environmental disturbance. If there is a significant difference between the two, it indicates that there is an impact of environmental disturbances. Specifically, by comparing the prior state and the actual state, it can be determined whether there is an environmental disturbance. If the difference exceeds a certain threshold or shows a specific pattern, it indicates the presence of environmental disturbances. If there is an environmental disturbance, another fuzzy logic controller will be used to estimate the impact of these disturbances and calculate the second rudder angle control increment. This increment is calculated based on the error between the prior state and the actual state through fuzzy rules. Specifically, another fuzzy logic controller uses fuzzy rules to process the error between the prior state and the actual state. These fuzzy rules define how to adjust the rudder angle according to the size and rate of change of the error to compensate for the impact of environmental disturbances. The first rudder angle control increment is calculated based on the ship's heading error, speed and acceleration, while the second rudder angle control increment is calculated based on the estimation of environmental disturbances. Combining these two increments, a comprehensive rudder angle control vector can be obtained. The final rudder angle control vector is used to control the actual operation of the ship, ensuring that the ship can accurately follow the planned route even in the presence of environmental disturbances.

[0046] In one embodiment, after the waypoint set is generated based on the route generation model, the waypoint set is simplified based on the Douglas-Peucker algorithm, so that the waypoints in the waypoint set are reduced.

[0047] Specifically, the Douglas-Peucker algorithm is a line simplification algorithm that reduces the number of waypoints by iteratively selecting key points in a polyline. In this embodiment, the goal of the algorithm is to reduce the number of waypoints while maintaining the overall shape of the polyline, thereby simplifying the route. The algorithm includes the following:

[0048] Select the starting and ending points: First determine the starting and ending points of the polyline. These two points will always be retained in the simplified route.

[0049] Calculate distance: Calculate the vertical distance from all other points on the polyline to the line connecting the starting point and the end point;

[0050] Find the maximum distance point: Find the point farthest from the line. This point has the greatest impact on the shape of the route, so it needs to be retained;

[0051] Recursive segmentation: Split the polyline into two parts, one from the starting point to the maximum distance point, and the other from the maximum distance point to the end point. Apply the Douglas-Peucker algorithm to each of the two parts.

[0052] Recursive termination condition: If the distance from all points on a certain polyline to the connecting line is less than the preset threshold, it is considered that this polyline can be simplified, and only the starting point and the end point are retained.

[0053] Through the above steps, the algorithm can effectively reduce the number of waypoints in the waypoint set. Each iteration checks whether all points are close enough to the simplified line. If so, these points can be removed. The simplified route requires less data to be processed, which reduces the amount of calculation. In the ship navigation system, the simplified route can be processed and displayed faster.

[0054] In one embodiment, when the fuzzy calculation of the heading angle error, velocity and acceleration is performed, it includes:

[0055] Mapping heading angle error, velocity and acceleration into multiple fuzzy sets;

[0056] According to the preset fuzzy rule base, the applicability of each rule is calculated and different rudder angle control fuzzy quantities are given;

[0057] Performing weighted calculation on all the rudder angle control fuzzy quantities to obtain a first comprehensive fuzzy output;

[0058] The first integrated fuzzy output is defuzzified to obtain the first rudder angle control increment.

[0059] In one embodiment, when performing fuzzy calculation on the error between the prior state and the actual state, it includes: mapping the error between the prior state and the actual state into multiple fuzzy sets; calculating the applicability of each rule according to a preset fuzzy rule base, and giving different rudder angle control fuzzy quantities; performing weighted calculation on all the rudder angle control fuzzy quantities to obtain a second comprehensive fuzzy output; defuzzifying the second comprehensive fuzzy output to obtain the impact force, roll force and yaw moment of the environmental disturbance; converting the impact force into the thrust to control the actual ship, and converting the roll force and yaw moment to control the rudder angle of the actual ship, and obtaining the second rudder angle control increment.

[0060] Among them, as an explanation, the error between the prior state provided by the Kalman filter and the actual state actually measured by the sensor is calculated, and this error is then mapped to multiple fuzzy sets. Fuzzy sets are concepts in fuzzy logic that are used to represent imprecise or uncertain information. In this case, the error is converted into membership in the fuzzy set, which indicates the degree to which the error belongs to each fuzzy set. According to the pre-set fuzzy rule base, each rule is evaluated to determine its applicability. These rules define the control actions that should be taken under different error conditions based on expert knowledge and previous experience. Each rule will give a fitness based on the membership of the error, which indicates the degree to which the rule is activated under the current error conditions. According to the fitness of each rule, different rudder angle control fuzzy quantities are calculated. These fuzzy quantities are based on the output of the rules and represent the suggestions for adjusting the rudder angle of the ship. Here, all the rudder angle control fuzzy quantities are weighted to obtain the second comprehensive fuzzy output. This second comprehensive output is a weighted average of all fuzzy quantities, where the weight of each fuzzy quantity is determined by the fitness of its corresponding rule. Then, the second comprehensive fuzzy output is defuzzified to obtain the impact force, roll force and yaw moment of the specific environmental disturbance. The defuzzification process can use the centroid method and the maximum membership method. The calculated impact force of the environmental disturbance is converted into the thrust to control the actual ship. That is, according to the magnitude of the impact force, the engine thrust of the ship is adjusted to offset the impact of the environmental disturbance. The roll force and yaw moment are converted into the rudder angle to control the actual ship. Specifically, according to the magnitude and direction of the roll force and yaw moment, the rudder angle of the ship is adjusted to keep the ship on the correct route. Finally, these adjustments are merged into the second rudder angle control increment and combined with the first rudder angle control increment calculated based on the ship's heading angle error to form the final rudder angle control vector, which is used to control the actual operation of the ship.

[0061] Based on the same idea, Figure 2 As shown, the embodiment of this specification also provides a ship route generation device based on artificial intelligence, the device comprising:

[0062] The model generation module 201 is used to construct a route generation model based on deep reinforcement learning training to generate a waypoint set. The environment of the route generation model is based on a nautical chart between the destination and the departure point, and is combined with the location data of the impassable area. Its actions include adjusting the speed and heading of the ship, and its state includes the current position, speed, and heading of the ship. The reward mechanism of the route generation model is set as follows: if the waypoint set generated at the t+1th time is less than the waypoint set generated at the tth time, or / and the actions of the tth and t+1th times in any waypoint set are the same, then the Q value is increased, and multiple waypoints in the waypoint set are connected into a route;

[0063] A fuzzy speed control module 202 is used to obtain a first rudder angle control increment after fuzzy calculation based on fuzzy rules, with the heading angle error, speed and acceleration as inputs when generating a rudder angle control vector by applying an action to the ship according to the route;

[0064] The fuzzy error estimation module 203 is used to estimate the prior state of the ship based on the Kalman filter, compare the prior state with the actual state actually measured by the sensor, determine whether there is an environmental disturbance, and when there is an environmental disturbance, based on fuzzy rules, use the error between the prior state and the actual state as input, obtain the second rudder angle control increment after fuzzy calculation, and combine the first rudder angle control increment, the second rudder angle control increment and the rudder angle control vector to control the operation of the ship.

[0065] The advantages of the device are as follows:

[0066] The optimal route is generated using machine learning technology based on the reinforcement learning algorithm, and deep reinforcement learning based on the Q-learning algorithm is introduced. The optimal route is generated in a way that ensures the safety of the ship while minimizing fuel consumption. The route is tracked based on fuzzy control, and the ship controller will not cause large overshoot, so that the ship will not deviate from the designated route. In addition, the ship's energy efficiency is improved due to the smooth change of the rudder angle. The environmental interference is estimated based on fuzzy control, and the size of the environmental interference is converted into the thrust and rudder angle that actually controls the ship. Even in the presence of environmental interference, the designated route can be accurately followed.

[0067] Based on the same idea, the embodiment of this specification also provides a ship route generation device based on artificial intelligence, such as Figure 3 shown.

[0068] The ship route generation device based on artificial intelligence can be the terminal device or server provided in the above embodiment.

[0069] The ship route generation device based on artificial intelligence may have relatively large differences due to different configurations or performances, and may include one or more processors 301 and memory 302, and one or more storage applications or data may be stored in the memory 302. Among them, the memory 302 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory unit, and may further include a read-only storage unit. The application stored in the memory 302 may include one or more program modules (not shown in the figure), such program modules include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of these examples or some combination may include the implementation of a network environment. Furthermore, the processor 301 can be configured to communicate with the memory 302 to execute a series of computer executable instructions in the memory 302 on the ship route generation device based on artificial intelligence. The artificial intelligence-based ship route generation device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more I / O interfaces (input and output interfaces) 305, one or more external devices 306 (such as keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a user to interact with the device, and / or communicate with any device that enables the device to communicate with one or more other computing devices (such as routers, modems, etc.). Such communication may be performed through the I / O interface 305. In addition, the device may also communicate with one or more networks (such as a local area network (LAN)) through the wired or wireless interface 304.

[0070] Specifically in this embodiment, the ship route generation device based on artificial intelligence includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions in the ship route generation device based on artificial intelligence, and the one or more programs are configured to be executed by one or more processors, including the following computer executable instructions:

[0071] A route generation model is constructed based on deep reinforcement learning training to generate a waypoint set. The environment of the route generation model is based on a nautical chart between a destination and a departure place, and is combined with location data of an impassable area. Its actions include adjusting the speed and heading of the ship, and its state includes the current position, speed, and heading of the ship. The reward mechanism of the route generation model is set as follows: if the waypoint set generated at the t+1th time is less than the waypoint set generated at the tth time, or / and the actions at the tth time and the t+1th time in any waypoint set are the same, then the Q value is increased, and multiple waypoints in the waypoint set are connected into a route;

[0072] When the rudder angle control vector is generated by applying an action to the ship according to the route, a first rudder angle control increment is obtained after fuzzy calculation based on fuzzy rules with the heading angle error, speed and acceleration as inputs;

[0073] The prior state of the ship is estimated based on the Kalman filter, and the prior state is compared with the actual state actually measured by the sensor to determine whether there is an environmental disturbance. When there is an environmental disturbance, based on fuzzy rules, the error between the prior state and the actual state is used as input, and a second rudder angle control increment is obtained after fuzzy calculation. The first rudder angle control increment, the second rudder angle control increment and the rudder angle control vector are combined to control the operation of the ship.

[0074] Based on the same idea, the exemplary embodiment of the present disclosure also provides a computer-readable storage medium on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Ship route generation method based on artificial intelligence" section of the present specification.

[0075] refer to Figure 4 As shown, a program product 400 for implementing the above method according to an exemplary embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, system or device.

[0076] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0077] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, system, or device.

[0078] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0079] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0080] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal system, or a network device, etc.) to execute the method according to the exemplary implementation of the present disclosure.

[0081] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiment of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0082] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0083] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A method for generating ship routes based on artificial intelligence, characterized in that: The method comprises: A route generation model is constructed based on deep reinforcement learning training to generate a waypoint set. The environment of the route generation model is based on a nautical chart between a destination and a departure place, and is combined with location data of an impassable area. Its actions include adjusting the speed and heading of the ship, and its state includes the current position, speed, and heading of the ship. The reward mechanism of the route generation model is set as follows: if the waypoint set generated at the t+1th time is less than the waypoint set generated at the tth time, or / and the actions at the tth time and the t+1th time in any waypoint set are the same, then the Q value is increased, and multiple waypoints in the waypoint set are connected into a route; When the rudder angle control vector is generated by applying an action to the ship according to the route, a first rudder angle control increment is obtained after fuzzy calculation based on fuzzy rules with the heading angle error, speed and acceleration as inputs; The prior state of the ship is estimated based on the Kalman filter, and the prior state is compared with the actual state actually measured by the sensor to determine whether there is an environmental disturbance. When there is an environmental disturbance, based on fuzzy rules, the error between the prior state and the actual state is used as input, and a second rudder angle control increment is obtained after fuzzy calculation. The first rudder angle control increment, the second rudder angle control increment and the rudder angle control vector are combined to control the operation of the ship.

2. The method for generating ship routes based on artificial intelligence according to claim 1, characterized in that: The impassable areas include: shallow water areas, restricted areas, islands, dangerous areas, and land.

3. The method for generating ship routes based on artificial intelligence according to claim 1, characterized in that: After the waypoint set is generated based on the route generation model, the waypoint set is simplified based on the Douglas-Peucker algorithm, so that the waypoints in the waypoint set are reduced.

4. The method for generating ship routes based on artificial intelligence according to claim 1, characterized in that: When performing fuzzy calculations on heading angle error, velocity, and acceleration, include: Mapping heading angle error, velocity and acceleration into multiple fuzzy sets; According to the preset fuzzy rule base, the applicability of each rule is calculated and different rudder angle control fuzzy quantities are given; Performing weighted calculation on all the rudder angle control fuzzy quantities to obtain a first comprehensive fuzzy output; The first integrated fuzzy output is defuzzified to obtain the first rudder angle control increment.

5. The method for generating ship routes based on artificial intelligence according to claim 1, characterized in that: When the error between the prior state and the actual state is fuzzily calculated, it includes: Mapping the error between the prior state and the actual state into a plurality of fuzzy sets; According to the preset fuzzy rule base, the applicability of each rule is calculated and different rudder angle control fuzzy quantities are given; Performing weighted calculation on all the rudder angle control fuzzy quantities to obtain a second comprehensive fuzzy output; Defuzzifying the second comprehensive fuzzy output to obtain the impact force, rolling force and yaw moment of the environmental disturbance; The impact force is converted into a thrust for controlling an actual ship, and the roll force and the yaw moment are converted into a rudder angle for controlling the actual ship, so as to obtain the second rudder angle control increment.

6. A ship route generation device based on artificial intelligence, characterized in that: The device comprises: A model generation module is used to construct a route generation model based on deep reinforcement learning training to generate a waypoint set. The environment of the route generation model is based on a nautical chart between the destination and the departure point, and is combined with location data of an impassable area. Its actions include adjusting the speed and heading of the ship, and its state includes the current position, speed, and heading of the ship. The reward mechanism of the route generation model is set as follows: if the waypoint set generated at the t+1th time is less than the waypoint set generated at the tth time, or / and the actions of the tth and t+1th times in any waypoint set are the same, then the Q value is increased, and multiple waypoints in the waypoint set are connected into a route; A fuzzy speed control module is used to obtain a first rudder angle control increment after fuzzy calculation based on fuzzy rules, with heading angle error, speed and acceleration as inputs when generating a rudder angle control vector by applying an action to the ship according to the route; The fuzzy error estimation module is used to estimate the prior state of the ship based on the Kalman filter, compare the prior state with the actual state actually measured by the sensor, determine whether there is an environmental disturbance, and when there is an environmental disturbance, based on fuzzy rules, use the error between the prior state and the actual state as input, obtain the second rudder angle control increment after fuzzy calculation, and combine the first rudder angle control increment, the second rudder angle control increment and the rudder angle control vector to control the operation of the ship.

7. A ship route generation device based on artificial intelligence, characterized in that: include: processor; and a memory arranged to store computer executable instructions which, when executed, cause the processor to: A route generation model is constructed based on deep reinforcement learning training to generate a waypoint set. The environment of the route generation model is based on a nautical chart between a destination and a departure place, and is combined with location data of an impassable area. Its actions include adjusting the speed and heading of the ship, and its state includes the current position, speed, and heading of the ship. The reward mechanism of the route generation model is set as follows: if the waypoint set generated at the t+1th time is less than the waypoint set generated at the tth time, or / and the actions at the tth time and the t+1th time in any waypoint set are the same, then the Q value is increased, and multiple waypoints in the waypoint set are connected into a route; When the rudder angle control vector is generated by applying an action to the ship according to the route, a first rudder angle control increment is obtained after fuzzy calculation based on fuzzy rules with the heading angle error, speed and acceleration as inputs; The prior state of the ship is estimated based on the Kalman filter, and the prior state is compared with the actual state actually measured by the sensor to determine whether there is an environmental disturbance. When there is an environmental disturbance, based on fuzzy rules, the error between the prior state and the actual state is used as input, and a second rudder angle control increment is obtained after fuzzy calculation. The first rudder angle control increment, the second rudder angle control increment and the rudder angle control vector are combined to control the operation of the ship.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the ship route generation method based on artificial intelligence as described in any one of claims 1 to 5 is implemented.

Images