Ship steering feedback control method and device based on bistable contribution factor

Abstract

The application relates to a ship steering feedback control method and device based on a bistable contribution factor, and comprises the following steps: step S1, initializing an experimental state value; step S2, randomly selecting a rudder angle in a predetermined rudder angle range value, and completing corresponding rudder angle operation after selection, and then updating the experimental state value; step S3, obtaining a real-time bistable contribution factor of the ship body based on the rudder angle operation; step S4, calculating the distance L between the ship body and the center coordinates of a target terminal based on the updated experimental state value; step S5, calculating a feedback value based on the real-time bistable contribution factor and the distance L, judging whether the bistable state is reached at the moment based on the feedback value, if yes, ending the control, and if not, repeatedly executing the steps S1 to S5. The application can train the ship intelligent agent with relatively few necessary variables, reduce the occurrence of excessively large steering amplitude, and increase the stability and safety of ship driving in practical scenes.

Description

Technical Field

[0001] This invention relates to the field of ship autopilot technology, and in particular to a ship steering feedback control method and device based on bistable contribution factors. Background Technology

[0002] With the increasing number of ships and the various unpredictable factors at sea, the requirements for ship navigation skills are constantly rising, which also means that the requirements for ship steering skills are becoming more stringent. For ships at sea, precise steering can greatly prevent maritime safety accidents and plays a vital role in the stability and safety of ship navigation.

[0003] The rapid development of modern technology has enabled the widespread application of reinforcement learning in the field of autonomous driving, and reinforcement learning-based ship autopilot technology is one such application. This technology provides innovative solutions for ships to achieve autonomous navigation and steering gear control. Especially in steering gear control, reinforcement learning can optimize control strategies, enabling autonomous steering of the ship and thus achieving unmanned operation.

[0004] Autonomous ships based on reinforcement learning can achieve automatic control of the steering gear. Specifically, each time an autonomous ship enters a new state, it selects a movement and a rudder angle to determine its course. However, in the process of selecting the rudder angle, autonomous ships are prone to excessive steering input. If this problem cannot be effectively solved, it is extremely detrimental to the stability and safety of the ship, posing a serious threat to the lives of the crew and potentially causing damage to the hull and resulting in significant economic losses. Therefore, solving the steering problem of reinforcement learning-based autonomous ships is essential. Summary of the Invention

[0005] The purpose of this invention is to at least address one of the shortcomings of the prior art and to provide a ship steering feedback control method and apparatus based on bistable contribution factors.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: Specifically, a ship steering feedback control method based on bistable contribution factors is proposed, including the following: Step S1: Initialize experimental state values, whereby the experimental state values ​​refer to the x-coordinate and y-coordinate of the ship's position on the sea surface. The initial coordinate values ​​defined by the heading angle; Step S2: The hull randomly selects a rudder angle within a predetermined range of rudder angle values, and performs the corresponding rudder angle control after selection, and then updates the experimental status value. Step S3: Obtain the real-time bistable contribution factor of the hull based on the rudder angle operation; Step S4: Calculate the distance L between the center coordinates of the ship and the target endpoint based on the updated experimental state values; Step S5: Calculate the feedback value based on the real-time bistable contribution factor and the distance L. Determine whether the bistable state has been reached based on the feedback value. If yes, end the control. Otherwise, repeat steps S1 to S5.

[0007] Furthermore, specifically, the initial experimental state value is (x, y, 0), that is, the initial value at the time of initialization. =0.

[0008] Furthermore, specifically, rudder angle operation refers to adjusting the rudder angle Φ towards the heading angle. Perform the conversion, and then adjust the heading angle. Adjustments were made, and the experimental state values ​​were updated accordingly. The rudder angle Φ is the heading angle. The conversion formula is as follows:

[0009] in, Φ is the heading angle, K and T are the maneuverability index and the follower index, respectively, both of which are human-defined constants, and t is the time parameter at the current moment.

[0010] Furthermore, specifically, K=0.08 and T=10.08.

[0011] Furthermore, specifically, the real-time bistable contribution factor of the hull is obtained based on the rudder angle operation, including: The rudder angle value Φ is defined as the action value a, and the bistable contribution factor includes the main contribution factor A and the auxiliary contribution factor. Then there is, A= ,in This represents the action value corresponding to the previous state. This represents the action value corresponding to the current state. ,in It is the difference between the heading angle before and after this action.

[0012] Furthermore, specifically, the distance L between the center coordinates of the hull and the target endpoint is calculated based on the updated experimental state values, including: The formula for calculating L is:

[0013] in , The x and y coordinates corresponding to the updated experimental state values. , These are the x and y coordinates of the endpoint center.

[0014] Furthermore, specifically, the feedback value is calculated based on the real-time bistable contribution factor and the distance L, including: The feedback value is calculated using a pre-set steady-state master control feedback function, which is as follows:

[0015] Where r = - This is the feedback value.

[0016] Furthermore, specifically, determining whether a bistable state has been reached based on the feedback value includes: Determine whether the feedback value is positive. If it is, it indicates that a bistable state has been reached; otherwise, it indicates that a bistable state has not been reached.

[0017] This invention also proposes a ship steering feedback control device based on a bistable contribution factor, comprising: The initialization module is used to initialize the experimental state values, which refer to the x-coordinate and y-coordinate of the ship's position on the sea surface. The initial coordinate values ​​defined by the heading angle; The status value update module is used to enable the ship to randomly select a rudder angle within a predetermined rudder angle range, and to perform the corresponding rudder angle control after selection, and then update the experimental status value. The bistable contribution factor calculation module is used to obtain the real-time bistable contribution factor of the hull based on the rudder angle operation. The distance calculation module is used to calculate the distance L between the center coordinates of the ship and the target endpoint based on the updated experimental state values; The loop control module is used to calculate the feedback value based on the real-time bistable contribution factor and the distance L, and to determine whether the bistable state has been reached based on the feedback value. If so, the control ends; otherwise, the initialization module, the state value update module, the bistable contribution factor calculation module, the distance calculation module, and the loop control module are executed repeatedly.

[0018] The beneficial effects of this invention are as follows: This invention proposes a ship steering feedback control method and device based on a bistable contribution factor. By acquiring the ship's coordinates, heading angle, and change values ​​of its actions, and combining these with the bistable contribution factor, the method guides the ship to sail stably, reduces heading changes, and encourages the ship's intelligent agent to move closer to the center of gravity. With relatively few necessary variables, the ship's intelligent agent can be trained to reduce the occurrence of excessive steering amplitude. In practical applications, this method can increase the stability and safety of ship navigation. Attached Figure Description

[0019] The above and other features of this disclosure will become more apparent from the detailed description of the embodiments illustrated in conjunction with the accompanying drawings. In the drawings of this disclosure, the same reference numerals denote the same or similar sampling monitoring points. Obviously, the drawings described below are merely some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort. In the drawings: Figure 1 The diagram shows a flowchart of the ship steering feedback control method based on bistable contribution factors according to the present invention. Figure 2 The diagram shows a flowchart of the single-cycle calculation of feedback values ​​in the ship steering feedback control method based on bistable contribution factors according to the present invention. Detailed Implementation

[0020] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The same reference numerals used throughout the accompanying drawings indicate the same or similar parts.

[0021] Example 1, referring to Figure 1 as well as Figure 2 This invention proposes a ship steering feedback control method based on a bistable contribution factor, comprising the following: Step S1: Initialize experimental state values, whereby the experimental state values ​​refer to the x-coordinate and y-coordinate of the ship's position on the sea surface. The initial coordinate values ​​defined by the heading angle; Step S2: The hull randomly selects a rudder angle within a predetermined rudder angle range, and performs the corresponding rudder angle control after selection, then updates the experimental state value; in a preferred embodiment, the rudder angle range can be (-35°, -15°, 0°, 15°, 35°) in combination with actual application. Step S3: Obtain the real-time bistable contribution factor of the hull based on the rudder angle operation; Step S4: Calculate the distance L between the center coordinates of the ship and the target endpoint based on the updated experimental state values; Step S5: Calculate the feedback value based on the real-time bistable contribution factor and the distance L. Determine whether the bistable state has been reached based on the feedback value. If yes, end the control. Otherwise, repeat steps S1 to S5.

[0022] In this embodiment 1, when the difference between two consecutive action values ​​is less than or equal to 10 (10 is a safe rudder difference value), it can be considered that the ship's intelligent agent has successfully reached a bistable state after feedback adjustment through the steady-state master control feedback function. In fact, the purpose of the steady-state master control feedback function is to train the ship to keep its steering amplitude within the safe rudder difference range, that is, to ensure that the difference between two consecutive rudder angles does not exceed 10. This is the goal of training the ship through the steady-state master control feedback function. Based on this idea, by obtaining the ship's coordinates, heading angle, and action change values, combined with the bistable contribution factor, the ship can be guided to sail stably, reducing heading changes and encouraging the ship's intelligent agent to move closer to the endpoint center. With relatively few necessary variables, the ship's intelligent agent can be trained to reduce the occurrence of excessive steering amplitude. In practical applications, this can increase the stability and safety of ship navigation.

[0023] In a preferred embodiment of the present invention, specifically, the initial experimental state value is (x, y, 0), that is, the initial value at the time of initialization. =0.

[0024] In this preferred embodiment, the initial experimental state value is the initial coordinate value of the ship agent. Where x and y are the initial horizontal and vertical coordinates of the ship's intelligent agent on the sea surface. Let be the heading angle, with an initial value of 0. Therefore, the initial experimental state value is...

[0025] In a preferred embodiment of the present invention, specifically, rudder angle operation refers to adjusting the rudder angle Φ towards the heading angle. Perform the conversion, and then adjust the heading angle. Adjustments were made, and the experimental state values ​​were updated accordingly. The rudder angle Φ is the heading angle. The conversion formula is as follows:

[0026] in, Φ is the heading angle, K and T are the maneuverability index and the follower index, respectively, both of which are human-defined constants, and t is the time parameter at the current moment.

[0027] In this preferred embodiment, the conversion between heading angle and rudder angle using the above method is derived based on the following formula.

[0028] In Formula 1, Ψ represents the heading angle. . The rudder angle value is defined in [the text]. The initial rudder angle is given. K and T are the maneuverability and follower indices, respectively, both constants with K = 0.08 and T = 10.08. t is an arbitrary time point, which can be adjusted to specific parameters according to the experimental requirements. In other words, t can be considered, in a sense, a self-defined time constant.

[0029] In a preferred embodiment of the present invention, specifically, the real-time bistable contribution factor of the hull is obtained based on the rudder angle operation, including... The rudder angle value Φ is defined as the action value a, and the bistable contribution factor includes the main contribution factor A and the auxiliary contribution factor. Then there is, A= ,in This represents the action value corresponding to the previous state. This represents the action value corresponding to the current state. ,in It is the difference between the heading angle before and after this action.

[0030] In this preferred embodiment, the main contribution factor A and the auxiliary contribution factor The rudder angle and heading angle are controlled separately. Therefore, it can be understood that in a bistable state, one part of the steady state is contributed by the rudder angle, and the other part is contributed by the heading angle.

[0031] This also explains the main contribution factor A and the auxiliary contribution factor. If the rudder angle and heading angle can be adjusted and controlled within the safe range for safe navigation of the ship (controlled by the steady-state main control feedback function below), then the ship will be penalized if the rudder angle or the course changes too much, thus ensuring the ship's smooth navigation and achieving the adjustment of navigation by the two steady-state factors.

[0032] In a preferred embodiment of the present invention, specifically, calculating the distance L between the center coordinates of the hull and the target endpoint based on the updated experimental state values ​​includes: The formula for calculating L is:

[0033] in , The x and y coordinates corresponding to the updated experimental state values. , These are the x and y coordinates of the endpoint center.

[0034] In this preferred embodiment, the ship agent obtains real-time state coordinates during S2. Based on these state coordinates, the distance L between the ship agent and the destination center coordinates can be calculated. Distance L encourages the ship agent to move towards the destination center coordinates. As a preferred embodiment of the present invention, specifically, the feedback value is calculated based on the real-time bistable contribution factor and the distance L, including: The feedback value is calculated using a pre-set steady-state master control feedback function, which is as follows:

[0035] Where r = - This is the feedback value.

[0036] In this preferred embodiment, after completing steps S3 and S4, the ship's intelligent agent is given corresponding feedback according to the steady-state master control feedback function. The steady-state master control feedback function aims to guide the ship's intelligent agent to take appropriate actions by rewarding stable heading, reducing heading changes, and moving closer to the destination center, so as to achieve safe and effective automatic shipboard control.

[0037] More specifically: A is the primary contributing factor. The A value, calculated through feedback, encourages the ship's agent to maintain a stable course, thus avoiding frequent directional changes. The calculation process is relatively simple; knowing the changes in the course before and after the action is sufficient to determine the magnitude of A, which is then used to control the ship's agent's course stability.

[0038] It is an auxiliary contribution factor, feedback It enables the ship's intelligent agent to minimize changes in the heading angle, that is, to maintain stability in the heading direction. The calculation only requires obtaining the change in the ship's forward and backward heading angles, which is relatively simple.

[0039] L is the distance between the ship agent and the destination center coordinates. The magnitude of L plays an important role in encouraging the ship agent to move towards the destination center coordinates.

[0040] In this preferred embodiment, specifically, determining whether a bistable state has been reached based on the feedback value includes: Determine whether the feedback value is positive. If it is, it indicates that a bistable state has been reached; otherwise, it indicates that a bistable state has not been reached.

[0041] In a preferred embodiment, the determination of whether a bistable state has been reached can be considered using a feedback value. This involves calculating the feedback value using the method r = - Considering The calculation process, if The value is positive, if , The value is negative, so we can directly use the final feedback value (i.e., r = -). The sign of the feedback value is used to determine whether a bistable state has been reached. If the final feedback value is positive, a bistable state has been reached; if it is negative, a bistable state has not been reached. This method allows for a quick determination of the result.

[0042] This invention also proposes a ship steering feedback control device based on a bistable contribution factor, comprising: The initialization module is used to initialize the experimental state values, which refer to the x-coordinate and y-coordinate of the ship's position on the sea surface. The initial coordinate values ​​defined by the heading angle; The status value update module is used to enable the ship to randomly select a rudder angle within a predetermined rudder angle range, and to perform the corresponding rudder angle control after selection, and then update the experimental status value. The bistable contribution factor calculation module is used to obtain the real-time bistable contribution factor of the hull based on the rudder angle operation. The distance calculation module is used to calculate the distance L between the center coordinates of the ship and the target endpoint based on the updated experimental state values; The loop control module is used to calculate the feedback value based on the real-time bistable contribution factor and the distance L, and to determine whether the bistable state has been reached based on the feedback value. If so, the control ends; otherwise, the initialization module, the state value update module, the bistable contribution factor calculation module, the distance calculation module, and the loop control module are executed repeatedly.

[0043] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0044] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or system capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0045] Although the description of the invention has been quite detailed and particularly of several described embodiments, it is not intended to limit it to any of these details or embodiments or any particular embodiment, but should be considered as providing a broad possible interpretation of the claims by referring to the appended claims and taking into account the prior art, thereby effectively covering the intended scope of the invention. Furthermore, the invention has been described above with respect to embodiments foreseeable by the inventors in order to provide a useful description, and non-substantial modifications to the invention that have not yet been foreseen may still represent equivalent modifications.

[0046] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any embodiment that achieves the technical effects of the present invention using the same means should fall within the protection scope of the present invention. Within the protection scope of the present invention, various modifications and variations can be made to the technical solutions and / or implementation methods.

Claims

1. A ship steering feedback control method based on bistable contribution factors, characterized in that, Includes the following steps: Step S1: Initialize experimental state values, whereby the experimental state values ​​refer to the x-coordinate and y-coordinate of the ship's position on the sea surface. The initial coordinate values ​​defined by the heading angle; Step S2: The hull randomly selects a rudder angle within a predetermined range of rudder angle values, and performs the corresponding rudder angle control after selection, and then updates the experimental status value. Step S3: Obtain the real-time bistable contribution factor of the hull based on the rudder angle operation; Step S4: Calculate the distance L between the center coordinates of the ship and the target endpoint based on the updated experimental state values; Step S5: Calculate the feedback value based on the real-time bistable contribution factor and the distance L. Determine whether the bistable state has been reached based on the feedback value. If yes, end the control. Otherwise, repeat steps S1 to S5. Specifically, rudder angle operation refers to adjusting the rudder angle Φ towards the heading angle. Perform the conversion, and then adjust the heading angle. Adjustments were made, and the experimental state values ​​were updated accordingly. The rudder angle Φ is in the direction of the heading angle. The conversion formula is as follows: in, Φ is the heading angle, K and T are the maneuverability index and the follower index, respectively, both of which are human-defined constants, and t is the time parameter at the current moment. Specifically, based on the rudder angle operation, the corresponding real-time bistable contribution factor of the hull is obtained, including: The rudder angle value Φ is defined as the action value a, and the bistable contribution factor includes the main contribution factor A and the auxiliary contribution factor. Then there is, A= ,in This represents the action value corresponding to the previous state. This represents the action value corresponding to the current state. ,in It is the difference between the heading angle before and after this action; Specifically, the feedback value is calculated based on the real-time bistable contribution factor and the distance L, including: The feedback value is calculated using a pre-set steady-state master control feedback function, which is as follows: Where r = - This is the feedback value; Specifically, determining whether a bistable state has been reached based on the feedback value includes: Determine whether the feedback value is positive. If it is, it indicates that a bistable state has been reached; otherwise, it indicates that a bistable state has not been reached.

2. The ship steering feedback control method based on bistable contribution factor according to claim 1, characterized in that, Specifically, the initial experimental state value is (x, y, 0), that is, the initial value at the time of initialization. =0.

3. The ship steering feedback control method based on bistable contribution factor according to claim 1, characterized in that, Specifically, K=0.08 and T=10.

08.

4. The ship steering feedback control method based on bistable contribution factor according to claim 1, characterized in that, Specifically, the distance L between the center coordinates of the hull and the target endpoint is calculated based on the updated experimental state values, including: The formula for calculating L is: in , The x and y coordinates corresponding to the updated experimental state values. , These are the x and y coordinates of the endpoint center.

5. A ship steering feedback control device based on a bistable contribution factor, characterized in that, The apparatus comprising the steps of the method according to any one of claims 1-4, wherein the method is applied, the apparatus includes: The initialization module is used to initialize the experimental state values, which refer to the x-coordinate and y-coordinate of the ship's position on the sea surface. The initial coordinate values ​​defined by the heading angle; The status value update module is used to enable the ship to randomly select a rudder angle within a predetermined rudder angle range, and to perform the corresponding rudder angle control after selection, and then update the experimental status value. The bistable contribution factor calculation module is used to obtain the real-time bistable contribution factor of the hull based on the rudder angle operation. The distance calculation module is used to calculate the distance L between the center coordinates of the ship and the target endpoint based on the updated experimental state values; The loop control module is used to calculate the feedback value based on the real-time bistable contribution factor and the distance L, and to determine whether the bistable state has been reached based on the feedback value. If so, the control ends; otherwise, the initialization module, the state value update module, the bistable contribution factor calculation module, the distance calculation module, and the loop control module are executed repeatedly.

Images