Compensation for inaccurate user input in ship operation

Abstract

The present invention provides a method for compensating for inaccurate user input in the operation of a vessel, a command adjustment device, a computer program, and a computer program product, as well as a ship control system and a ship equipped with a command adjustment device. [Solution] The device detects the environment of the vessel (22), determines at least one distance (d1, d2) to an object (38) in the environment of the vessel (22) when the vessel is operated using at least one control command issued by the vessel operator, where each distance is related to a corresponding point (P1, P2) on the vessel (22), analyzes at least one distance, determines at least one compensation (SC, VOC) for at least one control command based on the analysis, and provides at least one control command compensation for use in the operation of the vessel (22) being carried out by the operator.

Description

[Technical Field]

[0001] The present invention relates to a method for compensating inaccurate user input in ship steering, a command adjusting device, a computer program, a computer program product, a ship control system including the command adjusting device, and a ship including the ship control system. [Background Art]

[0002] Many developments centered on the automation and autonomy of ships are in progress worldwide in the marine industry.

[0003] Systems and solutions for collision avoidance in automatic operation, automatic docking and the like have been proposed. Most of the proposed solutions are either completely automatic until handover by a human operator, or completely manual.

[0004] However, it is expected that most ships will have a low level of steering automation in the near future. There are situation awareness systems and electronic chart display and information systems (ECDIS) that can be used by operators to help understand how to approach a quay and what the overall situation is.

[0005] In addition, many shipowners restrict the use of dynamic positioning (DP) systems during approach and maneuvering in ports. Rather, they consider it better to adopt a system that allows the crew to always keep control.

[0006] Therefore, in many cases, it is desirable for an operator to take charge of controlling the ship. Furthermore, automation solutions that assist the operator should be implemented to allow the operator to maintain control.

[0007] Therefore, a vessel can be operated manually using user interface devices such as a steering wheel, levers, buttons, or joysticks, and in such operation, it may be further necessary for the operator to use the user interface devices precisely to make the correct movement without causing a collision.

[0008] To provide safety for objects located above, under, or below the water surface, it would be interesting to enable operators to be less precise without the need to switch to autonomous control or DP operation mode. [Overview of the project]

[0009] Therefore, one of the objectives of the present invention is to enhance the safety of operator-controlled vessels.

[0010] The objective is achieved, according to the first aspect, by a method for compensating for inaccurate user input in the operation of a vessel, the method being performed by a command adjustment device. Detecting the environment of a ship, To determine at least one distance from a vessel to an object in the vessel's environment when the vessel is operated using at least one control command issued by the vessel's operator, where each distance is related to a corresponding point on the vessel. Analyzing at least one distance in relation to avoiding collisions with objects, Based on the analysis of at least one distance, determine at least one compensation for at least one control command, The invention includes providing at least one control command compensation for use in ship operations performed by an operator.

[0011] This objective is achieved, according to a second aspect, by a command adjustment device for compensating for inaccurate user input in the operation of a vessel, the command adjustment device comprising a processor, the processor is Detecting the environment of a ship, To determine at least one distance from a vessel to an object in the vessel's environment when the vessel is operated using at least one control command issued by the vessel's operator, where each distance is related to a corresponding point on the vessel. Analyze at least one distance, Based on the analysis of at least one distance, determine at least one compensation for at least one control command, It operates to provide at least one control command compensation for use in ship operations performed by the operator.

[0012] This objective is achieved, according to a third aspect, by a ship control system comprising a user interface device, a sensor unit, a ship control device, and the command adjustment device described in the second aspect.

[0013] This objective is achieved, according to the fourth aspect, by a vessel equipped with the ship control system described in the third aspect.

[0014] The objective is achieved, according to a fifth aspect, by a computer program for compensating for inaccurate user input in the operation of a vessel, the computer program comprising computer program code, which, when executed by the processor of a command coordination device, to the processor, Detecting the environment of a ship, To determine at least one distance from a vessel to an object in the vessel's environment when the vessel is operated using at least one control command issued by the vessel's operator, where each distance is related to a corresponding point on the vessel. Analyze at least one distance, Based on the analysis of at least one distance, determine at least one compensation for at least one control command, To provide at least one control command compensation for use in ship operations performed by the operator, and to enable such operation.

[0015] This objective is achieved, according to the sixth aspect, by a computer program product for compensating for inaccurate user input in the operation of a vessel, the computer program product comprising a data carrier having the computer program code described in the fifth aspect.

[0016] According to a first variation of the first embodiment, the method further comprises determining at least one velocity of a vessel toward an object when the vessel is operated by at least one control command, and analyzing at least one velocity, wherein determining at least one control command compensation is also based on the analysis of at least one velocity.

[0017] According to a corresponding variant of the second embodiment, the command adjustment device further operates to determine at least one velocity of the vessel toward an object when the vessel is operated by at least one control command, and to analyze at least one velocity, and the determination of at least one control command compensation is also based on the analysis of at least one velocity.

[0018] At least one control command compensation may include command compensation to reduce the ship's speed.

[0019] Speed ​​reduction can also be a reduction in speed that decreases as the distance to an object decreases, such as when a ship gets closer to an object and its speed decreases. Therefore, the analysis may involve analyzing the relationship between at least one distance, at least one speed, and time.

[0020] The decrease in velocity as distance decreases can be further calculated while taking into account the minimum constant time until encounter with an object.

[0021] The at least one control command compensation may be command compensation for reducing the speed of the ship in the direction towards an object when a collision risk is deemed to exist. A collision risk may be deemed to exist when the ship speed exceeds a ship speed threshold, and the ship speed threshold may be adaptive.

[0022] The adaptation may be based on user settings, or may be achieved by using the current capability of the ship to decelerate its movement. These capabilities may depend on which propulsion devices are active and / or how many generators are currently in operation. Alternatively, the ship speed threshold may be adapted to match the comfort level of passengers on board the ship.

[0023] Furthermore, a control command that causes speed or force in a direction contributing to movement towards an object may be compensated by control command compensation that causes speed or force in the opposite direction.

[0024] The control command compensation in the opposite direction may further be applied so as to increase gradually.

[0025] Determining the at least one distance may comprise determining a first distance to the object at a first point on the ship, and determining a second distance to the object at a second point on the ship.

[0026] In this case, determining the at least one speed may comprise determining the speed towards the object at each of these points.

[0027] The at least one control command compensation may further comprise at least one vessel orientation compensation.

[0028] In this case, in a further variant of the first aspect, the method may further comprise estimating a desired ship orientation based on the shape of the ship, the actual ship orientation, and the shape of the object.

[0029] In this case, in the corresponding variant of the second embodiment, the command adjustment device may further operate to estimate a desired ship orientation based on the shape of the ship, the actual orientation of the ship, and the shape of the object.

[0030] Determining at least one ship orientation compensation may further include determining ship orientation compensation to counteract the portion where the greater speed exceeds the other speed.

[0031] The first and second points are points on the first side of the vessel, at different distances from the object, and one of the points is assumed to have a faster velocity than the other if it is closer to the object. It is further possible that determining at least one vessel orientation compensation is to determine a vessel orientation compensation that reduces the difference in the distances from the two points to the object.

[0032] Here, it is further possible that the control command is provided to a point on the second side opposite to the point on the first side that is closer to the object than the other point on the first side.

[0033] Alternatively, the object can be considered a repulsive element that has the same virtual potential as the ship's virtual potential and causes a force or velocity change that is dependent on the distance / radius from the object. This type of virtual potential method can also be implemented so that a virtual potential field is determined between the object and the ship, and the changes in the ship's velocity and orientation are determined based on this field.

[0034] Another alternative is that the distance to at least one point is determined based on the current position and orientation of a fixed point on the vessel relative to a fixed location in the world coordinate system. In this case, the fixed point(s) on the vessel may be compared to a fixed point(s) on an object / in the world, and the distance and velocity to the point are used not only to control the velocity in its degree of freedom (e.g., laterally), but also to determine the angle and change in angle (angular velocity) to determine compensation in other degrees of freedom (e.g., forward).

[0035] Providing at least one control command compensation for use in ship operation may further include adding at least one control command compensation to a control command for a ship control device which is determined based on user input made by an operator.

[0036] The vessel may have a port side, a starboard side, a bow, and a stern, and at least one distance and at least one speed measurement may be performed at the bow, at least one distance and at least one speed measurement may be performed at the bow, at least two distances and at least two speed measurements may be performed on the port side, and at least two distances and at least two speed measurements may be performed on the starboard side.

[0037] At least one control command adjustment can be fed back to the operator via a user interface device.

[0038] According to another variation of the first embodiment, the method further comprises receiving user input, converting the user input into control commands, and supplying control commands to a ship control device for controlling a ship.

[0039] According to a corresponding variant of the second embodiment, the command adjustment device further operates to receive user input, convert the user input into control commands, and supply control commands to a ship control device for controlling the ship.

[0040] When used herein, the term “comprises / comprising” is used to identify the presence of a described feature, step, or component, but it should be emphasized that it does not preclude the presence or addition of one or more other features, steps, components, or groups thereof.

[0041] Next, the present invention will be described in detail with reference to the attached drawings. [Brief explanation of the drawing]

[0042] [Figure 1] A schematic representation of the first method for implementing a command adjustment device is provided below. [Figure 2] This describes a computer program product in the form of a CD-ROM containing computer program code used to implement a command adjustment device. [Figure 3] A schematic diagram of a vessel equipped with a command adjustment device, as well as a user interface device, a sensor unit, and a ship control device. [Figure 4] A schematic diagram of a ship control system comprising an input conversion module, a user interface device, a command adjustment device, a sensor unit, and a ship control device is shown. [Figure 5] Several method steps in a first embodiment of a method for compensating for inaccurate user input in ship operation are schematically shown. [Figure 6] A schematic diagram of a vessel that is operated on an object and has four propulsion systems. [Figure 7] A flowchart of several method steps in a second embodiment of a method for compensating for inaccurate user input in ship operation is shown. [Modes for carrying out the invention]

[0043] The following description includes specific details, such as particular architectures, interfaces, and techniques, for illustrative purposes only, not limiting purposes, to provide a complete understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be carried out in other embodiments departing from these specific details. In other instances, detailed descriptions of well-known devices, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary details.

[0044] Figure 1 schematically shows one implementation of the command coordination device CAD10. In this example, the command coordination device 10 comprises a processor PR12 and a memory or data storage 14 having computer program instructions or computer program code 16 that, when executed by the processor 12, implement the command coordination module and the perception module.

[0045] Therefore, the command adjustment device 10 may include a processor 12 having an associated program memory 14 containing computer program code 16 for implementing a command adjustment module and a perception module.

[0046] Computer programs can also be provided via computer program products in the form of non-temporary storage media such as CD-ROMs or memory sticks, or computer-readable storage media such as data carriers, which hold computer programs having computer program code that implements the aforementioned modules when loaded into a processor. One such computer program product in the form of a CD-ROM 20 having the aforementioned computer program code 16 is schematically shown in Figure 2.

[0047] The command adjustment device 10 can be advantageously installed inside the vessel.

[0048] Figure 3 schematically shows a vessel V22 equipped with a command adjustment device CAD10. The vessel V22 also includes a user interface device UID24 such as a joystick, lever, or steering wheel, and a vessel control device VCD26 for controlling the vessel V22, for example, for controlling how the vessel V22 moves within the water. The vessel V22 further includes a sensor unit SU28, which measures relevant quantities such as vessel velocity, and optionally the actual state of control signals. The sensor unit may also be equipped with the ability to collect sensor data such as radar echoes, lidar images, ultrasonic echoes, and / or camera images. The sensor unit may further include underwater sensors such as sonar sensors.

[0049] Both the ship control device 26 and the user interface device 24 can be used in the control of the ship 22. The ship 22 is further equipped with a group of propulsion systems, such as a group of thrusters.

[0050] In the marine industry, numerous developments are underway globally, primarily focusing on the automation and autonomy of ships.

[0051] Systems and solutions for collision avoidance in automated operations and automated docking have been proposed. Most of the solutions presented are either fully automated or fully manual until a human takes over.

[0052] However, most vessels are expected to have a low level of operational automation in the near future. There are situational awareness systems and electronic chart display systems (ECDIS) that operators can use to help them understand how to approach the quay and what the overall situation is. However, there is no system that can automatically use situational information to complement manual commands in a way that understands the situation and attempts to execute complex control commands, while simultaneously reducing the burden on the operator.

[0053] In some use cases, keeping a human in command of the vessel may be a much smaller step, but it is possible to benefit from autonomous technology by complementing human manual control with automation that helps ensure that the speed during maneuvering or in the open sea is safe, taking into account the distance to obstacles. This allows the crew to concentrate on their steering intent while the system makes small adjustments. There are many examples where this can be useful. For example, when docking a vessel and approaching a quay sideways, the crew must monitor the distance to the quay and alignment while manually controlling the vessel with levers or joysticks. Whether the operator can maintain a safe speed and rate of turn (ROT) during the approach to the quay depends entirely on the crew's experience and concentration. Furthermore, if something unexpected appears in front of the quay, the operator must manually issue complex control commands to stop or adjust the vessel's movement to avoid a collision with the unexpected object or quay structure. Moreover, in adverse environmental conditions such as strong winds or currents, it may be difficult for the operator to manage the vessel's speed and ensure that the approach is controlled.

[0054] When docking larger vessels such as cruise ships, operators typically control the vessel from the bridge wing on the docking side. The ship's bridge is typically located near the bow. The bridge wing is more than 30 meters above the waterline, and the ship's stern can be up to 300 meters from the bridge wing. Especially in adverse weather conditions, it is extremely difficult for a human to assess whether the ship is perfectly aligned with the quay and whether the bow and stern are approaching the quay at a constant speed. If a ship is misaligned or approaches the quay at too high a speed, it can cause serious damage to the ship's structure and the quay, and may also endanger cargo, crew, and passengers.

[0055] Furthermore, many shipowners and operators limit the use of dynamic positioning (DP) systems when approaching and maneuvering in port. They prefer to maintain a system that keeps the crew under constant control. Therefore, a low level of manual joystick system, where automation helps adjust commands to maintain safe tolerances (in terms of distance and time) relative to the surroundings, may be far more acceptable to crews than fully automated steering and docking systems, especially on larger vessels such as cruise ships.

[0056] A part of this disclosure relates to providing such automation support to manual control.

[0057] Next, the overall operation to achieve this will be described with reference to Figures 4 and 5. Figure 4 shows a block schematic of an implementation of a ship control system VCS36 that includes a command adjustment device 10 and uses a user interface device 24 and a sensor unit 28, and Figure 5 shows a flowchart of a first embodiment that compensates for inaccurate user input in ship operation, with the method steps performed by the command adjustment device 10.

[0058] As mentioned above, the command adjustment device comprises a command adjustment module CAM32 and a perception module PM30. There is also an input conversion module ICM34, which may or may not be part of the command adjustment device 10. In the example in Figure 4, it is located outside the command adjustment device 10. The separation of these modules is conceptual and may differ in actual implementations.

[0059] As can be seen in Figure 4, the command adjustment module CAM32, the perception module PM30, the input conversion module ICM34, the ship V22, the ship control device VCD26, the sensor unit SU28, and the user interface device UID24 can be considered to be part of the ship control system VCS36.

[0060] The ship control device 26 controls the movement, motion, or force of the ship, depending on a particular embodiment. The output of this device is a command that can ultimately control the propulsion system of the ship 22.

[0061] The user interface device 24 receives input commands, which are then forwarded to the input conversion module ICM 34, which processes the user commands. The user interface device 24 may also have the ability to provide haptic feedback to the operator.

[0062] The perception module 30 provides information (such as distance and speed) about the vessel's surroundings based on one or more sources, which may be electronic charts or perception sensors, but not limited to cameras, radar, or lidar. Alternatively, it may be virtual objects or other information entered by the operator to avoid specific areas.

[0063] The command adjustment module 32 utilizes input from the perception module 30 to provide adjustments to operator commands based on measured distance, speed, and user-defined parameters.

[0064] The sensor unit 28 performs and connects to necessary measurements, including but not limited to those of a Global Navigation Satellite System (GNSS), compass, radar, camera, lidar, local position reference system, motion sensor, and nautical chart, and feeds the measurement information to other parts of the ship's control system, such as the ship's control device 26, the perception module 30, and the command coordination module 32.

[0065] During operation, the user interface device 24 receives user input from the ship operator in the form of user operation of the device, such as pressing a button, moving the steering wheel, one or more control levers, a joystick, or a trackball. This user input is converted by the input conversion module 34 into at least one ship control command VCC, which is then supplied to the ship control device VCD 26 for controlling the ship. Thus, at least one control command VCC is issued by the ship operator. Then, based on one or more ship control commands VCC, the ship control device 26 determines one or more propulsion command PMCs for controlling one or more propulsion systems of the ship 22. The propulsion command PMCs are applied to the ship's propulsion systems, which results in the movement of the ship 22 in the water, and this movement can be considered as movement in a plane, corresponding to sway and surge. Next, the sensor unit 28 acquires various sensor measurements and provides sensor measurements that may be required for the control of the ship control device 26, such as wind data, speed, position, and heading. Furthermore, sensor data of the environment, such as radar echoes, lidar images, and / or camera images, may be acquired and provided to the perception module 30 of the command adjustment device 10. The sensor unit 28 may also directly provide other sensor data, such as wind measurements, to the command adjustment module 32.

[0066] Therefore, the sensor unit 28 acquires sensor measurements of the surroundings of the vessel, for example in the form of images, and provides these sensor measurements of the environment to the perception module 30, while other sensor measurements may be provided to the command adjustment module 32 and / or the vessel control device 26.

[0067] The perception module 30 analyzes the received sensor data to detect the environment of the vessel 22, i.e., the area surrounding the vessel 22 (S100). This detection may further include detecting whether there are any objects above, under, and / or below the water surface surrounding the vessel 22. Objects above the water surface may be cranes, bridges, platforms, terminal tunnels, or gangways. Objects underwater may be docks, islands, reefs, buoys, rafts, and other vessels. Objects below the water surface may be reefs, underwater rocks, and, if the seabed is shallow, parts of the seabed. In addition to or instead of receiving and analyzing sensor data, the perception module may receive and analyze digital chart / map data along with data about the vessel's position and, optionally, orientation, in order to detect the environment of the vessel. Objects can then be located through the information from the map / chart. If such an object is detected in the environment, the perception module 30 determines at least one distance from the vessel 22 to the object, where each distance is associated with a corresponding point on the (outer surface) of the vessel 22. Such distances can be obtained directly by analyzing sensor data, such as by analyzing LiDAR images. When a map is used, the distance from an object in the map is determined based on the position of the vessel itself and possibly on its orientation. Thus, at least one distance is at least one distance from the vessel 22 to an object in the vessel's environment. As can be seen from the above, at least one distance is also at least one distance obtained when the vessel is operated using at least one vessel control command VCC issued by the operator. Determining at least one distance may further include determining a first distance to an object at a first point on the vessel and determining a second distance to an object at a second point on the vessel. If velocity is determined, determining at least one velocity may in this case include determining the velocity toward the object at each of the points.

[0068] If the object is on the starboard side of the vessel, at least one distance may comprise a first distance and a second distance at two corresponding points on the starboard side; if the object is on the port side, at least one distance may comprise two distances at corresponding points on the port side; if the object is forward of the vessel, at least one or two distances from one or two positions on the bow may, in part, be distances from a point on the starboard side and / or from a point on the port side; and if the object is aft of the vessel, at least one or two distances may be determined from one or two positions on the stern, in part, be distances from a point on the starboard side and / or from a point on the port side. Furthermore, the point of interest may change during the maneuver of the vessel.

[0069] Optionally, the perception module 30 can also determine at least one velocity of the vessel 22 toward an object when the vessel is operated using at least one control command VCC, each of which such velocity is determined at a point where the distance is also determined. Thus, at each point where the distance is determined, a velocity can also be determined. The velocity can be determined through sensor data. Alternatively, it may be determined as the time derivative of the distance.

[0070] As can be seen from the above, at least one speed is obtained when the vessel 22 is operated using at least one control command issued by the operator.

[0071] Another alternative is that the distance to at least one point is determined based on the current position and orientation of a fixed point on the vessel relative to a fixed location in the world coordinate system. In this case, the fixed point(s) on the vessel may be compared to a fixed point(s) on an object / in the world, and the distance and velocity to the point are used not only to control the velocity in its degrees of freedom (e.g., laterally), but also to determine the angle and change in angle (angular velocity) to determine compensation in other degrees of freedom (e.g., forward). This is of interest when docking at a fixed location along a quay.

[0072] Next, the command adjustment module 32 analyzes at least one distance and at least one velocity at an optional distance (S120), and determines at least one compensation for at least one ship control command VCC based on the analysis of at least one distance and at least one velocity at an optional distance (S130), the at least one compensation being compensation to avoid collision with an object.

[0073] If a collision risk is deemed to exist, at least one control command compensation may include a command compensation to reduce the velocity in the direction toward the object.

[0074] Speed ​​reduction can also be a reduction in speed that decreases as the distance to an object decreases, such as when a ship gets closer to an object and its speed decreases. Therefore, the analysis may involve analyzing the relationship between at least one distance, at least one speed, and time.

[0075] The decrease in velocity as distance decreases can be further calculated while taking into account the minimum constant time until encounter with an object.

[0076] A control command that causes velocity or force in a direction contributing to the movement of a vessel toward an object can be compensated by a control command compensation that causes velocity or force in the opposite direction.

[0077] Opposite-direction control command compensation can also be applied so that the compensation gradually increases. For example, if the bow of a vessel approaches the quay too quickly with the current command, the bow force / velocity command may begin to decrease by adding a command with the opposite sign to the operator command. In this case, this added command can increase as the bow approaches the quay more quickly and in closer proximity. Thus, although an additional command is introduced (either by force feedback or by directly adding a command), the overall command does not necessarily change sign immediately if the operator command was initially non-zero.

[0078] When a ship's speed exceeds a ship speed threshold, there may be a risk of collision, and this ship speed threshold may be adaptive.

[0079] The adaptation may be based on user settings or by utilizing the ship's current capabilities to slow down the vessel's movement. These capabilities may depend on which propulsion devices are active and / or how many generators are currently running. Alternatively, the ship speed threshold may be adapted to match the passenger comfort level of the vessel.

[0080] Additionally, or instead, the command compensation may include a ship orientation compensation VOC that compensates for the ship's orientation relative to an object. Thus, it is compensation against the ship's rotation or yaw. In this case, the command adjustment module 32 can estimate the desired ship orientation based on the ship's shape, the actual ship orientation, and the object's shape.

[0081] Furthermore, it is possible to consider the object as having the same virtual potential as the ship and as a repulsive element that causes a force or velocity change dependent on the distance / radius from the object. This type of virtual potential method can also be implemented so that a virtual potential field is determined between the object and the ship, and the changes in the ship's velocity and orientation are determined based on this field. In this case, the object and the ship can be seen as magnets with the same poles.

[0082] If there are two velocities directed toward an object, determining at least one ship orientation compensation may further include determining a ship orientation compensation that counteracts the portion where the greater velocity exceeds the other velocity.

[0083] Next, the command adjustment module 32 provides the input conversion module 34 with compensation for use in the operation of the vessel 22 performed by the operator (S140). The input conversion module 34 may use the compensation to provide feedback to the operator, such as tactile feedback HF, via the user interface device 24. The input conversion module may, as an addition or alternative, provide compensation to the vessel control device 26 so that the compensation is applied to the control of the vessel 22. Thus, providing at least one control command compensation for use in the operation of the vessel may include providing feedback of at least one control command compensation to the operator via the user interface device and / or by adding at least one control command compensation to a control command for the vessel control device which is determined based on user input made by the operator. Alternatively, the feedback may be provided by visual means such as a light indicator or by displaying information on a screen.

[0084] By using haptic feedback HF, inaccurate operator inputs that could lead to collisions with objects can be subjected to some degree of resistance, thus making them more difficult to perform. Meanwhile, compensation supplied to the ship control device 26 can counteract portions of ship control commands that could lead to collisions with objects. This enhances safety and counteracts inaccuracies in operator input.

[0085] Next, a second embodiment will be described with reference to Figures 6 and 7. Figure 6 shows a vessel with four propulsion systems being moved toward an object OBJ38 to be avoided, and Figure 7 shows a flowchart of some method steps in the second embodiment of a method for compensating for inaccurate user input in maneuvering the vessel.

[0086] In this example, the input conversion module is part of the command adjustment device 10.

[0087] In the example shown in Figure 6, the first propulsion engine PM1 40 and the second propulsion engine PM2 42 are located on the starboard side of the ship 22, with the first propulsion engine 40 closer to the bow and the second propulsion engine 42 closer to the stern. Similarly, the third propulsion engine PM3 44 and the fourth propulsion engine PM4 46 are located on the port side of the ship 22, with the third propulsion engine 44 closer to the bow and the fourth propulsion engine 46 closer to the stern. The third propulsion engine 44 is located on the opposite side of the ship 22 from the first propulsion engine 40 with respect to the longitudinal axis A, and the fourth propulsion engine 46 is located on the opposite side of the ship 22 from the longitudinal axis A. There is also a line of symmetry SL that passes through the ship's center of gravity CoG perpendicular to the longitudinal axis A. The first propulsion engine 40 and the second propulsion engine 42 are located on either side of the line of symmetry SL, at the same distance from the line of symmetry SL. The third thruster 44 and the fourth thruster 46 are also installed on both sides of the symmetry line SL, at the same distance from the symmetry line SL.

[0088] In this example, the vessel is to dock at an object OBJ38 in the form of a quay, such that its starboard side is moved to the quay and positioned along the quay. Therefore, it is to be positioned adjacent to and parallel to the quay. To do this, there is at least one vessel control command VCC provided by the operator, in which a first propulsion unit 40 applies a first force F1 perpendicular to the starboard side and longitudinal axis A, and a second propulsion unit 42 applies a second force F2 perpendicular to the starboard side and longitudinal axis A. Due to inaccurate operator input, these forces F1 and F2 may be too strong, potentially causing the vessel to crash into the quay. Furthermore, the second force F2 may be greater than the first force F1, or vice versa, causing the vessel to rotate around its center of gravity CoG, thereby preventing the starboard side from properly aligning with the quay, which could also lead to a crash.

[0089] Next, we will explain how to avoid such crashes, referring to Figure 7.

[0090] The method may begin with the perception module 30 receiving sensor data from the sensor unit 28, analyzing the sensor data, and then determining the environment based on the analysis (S200). Thus, the environment of the ship is detected using the sensor data, in this case, this detection also includes detecting an object 38 in the environment. The input conversion module 34 may also receive user input via the user interface device 24 and convert it into at least one control command VCC (S210), which is supplied to the ship control device 26 for controlling the ship 22. In this example, there is a ship control command VCC that causes the ship control device 26 to control the first thruster 40 to generate a first force F1 and the second thruster 42 to generate a second force F2, thereby moving the ship toward the object 38.

[0091] Next, the perception module 30 determines at least one distance to an object in the environment of the vessel 22 when the vessel is operated using at least one control command VCC issued by the operator, each distance relating to a corresponding point on the vessel 22. In this case, it also determines at least one velocity of the vessel 22 toward the object 38 when the vessel 22 is operated using at least one control command. More specifically, the perception module 30 determines a first distance d1 to the object 38 at a first point P1 on the vessel 22, a second distance d2 to the object 38 at a second point P2 on the vessel 22, and a first velocity and a second velocity at the first point P1 and the second point P2 (S220).

[0092] In this example, the first side is the starboard side of the vessel 22, and the perception module determines a first distance d1 to object 38 at a first point P1 on the starboard side, which is aligned with the first propulsion engine 40, and a second distance d2 to object at a second point P2 on the starboard side, which is aligned with the second propulsion engine 42. The perception module 30 also determines a first velocity and a second velocity at the first point P1 and the second point P2, which may be determined as derivatives of the first distance d1 and the second distance d2. The first point P1 and the second point P2 may be located on either side of the line of symmetry SL, at the same or equal distance to the line of symmetry SL. It should be understood that these points do not need to be aligned with the propulsion engines, nor do they need to be symmetrical with respect to the line of symmetry SL. In practice, any point on the starboard side may be used. If there are objects in other directions, such as in front of the bow or behind the stern, the distances in those directions can also be determined. One such possible distance db in the direction away from the bow and one such possible distance ds in the direction away from the stern are schematically shown in Figure 6. Naturally, the distance from the port side can also be determined if there are objects there.

[0093] Therefore, it may be possible to measure at least one distance and at least one speed at the bow, at least one distance and at least one speed at the bow, at least two distances and at least two speeds at the port side, and at least two distances and at least two speeds at the starboard side.

[0094] Next, the command adjustment module 32 analyzes at least one distance and at least one velocity with respect to collision avoidance (S230). Then, based on the analysis of at least one distance and at least one velocity, it determines at least one compensation for at least one control command.

[0095] The velocity toward an object may be determined as the average of a first velocity and a second velocity. At least one control command compensation may include command compensation to reduce the ship's velocity toward an object when a risk of collision is deemed to exist. A risk of collision may be deemed to exist when the ship's velocity exceeds a ship's velocity threshold.

[0096] Therefore, the speed toward object 38 may be compared to a speed threshold, and if it exceeds the threshold, it may be determined that there is a risk of collision with object 38. The threshold may be more adaptive. The threshold may decrease as the vessel 22 approaches object 38, and the distance to the object may be determined as the average of a first distance and a second distance. If it is considered that moving at the speed used would pose a risk of collision with the object (S240), control command compensation to reduce the ship speed is determined. Here, a control command that causes a speed or force in the direction contributing to the ship's movement toward the object is compensated by a control command compensation that causes a speed or force in the opposite direction.

[0097] If speed is not considered a risk (S240), the command adjustment module 32 may continue to investigate the orientation of the vessel.

[0098] In this example, the forces F from both the third thruster 44 and the fourth thruster 46 are opposite to the first force F1 and the second force F2. SC A velocity compensation SC is determined that causes this. Thus, the third propulsion unit 44 and the fourth propulsion unit 46 actuate the ship 22 with the same force in the opposite direction to the first force F1 and the second force F2, and as a result, a velocity compensation SC is obtained that cancels out the ship's velocity toward the object. Alternatively, the four propulsion units may act in different ways to achieve the same result, for example, by generating a force in the same direction.

[0099] Furthermore, a ship orientation compensation may be investigated, and this ship orientation compensation investigation may be performed regardless of the results of the ship speed investigation. The ship orientation compensation investigation may include the command adjustment module 32 investigating whether the ship would collide with an object if it were in its actual orientation. This may include estimating a desired ship orientation based on the ship's shape, the actual ship orientation, and the shape of the object 38. In the example of a quay, this may include determining that the starboard side should be positioned parallel to the quay. Determining at least one ship orientation compensation may further include determining a ship orientation compensation to counteract the portion where the greater speed exceeds the other speed.

[0100] This decision can be simplified. When the first and second points are points on the first side of the vessel, at different distances to the object, determining at least one vessel orientation compensation may mean determining a vessel orientation compensation that reduces the difference in the distances to the object between the two points. In the example in Figure 6, the first side is the starboard side, with points P1 and P2 and first distances d1 and second distances d2. Therefore, whether vessel orientation compensation is necessary can be determined by comparing the first distance d1 and the second distance d2 with each other and determining that the vessel is considered to collide with object 38 when the first distance d1 and the second distance d2 are different (S260).

[0101] If these differ from each other, there is a rotation around the center of gravity that must be counteracted. The rotational speed can be determined based on two distances and / or based on a first speed and a second speed. Thus, there is a determination of command adjustments to counteract the difference in distance and speed (S270). Determining at least one ship orientation compensation may be determining a ship orientation compensation that reduces the difference in distance to the object at two points.

[0102] In this example, the second distance d2 is greater than the first distance d1, and therefore a command adjustment VOC is determined to counteract the rotation. When this command adjustment is applied by the ship control device 26, a force F is applied to the fourth propulsion unit 46 that is opposite to the second force F2 and has a quantity or number corresponding to the difference between the first force F1 and the second force F2. VOC This is applied to the fourth point P4. Thus, the control command is provided to a point on the second side (in this case, the port side) opposite to the point on the first side that is closer to the object than the other point on the first side.

[0103] If compensation is determined, these are supplied to the input conversion module 34 (S280), which then provides compensation to the operator through haptic feedback by providing compensation to the ship control device 26 via a joystick, trackball, one or more levers, etc. In this way, the command adjustment module 32 provides at least one control command compensation for use in ship operations being performed by the operator. Therefore, providing at least one control command compensation for use in ship operations includes feeding back at least one control command compensation to the operator via a user interface device and / or by adding at least one control command compensation to a control command for a ship control device which is determined based on user input made by the operator.

[0104] Therefore, solutions are provided to help manually control the vessel using levers, joysticks, or other user interface devices to automatically ensure that the vessel does not collide with any nearby structures or other vessels. Inaccurate user inputs made during vessel operation are thus compensated for.

[0105] Command coordination devices utilize information from onboard sensors, such as those implementing crew, perception systems, or chart systems, to determine the distance to surrounding obstacles and, based on that, complement / adjust / limit human control commands to ensure that the vessel approaches nearby objects in a controlled, smooth, and safe manner without colliding with surrounding obstacles.

[0106] The described instructions utilize measured, user-provided, or digital ambient information to calculate safe speeds in all directions of the vessel and, if necessary, smoothly adjust operator commands to enable a safe approach to obstacles.

[0107] The user interface device, along with the input conversion module, can operate in force joystick mode, velocity joystick mode, or manual control mode. They may also provide haptic feedback.

[0108] 1. In Force Joystick mode, surge and sway forces and yaw torque are automatically adjusted based on calculations from the command adjustment module to allow for deceleration of surge, sway, or yaw speeds when the vessel is approaching any object too quickly.

[0109] 2. In speed joystick mode, the surge, sway, and yaw velocities are automatically adjusted based on calculations from the command adjustment module to allow for a reduction in surge, sway, or yaw velocity if the vessel is approaching any object too quickly.

[0110] 3. In manual control mode (e.g., using the Azipod lever), adjustments to the allowable propeller pitch, speed, and angle are made based on calculations from the command adjustment module to allow for speed reduction related to surge, sway, or yaw if the vessel is approaching any object too quickly.

[0111] 4. When a vessel is approaching any object too quickly, haptic feedback may be provided via the user interface device to allow the user to manually adjust surge, sway, and yaw commands based on calculations from the command adjustment module, thereby reducing the velocity in relation to surge, sway, or yaw. In this mode, the operator may decide to maintain the current commands and ignore "advice" from the vessel control system. The user may also choose to turn off haptic feedback and rely on other information.

[0112] The operation sequence of the command adjustment device can also be described as follows:

[0113] 1. Obtain user input commands for the vessel from the user interface device.

[0114] 2. Determine whether adjustment is necessary and calculate the required adjustments to the input based on the output from the perception module.

[0115] 3. Adjust ship control commands automatically or by providing tactile or visual feedback to the operator. Depending on the embodiment, the adjustment may be performed by either an input conversion module or a ship control device.

[0116] 4. The ship's control device executes control commands (which may involve adjustments) that either directly control the propulsion system or provide control commands through another system (e.g., a thruster control system, a remote control system, etc.) that ultimately influences the ship's movement in a desired manner, thereby preventing collisions and enabling smooth approach to obstacles such as quays.

[0117] While some vessel categories (such as short-haul transport) are expected to move towards higher levels of automation in steering and navigation in the near future, other categories, such as cruise ships and offshore vessels, are most likely to take smaller steps toward automation, for example, by adopting advanced manual control solutions. Furthermore, this feature eliminates the gap between situational awareness, advice, and control without taking command from the crew. This only automatically supports the operator's safe operation while allowing the operator to maintain complete control. It should also be remembered that even in vessels with automatic / autonomous capabilities, the operator must sometimes take over commands. In situations where the level of automation is reduced to a lower level, it may still be beneficial to maintain a certain level of automation (by having the proposed feature as a fallback mode) rather than immediately moving towards full manual control.

[0118] The maneuvers described herein are primarily described in relation to docking a vessel at a quay. Low speeds are used in such docking. However, it should be understood that the invention is not limited to docking or low speed.

[0119] For example, the maneuvers described herein may be just right for use at passing speeds when using manual levers or rudder / mini-wheel and rpm control, in order to avoid collisions at high speeds when manually operated, by analyzing the distance / speed approaching other objects and then adjusting the speed and rotation rate (ROT) to reduce the risk of collision.

[0120] The present invention can be modified in many ways. Therefore, although the present invention is described in relation to what is currently considered the most practical and preferred embodiment, it should be understood that the present invention is not limited to the disclosed embodiment, but rather intended to cover a variety of modifications and equivalent configurations. Accordingly, the present invention is limited only by the following claims.

Claims

1. A method for compensating for inaccurate user input in the operation of a vessel (22), wherein the method is performed by a command adjustment device (10), The environment of the aforementioned vessel (22) is detected (S100; S200), Determining at least one distance (d1, d2) from the vessel to an object (38) in the environment of the vessel (22) when the vessel is operated using at least one control command (VCC) issued by the operator of the vessel (S110, S220), where each distance is related to a corresponding point (P1, P2) on the vessel (22), Analyzing at least one of the aforementioned distances (S120, S230), Based on the analysis of the at least one distance, determine at least one control command compensation for the at least one control command (S130; S250, S270), To provide the at least one control command compensation for use in the operation of the vessel (22) performed by the operator (S140; S280), Methods that include...

2. The method according to claim 1, further comprising determining at least one speed of the vessel toward the object (38) when the vessel is operated using the at least one control command (VCC) (S220), and analyzing the at least one speed (S230), wherein determining the at least one control command compensation (S250, S270) is also based on the analysis of the at least one speed.

3. The method according to claim 1 or 2, wherein the at least one control command compensation includes a command compensation for reducing the speed of the vessel (22) in the direction toward the object (S250) when there is a risk of collision (S240).

4. The method according to any one of claims 1 to 3, wherein determining at least one distance includes determining a first distance (d1, d2) from a first point (P1) on the vessel (22) to the object (38) (S220) and determining a second distance (d2) from a second point (P2) on the vessel to the object (38).

5. The method of claim 4, dependent on claim 2, wherein determining at least one velocity (S220) includes determining the velocity toward the object (38) at each of the points (P1, P2).

6. The method according to claim 4 or 5, wherein the at least one control command compensation includes at least one vessel orientation compensation (VOC).

7. The method of claim 6, dependent on claim 5, wherein determining at least one vessel orientation compensation (VOC) includes determining a vessel orientation compensation (VOC) that counteracts the portion in which the greater speed exceeds the other speed.

8. The method according to claim 6 or 7, wherein the first point (P1) and the second point (P2) are points on the first side of the vessel at different distances (d1, d2) from the object (38), and one of the points (P2) is assumed to have a faster velocity than the other (P1) if it is closer to the object (38), and determining at least one vessel orientation compensation (VOC) means determining a vessel orientation compensation that reduces the difference in the distances (d1, d2) from the two points (P1, P2) to the object (38).

9. The method according to any one of claims 1 to 8, wherein providing the at least one control command compensation for use in the operation of the vessel (S140; S280) comprises adding the at least one control command compensation to a control command (VCC) for a vessel control device (26), the control command (VCC) being determined based on user input made by the operator.

10. The method according to any one of claims 1 to 9, wherein the at least one control command compensation is fed back to the operator via a user interface device (24).

11. The method according to any one of claims 1 to 10, further comprising receiving user input, converting the user input into a control command (VCC), and supplying the control command to a ship control device (26) for controlling the ship.

12. A command adjustment device (10) for compensating for inaccurate user input in the operation of a vessel (22), wherein the command adjustment device (10) comprises a processor (12), and the processor is To detect the environment of the aforementioned vessel (22), Determining at least one distance (d1, d2) from the vessel to an object (38) in the environment of the vessel (22) when the vessel is operated using at least one control command (VCC) issued by the operator of the vessel, where each distance is related to a corresponding point (P1, P2) on the vessel (22), Analyzing at least one of the aforementioned distances, Based on the analysis of the at least one distance, determine at least one control command compensation for the at least one control command, To provide the at least one control command compensation for use in the operation of the vessel performed by the operator, A command adjustment device (10) that operates to perform the following actions.

13. A ship control system (36) comprising a user interface device (24), a sensor unit (28), a ship control device (26), and the command adjustment device described in claim 12.

14. A ship (22) equipped with the ship control system according to claim 13.

15. A computer program for compensating for inaccurate user input in the operation of a vessel (22), wherein the computer program comprises computer program code (16), and when the computer program code (16) is executed by the processor (12) of the command adjustment device (10), the processor (12) receives To detect the environment of the aforementioned vessel (22), Determine at least one distance (d1, d2) from the vessel to an object (38) in the vessel's environment when the vessel is operated using at least one control command (VCC) issued by the operator of the vessel (22), where each distance is related to a corresponding point (P1, P2) on the vessel (22). Analyzing at least one of the aforementioned distances, Based on the analysis of the at least one distance, determine at least one control command compensation for the at least one control command, To provide the at least one control command compensation for use in the operation of the vessel performed by the operator, A computer program that performs an action.

16. A computer program product for compensating for inaccurate user input in the operation of a vessel (22), comprising a data carrier (20) having the computer program code (16) described in claim 15.

Images