Application control for force feed back to vessel maneuvering stick

Abstract

To provide a navigation maneuvering system for a vessel.SOLUTION: A vessel maneuvering system includes a vessel control unit (30). The vessel control unit (30) is composed to execute: acquiring required operation for a maneuvering stick (24) indicating speed value and direction value for next navigation of a vessel; acquiring a proximity signal from a proximity sensor (10) in response to having an object (104) sensed in at least a direction indicated by the direction value of the maneuvering stick (24) in at least a part of surroundings (102) surrounding the vessel; specifying the distance between the vessel and the object (104) based on the vicinity signal; and controlling a force feedback device (22) so that force feedback is applied to the maneuvering stick (24) based on the specified distance.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure generally relates to navigation control in a ship. In a specific aspect, the present disclosure relates to the application control of force feedback to a ship's joystick. The present disclosure can be applied to ships such as leisure boats, ships, cruise ships, fishing boats, yachts, ferries, etc. among other types of vehicles. Although the present disclosure can be described with respect to a specific ship, the present disclosure is not limited to any specific ship.

Background Art

[0002] In a marine electronic vessel control (EVC) system for navigation, existing input devices are usually mechanical devices that do not have the ability to provide force feedback to the driver during use. Although some marine EVCs employ the application of force feedback to the input devices of a ship, the inventor has identified further areas for improvement with respect to force feedback in ship applications.

Summary of the Invention

[0003] According to a first aspect of the present disclosure, a marine navigation control system is provided. The navigation control system includes a proximity sensor configured to sense at least a part of the surroundings surrounding the ship, an input device adapted to provide a navigation command to the ship, the input device including a movable joystick, and a force feedback device adapted to apply force feedback to the joystick, a ship control unit operably connected to the proximity sensor and the input device, the ship control unit being configured to obtain a required operation of the joystick indicating a speed value and a direction value of the next navigation of the ship, obtain one or more proximity signals from the proximity sensor in response to sensing at least one object in the surroundings in at least the direction indicated by the direction value, identify a distance between the ship and at least one object based on the one or more proximity signals, and control the force feedback device to apply force feedback to the joystick based on the identified distance. The first aspect of the present disclosure may attempt to provide situation awareness to the operator of the ship. As a technical advantage, in relation to detecting at least one of the surrounding objects, a direct force feedback may be provided to the operator of the ship on the joystick being operated. With the provided feedback, the operator can intuitively understand the risks associated with the detected object, and further, the effectiveness of the required navigation command can be automatically adapted. This distance-based force feedback application can improve safety measures for both the ship and its operator, as well as the surrounding objects. Further, the operator of the ship can better understand the maneuverability of the ship.

[0004] In some embodiments, optionally including at least one preferred embodiment, the ship control unit is further configured to control the force feedback device by calculating the maximum allowable speed of the ship based on a specified distance, converting the maximum allowable speed into a maximum allowable joystick movement, and restricting the required movement of the joystick based on the maximum allowable joystick movement. As a technical advantage, the joystick input of the required movement can be restricted according to an allowable value, whereby the operator can intuitively understand the distance to nearby objects. This can result in improved safety and a better understanding of the maneuverability of the ship.

[0005] In some embodiments, optionally including at least one preferred embodiment, the ship control unit is further configured to control the force feedback device to apply a force feedback value that counteracts the force of the required movement of the joystick, thereby restricting the required movement of the joystick. As a technical advantage, it can be mentioned that counteracting the force of the required movement can reduce the risk of requesting unauthorized navigation commands, thereby improving the steering safety of the ship.

[0006] In some embodiments, optionally including at least one preferred embodiment, the ship control unit is further configured to control the force feedback device to gradually increase the applied force feedback when the distance between the ship and the object is decreasing. As a technical advantage, it can be mentioned that a more intuitive application of force feedback can be achieved, which may enable a warning to be given to the operator gradually stronger (or gradually weaker) according to the currently perceived distance to the object.

[0007] In some embodiments, including at least one preferred embodiment optionally, the ship control unit is further configured to control a force feedback device to gradually increase the applied force feedback when the speed value is increasing. As a technical advantage, it may be possible to provide a more intuitive application of force feedback that can gradually warn the operator more strongly (or more weakly) depending on the required speed value.

[0008] In some embodiments, including at least one preferred embodiment optionally, the applied force feedback can be disabled by an external force applied to the joystick that exceeds the value of the force feedback. As a technical advantage, it may be possible for the operator to regain control of the joystick, which can be useful for improving maneuverability in situations where the sensor measurements are incorrect or in advanced maneuvering situations where the operator cannot rely on the sensor measurements.

[0009] In some embodiments, including at least one preferred embodiment optionally, when a proximity signal indicates that there are a plurality of objects in the surroundings at least in the direction indicated by the direction value, the ship control unit is configured to determine the distance between the ship and each of the plurality of objects and to control a force feedback device to apply force feedback to the joystick based on each of the determined distances. As a technical advantage, it may be possible to take into account a plurality of objects in a composite control procedure, thereby improving feedback, intuitive understanding, and maneuverability in situations where there are a large number of objects such as in a mooring area.

[0010] In some embodiments, optionally including at least one preferred embodiment, the joystick can be moved in three degrees of freedom, the force feedback device includes respective force feedback units for each degree of freedom of the joystick, and the ship control unit is configured to control the application of force feedback to the joystick via one or more of the force feedback units. As a technical advantage, not only is the object sensed in a form associated with the force feedback by the respective force feedback units for different parts of the joystick, but also the location where the object is sensed is more often notified to the operator. Thereby, safety measures and / or the understanding of the maneuverability of the ship can be further improved.

[0011] In some embodiments, optionally including at least one preferred embodiment, the ship control unit is further configured to control the force feedback device based on the navigable water area conditions in which the ship is traveling, and the navigable water area conditions include one or more of wind speed, wave height, current strength, and weather conditions. As a technical advantage, it can be mentioned that the situation awareness of the conditions in which the ship is traveling is improved by more adaptable and versatile force feedback application control.

[0012] In some embodiments, optionally including at least one preferred embodiment, the ship control unit is further configured to process one or more proximity signals to identify at least one classification of at least one object and to control the force feedback device to apply force feedback to the joystick based on the identified at least one classification. As a technical advantage, it can be mentioned that the recognition of the type of object is improved by more adaptable and versatile force feedback application control.

[0013] In some embodiments, including at least one preferred embodiment arbitrarily, the ship control unit is further configured to control a force feedback device to apply a fixed or variable force feedback value. As a technical advantage, a specific type of force feedback value can be applied according to various situations, whereby the safety measures and / or the understanding of the maneuverability of the ship can be further improved.

[0014] According to a second aspect of the present disclosure, a ship is provided. The ship includes a navigation control system according to the first aspect. The second aspect of the present disclosure may attempt to provide situation awareness to the ship's operator. As a technical advantage, in relation to detecting at least one of the surrounding objects, a direct force feedback may be provided to the joystick that the ship's operator is operating. With the provided feedback, the operator can intuitively understand the risks associated with the detected object, and further, the effectiveness of the required navigation command can be automatically adapted. This distance-based force feedback application control can improve the safety measures for both the ship and its operator, as well as the surrounding objects. Further, the ship's operator can better understand the maneuverability of the ship.

[0015] According to a third aspect of the present disclosure, a computer-implemented method for a ship is provided. The computer-implemented method includes obtaining a required operation of a joystick indicating a speed value and a direction value for the next voyage of the ship, obtaining one or more proximity signals from a proximity sensor in response to detecting at least one object in the surrounding area in at least the direction indicated by the direction value, identifying a distance between the ship and the at least one object based on the one or more proximity signals, and controlling a force feedback device to apply force feedback to the joystick based on the identified distance. The third aspect of the present disclosure may attempt to provide situation awareness to the ship's operator. As a technical advantage, in relation to detecting at least one of the surrounding objects, a direct force feedback may be provided to the ship's operator on the operating joystick. The provided feedback enables the operator to intuitively understand the risks associated with the detected object, and further, the effectiveness of the required navigation command can be automatically adapted. This distance-based force feedback application control can improve safety measures for both the ship and its operator, as well as the surrounding objects. Further, the ship's operator can better understand the maneuverability of the ship.

[0016] According to a fourth aspect of the present disclosure, a computer program product is provided. The computer program product includes program code that, when executed by a processing circuit, performs the method of the third aspect. The fourth aspect of the present disclosure may attempt to provide situation awareness to the ship's operator. As a technical advantage, in relation to detecting at least one of the surrounding objects, a direct force feedback may be provided to the ship's operator on the operating joystick. The provided feedback enables the operator to intuitively understand the risks associated with the detected object, and further, the effectiveness of the required navigation command can be automatically adapted. This distance-based force feedback application control can improve safety measures for both the ship and its operator, as well as the surrounding objects. Further, the ship's operator can better understand the maneuverability of the ship.

[0017] According to a fifth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium includes instructions that, when executed by a processing circuit, cause the processing circuit to perform the method of the third aspect. The fifth aspect of the present disclosure may attempt to provide situation awareness to a ship's operator. As a technical advantage, in relation to detecting at least one of the surrounding objects, a direct force feedback may be provided to the ship's operator on the joystick being operated. With the provided feedback, the operator can intuitively understand the risks associated with the detected object, and further, the effectiveness of the required navigation command can be automatically adapted. This distance-based force feedback application control can improve safety measures for both the ship and its operator, as well as the surrounding objects. Further, the ship's operator can better understand the maneuverability of the ship.

[0018] As will be apparent to those skilled in the art, the aspects, embodiments (including any preferred embodiments), and / or the appended claims of the present disclosure may be appropriately combined with each other. Additional features and advantages are disclosed in the following description, the claims, and the drawings, some of which will be readily apparent to those skilled in the art from these, or will be recognized by practicing the present disclosure described herein.

[0019] Also disclosed herein are computer systems, control units, code modules, computer-implemented methods, computer-readable media, and computer program products associated with the aforementioned technical advantages.

[0020] Embodiments are described in more detail below with reference to the accompanying drawings.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0022] The detailed description set forth below provides the information and examples of the disclosed technology with sufficient detail to enable those skilled in the art to practice the present disclosure.

[0023] The present disclosure relates to controlling the application of force feedback of a ship's joystick based on distance. In a situation where a ship is operating in a body of water, for example, the sea, typically a plurality of different situations can occur in relation to interactions with surrounding objects (also referred to herein as targets). The present disclosure presents a technique for providing direct force feedback to the joystick being operated to a ship's operator in relation to detecting at least one of the surrounding objects. With the given feedback, the operator can intuitively understand the risks associated with the detected object, and furthermore, the effectiveness of the required navigation commands can be automatically adapted. This distance-based force feedback application control can improve safety measures for both the ship and its operator, as well as for the surrounding objects. Furthermore, the ship's operator can better understand the maneuverability of the ship.

[0024] FIG. 1 is a schematic view of a ship 100 to which a part of the inventive concept of the present disclosure can be applied. In a non-limiting example, the ship 100 is a leisure boat, a ship, a cruise ship, a fishing boat, a yacht, or a ferry, etc. The ship 100 is adapted to operate in a body of water such as, for example, the sea, ocean, lake, river, bay, inlet, strait, canal, reservoir, fjord, marsh, swamp, etc. The ship 100 can be an electric ship, a gasoline-powered ship, or a diesel-powered ship, etc., provided that it can be steered using an input device having a joystick. The ship is provided with a navigation control system 5, which is a navigation EVC adapted to control the operation of the ship 100.

[0025] The navigation control system 5 includes one or more proximity sensors 10, six in this particular embodiment. The proximity sensors 10 can be distributed at any position of the ship 100. In this embodiment, the first sensor pair 10-1, 10-2 are arranged on the respective rear sides of the ship 100, the second sensor pair 10-3, 10-4 are arranged on the respective central sides of the ship 100, and the third sensor pair 10-5, 10-6 are arranged on the respective front sides of the ship 100. The proximity sensors 10 can be arranged at any appropriate height of the ship 10 as both surface sensors or underwater sensors. In other embodiments, the proximity sensors 10 can be arranged at any location of the ship 100 provided that the proximity sensors 10 can sense the surrounding portion of the ship 100. When the proximity sensors 10 are arranged underwater, the sensed portion refers to the underwater area, i.e., below the water surface. The proximity sensors 10 are configured to sense at least a part of the periphery 102 surrounding the ship 100. The periphery 102 is usually a water area (above and / or below the water surface), but when the ship 100 is located within the sensor proximity range from land, the periphery 102 can also be land. Thus, "at least a part" of the periphery 102 refers to the spatial position near the ship 100, and the accessibility to the vicinity depends on the type(s) of sensing technology(ies) employed. The periphery 102 includes various objects that can be sensed, which include, but are not limited to, other ships, living organisms (e.g., humans, wildlife), buoys, lighthouses, rocks, underwater objects, floating objects, land masses, piers, mooring stations, docking facilities, and many more objects that can be easily envisioned by those skilled in the art. The proximity sensors 10 can be lidar devices, radar devices, sonar devices, ultrasonic devices, cameras, inductive proximity sensors, capacitive proximity sensors, infrared proximity sensors, and / or other suitable devices configured to sense the surroundings. In response to sensing an object in the periphery 102, the sensors 10 are individually and / or collectively configured to transmit a proximity signal(s) to the ship control unit 30, which will be further described shortly.

[0026] The navigation control system 5 includes an input device 20. The input device 20 includes a force feedback device 22 and a joystick 24. The input device 20 should be understood as a device adapted to provide navigation commands, such as commands regarding speed or direction, to the ship control unit 30.

[0027] The joystick 24 may include a handle, a lever, or a certain type of controllable axle. The joystick 24 can be arranged to be operated by the operator of the ship 100, for example, by the operator's hand. The joystick 24 may be capable of moving in three degrees of freedom, namely, pitch, roll, and yaw. The pitch motion refers to the up and down movement or rotation of the joystick 24 about a horizontal axis, that is, an imaginary line extending from port (left) to starboard (right) across the width of the ship 100. The roll motion refers to the lateral movement or rotation of the joystick 24 about the longitudinal axis, which is an imaginary line extending from the bow (front) to the stern (rear) of the ship 100. The yaw motion refers to the left and right movement or rotation of the joystick 24 about the vertical axis, corresponding to the movement by which the ship 100 turns or rotates due to a change in direction or course. These three degrees of freedom enable the joystick 24 to control the movement and orientation of the ship 100 in three-dimensional space.

[0028] The joystick 24 is arranged to be movable, for example, between a position herein referred to as the equilibrium position and one or more displacement positions. The equilibrium position should be understood as the neutral or default position of the joystick 24 that is assumed when no external force is applied to the joystick 24. In some embodiments, the external force is the force applied by the user. Thus, in these embodiments, it will be understood that when there is no force applied by the user to the joystick 24, the joystick 24 is maintained in the equilibrium position. For example, by the force feedback device 22, the joystick 24 is maintained in the equilibrium position as long as no other operating resistance is applied to the input device 20. The equilibrium position is typically the central position of the joystick 24 with respect to the mechanical end positions of the joystick 24 defined by the physical limitations of the joystick 24. However, in other joystick designs, other position details of the equilibrium position may also be included. The displacement position should be understood as a position displaced from the equilibrium position. The displacement position may correspond to the mechanical end positions of the joystick 24 defined by the physical limitations of the joystick 24. The displacement position may correspond to any position between the equilibrium position and the mechanical end position.

[0029] The joystick 24 may include a position sensor (not shown) configured to identify position data of the joystick 24. Using this information, it can be determined whether the joystick 24 is in a displacement position or in the equilibrium position. The position sensor can be a potentiometer, a Hall effect sensor, an optical encoder, a capacitive sensor, a resistive film sensor, a magnetic sensor, etc.

[0030] The force feedback device 22 is adapted to apply force feedback to the joystick 24. The force feedback can be applied in the form of tactile feedback, which corresponds to a physical sensation or force to the user in response to the interaction with the joystick 24. Thus, the force feedback device 22 is adapted to provide force feedback in response to the operator of the ship 100 manipulating the joystick 24 between the various positions described above.

[0031] By adjusting the operating resistance of the joystick 24, force feedback can be applied. The force feedback device 22 can be a mechanical device and / or an electrical device. In non-limiting embodiments, the force feedback device 22 can include an electric motor, an actuator, a piezoelectric device, a hydraulic device, a pneumatic device, a shape memory alloy, an electromagnetic device, or a mechanical linkage mechanism, etc. In an embodiment where the joystick 24 is operable in three degrees of freedom, the force feedback device 22 can include respective force feedback units for each degree of freedom. Thus, it is possible to apply a target force feedback towards a selected portion of the joystick 24 (e.g., via one or more of the force feedback units). The force feedback device 22 may be integrated with the joystick 24 or provided outside the joystick 24, but may be configured to transmit force feedback through a connection with the joystick 24. When the force feedback device 22 is for external use, it can include an external controller configured to send a signal to the controller of the joystick 24, whereby force feedback can be generated within the joystick 24.

[0032] The operating resistance of the joystick 24 can be adjusted by a fixed force value or a variable force value. For example, consider a scenario where a navigation requirement including a speed value of 10000 is requested. Applying a fixed force value means that the value 10000 is immediately reduced to a lower specific value such as 8000. In the case of a variable force value, the speed value 10000 can instead be gradually reduced from 10000 to 8000 via intermediate values such as 9500, 9000, 8500, or generally, at any subinterval of a granularity suitable for the current operating situation. The variable force value can be, for example, an integral value over time that functions as proportional-integral-derivative control (PID). Therefore, depending on the type of force value applied, the magnitude and direction of the force value may or may not change.

[0033] To provide force feedback, the direction of the force value is usually opposite to the direction of movement of the joystick 24 or the next direction of movement associated with the navigation requirement. For example, the movement of the joystick 24 from the displacement position to the equilibrium position may be accompanied by the application of a force value in the direction from the equilibrium position to the displacement position. Since the force value can vary, the operating speed of the joystick 24 from the displacement position to the equilibrium position can vary depending on the force value. The force value can lock the joystick 24 in a predetermined position by completely preventing the movement of the joystick 24 from the displacement position to the equilibrium position. Also, the force value can be made small enough so that the joystick 24 can move from the displacement position to the equilibrium position. This can be done by varying the magnitude so that the operating speed of the joystick 24 changes.

[0034] The marine steering system 5 includes a ship control unit 30. The ship control unit 30 is operatively connected to proximity sensor(s) 10 and an input device 20. The ship control unit 30 may include a processing circuit of the computer system of the ship 100. The ship control unit 30 is configured to acquire the required movement of the joystick 24 indicating the speed value and direction value of the next navigation of the ship 100. The ship control unit 30 is configured to acquire one or more proximity signals from the proximity sensor(s) 10 in response to sensing at least one object in the surrounding 102 in at least the direction indicated by the direction value. The ship control unit 30 is configured to identify the distance between the ship 10 and at least one object based on the one or more proximity signals. The ship control unit 30 is configured to control a force feedback device 22 to apply force feedback to the joystick 24 based on the identified distance. Various embodiments and descriptions of these functions performed by the ship control unit 30 are described later in the present disclosure.

[0035] The ship control unit 30 can be configured to acquire the longitudinal speed of the ship 100. The longitudinal speed can be acquired at least approximately in real time, which means that the longitudinal speed can be acquired continuously (or at least repeatedly). The longitudinal speed is the speed at which the ship 100 moves forward or backward along its length, which is also known as the speed over the ground. The longitudinal speed can be acquired by any known method of acquiring the longitudinal speed of a ship, such as an input from one or more of a speed sensor, an engine rotation sensor, a positioning system, a navigation system, a fleet management system, a light detection system, a radar detection system, a sonar detection system, or a nautical chart. In some embodiments, the ship control unit 30 can be configured to set a longitudinal speed threshold. In these embodiments, the ship control unit 30 can be configured to control the force feedback device 22 based on the result of comparing the longitudinal speed with the longitudinal speed threshold. The longitudinal speed threshold can be a fixed value such as 2 knots, 5 knots, 10 knots, or 20 knots, or any other similar speed value normally associated with the ship. The fixed value can be related to one or more speed limits of the ship 100. The speed limit can be a ship limit or an external limit. The ship limit is related to the characteristics of the ship 100 and can include one or more of the hull design, maximum power output, weight, and dimensional characteristics of the ship 100. The external limit is related to the characteristics surrounding the ship 100 that directly or indirectly affect the speed of the ship 100 and can include one or more of ocean conditions, weather conditions, navigation rules, and environmental regulations.

[0036] The ship steering system 5 includes one or more auxiliary sensors. In this embodiment, the auxiliary sensors are an inertial measurement unit (IMU) 118 and a position sensor (PS) 119. The IMU 118 is configured to provide real-time information regarding the orientation, acceleration, and / or angular velocity of the ship 100. The PS 119 is configured to provide the geographical coordinates of the ship 100, such as GPS / GNSS data.

[0037] The ship 100 is equipped with a propulsion system 110. The propulsion system 110 is responsible for generating the force necessary to propel the ship through the water. The design and operation of the propulsion system 110 can vary depending on the type of ship, including, for example, an electric engine, a diesel engine, a gas turbine, a steam turbine, or a water jet. This disclosure mainly relates to an electric propulsion system. This particular propulsion system 110 represents one type of electric propulsion system that can be controlled via computer control. The propulsion system 110 is adapted to move the ship 100 in one or more of the aforementioned three degrees of freedom. The propulsion system 110 can include several different units in various non-limiting embodiments. In this specific example, the propulsion system 110 includes a pair of engines 112-1, 112-2, gears 114-1, 114-2, and rudders 116-1, 116-2. The propulsion system 110 is involved in the coordination of the engines 112-1, 112-2, gears 114-1, 114-2, and rudders 116-1, 116-2 in response to navigation commands from the ship control unit 30, for example, via the steering station 40. The engines 112-1, 112-2, which can be AC motors or DC motors and can be DC commutator motors in some embodiments, are controlled by a throttle that adjusts the speed by regulating the engine rotation. The engines 112-1, 112-2 can include a power source such as a battery system, a fuel cell system, or another energy system. For example, the power source can be rechargeable, such as when the ship 100 is connected to an external power supply. The external power supply can be an onshore or offshore power facility. The power source can be the ship 100's solar system. The power source can be the ship 100's fuel cell stack. The gears 114-1, 114-2, which are often part of a transmission system (not shown), enable forward and reverse operations and allow the ship to navigate in different directions. The rudders 116-1, 116-2, which are typically arranged at the stern of the ship 100, are used for steering by changing the water flow around the hull of the ship 100 to change the direction of the ship 100. By integrating these components, the ship 100's operator can perform throttle input and steering input, for example, by operating the joystick 24.

[0038] Although not shown in FIG. 1, those skilled in the art will understand that the ship 100 may include additional (sub) systems commonly found in ships, such as an electrical system, a navigation system, a ballast system, a steering system, an HVAC system, an infotainment system, a hydraulic system, a safety system, a communication system, and an auxiliary sensing system.

[0039] The navigation and steering system 5 includes a steering station 40. The steering station 40 is operably connected to the components of the navigation and steering system 5 and functions as a control point for navigation and steering. The control may be related to the ship control unit 30, the IMU 118, the PS 119, the propulsion system 110, or the additional (sub) systems mentioned above.

[0040] Although not shown, it is assumed that the various lines in FIG. 1 refer to the various interfaces or peripherals through which the components communicate with each other. For this purpose, any known wired communication standard or wireless communication standard in the art may be adopted. The wireless communication standards may include IEEE 802.11, IEEE 802.15, ZigBee, WirelessHART, WiFi, Bluetooth®, BLE, RFID, WLAN, MQTT IoT, CoAP, DDS, NFC, AMQP, LoRaWAN, Z-Wave, Sigfox, Thread, EnOcean, mesh communication, or any other form of wireless communication signal between proximity-based devices such as LTE Direct. The wired communication standards may include Controller Area Network (CAN), Ethernet, Hybrid Communication Unit (HCU), Gigabit Multimedia Serial Link (GMSL), Local Interconnect Network (LIN), FlexRay, Media Oriented System Transport (MOST), Universal Serial Bus (USB). The choice of communication standard may depend on data transfer requirements, real-time capabilities, and the specific needs of the various components. It should be understood that the scope of the present disclosure is in no way limited to a specific communication standard.

[0041] Referring further to FIG. 2, an exemplary distance-based force application control is schematically depicted. The events shown in italics refer to actions performed by the vessel control unit 30 and are further described herein by additional embodiments of the present disclosure.

[0042] The first step is for the vessel control unit 30 to obtain a requested operation from the joystick 24. The requested operation indicates the speed value and direction value for the next voyage of the vessel 100. Thus, it should be understood that the requested operation is not the same as the actual operation. This event occurs before any action is taken by the controller of the vessel 100 regarding whether to activate propulsion and / or steering and how to activate them. The requested operation is typically obtained in response to the operator of the joystick 24 applying a force to the joystick 24, for example, from the equilibrium position of the joystick 24 towards the displacement position. However, this may also be done automatically, for example, in an autopilot mode, an automatic docking mode, or other operating modes of the vessel 100 that can be envisioned by those skilled in the art where the joystick 24 can be automatically controlled. The requested operation can vary depending on the type of maneuver, such as requests regarding acceleration, deceleration, steering, rotation, position holding, track holding, thruster control, automatic docking, autopilot, course correction, heading control, speed control, anchoring, mooring, or emergency maneuvering.

[0043] The magnitude of the force applied to the joystick 24 indicates the speed value. Thus, the vessel control unit 30 can be configured to convert the force applied to the joystick 24 into a speed value based on, for example, data obtained by the aforementioned position sensor of the joystick 24. By way of pure example, if the joystick 24 is moved to its maximum (as defined by physical limitations), it can indicate a maximum speed value of 10,000, if the joystick 24 is not moved at all, it can indicate a speed value of 0, and if any force is applied between these, it can correspond to a speed value between 0 and 10,000. Thus, a force application that is 2 / 10 of the maximum possible force application can correspond to a speed value of 2,000, a force application that is 5 / 10 of the maximum possible force application can correspond to a speed value of 5,000, and so on.

[0044] The direction in which the joystick 24 is moved indicates a direction value. Thus, the ship control unit 30 may be configured to convert the direction input given to the joystick 24 into a direction value. Purely as an example, when the joystick 24 is moved from the left position to the right position (roll operation), it may indicate a direction value of 90 degrees from the current front direction of the ship 100. In another example, when the joystick 24 is moved 1 / 4 rotation to the left from the equilibrium position (yaw operation), it may indicate a direction value of 90 degrees counterclockwise from the current front direction of the ship 100. In addition to the above, the direction value may indicate a direction in a plurality of dimensions. Thus, it should be understood that the direction value may indicate one or more of the required pitch rotation, yaw rotation, or roll rotation of the ship 100. It will be understood by those skilled in the art that any legitimate requested operation provided by the joystick 24, including both speed values and / or direction values, may be similarly contemplated, but is not intended to limit the scope of the present disclosure.

[0045] The second step is for the ship control unit 30 to acquire a proximity signal from the proximity sensor 10. These are acquired in the surroundings 102, in this case within the water area, in response to the object 104 being sensed in at least the direction indicated by the direction value. For purposes of illustration, although the ship 100 is not explicitly shown in the depiction of FIG. 2, the ship 100 is assumed to be within a distance range sufficient to acquire one or more proximity signals from the object 104. Since the required action includes a direction value, this step therefore has the prerequisite that the required action is acquired from the joystick 24. This is so that the ship control unit 30 can later determine, for example, whether there is a risk of colliding with the object 104 in the required direction and attempt to perform distance-based force feedback control accordingly. However, in some cases, other directions that are not immediately required by the required action may become relevant for the next navigation. This may apply in the case of an embodiment where the operator requests a clockwise rotation of the ship 100, which involves considering both the left rear of the ship 100 and the right front of the ship 100 due to the rotational nature of the action. Thus, the ship control unit 30 considers sensing an object in at least the direction indicated by the direction value, but as is apparent considering the foregoing, in some cases, the ship control unit 30 also considers sensing an object in additional directions that are not directly indicated but are indirectly indicated by the direction value.

[0046] The third step is for the ship control unit 30 to determine the distance between the ship 100 and the object 104 based on one or more proximity signals. The determined distance can be interpreted as a safety distance for avoiding the danger, which is the object 104 in this case. In some embodiments, the distance can be calculated from the position of the proximity sensor 10 from which the proximity signal was acquired. In some embodiments, the distance can be calculated from a weighted origin position, such as the center point of the ship 100, for example. The distance can be calculated using a distance function, for example, the Euclidean distance d from the origin (position x1, y1 of the ship 100) to the target point (position x2, y2 of the object 104) in a 2D space, that is

Equation

[0047] In embodiments where the proximity signal indicates that there are a plurality of objects in the surroundings 102 at least in the direction indicated by the direction value, the ship control unit 30 can be configured to specify the distances between the ship 100 and each of the plurality of objects. For this purpose, a plurality of distances are each specified and, for further control of the force feedback device 22, the plurality of distances can be considered individually or in combination.

[0048] The fourth step is for the vessel control unit 30 to control the force feedback device 22 to apply force feedback to the joystick 24 based on the identified distance. In FIG. 2, this step is shown as two separate events. The first event involves the vessel control unit 30 submitting one or more commands to the force feedback device 22 based on the distance. This may be understood as a control signal. The second event involves the force feedback device 22 executing the application of force feedback to the joystick 24 in response to the control signal. In FIG. 2, the force feedback is shown as a physical sensation that reaches into the operator's hand from the base of the joystick 24, through its handle. These two events cooperate to form the controlled operation of the force feedback device 22. Thus, the operator is intuitively notified via physical sensation that an object 104 exists within an unsafe distance range from the vessel 100 and experiences greater resistance (or less resistance in the case of a safe distance) when attempting to operate the joystick 24.

[0049] In some embodiments, controlling the force feedback device 22 based on a specified distance may involve a three-step procedure. The three-step procedure includes a first step of calculating a maximum allowable speed of the vessel 100 based on the specified distance. The maximum allowable speed corresponds to the propulsion force allowed in the next voyage of the vessel 100 that, for example, does not risk a collision or some other dangerous event. The maximum allowable speed may be based on one or more operating conditions of the vessel 100. The operating conditions may be propulsion characteristics (e.g., maximum possible energy processing capacity), braking characteristics (e.g., maximum possible braking force), the size or weight of the vessel 100, or the operating mode of the vessel 100 (e.g., whether the vessel 100 is operating in an automatic docking mode or an adaptive cruise mode), etc. The maximum allowable speed may additionally or alternatively be based on the environmental conditions of the surroundings 102. The environmental conditions may be related to general weather conditions (e.g., when wind force, wave height, temperature, precipitation, etc. exceed or fall below their respective predetermined safety thresholds), speed zones (e.g., port areas, mooring facilities, coastal areas, ocean zones), etc. The speed zones may be specified by nautical charts or obtained from a database storing information on the maximum allowable speeds in various geographical areas.

[0050] The second step of the three-step procedure mentioned above includes converting the maximum allowable speed into a maximum allowable joystick movement. Since the vessel control unit 30 knows the maximum allowable speed of the vessel 100, it is also clear how to convert the maximum allowable speed into a joystick movement that does not exceed the maximum allowable speed. This is referred to as the maximum allowable joystick movement. The third step of the three-step procedure includes restricting the required movement of the joystick 24 based on the maximum allowable joystick movement. In some embodiments, restricting the required movement of the joystick 24 may be performed by controlling the force feedback device 22 to apply a force feedback that opposes the force of the required movement. For this reason, the applied force feedback is greater than the force applied to the joystick 24 to initiate the required movement.

[0051] Here, the procedures of the above three steps will be described according to one possible embodiment. It should be noted that this embodiment is only for illustrative purposes and should never be construed as limiting the scope of the present disclosure. It can be apparent that there are numerous other similar embodiments having other objects (plural possible), distances (plural possible), speeds (plural possible), etc. that somehow affect the method of controlling the force feedback device 22 to apply force feedback to the joystick 24. In this embodiment, it is specified that the ship 100 exists within 100 meters of the object 104. The object 104 is another moving ship that is on the route of passing through the ship 100 from the front right side to the front left side of the ship 100. For the next navigation, in order to pass the moving ship without causing a collision, it is specified that the ship 100 must navigate at a speed of 5 m / s or less in the forward direction indicated by the direction value. For this reason, if the ship 100 continues to navigate at the maximum allowable speed of 5 m / s, the ship 100 will take about 20 seconds to reach the location of the moving ship (5 m / s × 20 s = 100 m), and by then the moving ship has already moved from that location to the front left side position. The ship control unit 30 knows which output value of the joystick 24 can cause the speed of the ship 100 to exceed 5 m / s, and thus, based on the maximum allowable joystick value, any operation request of the joystick 24 that is specified to exceed this output value is restricted. As will be apparent by referring to some of the following embodiments, it should be understood that since the force feedback is not necessarily applied uniformly, the control is not necessarily executed according to this method.

[0052] In some embodiments, the vessel control unit 30 may be configured to control the force feedback device 22 to gradually increase the applied force as the distance between the vessel and the object 104 decreases. Gradually increasing may be associated with a variable value in some embodiments, and thus will gradually increase in an accelerating manner. The same can be achieved in the reverse situation. In the reverse situation, the force feedback applied increases as the distance between the vessel 100 and the object 104 increases. Thus, as the proximity of a dangerous situation becomes higher / lower due to the distance to the object 104 being closer / farther, a larger / smaller force feedback is applied to the joystick 24. In the above embodiments, this means that, for example, in the first 50 m, the vessel 100 may be allowed to exceed a speed of 5 m / s, but in this case, a greater restriction on the operation of the joystick 24 is required such that the vessel 100 is below 5 m / s in the last 50 m. This can be controlled based on general situations and may be changed during navigation. For example, a proximity sensor 10 may indicate via an additional proximity signal that a moving vessel is moving faster than expected, for example by acceleration, and as a result, a smaller force feedback may be applied.

[0053] In some embodiments, the vessel control unit 30 may be configured to control the force feedback device 22 to gradually increase the applied force as the speed value increases. Gradually increasing may be associated with a variable value in some embodiments, and thus will gradually increase in an accelerating manner. The same can be achieved in the reverse situation. In the reverse situation, the force feedback applied decreases as the speed value decreases. Thus, as the speed value becomes higher / lower, a larger / smaller force feedback is applied to the joystick 24.

[0054] In some embodiments, the vessel control unit 30 may be configured to control the force feedback device 22 based on the navigable water conditions in which the vessel 100 is proceeding. The navigable water conditions can be any water conditions that can affect the way the joystick 24 is operated. For example, the wind speed, wave height, and / or strength of the tidal current in the water can affect the amount of force that must be applied to the input source 20 to control the behavior of the input source 20. Other conditions can include meteorological conditions such as water temperature and the presence of ice. The vessel control unit 30 may be configured to set a predetermined threshold associated with one or more of the navigable water conditions. The predetermined threshold can define limitations on the operable ways of the joystick 24 based on general conditions. The vessel control unit 30 may be configured to obtain the current navigable water conditions, compare them with their respective predetermined thresholds, and accordingly execute the control of the force feedback device 22. In some embodiments, in response to the navigable water conditions exceeding their respective predetermined thresholds, certain requested operations of the joystick 24 can be ignored. Ignoring a particular request can be useful in situations where the request is unintentionally triggered. Such unintentional triggering of the request can be, for example, as a result of the navigable water conditions affecting the controllability of the joystick 24, such as when severe rocking occurs on the vessel 100, causing the operator to release the joystick 24 or apply an unexpected force to the joystick 24. In some embodiments, the action request can be accommodated by the associated control of the force feedback device 22 in response to the navigable water conditions being below their respective predetermined thresholds. This can correspond to normal behavior where excessive navigable water conditions are not expected.

[0055] In some embodiments, the applied force feedback can be disabled by an external force applied to the joystick 24 that exceeds the value of the force feedback. This can be useful in certain scenarios where the operator of the joystick 24 wants to regain control of the joystick 24. Such scenarios can include cases where the sensor measurement values are incorrect (e.g., when an unnecessary force feedback application occurs due to the false detection of a non-existent object), advanced control situations where the operator cannot rely on the sensor measurement values, and the like.

[0056] In some embodiments, the vessel control unit 30 may be configured to cause control of the force feedback device 22 based on the classification of the object 104. As described above, since the object 104 is not limited to a specific type, in these embodiments, the vessel control unit 30 may identify one or more classifications of the object 104 by processing the proximity signal. For example, in embodiments where the proximity sensor 10 is embodied as a camera and is capable of capturing an image of the object 104, identification of the classification of the object may be performed. Thus, the proximity signal may include or be accompanied by one or more images. Classification identification may be performed by applying an image processing algorithm to these images. The image processing may be performed by the vessel control unit 30 and / or the proximity sensor 10 (e.g., in embodiments where the proximity sensor 10 is a smart sensor having computing capabilities). Known image processing algorithms for these in the art may include convolutional neural networks (CNNs), support vector machines (SVMs), k-nearest neighbors (KNNs), decision trees, random forests, feature extraction techniques, image segmentation, transfer learning, and the like.

[0057] The classification may include one or more of a movement attribute, a size attribute, a mass attribute, a living body attribute, or a material composition attribute. Based on the set classification type, the force feedback application can be appropriately controlled. For example, if the object 104 is classified as a moving object rather than a stationary object due to the movement attribute, an increase in safety risk may be involved, so it may be necessary to adopt additional safety restrictions. In another example, a smaller object rather than a larger object indicated by the size attribute may be ignorable (for example, if it is a small fish), so a very high force feedback may not be necessary. The mass attribute can be regarded in the same way. In yet another example, a living body object (such as a floating human) may require immediate action and the application of a higher force feedback compared to a non-living body object (such as a buoy). In a further example, a specific material composition (for example, a combustible object) is usually associated with a higher risk, so appropriate control can be performed based on this.

[0058] In an example where an object is associated with multiple classifications, the weighted values of the multiple classifications can be calculated, and control can be executed accordingly. For this purpose, additional safety measures may be provided. For example, if a human (due to the living body attribute) is identified and it is also identified that the human is swimming towards the ship 100 (due to the movement attribute), it may be even more dangerous.

[0059] Regarding FIG. 2, it has been discussed to some extent here by various examples. One or more of these examples may also be applicable to the exemplary traffic scenario shown in FIG. 3, and this will be discussed to some extent here.

[0060] Figure 3 shows a top view of a traffic scenario. The ship 100 moves in the water area within the surroundings 102. In the current depiction, the ship 100 is located at the first position 100-1 and desires to move to the second position 100-2 while avoiding any possible dangers. The operator of the ship 100 is notified of these dangers via the force feedback applied to the joystick 24 by control, as discussed herein. In the surroundings 102, a plurality of objects 104 are apparent. The objects 104 include a buoy 104-1, a larger ship 104-2, a dock 104-3, a small ship 104-4, and a wave 104-5 in the water area. To reach the second position 100-2, the ship 100 needs to navigate through these different objects.

[0061] Since the buoy 104-1, the small ship 104-4, and the wave 104-5 do not exist in the direction indicated by the direction value, it is determined that they are not relevant to this specific required operation. As the required operation causes the rotation of the ship 100, the stern of the ship 100 faces at least partially the buoy 104-1 and the wave 104-5, so the proximity signal senses the buoy 104-1 and the wave 104-5, but the buoy 104-1 is too far away and the wave 104-5 is classified as not large enough to require the application of force feedback. The small ship 104-4 is not sensed because it does not appear in the direction required by the direction value.

[0062] Taking the above into consideration, the perceived objects are the large ship 104-2 and the quay 104-3. In this specific embodiment, accordingly, the force feedback is applied in two different ways according to (1) and (2) as shown in FIG. 3. In the figure shown in FIG. 3, when the ship 100 is passing by the left side of the large ship 104-2, the first force feedback is applied, and optionally, as the large ship 104-2 approaches the ship 100, the first force feedback gradually increases. Therefore, the force feedback is preferably applied to the front left part of the handle of the joystick 24 as shown in (1). When the bow of the ship 100 is approaching the quay 104-3, the second force feedback is applied. Since this indicates the danger immediately in front of the ship 100, it is preferable that the force feedback is applied to the front part of the handle of the joystick 24 as shown in (2).

[0063] In some embodiments, control of the residual application of force feedback may be applied. This may be performed by the vessel control unit 30. In this specific embodiment, this is performed as a residual effect after the first two force feedbacks are applied, such that the force feedback still remains even after bypassing the two objects 104-2, 104-3. This is shown in (3). In these embodiments, some final force feedback may be applied to each side portion of the handle of the joystick 24, whereby the operator can pass the object without problems and receive feedback that the object is currently located on each side (left and right) of the vessel 100. For this reason, in some embodiments, the vessel control unit 30 may be configured to provide control of the force feedback device 22 so as to produce a force feedback residual effect. The handle portion to which the force feedback residual effect is applied may change or remain the same compared to the original portion of the force feedback effect, as shown in this embodiment. Since the force feedback residual effect may indicate that the danger is not as imminent as before, it is usually smaller than the previous force feedback. The handle portion to which the remaining force feedback is applied may decrease over time until the force feedback completely disappears as the vessel 100 moves away from the sensed object. Thus, the operator can advantageously receive feedback on the successfully avoided danger via the joystick 24. These embodiments can be understood as providing "following" force feedback on the operation of the vessel 100 with respect to the sensed objects 104-2, 104-3.

[0064] Referring further to FIG. 4, a computer-implemented method 200 for a vessel 100 is shown. The method 200 can be implemented by the processing circuitry of the vessel control unit 30 as discussed herein. The method 200 includes obtaining 210 a requested movement of the joystick 24 indicating a speed value and a direction value for the next voyage of the vessel 100. The method 200 includes obtaining 220 one or more proximity signals from the proximity sensor 10 in response to at least one object 104 being sensed in the surroundings 102 in at least the direction indicated by the direction value. The method 200 includes determining 230 the distance between the vessel 100 and at least one object 104 based on the one or more proximity signals. The method 200 includes controlling 240 the force feedback device 22 to apply force feedback to the joystick 24 based on the determined distance.

[0065] Referring further to FIG. 5, the controlling 240 shown in FIG. 4 is described in more detail in one embodiment. The controlling 240 includes calculating 242 a maximum allowable speed of the vessel 100 based on the determined distance. The controlling 240 includes converting 244 the maximum allowable speed into a maximum allowable joystick movement. The controlling 240 includes restricting 246 the requested movement of the joystick 24 based on the maximum allowable joystick movement. In some embodiments, the restricting 246 can include controlling 248 the force feedback device 22 to apply a force feedback value that opposes the force of the requested movement.

[0066] FIG. 6 is a schematic diagram of a computer system 600 for implementing the embodiments disclosed herein. The computer system 600 is adapted to execute instructions from a computer-readable medium to perform the functions or processes described herein and / or any of them. The computer system 600 can be connected (e.g., network-connected) to other machines within a LAN (Local Area Network), LIN (Local Interconnect Network), automotive network communication protocol (e.g., FlexRay), intranet, extranet, or the Internet. Although only a single device is shown, the computer system 600 can include any set of devices that individually or jointly execute a set (or sets) of instructions to perform any one or more of the methods discussed herein. Thus, any reference in this disclosure and / or the claims to a computer system, computing system, computer device, computing device, control system, control unit, electronic control unit (ECU), processor device, processing circuit, etc. includes a reference to one or more devices that individually or jointly execute a set (or sets) of instructions to perform any one or more of the methods discussed herein. For example, a control system can include a single control unit, or can include a plurality of control units interconnected or otherwise communicatively coupled such that any function performed is distributed among the control units as desired. Further, such devices can communicate with each other or with other devices according to various system architectures, e.g., directly or via a controller area network (CAN) bus, etc.

[0067] The computer system 600 may comprise at least one computing device or electronic device, which can include firmware, hardware, and / or execute software instructions for implementing the functions described herein. The computer system 600 may include a processing circuit 602 (e.g., a processing circuit including one or more processor devices or control units), a memory 604, and a system bus 606. The computer system 600 may include at least one computing device having the processing circuit 602. The system bus 606 provides an interface to system components including, but not limited to, the memory 604 and the processing circuit 602. The processing circuit 602 may include any number of hardware components for performing data processing or signal processing, or any number of hardware components for executing computer code stored in the memory 604. The processing circuit 602 may include, for example, a general-purpose processor, an application-specific processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a circuit including processing components, a group of distributed processing components, a group of distributed computers configured for processing, or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. The processing circuit 602 may further include computer-executable code for controlling the operation of the programmable device.

[0068] The system bus 606 can be any of several types of bus structures that can further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and / or a local bus using any of various bus architectures. The memory 604 can be one or more devices for storing data and / or computer code that completes or facilitates the methods described herein. The memory 604 can include a database component, an object code component, a script component, or other types of information structures for corresponding to various activities herein. Any distributed memory device or local memory device can be utilized with the systems and methods herein. The memory 604 can be communicatively coupled to the processing circuit 602 (e.g., via circuitry, or any other wired connection, wireless connection, or network connection), and can include computer code for performing one or more processes described herein. The memory 604 can include non-volatile memory 608 (e.g., read only memory (ROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc.), and volatile memory 610 (e.g., random access memory (RAM)), or any other medium that can be used to carry or store desired program code in the form of machine-executable instructions or data structures and that can be accessed by a computer or other machine having the processing circuit 602. The basic input / output system (BIOS) 612 can be stored in the non-volatile memory 608 and can include basic routines that help to transfer information between elements within the computer system 600.

[0069] The computer system 600 may further include or be connected to a non-transitory computer-readable storage medium such as a storage device 614. The non-transitory computer-readable storage medium may include, for example, an internal or external hard disk drive (HDD) (e.g., Enhanced Integrated Drive Electronics (EIDE) or Serial Advanced Technology Attachment (SATA)), a storage HDD (e.g., EIDE or SATA), or a flash memory. The storage device 614, as well as other drives associated with the computer-readable media and computer-usable media, may provide non-volatile storage of data, data structures, and computer-executable instructions, etc.

[0070] Hard-coded or soft-coded computer code may be provided in the form of one or more modules. The module(s) may be implemented as software to perform all or part of the functions described herein and / or may be hard-coded in a circuit. The module may be stored in the storage device 614 and / or the volatile memory 610, and the volatile memory 610 may include an operating system 616 and / or one or more program modules 618. All or part of the embodiments disclosed herein may be implemented as a computer program 620 stored in a temporary or non-temporary computer-usable medium or computer-readable storage medium (e.g., a single medium or multiple media) such as the storage device 614, and the computer program 620 includes complex programming instructions (e.g., complex computer-readable program code) for causing the processing circuit 602 to execute the actions described herein. Thus, the computer-readable program code of the computer program 620 may include software instructions that, when executed by the processing circuit 602, implement the functions of the embodiments described herein. In some embodiments, the storage device 614 may be a computer program product (e.g., a readable storage medium) that stores the computer program 620 in itself, and at least a portion of the computer program 620 may be loadable (e.g., into a processor) to implement the functions of the embodiments described herein when executed by the processing circuit 602. The processing circuit 602 may function as a controller or control system of the computer system 600 for implementing the functions described herein.

[0071] Computer system 600 may include an input device interface 622 configured to receive inputs and selections communicated to the computer system 600 when executing instructions from a keyboard, mouse, touch-sensitive surface, etc. Such input devices may be connected to processing circuitry 602 via an input device interface 622 connected to system bus 606, but may also be connected via other interfaces such as a parallel port, Institute of Electrical and Electronic Engineers (IEEE) 1394 serial port, Universal Serial Bus (USB) port, and IR interface. Computer system 600 may include an output device interface 624 configured to transfer outputs to a display, video display unit (e.g., liquid crystal display (LCD) or cathode ray tube (CRT)), etc. Computer system 600 may include a communication interface 626 suitable for communicating with a network, as appropriate or as desired.

[0072] Any of the illustrative aspects described herein are described for purposes of providing examples and discussion. The actions may be performed by hardware components, embodied by machine-executable instructions that cause a processor to perform the actions, or performed by a combination of hardware and software. Although a particular order of method actions may be shown or described, the order of the actions may be different. Further, two or more actions may be performed simultaneously or partially simultaneously.

[0073] Example 1: A navigation control system (5) for a ship (100), comprising a proximity sensor (10) configured to sense at least a part of the surroundings (102) surrounding the ship (100), an input device (20) adapted to provide a navigation command to the ship (100), the input device (20) comprising a movable joystick (24) and a force feedback device (22) adapted to apply force feedback to the joystick (24), and a ship control unit (30) operably connected to the proximity sensor (10) and the input device (20), wherein the ship control unit (30) is configured to obtain a required operation of the joystick (24) indicating a speed value and a direction value for the next navigation of the ship (100), obtain one or more proximity signals from the proximity sensor (10) in response to sensing at least one object (104) in the surroundings (102) in at least the direction indicated by the direction value, identify a distance between the ship (100) and the at least one object (104) based on the one or more proximity signals, and control the force feedback device (22) to apply force feedback to the joystick (24) based on the identified distance. The navigation control system (5).

[0074] Example 2: The ship control unit (30) is further configured to control the force feedback device (22) by calculating a maximum allowable speed of the ship (100) based on the identified distance, converting the maximum allowable speed into a maximum allowable joystick operation, and restricting the required operation of the joystick (24) based on the maximum allowable joystick operation. The navigation control system (5) according to Example 1.

[0075] Example 3: The ship control unit (30) is further configured to control the force feedback device (22) to apply a force feedback value that counteracts the force of the required operation of the joystick (24), thereby restricting the required operation of the joystick (24). The navigation control system (5) according to Example 2.

[0076] Example 4: The navigation control system (5) according to Example 2 or 3, wherein the maximum allowable speed is further based on one or more operating conditions of the ship (100).

[0077] Example 5: The navigation control system (5) according to any one of Examples 2 to 4, wherein the maximum allowable speed is further based on one or more environmental conditions of the surroundings in which the ship (100) is traveling.

[0078] Example 6: The navigation control system (5) according to any one of Examples 1 to 5, wherein the ship control unit (30) is further configured to control the force feedback device (22) to gradually increase the applied force feedback when the distance between the ship (100) and the object (104) is decreasing.

[0079] Example 7: The navigation control system (5) according to any one of Examples 1 to 6, wherein the ship control unit (30) is further configured to control the force feedback device (22) to gradually increase the applied force feedback when the speed value is increasing.

[0080] Example 8: The navigation control system (5) according to any one of Examples 1 to 7, wherein the ship control unit (30) is further configured to control the force feedback device (22) to apply a fixed force feedback value.

[0081] Example 9: The navigation control system (5) according to any one of Examples 1 to 7, wherein the ship control unit (30) is further configured to control the force feedback device (22) to apply a variable force feedback value.

[0082] Example 10: The navigation control system (5) according to any one of Examples 1 to 9, wherein the applied force feedback can be invalidated by an external force applied to the joystick (24) that exceeds the value of the force feedback.

[0083] Embodiment 11: When the proximity signal indicates that there are a plurality of objects (104-1, 104-2, 104-3, 104-4, 104-5) in at least the direction indicated by the direction value in the periphery (102), the ship control unit (30) determines the distance between the ship (100) and each of the plurality of objects (104-1, 104-2, 104-3, 104-4, 104-5), and controls the force feedback device (22) to apply force feedback to the joystick (24) based on each of the determined distances. The marine steering system (5) according to any one of Embodiments 1 to 10 is configured to perform the above operations.

[0084] Embodiment 12: The joystick (24) can be moved in three degrees of freedom, the force feedback device (22) includes respective force feedback units for each degree of freedom of the joystick (24), and the ship control unit (30) is configured to control the application of the force feedback to the joystick (24) through one or more of the force feedback units. The marine steering system (5) according to any one of Embodiments 1 to 11 is configured to perform the above operations.

[0085] Embodiment 13: The ship control unit (30) is further configured to control the force feedback device (22) based on the navigable water area conditions in which the ship (100) is traveling, and the navigable water area conditions include one or more of wind speed, wave height, tidal current strength, and weather conditions. The marine steering system (5) according to any one of Embodiments 1 to 12 is configured to perform the above operations.

[0086] Example 14: The marine control unit (30) is further configured to process the one or more proximity signals to identify at least one classification of the at least one object (104), and to apply force feedback to the joystick (24) based on the identified at least one classification, so as to control the force feedback device (22). The marine steering system (5) according to any one of Examples 1 to 13.

[0087] Example 15: The marine steering system (5) according to Example 14, wherein the at least one classification includes one of a movement attribute, a size attribute, a mass attribute, a biological attribute, or a material composition attribute.

[0088] Example 16: The marine control unit (30) is further configured to calculate weighted values of a plurality of classifications from among the at least one classification, and to apply force feedback to the joystick based on the calculated weighted values, so as to control the force feedback device (22). The marine steering system (5) according to Example 14 or 15.

[0089] Example 17: The marine steering system (5) according to any one of Examples 1 to 16, wherein the proximity sensor (10) is one of a lidar device, a radar device, a sonar device, an ultrasonic device, or a camera.

[0090] Example 18: In response to an operation of the joystick (24) started by one of manual or automatic, the requested operation of the joystick (24) is received. The marine steering system (5) according to any one of Examples 1 to 17.

[0091] Example 19: The marine steering system (5) according to any one of Examples 1 to 18, wherein the force feedback device (22) includes a DC commutator motor.

[0092] Example 20: The marine control unit (30) further controls the force feedback device (22) to perform force feedback residual application adapted to follow the operation of the ship (20) with respect to the at least one object (104), the marine steering system (5) according to any one of Examples 1 to 19.

[0093] Example 21: A ship (100) equipped with the marine steering system (5) according to any one of Examples 1 to 20.

[0094] Example 22: The ship (100) according to Example 21, further comprising a steering station (40), an auxiliary sensor unit (118, 119) which is one of an inertial measurement unit (118) or a positioning sensor (119), and a propulsion system (110).

[0095] Example 23: The ship (100) according to Example 22, wherein the propulsion system (110) comprises a pair of engines (112-1, 112-2), a pair of gears (114-1, 114-2), and a pair of rudders (116-1, 116-2).

[0096] Example 24: A computer-implemented method (200) for a ship (100), comprising: obtaining (210) a required operation of a joystick (24) indicating a speed value and a direction value for the next voyage of the ship (100); obtaining (220) one or more proximity signals from a proximity sensor (10) in response to sensing at least one object (104) in the surroundings (102) in at least the direction indicated by the direction value; specifying (230) a distance between the ship (100) and the at least one object (104) based on the one or more proximity signals; and controlling (240) a force feedback device (22) to apply force feedback to the joystick (24) based on the specified distance.

[0097] Example 25: The controlling (240) further includes calculating (242) a maximum allowable speed of the ship (100) based on the identified distance, converting (244) the maximum allowable speed into a maximum allowable joystick operation, and restricting (246) the required operation of the joystick (24) based on the maximum allowable joystick operation, the method (200) according to Example 24.

[0098] Example 26: The restricting (246) further includes controlling (248) the force feedback device (22) to apply a force feedback value that counteracts the force of the required operation, the method (200) according to Example 25.

[0099] Example 27: A computer program product including program code that, when executed by a processing circuit (602), performs the method according to Example 24, the computer program product.

[0100] Example 28: A non-transitory computer-readable storage medium including instructions that, when executed by the processing circuit (602), cause the processing circuit (602) to perform the method according to Example 24, the non-transitory computer-readable storage medium.

[0101] The terms used in this specification are for the sole purpose of describing particular embodiments and are not intended to limit the disclosure. The singular forms "a", "an", and "the" used in this specification are intended to include the plural forms as well, unless the context clearly dictates otherwise. The term "and / or" used in this specification encompasses any and all combinations of one or more of the listed related items. The terms "comprises", "comprising", "includes", and / or "including", when used in this specification, specify the presence of the described features, elements, actions, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, actions, steps, operations, elements, components, and / or groups thereof.

[0102] In this specification, terms such as first, second, etc. may be used to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the present disclosure, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element.

[0103] Relative terms such as "downward", "upward", "upper", "lower", "horizontal", "vertical", etc. may be used in this specification to describe the relationship between one element shown in the figure and another element. It should be understood that these terms, as well as the aforementioned terms, are intended to encompass various orientations of the device in addition to the orientation depicted in the figure. When an element is referred to as being "connected" or "coupled" to another element, it should be understood that the element may be directly connected or coupled to the other element, or intervening elements may be present. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, no intervening elements are present.

[0104] Unless otherwise defined, all terms (including technical and scientific terms) used in this specification shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It should be further understood that terms used in this specification shall be interpreted as having a meaning consistent with their meaning in the context of this specification and the relevant art, and shall not be interpreted in an idealized or overly formalized sense unless explicitly defined herein.

[0105] It should be understood that the present disclosure is not limited to the embodiments described above and shown in the drawings, but rather many changes and modifications may be made within the scope of the present disclosure and the appended claims, as will be recognized by those skilled in the art. The embodiments are disclosed in the drawings and the specification for illustrative purposes only, not for limiting purposes, and the scope of the present disclosure is defined in the following claims.

Claims

1. 1. A marine navigation system for a marine vessel, comprising: a proximity sensor configured to sense at least a portion of a perimeter surrounding the marine vessel; an input device adapted to provide navigation commands to the vessel, the input device comprising a movable joystick and a force feedback device adapted to apply force feedback to the joystick; a vessel control unit operatively connected to the proximity sensor and the input device, the vessel control unit comprising: obtaining a desired movement of the joystick indicative of speed and direction values ​​for an upcoming journey of the vessel; obtaining one or more proximity signals from the proximity sensor in response to at least one object being sensed in the surroundings at least in a direction indicated by the directional value; determining a distance between the vessel and the at least one object based on the one or more proximity signals; controlling the force feedback device to apply force feedback to the control stick based on the determined distance; The marine navigation system is configured to execute the following steps:

2. The vessel control unit further comprises: calculating a maximum allowable speed of the vessel based on the determined distance; converting said maximum allowable velocity into a maximum allowable column motion; limiting the requested movement of the control stick based on the maximum allowable control stick movement; 2. The marine navigation system of claim 1, configured to control the force feedback device by executing:

3. 3. The marine navigation system of claim 2, wherein the vessel control unit is further configured to limit the demanded movement of the control stick by controlling the force feedback device to apply a force feedback value that opposes a force of the demanded movement.

4. The marine navigation system of claim 2 , wherein the maximum allowable speed is further based on one or more operating conditions of the vessel.

5. 3. The marine navigation system of claim 2, wherein the maximum allowable speed is further based on one or more environmental conditions of the surroundings in which the marine vessel is traveling.

6. 2. The marine navigation system of claim 1, wherein the vessel control unit is further configured to control the force feedback device to gradually increase the applied force feedback when the distance between the vessel and the object is decreasing.

7. 2. The marine navigation system of claim 1, wherein the vessel control unit is further configured to control the force feedback device to gradually increase the applied force feedback when the velocity value is increasing.

8. The marine navigation system of claim 1 , wherein the applied force feedback can be overridden by an external force applied to the control stick that exceeds the force feedback value.

9. When the proximity signal indicates that a plurality of objects are present in the surroundings at least in the direction indicated by the direction value, the vessel control unit: determining a distance between the vessel and each of the plurality of objects; controlling the force feedback device to apply force feedback to the control stick based on each of the determined distances; 2. The marine navigation system of claim 1 configured to execute:

10. 2. The marine navigation system of claim 1, wherein the control stick is movable in three degrees of freedom, the force feedback device comprises a respective force feedback unit for each degree of freedom of the control stick, and the vessel control unit is configured to control the application of the force feedback to the control stick via one or more of the force feedback units.

11. 2. The marine navigation system of claim 1, wherein the vessel control unit is further configured to control the force feedback device based on navigable water conditions in which the vessel is traveling, the navigable water conditions including one or more of wind speed, wave height, current strength, and weather conditions.

12. The vessel control unit further comprises: processing the one or more proximity signals to identify at least one classification of the at least one object; controlling the force feedback device to apply force feedback to the control stick based on the determined at least one classification; 2. The marine navigation system of claim 1 configured to execute:

13. The marine navigation system of claim 12 , wherein the at least one classification includes one of a movement attribute, a size attribute, a mass attribute, a biometric attribute, or a material composition attribute.

14. The vessel control unit further comprises: calculating weighted values ​​for a plurality of categories from the at least one category; controlling the force feedback device to apply force feedback to the control stick based on the calculated weighting values; 13. The marine navigation system of claim 12 configured to execute:

15. The marine navigation system of claim 1 , wherein the requested movement of the control stick is received in response to one of manually or automatically initiated movement of the control stick.

16. 2. The marine navigation system of claim 1, wherein the vessel control unit is further configured to control the force feedback device to apply a fixed or variable force feedback value.

17. A marine vessel comprising the navigation and steering system according to claim 1.

18. 1. A computer-implemented method for a marine vessel, comprising: obtaining a desired stick movement indicative of speed and direction values ​​for an upcoming voyage of the vessel; obtaining one or more proximity signals from a proximity sensor in response to at least one object being sensed in the surroundings at least in a direction indicated by the directional value; determining a distance between the vessel and the at least one object based on the one or more proximity signals; controlling a force feedback device to apply force feedback to the control stick based on the determined distance; The method comprising:

19. 20. A computer program product comprising program code, the program code being configured to perform the method of claim 18 when executed by a processing circuit.

20. 20. A non-transitory computer readable storage medium containing instructions that, when executed by a processing circuit, cause the processing circuit to perform the method of claim 18.

Images