A ship berthing danger early warning method and device and electronic equipment

Abstract

The application discloses a ship berthing danger early warning method and device and electronic equipment, and the method comprises the following steps: collecting video images in the ship berthing process through an image collection device, wherein the image collection device is installed on the ship side; extracting the ship side line and the wharf shoreline from the video images; determining the first distance between the ship bow and the wharf, the second distance between the ship stern and the wharf, and the real included angle between the ship side line and the wharf shoreline based on the extracted ship side line and the wharf shoreline; and judging whether the ship berthing reaches the danger early warning condition based on the preset early warning rule, the first distance, the second distance and the real included angle, and sending an alarm information when the danger early warning condition is reached.

Description

Technical Field

[0001] This invention relates to the field of ship early warning technology, and more specifically, to a method, device, and electronic device for early warning of ship berthing danger. Background Technology

[0002] Current ship berthing warning methods generally use lidar to measure the distance of a ship to the dock and issue a warning when the measured distance falls below a certain threshold, thus indicating the potential for a collision with the dock. However, existing ship berthing warning methods have the following drawbacks:

[0003] 1) Laser ranging technology is relatively expensive;

[0004] 2) Installation is inconvenient for ships. The optimal installation position for lidar should be parallel to the dock, but ships cannot drill holes for installation at this position. In addition, there is usually a longitudinal height difference between the installation position on a ship and the dock, which causes a certain error in lidar ranging.

[0005] 3) LiDAR is susceptible to the effects of humid weather, which will reduce its performance. In addition, LiDAR has a short detection range for docks that are generally dark in color, making it unsuitable for all types of ships and docks. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method, device, and electronic equipment for early warning of ship berthing dangers.

[0007] According to one aspect of the present invention, a method for early warning of ship berthing danger is provided, comprising:

[0008] Video images of the ship during the berthing process are captured by an image acquisition device installed on the side of the ship.

[0009] Extract the ship's hull line and the dock shoreline from video images;

[0010] Based on the extracted hull line and wharf shoreline, the first distance between the bow of the ship and the wharf, the second distance between the stern of the ship and the wharf, and the true angle between the hull line and the wharf shoreline are determined.

[0011] Based on preset warning rules, first distance, second distance, and actual angle, it determines whether the vessel's berthing meets the danger warning conditions, and issues an alarm message when the danger warning conditions are met.

[0012] Preferably, the image acquisition device includes at least two cameras, one mounted at the front of the ship's side and the other mounted at the rear of the ship's side; and

[0013] The extraction of the ship's hull line and the dock shoreline from the video images includes:

[0014] Stitch together the video images captured by the two cameras;

[0015] The system uses a pre-defined neural network model to identify the stitched video images and extract the ship's hull line and the dock shoreline.

[0016] Preferably, determining the first distance between the bow of the vessel and the dock, the second distance between the stern of the vessel and the dock, and the true angle between the bow line and the dock line based on the extracted hull line and dock shoreline includes:

[0017] Based on the extracted sideline and wharf shoreline, the distance between the bow of the ship and the first pixel point of the wharf, the distance between the stern of the ship and the second pixel point of the wharf, and the virtual angle between the sideline and the wharf shoreline in the video image are determined.

[0018] By using the extrinsic parameters of the image acquisition device, the geometric relationship between the pixel distance in the video image and the actual distance is determined;

[0019] Based on geometric relationships, the distance between the first and second pixels, and the virtual angle, the first distance between the bow of the ship and the dock, the second distance between the stern of the ship and the dock, and the actual angle between the hull line and the dock shoreline are determined.

[0020] Preferably, the preset warning rules include at least one of the following:

[0021] When the distance between the bow and stern of a vessel and the dock is less than a distance threshold, if the speed at which the bow and stern of the vessel approach the dock is greater than a speed threshold, the vessel's berthing will meet the danger warning conditions and an alarm message will be issued.

[0022] When the absolute value of the actual angle between the ship's side line and the quay line is greater than the angle threshold, the ship's berthing meets the danger warning conditions and an alarm message is issued.

[0023] Preferably, the ship berthing hazard warning method further includes setting a distance threshold, a speed threshold, and an angle threshold.

[0024] Preferably, the step of determining whether the vessel's berthing meets the danger warning conditions based on preset warning rules, a first distance, a second distance, and the actual included angle includes:

[0025] Determine whether the first distance and the second distance are less than the distance threshold;

[0026] When the first distance and the second distance are less than the distance threshold, the first speed of the ship's bow approaching the dock and the second speed of the ship's stern approaching the dock are calculated based on the first distance and the second distance.

[0027] Determine whether the first and second velocities are greater than the speed threshold;

[0028] If the first and second speeds are greater than the speed thresholds, the vessel is deemed to have met the danger warning conditions for berthing; otherwise, it is not.

[0029] Preferably, the step of determining whether the vessel's berthing meets the danger warning conditions based on preset warning rules, a first distance, a second distance, and the actual included angle includes:

[0030] Determine whether the absolute value of the actual included angle is greater than the angle threshold;

[0031] When the absolute value of the actual included angle is greater than the angle threshold, the vessel is deemed to have met the danger warning conditions for berthing; otherwise, it is not.

[0032] According to another aspect of the present invention, a ship berthing danger warning device is provided, comprising:

[0033] The image acquisition module is used to acquire video images of the ship during the berthing process through an image acquisition device, wherein the image acquisition device is installed on the side of the ship.

[0034] The extraction module is used to extract the ship's hull line and the dock shoreline from video images;

[0035] The determination module is used to determine the first distance between the bow of the ship and the dock, the second distance between the stern of the ship and the dock, and the true angle between the hull line and the dock shoreline based on the extracted hull line and dock shoreline.

[0036] The danger warning module is used to determine whether the vessel's berthing meets the danger warning conditions based on preset warning rules, first distance, second distance, and actual angle, and to issue an alarm message when the danger warning conditions are met.

[0037] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the methods described in any of the above aspects of the present invention.

[0038] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the executable instructions to implement the method described in any of the preceding aspects of the present invention.

[0039] Therefore, this invention uses an image acquisition device installed on the ship's side to collect video images during the berthing process. Visual recognition technology is used to extract the ship's side line and the dock line from the video images. The distance between the side line and the dock line is obtained based on the distance data calibrated by the image acquisition device. Geometric calculations are then used to obtain the angle between the ship and the dock. Finally, the distance and angle are used to determine whether there is a risk of collision during berthing. This invention uses an image acquisition device as a sensor to perceive the relative position information of the ship and the dock, which is more economical and has better environmental adaptability compared to lidar solutions, while also being easier to install and implement. The berthing warning rules designed using parameters obtained from the image acquisition device are not limited by the type of ship or dock, making it suitable for all types of ships and docks. Attached Figure Description

[0040] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:

[0041] Figure 1 This is a flowchart illustrating a ship berthing danger early warning method provided in an exemplary embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the camera installation position provided in an exemplary embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the quayline and ship hull line abstracted using a straight line fitting method, provided by an exemplary embodiment of the present invention.

[0044] Figure 4 This is a schematic diagram of the wharf shoreline and ship hull line obtained by mapping geometric relationships according to an exemplary embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of the structure of a ship berthing danger early warning device provided in an exemplary embodiment of the present invention;

[0046] Figure 6 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. Detailed Implementation

[0047] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0048] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0049] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of the present invention are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0050] It should also be understood that in the embodiments of the present invention, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0051] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more unless explicitly defined or given contrary instructions in the context.

[0052] Furthermore, the term "and / or" in this invention is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this invention generally indicates that the preceding and following related objects have an "or" relationship.

[0053] It should also be understood that the description of the various embodiments in this invention emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0054] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0055] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0056] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0057] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0058] The embodiments of this invention can be applied to electronic devices such as communication terminals, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Well-known examples of communication terminals, computing systems, environments, and / or configurations suitable for use with electronic devices such as communication terminals, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0059] Electronic devices such as communication terminals, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0060] Exemplary methods

[0061] Figure 1 This is a flowchart illustrating a ship berthing danger early warning method provided in an exemplary embodiment of the present invention. Figure 1 As shown, the ship berthing hazard warning method includes the following steps:

[0062] Step 1: Acquire video images of the ship during the berthing process using an image acquisition device installed on the side of the ship.

[0063] Preferably, the image acquisition device includes at least two cameras, one of which is mounted at the front of the ship's side and the other is mounted at the rear of the ship's side.

[0064] In an embodiment of the present invention, Figure 2 A schematic diagram showing the components of a ship berthing hazard warning system and the installation locations of its cameras is provided. Figure 2 As shown, the system consists of a side-mounted camera and a berthing warning calculation unit. The side-mounted camera is used to collect video images of the ship when it is berthing, and is generally installed at the front and rear of the ship's side, covering the bow and stern. The berthing warning calculation unit processes and recognizes the images generated by the camera to obtain the distance and angle between the bow and stern and the dock, and then obtains the warning result.

[0065] Step 2: Extract the ship's hull line and the dock shoreline from the video images.

[0066] Preferably, the step of extracting the ship's hull line and the dock shoreline from the video images includes: stitching together video images captured by two cameras; and using a preset neural network model to identify the stitched video images and extract the ship's hull line and the dock shoreline.

[0067] In this embodiment of the invention, a top-down view of the ship's berthing process is captured by two cameras, resulting in video images. The images from the two cameras are then stitched together, and a trained CNN model is used to identify and extract the quay line and the ship's side line from the stitched image. The quay line and the ship's side line are then abstracted using a straight-line fitting method, as shown below. Figure 3 As shown.

[0068] Step 3: Based on the extracted hull line and wharf shoreline, determine the first distance between the bow of the ship and the wharf, the second distance between the stern of the ship and the wharf, and the true angle between the hull line and the wharf shoreline.

[0069] Preferably, determining the first distance between the bow of the vessel and the dock, the second distance between the stern of the vessel and the dock, and the true angle between the lateral line and the dock line based on the extracted lateral line and dock line includes: determining the first pixel distance between the bow of the vessel and the dock, the second pixel distance between the stern of the vessel and the dock, and the virtual angle between the lateral line and the dock line in the video image based on the extracted lateral line and dock line; determining the geometric relationship between the pixel distances and the true distances in the video image through the extrinsic parameters of the image acquisition device; and determining the first distance between the bow of the vessel and the dock, the second distance between the stern of the vessel and the dock, and the true angle between the lateral line and the dock line based on the geometric relationship, the first pixel distance, the second pixel distance, and the virtual angle.

[0070] In this embodiment of the invention, using the installation positions of the front and rear cameras as starting points, perpendicular lines are drawn to the quayline, respectively. Distances d1 and d2, as well as the virtual angle α between the hull side line and the quayline, are obtained on the abstract image. Figure 3 As shown. Then, according to the imaging principle of the camera, there is a one-to-one correspondence between the pixel distances in the image captured by the camera and the actual distances, and this correspondence can be obtained through the external parameter calibration of the camera. The external parameter calibration uses a checkerboard method to map the distances of the checkerboard squares to the pixel distances in the image. Based on this geometric relationship, the actual distances D1 and D2 from the side-mounted camera to the dock can be obtained. Using the principle of similarity, the actual angle is equal to the virtual angle in the abstract image, which is also α, as shown. Figure 4 As shown.

[0071] Step 4: Based on the preset warning rules, first distance, second distance, and actual angle, determine whether the vessel's berthing meets the danger warning conditions, and issue an alarm message when the danger warning conditions are met.

[0072] Preferably, the preset warning rules include at least one of the following: when the distance between the bow and stern of the ship and the dock is less than a distance threshold, if the speed at which the bow and stern of the ship approach the dock is greater than a speed threshold, the ship's berthing meets the danger warning conditions and an alarm message is issued; when the absolute value of the actual angle between the ship's side line and the dock shoreline is greater than an angle threshold, the ship's berthing meets the danger warning conditions and an alarm message is issued.

[0073] Preferably, the ship berthing hazard warning method further includes setting a distance threshold, a speed threshold, and an angle threshold.

[0074] In this embodiment of the invention, the alarm logic uses three variables—distance, speed, and angle—for judgment. If the distance is too close and the speed is too high, or if the angle is too large, a collision risk is considered to exist. Therefore, the ship berthing hazard warning rules can be divided into two parts:

[0075] 1) The distance between the bow and stern of the ship and the dock is less than the distance threshold D. g Under the premise that the bow and stern approach the dock at speeds V1 and V2 greater than the speed threshold V... g When a ship berths, the conditions for a danger warning are met;

[0076] 2) The absolute value of the true included angle α, |α|, is greater than the angle threshold α. g When a ship docks, the conditions for a danger warning are met.

[0077] The corresponding warning logic is as follows:

[0078] ((D1 <D g )&&(V1>V g ))

[0079] ||((D2 <D g )&&(V2>V g ))

[0080] ||(|α|>α g )

[0081] Furthermore, the distance threshold, speed threshold, and angle threshold can be set according to the actual situation. For example, but not limited to, the distance threshold is generally set to twice the ship's width or once the ship's width, the speed threshold is generally set to 0.8 knots or more, and the angle threshold generally needs to be set to less than 5°, depending on the ship type.

[0082] Preferably, the step of determining whether the vessel's berthing meets the danger warning conditions based on preset warning rules, a first distance, a second distance, and the actual angle includes: determining whether the first distance and the second distance are less than a distance threshold; when the first distance and the second distance are less than the distance threshold, calculating the first speed of the vessel's bow approaching the dock and the second speed of the vessel's stern approaching the dock based on the first distance and the second distance; determining whether the first speed and the second speed are greater than a speed threshold; when the first speed and the second speed are greater than the speed threshold, determining that the vessel's berthing meets the danger warning conditions; otherwise, not.

[0083] Preferably, the step of determining whether the berthing of a vessel meets the danger warning conditions based on preset warning rules, a first distance, a second distance, and the actual angle includes: determining whether the absolute value of the actual angle is greater than an angle threshold; if the absolute value of the actual angle is greater than the angle threshold, determining that the berthing of the vessel meets the danger warning conditions; otherwise, not.

[0084] In this embodiment of the invention, based on preset warning rules, a first distance, a second distance, and the actual angle, it can be determined whether the vessel's berthing meets the danger warning conditions. An alarm is issued when the distance, speed, and angle between the bow and stern and the dock meet the judgment conditions. The first speed V1 of the bow approaching the dock and the second speed V2 of the stern approaching the dock are obtained by discretely differentiating D1 and D2, i.e.:

[0085] V1=ΔD1 / Δt

[0086] V2=ΔD2 / Δt

[0087] This invention employs visual recognition methods to extract features from the ship's hull and the dock's edge. Through geometric calculations, it obtains the distances from the bow and stern to the dock, as well as the angle between the ship and the dock, optimizing the sensor configuration for berthing warnings. Using the distances and angles obtained through visual recognition, a berthing assistance warning logic is designed that conforms to actual application scenarios and the characteristics of the configured sensors, thereby improving berthing safety.

[0088] In addition, current cameras are equipped with infrared illumination, allowing them to function normally even in the dark lighting conditions of the dock. In foggy or rainy weather, defogging and deraining algorithms or infrared thermal imaging cameras can be used to enhance the camera footage.

[0089] In summary, this invention proposes a method for early warning of ship berthing hazards. It acquires video images of the ship during berthing using an image acquisition device installed on the ship's side. Visual recognition technology is used to extract the ship's sideline and the dock line from the video images. The distance between the sideline and the dock line is obtained based on distance data calibrated by the image acquisition device. Geometric calculations are then performed to determine the angle between the ship and the dock. Finally, the distance and angle are used to determine whether there is a risk of collision during berthing. This invention uses an image acquisition device as a sensor to perceive the relative position information of the ship and the dock, which is more economical and has better environmental adaptability compared to lidar solutions, while also being easier to install and implement. The berthing warning rules designed using parameters obtained from the image acquisition device are not limited by the type of ship or dock, making it suitable for all types of ships and docks.

[0090] Exemplary device

[0091] Figure 5 This is a schematic diagram of the structure of a ship berthing danger warning device 500 provided in an exemplary embodiment of the present invention. Figure 5 As shown, the device includes:

[0092] The image acquisition module is used to acquire video images of the ship during the berthing process through an image acquisition device, wherein the image acquisition device is installed on the side of the ship.

[0093] The extraction module is used to extract the ship's hull line and the dock shoreline from video images;

[0094] The determination module is used to determine the first distance between the bow of the ship and the dock, the second distance between the stern of the ship and the dock, and the true angle between the hull line and the dock shoreline based on the extracted hull line and dock shoreline.

[0095] The danger warning module is used to determine whether the vessel's berthing meets the danger warning conditions based on preset warning rules, first distance, second distance, and actual angle, and to issue an alarm message when the danger warning conditions are met.

[0096] Preferably, the image acquisition device includes at least two cameras, one mounted at the front of the ship's side and the other mounted at the rear of the ship's side; and

[0097] The image acquisition module is specifically used for:

[0098] Stitch together the video images captured by the two cameras;

[0099] The system uses a pre-defined neural network model to identify the stitched video images and extract the ship's hull line and the dock shoreline.

[0100] Preferably, the determining module is specifically used for:

[0101] Based on the extracted sideline and wharf shoreline, the distance between the bow of the ship and the first pixel point of the wharf, the distance between the stern of the ship and the second pixel point of the wharf, and the virtual angle between the sideline and the wharf shoreline in the video image are determined.

[0102] By using the extrinsic parameters of the image acquisition device, the geometric relationship between the pixel distance in the video image and the actual distance is determined;

[0103] Based on geometric relationships, the distance between the first and second pixels, and the virtual angle, the first distance between the bow of the ship and the dock, the second distance between the stern of the ship and the dock, and the actual angle between the hull line and the dock shoreline are determined.

[0104] Preferably, the preset warning rules include at least one of the following:

[0105] When the distance between the bow and stern of a vessel and the dock is less than a distance threshold, if the speed at which the bow and stern of the vessel approach the dock is greater than a speed threshold, the vessel's berthing will meet the danger warning conditions and an alarm message will be issued.

[0106] When the absolute value of the actual angle between the ship's side line and the quay line is greater than the angle threshold, the ship's berthing meets the danger warning conditions and an alarm message is issued.

[0107] Preferably, the ship berthing hazard warning method further includes setting a distance threshold, a speed threshold, and an angle threshold.

[0108] Preferably, the hazard warning module is specifically used for:

[0109] Determine whether the first distance and the second distance are less than the distance threshold;

[0110] When the first distance and the second distance are less than the distance threshold, the first speed of the ship's bow approaching the dock and the second speed of the ship's stern approaching the dock are calculated based on the first distance and the second distance.

[0111] Determine whether the first and second velocities are greater than the speed threshold;

[0112] If the first and second speeds are greater than the speed thresholds, the vessel is deemed to have met the danger warning conditions for berthing; otherwise, it is not.

[0113] Preferably, the hazard warning module is further configured to:

[0114] Determine whether the absolute value of the actual included angle is greater than the angle threshold;

[0115] When the absolute value of the actual included angle is greater than the angle threshold, the vessel is deemed to have met the danger warning conditions for berthing; otherwise, it is not.

[0116] The ship berthing danger warning device of the present invention corresponds to the ship berthing danger warning method of another embodiment of the present invention, and will not be described again here.

[0117] Exemplary electronic devices

[0118] Figure 6 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. For example... Figure 6 As shown, the electronic device 60 includes one or more processors 61 and a memory 62.

[0119] The processor 61 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0120] The memory 62 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 61 may execute the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above, and / or other desired functions. In one example, the electronic device may also include an input device 63 and an output device 64, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0121] In addition, the input device 63 may also include, for example, a keyboard, a mouse, etc.

[0122] The output device 64 can output various information to the outside. The output device 64 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0123] Of course, for the sake of simplicity, Figure 6 Only some of the components of the electronic device relevant to the present invention are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0124] Exemplary computer program products and computer-readable storage media

[0125] In addition to the methods and apparatus described above, embodiments of the present invention may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of the present invention described in the "Exemplary Methods" section above.

[0126] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0127] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of the present invention described in the "Exemplary Methods" section above.

[0128] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0129] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0131] The block diagrams of devices, systems, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, systems, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0132] The methods and systems of the present invention may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of the present invention are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present invention may also be implemented as a program recorded on a recording medium, the program comprising machine-readable instructions for implementing the methods according to the present invention. Thus, the present invention also covers recording media storing programs for performing the methods according to the present invention.

[0133] It should also be noted that in the systems, apparatus, and methods of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered equivalents of the present invention. The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0134] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for early warning of ship berthing danger, characterized in that, include: Video images of the ship during the berthing process are captured by an image acquisition device installed on the side of the ship. The image acquisition device includes at least two cameras, one mounted at the front of the ship's side and the other mounted at the rear of the ship's side; and Extract the ship's hull line and the dock shoreline from video images, including: Stitch together the video images captured by the two cameras; The system uses a pre-defined neural network model to identify the stitched video images and extract the ship's hull line and the dock shoreline. Extract the ship's hull line and the dock shoreline from video images; Based on the extracted hull line and wharf shoreline, the first distance between the bow of the ship and the wharf, the second distance between the stern of the ship and the wharf, and the true angle between the hull line and the wharf shoreline are determined. The determination of the first distance between the ship's bow and the dock, the second distance between the ship's stern and the dock, and the true angle between the ship's bow and the dock line, based on the extracted hull line and dock shoreline, includes: Based on the extracted sideline and wharf shoreline, the distance between the bow of the ship and the first pixel point of the wharf, the distance between the stern of the ship and the second pixel point of the wharf, and the virtual angle between the sideline and the wharf shoreline in the video image are determined. By using the extrinsic parameters of the image acquisition device, the geometric relationship between the pixel distance in the video image and the actual distance is determined; Based on geometric relationships, the distance between the first and second pixels, and the virtual angle, the first distance between the bow of the ship and the dock, the second distance between the stern of the ship and the dock, and the actual angle between the hull line and the dock shoreline are determined. Based on preset warning rules, first distance, second distance and actual angle, determine whether the berthing of the ship meets the danger warning conditions, and issue an alarm message when the danger warning conditions are met; The preset warning rules include at least one of the following: When the distance between the bow and stern of a vessel and the dock is less than a distance threshold, if the speed at which the bow and stern of the vessel approach the dock is greater than a speed threshold, the vessel's berthing will meet the danger warning conditions and an alarm message will be issued. When the absolute value of the actual angle between the ship's side line and the quay line is greater than the angle threshold, the ship's berthing meets the danger warning conditions and an alarm message is issued. It also includes: setting distance thresholds, speed thresholds, and angle thresholds; The determination of whether a vessel's berthing meets the danger warning conditions, based on preset warning rules, a first distance, a second distance, and the actual angle, includes: Determine whether the first distance and the second distance are less than the distance threshold; When the first distance and the second distance are less than the distance threshold, the first speed of the ship's bow approaching the dock and the second speed of the ship's stern approaching the dock are calculated based on the first distance and the second distance. Determine whether the first and second velocities are greater than the speed threshold; If the first and second speeds are greater than the speed thresholds, it is determined that the vessel's berthing has met the danger warning conditions; otherwise, it is not. The determination of whether a vessel's berthing meets the danger warning conditions, based on preset warning rules, a first distance, a second distance, and the actual angle, includes: Determine whether the absolute value of the actual included angle is greater than the angle threshold; When the absolute value of the actual included angle is greater than the angle threshold, the vessel is deemed to have met the danger warning conditions for berthing; otherwise, it is not.

2. A ship berthing danger early warning device, characterized in that, include: The image acquisition module is used to acquire video images of the ship during the berthing process through an image acquisition device, wherein the image acquisition device is installed on the side of the ship. The extraction module is used to extract the ship's hull line and the dock shoreline from video images; The determination module is used to determine the first distance between the bow of the ship and the dock, the second distance between the stern of the ship and the dock, and the true angle between the hull line and the dock shoreline based on the extracted hull line and dock shoreline. The danger warning module is used to determine whether the vessel's berthing meets the danger warning conditions based on preset warning rules, first distance, second distance, and actual angle, and to issue an alarm message when the danger warning conditions are met.

3. A computer-readable storage medium storing a computer program for performing the method of claim 1.

4. An electronic device, characterized in that, The electronic device includes: processor; A memory for storing executable instructions of the processor; the processor for reading the executable instructions from the memory and executing the executable instructions to implement the method of claim 1.

Images