Navigation Obstacle Lookout System and Method for Ships, and Electronic Equipment

Abstract

The present invention relates to the field of ships, and provides an obstruction lookout system and method for ships, as well as an electronic device. The system includes an on-board system and a shore-based system. The on-board system includes: an information acquisition unit for acquiring the environmental information of the ship; an obstruction recognition unit for recognizing the environmental information based on an obstruction recognition model and obtaining a recognition result, where the obstruction recognition model is pre-trained by the shore-based system; and a warning unit for giving a warning based on the recognition result. The shore-based system is used to acquire the environmental information of the ship and train and update the obstruction recognition model based on the environmental information. It is used to solve the defect that the obstruction recognition method in the related art cannot accurately recognize the obstructions on the sea surface and the safety of ship navigation cannot be guaranteed. In the solution of the present application, the obstructions on the sea surface can be recognized based on the pre-trained recognition model, which can effectively improve the safety of ship navigation.

Description

Technical Field

[0001] The present invention relates to the technical field of ships, and particularly relates to an obstacle lookout system and method for ships, and an electronic device. Background Art

[0002] With the rapid development of offshore aquaculture, marine ranching, offshore wind power, and offshore leisure tourism, the number of lost fishing nets, navigational floating obstacles, small boats, etc. is increasing continuously, which not only poses a huge threat to the navigation safety of ships, but also brings great challenges to the governance of navigation order.

[0003] In the current related technologies, navigation ships are all equipped with active detection devices such as radars. However, due to the very small electromagnetic reflection cross-sections of navigational floating obstacles, small boats, etc., it is difficult for radars to detect them. Even if the ship's crew strengthens the lookout, due to the limitation of the human eye's visual range, they often find that they are too close when they discover, and it is difficult to operate the ship to avoid, thus resulting in safety accidents. Moreover, some difficult-to-detect navigational obstacles are more likely to cause major accidents. For example, floating fishing nets on the sea are extremely difficult to detect. Once the ship's propeller is entangled by the fishing net, it is not only difficult to clean, but in severe cases, the ship will lose power, and it is extremely easy to cause major maritime disasters.

[0004] In the related technologies, there are also methods of applying neural networks to identify navigational obstacles. However, such algorithms in the related technologies cannot identify navigational obstacles on the sea in real time, and the navigation safety of ships cannot be guaranteed either. Summary of the Invention

[0005] The present invention provides an obstacle lookout system and method for ships, and an electronic device, which are used to solve the defect that the obstacle identification methods in the related technologies cannot accurately identify navigational obstacles on the sea and the navigation safety of ships cannot be guaranteed. In the solution of this application, the navigational obstacles on the sea can be identified based on a pre-trained identification model. In this way, the ship itself can identify navigational obstacles based on the identification model. Compared with marine radars and human lookouts, the real-time performance of obstacle identification can be improved, and the identification model can also be continuously updated during the use process to continuously improve the identification accuracy, which can effectively improve the navigation safety of ships.

[0006] The present invention provides an obstacle lookout system for ships, which includes an on-ship system arranged on the ship and a shore-based system arranged on the shore. The on-ship system and the shore-based system are wirelessly connected;

[0007] The on-ship system includes:

[0008] An information acquisition unit, which is used to acquire the environmental information of the ship, and the environmental information represents the surrounding environment in the forward direction of the ship;

[0009] An obstacle identification unit, configured to identify the environmental information based on an obstacle identification model, and obtain an identification result, where the obstacle identification model is pre-trained by the shore-based system;

[0010] An early warning unit, configured to give an early warning based on the identification result;

[0011] The shore-based system is configured to obtain the environmental information of a ship, and train and update the obstacle identification model based on the environmental information.

[0012] According to the obstacle lookout system for ships provided by the present invention, the shore-based system is communicatively connected to the on-board system, and the on-board system is arranged in the corresponding ship;

[0013] The on-board system is configured to send the environmental information of the corresponding ship to the shore-based system, and the shore-based system is configured to train an obstacle identification standard model based on the environmental information of the corresponding ship obtained by the on-board system, where the obstacle identification standard model is stored in the shore-based system;

[0014] The shore-based system is further configured to update the obstacle identification model of the on-board system based on the trained obstacle identification standard model.

[0015] According to the obstacle lookout system for ships provided by the present invention, the shore-based system updates the obstacle identification model of the on-board system based on the trained obstacle identification standard model according to a preset update rule.

[0016] According to the obstacle lookout system for ships provided by the present invention, the shore-based system is further configured to send a remote control instruction to the on-board system, where the remote control instruction includes obtaining the control right of the camera of the on-board system to obtain video information of the sea area where the ship is located.

[0017] According to the obstacle lookout system for ships provided by the present invention, the on-board system further includes a first communication management unit, and the first communication management unit is configured to automatically select a communication method based on the status information of the ship, where the status information of the ship includes the signal strength of 700M 5G and 5G communication networks, and the communication methods include 700M 5G, 5G communication, and satellite communication.

[0018] According to the obstacle lookout system for ships provided by the present invention, the shore-based system includes a second communication management unit, and the second communication management unit is configured to determine the availability of a communication link based on the communication importance level and communication cost economy, where the communication link includes 700M 5G, 5G communication networks, and satellite communication.

[0019] According to the navigation obstacle lookout system for ships provided by the present invention, the early warning unit is further configured to divide the recognition result into early warning levels, and determine the notification method and the notified personnel of the early warning information based on a preset early warning plan and the early warning level. The notification methods include the system client, WeChat applet, and text message.

[0020] According to the navigation obstacle lookout system for ships provided by the present invention, the environmental information of the ship includes image information;

[0021] The navigation obstacle recognition unit is further configured to preprocess the acquired image information, and the preprocessing is used to improve the clarity of the image information.

[0022] The preprocessing includes feature extraction and background noise elimination of the image information.

[0023] The present invention also provides a navigation obstacle lookout method for ships, which is executed by an on-board system and includes:

[0024] Obtain the environmental information of the ship, where the environmental information represents the surrounding environment in the forward direction of the ship;

[0025] Identify the environmental information based on a navigation obstacle recognition model and obtain a recognition result. The navigation obstacle recognition model is pre-trained, and the recognition result represents the navigation obstacles around the waterway in the forward direction of the ship;

[0026] Give an early warning based on the recognition result.

[0027] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements any one of the above-mentioned navigation obstacle lookout methods for ships.

[0028] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the above-mentioned navigation obstacle lookout methods for ships.

[0029] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements any one of the above-mentioned navigation obstacle lookout methods for ships.

[0030] The obstacle lookout system for ships provided in this application includes a shipborne system respectively installed on the ship and a shore-based system installed on the shore. The shipborne system is equipped with an obstacle recognition model. In this way, the ship itself can identify obstacles based on the recognition model, which can improve the real-time performance of obstacle recognition compared with marine radars and human lookouts. The obstacle recognition model can identify obstacles in real time based on the surrounding environment information in the forward direction. If an obstacle is recognized, timely warnings can be issued to improve the safety of ship navigation. At the same time, the shore-based system can also obtain the environmental information of the ship, and the shore-based system obtains the environmental information of the ship for continuously training and updating the obstacle recognition model. In this way, as the obstacle lookout system is used, the accuracy of the obstacle recognition model will become higher and higher, further improving the safety of ship navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0032] Figure 1 is a schematic structural diagram of the obstacle lookout system provided by an embodiment of the present invention;

[0033] Figure 2 is a schematic flowchart of one of the obstacle lookout methods provided by an embodiment of the present invention;

[0034] Figure 3 is a schematic flowchart of another obstacle lookout method provided by an embodiment of the present invention;

[0035] Figure 4 is a schematic physical structure diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0037] Figure 1 is a schematic structural diagram of the obstacle lookout system provided by an embodiment of the present invention.

[0038] Such as Figure 1As shown in the figure, this embodiment provides an obstacle lookout system for ships, which includes an on-board system installed on the ship and a shore-based system installed on the shore. The on-board system and the shore-based system are wirelessly connected;

[0039] The on-board system includes:

[0040] An information acquisition unit for acquiring the environmental information of the ship, where the environmental information characterizes the surrounding environment in the forward direction of the ship;

[0041] An obstacle recognition unit for recognizing the environmental information based on an obstacle recognition model and obtaining a recognition result. The obstacle recognition model is pre-trained by the shore-based system;

[0042] An early warning unit for giving an early warning based on the recognition result;

[0043] The shore-based system is used to acquire the environmental information of the ship and train and update the obstacle recognition model based on the environmental information.

[0044] In practical applications, the shore-based system is equivalent to a fixed server, and the on-board system is equivalent to a mobile terminal. The shore-based system acting as a server can be connected to multiple on-board systems at the same time.

[0045] In practical applications, after the obstacle recognition unit recognizes the obstacles around the ship, it can automatically load the obstacle markings in the acquired environmental data and generate corresponding alarm information to provide data services for the alarm management and visualization sharing management units. After that, the early warning unit can apply target tracking algorithms to highlight the marked obstacles on the display end to improve the identification efficiency of the crew and passengers and timely avoid navigation risks. The target tracking algorithms can include existing tracking algorithms such as the TLD (Tracking-Learning-Detection) algorithm and the optical flow algorithm.

[0046] In implementation, the obstacle identification model in the obstacle identification unit of the on-board system can be pre-trained by the shore-based system and transmitted to the on-board system. At the same time, the shore-based system stores the obstacle identification standard model, that is, a backup of the obstacle identification model. While the on-board system performs obstacle identification based on the obstacle identification model, the shore-based system can also synchronously train and optimize the obstacle identification standard model. In practical applications, the shore-based system can use the YOLO detection algorithm and the SSD detection algorithm as the basis of the model, and use the images or video materials of obstacles under the actual sea conditions as the training set. Through a large number of refined classification and localization trainings, multi-scale feature maps of obstacles such as fishing net floats, harmful floating objects, and ships under different sea conditions are constructed, and finally the maritime obstacle identification standard model is trained. Among them, when training for the first time, the data of the training set can be obtained from test ships and the Internet. After that, during the use process, the obstacle identification standard model can be further deeply trained based on the environmental information during the actual navigation of the ship. In addition, the refined classification and localization training of the training set means distinguishing the obstacles in the training set by type, such as classifying them into reef types, ship types, and harmful floating object types, etc. After that, the obstacle identification model can be trained separately according to different categories. In this way, the recognition accuracy of the trained obstacle identification model for each type of obstacle can be improved, and the actually detected obstacles can be accurately classified into the corresponding categories, which is convenient for evaluating the safety level of the obstacle.

[0047] In this embodiment, the on-board system can independently complete all functions of obstacle lookout, including functions such as data acquisition, obstacle identification, and early warning. The obstacles here can include any object on the sea surface that hinders the current ship's navigation, including naturally existing reefs, other ships, fishing nets on the sea surface, and other garbage that may affect the ship's navigation. In practical applications, the on-board system can identify almost all obstacles on the sea surface based on the obstacle identification model. Of course, there are also different levels among the obstacles, and different obstacles may have different impacts on the ship's navigation. On this basis, when the early warning unit of the on-board system issues an early warning based on the recognition result, it can also divide different safety levels based on the different obstacles and issue different forms of early warnings according to different safety levels. For example, if the obstacle identification unit identifies that there are reefs or other ships in front of the ship's progress, the highest-level early warning can be issued. If the obstacle identification unit identifies that there are obstacles that do not endanger the navigation safety in front of the ship's progress, a low-level early warning can be issued.

[0048] In practical applications, the shore-based system can obtain the environmental information around the ship through the on-board system. Specifically, after the on-board system obtains the environmental information of the ship through the information acquisition unit, it can upload it to the shore-based system in real time, or the shore-based system can send an instruction to the on-board system to obtain the environmental information for a specified period or in real time. The environmental information of the ship can include images captured by the ship's cameras and videos taken, etc.

[0049] The obstacle detection system for ships provided in this embodiment includes an on-board system respectively arranged on the ship and a shore-based system arranged on the shore. Among them, the on-board system is equipped with an obstacle detection model. In this way, the ship itself can perform obstacle detection based on the detection model, which can improve the real-time performance of obstacle detection compared with transmitting data to the server for remote decision-making. The obstacle detection model can perform real-time detection of obstacles in the ship's surrounding environment based on the acquired environmental information. If an obstacle is detected, timely warnings can be issued to improve the safety of ship navigation. At the same time, the shore-based system can also obtain the environmental information of the ship. The shore-based system obtains the environmental information of the ship for continuously training and updating the obstacle detection model. In this way, as the obstacle detection system is used, the accuracy of the obstacle detection model will become higher and higher, further improving the safety of ship navigation.

[0050] In an exemplary embodiment, the shore-based system is communicatively connected to the on-board system, and the on-board system is arranged in the corresponding ship;

[0051] The on-board system is used to send the environmental information of the corresponding ship to the shore-based system. The shore-based system is used to train the obstacle detection standard model based on the environmental information of the corresponding ship obtained by the on-board system. The obstacle detection standard model is stored in the shore-based system;

[0052] The shore-based system is further used to update the obstacle detection model of the on-board system based on the trained obstacle detection standard model.

[0053] In implementation, as described in the above embodiments, after the shore-based system obtains the navigational obstacle recognition model through the first training, it sends the navigational obstacle recognition model to the on-board system for actual use. The shore-based system itself retains a backup of the navigational obstacle recognition model, that is, the standard navigational obstacle recognition model. In actual application, the shore-based system can be connected to multiple on-board systems. Along with the actual working process of the on-board system on the ship, each ship equipped with the on-board system can obtain a large amount of training set data containing navigational obstacles on the sea surface. At the same time, all the training set data obtained by the ships can be sent to the shore-based system. Based on this continuous and huge training set, the shore-based system continuously trains and updates the standard navigational obstacle recognition model. After the shore-based system trains and updates the standard navigational obstacle recognition model, it can also update the navigational obstacle recognition models in the on-board systems of all the connected ships based on the updated standard navigational obstacle recognition model. That is to say, the update of the navigational obstacle recognition model in the on-board system is not linear but step-shaped, and it is updated step by step, providing technical support for improving the obstacle recognition efficiency and accuracy of the on-board AI video navigational obstacle lookout system.

[0054] In this embodiment, the shore-based system can uniformly train the navigational obstacle recognition model based on the training set data collected from all the ships within the connected system. In this way, compared with training each navigational obstacle recognition model separately, the training set data is more sufficient, and the navigational obstacle recognition model obtained by this training method is consistent in each on-board model. A unified navigational obstacle recognition model can adapt to various working conditions and has sufficient accurate recognition ability for various different navigational obstacles.

[0055] In an exemplary embodiment, the shore-based system updates the navigational obstacle recognition model of the on-board system based on the trained standard navigational obstacle recognition model according to a preset update rule.

[0056] In actual application, the preset update rule can be to update the navigational obstacle recognition model of the on-board system every set time. For example, the set time can be 10 days. In implementation, the shore-based system can also set the training stage, that is, set the training goal. When the standard navigational obstacle recognition model in the shore-based system reaches each training stage, or completes each training goal, the navigational obstacle recognition model of the on-board system is updated.

[0057] In an exemplary embodiment, the shore-based system is further configured to send a remote control instruction to the on-board system, and the remote control instruction includes obtaining the control right of the camera of the on-board system to obtain video information of the sea area where the ship is located.

[0058] In practical applications, the shore-based system includes a data sharing unit, which is used to build a shore-based maritime video data sharing platform. Using a broadband Internet connection, according to a preset maritime video and alarm information breakdown plan, it provides technical support for relevant agencies to immediately receive obstruction navigation avoidance alarm information and view real-time and historical maritime video data.

[0059] Specifically, although the on-board system is in a wireless connection state with the shore-based system, the on-board system has a very large degree of autonomy. During normal ship operation, almost all obstruction navigation objects encountered can be independently identified and warned by the on-board system. However, it should be noted that when the ship discovers special situations such as shipwrecks or other suspicious objects during navigation, it can notify the shore-based system, and the shore-based system issues a remote control command to the on-board system to obtain the control right of the camera, extending the visual range of the shore-based system and providing intuitive visual information support for shipwreck rescue and suspicious object identification.

[0060] In addition, when the shore-based system completes the training of the obstruction navigation object recognition standard model and hopes to update the obstruction navigation object recognition model of the on-board system based on the trained obstruction navigation object recognition standard model, it can also issue a remote control command to update the system to the on-board system of each ship in the system. After receiving this remote control command, the on-board system can receive the updated obstruction navigation object recognition standard model from the shore-based system and update the obstruction navigation object recognition model it loads.

[0061] In an exemplary embodiment, the on-board system further includes a first communication management unit, which is used to automatically select a communication method based on the status information of the ship. The status information of the ship includes the signal strength of 700M 5G and 5G communications, and the communication methods include 700M 5G, 5G communication links, and satellite communication links.

[0062] In an exemplary embodiment, the shore-based system includes a second communication management unit, which is used to determine the availability of communication links based on the communication importance level and communication cost economy. The communication links include 700M 5G, 5G communication networks, and satellite communication.

[0063] In practical applications, communication can be carried out between the shore-based system and the on-board system through various communication methods. In implementation, the shore-based system can communicate with the on-board system through a data communication unit. The data communication unit can be combined with multiple communication modules such as 700M 5G, 5G, satellite, and Beidou satellite short message. That is to say, the shore-based system can include various communication methods such as 700M 5G, 5G, satellite communication, and Beidou satellite short message. In practical applications, the most suitable communication method can be flexibly selected based on the differences in the on-board system. Specifically, in this embodiment, before the shore-based system communicates with a certain on-board system, it can first determine the status information of the ship, specifically including data such as the configuration, working status, and communication conditions of the communication equipment installed on the ship, and comprehensively determine the optimal communication method, communication link, and communication time, etc. Among them, the criteria for selecting the communication method can include selecting the method with the lowest economic cost, selecting the method with the highest communication security, selecting the method with the fastest information transmission, etc. Specifically, the communication method can be selected according to actual needs. For example, when the obstacle identification model needs to be updated, since the data communication capacity of the obstacle identification model is relatively high, 700M 5G or 5G broadband communication methods can be selected. When encountering emergencies such as shipwrecks or suspicious objects, satellite communication or Beidou short message methods can be selected to establish a communication connection with the shore-based system in the fastest information transmission method without being restricted by the region.

[0064] In this embodiment, through the communication management unit of the on-board system and the communication management unit of the shore-based system, the obstacle lookout system forms a configurable, multi-channel, and intelligent ship-shore communication system, providing technical support for efficient and economic information interaction between the ship and the shore in the whole time domain, and realizing the efficient and economic interconnection of ship-shore video and alarm information.

[0065] In an exemplary embodiment, the warning unit is further configured to classify the recognition result into warning levels, and determine the notification method and notification personnel of the warning information based on a preset warning plan and the warning level. The notification methods include system client, WeChat mini-program, and short message.

[0066] In practical applications, after the on-board system identifies obstacles in the ship's surrounding environment based on the obstacle identification unit, corresponding warnings can be issued based on a pre-set notification management plan. Specifically, different responsible persons on the ship can be warned according to different recognition results. The warning methods can include providing a man-machine interaction window for relevant responsible persons to receive and view obstacle avoidance alarms, shore-based remote control and remote signaling notifications, etc. in real time. The relevant responsible persons can obtain the warning information of the obstacles based on this man-machine interaction window, and corresponding processing methods can also be provided on this man-machine interaction window. The relevant responsible persons can make corresponding treatments based on this.

[0067] In an exemplary embodiment, the environmental information of the ship includes image information;

[0068] The obstacle identification unit is further configured to preprocess the acquired image information, and the preprocessing is used to improve the clarity of the image information.

[0069] The preprocessing includes feature extraction and background noise elimination of the image information.

[0070] Specifically, an adaptive noise cancellation algorithm can be used to eliminate ocean background noise. The adaptive noise cancellation algorithm can use existing technologies, such as the LMS algorithm or the RLS algorithm, etc. This embodiment does not limit this.

[0071] After the preprocessing is completed, combined data such as video frames, camera longitude and latitude coordinates, compass, illuminance, etc. can be formed and stored based on time. These combined data can be stored in the on-board system and the shore-based system to provide data services for the high-quality display of video information by the shipborne AI video obstacle lookout system.

[0072] Next, the obstacle lookout method provided by the present invention for ships will be described. The obstacle lookout method for ships described below can be mutually corresponding and referred to the obstacle lookout system for ships described above.

[0073] Figure 2 is one of the flow diagrams of the obstacle lookout method provided by the embodiment of the present invention.

[0074] As Figure 2 shown, the obstacle lookout method for ships provided by this embodiment is executed by the on-board system and includes:

[0075] Step 201, obtain the environmental information of the ship, where the environmental information characterizes the surrounding environment in the forward direction of the ship;

[0076] Step 202, identify the environmental information based on the obstacle identification model and obtain an identification result. The obstacle identification model is pre-trained, and the identification result characterizes the obstacles around the ship;

[0077] Step 203, issue a warning based on the identification result.

[0078] Figure 3 is another flow diagram of the obstacle lookout method provided by the embodiment of the present invention.

[0079] Next, the obstacle lookout method provided by this application will be described in conjunction with Figure 3 to illustrate.

[0080] First, environmental information around the ship's navigation direction can be collected based on devices such as high-definition over-the-horizon cameras and environmental sensors installed on the ship. The on-board system integrates and caches the collected environmental information, and processes the ocean background noise. Then, based on the obstacle identification model, obstacles in the data can be identified, and an alarm can be issued based on the identification result. At the same time, the alarm data can also be stored for future use.

[0081] The specific implementation method of the obstacle lookout method provided in this embodiment can be implemented with reference to the above embodiment, and will not be elaborated here.

[0082] In summary, the obstacle lookout system for ships provided by this application has the following beneficial effects:

[0083] (1) The shore-based system accumulates various weather and sea conditions training materials for the in-depth training of the standard model for identifying obstacles at sea by collecting and storing a large number of videos and obstacle alarm data uploaded by the on-board AI video obstacle lookout system. Applying the YOLO detection algorithm and the SSD detection algorithm, based on the obstacle video materials under the actual sea conditions accumulated, through a large number of refined classification and positioning trainings, multi-scale feature maps of obstacles such as fishing net floats, harmful floating objects, and small ships are constructed, forming a standard model database for identifying obstacles at sea, and providing an updated service for the standard model for identifying obstacles at sea for the on-board AI video obstacle lookout system, maximizing the efficiency and accuracy of obstacle identification;

[0084] (2) The on-board system uses a high-definition, high-over-the-horizon video camera system and AI video recognition technology to efficiently discover and identify obstacles while extending the lookout horizon of the ship's personnel, making up for the detection blind area of small obstacles by the on-board radar, improving the lookout horizon and efficiency of the personnel, and providing technical support for promptly prompting the crew to identify potential navigation safety risks and taking scientific ship operation measures in a timely manner to avoid marine accidents.

[0085] (3) The on-board system can also receive remote control and telemetry commands from the shore-based marine AI video obstacle lookout system, connect and share real-time and historical video data and obstacle alarm data at sea in a manageable state, use the ship in navigation as a vehicle for the high-definition camera system, make up for the lack of visualization range in the deep sea and far sea, and provide information support for improving the governance of sea area navigation order, maritime search and rescue, sea area environmental protection, and sea area national defense security control capabilities.

[0086] Figure 4 Illustrates a schematic physical structure diagram of an electronic device, such as Figure 4As shown in the figure, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logical instructions in the memory 430 to execute the method for observing navigation obstacles applied to ships, and this method includes:

[0087] Obtain the environmental information of the ship, where the environmental information characterizes the surrounding environment of the ship;

[0088] Identify the environmental information based on the navigation obstacle recognition model and obtain the recognition result, where the navigation obstacle recognition model is pre-trained;

[0089] Give an early warning based on the recognition result.

[0090] In addition, when the logical instructions in the above-mentioned memory 430 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0091] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for observing navigation obstacles applied to ships provided by the above-mentioned various methods, and this method includes:

[0092] Obtain the environmental information of the ship, where the environmental information characterizes the surrounding environment in the forward direction of the ship;

[0093] Identify the environmental information based on the navigation obstacle recognition model and obtain the recognition result, where the navigation obstacle recognition model is pre-trained;

[0094] Give an early warning based on the recognition result.

[0095] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is used to execute the method for observing navigation obstacles applied to a ship provided by the above various methods. The method includes:

[0096] Obtain the environmental information of the ship, where the environmental information characterizes the surrounding environment in the forward direction of the ship;

[0097] Identify the environmental information based on an obstacle recognition model, and obtain an identification result. The obstacle recognition model is pre-trained;

[0098] Give an early warning based on the identification result.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0100] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0101] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. An obstruction lookout system applied to ships, characterized in that, It includes an on-board system installed on a ship and a shore-based system installed on shore, and the on-board system and the shore-based system are wirelessly connected; The on-board system includes: An information acquisition unit for acquiring the environmental information of the ship, where the environmental information characterizes the surrounding environment in the forward direction of the ship; An obstacle identification unit for identifying the environmental information based on an obstacle identification model and obtaining an identification result, where the obstacle identification model is pre-trained by the shore-based system; An early warning unit for giving an early warning based on the identification result; The shore-based system is used to acquire the environmental information of the ship and train and update the obstacle identification model based on the environmental information; The shore-based system is communicatively connected to the on-board system, and the on-board system is installed in the corresponding ship; The on-board system is used to send the environmental information of the corresponding ship to the shore-based system, and the shore-based system is used to train an obstacle identification standard model based on the environmental information of the corresponding ship acquired by the on-board system, and the obstacle identification standard model is stored in the shore-based system; The shore-based system is further used to update the obstacle identification model of the on-board system based on the trained obstacle identification standard model according to a preset update rule; The preset update rule includes updating the obstacle identification model of the on-board system at regular intervals; The shore-based system is connected to multiple on-board systems. During the actual working process of the on-board systems in the ships, each ship equipped with an on-board system obtains a large amount of training set data containing obstacles on the sea surface. At the same time, the training set data obtained by all ships is sent to the shore-based system, and the shore-based system then continuously trains and updates the obstacle identification standard model based on the training set. The shore-based system trains and updates the obstacle identification standard model and also updates the obstacle identification models in the on-board systems of all ships connected to the shore-based system based on the updated obstacle identification standard model.

2. The navigation obstruction lookout system for ships according to claim 1, wherein The shore-based system is further used to send a remote control instruction to the on-board system, and the remote control instruction includes obtaining the control right of the camera of the on-board system to obtain video information of the sea area where the ship is located.

3. The navigation obstacle lookout system for ships according to claim 1, wherein The on-board system further includes a first communication management unit, and the first communication management unit is used to automatically select a communication method based on the status information of the ship, where the status information of the ship includes the signal strength of the 5G communication network, and the communication methods include 5G communication and satellite communication.

4. The navigation obstacle lookout system for ships according to claim 1, wherein The shore-based system includes a second communication management unit, and the second communication management unit is used to determine the availability of a communication link based on the communication importance level and communication cost economy, and the communication link includes a 5G communication network and satellite communication.

5. The navigation hazard lookout system for ships according to claim 1, characterized in that, The early warning unit is further used to divide the warning level for the identification result and determine the notification method and notification personnel of the early warning information based on a preset early warning plan and the warning level, and the notification methods include a system client, a WeChat mini-program, and a text message.

6. The navigation obstacle lookout system for ships according to claim 1, characterized in that, The environmental information of the ship includes image information; The navigation obstacle recognition unit is further configured to preprocess the acquired image information, and the preprocessing is used to improve the clarity of the image information; The preprocessing includes feature extraction and background noise elimination of the image information.

7. A method for observing navigation hazards applied to a ship, which is executed by an on-board system of the navigation hazard observation system applied to a ship according to any one of claims 1-6, characterized in that, It includes: Obtain the environmental information of the ship, and the environmental information characterizes the surrounding environment in the forward direction of the ship; Identify the environmental information based on the navigation obstacle recognition model, and obtain an identification result. The navigation obstacle recognition model is pre-trained, and the identification result characterizes the navigation obstacles around the waterway in the forward direction of the ship; Give an early warning based on the identification result.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the navigation obstacle lookout method for ships as described in claim 7.

Images