Display method, control method and apparatus for movable platform, and movable platform

Abstract

A display method, a control method and apparatus for a movable platform, and a movable platform. The display method is applied to a display device, and the display device is communicatively connected to a movable platform. The display method comprises: acquiring a captured image from a movable platform, wherein the captured image comprises a target vessel; and displaying the identity information of the target vessel in the captured image, wherein the identity information of the target vessel is determined on the basis of first position-related information of the target vessel and second position-related information that is determined by a vessel management system, and the first position-related information is determined on the basis of the position of the movable platform. The technical solution provided by the present embodiments effectively allows the display device to quickly acquire the captured image comprising the target vessel, thereby ensuring, to a certain extent, the accuracy of vessel information displayed on a nautical chart; and a user can quickly and visually view the identity information of the target vessel by means of the captured image, thereby being conducive to improving the timeliness and reliability of performing maritime management on the basis of the identity information of the target vessel.

Description

Display method, control method and device for a movable platform, and movable platform Technical Field

[0001] The embodiments of the present invention relate to the field of mobile platform technology, and in particular to a display method, a control method for a mobile platform, an apparatus, and a mobile platform. Background Technology

[0002] Traditional maritime management methods often rely on Automatic Identification System (AIS). AIS receivers can be deployed on land or on patrol vessels. The AIS information corresponding to the vessels can then be displayed on marine electronic maps (hereinafter referred to as nautical charts) to enable waterway vessel and waterway management operations based on AIS information.

[0003] However, the aforementioned method of displaying ship information through nautical charts is not intuitive enough for users, as they cannot observe the actual environment in which the ship is located. Therefore, when using nautical charts for maritime management, the timeliness and reliability of the management of ships and waterways in the waters cannot be guaranteed. Summary of the Invention

[0004] This invention provides a display method, a control method for a movable platform, an apparatus, and a movable platform, which can, to a certain extent, ensure the accuracy of ship information displayed on nautical charts and help ensure the timeliness and reliability of maritime management operations.

[0005] A first aspect of the present invention is to provide a display method applied to a display device, the display device being communicatively connected to a mobile platform; the method comprising:

[0006] The mobile platform captures images of the target vessel.

[0007] The target vessel's identity information is displayed in the captured image.

[0008] The identity information of the target vessel is determined based on the first location-related information of the target vessel and the second location-related information determined by the vessel management system. The first location-related information is determined based on the location of the mobile platform itself.

[0009] A second aspect of the present invention is to provide a control method for a mobile platform, the mobile platform being communicatively connected to a display device, the method comprising:

[0010] The mobile platform captures images of the target vessel.

[0011] Obtain first location-related information of the target vessel, wherein the first location-related information is determined based on the location of the mobile platform itself;

[0012] Obtain information related to the second location determined by the ship management system;

[0013] The captured image is transmitted to a display device so that the display device can display the identity information of the target vessel in the captured image, wherein the identity information of the target vessel is determined based on the first location-related information and the second location-related information.

[0014] A third aspect of the present invention is to provide a display device communicatively connected to a mobile platform; the display device includes: a memory and a processor;

[0015] The memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the display method described in the first aspect above.

[0016] A fourth aspect of the present invention is to provide a control device for a mobile platform, the mobile platform being communicatively connected to a display device, the control device comprising: a memory and a processor;

[0017] The memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the control method of the mobile platform described in the second aspect above.

[0018] A fifth aspect of the present invention is to provide a portable platform communicatively connected to a display device; the portable platform comprising:

[0019] The control device for the movable platform described in the fourth aspect.

[0020] A sixth aspect of the present invention is to provide a computer-readable storage medium storing program instructions for use in any one of the first to second aspects described above.

[0021] The display method, mobile platform control method, device, and mobile platform provided in this invention effectively enable the display device to quickly acquire images of the target vessel via the mobile platform. This allows users to quickly and intuitively view the target vessel's identity information through the captured images, and then perform ship management or maritime management operations based on the target vessel's identity information. This improves the timeliness and reliability of maritime management operations and further ensures the practicality of the method. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0023] Figure 1 is a flowchart illustrating a display method provided in an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of a display method provided in an embodiment of the present invention;

[0025] Figure 2a is a schematic diagram of determining the first position-related information of the target ship according to an embodiment of the present invention;

[0026] Figure 2b is a schematic diagram of determining the first position-related information of the target ship according to an embodiment of the present invention;

[0027] Figure 3 is a flowchart illustrating another display method provided in an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of the process of displaying the identity information of the target vessel in the captured image according to an embodiment of the present invention;

[0029] Figure 5 is a flowchart illustrating a ship management method provided in an application embodiment of the present invention;

[0030] Figure 6 is a flowchart illustrating a control method for a mobile platform provided in an application embodiment of the present invention;

[0031] Figure 7 is a schematic diagram of a display device provided in an embodiment of the present invention;

[0032] Figure 8 is a schematic diagram of the structure of a control device for a mobile platform provided in an embodiment of the present invention;

[0033] Figure 9 is a structural schematic diagram of a mobile platform provided in an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0036] To facilitate understanding of the specific implementation process of the technical solution in this embodiment, the relevant technologies are briefly described below:

[0037] The Automatic Identification System (AIS) is a maritime navigation system adopted by the International Association of Lighthouse Authorities (IALA), the International Telecommunication Union (ITU), and the International Electrotechnical Commission (IEC) for maritime safety. It is a crucial means of ensuring safe navigation at sea. Since the 1990s, AIS technology has introduced digital radio communication into ship navigation, ushering in the era of intelligent and digital ship navigation. The main function of AIS is to exchange short messages between ships, coastal radio stations, and navigation equipment. These messages include static information (ship identification code, ship type, port of destination, etc.), dynamic information (position, heading, speed, navigation status, etc.), and safety-related information, and display ship name, speed, heading, and route information on an electronic map in real time. The application of this system has greatly improved maritime communication capabilities, effectively enhancing the smoothness and safety of ship navigation.

[0038] Traditional maritime management methods often rely on land-based AIS receivers or AIS systems installed on patrol vessels. Specifically, the AIS system can acquire the AIS reception information of ships, which can then be displayed on a marine electronic map (hereinafter referred to as a nautical chart). Based on the AIS reception information displayed on the nautical chart, waterway and vessel management operations can be carried out.

[0039] However, since the area of ​​water that an AIS system can cover is limited, when it is necessary to manage a pre-defined maritime area, a large number of ground devices need to be deployed to cover the aforementioned maritime area. This results in high maintenance costs for the AIS system in terms of construction and use.

[0040] Furthermore, the aforementioned method of displaying ship information through nautical charts is not intuitive enough for users, as they cannot observe the actual environment in which the ship is located. Thus, when using nautical charts for maritime management, the timeliness and reliability of the management of ships and waterways in the waters cannot be guaranteed.

[0041] To address the aforementioned technical challenges, and given the widespread application of small unmanned aerial vehicle (UAV) technology, UAVs can quickly reach the scene and perform automatic patrol operations for frontline waterway management tasks, thereby increasing the frequency and efficiency of patrol operations. Based on this, this embodiment provides a display method, a control method for a mobile platform, a device, and a mobile platform. The display method is executed by a display device, meaning the display method can be applied to a display device, which can communicate with the mobile platform. Specifically, to improve the timeliness and reliability of maritime management operations, images captured by the mobile platform can be acquired first. These images include target vessels, which can be any one or more of multiple vessels in the captured images.

[0042] To enable users to quickly and intuitively obtain the target vessel's identity information through the captured footage, this information can be displayed within the captured image. The target vessel's identity information can be determined based on first location-related information and second location-related information determined by the vessel management system. The first location-related information is determined based on the location of the mobile platform itself. The aforementioned vessel management system can be an Automatic Identification System (AIS), and it is communicatively connected to the display device and / or the mobile platform. This allows the display device to quickly and directly acquire the captured image, including the target vessel, through the mobile platform. Users can then quickly and intuitively view the target vessel's identity information through the captured image, thereby improving vessel management or maritime management operations based on the target vessel's identity information, ensuring the quality and efficiency of maritime management, and thus enhancing the practicality of the method to a certain extent. It is understood that the vessel management system in this application can also include third-party management systems other than the AIS system, as long as it includes a system capable of collecting vessel information.

[0043] The following detailed description, with reference to the accompanying drawings, outlines some embodiments of a display method, a control method for a movable platform, an apparatus, and a movable platform according to the present invention. Where there is no conflict between the embodiments, the following embodiments and features thereof can be combined with each other.

[0044] Figure 1 is a flowchart illustrating a display method according to an embodiment of the present invention; Figure 2 is a scene illustration illustrating a display method according to an embodiment of the present invention. Referring to Figures 1 and 2, this embodiment provides a display method, the execution subject of which is a display device, i.e., the display method can be applied to a display device, wherein the display device can be implemented as software, or a combination of software and hardware; in some instances, the display device can be implemented as a handheld remote control corresponding to a mobile platform. Specifically, the display device can be communicatively connected to the mobile platform, and the mobile platform can be equipped with an image acquisition device capable of shooting operations, so that the target ship and its identity information can be quickly displayed in the captured image based on the mobile platform. In other embodiments, the display device and the remote control of the mobile platform are not the same device. In this case, the display device can be directly communicatively connected to the mobile platform, or it can be communicatively connected to the mobile platform through the control terminal of the mobile platform.

[0045] In this embodiment, the mobile platform refers to a system or device capable of moving freely between different locations, including but not limited to the following: aircraft, vehicles, ships, ground robots (such as sweeping robots, humanoid robots), etc. Taking aircraft as an example, in this embodiment, aircraft are classified according to whether they are piloted in the cabin, including unmanned aircraft and manned aircraft; according to configuration, including rotary-wing drones, fixed-wing drones, and drones combining rotary-wing and fixed-wing configurations; and according to purpose, including aerial photography drones, agricultural drones, surveying drones, logistics drones, etc.

[0046] Specifically, the display method implemented based on the above-mentioned display device may include:

[0047] Step S101: Acquire the footage captured by the mobile platform, including the target vessel.

[0048] When a mobile platform is conducting inspections or operations in a pre-defined sea area, the image acquisition device mounted on the mobile platform can perform image acquisition operations, thereby obtaining the images captured by the mobile platform. The captured images may include sea area information within a pre-defined field of view (FOV) or the field of view.

[0049] In some instances, the acquired footage may include a target vessel, which may be at least one of multiple vessels in the footage. To improve the identification of the target vessel, after acquiring the footage from the mobile platform, the method in this embodiment may further include: displaying at least one vessel included in the footage, wherein the at least one vessel includes the target vessel.

[0050] After acquiring footage captured by the mobile platform, the footage can be analyzed and identified to determine at least one vessel included in the footage. In some instances, at least one vessel can be determined through artificial intelligence (AI) algorithms analyzing the footage. Specifically, after acquiring footage captured by the mobile platform, AI algorithms can be used to identify vessels within the footage, thus ensuring the accuracy and reliability of identifying at least one vessel to a certain extent.

[0051] To facilitate the identification or confirmation of target vessels, after obtaining at least one vessel included in the captured image, the identified at least one vessel can be displayed on the captured image. This allows users to view the identified at least one vessel through the captured image, that is, not only the target vessel, but also other vessels besides the target vessel.

[0052] Furthermore, the target vessel can be any one of at least one vessel. In some instances, the target vessel can be determined based on the vessel's position information within the captured image. For example, the target vessel can be a vessel located in the center of the captured image; or, the target vessel can be a vessel located in a non-center position within the captured image.

[0053] In other instances, the target vessel can be automatically identified from the captured footage based on preset events. For example, the preset event could be "the speed exceeds a preset threshold," in which case the target vessel could be a vessel whose speed exceeds the preset threshold; or, the preset event could be "the navigation channel is not compliant," in which case the target vessel could be a vessel whose navigation channel is not compliant, and so on. This effectively achieves the automatic identification of the target vessel.

[0054] In some other instances, the target vessel can also be obtained through human-computer interaction. In this case, after displaying at least one vessel included in the captured image, the method in this embodiment can further include: in response to the user's selection operation for any number of vessels, obtaining the target vessel included in the captured image. This also ensures the accuracy and reliability of the mobile platform in acquiring the target vessel.

[0055] Furthermore, in order to enable users to quickly and intuitively view the target vessel through the captured images, the image acquisition device on the mobile platform performs image acquisition operations and obtains the captured images. The display device can then acquire the captured images through either the mobile platform or the image acquisition device. Specifically, the image acquisition device / mobile platform can actively send the captured images to the display device, allowing the display device to passively acquire them; or, the display device can actively interact with the mobile platform to obtain the captured images, thus ensuring the accuracy and reliability of the captured images to a certain extent.

[0056] Step S102: Display the target vessel's identity information in the captured image.

[0057] After acquiring the captured image from the mobile platform (specifically, through an image acquisition device located on the mobile platform), in order to facilitate users to obtain relevant information about the target vessel in a timely manner through the captured image, the captured image can not only display the visual information of the target vessel, but also the identity information of the target vessel. The identity information of the target vessel may include at least one of the following: vessel name, vessel type, vessel call sign, port of registry, registration number, etc.

[0058] Regarding the identity information of the target vessel, it can be determined based on the target vessel's first location-related information and the second location-related information determined by the vessel management system. It is understood that the first location-related information may include the first location itself, or it may include information that can deduce the first location; similarly, the second location-related information may include the second location itself, or it may include information that can deduce the second location.

[0059] In some instances, the first location-related information can be determined based on the location of the mobile platform itself. The location of the mobile platform itself can be obtained by positioning operations performed by a positioning module (e.g., Global Positioning System, GPS) deployed on the mobile platform. After obtaining the location of the mobile platform itself, the location can be analyzed and processed. Specifically, a preset location mapping algorithm or a pre-trained neural network model can be used to analyze and process the location of the mobile platform itself, thereby stably determining the first location-related information.

[0060] In other instances, the first position-related information can also be determined based on the position of the mobile platform itself and the relative positional relationship between the mobile platform and the target vessel. The relative positional relationship can include at least one of the following: the relative angle information between the mobile platform and the target vessel, and the distance information between the mobile platform and the target vessel. For example, the relative angle information can be implemented as (Δyaw angle, Δroll angle, Δpitch angle), and the distance information D between the mobile platform and the target vessel can be implemented as (Δx, Δy, Δz). Furthermore, the above-mentioned relative positional relationship can be determined by a laser ranging module, wherein the laser ranging module is deployed on the mobile platform.

[0061] After obtaining the relative positional relationship between the mobile platform and the target vessel through the laser ranging module, the position of the mobile platform itself and / or the relative positional relationship between the mobile platform and the target vessel can be analyzed and processed to determine the first position-related information of the target vessel.

[0062] Example 1: Referring to Figure 2a, taking a mobile platform including a drone as an example, when the relative positional relationship includes the relative angle information between the drone and the target ship, the first position-related information of the target ship can be determined based on the drone's own position and the relative angle information. Specifically, the relative angle information can include the angle information formed by the line segment formed between the drone and the target ship in the world coordinate system XYZ, which specifically includes: the angle a1 of the line segment formed between the drone and the target ship relative to the X-axis, the angle a2 of the line segment formed between the drone and the target ship relative to the Y-axis, and the angle a3 of the line segment formed between the drone and the target ship relative to the Z-axis.

[0063] After obtaining the location of the UAV itself, the UAV's location can be analyzed and calculated based on the aforementioned relative angle information (a1, a2, a3), thereby obtaining the first location-related information of the target ship. This first location-related information may include at least the target ship's positioning information.

[0064] Example 2: Referring to Figure 2b, taking a mobile platform including a drone as an example, when the relative positional relationship includes the distance information D between the drone and the target ship, the first position-related information of the target ship can be determined based on the drone's own position and the distance information between the drone and the target ship.

[0065] Specifically, the distance information D between the UAV and the target ship can be obtained by detecting the distance using a distance detection module, which can be deployed on either the UAV or the target ship. After obtaining the distance information D between the UAV and the target ship, a preset position transformation relationship or a pre-trained neural network model can be used to analyze and calculate the distance information D and the UAV's own position, thereby obtaining the first position-related information of the target ship.

[0066] Example 3: Taking a mobile platform including a drone as an example, when the relative positional relationship can include the relative angle information between the drone and the target ship, and the distance information between the drone and the target ship, the first position-related information of the target ship can be determined based on the drone's own position, the relative angle information between the drone and the target ship, and the distance information between the drone and the target ship. This also ensures the accuracy and reliability of determining the first position-related information of the target ship.

[0067] Regarding the first location information of the target vessel, since it is determined based on the location of the mobile platform itself, and the positioning accuracy of the mobile platform itself is high, the positioning accuracy of the first location information of the target vessel is also high. That is, the positioning accuracy of the first location information of the target vessel can be greater than or equal to a preset threshold, which helps to improve the accuracy of determining the identity information of the target vessel based on the first location information of the target vessel.

[0068] Since the first location-related information is determined solely by the location of the mobile platform itself, in order to improve the accuracy and reliability of obtaining the identity information of the target vessel to a certain extent, a second location-related information determined by the ship management system can be obtained. The ship management system may include an Automatic Identification System (AIS), and the second location-related information may include information on the target vessel's location detection based on the ship management system and its proactive reporting to the display device or mobile platform.

[0069] After obtaining the first location-related information of the target vessel and the second location-related information determined based on the vessel management system, since the first location-related information and the second location-related information are the target vessel's location-related information determined using different methods, the accuracy and reliability of determining the target vessel's identity information can be improved to a certain extent when determining the target vessel's identity information based on the first location-related information and the second location-related information.

[0070] Furthermore, when the display device displays at least one ship included in the captured image, in order to enable the user to quickly and directly understand the relevant information of each ship, the location-related information of each ship can also be displayed in the captured image. In this case, displaying at least one ship in the captured image in this embodiment may include: displaying the location-related information of each of the at least one ship in the captured image.

[0071] The position-related information of each vessel is determined based on the first position-related information of the target vessel, or the position-related information of each vessel can be determined based on the first position-related information and the second position-related information of the target vessel.

[0072] After obtaining the location information of each vessel, the location information of at least one vessel can be displayed on the captured image. This location information may include at least one of the following: vessel positioning information, vessel identification information, vessel navigation information, etc. This allows users to not only view at least one vessel through the captured image, but also to view the location information of at least one vessel, thus facilitating vessel management operations based on at least one vessel and its corresponding location information.

[0073] The display method provided in this embodiment acquires the captured image from a mobile platform and then displays the target vessel's identity information in the captured image. This effectively enables the display device to quickly acquire the captured image, including the target vessel, through the mobile platform. This allows users to quickly and intuitively view the target vessel's identity information through the captured image, and then perform ship management or maritime management operations based on the target vessel's identity information. This helps improve the timeliness and reliability of maritime management operations and further ensures the practicality of the method.

[0074] Figure 3 is a schematic diagram of another display method provided by an embodiment of the present invention. Based on the above embodiment, referring to Figure 3, when a captured image is obtained and the captured image includes at least one ship, before displaying the position-related information of each of the at least one ship in the captured image, the position-related information of each ship can be determined based on the relative display position relationship between each ship in the captured image and the target ship. At this time, the method in this embodiment may include:

[0075] Step S301: Obtain the relative display position relationship between each ship in the captured image and the target ship.

[0076] In situations where the captured image includes at least one vessel, and that vessel includes the target vessel, to ensure the accuracy and reliability of determining the positional information of each vessel, the captured image can be analyzed and processed after acquisition to obtain the relative display positional relationship between each vessel and the target vessel. This relative display positional relationship can include the relative relationship within the pixel coordinate system of the captured image. For example, the relative display positional relationship can be implemented as (Δx, Δy), where Δx represents the relative positional relationship along the x-axis of the image coordinate system, and Δy represents the relative positional relationship along the y-axis. Alternatively, the relative display positional relationship can be implemented as (Δw, Δh), where Δw represents the relative horizontal positional relationship along the W-axis of the image coordinate system, and Δh represents the relative vertical positional relationship along the H-axis of the image coordinate system.

[0077] In some instances, the relative display positional relationships can be determined by analyzing and processing the captured images using a pre-trained image analysis model. In this case, obtaining the relative display positional relationships between each ship in the captured images and the target ship may include: calling the image analysis model used to analyze and process the captured images; and using the image analysis model to analyze and process the captured images, thereby stably obtaining the relative display positional relationships between each ship and the target ship.

[0078] In other instances, relative display position relationships can be determined not only through image analysis models but also through object recognition algorithms. In this case, obtaining the relative display position relationships between each vessel in the captured image and the target vessel can include: invoking an object recognition algorithm to analyze the captured image; using the object recognition algorithm to identify and locate at least one vessel in the captured image, obtaining the display position information of each vessel in the corresponding pixel coordinate system of the captured image, where each vessel includes the target vessel; and then determining the relative display position relationships between each vessel and the target vessel based on the display position information of each vessel in the corresponding pixel coordinate system of the captured image and the target vessel in the corresponding pixel coordinate system of the captured image. This also ensures the accuracy and reliability of obtaining the relative display position relationships.

[0079] Step S302: Based on the first position-related information and the relative displayed position relationship, determine the position-related information of each ship.

[0080] After obtaining the relative display position relationship, the first position information of the target vessel and the relative display position relationship between each vessel and the target vessel can be analyzed and processed to determine the position information of each vessel.

[0081] Specifically, the position-related information of each vessel can be determined by analyzing and processing the first position-related information and the relative displayed position relationship based on a pre-trained neural network model and a preset algorithm. In this case, determining the position-related information of each vessel based on the first position-related information and the relative displayed position relationship may include: acquiring a preset algorithm / neural network model for determining the position-related information of each vessel; and using the preset algorithm / neural network model to analyze and process the first position-related information and the relative displayed position relationship to obtain the position-related information of each vessel. This effectively ensures the accuracy and reliability of determining the position-related information of each vessel.

[0082] In this embodiment, the relative display position relationship between each ship in the captured image and the target ship is obtained, and then the position-related information of each ship is determined based on the first position-related information and the relative display position relationship. In this way, the position-related information of each ship is accurately determined, thereby improving the accuracy and reliability of displaying position-related information in the captured image.

[0083] Based on the above embodiments, before displaying the target vessel's identity information in the captured image, the method in this embodiment may further include: obtaining the target vessel's identity information.

[0084] The identity information of the target vessel is determined not only by analyzing and processing the first location-related information of the target vessel and the second location-related information determined by the ship management system based on a preset algorithm or a pre-trained neural network model, but also by analyzing and matching the first and second location-related information. In this case, obtaining the identity information of the target vessel may include: obtaining the identity information based on the matching results of the first and second location-related information.

[0085] Specifically, for the target vessel, the first location-related information is determined based on the location of the mobile platform itself, and the second location-related information is determined based on the vessel management system. In some instances, the second location-related information may include at least the reported positioning information and the identity information of the reporting vessel. The reporting vessel may be a vessel recorded in or managed by the vessel management system.

[0086] Since both the first and second location-related information can identify the location-related information of the target vessel, and since the first and second location-related information are obtained using different methods or different dimensions, in order to accurately determine the identity information of the target vessel, after obtaining the first and second location-related information, an analysis and matching operation can be performed on the first and second location-related information to obtain the matching results. Then, the identity information of the target vessel can be determined based on the matching results.

[0087] In some instances, obtaining identity information based on the matching results of the first location-related information and the second location-related information may include: when the positioning information of the target vessel represented by the first location-related information matches the reported positioning information, it means that the deviation between the positioning information of the target vessel represented by the first location-related information and the reported positioning information is small. Specifically, the deviation between the positioning information of the target vessel and the reported positioning information may be less than or equal to a preset positioning threshold, and then the identity information of the reported vessel can be determined as the identity information of the target vessel.

[0088] Correspondingly, when the positioning information of the target vessel represented by the first location-related information does not match the reported positioning information in the second location-related information, it means that the deviation between the positioning information of the target vessel represented by the first location-related information and the reported positioning information is large. Specifically, the deviation between the positioning information of the target vessel and the reported positioning information can be greater than the preset positioning threshold. At this time, it can be determined that the target vessel represented by the first location-related information is not included in the vessels collected or managed by the vessel management system, and therefore the identity information of the reported vessel cannot be determined as the identity information of the target vessel.

[0089] Furthermore, the method in this embodiment may also include: identifying illegally operating vessels in the captured image, wherein illegally operating vessels include vessels in the captured image that fail to match the reported positioning information.

[0090] Specifically, when the location information of the target vessel represented by the first location-related information does not match the location information reported by the vessel management system, it indicates that the target vessel in the captured image is a vessel that fails to match the reported location information. In this case, the presence of vessels that fail to match the reported location information in the captured image indicates that the vessel management system has not included or registered the aforementioned target vessels, and therefore cannot perform any management or control operations on the aforementioned vessels. In this way, vessels in the captured image that fail to match the reported location information can be identified as illegally operating vessels.

[0091] To enable users to quickly and intuitively understand information about illegally operating vessels, these vessels can be marked in the captured footage. Specifically, this can be done by highlighting the vessel's color or outline. This allows users to quickly and directly obtain information about illegally operating vessels from the captured footage, facilitating effective management of these vessels based on the information displayed in the footage.

[0092] In this embodiment, the identity information of the target vessel is obtained by analyzing and matching the first location-related information and the second location-related information, thus achieving a certain degree of flexibility and reliability in obtaining the identity information of the target vessel.

[0093] For example, based on any of the above embodiments, in order to ensure to a certain extent the acquisition or reception of information related to the second location determined by the ship management system, the mobile platform may be equipped with an information receiving device that is communicatively connected to the Automatic Identification System (AIS). This information receiving device can be used to receive information sent by the AIS, which may include the official system of a maritime authority. In some instances, the mobile platform can communicate with the information receiving device through a preset expansion interface, or the mobile platform can communicate with the information receiving device through a wireless communication component.

[0094] For an information receiving device on a mobile platform, it can be implemented in at least one of the following ways:

[0095] 1) When the mobile platform is implemented as a drone, the information receiving device can be implemented as a receiver box of the AIS system installed on the drone. The receiver box of the AIS system can communicate with the transmitter box in the AIS system so as to receive the information sent by the AIS system.

[0096] 2) The mobile platform is implemented as a drone, and the AIS system receiver box is installed on the land-based equipment. The drone can wirelessly connect with the AIS system receiver box. After the AIS receiver box obtains the information sent by the AIS system, it can forward the received information to the drone, so that the drone can quickly obtain the information sent by the AIS system through the receiver box of the AIS system on the land-based equipment.

[0097] In this embodiment, the information sent by the Automatic Identification System (AIS) is obtained through an information receiving device mounted on a mobile platform, which to a certain extent ensures the accuracy and reliability of obtaining the information sent by the AIS.

[0098] Figure 4 is a schematic flowchart illustrating the display of the target vessel's identity information in a captured image according to an embodiment of the present invention. Based on any of the above embodiments, referring to Figure 4, the target vessel's identity information can be displayed based on its display position. In this case, displaying the target vessel's identity information in the captured image may include:

[0099] Step S401: Determine the display position of the target vessel in the captured image.

[0100] In order to accurately display the identity information of the target vessel, we can first determine the display position of the target vessel in the shooting frame, and then display the identity information of the target vessel based on its display position in the shooting frame.

[0101] In some instances, the display position can be obtained by identifying the target vessel in the captured image. In this case, determining the display position of the target vessel in the captured image may include: identifying and locating the target vessel in the captured image to obtain the display position of the target vessel in the captured image. This, to a certain extent, ensures the accuracy and reliability of determining the display position.

[0102] In other instances, the display position can also be determined through human-computer interaction. In this case, determining the display position of the target vessel in the captured image may include: displaying the captured image; obtaining the user's confirmation operation input on the target vessel in the captured image; and determining the display position of the target vessel in the captured image based on the confirmation operation. This also ensures the accuracy and reliability of determining the display position.

[0103] In another example, the display position can be obtained not only by identifying the target vessel in the captured image, but also by determining it based on camera extrinsic parameters and the target vessel's initial position information. In this case, determining the target vessel's display position in the captured image can include: acquiring camera extrinsic parameters corresponding to the captured image. These extrinsic parameters can be parameters describing the camera's position and attitude in the world coordinate system, and can include at least one of the following: rotation matrix, translation vector, etc.; then, the target vessel's display position in the captured image can be determined based on the camera extrinsic parameters, the target vessel's initial position information, and the position of the movable platform itself. Specifically, a preset algorithm or a pre-trained neural network model can be used to analyze and process the camera extrinsic parameters, the target vessel's initial position information, and the movable platform's own position to obtain the target vessel's display position in the captured image, thus effectively ensuring the accuracy and reliability of the determined display position.

[0104] Step S402: Based on the display location, display the target vessel's identity information in the captured image.

[0105] After obtaining the target vessel's display position in the captured image, its identification information can be displayed based on that position. In some instances, the target vessel's identification information can be displayed directly at the target vessel's location in the captured image, i.e., at the aforementioned display position. Alternatively, the target vessel's identification information can be displayed in a location other than the target vessel in the captured image. For example, when the target vessel is being captured, its identification information can be displayed on the right / left side of the captured image; or, if the target vessel is located in the center of the captured image, its identification information can be displayed above or below the captured image, etc. The key is to ensure that the display device can accurately display the target vessel's identification information; further details will not be elaborated upon here.

[0106] In other instances, the target vessel's identity information can include not only its name, type, call sign, port of registry, and registration number, but also dynamically changing navigation information such as its attitude, direction, speed, and navigation path. To ensure the accuracy and reliability of the display device's representation of the target vessel's identity information, the information is dynamically updated based on the attitude information of the mobile platform and the display device.

[0107] The attitude information of the mobile platform can be obtained by detecting the mobile platform using attitude sensors mounted on it. To enable users to view and be aware of potential dynamic changes in the target vessel in a timely manner, the target vessel's identity information can be dynamically displayed. Therefore, displaying the target vessel's identity information in the captured image in this embodiment can include dynamically displaying the target vessel's identity information within the captured image.

[0108] Example 1: When the target vessel's identity information includes the target vessel's name and speed, the target vessel's speed can change as the vessel sails. For example, at time t1, the target vessel's speed is V1; at time t2, the target vessel's speed is V2, where V1 and V2 are different.

[0109] Therefore, in order to enable users to understand the target vessel's speed through the captured footage, the target vessel's name and speed can be dynamically displayed in the captured footage. Specifically, at time t1, the target vessel's name and speed V1 can be displayed in the captured footage; at time t2, the target vessel's name and speed V2 can be displayed in the captured footage. This achieves dynamic display of the target vessel's identity information in the captured footage, and also allows for dynamic updating of the displayed target vessel's identity information (such as speed).

[0110] Example 2: When the target vessel's identity information includes the target vessel's name and attitude information, the target vessel's attitude information can change as the vessel navigates or the mobile platform's shooting attitude changes. For example, at time t1, the target vessel's attitude information is Z1; at time t2, the target vessel's attitude information is Z2, where Z1 and Z2 are different.

[0111] Therefore, in order to enable users to understand the attitude information of the target vessel through the captured image, the vessel's name and attitude information can be dynamically displayed in the captured image. Specifically, at time t1, the vessel's name and attitude information Z1 can be displayed in the captured image; at time t2, the vessel's name and attitude information Z2 can be displayed in the captured image. This achieves the dynamic display of the target vessel's identity information in the captured image, and also allows for the dynamic updating of the displayed target vessel's identity information (e.g., attitude information).

[0112] In this embodiment, by determining the display position of the target vessel in the captured image, and then displaying the target vessel's identity information in the captured image based on the display position, the timely and reliable display of the target vessel's identity information in the captured image is effectively achieved.

[0113] Based on any of the above embodiments, in order to enable users to understand the target vessel's navigation information through the captured image, the method in this embodiment may further include: displaying the target vessel's navigation information in the captured image.

[0114] For the target vessel, not only can the target vessel's identity information be displayed in the captured image, but the target vessel's navigation information can also be displayed in the captured image. The target vessel's navigation information may include at least one of the following: navigation speed and navigation route.

[0115] Since the target vessel's navigation information can change as the target vessel operates, in order to enable users to understand the target vessel's navigation information in a timely and quick manner, the target vessel's navigation information can be directly displayed in real time on the captured screen after it is obtained.

[0116] Alternatively, in order to ensure the accuracy and reliability of acquiring the target vessel's driving information, when displaying the target vessel's driving information using the captured image, the mobile platform can be controlled to follow the target vessel. In this case, displaying the target vessel's driving information in the captured image in this embodiment may include: displaying the target vessel's driving information in the captured image while the mobile platform follows the target vessel.

[0117] When a target vessel is navigating in the sea, its navigation information may change. To ensure the timely and accurate display of the target vessel's identity and navigation information, a mobile platform follows the target vessel. This allows the mobile platform to collect or gather the target vessel's navigation information while following it, thus ensuring the accuracy and reliability of the displayed navigation information to a certain extent.

[0118] In some instances, different navigation information for a target vessel may correspond to different compliance review results. For example, the navigation information may meet preset navigation standards, or it may not. In such cases, the navigation information can be displayed in different ways based on different compliance review results. For instance, displaying the target vessel's navigation information in the captured footage could include: when the target vessel's navigation information does not meet preset navigation standards, the navigation information of the target vessel can be displayed differently in the captured footage.

[0119] After obtaining the target vessel's driving information, the driving information can be analyzed and processed based on preset driving standards to identify whether the target vessel's driving information meets the preset driving standards. Specifically, the analysis and processing of the target vessel's driving information based on preset driving standards may include: obtaining the driving speed threshold corresponding to the preset driving standards; analyzing and comparing the driving information with the driving speed threshold; if the driving information is less than or equal to the driving speed threshold, it is determined that the target vessel's driving information meets the preset driving standards; if the driving information is greater than the driving speed threshold, it is determined that the target vessel's driving information does not meet the preset driving standards.

[0120] When the target vessel's driving information meets the preset driving standards, it means that the target vessel's driving information is compliant or legal. In this case, the user theoretically does not need to perform any additional management operations on the target vessel, and can simply display the target vessel's driving information normally in the captured image.

[0121] Correspondingly, if the target vessel's driving information does not meet the preset driving standards, it indicates that the target vessel's driving information is non-compliant or illegal. In this case, in order to enable users to quickly and intuitively understand that the target vessel's driving information does not meet the preset driving standards, the target vessel's driving information can be displayed differently in the captured image. Specifically, the target vessel's driving information can be displayed differently from the target vessel's identity information. For example, the display color of the target vessel's driving information (e.g., red, green, etc.) can be different from the display color of the target vessel's identity information (e.g., black, gray, etc.). As long as users can quickly learn about the target vessel's driving information that does not meet the preset driving standards through the captured image, it will not be elaborated further here.

[0122] Furthermore, to more reliably ensure that users are aware of the state that "the target vessel's driving information does not meet the preset driving standards," the method in this embodiment may further include: when the target vessel's driving information does not meet the preset driving standards, a prompt message can be generated and output based on the above situation. The prompt message can be displayed through a prompt box, message pop-up, or floating window. Since the prompt message can be used to inform the user that the target vessel is in an abnormal driving state, the user can quickly and directly learn about the abnormal driving state of the target vessel through the prompt message, and then facilitate the user to manage the target vessel based on the abnormal driving state of the target vessel, thereby improving the practicality of the method.

[0123] In other instances, to facilitate users' more intuitive perception of the target vessel's navigation information, the method in this embodiment may further include: displaying a ship-view image of the target vessel in the captured image.

[0124] Example 1: A mobile platform is equipped with multiple image acquisition devices for image capture operations. Some of these image acquisition devices can capture images of the sea area where the target ship is located, and some of them can also capture images from the target ship's perspective, thereby obtaining images from the target ship's perspective.

[0125] After acquiring the ship's perspective image, the target ship's perspective image can be displayed in the captured screen. This allows users to view not only the target ship's identity information but also its perspective image in the captured screen, thus facilitating users' perception and operation of the target ship's navigation information. This helps improve the stability and reliability of users' management and operation of the target ship based on the ship's perspective image.

[0126] Example 2: The target vessel is equipped with an image acquisition device for capturing images. This device can acquire images from the target vessel's perspective, thus obtaining an image of the target vessel from that perspective. The image acquisition device is communicatively connected to a display device or a mobile platform. The image acquisition device actively or passively transmits the image from the target vessel's perspective to the display device or mobile platform, which can then display the image from the target vessel's perspective on the captured image.

[0127] In some instances, the ship's perspective image can be displayed in a picture-in-picture format within the captured image; alternatively, the ship's perspective image and the captured image can be displayed in different display areas. This allows users to view not only the target ship's identity information but also its perspective image within the captured image, facilitating their perception and operation of the target ship's navigation information. This improves the accuracy and reliability of user-based management operations on the target ship using the perspective image, further enhancing the practicality of the method.

[0128] In a specific application, referring to Figure 5, taking a drone as a mobile platform as an example, this application embodiment provides a ship management method based on a drone. This method can be executed by a ship management device to form a new intelligent technology management operation in the field of maritime management. After execution by the ship management device, the problems of low accuracy and low efficiency in maritime management scenarios can be solved. Specifically, the method may include the following steps:

[0129] Step 1: Obtain the drone's own drone location information.

[0130] The drone is equipped with a real-time kinematic (RTK) module for positioning operations. The RTK module can accurately obtain the drone's positioning information, and the positioning accuracy of the drone's positioning information is greater than or equal to a preset threshold.

[0131] Step 2: Acquire images using cameras deployed on the drone, which may include the target vessel.

[0132] The camera deployed on the drone can be a visible light camera, and the number of visible light cameras can be three. All three visible light cameras (1x, 3x, 7x) are deployed on the drone, and the optical centers of the three visible light cameras are calibrated and aligned to form a zoom camera. In this way, the three visible light cameras can accurately acquire the shooting image.

[0133] After acquiring the captured footage, the onboard AI detection model can be used to analyze and identify the footage to identify at least one vessel and its visual information (including: vessel shape information, vessel color information, vessel type information, etc.). The at least one vessel may include the target vessel to be investigated, and the number of target vessels may be one or more.

[0134] Step 3: Determine the relative positional relationship between the UAV and the target ship, and determine the positional information of the target ship based on the relative positional relationship.

[0135] The drone is equipped with a laser ranging and marking module, which is aligned with the optical center of the camera module consisting of visible light cameras. The relative positional relationship between the drone and the target ship can then be determined based on the camera module consisting of visible light cameras and the laser ranging and marking module.

[0136] In some instances, the laser ranging and marking module can be aligned with a target vessel in the center of the image to obtain the relative positional relationship between the UAV and the target vessel. This relative positional relationship can include: relative angle information (yaw angle, pitch angle, roll angle) and relative distance information (ΔX, ΔY, ΔZ) between the UAV and the target vessel.

[0137] Specifically, determining the relative positional relationship between the UAV and the target vessel can include: acquiring the extrinsic parameters of each camera (including rotation matrix, translation vector, etc.); acquiring the attitude angle of the camera gimbal through the attitude sensor mounted on the UAV, wherein the camera can be mounted on the UAV via the camera gimbal; then acquiring the attitude angle of the camera gimbal; and acquiring the real-time ranging distance based on the laser ranging standard module; and then calculating the position information of the target vessel in the camera coordinate system in the captured image (i.e., the positioning information corresponding to the first position-related information in the above embodiment) based on the UAV positioning information, the camera extrinsic parameters, the attitude angle of the camera gimbal, and the real-time ranging distance, thus effectively ensuring that the target vessel performs accurate positioning operations.

[0138] Step 4: The drone can receive ship broadcast information in real time through the AIS receiver module.

[0139] The drone can be equipped with an AIS receiver module. Specifically, the AIS receiver module can communicate with the drone through the fuselage expansion interface. In this case, the AIS receiver module can directly receive the ship broadcast information sent by the AIS land base station to ships in the airspace and water area where it is located.

[0140] Alternatively, if the AIS receiver module is not deployed on the drone, but rather on an AIS land-based base station, the drone can communicate with it via a wireless communication module. In this way, the AIS receiver module can obtain ship broadcast information from the AIS land-based base station and then transmit the ship broadcast information to the drone via the wireless communication module, allowing the drone to receive the ship broadcast information in real time through the AIS receiver module.

[0141] In some instances, the ship broadcast information may include at least one of the following: ship name, ship MMSI number, ship length and width dimensions, ship reported positioning information (latitude and longitude information of the ship in the world coordinate system, etc.), ship type, etc.

[0142] Step 5: Based on the reported positioning information in the ship broadcast information and the target ship's location information, determine the target ship's information and display it.

[0143] The target vessel's information may include static information and dynamic information. The static information may include at least one of the following: vessel identification code, vessel type, port of destination, etc. The dynamic information may include at least one of the following: vessel positioning information, vessel heading information, vessel speed, vessel navigation status, etc.

[0144] In some instances, some of the target vessel's information can be pre-stored in a cloud database or cloud server. Once the target vessel's identification code or vessel number is obtained, data can be retrieved from the cloud database or cloud server using the identification code or vessel number to obtain other information about the target vessel, such as its vessel type, port of destination, etc.

[0145] Drones can communicate with and display devices. After acquiring ship broadcast information, they can merge the ship broadcast information and the target ship into a single display. In some cases, augmented reality (AR) display technology can be used to display the target ship and the ship broadcast information in the real-time video captured by the drone.

[0146] Step 6: Based on the target vessel's location information GPS1 and the captured image, calculate the location information GPSN of all vessels in the captured image.

[0147] When the target vessel is any one of at least one vessels in the captured image, after obtaining the target vessel's position information GPS1, the captured image can be analyzed and processed to obtain the relative pixel coordinate relationship between the target vessel and any other vessel in the captured image; based on the relative pixel coordinate relationship and the target vessel's position information GPS1, the position information GPSN of all vessels in the captured image can be determined.

[0148] The relative pixel coordinate relationship can be determined by the position of the target ship in the captured image and the projection positions of other ships in the captured image. The projection positions of other ships in the captured image can be obtained by analyzing and calculating based on the external parameters of the camera gimbal, the positioning information of the UAV itself, and the positioning information of the target ship.

[0149] Step 7: Based on the reported positioning information in the ship broadcast information and the GPSN position information of all ships, determine the ship information of each ship and display the ship information of each ship.

[0150] When the reported positioning information in the ship broadcast information is implemented as AGPSN, the obtained reported positioning information AGPSN can be analyzed and compared with the position information GPSN of each ship. When the reported positioning information AGPSi matches the position information GPSj of the ship, the reported information corresponding to the reported positioning information AGPSi can be determined as the ship-related information corresponding to that ship. Then, the ship information of that ship can be determined based on the ship broadcast information corresponding to the reported positioning information AGPSi.

[0151] Similarly, by continuously reporting positioning information and matching it with the GPSN location information of each vessel, the vessel information of each vessel in the captured image can be determined. Then, the vessel information of each vessel can be displayed in the captured image. This allows users to directly view the relevant information of the target vessel and other vessels through the captured image, which helps to improve the quality and efficiency of vessel navigation management.

[0152] Step 8: Based on the drone's attitude and the camera gimbal's attitude, dynamically refresh the display of ship information in the captured footage in real time.

[0153] Since ship information may change over time, in order to ensure that the display device can display ship information in a timely and accurate manner, the ship information in the captured image can be dynamically refreshed and displayed in real time according to the attitude of the drone and the camera gimbal.

[0154] Step 9: Perform management operations on the vessel based on the vessel information displayed in the captured footage.

[0155] To further enhance and optimize vessel position and management, vessel management operations can be implemented based on vessel information provided by captured footage or other visual information. In some examples, vessel routes can be calculated and tracked in real time, and warnings can be issued for vessel routes and speeds. For instance, since different waterways or routes may have different draft requirements, warnings can be issued when a vessel's route deviates from the expected route, or when a vessel's speed exceeds a preset threshold. This can improve the safety and reliability of vessel navigation to a certain extent.

[0156] In addition, the camera can not only manage vessels registered or included in the AIS system, but also issue alarms for unregistered vessels. For example, when an unregistered vessel is detected in the camera footage, it can be determined that the vessel is an illegal vessel or has performed an illegal operation (e.g., the vessel has illegally shut down the AIS system). In this case, illegal operation instruction information for the vessel can be generated and displayed in the camera footage, so that users can quickly and directly learn about the illegal behavior of the vessel through the camera footage, thereby facilitating timely management and response to illegal behavior.

[0157] The ship management method provided in this application embodiment can realize operations such as navigation safety operation, marine monitoring operation, or ship traffic management based on ship detection information. Specifically, the method can utilize the mobility of UAVs and combine them with airborne payload equipment to accurately obtain the UAV's own RTK high-precision positioning information. Through the UAV's own RTK high-precision positioning and accurate ship identification algorithm, the method can accurately mark the position of the target ship. Then, the position information of the target ship obtained by the above position marking operation can be matched with the ship reported positioning information in the AIS received information. Based on the matching result, a relatively accurate ship information of the target ship can be determined, thus ensuring the accuracy and reliability of the determination of the ship information of the target ship to a certain extent.

[0158] Furthermore, the aforementioned vessel information can be precisely projected and overlaid onto the video footage captured by the drone. Specifically, aerial images captured by the drone can be obtained first, and then the acquired vessel information can be overlaid onto the aerial images in real time. This allows frontline patrol and law enforcement personnel to obtain relevant information about target vessels immediately, making the supervision of vessels in the waters more intuitive, efficient, and readily apparent. It also enables specific management operations for specific vessels, providing a degree of targeting (e.g., whether the passing vessel is using AIS, whether it complies with the waterway management requirements of the waterway, etc.). In addition, using drones for waterway management can be more efficient and frequent, with wider coverage, thus forming a new technological management method that helps save management costs, improves operational efficiency, and further enhances the practicality of the method.

[0159] Figure 6 is a flowchart illustrating a control method for a mobile platform provided in an application embodiment of the present invention. Referring to Figure 6, this embodiment provides a control method for a mobile platform. The execution subject of this method is a control device for the mobile platform. The control device for the mobile platform can be implemented as software or a combination of software and hardware. The control device for the mobile platform can be communicatively connected to the mobile platform, or the control device for the mobile platform can be mounted on the mobile platform. The mobile platform is equipped with an image acquisition device and can be communicatively connected to a display device. Through the cooperation between the mobile platform and the display device, users can quickly and intuitively display the identity information of the target vessel in the captured image.

[0160] In this embodiment, the mobile platform refers to a system or device capable of moving freely between different locations, including at least one of the following: aircraft, vehicles, ships, ground robots (such as robotic vacuum cleaners), etc. Taking aircraft as an example, in this embodiment, aircraft are classified according to whether they are piloted inside the cabin, including unmanned aircraft and manned aircraft; according to configuration, including rotary-wing drones, fixed-wing drones, and drones combining rotary-wing and fixed-wing configurations; and according to purpose, including aerial photography drones, agricultural drones, surveying drones, logistics drones, etc.

[0161] In addition, the image acquisition device in this embodiment refers to a system or device capable of performing image acquisition operations, including at least one of the following: visible light camera, infrared camera, lidar camera, etc. Those skilled in the art can arbitrarily adjust or configure the image acquisition device according to specific application scenarios or application requirements.

[0162] Specifically, the control method for the mobile platform implemented based on the aforementioned control device for the mobile platform may include:

[0163] Step S601: Acquire the captured footage from the mobile platform, including the target vessel.

[0164] Step S602: Obtain the first position-related information of the target vessel, which is determined based on the position of the mobile platform itself.

[0165] Step S603: Obtain information related to the second location determined by the ship management system.

[0166] Step S604: Transmit the captured image to the display device so that the display device can display the identity information of the target vessel in the captured image, wherein the identity information of the target vessel is determined based on the first location-related information and the second location-related information.

[0167] The specific implementation methods and effects of each step in the above embodiments are similar to those of each step in the display method shown in Figures 1-2. For details, please refer to the descriptions in the embodiments shown in Figures 1-2. For parts not described in detail in this embodiment, please refer to the relevant descriptions of the embodiments shown in Figures 1-2. The execution process and technical effects of this technical solution are described in the embodiments shown in Figures 1-2, and will not be repeated here.

[0168] The mobile platform control method provided in this embodiment acquires images captured by the mobile platform, then obtains first location-related information of the target vessel and second location-related information determined by the vessel management system, and transmits the captured images to a display device so that the display device can display the target vessel's identity information in the captured images. Since the target vessel's identity information is determined based on the first and second location-related information, this ensures the accuracy and reliability of the determination of the target vessel's identity information to a certain extent. In addition, by displaying the target vessel's identity information in the captured images through the display device, users can quickly and intuitively view the target vessel's identity information through the captured images, thereby improving the quality and efficiency of vessel management or maritime management based on the target vessel's identity information, thus ensuring the practicality of the method.

[0169] In some instances, obtaining the first position-related information of the target vessel may include: obtaining the position of the mobile platform itself; determining the relative positional relationship between the mobile platform and the target vessel; and determining the first position-related information of the target vessel based on the position of the mobile platform itself and the relative positional relationship.

[0170] In some instances, the relative positional relationship may include at least one of the following: relative angle information between the mobile platform and the target vessel, and distance information between the mobile platform and the target vessel.

[0171] In some instances, relative positional relationships are determined by laser ranging modules deployed on mobile platforms.

[0172] In some instances, the positioning accuracy of the first location-related information is greater than or equal to a preset threshold.

[0173] In some instances, after acquiring footage captured by a mobile platform, the method in this embodiment may further include: identifying at least one vessel included in the footage, wherein the at least one vessel includes the target vessel.

[0174] In some instances, at least one vessel was identified by analyzing and recognizing footage captured using artificial intelligence (AI) algorithms.

[0175] In some instances, the target vessel is located in the center of the captured image.

[0176] In some instances, the target vessel was located off-center in the frame of the shot.

[0177] In some instances, after identifying at least one vessel in the captured image, the method in this embodiment may further include: determining the location-related information corresponding to each of the at least one vessel based on the first location-related information; and sending the location-related information corresponding to each of the at least one vessel to a display device so that the display device displays the location-related information corresponding to each of the at least one vessel in the captured image.

[0178] In some instances, determining the location-related information of at least one vessel based on the first location-related information may include: acquiring the relative display positional relationship between each vessel in the captured image and the target vessel; and determining the location-related information of each vessel based on the first location-related information and the relative display positional relationship.

[0179] In some instances, relative display positional relationships include the relative relationships between pixel coordinates located within the captured image.

[0180] In some instances, the method in this embodiment may further include obtaining the identity information of the target vessel before transmitting the captured image to the display device.

[0181] In some instances, obtaining the identity information of a target vessel may include: obtaining the identity information based on the matching results of a first location-related information and a second location-related information.

[0182] In some instances, the second location-related information includes at least the reported location information of the reporting vessel and the identity information of the reporting vessel.

[0183] In some instances, obtaining identity information based on the matching results of the first location-related information and the second location-related information may include: when the positioning information of the target vessel represented by the first location-related information matches the reported positioning information, the identity information of the reported vessel is determined as the identity information of the target vessel.

[0184] In some instances, the identity information includes at least one of the following: vessel name, vessel type, vessel call sign, port of registry, and registration number.

[0185] In some instances, the mobile platform is equipped with an information receiving device that can receive information sent by the Automatic Identification System (AIS).

[0186] In some instances, the mobile platform communicates with the information receiving device via a pre-defined expansion interface.

[0187] In some instances, the method in this embodiment may further include: sending the identity information of the target vessel to a display device.

[0188] In some instances, the method in this embodiment may further include: acquiring the driving information of the target vessel; sending the driving information to a display device so that the display device displays the driving information of the target vessel in the captured image.

[0189] In some instances, after obtaining the target vessel's navigation information, the method in this embodiment may further include: performing navigation management on the target vessel based on the navigation information.

[0190] In some instances, driving information includes at least one of the following: driving speed, driving route.

[0191] In some instances, sending driving information to a display device may include: sending driving information to a display device when the driving information of the target vessel does not meet preset driving standards, so that the display device can output a prompt message, which can be used to inform the user that the target vessel is in an abnormal driving state.

[0192] In some instances, the method in this embodiment may further include: acquiring a ship-view image of the target vessel; sending the ship-view image to a display device so that the display device can display the ship-view image of the target vessel in the captured image.

[0193] In some instances, after controlling the mobile platform to follow the target vessel, the method in this embodiment may further include: generating alarm information related to the target vessel; and sending the alarm information to a display device so that the display device can display the alarm information in the captured image.

[0194] In some instances, the navigation information includes the speed; navigation management of the target vessel based on the navigation information may include: allowing the target vessel to continue navigation operations when the speed information is less than or equal to a preset speed threshold; and issuing an overspeed warning to the target vessel when the speed information is greater than the preset speed threshold.

[0195] The specific implementation methods and effects of each scheme in the above embodiments are similar to the specific implementation methods and effects of each step in the display methods shown in Figures 1-5. For details, please refer to the descriptions in the embodiments shown in Figures 1-5. For parts not described in detail in this embodiment, please refer to the relevant descriptions of the embodiments shown in Figures 1-5. The execution process and technical effects of this technical solution are described in the embodiments shown in Figures 1-5, and will not be repeated here.

[0196] Figure 7 is a schematic diagram of a display device provided in an embodiment of the present invention; referring to Figure 7, this embodiment provides a display device that is communicatively connected to a mobile platform; the display device can execute the display methods shown in Figures 1-5 above. Specifically, the display device may include: a first memory 11 and a first processor 12.

[0197] The first memory 11, wherein the memory is used to store one or more computer instructions.

[0198] The first processor 12 is communicatively connected to the first memory 11 and is used to execute one or more computer instructions stored in the first memory 11 to achieve the following steps: acquiring a captured image from a mobile platform, the captured image including the target vessel; displaying the identity information of the target vessel in the captured image; wherein the identity information of the target vessel is determined based on the first location-related information of the target vessel and the second location-related information determined by the ship management system, and the first location-related information is determined based on the position of the mobile platform itself.

[0199] Furthermore, the first processor 12 is also used to execute all or part of the steps in the embodiments shown in Figures 1-5 above.

[0200] The structure of the electronic device may also include a first communication interface 13 for communication between the electronic device and other devices or communication networks.

[0201] Figure 8 is a schematic diagram of the structure of a control device for a mobile platform provided in an embodiment of the present invention. Referring to Figure 8, this embodiment provides a control device for a mobile platform, wherein the mobile platform is communicatively connected to a display device. The control device for the mobile platform can execute the control method for the mobile platform shown in Figure 6 above. Specifically, the control device for the mobile platform may include: a second memory 21 and a second processor 22.

[0202] The second memory 21 is used to store one or more computer instructions.

[0203] The second processor 22, communicatively connected to the second memory 21, is used to execute one or more computer instructions stored in the second memory 21 to achieve the following steps: acquiring a captured image from a mobile platform, the captured image including the target vessel; acquiring first location-related information of the target vessel, the first location-related information being determined based on the location of the mobile platform itself; acquiring second location-related information determined by the ship management system; and transmitting the captured image to a display device so that the display device can display the identity information of the target vessel in the captured image, wherein the identity information of the target vessel is determined based on the first location-related information and the second location-related information.

[0204] Furthermore, the second processor 22 is also used to perform all or part of the steps in the embodiment shown in FIG6 above.

[0205] The structure of the electronic device may also include a second communication interface 23 for communication between the electronic device and other devices or communication networks.

[0206] Figure 9 is a structural schematic diagram of a mobile platform provided in an embodiment of the present invention. Referring to Figure 9, this embodiment provides a mobile platform 200, which is communicatively connected to a display device. The mobile platform 200 may include: a control device 201 for the mobile platform shown in Figure 8 above, which is used to control the display device to display the identity information of the target ship in the captured image based on the image acquisition device.

[0207] The implementation principle and effect of the mobile platform provided in the embodiment shown in Figure 9 are consistent with the implementation principle and effect of the control device of the mobile platform corresponding to Figure 8. For details, please refer to the above description, which will not be repeated here.

[0208] In addition, embodiments of the present invention provide a computer-readable storage medium, wherein the storage medium stores program instructions for implementing the methods in the embodiments corresponding to Figures 1-6 above.

[0209] The technical solutions and features in the above embodiments can be used individually or in combination if they conflict with this invention. As long as they do not exceed the knowledge of those skilled in the art, they are all equivalent embodiments within the scope of protection of this application.

[0210] In the several embodiments provided by this invention, it should be understood that the disclosed remote control devices and methods can be implemented in other ways. For example, the remote control device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection of the remote control device or unit may be electrical, mechanical, or other forms.

[0211] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0212] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0213] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0214] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

[0215] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A display method, characterized in that, Applied to a display device, the display device being communicatively connected to a mobile platform; the method includes: The mobile platform captures images of the target vessel. The target vessel's identity information is displayed in the captured image. The identity information of the target vessel is determined based on the first location-related information of the target vessel and the second location-related information determined by the vessel management system. The first location-related information is determined based on the location of the mobile platform itself.

2. The method according to claim 1, characterized in that, The first location-related information is determined based on the location of the mobile platform itself and the relative positional relationship between the mobile platform and the target vessel.

3. The method according to claim 2, characterized in that, The relative positional relationship includes at least one of the following: the relative angle information between the mobile platform and the target vessel, and the distance information between the mobile platform and the target vessel.

4. The method according to claim 2, characterized in that, The relative positional relationship is determined by a laser ranging module, which is deployed on the movable platform.

5. The method according to claim 2, characterized in that, The positioning accuracy of the first location-related information is greater than or equal to a preset threshold.

6. The method according to claim 1, characterized in that, After acquiring the footage captured by the mobile platform, the method further includes: The image shows at least one vessel included in the captured footage, and the target vessel is among the at least one vessel.

7. The method according to claim 6, characterized in that, The at least one vessel was identified by analyzing and recognizing the captured footage using artificial intelligence (AI) algorithms.

8. The method according to claim 6, characterized in that, The target vessel is the one located in the center of the captured image.

9. The method according to claim 6, characterized in that, The target vessel is a vessel located in a non-center position within the captured image.

10. The method according to claim 6, characterized in that, Displaying at least one vessel in the captured footage, including: The captured image displays the location-related information of each of the at least one vessel, wherein the location-related information of each vessel is determined based on the first location-related information.

11. The method according to claim 10, characterized in that, Before displaying the position-related information of each of the at least one vessel in the captured image, the method further includes: Obtain the relative display position relationship between each ship in the captured image and the target ship; Based on the first location-related information and the relative display position relationship, the location-related information of each ship is determined.

12. The method according to claim 11, characterized in that, The relative display position relationship includes the relative relationship between pixel coordinates located in the captured image.

13. The method according to claim 1, characterized in that, Before displaying the target vessel's identity information in the captured image, the method further includes: Obtain the identity information of the target vessel.

14. The method according to claim 13, characterized in that, Obtaining the identity information of the target vessel includes: The identity information is obtained based on the matching results of the first location-related information and the second location-related information.

15. The method according to claim 14, characterized in that, The second location-related information includes at least the reported location information of the reporting vessel and the identity information of the reporting vessel.

16. The method according to claim 15, characterized in that, Based on the matching results of the first location-related information and the second location-related information, the identity information is obtained, including: When the positioning information of the target vessel, as represented by the first location-related information, matches the reported positioning information, the identity information of the reported vessel is determined as the identity information of the target vessel.

17. The method according to claim 13, characterized in that, The identity information includes at least one of the following: vessel name, vessel type, vessel call sign, port of registry, and registration number.

18. The method according to claim 13, characterized in that, The mobile platform is equipped with an information receiving device, which can be used to receive information sent by the Automatic Identification System (AIS).

19. The method according to claim 18, characterized in that, The mobile platform is connected to the information receiving device via a pre-defined expansion interface.

20. The method according to claim 16, characterized in that, The method further includes: The illegally operating vessels are identified in the captured footage, including those vessels in the captured footage that do not match the reported location information.

21. The method according to any one of claims 1-20, characterized in that, The identity information of the target vessel is displayed in the captured image, including: Determine the display position of the target vessel in the captured image; Based on the display location, the identity information of the target vessel is displayed in the captured image.

22. The method according to claim 21, characterized in that, The display position of the target vessel in the captured image is determined based on camera extrinsic parameters, the target vessel's first position-related information, and the position of the movable platform itself.

23. The method according to claim 21, characterized in that, The target vessel's identity information is displayed on the target vessel in the captured image.

24. The method according to claim 23, characterized in that, The identity information of the target vessel is displayed in the non-target vessel area of ​​the captured image.

25. The method according to any one of claims 1-20, characterized in that, The identity information of the target vessel is displayed in the captured image, including: The identity information of the target vessel is dynamically displayed in the captured image.

26. The method according to claim 25, characterized in that, The identity information of the target vessel is dynamically displayed in the captured image, including: The identity information of the target vessel displayed in the captured image is dynamically updated.

27. The method according to claim 26, characterized in that, The target vessel's identity information is dynamically updated based on the attitude information of the mobile platform and the attitude information of the display device.

28. The method according to any one of claims 1-20, characterized in that, The method further includes: The target vessel's navigation information is displayed in the captured image.

29. The method according to claim 28, characterized in that, The navigation information of the target vessel is displayed in the captured image, including: When the mobile platform follows the target vessel, the navigation information of the target vessel is displayed in the captured image.

30. The method according to claim 28, characterized in that, The navigation information of the target vessel is displayed in the captured image, including: When the target vessel's driving information does not meet the preset driving standards, the target vessel's driving information is displayed differently in the captured image.

31. The method according to claim 30, characterized in that, The method further includes: When the target vessel's driving information does not meet the preset driving standards, a prompt message is output, which can be used to inform the user that the target vessel is in an abnormal driving state.

32. The method according to claim 30, characterized in that, The driving information includes at least one of the following: driving speed and driving route.

33. The method according to claim 28, characterized in that, The method further includes: The captured image displays a ship-view image of the target vessel.

34. A control method for a mobile platform, characterized in that, The mobile platform is communicatively connected to a display device, and the method includes: The mobile platform captures images of the target vessel. Obtain first location-related information of the target vessel, wherein the first location-related information is determined based on the position of the mobile platform itself; Obtain information related to the second location determined by the ship management system; The captured image is transmitted to a display device so that the display device can display the identity information of the target vessel in the captured image, wherein the identity information of the target vessel is determined based on the first location-related information and the second location-related information.

35. The method according to claim 34, characterized in that, Obtaining the first location-related information of the target vessel, including: Obtain the position of the mobile platform itself; Determine the relative positional relationship between the mobile platform and the target vessel; Based on the position of the mobile platform itself and the relative positional relationship, the first position-related information of the target vessel is determined.

36. The method according to claim 35, characterized in that, The relative positional relationship includes at least one of the following: the relative angle information between the mobile platform and the target vessel, and the distance information between the mobile platform and the target vessel.

37. The method according to claim 35, characterized in that, The relative positional relationship is determined by a laser ranging module, which is deployed on the movable platform.

38. The method according to claim 35, characterized in that, The positioning accuracy of the first location-related information is greater than or equal to a preset threshold.

39. The method according to claim 34, characterized in that, After acquiring the captured footage from the mobile platform, the method further includes: Identify at least one vessel included in the captured footage, wherein the target vessel is among the at least one vessel.

40. The method according to claim 39, characterized in that, The at least one vessel was identified by analyzing and recognizing the captured footage using artificial intelligence (AI) algorithms.

41. The method according to claim 39, characterized in that, The target vessel is the one located in the center of the captured image.

42. The method according to claim 39, characterized in that, The target vessel is a vessel located in a non-center position within the captured image.

43. The method according to claim 39, characterized in that, After identifying at least one vessel in the captured footage, the method further includes: Based on the first location-related information, determine the location-related information corresponding to each of the at least one vessel; The location-related information corresponding to each of the at least one vessel is sent to the display device, so that the display device displays the location-related information corresponding to each of the at least one vessel in the captured image.

44. The method according to claim 43, characterized in that, Based on the first location-related information, determine the location-related information corresponding to each of the at least one vessel, including: Obtain the relative display position relationship between each ship in the captured image and the target ship; Based on the first location-related information and the relative display position relationship, the location-related information of each ship is determined.

45. The method according to claim 44, characterized in that, The relative display position relationship includes the relative relationship between pixel coordinates located in the captured image.

46. ​​The method according to claim 34, characterized in that, Before transmitting the captured image to the display device, the method further includes: Obtain the identity information of the target vessel.

47. The method according to claim 46, characterized in that, Obtaining the identity information of the target vessel includes: The identity information is obtained based on the matching results of the first location-related information and the second location-related information.

48. The method according to claim 47, characterized in that, The second location-related information includes at least the reported location information of the reporting vessel and the identity information of the reporting vessel.

49. The method according to claim 48, characterized in that, Based on the matching results of the first location-related information and the second location-related information, the identity information is obtained, including: When the positioning information of the target vessel, as represented by the first location-related information, matches the reported positioning information, the identity information of the reported vessel is determined as the identity information of the target vessel.

50. The method according to claim 46, characterized in that, The identity information includes at least one of the following: vessel name, vessel type, vessel call sign, port of registry, and registration number.

51. The method according to claim 46, characterized in that, The mobile platform is equipped with an information receiving device, which can be used to receive information sent by the Automatic Identification System (AIS).

52. The method according to claim 51, characterized in that, The mobile platform is connected to the information receiving device via a pre-defined expansion interface.

53. The method according to claim 46, characterized in that, The method further includes: The identity information of the target vessel is sent to the display device.

54. The method according to any one of claims 34-53, characterized in that, The method further includes: Obtain the navigation information of the target vessel; The driving information is sent to the display device so that the display device displays the driving information of the target vessel in the captured image.

55. The method according to claim 54, characterized in that, After obtaining the navigation information of the target vessel, the method further includes: The navigation management of the target vessel is carried out based on the driving information.

56. The method according to claim 54, characterized in that, The driving information includes at least one of the following: driving speed and driving route.

57. The method according to claim 54, characterized in that, Sending the driving information to the display device includes: When the driving information of the target vessel does not meet the preset driving standards, the driving information is sent to the display device so that the display device can output a prompt message, which can be used to inform the user that the target vessel is in an abnormal driving state.

58. The method according to any one of claims 34-53, characterized in that, The method further includes: Obtain a ship-view image of the target vessel; The ship's perspective image is sent to the display device so that the display device can display the ship's perspective image of the target ship in the captured image.

59. A display device, characterized in that, The display device is communicatively connected to the mobile platform; the display device includes: a memory and a processor; The memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the display method of any one of claims 1-33.

60. A control device for a mobile platform, characterized in that, The mobile platform is communicatively connected to a display device, and the control device includes: a memory and a processor; The memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the control method of the mobile platform according to any one of claims 34-58.

61. A mobile platform, characterized in that, The communication connection includes a display device; the movable platform includes: The control device for the mobile platform as described in claim 60.

62. A computer-readable storage medium, characterized in that, The storage medium is a computer-readable storage medium that stores program instructions for implementing the method described in any one of claims 1-58.

Images