System and method for recognizing marine object based on radar attention in ship

Abstract

A method for recognizing a marine object based on radar attention in a ship, according to the present invention, comprises the steps of: receiving a transmission signal of a radar reflected from an object; determining horizontal bearing information of the object on the basis of the received signal; generating an object region of interest by applying vertical beam width information of the radar to the horizontal bearing information; applying the object region of interest onto an image captured through a camera installed on the ship; and performing object recognition on the object region of interest on the image.

Description

Radar Attention-Based Marine Object Recognition System and Method on a Ship

[0001] The present invention relates to a radar attention-based maritime object recognition system and method on a ship.

[0002] Ships operating at sea utilize various devices such as radar, lidar, and cameras to recognize objects such as other ships, and based on the recognition results, they set routes to avoid collisions.

[0003] The conventional approach, however, suffers from the problem that radar and camera each acquire data independently and provide their own object recognition results. In other words, radio-based radar and optical imaging-based cameras operate as separate sensors, independently acquiring data and providing object recognition results. This reduces the management of separated data and the accuracy and reliability of object recognition and tracking.

[0004] For example, radar is generally good for long-distance detection and is effective in identifying relatively large objects. However, it has limitations in situations requiring small, distant objects or fine details. Specifically, it has been difficult to identify details such as small vessels or floating objects from a distance, requiring visual identification by a navigator to compensate.

[0005] Furthermore, camera-based object recognition suffers from the significant computing resources required to analyze high-resolution images or videos, making real-time processing difficult. Furthermore, the camera's detection range is limited compared to radar, making proactive object detection and tracking impossible based solely on camera-based object recognition results.

[0006] The problem to be solved by the present invention is to provide a system and method for maritime object recognition based on radar attention on a ship, which enables faster and more precise object recognition at sea by setting a region of interest based on the result of radar-based object recognition and applying it to an image acquired by a camera.

[0007] However, the problems to be solved by the present invention are not limited to the problems described above, and other problems may exist.

[0008] According to a first aspect of the present invention for solving the above-described problem, a method for recognizing a maritime object based on radar attention on a ship comprises the steps of: receiving a radar transmission signal reflected from an object; determining horizontal bearing information of the object based on the received signal; generating an object region of interest by applying vertical beam width information of the radar to the horizontal bearing information; applying the object region of interest to an image captured by a camera installed on the ship; and performing object recognition on the object region of interest on the image.

[0009] Some embodiments of the present invention may further include a step of estimating distance information between the object and the camera; and a step of correcting a point on an image plane corresponding to the estimated distance information in the region of interest of the object to a horizontal lower limit of the region of interest of the object.

[0010] In some embodiments of the present invention, the step of estimating distance information between the object and the camera may estimate distance information corresponding to a signal with the shortest distance among a plurality of signals received in response to the transmission signal of the radar as distance information between the object and the camera.

[0011] In some embodiments of the present invention, the step of generating an object region of interest by applying the vertical beam width information of the radar to the horizontal bearing information may include the step of detecting a horizontal line on an image captured by the camera; and the step of correcting the horizontal line to a horizontal upper limit of the object region of interest.

[0012] Some embodiments of the present invention may further include the steps of: receiving AIS information from another vessel; determining whether a timestamp included in the AIS information matches the reception time of a received signal from the radar; if the determination result matches, extracting pixel information corresponding to an object on the image based on location information and size information of the other vessel included in the AIS information; and correcting a region of interest of the object based on the pixel information corresponding to the extracted object.

[0013] In addition, a radar attention-based maritime object recognition system on a ship according to a second aspect of the present invention includes a radar system that receives a transmission signal of a radar reflected from an object to generate an object detection result, applies vertical beam width information of the radar to horizontal bearing information of the object determined based on the received signal to generate an object region of interest, and a camera system that applies the object region of interest to an image captured by a camera installed on the ship and performs object recognition on the object region of interest.

[0014] In some embodiments of the present invention, the camera system generates feedback information on the result of object recognition performed based on the image and provides the feedback information to the radar system, and the radar system can modify and improve the result of the radar-based object detection by receiving and reflecting the feedback information.

[0015] In some embodiments of the present invention, the radar system can identify fixed obstacle information including land, islands, and navigational buoys on an electronic chart, and provide target information excluding the identified fixed obstacles by matching them with the results of performing the object detection.

[0016] According to another aspect of the present invention for solving the above-described problem, a computer program is coupled with a computer as hardware to execute a radar attention-based maritime object recognition system and method on a ship, and is stored in a computer-readable recording medium.

[0017] Other specific details of the present invention are included in the detailed description and drawings.

[0018] According to one embodiment of the present invention described above, radar and camera information can be fused to provide accurate detection results for maritime objects. Specifically, detailed object information, previously unobtainable through radar, can be obtained through camera information. Furthermore, the system fuses information generated through a mutual detection process, rather than through independent object detection, thereby enabling faster and more accurate object detection.

[0019] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0020] FIG. 1 is a block diagram of a maritime object recognition system according to one embodiment of the present invention.

[0021] FIGS. 2 to 4 are drawings for explaining a radar system in one embodiment of the present invention.

[0022] FIG. 5 is a drawing for explaining the contents of creating an object area of ​​interest in one embodiment of the present invention.

[0023] FIG. 6 is a drawing for explaining the contents for setting a horizontal upper limit in one embodiment of the present invention.

[0024] FIG. 7 and FIG. 8 are drawings for explaining an application example of a maritime object recognition system according to one embodiment of the present invention.

[0025] FIG. 9 is a drawing for explaining the contents of recognizing a maritime object using an electronic chart in one embodiment of the present invention.

[0026] FIG. 10 is a drawing for explaining the contents of recognizing a maritime object using AIS information in one embodiment of the present invention.

[0027] FIG. 11 is a flowchart of a method performed by a radar attention-based maritime object recognition system on a ship according to one embodiment of the present invention.

[0028] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined solely by the scope of the claims.

[0029] The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like components throughout the specification, and "and / or" includes each and any combination of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Therefore, it should be understood that a first component mentioned below may also be a second component within the technical spirit of the present invention.

[0030] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those skilled in the art to which the present invention pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0031] FIG. 1 is a block diagram of a maritime object recognition system according to one embodiment of the present invention.

[0032] A maritime object recognition system according to one embodiment of the present invention includes a radar system and a camera system.

[0033] A radar system receives a radar signal reflected from an object and generates an object detection result. The radar system applied to the present invention relates to a radar system operated on a ship, and differs significantly from a vehicle radar system in terms of configuration and operation. Specifically, vehicle radar is used to detect and track objects within a relatively short distance, operating within a range of several meters to several hundred meters. In contrast, maritime radar can detect objects at a longer range than vehicle radar, with a range of several kilometers. It is used to monitor a wide maritime area and prevent collisions with other vessels.

[0034] In particular, in one embodiment of the present invention, the radar system is characterized in that it creates a region of interest for an object (hereinafter, object region of interest) by applying vertical beam width information of the radar to horizontal bearing information of an object determined based on a received signal.

[0035] The camera system applies an object region of interest generated by the radar system to an image captured by a camera installed on a ship, and performs object recognition targeting the object region of interest.

[0036] Meanwhile, according to an embodiment of the present invention, the radar and camera can be mechanically rigidly fixed. Therefore, even when the vessel moves, the relative position and attitude angle between the two sensors can remain constant. This mechanical fixation minimizes the influence of vessel motion or vibration, enables stable acquisition of sensor data from the radar and camera, and further ensures that the relative relationship between the radar and camera remains constant, enabling continuous acquisition of reliable data for fusion.

[0037] FIGS. 2 to 4 are drawings for explaining a radar system in one embodiment of the present invention.

[0038] Referring to Figure 2, the radar system determines horizontal bearing information (210) of an object based on the received signal. The radar system radiates signals through an antenna, interacts with the object, and the signals are reflected back. The signals received by the antenna contain information about the reflected object, which is expressed as data such as time and signal strength.

[0039] A radar system can calculate the distance to an object by measuring the emission and reception times of a received signal, and can determine the direction in which the radar beam was emitted and received based on the direction of the antenna. The radar system can derive horizontal bearing information (210) for the object by calculating the angle between the distance to the object and the direction of the antenna. The horizontal bearing information (210) is used to display direction information on a horizontal map and can be expressed as angle information.

[0040] FIG. 3 illustrates the vertical beam width information (220) of the radar, which refers to the detection area in the vertical direction, indicating how widely or narrowly the radar beam is radiated in the vertical direction. This vertical beam width information (220) can be expressed in an angular range. In the case of a ship, there are various directions of shaking during operation, such as side-to-side shaking and front-to-back shaking. In this case, if the vertical beam width of the radar is narrow, there may be cases where the target object is not detected and is lost. Therefore, in the case of a maritime radar system, there is a regulation that it must have a vertical beam width of at least a certain degree. One embodiment of the present invention can create an object region of interest, which will be described later, by using this vertical beam width information (220).

[0041] Fig. 4 shows the distance between a radar beam and an object. It is impossible to determine the shape of an object simply by emitting and receiving a radar beam. That is, as shown in Fig. 4, due to the shape of the object, the radar antenna receives signals (230) having different distances. This is because even if the radar beam is radiated with a vertical beam width, the radiated beams are not received in a sequential order that can express the shape of the object.

[0042] Ultimately, unlike LiDAR, radar has poor range and angular resolution, so object detection results are expressed as individual cells on a horizontal map being filled in. This only allows for the acquisition of distance and angle information about the detected object on the horizontal map, and no information about the object's vertical direction. Therefore, only distance and angle information about the object can be confirmed, but specific confirmation of the object is impossible.

[0043] One embodiment of the present invention is characterized in that it can generate an object recognition result that can identify the type, shape, etc. of an object by fusing images captured by a camera to solve such a problem.

[0044] FIG. 5 is a drawing for explaining the contents of creating an object area of ​​interest in one embodiment of the present invention.

[0045] Before performing object recognition on an image captured by a camera installed on a ship, one embodiment of the present invention can generate an object region of interest to be applied to the image.

[0046] First, the initial object area of ​​interest (310) can be generated as a bounding box configured based on horizontal bearing information and vertical beam width information of the radar. That is, the horizontal position of the object can be determined using the horizontal bearing information acquired by the radar system, the vertical position of the object can be determined using the minimum and maximum vertical beam widths of the radar, and the bounding box configured using the information can be set as the object area of ​​interest (310).

[0047] When such an object area of ​​interest (310) is applied to a camera image, faster object recognition is possible on the image, and accurate detailed information such as the size, shape, and type of the object can be confirmed based on the image. In particular, one embodiment of the present invention differs significantly from traditional techniques for setting an area of ​​interest in that it sets an area of ​​interest (310) using radar information rather than setting an area of ​​interest for object recognition using information on the image.

[0048] Furthermore, one embodiment of the present invention can perform a correction process to further precisely reduce the size of the set object region of interest (310). Further reducing the size of the object region of interest (310) has the significant advantage of further reducing the analysis time and computing resources required for object recognition on high-resolution images, thereby further enabling real-time analysis.

[0049] To this end, one embodiment of the present invention estimates distance information between an object and a camera. As illustrated in FIG. 4, the distance information between the object and the camera can be estimated as the distance information corresponding to the signal with the shortest distance among multiple signals received in response to a radar transmission signal.

[0050] Next, the point on the image plane corresponding to the distance information between the estimated object and the camera in the object's area of ​​interest (310) can be corrected to be reduced to the horizontal lower limit of the object's area of ​​interest (320).

[0051] Specifically, for the image plane where the camera's image is generated, the bottom point on the image plane changes depending on the horizontal relative distance between the object and the camera. In this case, the shorter the horizontal relative distance between the object and the camera, the lower the bottom point on the image plane.

[0052] Therefore, one embodiment of the present invention is a signal having the shortest distance from an object among the received signals corresponding to the radar transmission signal (r in FIG. 2c). min ) to calculate minimum distance information, and correct (reduce) the horizontal lower limit, which is the lower part of the radar vertical beam width, to the position of the lower point on the image plane corresponding to the minimum distance information.

[0053] FIG. 6 is a drawing for explaining the contents for setting a horizontal upper limit in one embodiment of the present invention.

[0054] In addition, one embodiment of the present invention can detect a horizontal line (400) on an image captured by a camera and correct the horizontal line (400) to the horizontal upper limit of an object's region of interest (330). When the horizontal line (400) is detected, the object is located between the horizontal line (400) and the camera. In other words, since the object is above water, the horizontal upper limit of the bounding box can be set to the horizontal line (400).

[0055] At this time, one embodiment of the present invention can correct the horizontal upper limit based on distance information to the horizon. As an object existing in an image gets farther away from the system (100), it appears closer to the horizon due to the perspective effect. Using this, pixel information of the horizon can be determined based on distance information from the radar system (110) for the object determined through the maximum distance among the reception distances and distance information to the horizon, and correction of the horizontal upper limit is possible based on the pixel information. That is, pixel information according to the radar distance can be extracted in a proportional relationship from the pixel coordinates of the horizon and the distance information from the radar for the object. This means that as the distance increases, the pixel coordinates of the horizon also increase proportionally, and as the distance decreases, the pixel coordinates also decrease proportionally.

[0056] At this time, distance information to the horizon can be calculated based on Earth radius information. That is, in the former case, distance information to the horizon can be calculated based on height information of the camera system (120) and length information of the Earth radius. Alternatively, distance information to an object can be obtained from the radar system (110) when the object exists above the horizon.

[0057] If the horizon (400) cannot be detected on the image (e.g., when viewing land), the upper horizontal limit can be corrected using electronic chart information (e.g., coastline).

[0058] Referring back to Figure 3, once the object region of interest (ROI) has been created, the camera system applies the ROI to the captured image and then performs object recognition within the ROI. That is, the camera system uses image processing and computer vision technologies to identify objects within the ROI and obtain detailed information about the objects, such as their size, shape, and color.

[0059] Meanwhile, in some embodiments, when the camera system is installed low, the camera looks upward at the horizon. In this case, a portion of the object in the image captured by the camera may be located above the horizon. In other words, a portion of the object may be located below the horizon, while the remaining portion may be located above the horizon. Accordingly, the region of interest generated by the radar system may not encompass the entire area of ​​the object. One embodiment of the present invention handles this case as an exception, enabling the camera system to identify the remaining portion of the object located above the horizon.

[0060] FIG. 7 and FIG. 8 are drawings for explaining an application example of a maritime object recognition system according to one embodiment of the present invention.

[0061] When applying a maritime object recognition system according to one embodiment of the present invention, noise can be removed from object detection results obtained by a radar system. Specifically, the camera system can generate feedback information regarding the results of image-based object recognition and provide it to the radar system. The radar system, upon receiving the feedback information, can then modify and reflect the radar-based object detection results.

[0062] For example, referring to Figure 7, in the radar system-based object detection results, if a sea reflection (510) occurs, it is recognized as an object and displayed on the screen (500). Therefore, unless the navigator visually identifies the object, it is impossible to clearly determine whether it is an actual object.

[0063] In contrast, one embodiment of the present invention creates an object region of interest for a portion detected as an object by a radar system, and the camera system performs object detection only on the region of interest rather than performing vision processing on the entire domain of the image, thereby easily determining whether it is an actual object or noise such as sea reflection or rain reflection. In addition, if it is determined to be noise such as sea reflection, this can be generated as feedback information to remove the noise portion from the screen of the radar system (520).

[0064] As another example, referring to FIG. 8, there are cases where a radar system recognizes a false image (610) generated by a side lobe (600) when there is a strong reflector at a close distance as an object. That is, a radar signal may be composed of a main beam and side lobes (600), which are secondary radar signals generated around the main beam. The main beam is used for actual object detection, but the side lobes (600) may act as a factor of a false image (610) in situations such as ambient noise or multiple reflections.

[0065] Even if such a false image (610) occurs, one embodiment of the present invention has the advantage of being able to easily remove the false image through fusion processing with a camera image.

[0066] FIG. 9 is a drawing for explaining the content of recognizing a maritime object using an electronic chart (700) in one embodiment of the present invention.

[0067] In one embodiment, the radar system can identify fixed obstacle information (710), including land, islands, and navigational buoys, on an electronic chart (700). Fixed obstacle information (710), such as land, islands, and navigational buoys, is pre-established on the electronic chart. Therefore, after identifying such fixed obstacle information (710), the radar system can match the remaining target information, excluding the identified fixed obstacles, with the results of object detection and provide the information.

[0068] As a result, the maritime object recognition system can quickly generate detection results without considering fixed obstacle information (710) provided on the chart when detecting an object. The radar system matches this with the object recognition results recognized by the camera system and provides this, allowing navigators to immediately check image information (720) when clicking on an object of interest, and has the advantage of being able to check not only motion information but also class and shape information (size) at a glance.

[0069] FIG. 10 is a drawing for explaining the contents of recognizing a maritime object using AIS information in one embodiment of the present invention.

[0070] Meanwhile, when a maritime object recognition system according to one embodiment of the present invention receives AIS (Automatic Identification System) information from another vessel (800), it can perform recognition of another vessel more quickly by further reducing the object area of ​​interest by further utilizing the AIS information.

[0071] Specifically, when receiving AIS information from another vessel (800), it is determined whether the timestamp included in the received AIS information matches the reception time of the radar signal. If there is no timestamp in the received AIS information, it can be determined whether there is a significant match with the AIS reception time. Not all vessels are equipped with AIS, and the AIS information transmission cycle varies from 2 seconds to as long as 10 minutes depending on the moving speed and hazard level. Therefore, there is a problem in terms of reliability when recognizing the location of another vessel and performing collision avoidance, etc. using only AIS information. However, since AIS information includes various static information about other vessels (ship name, vessel type, length, width, identification number, etc.), dynamic information (vessel position, UTC time, ground course, ground speed, sailing status, turning speed, etc.), and navigation information, utilizing such information when receiving AIS information is very advantageous in terms of object recognition. Additionally, when receiving AIS information, objects can be accurately identified and effectively classified based on static and dynamic information contained in the AIS information in densely packed objects.

[0072] Accordingly, one embodiment of the present invention can extract pixel information corresponding to an object on an image based on the location and size information of another vessel included in the AIS information at a point in time when the radar reception time and the AIS information reception time substantially coincide. Furthermore, based on the pixel information corresponding to the extracted object, the horizontal upper and lower limits of the object's region of interest can be corrected to be closest to the object's size.

[0073] In this way, by further reducing the object area of ​​interest to correspond to the object's location and size information at the moment when the radar signal and AIS information reception time coincide, reliability can be guaranteed while simultaneously enabling faster image processing.

[0074] Furthermore, one embodiment of the present invention can construct training data for AI model-based object recognition by labeling object regions of interest (ROIs) generated based on radar signals and AIS information and object recognition results based on these regions of interest. Using the constructed training data to train an AI model has the advantage of generating object regions of interest that better match the size of real objects.

[0075] FIG. 11 is a flowchart of a method performed by a radar attention-based maritime object recognition system (100) on a ship according to one embodiment of the present invention.

[0076] In one embodiment of the present invention, first, a transmission signal of a radar reflected on an object is received (S910).

[0077] Next, based on the received signal, horizontal bearing information of the object is determined (S920), and the vertical beam width information of the radar is applied to the horizontal bearing information to create an object area of ​​interest (S930).

[0078] Next, an object area of ​​interest is applied to an image captured by a camera installed on a ship (S940), and object detection and identification are performed targeting the object area of ​​interest on the image (S950).

[0079] Meanwhile, in the above description, steps S910 to S950 may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present invention. Furthermore, some steps may be omitted as needed, and the order of steps may be changed. Furthermore, the contents of FIGS. 1 to 10 and FIG. 11 may be mutually applicable.

[0080] The embodiments of the present invention described above have been explained based on an example in which a ship on which a radar is installed does not rock left and right or back and forth. In the case of rocking of the ship, the ship's navigation information (position and attitude angle) is utilized to compensate for the movement of the body on which the radar and camera are fixed, thereby enabling stable and accurate data acquisition.

[0081] Meanwhile, embodiments of the present invention may be implemented as a program (or application) to be executed in combination with a hardware computer and stored in a medium.

[0082] The above-described program may include codes coded in a computer language, such as C, C++, JAVA, Ruby, or machine language, that can be read by the processor (CPU) of the computer through the device interface of the computer, so that the computer reads the program and executes the methods implemented as a program. Such codes may include functional codes related to functions that define functions necessary to execute the methods, and may include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. In addition, such codes may further include memory reference-related codes regarding which location (address address) of the internal or external memory of the computer should reference additional information or media necessary for the processor of the computer to execute the functions. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to send and receive during communication.

[0083] The above storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program can be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. In addition, the medium can be distributed across network-connected computer systems, so that computer-readable code can be stored in a distributed manner.

[0084] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0085] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

1. In a method performed by a computer, A step of receiving a radar transmission signal reflected from an object; A step of determining horizontal bearing information of an object based on the received signal; A step of generating an object region of interest by applying the vertical beam width information of the radar to the horizontal bearing information; A step of applying the object area of interest to an image captured by a camera installed on the ship; and Comprising a step of performing object recognition on an object area of interest on the image above. A method for maritime object recognition based on radar attention on a ship.

2. In paragraph 1, A step of estimating distance information between the object and the camera; and Further comprising a step of correcting a point on an image plane corresponding to the estimated distance information in the region of interest of the object to a horizontal lower limit of the region of interest of the object. A method for maritime object recognition based on radar attention on a ship.

3. In paragraph 2, The step of estimating the distance information between the above object and the camera is: In response to the transmission signal of the above radar, the distance information corresponding to the signal with the shortest distance among the multiple signals received is estimated as the distance information between the object and the camera. A method for maritime object recognition based on radar attention on a ship.

4. In paragraph 2, The step of generating an object area of interest by applying the vertical beam width information of the radar to the horizontal bearing information is as follows. A step of detecting a horizontal line on an image captured by the above camera; and Comprising a step of correcting the above horizontal line to the horizontal upper limit of the object area of interest, A method for maritime object recognition based on radar attention on a ship.

5. In paragraph 4, The step of correcting the above horizontal line to the horizontal upper limit of the object area of interest is: A step of calculating distance information to the above-mentioned horizon; A step of determining pixel information of the detected horizon using distance information to the horizon and radar-based distance information for the object; and Comprising a step of correcting the horizontal upper limit based on pixel information of the determined horizontal line, A method for maritime object recognition based on radar attention on a ship.

6. In paragraph 1, Step of receiving AIS information from another vessel; A step of determining whether the timestamp included in the above AIS information and the reception time of the received signal of the radar match; If the above judgment result matches, a step of extracting pixel information corresponding to the object on the image based on the location information and size information of the other vessel included in the AIS information; and Further comprising a step of correcting the region of interest of the object based on pixel information corresponding to the extracted object. A method for maritime object recognition based on radar attention on a ship.

7. A radar system that generates an object detection result by receiving a radar transmission signal reflected from an object, and generates an object area of interest by applying the vertical beam width information of the radar to the horizontal bearing information of the object determined based on the received signal. A camera system that applies the object region of interest to an image captured by a camera installed on the ship and performs object recognition targeting the object region of interest. Radar attention-based maritime object recognition system on ships.

8. In paragraph 7, The above camera system generates feedback information on the result of object recognition performed based on the above image and provides it to the radar system. The above radar system modifies and reflects the radar-based object detection results upon receiving the above feedback information. Radar attention-based maritime object recognition system on ships.

9. In paragraph 7, The above radar system identifies fixed obstacle information including land, islands and navigation buoys on an electronic chart, and matches the remaining target information excluding the identified fixed obstacles with the results of performing the object detection, and provides the information. Radar attention-based maritime object recognition system on ships.

Images