Ship monitoring system, method and program

Abstract

To provide a ship monitoring system, a ship monitoring method, and a ship monitoring program capable of determining in real time.SOLUTION: A ship monitoring system 1 has an analysis device 3 that includes a CPU 30 configured to process an image from a thermal camera 2 and a storage unit 31 configured to store the image from the thermal camera 2. The CPU 30 includes a contour extraction unit 300, a moving object detection unit 301, a filter processing unit 302, and an AI discrimination unit 303. The contour extraction unit 300 calculates temperature differences in an image and extracts contours of objects in the image. The moving object detection unit 301 compares pixel values at the same position in relation to another image obtained at a time different from the time when the image was acquired, and extracts contours of objects that are moving objects. The filter processing unit 302 executes filter processing on the objects and extracts objects that are ship candidates. The AI discrimination unit 303 executes image analysis processing using artificial intelligence on the objects and discriminates ships from the objects.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a ship monitoring device, method, and program, and in particular to a ship monitoring system, a ship monitoring method, and a ship monitoring program that analyze images captured by a surveillance camera to monitor ships within a monitoring area. [Background technology]

[0002] For example, Patent Document 1 and Patent Document 2 are known as devices and methods for monitoring ships navigating within a monitoring area. Patent Document 1 describes a device that determines the presence or absence of a navigating ship by a ship sensor (using ultrasonic waves or electromagnetic waves) that receives a signal from a water level gauge and rises and falls according to the fluctuation of the water level. Patent Document 2 describes a device that determines the presence or absence of a navigating ship from the acceleration value obtained from the acceleration sensor unit by installing several floating bodies (buoys) equipped with an acceleration sensor unit, a wireless communication means, and a self-contained power source within the monitoring area. However, since Patent Document 1 and Patent Document 2 both use sensors that are installed within the monitoring area, if the monitoring area is wide or there are many monitoring areas, it is necessary to install many sensors. In addition, the cost of maintaining such many sensors is also required. Therefore, in Patent Document 1 and Patent Document 2, it is difficult to install them inexpensively when the monitoring area is wide.

[0003] On the other hand, for example, Patent Document 3 is known as an apparatus and method for monitoring a monitoring area by analyzing images acquired from a camera without using the above-mentioned sensor. Patent Document 3 describes an automatic poacher recognition system that includes a person / animal silhouette database storing images of approaching objects captured by a thermography monitoring camera and storing the images of the silhouettes of people and animals, and a means for determining recreational fishing permit holders or poachers by referring to the database. However, Patent Document 3 requires that the silhouettes of people and animals be stored in a database in advance in order to match them with the images of approaching objects acquired from the camera. In order to match the data stored in such a database with the images of approaching objects, a server or computer with a relatively high processing capacity is required. In addition, since ships have various shapes compared to people and animals, it is difficult to store data on all silhouettes of such various ship shapes in the database described in Patent Document 3 in advance.

[0004] Generally, as the size of the image to be analyzed increases, it takes time to process the data, making it difficult to determine in real time whether an approaching object is a ship or not. In particular, when artificial intelligence (AI) is used to identify ships on a computer with relatively low processing power, it takes more time to determine whether an object is a ship or not. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 7-108987 [Patent Document 2] JP 2009-211570 A [Patent Document 3] JP 2006-268471 A Summary of the Invention [Problem to be solved by the invention]

[0006] There is a demand for a ship monitoring system and method that does not require sensors such as ultrasonic sensors or electromagnetic wave sensors, and can determine in real time whether or not an object is a ship, even with a computer having relatively low processing power, without comparing the data with a database that stores silhouette data, etc. of the object in image analysis processing. [Means for solving the problem]

[0007] According to the present invention, the following ship monitoring system can be provided. That is, a thermal camera for acquiring images of a monitored area at predetermined time intervals; An analysis device including a central processing unit that processes the image received from the thermal camera, and a storage unit that stores the image received from the thermal camera, the central processing unit including a contour extraction unit, a moving object detection unit, a filter processing unit, and an AI discrimination unit for processing the image. A vessel monitoring system comprising: The contour extraction unit calculates a temperature difference in the image and extracts a contour of an object in the image; the moving object detection unit compares pixel values ​​at the same positions between two images captured at different times, the image being captured at the same time, to extract a contour of the moving object; The filtering unit performs filtering on the objects to extract the objects that may be candidates for ships; The AI ​​discrimination unit performs image analysis processing using artificial intelligence (AI) on the object and discriminates a ship from the object. It is possible to provide a ship monitoring system characterized by the above.

[0008] Here, the contour extraction unit may further obtain a contour of the object from a temperature difference equal to or greater than a predetermined threshold value. Also, the moving object detection unit may process the image using a background subtraction method. The filter processing unit: a spatio-temporal filter that compares pixel values ​​at the same position between a target frame at time t corresponding to the image and another frame at time t-1 and time t-2, which are different from the time t, and determines a frame having a minimum brightness as an analysis target; a horizon filter that calculates the position of a horizon from the installation position and depression angle of the thermal camera and excludes the object located above the horizon from analysis; A size filter that calculates the size of the object, which is the moving body, from the installation position and depression angle of the thermal camera, and excludes the object from analysis if the size is less than a predetermined size; a standard deviation filter that calculates a standard deviation of the pixel values ​​and excludes the pixel values ​​from analysis if the pixel values ​​are less than a predetermined value; a moving direction filter for calculating a moving direction of the object, which is a moving body; A mask filter that excludes areas where ships cannot pass from the analysis The embodiment may include at least one of the above.

[0009] The AI ​​discrimination unit may execute a process of attaching a mark to distinguish the ship in the image. Here, it is preferable that the AI ​​discrimination unit includes a program having an algorithm using a VGG16 model structure with a modified number of intermediate layers, and that the AI ​​discrimination unit repeats learning using images including ships photographed in different weather conditions or different seasons.

[0010] In addition, according to the present invention, the following ship monitoring method can be provided. That is, A ship monitoring method in a ship monitoring system, comprising: The ship monitoring system includes: a thermal camera for acquiring images of a monitored area at predetermined time intervals; An analysis device including a central processing unit that processes the image received from the thermal camera, and a storage unit that stores the image received from the thermal camera, the central processing unit including a contour extraction unit, a moving object detection unit, a filter processing unit, and an AI discrimination unit for processing the image. Contains calculating a temperature difference in the image by the contour extraction unit and extracting a contour of an object in the image; a step of comparing pixel values ​​at the same positions between an image captured at a time different from the time the image was captured by the moving object detection unit, and extracting a contour of the object that is a moving object; A step of performing a filter process on the objects by the filter processing unit, and extracting the objects that may be candidates for ships; executing an image analysis process using artificial intelligence (AI) on the object by the AI ​​discrimination unit, and discriminating a ship from the object; A method for monitoring a vessel may be provided, comprising:

[0011] Here, the contour extraction unit may further obtain the contour of the object from a temperature difference equal to or greater than a predetermined threshold value. Also, the moving object detection unit may further process the image using a background subtraction method. The filter processing unit: a spatio-temporal filter that compares pixel values ​​at the same position between a target frame at time t corresponding to the image and another frame at time t-1 and time t-2, which are different from the time t, and determines a frame having a minimum brightness as an analysis target; a horizon filter that calculates the position of a horizon from the installation position and depression angle of the thermal camera and excludes the object located above the horizon from analysis; A size filter that calculates the size of the object from the installation position and depression angle of the thermal camera, and excludes the object from analysis if the size is less than a predetermined size; a standard deviation filter that calculates a standard deviation of the pixel values ​​and excludes the pixel values ​​from analysis if the pixel values ​​are less than a predetermined value; A travel direction filter for calculating a travel direction of the object; A mask filter that excludes areas where ships cannot pass from the analysis The embodiment may include at least one of the above.

[0012] The AI ​​discrimination unit may further execute a process of attaching a mark to distinguish the ship in the image, and the AI ​​discrimination unit may include a program having an algorithm using a model structure in which the number of intermediate layers is changed from the VGG16 model structure. Here, it is preferable that the AI ​​discrimination unit is trained by repeatedly learning using images including ships taken in different weather conditions or different seasons.

[0013] Furthermore, according to the present invention, it is possible to provide a computer-readable program that can be stored in the memory unit and causes the central processing unit to execute any of the ship monitoring methods described above. Effect of the Invention

[0014] It is possible to provide a ship monitoring system that has few false positives due to people or waves and can determine in real time whether or not an object is a ship, even with a computer having a relatively low processing power. [Brief description of the drawings]

[0015] [Figure 1] 1 is a diagram showing a ship monitoring system 1 according to an embodiment of the present invention. [Diagram 2] 4 is a flowchart showing the flow of processing executed in the ship monitoring system 1. [Diagram 3] 4 is a flowchart showing the contents of each process executed by the analysis device 3 of the ship monitoring system 1. [Figure 4]13 is a flowchart showing the contents of a contour extraction process S31 performed in the contour detection unit 300. [Diagram 5] 13 is a flowchart showing the contents of a moving object detection process S32 performed in the moving object detection section 301. [Figure 6] FIG. 2 is a diagram showing a background difference screen obtained by using a background difference method to detect a moving object region in the vessel monitoring system 1. [Figure 7] 13 is a flowchart showing steps for setting a mask function of the filter process S33 performed in the filter processor 312. [Figure 8] 8 is a diagram showing an image displayed on the display device 4 when the mask function of FIG. 7 is set. FIG. [Figure 9] FIG. 1 is a diagram showing a VGG16 model structure that can be used for image analysis processing in the AI ​​discrimination unit 313 of the ship monitoring system 1. [Figure 10] FIG. 1A is a diagram showing a screen shot taken in summer when a ship is detected using the ship monitoring system 1 according to an embodiment of the present invention. FIG. 1B is a diagram showing a screen shot taken in winter when a ship is detected using the ship monitoring system 1 according to an embodiment of the present invention. [Figure 11] 1A is a conceptual diagram showing a cycle for improving the detection rate by relearning to determine whether or not an object is a ship in the AI ​​determination unit, and FIG. 1B is a diagram showing that the detection rate is improved by repeating the relearning cycle shown in A. [Figure 12] 1 is a diagram showing a screen displayed on a display device 4 of a ship monitoring system 1 according to an embodiment of the present invention when a ship is detected. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] A ship monitoring system 1 according to an embodiment of the present invention will be described with reference to FIG. 1. The ship monitoring system 1 includes a thermal camera 2 that captures images of a monitoring area such as the sea or a river, an analysis device 3 that receives the images captured by the thermal camera 2, and a display device 4 that displays the results of analysis by the analysis device 3. The analysis device 3 includes a central processing unit (CPU) 30 that performs calculations such as image analysis processing, and a storage unit 31 that is connected to the CPU 30 and stores analysis programs and various data for analysis and results. The CPU 30 includes a contour extraction unit 300, a moving object detection unit 301, a filter processing unit 302, and an AI discrimination unit 303. The contour extraction unit 300, the moving object detection unit 301, the filter processing unit 302, and the AI ​​discrimination unit 303 are programs that analyze the images captured by the thermal camera 2 in cooperation with the CPU 30. The analyzed images are displayed on the display device 4 connected to the analysis device 3. Here, the thermal camera 2 and the analysis device 3 may be connected by wire, for example, a communication cable, or may be connected wirelessly via an Internet line. Similarly, the analysis device 3 and the display device 4 may be connected by wire, for example, a communication cable, or may be connected wirelessly via an Internet line.

[0017] The process flow of the image analysis process executed by the ship monitoring system 1 will be described with reference to the flowchart of FIG. 2. In step S1, the contour of an object is extracted by image analysis to detect a moving object. Specifically, the contour extraction unit 300 calculates the temperature difference from the image received from the thermal camera 2 to extract the contour of the object. Next, the moving object detection unit 301 detects whether there is a movement by applying a background subtraction method to the image including the contour of the extracted object. Incidentally, it is also possible to detect the movement first and then extract the contour. In step S2, ship candidates are extracted from the detected moving objects. Specifically, the filter processing unit 302 extracts moving objects that are ship candidates from the moving objects detected in step S1. Then, the filter processing unit 302 creates a small-sized image including the moving object that is the ship candidate. In step S3, the moving object image of the ship candidate is discriminated by AI. Specifically, the AI ​​discrimination unit 303 processes the small-sized image created in step S2 and discriminates whether the moving object that is the ship candidate is a ship or not. In step S4, the identified ship is marked in real time together with the time of detection. Specifically, the AI ​​discrimination unit 303 marks the moving object that is the ship obtained in step S3 in the image received from the thermal camera 2. For example, marking here includes surrounding the object that is the ship with a circle, a square, or the like. The AI ​​discrimination unit 303 can also store the time when the object that is the ship obtained in step S3 was actually detected.

[0018] The contents of each process executed by the analysis device 3 of the vessel monitoring system 1 will be described in more detail with reference to the flowchart in Fig. 3. In step S30, the image acquired from the thermal camera 2 is stored in the storage unit 31 of the analysis device 3. Here, the CPU 30 can also store in the storage unit 31 a video (video data) such as a series of images continuously captured by the thermal camera 2. In step S31, the contour extraction unit 300 extracts the contour of an object from temperature differences in a video (video data) such as a series of images. In step S32, the moving object detection unit 301 executes a process of determining whether or not an object in the video is a moving object. If the moving object detection unit 301 determines that the object in the video is a moving object, the process proceeds to step S33. On the other hand, if the moving object detection unit 301 determines that the object in the video is not a moving object (i.e., if the video contains only stationary objects), the process proceeds to step S34, where the process for the video ends without doing anything. In step S33, the filter processing unit 302 executes a filter process on the video including the moving object. Here, if the filter processing unit 302 determines that the moving object is a candidate ship, the process proceeds to step S35. On the other hand, if the filter processing unit 302 determines that the moving object is not a candidate ship, the process proceeds to step S34, where the process for that video ends without doing anything. Next, in step S35, the AI ​​discrimination unit 303 executes an image analysis process using AI according to a predetermined model structure on the video including the candidate ship (see FIG. 9 for the predetermined model structure). Here, if the AI ​​discrimination unit 303 determines that the candidate ship is a ship, the process proceeds to step S36, where the AI ​​discrimination unit 303 executes a process of attaching a mark to the part of the video that is determined to be a ship. On the other hand, if the AI ​​discrimination unit 303 determines that the candidate ship is something other than a ship, the process proceeds to step S34, where the process for that video ends without doing anything.

[0019] Next, the contents of the contour extraction process S31 executed by the contour extraction unit 300 will be described with reference to the flowchart of FIG. 4. In step S31-1, as shown in FIG. 4A, an image is captured from the thermal camera 2. In step S31-2, as shown in FIG. 4B, a temperature difference in the image captured from the thermal camera 2 in step S31-1 is calculated. Such a temperature difference can be obtained, for example, from the difference in the amount of energy of infrared rays (wavelength: about 750 nm to about 1 mm) emitted from an object in the image. In addition, the display device 4 can display the temperature difference in the image using different colors according to the size of such a temperature difference. For example, a threshold value corresponding to a predetermined temperature difference can be set, and an area exceeding the threshold value can be displayed in a different color. Then, in step S31-3, the contour of the object is extracted as shown in FIG. 4C.

[0020] Next, the contents of the moving object detection process S32 executed by the moving object detection unit 301 will be described with reference to the flowchart of FIG. 5. In step S32-1, a live image is captured from the thermal camera. Specifically, as shown in FIG. 5A, a live image (video data that may include a series of still images) including the contour of a moving object is captured from the thermal camera 2. Note that FIG. 5A shows an example of an image in which the moving object is a vehicle. In step S32-2, a background image is calculated. Specifically, as shown in FIG. 5B, a background image is calculated by removing the image of the vehicle from FIG. 5A. Such a background image can be obtained by, for example, comparing images before and after a certain image in a series of images in FIG. 5A to identify an area in which the position has not changed (a stationary area). Then, in step S32-3, a moving object is detected from the difference between the live image and the background image. Specifically, as shown in FIG. 5C, an image of a moving object (here, a vehicle as an example) can be obtained by calculating the difference in pixel values ​​between the live image in FIG. 5A and the background image in FIG. 5B. Such background subtraction methods are described at https: / / algorithm.joho.info / image-processing / background-subtraction-method / .

[0021] With reference to FIG. 6, a screen obtained by using the background subtraction method when detecting a moving ship in the ship monitoring system 1 will be described. Here, the "background subtraction method" is a method for detecting a moving object with high accuracy by creating a background image in which no moving object exists from a captured image and calculating the difference (such as the temperature difference) between the captured image and the background image. Comparing FIG. 4C in which the object contour is extracted in step S31-3 with FIG. 6 corresponding to the difference between the live image and the background image in step S32-3, it can be seen that the contour of a moving ship is displayed in a different color and more emphasized than the contours of stationary objects such as a quay or rocks by calculating the temperature difference. Therefore, by using the background subtraction method in moving object detection, a moving ship can be found with high accuracy.

[0022] With reference to the flowchart of FIG. 7, each step of setting the mask function of the filter processing S33 performed in the filter processing unit 302 will be described. In step S33-1, the mask function is set in an area where ships do not pass. In step S33-2, the area where the mask function is set is specified on the setting screen. For such an area where the mask function is set, the user can specify the area where the mask function is set using a mouse or the like on the setting screen of the display device 4, as shown in FIG. 8A, for example. Then, in step S33-3, even if a moving object is detected in the area where the mask function is set, the process ends without proceeding to the subsequent flow. Specifically, as shown in FIG. 8B, when a moving object is detected in the area where the mask function is set in FIG. 8A, the process for the moving object ends without performing subsequent processing.

[0023] Here, the filter processing unit 302 may have a spatio-temporal filter that compares pixel values ​​at the same position between a target frame at time t corresponding to the image and another frame at time t-1 and time t-2, which are different times from time t, and determines the frame with the minimum brightness as the frame to be analyzed. Furthermore, the filter processing unit 302 may have a horizon filter that calculates the position of the horizon from the installation position and depression angle of the thermal camera 2, and excludes the object located above the horizon from the analysis target. Furthermore, the filter processing unit 302 may have a size filter that calculates the size of the moving object from the installation position and depression angle of the thermal camera 2, and excludes the object from analysis if the size is less than a predetermined size. In addition, the filter processing unit 302 may have a standard deviation filter that calculates the standard deviation of the pixel values ​​and excludes pixel values ​​that are less than a predetermined value from the analysis. The filter processing unit 302 may also include a movement direction filter that calculates the movement direction of the object, which is a moving body. The filter processing unit 302 may have a mask filter that excludes areas where ships cannot pass from the analysis target. The filtering unit 302 may include at least one of the filters described above.

[0024] Next, an example of a model structure that can be used for image analysis processing in the AI ​​discrimination unit 303 of the ship monitoring system 1 will be described with reference to Fig. 9. Here, Fig. 9 shows the VGG16 model structure proposed by a research group at Oxford University. This VGG16 model structure is a neural network consisting of a total of 16 layers, including 13 convolutional layers and 3 fully connected layers, and is used in transfer learning, etc. Such a VGG16 model structure is, for example, It is described at https: / / newtechnologylifestyle.net / vgg16originalpicture / . The AI ​​discrimination unit 303 of the ship monitoring system 1, which is an embodiment of the present application, uses the model structure shown in Table 1, which is obtained by changing the number of intermediate layers in the VGG16 model structure shown in Fig. 9. Here, "intermediate layers" refers to all layers in the VGG16 model structure shown in Fig. 9, except for the softmax layer shown as "1x1x1000". In other words, the intermediate layers include two "convolution+ReLU" layers displayed as "224×224×64", one "max pooling" layer and two "convolution+ReLU" layers displayed as "112×112×128", one "max pooling" layer and three "convolution+ReLU" layers displayed as "56×56×256", one "max pooling" layer and three "convolution+ReLU" layers displayed as "28×28×512", one "max pooling" layer and three "convolution+ReLU" layers displayed as "14×14×512", one "max pooling" layer displayed as "7×7×512", two "fully connected+ReLU" layers displayed as "1×1×4096", and one "fully connected+ReLU" layer displayed as "1×1×1000". The AI ​​discrimination unit 303 then performs learning on the model structure shown in Table 1 using approximately 8,000 images of ships. [Table 1]

[0025] FIG. 10A is a diagram showing a screen captured during a ship detection in summer (July 23, 2019) using the ship monitoring system 1 according to an embodiment of the present invention. FIG. 10B is a diagram showing a screen captured during a ship detection in winter (December 5, 2019) using the ship monitoring system 1 according to an embodiment of the present invention. Note that, when comparing FIG. 10A with FIG. 10B, the sea surface conditions vary greatly depending on the weather and season. In other words, if there is a weather or seasonal bias in the ship images used in learning for the AI ​​discrimination unit 303 of the ship monitoring system 1, the image recognition accuracy of the ship during the demonstration experiment may be extremely reduced. Therefore, it is preferable to perform learning using images of ships and waves that are not biased in terms of weather or season, for example.

[0026] Now, with reference to FIG. 11A, the re-learning performed by the AI ​​discrimination unit 303 of the ship monitoring system 1 will be described. FIG. 11A shows a method of repeatedly learning using 8,000 images of ships and waves without weather or seasonal bias in the model structure shown in Table 1 (in which the number of intermediate layers in the VGG16 model structure has been changed). Next, FIG. 11B shows the image recognition accuracy of ships when the model structure shown in Table 1 is used to re-learn as shown in FIG. 11A. FIG. 11B shows the image recognition accuracy of ships when 8,000 images of ships are repeatedly re-learned up to 50 times, with the number of re-learning times corresponding to the horizontal axis. FIG. 11B shows that the image recognition accuracy tends to improve as the number of re-learning times increases.

[0027] Next, the actual ship detection rate results when the AI ​​discrimination unit 303 re-learned in Figure 11A using the model structure shown in Table 1 are shown below (here, the AI ​​discrimination unit 303 repeatedly learned 50 times using approximately 8,000 ship images (corresponding to the graph on the far right of Figure 11B)). The actual ship detection rate and accuracy rate both achieved an accuracy of over 90%. Total number of detections (including detection of waves as ships): 68 Number of ships correctly detected: 64 Number of times waves or ships were detected: 4 Number of times no ships were detected: 5 Accuracy rate for all detections: 64 / 68 = 94.1% Detection rate for all ships: 64 / 69 = 92.8%

[0028] With reference to FIG. 12, a screen displayed on the display device 4 of the vessel monitoring system 1 according to an embodiment of the present invention when a vessel is detected will be described. FIG. 12 includes an image 41 of the monitoring area photographed by the thermal camera 2, a slide bar 42 for searching for images of the monitoring area photographed by the thermal camera 2, and a column 43 indicating the date and time when the image 41 was photographed in conjunction with the slide bar 42. Here, FIG. 12 displays a vessel with a mark (surrounded by a square) indicating that the CPU 30 and the AI ​​discrimination unit 303 have detected the vessel. In this way, the vessel monitoring system 1 according to an embodiment of the present invention can display an image of the vessel with a predetermined mark on the display device 4. In addition, by searching for an image of a vessel with a predetermined mark, the date and time when the vessel was photographed (the date and time when the vessel was sailing) can be easily confirmed on the display device 4.

[0029] Although the present invention has been particularly described herein with reference to certain embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention, and that those skilled in the art will recognize that numerous modifications can be made to the illustrative embodiments and that other configurations can be adopted without departing from the spirit and scope of the present invention as defined by the appended claims. In order to maintain the disclosure matters at the time of filing of the present application, the contents of claims 1 to 15 at the time of filing of the present application are added as follows. (Claim 1) a thermal camera for acquiring images of a monitored area at predetermined time intervals; An analysis device including a central processing unit that processes the image received from the thermal camera, and a storage unit that stores the image received from the thermal camera, the central processing unit including a contour extraction unit, a moving object detection unit, a filter processing unit, and an AI discrimination unit for processing the image. A vessel monitoring system comprising: The contour extraction unit calculates a temperature difference in the image and extracts a contour of an object in the image; the moving object detection unit compares pixel values ​​at the same positions between two images captured at different times, the image being captured at the same time, to extract a contour of the moving object; The filtering unit performs filtering on the objects to extract the objects that may be candidates for ships; The AI ​​discrimination unit performs image analysis processing using artificial intelligence (AI) on the object and discriminates a ship from the object. A vessel monitoring system comprising: (Claim 2) The vessel monitoring system according to claim 1 , further comprising: the contour extraction unit determining a contour of the object from a temperature difference equal to or greater than a predetermined threshold value. (Claim 3) The vessel monitoring system according to claim 1 , wherein the motion detector further processes the images using a background subtraction method. (Claim 4) The filter processing unit includes: a spatio-temporal filter that compares pixel values ​​at the same position between a target frame at time t corresponding to the image and another frame at time t-1 and time t-2, which are different from the time t, and determines a frame having a minimum brightness as an analysis target; a horizon filter that calculates the position of a horizon from the installation position and depression angle of the thermal camera and excludes the object located above the horizon from analysis; A size filter that calculates the size of the object from the installation position and depression angle of the thermal camera, and excludes the object from analysis if the size is less than a predetermined size; a standard deviation filter that calculates a standard deviation of the pixel values ​​and excludes the pixel values ​​from analysis if the pixel values ​​are less than a predetermined value; a moving direction filter for calculating a moving direction of the object, which is a moving body; A mask filter that excludes areas where ships cannot pass from the analysis The vessel monitoring system of claim 1 , comprising at least one of: (Claim 5) The ship monitoring system of claim 1 , wherein the AI ​​discrimination unit performs a process of attaching a mark to distinguish the ship in the image. (Claim 6) The ship monitoring system of claim 1, wherein the AI ​​discrimination unit includes a program having an algorithm using a VGG16 model structure with a modified number of hidden layers. (Claim 7) The vessel monitoring system of claim 6, wherein the AI ​​discrimination unit repeats learning using images including vessels photographed in different weather conditions or different seasons. (Claim 8) A ship monitoring method in a ship monitoring system, comprising: The ship monitoring system includes: a thermal camera for acquiring images of a monitored area at predetermined time intervals; An analysis device including a central processing unit that processes the image received from the thermal camera, and a storage unit that stores the image received from the thermal camera, the central processing unit including a contour extraction unit, a moving object detection unit, a filter processing unit, and an AI discrimination unit for processing the image. Contains calculating a temperature difference in the image by the contour extraction unit and extracting a contour of an object in the image; a step of comparing pixel values ​​at the same positions between an image captured at a time different from the time the image was captured by the moving object detection unit, and extracting a contour of the object that is a moving object; A step of performing a filter process on the objects by the filter processing unit, and extracting the objects that may be candidates for ships; The AI ​​discrimination unit executes an image analysis process using artificial intelligence (AI) for the object, and discriminates a ship from the object. A ship monitoring method comprising the steps of: (Claim 9) 9. The vessel monitoring method according to claim 8, wherein the contour extraction unit further obtains the contour of the object from a temperature difference equal to or greater than a predetermined threshold value. (Claim 10) 9. The vessel monitoring method according to claim 8, wherein the moving object detection unit further processes the image using a background subtraction method. (Claim 11) The filter processing unit includes: a spatio-temporal filter that compares pixel values ​​at the same position between a target frame at time t corresponding to the image and another frame at time t-1 and time t-2, which are different from the time t, and determines a frame having a minimum brightness as an analysis target; a horizon filter that calculates the position of a horizon from the installation position and depression angle of the thermal camera and excludes the object located above the horizon from analysis; A size filter that calculates the size of the object from the installation position and depression angle of the thermal camera, and excludes the object from analysis if the size is less than a predetermined size; a standard deviation filter that calculates a standard deviation of the pixel values ​​and excludes the pixel values ​​from analysis if the pixel values ​​are less than a predetermined value; A travel direction filter for calculating a travel direction of the object; A mask filter that excludes areas where ships cannot pass from the analysis 9. The vessel monitoring method of claim 8, further comprising at least one of: (Claim 12) 9. The vessel monitoring method according to claim 8, wherein the AI ​​discrimination unit further performs a process of attaching a mark to distinguish the vessel in the image. (Claim 13) The ship monitoring method according to claim 8, wherein the AI ​​discrimination unit includes a program having an algorithm using a VGG16 model structure with a modified number of hidden layers. (Claim 14) 14. The ship monitoring method of claim 13, wherein the AI ​​discrimination unit is trained by repeatedly learning using images including ships taken in different weather conditions or different seasons. (Claim 15) A computer-readable program that can be stored in the storage unit and causes the central processing unit to execute the ship monitoring method according to any one of claims 8 to 14. [Explanation of symbols]

[0030] 1. Ship monitoring system 2. Thermal camera 3 Analysis device (analysis server) 30 Central processing unit (CPU) 31 Storage section 300 Contour Extraction Unit 301 Motion detection unit 302 Filter processing section 303 AI discrimination section 4 Display device

Claims

1. a thermal camera for acquiring images of a monitored area at predetermined time intervals; An analysis device including a central processing unit that processes the image received from the thermal camera, and a storage unit that stores the image received from the thermal camera, the central processing unit including a contour extraction unit, a moving object detection unit, a filter processing unit, and an AI discrimination unit for processing the image. A vessel monitoring system comprising: The contour extraction unit calculates a temperature difference in the image and extracts a contour of an object in the image; the moving object detection unit detects whether the object is a moving object from a difference in pixel values ​​obtained by comparing pixel values ​​at the same positions between an image captured at a time different from the time the image was captured, and The filtering unit performs filtering on the video including the moving object, and extracts the object that may be a candidate for a ship; The AI ​​discrimination unit performs image analysis processing using artificial intelligence (AI) on the object that may be a candidate for a ship, and discriminates the ship from the object. A vessel monitoring system comprising:

2. The vessel monitoring system according to claim 1 , further comprising: the contour extraction unit determining a contour of the object from a temperature difference equal to or greater than a predetermined threshold value.

3. The vessel monitoring system according to claim 1 , wherein the motion detector further processes the images using background subtraction.

4. The filter processing unit includes: a spatio-temporal filter that compares pixel values ​​at the same position between a target frame at time t corresponding to the image and another frame at time t-1 and time t-2, which are different from the time t, and determines the frame having the smallest pixel value between the target frame and the other frame as the frame to be analyzed; a horizon filter that calculates the position of the horizon from the installation position and depression angle of the thermal camera, and excludes the object located above the horizon from the analysis; a size filter that calculates the size of the object from the installation position and depression angle of the thermal camera and excludes the object whose size is less than a predetermined size from analysis; a standard deviation filter that calculates a standard deviation of the pixel values ​​and excludes the object from analysis if the pixel value is less than the standard deviation value; A mask filter that excludes areas where ships cannot pass from the analysis The vessel monitoring system of claim 1 , comprising at least one of:

5. The vessel monitoring system according to claim 1 , wherein the AI ​​discrimination unit performs a process of attaching a mark to distinguish the vessel in the image.

6. The vessel monitoring system according to claim 1 , wherein the AI ​​discrimination unit includes a program having an algorithm using a VGG16 model structure with a modified number of hidden layers.

7. The vessel monitoring system according to claim 6 , wherein the AI ​​discrimination unit repeats learning using images including vessels photographed in different weather conditions or different seasons.

8. A ship monitoring method in a ship monitoring system, comprising: The ship monitoring system includes: a thermal camera for acquiring images of a monitored area at predetermined time intervals; An analysis device including a central processing unit that processes the image received from the thermal camera, and a storage unit that stores the image received from the thermal camera, the central processing unit including a contour extraction unit, a moving object detection unit, a filter processing unit, and an AI discrimination unit for processing the image. Contains calculating a temperature difference in the image by the contour extraction unit and extracting a contour of an object in the image; a step of detecting, by the moving object detection unit, whether the object is a moving object based on a difference in pixel values ​​obtained by comparing pixel values ​​at the same positions between an image captured at a time different from the time the image was captured; a step of performing a filter process on the video including the moving object by the filter processing unit, and extracting the object that may be a candidate for a ship; a step of executing an image analysis process using artificial intelligence (AI) on the object that may be a candidate for a ship by the AI ​​discrimination unit, and discriminating the ship from the object; A ship monitoring method comprising the steps of:

9. The vessel monitoring method according to claim 8 , wherein the contour extraction unit further obtains the contour of the object from a temperature difference equal to or greater than a predetermined threshold value.

10. The method for monitoring vessels according to claim 8 , wherein the moving object detection unit further processes the images using a background subtraction method.

11. The filter processing unit includes: a spatio-temporal filter that compares pixel values ​​at the same position between a target frame at time t corresponding to the image and another frame at time t-1 and time t-2, which are different from the time t, and determines a frame having a minimum pixel value between the target frame and the other frame as an analysis target; a horizon filter that calculates the position of a horizon from the installation position and depression angle of the thermal camera and excludes the object located above the horizon from analysis; a size filter that calculates the size of the object from the installation position and depression angle of the thermal camera and excludes the object whose size is less than a predetermined size from analysis; a standard deviation filter that calculates a standard deviation of the pixel values ​​and excludes the object whose pixel value is less than a predetermined value from the analysis; A mask filter that excludes areas where ships cannot pass from the analysis 9. The vessel monitoring method of claim 8, further comprising at least one of:

12. The vessel monitoring method according to claim 8 , wherein the AI ​​discrimination unit further performs a process of attaching a mark to distinguish the vessel in the image.

13. The vessel monitoring method according to claim 8 , wherein the AI ​​discrimination unit includes a program having an algorithm using a VGG16 model structure with a modified number of intermediate layers.

14. The vessel monitoring method according to claim 13, wherein the AI ​​discrimination unit is trained by repeatedly learning using images including vessels photographed in different weather conditions or different seasons.

15. A computer-readable program that can be stored in the storage unit and causes the central processing unit to execute the ship monitoring method according to any one of claims 8 to 14.

Images