Marine camera apparatus optimized for maritime environments

Abstract

A marine camera device optimized for a marine environment is disclosed. The marine camera device comprises: optical cameras for capturing optical images in different directions; infrared cameras for capturing thermal images in different directions; a main body case having a plurality of camera holes formed on the front, right side, and left side, respectively; and an integrated bracket inserted into the main body case with the optical cameras and the infrared cameras combined on the upper surface, wherein the optical cameras and the infrared cameras can be combined such that at least one of the optical cameras and at least one of the infrared cameras are positioned on the front, right side, and left side, respectively, of the main body case.

Description

Technology Field

[0001] The present invention relates to a marine camera device optimized for marine environments, and more specifically, to a marine camera device capable of perceiving the situation in marine environments both day and night. Background Technology

[0002] Due to the nature of the maritime environment, situations with poor visibility frequently occur, and there are instances where ships must navigate even in adverse weather conditions.

[0003] When a vessel navigates in adverse weather conditions, it relies on radar reflections from surrounding objects; however, if a hazardous object is not detected by radar or exists at close range (from the vicinity of the vessel to several hundred meters) within a radar blind spot, the risk of collision cannot be effectively notified to the user, significantly increasing the danger.

[0004] Conventional marine cameras had difficulty determining the consistency of each piece of information because they positioned daytime optical cameras (EO, electro-optical) and nighttime infrared cameras (IR, infrared) at different heights and locations.

[0005] Therefore, there is a demand for a marine camera device that reduces errors occurring during the alignment process between optical and thermal images. The problem to be solved

[0006] The present invention provides a marine camera device to replace human lookout by widely distributing optical cameras to satisfy the 225-degree lookout requirement of the Safety of Life at Sea Convention (SOLAS) and designing an external camera module suitable for harsh marine environments according to the distributed optical cameras.

[0007] In addition, the present invention provides a marine camera device that reduces errors occurring during the alignment process of optical and thermal images by horizontally arranging an optical camera and an infrared camera in each direction, thereby capturing optical and thermal images in the same direction and at the same height.

[0008] In addition, the present invention includes an Ethernet communication module that transmits images transmitted by cameras via a USB cable to a server on a ship equipped with a marine camera device using Ethernet communication, and by allowing the ship's server to process the images, it provides a camera device with high operational reliability compared to conventional camera devices, which are susceptible to errors caused by thermal temperature conditions in a marine environment and unstable voltage conditions during the process of processing images with an embedded processor.

[0009] In addition, the present invention provides a marine camera device in which an optical camera and an infrared camera are combined and an integrated bracket is inserted into a main body case formed as a single layer structure, thereby allowing the optical camera and the infrared camera to be serviced simply by removing the integrated bracket during maintenance.

[0010] Furthermore, the present invention provides a method for stabilizing a panoramic image by projecting optical images or thermal images onto a cylindrical coordinate system plane to generate a panoramic image, so that a discontinuous state does not occur in the matching area of ​​the panoramic image even when the orientation of the optical cameras and infrared cameras is changed by the rolling of the ship. means of solving the problem

[0011] A marine camera device according to one embodiment of the present invention comprises: optical cameras for capturing optical images in different directions; infrared cameras for capturing thermal images in different directions; a main body case having a plurality of camera holes formed on the front, right side, and left side, respectively; and an integrated bracket inserted into the main body case with the optical cameras and the infrared cameras coupled thereto on the upper surface, wherein the optical cameras and the infrared cameras may be coupled to the integrated bracket such that at least one of the optical cameras and at least one of the infrared cameras are positioned on the front, right side, and left side, respectively, of the main body case.

[0012] A marine camera device according to an embodiment of the present invention The above integrated bracket is combined with an Ethernet communication module that outputs information received via a USB (Universal Serial Bus) cable via Ethernet communication, and the Ethernet communication module can transmit thermal images transmitted by the infrared cameras via a USB cable and optical images transmitted by the optical cameras via a USB cable to a server on a vessel equipped with a marine camera device using Ethernet communication.

[0013] The server of the marine camera device according to one embodiment of the present invention can generate a panoramic image by projecting the optical images and the thermal images onto a cylindrical coordinate system plane.

[0014] The integrated bracket of the marine camera device according to one embodiment of the present invention may further include an IMU (Inertial Measurement Unit) that measures the movement of a vessel on which the marine camera device is installed and outputs the measured result as IMU sensor data.

[0015] The panoramic image can be corrected based on the IMU sensor data of the marine camera device according to one embodiment of the present invention.

[0016] A marine camera device according to one embodiment of the present invention further includes a sun visor coupled to the upper part of the main body case, and the sun visor may be manufactured with an aerodynamic shape that takes into account wind resistance.

[0017] The main body case of a marine camera device according to one embodiment of the present invention may have an engraved pattern for heat dissipation formed on its upper surface.

[0018] The main body case of a marine camera device according to one embodiment of the present invention may include a camera waterproof structure in which a waterproof sealing of the hole, a transparent window, a waterproof sealing of the transparent window, and a fixing case are sequentially coupled to the front of each of the camera holes.

[0019] A marine camera device according to one embodiment of the present invention further includes a stand for fixing the main body case to a ship, wherein the stand includes a plurality of horizontal rotation holes formed long in a horizontal rotation direction on an upper surface, and a bolt passing through the horizontal rotation holes is inserted and coupled into a hole formed in the lower part of the main body, and the range of horizontal rotation of the stand may be determined according to the length of the horizontal rotation holes.

[0020] The tilting part of a marine camera device according to one embodiment of the present invention includes a plurality of tilting holes formed according to a tilting angle in the coupling plate, and a bolt passing through the tilting holes is inserted and coupled to the vertical plate of the horizontal rotation part, and the tilting range of the stand can be determined according to the length and shape of the tilting holes.

[0021] An image stabilization method for a marine camera device according to an embodiment of the present invention includes the steps of: receiving camera images generated by the cameras of the marine camera device from the marine camera device; and projecting the camera images onto a cylindrical coordinate system plane to generate a panoramic image, wherein the marine camera device may have at least one optical camera and at least one infrared camera disposed on the front, right side, and left side of a main body case, respectively.

[0022] The camera images of the image stabilization method of a marine camera device according to one embodiment of the present invention may include at least one of optical images generated by the optical cameras capturing different directions and thermal images generated by the infrared cameras capturing different directions.

[0023] A method for image stabilization of a marine camera device according to one embodiment of the present invention further comprises: a step of identifying whether the surrounding environment of a vessel on which the marine camera device is installed is rough water; and a step of correcting the panoramic image based on IMU sensor data when the surrounding environment of the vessel on which the marine camera device is installed is not rough water, wherein the IMU sensor data may be generated by the Inertial Measurement Unit (IMU) of the marine camera device measuring the movement of the vessel on which the marine camera device is installed.

[0024] The step of correcting based on the IMU sensor data of the image stabilization method for a marine camera device according to one embodiment of the present invention can correct the panoramic image by applying a coordinate transformation between the individual camera coordinate system of each of the optical cameras and infrared cameras and the origin coordinate system of the vessel on which the marine camera device is installed, based on the IMU sensor data, to the panoramic image.

[0025] An image stabilization method for a marine camera device according to one embodiment of the present invention may further include the step of generating a Surround View Monitoring (SVM) image using the camera images; and the step of correcting the SVM image based on the IMU sensor data. Effects of the invention

[0026] According to one embodiment of the present invention, by widely distributing optical cameras to satisfy the 225-degree lookout requirement of the SOLAS Convention and designing an external camera module suitable for harsh marine environments according to the distributed optical cameras, a marine camera device to replace human lookout can be provided.

[0027] In addition, according to one embodiment of the present invention, by horizontally arranging an optical camera and an infrared camera in each direction, an optical image and a thermal image can be captured at the same direction and at the same height, thereby providing a marine camera device that reduces errors occurring during the alignment process of the optical image and the thermal image.

[0028] In addition, according to one embodiment of the present invention, an Ethernet communication module is included to transmit images transmitted by cameras via a USB cable to a server on a ship equipped with a marine camera device using Ethernet communication, and by allowing the server on the ship to process the images, a camera device with high operational reliability can be provided compared to a conventional camera device in which errors may occur due to thermal temperature conditions in a marine environment and unstable voltage conditions during the process of processing images with an embedded processor.

[0029] In addition, according to one embodiment of the present invention, by inserting an integrated bracket combining an optical camera and an infrared camera into a main body case formed as a single layer structure, the optical camera and the infrared camera can be serviced simply by removing the integrated bracket during maintenance, thereby making maintenance easier compared to conventional camera devices in which cameras are each placed in a multi-layer structure.

[0030] In addition, according to one embodiment of the present invention, by generating a panoramic image by projecting optical images or thermal images onto a cylindrical coordinate system plane, the panoramic image can be stabilized so that a discontinuous state does not occur in the matching area of ​​the panoramic image even if the orientation of the optical cameras and infrared cameras is changed by the rolling of the ship. Brief explanation of the drawing

[0031] FIG. 1 is a drawing illustrating a marine camera device according to an embodiment of the present invention. FIG. 2 is a drawing showing the detailed configuration of a marine camera device according to an embodiment of the present invention. FIG. 3 is a drawing illustrating a waterproof structure of a camera for marine use according to an embodiment of the present invention. FIG. 4 is a drawing illustrating the detailed configuration of the shooting unit of a marine camera device according to an embodiment of the present invention. FIG. 5 is a drawing showing the lower view of a marine camera device according to an embodiment of the present invention. Figure 6 is an example of a panoramic image generated by a conventional marine camera device. FIG. 7 is a drawing showing a ship equipped with a marine camera device according to an embodiment of the present invention. FIG. 8 is a diagram illustrating the process of a marine camera device generating a panoramic image according to an embodiment of the present invention. FIG. 9 is an example of a process in which a server of a ship equipped with a marine camera device according to an embodiment of the present invention corrects an SVM image. FIG. 10 is an example of an SVM image captured by a marine camera device according to an embodiment of the present invention. FIG. 11 is an example of an image corrected by a server of a ship equipped with a marine camera device according to an embodiment of the present invention based on IMU sensor data. FIG. 12 is a flowchart illustrating an image stabilization method for a marine camera device according to an embodiment of the present invention. Specific details for implementing the invention

[0032] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, various modifications may be made to the embodiments, and thus the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, and substitutions to the embodiments are included within the scope of the rights.

[0033] The terms used in the embodiments are for illustrative purposes only and should not be interpreted as intended to be limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0034] In addition, when describing with reference to the attached drawings, identical components are assigned the same reference numeral regardless of drawing symbols, and redundant descriptions thereof are omitted. In describing the embodiments, if it is determined that a detailed description of related prior art could unnecessarily obscure the essence of the embodiments, such detailed description is omitted.

[0035] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0036] FIG. 1 is a drawing illustrating a first embodiment of a marine camera device according to an embodiment of the present invention.

[0037] A marine camera device according to one embodiment of the present invention may be composed of a main body (110) and a stand (120) that fixes the main body (110) to a ship.

[0038] FIG. 2 is a drawing showing the detailed configuration of the main body (110) of a marine camera device according to an embodiment of the present invention.

[0039] The main body (110) may be composed of a main body case (210), a main body bottom surface (220), and a shooting unit (240).

[0040] The main body case (210) is formed as a single-layer structure as shown in FIG. 2, and a plurality of camera holes may be formed on the front, right side, and left side, respectively. At this time, the right side and left side of the main body case (210) may be formed such that the angle from the right side to the left side is 225 degrees or more. In addition, an engraved pattern (211) for heat dissipation may be formed on the upper surface of the main body case (210) as shown in FIG. 2.

[0041] The lower surface of the main body (220) may have a shooting unit (240) attached to the middle of the upper surface and a main body case (210) attached to the outer edge of the upper surface. Additionally, the lower part of the lower surface of the main body (220) may be attached to a stand (120). Furthermore, the lower surface of the main body (220) may be manufactured so that a portion of the lower surface is open to expose the interface of the shooting unit (240).

[0042] The sun visor (230) is attached to the upper part of the main body case (210) to form shade on the main body case (210) and prevent sunlight from shining directly on the main body case (210), thereby preventing the shooting unit (240) from overheating. The sun visor (230) can be manufactured with an aerodynamic shape that takes into account wind resistance. Specifically, as shown in FIG. 2, the sun visor (230) is manufactured with a structure that is flat without a step in the front-rear direction, and has a step formed in the left-right direction with the central part protruding upward, thereby minimizing the lift force generated by the wind blowing from the front of the main body (110).

[0043] The shooting unit (240) may include optical cameras and infrared cameras. The shooting unit (240) may generate panoramic images based on images captured by the optical cameras and infrared cameras. At this time, the generated panoramic images may be transmitted to a server on a vessel equipped with a marine camera device through the interface of the shooting unit (240).

[0044] FIG. 3 is a drawing illustrating a waterproof structure of a camera for marine use according to an embodiment of the present invention.

[0045] A waterproof camera structure can be formed by sequentially combining a waterproof sealing (310) of the hole, a transparent window (320), a waterproof sealing (330) of the transparent window, and a fixing case (340) on the front of each camera hole formed in the main body case (210). Additionally, by fitting a rubber cover (350) on the front of the fixing case (340), the bolts used to connect the fixing case (340) to the camera hole can be prevented from being exposed.

[0046] At this time, a groove to which camera waterproof structures are to be attached may be formed on the front of each camera hole of the main body case (210) according to the area of ​​the fixed case (340) as shown in FIG. 4. In addition, the depth of the groove to which the camera waterproof structures are to be attached may be the thickness of the waterproof sealing (310) of the hole + the thickness of the fixed case (340) + the thickness of the rubber cover (350).

[0047] FIG. 4 is a drawing illustrating the detailed configuration of the shooting unit of a marine camera device according to an embodiment of the present invention.

[0048] The shooting unit (240) may be composed of an integrated bracket (480) having optical cameras (410, 430, 450) and infrared cameras (420, 440, 460) combined on the upper surface.

[0049] Optical cameras (410, 430, 450) can capture optical images in different directions. In addition, infrared cameras (420, 440, 460) can capture thermal images in different directions.

[0050] The integrated bracket (480) may be configured such that at least one of the optical cameras (410, 430, 450) and at least one of the infrared cameras (420, 440, 460) are positioned on the front, right side, and left side of the main body case (210), respectively, so that the optical cameras (410, 430, 450) and the infrared cameras (420, 440, 460) are combined.

[0051] Specifically, by combining an optical camera (410) and an infrared camera (420) at the position of an integrated bracket (480) corresponding to two camera holes on the front of the main body case (210), the optical camera (410) and the infrared camera (420) can each capture an optical image and a thermal image of the front of the marine camera device. Additionally, by combining an optical camera (430) and an infrared camera (440) at the position of an integrated bracket (480) corresponding to two camera holes on the right side of the main body case (210), the optical camera (430) and the infrared camera (440) can each capture an optical image and a thermal image of the right side of the marine camera device. And, by combining an optical camera (450) and an infrared camera (460) at the position of an integrated bracket (480) corresponding to two camera holes on the left side of the main body case (210), the optical camera (450) and the infrared camera (460) can each capture an optical image and a thermal image of the left side of the marine camera device.

[0052] In other words, by horizontally arranging an optical camera and an infrared camera in each direction, the marine camera device can capture optical images and thermal images from the same direction and at the same height.

[0053] At this time, the spacing between the optical cameras (410, 430, 450) and the infrared cameras (420, 440, 460) may all be the same, or they may differ depending on the type of camera. Specifically, the arrangement of the optical cameras (410, 430, 450) and the infrared cameras (420, 440, 460) may be determined by considering the characteristic that the field of view of the optical camera is larger than the field of view of the infrared camera. For example, since the angle between the optical camera (410) and the optical camera (450) is influenced by the field of view of the optical camera, and the angle between the infrared camera (420) and the infrared camera (440) is influenced by the field of view of the infrared camera, the optical camera (410) and the optical camera (450) may be arranged so that the angle between the optical camera (410) and the optical camera (450) is smaller than the angle between the infrared camera (420) and the infrared camera (440). Additionally, the infrared camera (420) and the infrared camera (440) may be positioned such that the angle between the infrared camera (420) and the infrared camera (440) is greater than the angle between the optical camera (410) and the optical camera (450).

[0054] In addition, an IMU (Inertial Measurement Unit) (470) for measuring the movement of a ship equipped with a marine camera device can be combined with the integrated bracket (480).

[0055] The IMU (470) can measure the movement of a ship equipped with a marine camera device and output the measured result as IMU sensor data.

[0056] At this time, the integrated bracket (480) can be manufactured so that cameras of the same type are placed adjacently. Specifically, as shown in FIG. 4, the optical camera (410) and the optical camera (450) are placed adjacently, and the infrared camera (420) and the infrared camera (440) are placed adjacently, thereby securing placement space for the IMU (470) and IR conversion boards (425, 445, 465).

[0057] The IR conversion board (425) is connected to the infrared camera (420) and can generate a thermal image of the infrared camera (420) by converting the infrared signal output from the infrared camera (420) into a digital signal. Additionally, the IR conversion board (445) is connected to the infrared camera (440) and can generate a thermal image of the infrared camera (440) by converting the infrared signal output from the infrared camera (440) into a digital signal. Furthermore, the IR conversion board (465) is connected to the infrared camera (460) and can generate a thermal image of the infrared camera (460) by converting the infrared signal output from the infrared camera (460) into a digital signal.

[0058] The IR conversion boards (425, 445, 465) are connected to an Ethernet communication module via a USB cable and can transmit thermal images to the Ethernet communication module via the USB cable. According to an embodiment, an IR conversion board may be embedded in each of the infrared cameras (420, 440, 460), and the infrared cameras (420, 440, 460) may transmit thermal images to the Ethernet communication module via a USB cable.

[0059] Additionally, an Ethernet communication module that outputs information received via a USB (Universal Serial Bus) cable via Ethernet communication may be combined with the integrated bracket (480). At this time, the Ethernet communication module can transmit thermal images transmitted via USB cables by infrared cameras (420, 440, 460) and optical images transmitted via USB cables by optical cameras (410, 430, 450) to a server on a vessel equipped with a marine camera device using Ethernet communication. The server on the vessel equipped with the marine camera device may be an object detection device that detects objects located on the water surface using optical images captured by optical cameras (410, 430, 450) and thermal images captured by infrared cameras (420, 440, 460), or a server equipped with an object detection program.

[0060] The Ethernet communication module can issue a synchronized timestamp and, based on the timestamp, synchronize and output optical images captured by optical cameras (410, 430, 450), thermal images captured by infrared cameras (420, 440, 460), and IMU sensor data output by the IMU (470).

[0061] The integrated bracket (480) may include a processor that outputs optical images captured by optical cameras (410, 430, 450) and thermal images captured by infrared cameras (420, 440, 460) with horizon correction or stabilization.

[0062] A processor in the server of a vessel equipped with a marine camera device can generate a panoramic image by projecting optical images captured by optical cameras (410, 430, 450) and thermal images captured by infrared cameras (420, 440, 460) onto a cylindrical coordinate system plane. Additionally, the processor can identify the surrounding environment of the vessel equipped with the marine camera device. If the surrounding environment of the vessel equipped with the marine camera device is not rough seas, the processor can correct the panoramic image based on IMU sensor data and then output it. Furthermore, if the surrounding environment of the vessel equipped with the marine camera device is rough seas, the processor can output the panoramic image without correction. Rough seas is a weather classification that refers to a dangerous sea condition characterized by high waves.

[0063] FIG. 5 is a drawing showing the lower view of a marine camera device according to an embodiment of the present invention.

[0064] The interface (510) of the imaging unit (240) is configured to be connected to an Ethernet communication cable, and in FIG. 5, it is exposed to the outside through an open space at the bottom of the main body bottom surface (220), but depending on the embodiment, it may be exposed to the outside through the rear or side of the main body case (210). For example, the interface (510) may be composed of seven sockets for outputting optical images captured by optical cameras (410, 430, 450), thermal images captured by infrared cameras (420, 440, 460), and IMU sensor data output by the IMU (470).

[0065] The upper surface of the stand (120) may include a plurality of horizontal rotation holes (520) formed in a horizontal rotation direction as shown in FIG. 5. A bolt (530) passing through the horizontal rotation holes (520) may be inserted and coupled into a hole formed in the lower part of the lower surface (220) of the main body. At this time, the range of horizontal rotation of the stand (120) may be determined according to the length and degree of curvature of the horizontal rotation holes (520).

[0066] Figure 6 is an example of a panoramic image generated by a conventional marine camera device.

[0067] A panoramic image generated by a conventional marine camera device is created by synthesizing an image (610) captured by a left camera, an image (620) captured by a central camera, and an image (630) captured by a right camera. At this time, by including an overlapping alignment area (640) between the image (610) and the image (620), and an overlapping alignment area (650) between the image (620) and the image (630), the occurrence of discontinuities in the image is prevented.

[0068] However, if rolling occurs as the ship shakes due to waves, a difference in the sea level in each of the images captured by the left camera (610), the central camera (620), and the right camera (630) may occur, which may cause problems in the panoramic image.

[0069] For example, if the ship is tilted to the left by waves, the left camera moves closer to the sea surface compared to the state where no rolling occurs, so the sea surface included in the image (610) may be at a higher position compared to the sea surface where no rolling occurs. Also, since the right camera moves further away from the sea surface compared to the state where no rolling occurs, the sea surface included in the image (630) may be at a lower position compared to the sea surface where no rolling occurs. Furthermore, since the front camera takes pictures while tilted according to the angle of the ship, the sea surface included in the image (610) may have a slope according to the angle of the ship.

[0070] Accordingly, the sea surface of the panoramic image generated by synthesizing images (610), (620), and (630) while the ship is tilted to the left by waves may have discontinuous states in the matching area (640) and matching area (650) as shown in FIG. 6.

[0071] FIG. 7 is a drawing showing a ship equipped with a marine camera device according to an embodiment of the present invention.

[0072] A server (720) may be installed on a vessel (700) on which a marine camera device (710) is installed. The processor of the server (720) can generate a panoramic image using optical images or thermal images received from the marine camera device (710) via Ethernet communication. At this time, the marine camera device (710) may be installed at the bow of the vessel (700) as shown in Case 1 of FIG. 7, or on the upper part of the bridge (730) of the vessel (700) as shown in Case 2 of FIG. 7. Additionally, depending on the embodiment, a marine camera device (710) may be additionally installed on the side or stern of the vessel (700).

[0073] Additionally, the processor of the server (720) can correct the panoramic image based on IMU sensor data received from the marine camera device (710) via Ethernet communication. Specifically, the processor can correct the panoramic image by applying a coordinate transformation between the individual camera coordinate systems of each of the optical cameras and infrared cameras and the origin coordinate system of the vessel (700) to the panoramic image based on the IMU sensor data.

[0074] When a Surround View Monitoring (SVM) system including a marine camera device (710) is installed on a vessel (700), the processor of the server (720) can generate an SVM image using optical images and thermal images received from multiple marine camera devices (710). Additionally, the processor of the server (720) can correct the SVM image based on IMU sensor data received from multiple marine camera devices (710).

[0075] FIG. 8 is a diagram illustrating the process of a marine camera device generating a panoramic image according to an embodiment of the present invention.

[0076] The processor of the server of a ship equipped with a marine camera device can generate a panoramic image by projecting optical images or thermal images onto a cylindrical coordinate system plane (800) as shown in FIG. 8, thereby preventing discontinuity from occurring in the alignment area of ​​the panoramic image even if the orientation of the optical cameras and infrared cameras is changed by the rolling of the ship.

[0077] Therefore, the processor of the server on a vessel equipped with a marine camera device can provide a seamless ultra-wide angle view even in situations where the vessel is rolling.

[0078] FIG. 9 is an example of a process in which a server of a ship equipped with a marine camera device according to an embodiment of the present invention corrects an SVM image.

[0079] A server on a ship equipped with a marine camera device can receive images (910) captured by each of the marine camera devices installed at different locations on the ship.

[0080] A server on a ship equipped with a marine camera device can generate an SVM image (920) using images among the received images (910) in which the score threshold indicating reliability is 50 or higher, and can correct the SVM image (920) using IMU sensor data.

[0081] For example, a server on a ship equipped with a marine camera device can output an image as shown in Fig. 11 by correcting an SVM image as shown in Fig. 10 based on IMU sensor data.

[0082] FIG. 12 is a flowchart illustrating an image stabilization method for a server of a ship equipped with a marine camera device according to an embodiment of the present invention.

[0083] In step (1210), the processor can receive camera images from the cameras of the marine camera device. At this time, the camera images may be one of optical images generated by multiple optical cameras capturing different directions, and thermal images generated by multiple infrared cameras capturing different directions.

[0084] In step (1220), the processor can generate a panoramic image by projecting the camera images received in step (1210) onto a cylindrical coordinate system plane.

[0085] In step (1230), the processor can identify whether the surrounding environment of the vessel equipped with the marine camera device is rough. If the surrounding environment of the vessel equipped with the marine camera device is not rough, the processor can perform step (1250). If the surrounding environment of the vessel equipped with the marine camera device is rough, the processor can perform step (1240).

[0086] In step (1240), the processor can correct the panoramic image generated in step (1220) based on the IMU sensor data.

[0087] In step (1240), the processor can generate an SVM image using the camera images received in step (1210). Additionally, the processor can correct the SVM image based on IMU sensor data.

[0088] The present invention can provide a marine camera device to replace human lookout by widely distributing optical cameras to satisfy the 225-degree lookout requirement of the SOLAS convention and designing an external camera module suitable for harsh marine environments according to the distributed optical cameras.

[0089] The marine camera device of the present invention can reduce errors occurring during the alignment process between optical and thermal images by horizontally arranging an optical camera and an infrared camera in each direction, thereby capturing optical and thermal images from the same direction and at the same height.

[0090] In addition, the present invention includes an Ethernet communication module that transmits images transmitted by cameras via a USB cable to a server on a ship equipped with a marine camera device using Ethernet communication, and by allowing the ship's server to process the images, it provides a camera device with high operational reliability compared to conventional camera devices, which are susceptible to errors caused by thermal temperature conditions in a marine environment and unstable voltage conditions during the process of processing images with an embedded processor.

[0091] In addition, the present invention allows for maintenance of the optical camera and the infrared camera by inserting an integrated bracket combined with the optical camera and the infrared camera into a main body case formed as a single layer structure, thereby enabling maintenance of the optical camera and the infrared camera simply by removing the integrated bracket. This makes maintenance easier compared to conventional camera devices in which cameras are placed in separate multi-layer structures.

[0092] In addition, the present invention can stabilize a panoramic image by generating a panoramic image by projecting optical images or thermal images onto a cylindrical coordinate system plane, so that a discontinuous state does not occur in the matching area of ​​the panoramic image even if the orientation of the optical cameras and infrared cameras is changed by the rolling of the ship.

[0094] Although this specification contains details of a number of specific embodiments, they should not be understood as limiting the scope of any invention or claimables, but rather as descriptions of features that may be characteristic of a specific embodiment of a specific invention. Specific features described in this specification in the context of individual embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any appropriate sub-combination. Furthermore, while features may operate in a specific combination and be described as initially claimed, one or more features from the claimed combination may be excluded from the combination in some cases, and the claimed combination may be changed to a sub-combination or a variation of the sub-combination.

[0095] Likewise, although operations are depicted in the drawings in a specific order, this should not be understood as requiring that such operations be performed in that specific or sequential order depicted to obtain a desirable result, or that all depicted operations must be performed. In certain cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various device components of the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and devices can generally be integrated together into a single software product or packaged into multiple software products.

[0096] Meanwhile, the embodiments of the present invention disclosed in this specification and drawings are merely specific examples provided to aid understanding and are not intended to limit the scope of the present invention. It is obvious to those skilled in the art that other variations based on the technical concept of the present invention are possible in addition to the embodiments disclosed herein.

Claims

Claim 1 A marine camera device comprising: optical cameras for capturing optical images in different directions; infrared cameras for capturing thermal images in different directions; a main body case having a plurality of camera holes formed on the front, right side, and left side, respectively; and an integrated bracket inserted into the main body case with the optical cameras and the infrared cameras combined on the upper surface, wherein the optical cameras and the infrared cameras are combined such that at least one of the optical cameras and at least one of the infrared cameras are positioned on the front, right side, and left side, respectively, of the main body case. Claim 2 A marine camera device according to claim 1, wherein the integrated bracket is coupled with an Ethernet communication module that outputs information received via a USB (Universal Serial Bus) cable via Ethernet communication, and the Ethernet communication module transmits thermal images transmitted by the infrared cameras via a USB cable and optical images transmitted by the optical cameras via a USB cable to a server on a vessel equipped with a marine camera device using Ethernet communication. Claim 3 In paragraph 2, the server is a marine camera device that generates a panoramic image by projecting the optical images and the thermal images onto a cylindrical coordinate system plane. Claim 4 In paragraph 3, the integrated bracket further comprises an IMU (Inertial Measurement Unit) that measures the movement of a vessel on which a marine camera device is installed and outputs the measured result as IMU sensor data, for a marine camera device. Claim 5 In paragraph 4, the server is a marine camera device that corrects the panoramic image based on the IMU sensor data. Claim 6 A marine camera device according to claim 1, further comprising a sun visor coupled to the upper part of the main body case, wherein the sun visor is manufactured with an aerodynamic shape that takes into account wind resistance. Claim 7 In claim 1, the main body case is a marine camera device having an engraved pattern for heat dissipation formed on its upper surface. Claim 8 A marine camera device according to claim 1, wherein the main body case comprises a camera waterproof structure in which a waterproof sealing of the hole, a transparent window, a waterproof sealing of the transparent window, and a fixed case are sequentially combined on the front of each of the camera holes. Claim 9 A marine camera device according to claim 1, further comprising a stand for fixing the main body case to a vessel, wherein the stand comprises a plurality of horizontal rotation holes formed lengthwise in a horizontal rotation direction on the upper surface, a bolt passing through the horizontal rotation holes is inserted and coupled into a hole formed in the lower part of the main body, and the range of horizontal rotation of the stand is determined according to the length of the horizontal rotation holes. Claim 10 A method for image stabilization of a marine camera device, comprising: receiving camera images generated by cameras of a marine camera device from a marine camera device; and projecting the camera images onto a cylindrical coordinate system plane to generate a panoramic image, wherein the marine camera device comprises at least one optical camera and at least one infrared camera disposed on the front, right side, and left side of a main body case, respectively. Claim 11 A method for image stabilization of a marine camera device according to claim 10, wherein the camera images include at least one of optical images generated by the optical cameras capturing different directions and thermal images generated by the infrared cameras capturing different directions. Claim 12 A method for stabilizing an image of a marine camera device according to claim 10, further comprising: a step of identifying whether the surrounding environment of a vessel equipped with a marine camera device is rough; and a step of correcting the panoramic image based on IMU sensor data when the surrounding environment of the vessel equipped with the marine camera device is not rough, wherein the IMU sensor data is generated by the Inertial Measurement Unit (IMU) of the marine camera device measuring the movement of the vessel equipped with the marine camera device. Claim 13 In claim 12, the step of correcting based on the IMU sensor data is to correct the panoramic image by applying a coordinate transformation between the individual camera coordinate system of each of the optical cameras and infrared cameras and the origin coordinate system of the vessel on which the marine camera device is installed, based on the IMU sensor data, to the panoramic image. Claim 14 A method for image stabilization of a maritime camera device according to claim 12, further comprising: a step of generating a Surround View Monitoring (SVM) image using the camera images; and a step of correcting the SVM image based on the IMU sensor data.

Images