Method of provide an omnidirectional around view image of a ship

Abstract

The present invention relates to a method for providing an all-around view image of a ship that can prevent collision accidents by displaying the 360° surrounding situation of the ship in real time, and is characterized by comprising: a distance information acquisition step in which a plurality of ultrasonic sensors acquire distance information with respect to obstacles around the ship; a shooting step in which an all-around image of the ship is captured through a plurality of cameras; an around view image generation step in which an around view image is generated based on the distance information and the all-around image; and a display step in which the around view image is displayed.

Description

Technology Field

[0001] The present invention relates to a method for providing an all-around view image of a ship, and more specifically, to a method for providing an all-around view image of a ship that can prevent collision accidents by displaying the 360° surrounding situation of the ship in real time. Background Technology

[0003] In general, to minimize damage from marine pollution caused by maritime accidents, prompt initial measures, the establishment of efficient response strategies, and the rapid mobilization of response equipment are essential. In particular, since spilled oil spreads at sea due to external environmental forces such as tides, currents, and winds, it is crucial to accurately identify the diffusion path of the spilled oil by considering real-time environmental forces at the time of the accident for efficient response.

[0004] Oil spills caused by ship accidents damage marine ecosystems and sensitive resources, resulting in massive financial losses; consequently, various containment technologies and equipment are mobilized to carry out cleanup operations in the affected area.

[0005] For example, Korean registered patent No. 10-1567431 discloses a method for predicting the diffusion of spilled oil for establishing efficient control measures in the event of a marine oil pollution accident, comprising the steps of: connecting a weather forecasting system, a satellite image receiving system, a tide gauge station, a server, and a client to the Internet, and the server receiving weather data, water temperature data, and tidal information in real time from the weather forecasting system, the satellite image receiving system, and the tide gauge station, respectively; predicting tides and wind-driven currents using the weather data, water temperature data, tidal information, and a numerical model of seawater flow stored in the server; and predicting seawater flow using the tides and wind-driven currents and predicting the diffusion of spilled oil in real time using the seawater flow and weather data; wherein the tide prediction is performed using a processing model that reproduces real-time tide levels and tides by combining real-time tide level information from a tide gauge station based on a harmonic method for each tide type and spatial distribution information of tides based on tidal numerical modeling through modulated tides.

[0006] However, according to the aforementioned prior art literature, there is a problem in that there are limitations in providing detailed information, such as the location of rescue vessels, the spread of spilled oil, the location of emulsified oil, and the degree of weathering, when predicting the diffusion of spilled oil.

[0007] Furthermore, according to the aforementioned prior art literature, there is a problem in that it is difficult to utilize predicted information based on the maritime conditions of the day when an actual ship accident occurs, making it difficult to use pollution control strategies based solely on the prediction of tides and wind currents. Prior art literature

[0009] Republic of Korea Published Patent Application No. 10-2021-0054708 The problem to be solved

[0010] The present invention has been devised to solve the aforementioned problems, and the objective of the present invention is to provide a method for providing an all-around view image of a ship that allows the viewpoint position, viewpoint direction, magnification ratio, and angle of view to be freely adjusted in real time.

[0012] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0014] To achieve the above objective, the present invention is characterized by comprising: a distance information acquisition step in which a plurality of ultrasonic sensors acquire distance information to obstacles around a vessel; a shooting step in which a vessel-wide image is captured through a plurality of cameras; an around-view image generation step in which an around-view image is generated based on the distance information and the around-view image; and a display step in which the around-view image is displayed.

[0015] In addition, the surround view image generation step is characterized by including a virtual image generation step that generates a virtual partial image visualizing distance information to the obstacle, and a virtual surround view image generation step that generates a virtual surround view image by combining the virtual partial image and the captured video.

[0016] In addition, the above-described around-view image generation step is characterized by adjusting the camera image magnification according to the location where the camera is installed, based on the distance information obtained in the above-described distance information acquisition step. Effects of the invention

[0018] The present invention provides an around-view system by measuring images and distances around a ship based on multiple ultrasonic sensors and multiple cameras, thereby providing the effect of preventing collision accidents during ship operation. Brief explanation of the drawing

[0020] FIG. 1 is a flowchart illustrating a method for providing an all-around view image of a ship according to a preferred embodiment of the present invention. FIGS. 2 and FIGS. 3 are exemplary diagrams showing the state in which a method for providing an all-around view image of a ship according to a preferred embodiment of the present invention is applied. Specific details for implementing the invention

[0021] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0022] Specific details for implementing the present invention will be described in detail with reference to the drawings attached below. Regardless of the drawings, identical reference numerals refer to identical components, and "and / or" includes each of the mentioned items and all combinations of one or more.

[0023] Although terms such as "first," "second," etc., are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical scope of the present invention.

[0024] The terms used herein are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used herein, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0025] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in a meaning that is commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0026] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0028] FIG. 1 is a flowchart illustrating a method for providing an all-around view image of a ship according to a preferred embodiment of the present invention, and FIG. 2 and FIG. 3 are exemplary diagrams illustrating a state in which the method for providing an all-around view image of a ship according to a preferred embodiment of the present invention is applied.

[0030] As illustrated in FIGS. 1 to 3, the method for providing an all-around view image of a ship according to the present invention comprises a distance information acquisition step (S100), a shooting step (S200), an image generation step (S300), and a display step (S400).

[0031] First, the distance information acquisition step (S100) is a step in which multiple ultrasonic sensors acquire distance information to obstacles around the ship, and multiple ultrasonic sensors are installed from the bow to the stern of the ship to measure the distance to obstacles from the side.

[0032] And the above shooting step (S200) is a step of capturing a full-view image of the ship through a plurality of cameras, which are installed at the front, rear, left, right, and center of the ship, respectively, and the number of cameras installed may increase depending on the size of the ship.

[0033] Preferably, 6 to 8 cameras are installed, and 2 to 4 ultrasonic sensors are installed.

[0034] Next, the around view image generation step (S300) is an around view image generation step that generates an around view image based on the distance information and the omnidirectional image, and consists of a virtual image generation step (S310) and a virtual around view image generation step (S320).

[0035] Here, the virtual image generation step (S310) is a step of generating a virtual partial image that visualizes the distance information with respect to the obstacle.

[0036] And, the virtual around-view image generation step (S320) is a step of generating a virtual around-view image by combining a virtual partial image and a captured video.

[0037] In addition, the above-mentioned around-view image generation step (S300) can generate an around-view image by adjusting the camera image magnification ratio according to the location where the camera is installed, based on the distance information obtained in the distance information acquisition step (S100).

[0038] Next, the display step (S400) is a step of displaying the around-view image, and displays the around-view image in the captain's cabin so that the navigator can check it in real time.

[0039] Meanwhile, the method for providing an all-around view image of a ship according to the present invention may utilize an all-around view image providing system (1000) of a ship.

[0040] At this time, the all-around view image providing system (1000) of the above-mentioned vessel is configured to include a camera (100), an image input unit (200), a coordinate storage unit (300), a coordinate estimation unit (400), an image processing unit (500), an ultrasonic sensor (600), an analysis unit (700), a control unit (800), and a monitor unit (900).

[0041] First, the camera (100) outputs at least one of a ship's front image, a ship's rear image, and a ship's side image.

[0042] Next, the image input unit (200) receives images from a plurality of cameras (100). The image input unit (200) obtains color representation information or image coordinate information in units of image output pixels of the received images. The image input unit (200) outputs the coordinate information to the coordinate storage unit (300).

[0043] The above coordinate storage unit (300) stores a plurality of lookup tables composed of records in which color expression information or image coordinate information is recorded in units of image output pixels.

[0044] At this time, the coordinate storage unit (300) stores a lookup table (luta) composed of records in which coordinate information of the screen currently set by the driver is recorded, and a lookup table (lutb) composed of records in which coordinate information of the screen to be selected by the driver is recorded.

[0045] And the coordinate estimation unit (400) calculates a new lookup table (lutc) by calculating interpolation with weights for the data of at least two of the multiple lookup tables (luta, lutb).

[0046] That is, the coordinate estimation unit (400) gradually changes the weights of the current screen and the selected screen to prevent discontinuous switching from the current screen to the selected screen when the driver selects a screen switch.

[0047] In addition, the coordinate estimation unit (400) calculates interpolation by changing the value of interpolation for each frame.

[0048] Here, the coordinate estimation unit (400) calculates the coordinate information of the lookup table (lutc) of the intermediate transformed screen by giving a weight (α) to the coordinate information of the lookup table (luta) of the current screen in a preset algorithm and giving a weight (1-α) to the coordinate information of the lookup table (lutb) of the transformed selected screen.

[0049] And the coordinate estimation unit (400) changes the interpolation value by decreasing a constant value at set unit times.

[0050] According to one embodiment, the coordinate estimation unit (400) decreases the weight (α) by 0.1 at each unit time set from 1.

[0051] Furthermore, the weights set in the coordinate estimation unit (400) can be changed according to the settings and are not limited to any specific value.

[0052] Next, the image processing unit (500) synthesizes an image by considering at least one of the stored lookup tables.

[0053] Here, the image processing unit (500) synthesizes an image by considering at least one of the considered lookup tables (luta, lutb) and the new lookup table (lutc).

[0054] The image processing unit (500) synthesizes the image by considering a new lookup table (lutc) in the middle of the lookup tables (luta, lutb) used for the interpolation calculation when converting the synthesized image.

[0055] The image processing unit (500) outputs the image synthesized by the new lookup table (lutc) to the monitor unit (900).

[0056] The above-mentioned monitor unit (900) outputs the received video to the screen. The monitor unit (900) may be an AVM (Around-View Monitoring) system screen or a HUD (Head-Up Display).

[0057] The operation of the image generation method in the ship omnidirectional surveillance camera system for preventing safety accidents on a ship according to the present invention, configured as described above, is as follows.

[0058] Next, the ultrasonic sensors (600) are installed in multiple numbers on the side of the vessel and are configured to measure the distance between the side of the vessel and surrounding objects.

[0059] Here, it is preferable that the ultrasonic sensors (600) be installed at the bow, center, and stern of the ship, respectively.

[0060] At this time, the ultrasonic sensor (600) calculates the distance between the side of the ship and the obstacle, and the distance can be measured through a preset algorithm.

[0061] And, the analysis unit (700) analyzes the image or photo measured by the camera (100) to determine the type of obstacle around the ship.

[0062] Accordingly, the analysis unit (700) determines the type of obstacle and provides an alarm to the navigator (pilot) to prevent the vessel from colliding with or coming into contact with a port or another vessel, thereby enabling the navigator (pilot) to make a safe choice regarding the risk of a vessel accident.

[0063] Next, the integrated control unit (800) measures the inclination from the bow to the stern of the ship through the distance to surrounding objects measured by the plurality of ultrasonic sensors (600), and corrects the magnification of the image measured according to the position of the plurality of cameras (100) through the measured inclination.

[0064] That is, as illustrated in FIG. 3, the tilt of the ship is measured according to the difference in distance between the ship and the terminal while the ship is approaching the terminal, and the image magnification of the cameras (100) installed at the bow and stern, respectively, is adjusted according to the tilt of the ship so that no error occurs in the image displayed on the monitor unit (900).

[0065] Next, the operation sequence of the ship omnidirectional surveillance camera system for preventing safety accidents on a ship according to the present invention will be explained.

[0066] The camera (100) outputs at least one of a ship's front image, a ship's rear image, and a ship's side image. The image input unit (200) receives images from a plurality of cameras (S100).

[0067] The coordinate storage unit (300) stores a plurality of lookup tables composed of records in which color expression information or coordinate information of an image is recorded in units of image output pixels (S110).

[0068] The coordinate storage unit (300) stores a lookup table (luta) composed of records in which coordinate information of the screen currently set by the driver is recorded, and a lookup table (lutb) composed of records in which coordinate information of the screen to be selected by the driver is recorded.

[0069] The coordinate estimation unit (400) calculates a new lookup table (lutc) by applying weights to the data of at least two of the multiple lookup tables and performing interpolation calculations (S120). The coordinate estimation unit (400) performs interpolation calculations by changing the interpolation value for each frame. The coordinate estimation unit (400) changes the interpolation value by decreasing a constant value at set unit times.

[0070] The coordinate estimation unit (400) gradually changes the weights of the current screen and the selected screen to prevent discontinuous changes from the current screen to the selected screen when the driver selects a screen switch. The coordinate estimation unit (400) calculates interpolation by changing the interpolation value for each frame.

[0071] Then, the distance measured by the above-mentioned ultrasonic sensor (600) is displayed on the monitor unit (900), and the magnification of the camera (100) at each position according to the tilt of the ship is adjusted so that the navigator can check an accurate full-directional image of the ship.

[0073] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without changing its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. Explanation of the symbols

[0075] S100: Distance information acquisition stage S200: Shooting stage S300: Around View Image Generation Step S310: Virtual image generation step S320: Virtual Around View Image Generation Step S400: Display step

Claims

Claim 1 A method for providing an all-around view image of a ship, comprising: a distance information acquisition step in which a plurality of ultrasonic sensors acquire distance information to obstacles around the ship; a shooting step in which a holographic image of the ship is captured through a plurality of cameras; an around view image generation step in which an around view image is generated based on the distance information and the holographic image; and a display step in which the around view image is displayed; wherein the around view image generation step comprises a virtual image generation step in which a virtual partial image is generated by visualizing the distance information to obstacles, and a virtual around view image generation step in which a virtual around view image is generated by combining the virtual partial image and the captured image; and furthermore, the around view image generation step is characterized by measuring the inclination from the bow to the stern of the ship based on the distance information acquired in the distance information acquisition step, and adjusting the camera image magnification according to the position where the camera is installed through the measured inclination. Claim 2 delete Claim 3 delete

Images