Ship safety control methods and systems

Abstract

This disclosure relates to a method and system for the safe control of a ship. According to one embodiment of this disclosure, a method for the safe control of a ship may include the steps of: determining the sea conditions in the area where the ship is located based on video features of a first video of the ship's condition; preprocessing the first video based on the sea conditions; designating a control area in the second video based on a second video generated by the preprocessing; and detecting the occurrence of at least one object or event within the control area.

Description

Technical Field

[0001] The present disclosure relates to a method and system for ship safety control.

Background Art

[0002] In recent years, with the automation of ships, the number of crew members on board has decreased and the size of ships has increased.

[0003] This has led to the problem that it is difficult to quickly respond to various accidents occurring on ships. For example, the occurrence of a fire on a container ship causes damage of $20 billion annually.

[0004] Conventional ship control systems had the drawback of being difficult to grasp in real time information related to ship operation, such as crew members and the operating state of the ship. For example, when monitoring using a photographing device, even if an event was detected, the area where the event occurred could not be immediately displayed, and it was difficult to provide immediate countermeasures. However, since most ship accidents occur at sea, a delayed response may lead to greater damage.

[0005] This has led to a situation where there is a need to develop a system that can efficiently recognize accidents occurring during ship operation and take prompt action.

Summary of the Invention

Problems to be Solved by the Invention

[0006] The present disclosure aims to provide a method and system for ship safety control. The problems to be solved by the present invention are not limited to the problems described above, and other problems and advantages of the present invention not mentioned can be understood from the following description and will be more clearly understood in the embodiments of the present invention. Furthermore, it will be understood that the problems and advantages to be solved by the present invention can be realized by the means and combinations thereof shown in the claims.

Means for Solving the Problems

[0007] As a technical means for achieving the technical challenges described above, a first aspect of this disclosure may include, as a method for safe control of a vessel, the steps of: determining the maritime conditions in the area where the vessel is located based on video features of a first video of the vessel's condition; preprocessing the first video based on the maritime conditions; specifying a control area in the second video based on a second video generated by the preprocessing; and detecting the occurrence of at least one object or event within the control area.

[0008] A second aspect of the present disclosure is a ship safety control device comprising at least one memory and at least one processor, the at least one processor capable of determining the maritime conditions in the area where the ship is located based on video features of a first video capturing the ship's condition, preprocessing the first video based on the maritime conditions, designating a control area in the second video based on the second video generated by the preprocessing, and detecting at least one occurrence of an object or event within the control area.

[0009] A third aspect of this disclosure can provide a computer-readable recording medium that stores a program for performing the method according to the first aspect on a computer.

[0010] In addition, other methods, other systems, and computer-readable recording media storing computer programs for implementing the present invention can be further provided.

[0011] Other aspects, features, and advantages not mentioned above will become clear from the following drawings, claims, and detailed description of the invention. [Effects of the Invention]

[0012] According to the means for solving the problems of this disclosure described above, this disclosure enables stable control of ships in response to conditions that occur during ship operation, such as sea fog and rainfall.

[0013] Furthermore, this disclosure enables efficient ship control based on various scenarios for event detection.

[0014] Furthermore, this disclosure enables immediate response to accidents by presenting and implementing countermeasures based on the detection results of objects or events. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram illustrating a ship safety control system according to one embodiment. [Figure 2] This is a block diagram of a ship safety control system according to one embodiment. [Figure 3] This is an illustrative configuration diagram of a system including a ship safety control device and external devices according to one embodiment. [Figure 4] This is a flowchart illustrating a ship safety control method according to one embodiment. [Figure 5] This flowchart illustrates a method for detecting objects and event occurrences based on a first image acquired through a camera according to one embodiment, and for executing corresponding countermeasures. [Figure 6] This is an illustrative diagram illustrating a pretreatment method for sea fog conditions according to one embodiment. [Figure 7] This is a flowchart illustrating a pretreatment method for sea fog conditions according to one embodiment. [Figure 8] This is an illustrative diagram illustrating a pretreatment method under rainfall conditions according to one embodiment. [Figure 9] This is a flowchart illustrating a pretreatment method under rainfall conditions according to one embodiment. [Figure 10] This is an illustrative diagram illustrating a method for dividing a region according to one embodiment. [Figure 11] It is an exemplary diagram for explaining a method of maintaining a user-specified area when the shooting area of a shooting device according to an embodiment is changed. [Figure 12] It is an exemplary diagram for explaining a method of detecting a fire event according to an embodiment. [Figure 13] It is an exemplary diagram for explaining a method of detecting a falling event according to an embodiment. [Figure 14] It is an exemplary diagram for explaining a method of detecting an event of not wearing safety equipment according to an embodiment. [Figure 15] It is an exemplary diagram for explaining a method of detecting an event associated with an increase in moving speed according to an embodiment. [Figure 16] It is an exemplary diagram for explaining a method of detecting an event due to the occurrence of a crowd according to an embodiment. [Figure 17] It is an exemplary diagram for explaining a method of detecting an event due to intrusion into a danger area according to an embodiment. [Figure 18] It is an exemplary diagram for explaining a method of detecting a maritime fall event according to an embodiment. [Figure 19] It is an exemplary diagram for explaining a sorting control method according to an embodiment.

Best Mode for Carrying Out the Invention

[0016] As a ship safety control method, based on the video feature of the first video that captures the ship situation, determining the maritime situation of the area where the ship is located, preprocessing the first video based on the maritime situation, designating a control area within the second video based on the second video generated by the preprocessing, and detecting at least one of the occurrence of an object or an event within the control area.

[0017] The advantages and features of the present invention, as well as methods for achieving them, will become apparent with reference to the embodiments described in detail with the accompanying drawings. However, the present invention is not limited to the embodiments presented below and can be realized in different forms, and should be understood to include all transformations, equivalents, or substitutions that fall within the spirit and technical scope of the invention. The embodiments presented below are provided to complete the disclosure of the present invention and to fully inform those who are ordinary skill in the art to which the invention pertains of the invention of the scope. Where it is determined that a specific description of the relevant prior art would hinder the essence of the invention, such detailed description will be omitted in describing the present invention.

[0018] The terms used in this application are used solely to describe specific embodiments and are not intended to limit the invention. A singular expression includes plural expressions unless the context clearly indicates otherwise. In this application, terms such as “includes” or “having” specify the presence of features, figures, steps, actions, components, parts, or combinations thereof described herein, and should be understood not to preemptively exclude the possibility of the presence or addition of one or more other features, figures, steps, actions, components, parts, or combinations thereof.

[0019] Some embodiments of this disclosure can be represented by functional block configurations and various processing steps. Some or all of such functional blocks can be implemented by various hardware and / or software configurations that perform a particular function. For example, a functional block of this disclosure may be implemented by one or more microprocessors, or by a circuit configuration for a given function. Alternatively, for example, a functional block of this disclosure may be implemented in various programming or scripting languages. A functional block can be implemented by an algorithm that runs on one or more processors. Furthermore, this disclosure may employ prior art for electronic environment setup, signal processing, and / or data processing. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations.

[0020] Furthermore, the connecting lines or members between components shown in the drawings are merely illustrative examples of functional and / or physical or circuit connections. In actual devices, connections between components may be indicated by a variety of alternative or added functional, physical, or circuit connections.

[0021] The present disclosure will be described in detail below with reference to the attached drawings.

[0022] Figure 1 is a conceptual diagram illustrating a ship safety control system according to one embodiment.

[0023] Referring to Figure 1, the vessel 10 can quickly detect an accident occurring on the vessel 10 based on the ship safety control system 1000, and immediately implement appropriate countermeasures according to the detection results.

[0024] For example, the ship safety control system 1000 may include a camera 20 and a ship safety control device 30.

[0025] The imaging device 20 is installed on the deck of the ship 10, inside the ship 10, in the yard, shipyard, etc., and can photograph the area that the ship safety control system 30 wants to monitor. For example, the imaging device 20 may include, but is not limited to, a PTZ (Pan, Tilt and Zoom) camera, a closed-circuit television (CCTV) system, and an image sensor.

[0026] The ship safety control system 30 can detect events occurring in multiple areas based on video footage received from at least one imaging device 20.

[0027] Conventionally, when performing control during ship operation, accidents occurring inside the ship were primarily detected and addressed. However, the ship safety control system 30 can efficiently detect and address accidents occurring outside the ship, such as objects falling into the sea and fires, based on multiple imaging devices 20 installed not only inside the ship but also on the exterior, such as the deck.

[0028] Furthermore, the ship safety control system 30 can detect events occurring on the ship 10 using multiple sensors in addition to the imaging device 20. For example, the ship safety control system 30 can detect events occurring on the ship 10 using at least one of the following sensors: a fire detection sensor, a gas concentration sensor, and an inter-ship recognition sensor (ISS).

[0029] The ship safety control device 30 can perform preprocessing 31 on the first video received from the camera device 20. For example, the ship 10 may face various conditions such as rain, sea fog, nighttime, and fire.

[0030] For example, if the first video received from the camera 20 has image quality that makes it difficult to detect the occurrence of objects and events, the ship safety control device 30 can preprocess the first video 31 to generate a second video suitable for detecting the occurrence of objects and events.

[0031] The ship safety control device 30 can designate a control area 32 for detecting the occurrence of objects and events based on a second image generated by a preprocessing process.

[0032] For example, the ship safety control device 30 can designate an area specified by a crew member as the control area 32. If no area specified by a crew member exists, the ship safety control device 30 can separate the sea area and the ship 10 area in the second image and designate the ship 10 area as the control area 32.

[0033] The ship safety control device 30 can detect objects 33 based on a designated control area.

[0034] For example, the ship safety control device 30 can detect at least one of the following within its control area: persons, safety equipment, fire, smoke, and containers. However, it is not limited to these.

[0035] For example, the ship safety control system 30 can recognize objects based on an object recognition model. Here, the object recognition model can be a DCNN (Deep Convolutional Neural Network) based object recognition model. For example, DCNN-based object recognition models may include, but are not limited to, R-CNN, ViT, Fast / Faster R-CNN, RFCN, SSD, YOLO, etc.

[0036] For example, the YOLO object recognition model can process objects by extracting feature maps from video data via the Backbone-Neck-Head step, refining and reconstructing them, localizing the extracted feature maps, predicting the class and object region, and then generating feature maps of multiple sizes for multi-scale prediction.

[0037] Furthermore, the ship safety control device 30 can output and display the position coordinates of the detected object. For example, a crew member can respond to an accident efficiently based on the position coordinates of the object displayed on the ship safety control device 30.

[0038] Furthermore, the ship safety control device 30 can output class information of the detected objects. For example, if multiple people are detected in the control area, the ship safety control device 30 can assign a number to each person. The ship safety control device 30 can display the numbers assigned to the people among the detected individuals who are not wearing safety equipment.

[0039] The ship safety control system 30 can detect the occurrence of an event based on a designated control area.

[0040] For example, an event may include, but is not limited to, at least one of the following: fire, a person falling, a worker not wearing safety equipment, a crowd gathering, entering a hazardous area, and drowning.

[0041] The following describes in detail how the ship safety control system 30 detects the occurrence of an event 33, with reference to Figures 12 to 18.

[0042] The ship safety control system 30 can take countermeasures based on the detection of at least one of the occurrences of an object or an event.

[0043] For example, if a fire is detected on board the vessel 10, the ship safety control system 30 can automatically take action to extinguish the fire. Conventionally, in order to extinguish a fire on the vessel 10, a crew member who became aware of the fire detection result would have to extinguish the fire directly. However, the ship safety control system 30 can immediately extinguish the fire upon rapid fire detection.

[0044] Furthermore, the ship safety control device 30 can provide evacuation routes for crew members to take refuge within the ship 10. For example, the ship safety control device 30 can pre-designate evacuation routes for crew members to evacuate, and then activate lighting in the event of an accident so that crew members can recognize the evacuation routes.

[0045] Furthermore, although not shown in Figure 1, the ship safety control system 30 can generate and provide an event log based on the type and frequency of previously detected events. This allows the crew to recognize events that frequently occur on the ship 10 and to formulate appropriate countermeasures in advance.

[0046] Figure 2 is a block diagram of a ship safety control system according to one embodiment.

[0047] Referring to Figure 2, the ship safety control system 100 includes a processor 110, memory 120, input / output interface 130, and communication module 140. For convenience of explanation, only components relevant to the present invention are shown in Figure 2. Therefore, the ship safety control system 100 may further include other general-purpose components in addition to those shown in Figure 2. Furthermore, it is obvious to those ordinary skill in the art related to the present invention that the processor 110, memory 120, input / output interface 130, and communication module 140 shown in Figure 2 can be implemented as independent devices.

[0048] In this case, the ship safety control device 100 may be the same device as the safety control device 30 in Figure 1.

[0049] The processor 110 can process computer program instructions by performing basic arithmetic, logic, and input / output operations. Instructions may be provided from memory 120 or an external device. The processor 110 can also provide overall control over the operation of other components included in the ship safety control system 100.

[0050] For example, the processor 110 can preprocess the first video received from the imaging device to generate a second video suitable for detecting the occurrence of objects and events.

[0051] Furthermore, the processor 110 can specify a control area based on the pre-processed second image. Here, the control area can be specified arbitrarily, or it may be specified by image segmentation technology.

[0052] Furthermore, the processor 110 can detect at least one of the occurrences of objects and events within the control area. For example, detected objects may include at least one of a person, safety equipment, fire, smoke, and a container. Furthermore, detected events may include at least one of a fire, a person falling, a worker not wearing safety equipment, a crowd gathering, entry into a hazardous area, and drowning.

[0053] Furthermore, the processor 110 can execute countermeasures based on the detection results.

[0054] For example, the processor 110 can detect at least one object or event using a neural network model. For example, the ship safety control device 100 can train the neural network model using a first video captured from a camera that captures the ship's condition as input data for the neural network model, and at least one detection result of an object or event as output data. The ship safety control device 100 can also input the first video captured from the camera that captures the ship's condition into the trained neural network model to obtain a detection result for at least one object or event.

[0055] In machine learning techniques and cognitive science, a neural network model refers to a statistical learning algorithm or a structure that executes such an algorithm, which is based on the structure of a biological neural network.

[0056] For example, a neural network model can represent a model with problem-solving capabilities, where nodes, which are artificial neurons forming a network with synaptic connections in a biological nervous system, repeatedly adjust the weights of the synapses and learn to reduce the error between the correct output corresponding to a specific input and the inferred output. For example, neural network models can include any probabilistic model or neural network model used in artificial intelligence learning methods such as machine learning and deep learning.

[0057] For example, a neural network model can be implemented using a multilayer perceptron (MLP), which consists of multiple layers of nodes and connections between them. The neural network model according to this embodiment can be implemented using one of various artificial neural network model structures, including MLPs. For example, a neural network model may consist of an input layer that receives input signals or data from the outside, an output layer that outputs output signals or data corresponding to the input data, and at least one hidden layer located between the input and output layers that receives signals from the input layer, extracts characteristics, and transmits them to the output layer. The output layer receives signals or data from the hidden layer and outputs them to the outside.

[0058] The processor 110 may be implemented as an array of multiple logic gates, or as a combination of a general-purpose microprocessor and memory storing a program that can be executed by the microprocessor. For example, the processor 110 may include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some environments, the processor 110 may also include application-specific semiconductors (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), and the like. For example, the processor 110 may refer to a combination of processing units such as a combination of a digital signal processor (DSP) and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors coupled to a digital signal processor (DSP) core, or any other combination of such configurations.

[0059] Memory 120 may include a non-temporary, computer-readable recording medium. For example, memory 120 may include a permanent mass storage device such as RAM (random access memory), ROM (read-only memory), a disk drive, an SSD (solid-state drive), or flash memory. In other examples, the permanent mass storage device such as ROM, SSD, flash memory, or disk drive may be a separate permanent storage device distinct from memory. Memory 120 may also store an operating system (OS) and at least one program code (for example, code for the processor 110 to perform the operations described later with reference to Figures 3 to 19).

[0060] Such software components can be loaded from a computer-readable recording medium other than memory 120. Such a computer-readable recording medium may be a recording medium that can be directly connected to the ship safety control device 100, and may include, for example, input / output computer-readable recording media such as floppy drives, disks, tapes, DVD / CD-ROM drives, and memory cards. Alternatively, the software components can be loaded into memory 120 via the communication module 130 instead of a computer-readable recording medium. For example, at least one program may be loaded into memory 120 based on a computer program (for example, a computer program for the processor 110 to perform the operations described later with reference to Figures 3 to 15) that is installed by a file provided via the communication module 130 by a developer or a file distribution system that distributes application installation files.

[0061] The input / output interface 130 may also be means for interface with input and / or output devices (e.g., keyboard, mouse, etc.) connected to or included in the ship safety control system 100. In Figure 2, the input / output interface 130 is shown as an element configured separately from the processor 110, but is not limited to this, and the input / output interface 130 may be configured to be included in the processor 110. For example, the processor 110 may designate an input user-specified area as a control area based on the input / output interface 130.

[0062] The communication module 140 can provide configurations or functions for the ship safety control system 100 to communicate with external devices (not shown) via a network. The communication module 140 can also provide configurations or functions for the ship safety control system 100 to communicate with other external devices. For example, control signals, instructions, data, etc., provided in accordance with the control of the processor 110 can be transmitted to external devices via the communication module 140 and the network.

[0063] Although not shown in Figure 2, the ship safety control system 100 may also include a display module. For example, the ship safety control system 100 can display a second video showing the event detection results via the display module.

[0064] Figure 3 is an illustrative diagram of a system including a ship safety control device and external devices according to one embodiment.

[0065] Referring to Figure 3, the ship safety control device 310 may include any kind of server that manages a web and / or application that can provide artificial intelligence services. Alternatively, the ship safety control device 310 in Figure 3 may be the same device as the ship safety control device 30 in Figure 1 and / or the ship safety control device 100 in Figure 2.

[0066] External device 320 refers to an entity that provides the ship safety control system 310 with the information necessary for the ship safety control system to detect the occurrence of objects and events and to take countermeasures. External device 320 may include any type of server that manages various types of information. External device 320 may include, but is not limited to, a database and a server that manages a web service API that can provide information. For example, external device 320 may be a camera that photographs the ship's condition and provides video. External device 320 may also be a database that stores countermeasures corresponding to detected events.

[0067] The ship safety control device 310 and the external device 320 can communicate with each other and / or with other devices via a network. The network is a data network in a comprehensive sense that enables different entities to communicate smoothly with each other, and can include wired internet, wireless internet, and mobile wireless networks. For example, the network can include local area networks (LANs), wide area networks (WANs), value-added networks (VANs), mobile radio communication networks, satellite networks, and combinations thereof. Wireless communication can be, but is not limited to, Wi-Fi, Bluetooth, Bluetooth low energy, ZigBee, Wi-Fi Direct (WFD), ultrawideband (UWB), infrared data association (IrDA), and near-field communication (NFC).

[0068] The ship safety control device 310 can communicate with the external device 320 via a network. By communicating via the network, the ship safety control device 310 can receive data from the external device 320 and provide a response based on the received data.

[0069] Figure 4 is a flowchart illustrating a ship safety control method according to one embodiment.

[0070] Referring to Figure 4, the ship safety control method consists of steps processed chronologically by the ship safety control device 100 and / or processor 110 shown in Figure 2. Therefore, even if the details are omitted below, the above information regarding the ship safety control device 100 or processor 110 shown in Figure 2 can also be applied to the ship safety control method in Figure 4.

[0071] In step 410, the ship safety control system can determine the maritime conditions in the area where the ship is located based on the video features of the first video that captures the ship's condition.

[0072] The ship's safety control system can calculate image features based on a first image acquired from the imaging device. Here, the image features may include at least one of the following features: an illuminance feature, an RGB feature, and a dark channel feature.

[0073] Furthermore, the ship safety control system can determine the maritime conditions in the area where the ship is located based on the calculated video features. For example, if the calculated video features exceed a predetermined threshold, the ship safety control system can determine that the maritime conditions in the area where the ship is located correspond to at least one of the following: nighttime, rainfall, or sea fog.

[0074] In step 420, the ship safety control system can preprocess the first video based on the determined sea conditions.

[0075] For example, the ship safety control system can perform at least one of the following pre-processing steps based on the determined sea conditions: nighttime pre-processing, rainfall pre-processing, and sea fog pre-processing. The ship safety control system can generate a second image by pre-processing the first image.

[0076] In step 430, the ship safety control system can designate a control area within the second image based on the second image generated by the preprocessing. For example, the ship safety control system may designate a specified area entered by the user as the control area, or it may analyze the second image to determine the control area. For example, the ship safety control system may divide the second image into a sea area and a ship area and designate only the ship area as the control area.

[0077] In step 440, the ship's safety control system can detect at least one object or event occurring within the control area.

[0078] For example, a ship's safety control system can detect at least one object from among people, safety equipment, fire, smoke, and containers. Furthermore, a ship's safety control system can detect at least one event from among fire, people falling, workers not wearing safety equipment, crowds, intrusion into hazardous areas, and drowning.

[0079] Furthermore, in addition to video, the ship's safety control system can detect the occurrence of objects and events based on data acquired from at least one of the following sensors: a fire detection sensor, a gas concentration sensor, and an other ship recognition sensor.

[0080] In step 450, the ship safety control system can implement countermeasures in response to the detection results.

[0081] For example, countermeasures may include, but are not limited to, at least one of the following: fire suppression, alarm activation, display of the event area, expansion of the event area, and display of evacuation routes.

[0082] Figure 5 is a flowchart illustrating a method for detecting objects and event occurrences based on a first image acquired through a camera according to one embodiment, and for executing corresponding countermeasures.

[0083] Referring to Figure 5, the ship safety control method consists of steps processed chronologically by the ship safety control device 100 and / or processor 110 shown in Figure 2. Therefore, even if the details are omitted below, the above information regarding the ship safety control device 100 or processor 110 shown in Figure 2 can also be applied to the ship safety control method in Figure 5.

[0084] In step 511, the ship safety control system can acquire a first image from the camera.

[0085] In this context, the first image refers to the original image captured by the camera. However, since ships may encounter various conditions such as rainfall and sea fog during operation, pre-processing of the original image is necessary to obtain accurate detection results.

[0086] In step 512, the ship safety control system can calculate video features from the first video.

[0087] For example, the video feature may include at least one of the following features: an illumination feature, an RGB feature, and a dark channel feature. The video feature may be used to determine whether a preprocessing process has been performed on the first video of the ship's safety control system.

[0088] In step 513, the ship safety control system can determine whether the video features calculated from the first video exceed a predetermined threshold.

[0089] For example, a ship safety control system can determine whether the value of an illuminance feature exceeds a first threshold. It can also determine whether the value of an RGB feature exceeds a second threshold. Furthermore, it can determine whether the value of a dark channel feature exceeds a third threshold.

[0090] In step 521, if all of the features included in the video feature do not exceed a predetermined threshold, the ship safety control system can determine whether or not there is a user-specified area within the first video without performing any pre-processing on the first video.

[0091] In step 514, if at least one feature among the features included in the video feature exceeds a predetermined threshold, the ship safety control system can determine a first video preprocessing method.

[0092] The ship safety control system can perform at least one of the following pre-treatment processes: nighttime pre-treatment 515, rainfall pre-treatment 516, and sea fog pre-treatment 517.

[0093] For example, if the illuminance feature value exceeds a first threshold, the ship safety control system can execute a nighttime pre-processing process. Similarly, if the RGB feature value exceeds a second threshold, the ship safety control system can execute a rainfall pre-processing process. Furthermore, if the dark channel feature value exceeds a third threshold, the ship safety control system can execute a sea fog pre-processing process.

[0094] For example, if the illuminance feature value is less than 0.1 lux, the ship safety control system can perform a nighttime pre-processing process. As a result of the nighttime pre-processing process, the ship safety control system may generate images with improved visibility.

[0095] For example, a ship safety control system can use night vision technology as a night pre-processing step. Night vision technology, for instance, means controlling the camera so that the infrared cut film applied to the camera is removed. This allows the ship safety control system to generate a second image with improved visibility.

[0096] The ship safety control system can perform multiple preprocessing processes on the first video if multiple features among the features included in the video feature exceed a predetermined threshold.

[0097] The pre-processing processes 515, 516, and 517 for the first video will be described in detail below with reference to Figures 6 to 9.

[0098] In step 518, the ship safety control system may perform preprocessing on the first image according to a determined method. The ship safety control system may generate a second image by performing preprocessing on the first image.

[0099] In step 521, the ship safety control system can determine whether the user-specified area is present in the second video.

[0100] In step 522, if the user-specified area does not exist in the second image, the ship safety control system can divide the area in the second image. For example, the ship safety control system can divide the area in the second image into a sea area and a ship area.

[0101] The method of dividing the region will be explained in detail below with reference to Figure 10.

[0102] In step 525, the ship safety control system can designate a control area within the second video. For example, the ship safety control system can designate at least one area from the sea area or the ship area as the control area.

[0103] In step 523, if the user-specified area exists within the second image, the ship safety control system can determine whether the shooting area of ​​the first image and / or the orientation of the shooting ship safety control system have changed. For example, if the shooting area of ​​the first image and / or the orientation of the shooting device changes, the position of the user-specified area within the first image also changes, so it is necessary to maintain and recognize the user-specified area in accordance with the change in the shooting area.

[0104] In step 525, if the shooting area and / or orientation of the shooting device of the first image has not changed compared to before, the ship safety control device may designate the user-specified area as the control area.

[0105] In step 524, if the shooting area and / or orientation of the shooting device of the first image has changed compared to before, the ship's safety control system may adjust the user-specified area to reflect the changed shooting area and / or orientation of the shooting device.

[0106] Furthermore, if the shooting area of ​​the camera is changed, the area where the user-specified area is located in the first image may change. The ship control system can maintain the area to be observed and detected by adjusting the user-specified area as the position of the user-specified area in the first image changes.

[0107] The method for maintaining and recognizing the user-specified area when the shooting area is changed will be explained in detail below with reference to Figure 11.

[0108] In step 525, the ship safety control device may designate a newly recognized user-specified area as the control area in response to a change in the shooting area of ​​the first image and / or the orientation of the shooting device.

[0109] In step 530, the ship safety control system can detect at least one object or event occurring within the control area. For example, the ship safety control system can detect at least one object from among people, safety equipment, fire, smoke, and containers. Furthermore, the ship safety control system can detect at least one event from among fire, a person falling, a worker not wearing safety equipment, a crowd formation, entry into a hazardous area, and drowning.

[0110] In step 540, the ship's safety control system can take action to respond to at least one detection result of the occurrence of an object or event.

[0111] For example, countermeasures may include, but are not limited to, at least one of the following: fire suppression, alarm activation, display of the event area, expansion of the event area, and display of evacuation routes.

[0112] Figure 6 is an illustrative diagram illustrating a pretreatment method in sea fog conditions according to one embodiment.

[0113] Referring to Figure 6, Image 610 corresponds to an image taken by the camera when no sea fog was detected. Image 620 corresponds to an image taken by the camera when sea fog was detected.

[0114] Traditionally, the Structural Similarity Index (SSIM) value was used to determine whether or not sea fog could be detected. SSIM is an index used to measure the quality of an image or video, and is used to compare the similarity between two different images or videos. However, the SSIM value had a problem in that it was difficult to use in typical sea fog conditions because it required a control image in which no sea fog was detected.

[0115] However, the ship safety control system according to the present invention can determine the sea fog situation in the first video acquired from the imaging device without using a reference image. The ship safety control system according to the present invention can calculate dark channel features in the first video to determine the sea fog situation.

[0116] For example, a ship safety control system can calculate the luminance value of each RGB channel for each frame of a first video. Furthermore, the ship safety control system can determine the channel with the minimum luminance value among the RGB channels. In addition, the ship safety control system can calculate the average luminance value of the pixels in the first video for the determined channel. The ship safety control system can also determine the dark channel feature value based on the average luminance value.

[0117] For example, a ship safety control system can execute a sea fog pretreatment process if the calculated dark channel feature value exceeds a predetermined threshold.

[0118] According to one embodiment, a predetermined threshold can be determined based on the dark channel feature value of each frame of the first video and / or the average luminance value of the pixels in each RGB channel of each frame.

[0119] According to one embodiment, the predetermined threshold may be set as the difference between the average values ​​of the luminance values ​​between the RGB channels. For example, the ship safety control system can perform a sea fog pretreatment process if the luminance values ​​of all RGB channels do not exceed the predetermined threshold.

[0120] According to one embodiment, a predetermined threshold can be determined using a feature in which the luminance value of at least one channel among the RGB channels of a first image taken in a marine environment where sea fog is absent tends to be significantly lower than the luminance values ​​of the other channels.

[0121] According to the embodiment, a predetermined threshold can be determined based on background conditions, including the amount of sunlight. In this case, if the amount of sunlight measured through the first video is higher than the average amount of sunlight, the ship safety control device can adjust the predetermined threshold to a higher value than conventionally.

[0122] The ship safety control system can perform a sea fog preprocessing process on image 620 in which the value of the dark channel feature exceeds a predetermined threshold. The ship safety control system may include a process to sharpen the image. For example, the ship safety control system can perform a preprocessing process to sharpen the first image using a dehaze algorithm. As a result, the luminance value of at least one channel among the RGB channels of the preprocessed second image may tend to be significantly lower than the luminance values ​​of the other channels.

[0123] Whether the luminance value of at least one of the RGB channels of a video tends to be significantly lower than the luminance values ​​of the other channels can be determined by using video with and without sea fog as references. According to one embodiment, whether the luminance value of at least one of the RGB channels of a video tends to be significantly lower than the luminance values ​​of the other channels can be determined by a reference value entered by the user, or by using a reference value extracted from a first acquired video.

[0124] Furthermore, the ship safety control system can further optimize the dehaze algorithm to reduce the time required for the sea fog preprocessing process. For example, the ship safety control system can perform a process to compress the original image in which sea fog is detected, and then perform a filtering process. After the filtering process is performed, the ship safety control system can perform a process to re-decompress the compressed image.

[0125] For example, a ship safety control system can accurately detect the occurrence of events such as fire 630 based on a second video showing the sea fog pre-treatment process being performed.

[0126] Figure 7 is a flowchart illustrating a pretreatment method in sea fog conditions according to one embodiment.

[0127] In step 710, the ship safety control system can calculate the luminance value of each RGB channel for each frame of the first video.

[0128] In step 720, the ship's safety control system can determine the channel having the minimum luminance value among the RGB channels.

[0129] In step 720, the ship safety control system can calculate the average brightness value of pixels in the first video for the determined channel.

[0130] In step 720, the ship safety control system can determine the average luminance value as the dark channel feature value.

[0131] In step 750, the ship safety control system can determine whether the dark channel feature exceeds a predetermined threshold.

[0132] Here, a predetermined threshold can be determined based on the dark channel feature value of each frame of the first video and / or the average luminance value of the pixels in each RGB channel of each frame.

[0133] Furthermore, a predetermined threshold can be determined by using the fact that, in the absence of sea fog, the luminance value of at least one of the RGB channels tends to be significantly lower than the luminance values ​​of the other channels.

[0134] Furthermore, the predetermined threshold can be determined based on background conditions. For example, the background conditions may include solar radiation. If the measured solar radiation is higher than the average solar radiation, the ship's safety control system can adjust it to a lower value than the predetermined threshold.

[0135] In step 760, the ship safety control system may perform sea fog preprocessing if the dark channel features exceed a predetermined threshold value. For example, the ship safety control system may perform a sea fog preprocessing process based on a dehaze algorithm. Thus, the ship safety control system can reduce the difference between pixel values ​​in the first image to create a second image with a clearer image.

[0136] Figure 8 is an illustrative diagram illustrating a pretreatment method under rainfall conditions according to one embodiment.

[0137] Referring to Figure 8, Image 800 corresponds to the first video captured by the camera during rainfall.

[0138] The ship's safety control system can calculate RGB features based on the first image acquired from the imaging device.

[0139] For example, a ship safety control system can calculate the luminance value of each pixel contained in a frame within a first video. Furthermore, the ship safety control system can calculate the luminance value of each pixel contained in the frame adjacent to the frame for which the luminance value was calculated. The ship safety control system can also calculate the change in the luminance value of each pixel. Additionally, the ship safety control system can determine the number of pixels whose change in luminance value exceeds a predetermined threshold as the value of the RGB feature.

[0140] Furthermore, the ship's safety control system can determine the value of the RGB feature by summing the changes in the brightness value of each pixel.

[0141] The ship safety control system can determine that rainfall is occurring if the calculated RGB feature values ​​exceed a predetermined threshold. In this case, the ship safety control system can perform a rainfall pre-processing process on the first video.

[0142] According to one embodiment, the predetermined threshold may be the ratio of the RGB feature value to the total number of pixels included in the frame.

[0143] According to one embodiment, a predetermined threshold can be determined based on the average luminance value of the RGB channels of each frame of the first video.

[0144] According to one embodiment, a predetermined threshold can be determined using the characteristic that the change in the average luminance value of the RGB channels of video captured in a sea environment without rainfall is higher than the change in the average luminance value of the RGB channels of video captured in a sea environment with rainfall.

[0145] According to the embodiment, a predetermined threshold can be determined based on background conditions, including the amount of sunlight. In this case, if the amount of sunlight measured through the first video is higher than the average amount of sunlight, the ship safety control device can adjust the predetermined threshold to a higher value than conventionally.

[0146] The ship safety control system can perform rainfall preprocessing by using the RGB values ​​of pixels in adjacent frames to restore pixels corresponding to the areas obscured by the water droplets 810 in the image. By detecting pixels whose average luminance change in the RGB channels is relatively lower than that of other areas and recognizing them as areas obscured by the water droplets 810, the system can reverse-calculate the color information of pixels in areas not obscured by the water droplets 810 during the process of compressing and restoring the image (or frame), and use the calculated color information to correct the color information of pixels in areas obscured by the water droplets 810. The compression and restoration may be performed only once or multiple times. The areas not obscured by the water droplets 810 may be determined using pixels whose average luminance change in the RGB channels is relatively higher than that of other areas.

[0147] Whether a pixel has a relatively high or low change in the average brightness value of its RGB channels compared to other parts may be determined using a reference value, or it may be determined proportionally by aligning the values ​​within the frame. For example, the reference value may be determined based on a reference image or video of rainfall conditions. For example, the change in the average brightness value of the RGB channels can be calculated for each pixel, and pixels with values ​​within the top or bottom 5% of the total value of all pixels can be determined as pixels with relatively high or low values.

[0148] For example, a ship safety control system can use a Discrete Cosine Transform (DCT) algorithm to compress the image and an inverse DCT algorithm to restore the image in order to restore the portion of the image obscured by water droplets 810.

[0149] Figure 9 is a flowchart illustrating a pretreatment method under rainfall conditions according to one embodiment.

[0150] In step 910, the ship safety control system can calculate the brightness value of each pixel contained in the frame of the first video.

[0151] In step 920, the ship safety control system can compare the brightness value of each pixel in a frame with the brightness value of an adjacent frame to calculate the change in the brightness value of each pixel.

[0152] In step 930, the ship safety control device can determine the number of pixels whose change in brightness value exceeds a predetermined threshold as the value of the RGB feature.

[0153] Here, the value of the RGB feature represents the difference in RGB values ​​of each pixel contained in adjacent frames.

[0154] Furthermore, the ship safety control system can determine the value of the RGB feature as the proportion of pixels within a frame whose change in brightness value exceeds a predetermined threshold.

[0155] Furthermore, the ship safety control system can calculate the values ​​of RGB features through the normalization of the number of pixels whose change in brightness exceeds a predetermined threshold and the proportion of pixels in the frame whose change in brightness exceeds a predetermined threshold.

[0156] In step 940, the ship safety control system can determine whether the determined RGB feature exceeds a predetermined threshold.

[0157] Here, the predetermined threshold may be the ratio of the RGB feature value to the total number of pixels included in the frame.

[0158] In step 950, the ship safety control system may perform rainfall preprocessing if the RGB features exceed a predetermined threshold. For example, the ship safety control system may perform a preprocessing process for rainfall conditions based on the Discrete Cosine Transform (DCT) algorithm and the inverse DCT algorithm.

[0159] For example, a ship safety control system can compress a first image through a DCT algorithm. Then, in the process of restoring the compressed first image through an inverse DCT algorithm, the ship safety control system can restore pixels in areas recognized as water droplets using the RGB values ​​of adjacent pixels.

[0160] Figure 10 is an illustrative diagram illustrating a region division method according to one embodiment.

[0161] Referring to Figure 10, the image captured by the camera may include the ship region 1010 and the sea region 1020.

[0162] Ship safety control systems can use image segmentation models to specify control areas. Image segmentation models are used to identify and separate objects or regions within an image as a method of separating an image at the pixel level.

[0163] For example, an image segmentation model can divide a second image into pixel units and determine whether each pixel belongs to the sea area 1020 or the ship area 1010. The image segmentation model can determine the pixel division criteria based on previously learned information.

[0164] Furthermore, the ship safety control system may generate a virtual line 1030 connecting pixels corresponding to the portion where the sea area 1020 and the ship area 1010 are connected. For example, the ship safety control system can designate at least one of the sea area 1020 and the ship area 1010 as a control area based on the virtual line 1030.

[0165] For example, if an object detected in the ship area 1010 moves to the sea area 1020 and is detected in the sea area 1020 for a predetermined period of time, the ship safety control system can determine that a drowning has occurred.

[0166] Figure 11 is an illustrative diagram illustrating a method for maintaining a user-specified area when the imaging area of ​​an imaging device according to one embodiment is changed.

[0167] Referring to Figure 11, the imaging area 1110 of the imaging device may include a user-designated area 1120 for safety control. For example, the user can arbitrarily set the user-designated area 1120 for safety control in advance via an input / output interface included in the ship's safety control device.

[0168] For example, if the imaging area 1110 of the imaging device is changed due to ship vibration or control of the imaging device, it is necessary to maintain the user-specified area 1120 in order to detect the occurrence of an event in the area specified in advance by the user.

[0169] The ship safety control system can extract feature points 1121 from the user-specified region 1120 and maintain the user-specified region 1120.

[0170] For example, the ship safety control system can extract feature points 1121 from a user-specified region 1120 where the pixel value changes rapidly. Furthermore, the ship safety control system can extract feature points 1121 from the user-specified region 1120 using the ORB (Oriented FAST and Rotated BRIEF) algorithm and the FAST (Features from Accelerated Segment Test) algorithm.

[0171] The ship safety control system can maintain and recognize the user-specified region 1120 by tracking extracted feature points 1121 when the imaging area 1110 changes. Because this tracking of feature points 1121 is based on frame continuity, the user-specified region 1120 can be accurately maintained and recognized.

[0172] For example, the ship safety control system can estimate a homography matrix showing the transformation relationship between the initial imaging area and the modified imaging area based on the tracked feature points 1121. The ship safety control system can calculate the transformed region according to the estimated homography matrix and maintain and recognize the user-specified region 1120.

[0173] Furthermore, if a moving object excluding the feature point 1121 is detected in the user-specified region 1120, the ship safety control system can mask the moving object to improve the accuracy of the recognition of the user-specified region 1120.

[0174] Figure 12 is an illustrative diagram illustrating a method for detecting a fire event according to one embodiment.

[0175] Referring to Figure 12, the ship safety control system can detect fire 1230 or smoke in a designated control area 1220 within a pre-processed second image 1210 using an object detection algorithm.

[0176] For example, a ship safety control system can determine that a fire event has occurred if fire 1230 or smoke is detected within the control area 1220. However, if the ship safety control system detects fire 1230 due to an error and immediately determines that a fire event has occurred, a problem may arise in that the accuracy of fire recognition is reduced.

[0177] The ship's safety control system can determine that a fire event has occurred if the number of times fire or smoke is detected within a predetermined period exceeds a predetermined threshold.

[0178] For example, a ship's safety control system can detect fire or smoke every two seconds. If fire or smoke is detected four or more times within a 20-second period, the ship's safety control system can determine that a fire event has occurred.

[0179] Figure 13 is an illustrative diagram illustrating a method for detecting a fall event according to one embodiment.

[0180] Referring to Figure 13, the ship's safety control system can detect when a person 1330 falls in the control area 1310.

[0181] The ship's safety control system can determine the bounding box of person 1330. Here, the bounding box means the smallest rectangle containing person 1330.

[0182] A ship's safety control system can calculate the aspect ratio of the bounding box. For example, in typical circumstances, the horizontal length of a person's bounding box 1320 is shorter than its vertical length. In this case, the ratio of the vertical length to the horizontal length of the bounding box 1320 may be greater than 1.

[0183] However, if person 1330 falls or tumbles, the horizontal length of person 1330's bounding box 1340 will be longer than its vertical length. In this case, the ratio of the vertical length to the horizontal length of the bounding box 1340 may be less than 1.

[0184] The ship's safety control system can determine that a tipping event has occurred if, within a predetermined period, the ratio of the length to the width of the bounding box is less than 1 for a predetermined amount of time equal to or greater than a predetermined threshold.

[0185] For example, a ship safety control system can determine that a tipping event has occurred if, within a 30-second period, the ratio of the length to the width of the bounding box is less than 1 for 5 seconds or more.

[0186] Furthermore, the ship's safety control system can recognize when person 1330 has fallen or is performing a rescue signal based on a human pose estimation algorithm. For example, the human pose estimation algorithm can estimate the posture of person 1330 based on the joint positions. If the estimated posture of person 1330 corresponds to the posture of a rescue signal action stored in the algorithm, the ship's safety control system can determine that person 1330 is transmitting a rescue signal.

[0187] Figure 14 is an illustrative diagram illustrating a method for detecting an event of not wearing safety equipment according to one embodiment.

[0188] Referring to Figure 14, within the pre-processed second video 1410, the person 1430 and safety equipment 1440 can be detected in the designated control area 1420 using an object detection algorithm.

[0189] Furthermore, the ship's safety control system can calculate the distance 1450 between the person 1430 and the safety equipment 1440. If the calculated distance 1450 exceeds a predetermined threshold, the ship's safety control system can determine that a safety equipment non-wearing event has occurred.

[0190] Furthermore, the ship's safety control system can determine that a safety equipment non-wearing event has occurred if the time during which the calculated distance of 1450 within a predetermined period exceeds a predetermined threshold is greater than or equal to a predetermined threshold.

[0191] For example, a ship safety control system can determine that a tipping event has occurred if the distance of 1450 calculated within a 20-second period exceeds a predetermined threshold for 4 seconds or more.

[0192] The ship's safety control system can detect the occurrence of an event where safety equipment is not being worn, based on a long-range object detection model and a short-range object detection model. For example, the ship's safety control system can detect a person 1430 using the long-range object detection model. Safety equipment 1440, which is smaller in size than a person 1430, may not be detected by the long-range object detection model.

[0193] If a person 1430 is detected by the long-range object detection model, the ship's safety control system can zoom in on the area near the detected person 1430. The ship's safety control system can then use the short-range object detection model to detect safety equipment 1440 in the zoomed-in area.

[0194] Figure 15 is an illustrative diagram illustrating a method for detecting events associated with an increase in movement speed according to one embodiment.

[0195] Referring to Figure 15, Image 1510 shows the labeling results of objects detected in the work area. Image 1520 shows the calculated distance and direction of movement for the detected objects.

[0196] Referring to Figure 1510, the ship safety control system can detect people in the work area using an object detection model. Furthermore, the ship safety control system can perform labeling of people detected in the work area. Through labeling, the ship safety control system can issue an alarm to draw attention only to people who exceed a predetermined movement speed.

[0197] Referring to Figure 1520, the ship safety control system can calculate the relative speed of movement based on the bounding box 1521 and the distance traveled 1522 of a person. For example, the ship safety control system can correct for distortions that occur based on the field of view (FoV) of the camera.

[0198] The ship's safety control system can determine that an event associated with an increase in movement speed has occurred if the movement speed of a person exceeds a predetermined threshold.

[0199] Furthermore, the ship safety control system can determine that an event associated with an increase in speed has occurred if the time during which the ship's speed exceeds a predetermined threshold is greater than or equal to a predetermined threshold within a predetermined period.

[0200] For example, a ship safety control system can determine that an event associated with an increase in speed has occurred if the ship's speed exceeds a predetermined threshold for 3 seconds or more over a 20-second period.

[0201] Figure 16 is an illustrative diagram illustrating a method for detecting crowd-generating events according to one embodiment.

[0202] Referring to Figure 16, Image 1610 corresponds to an image of the imaging area where the control area 1611 is designated. Furthermore, Image 1620 corresponds to an image of the imaging area showing the bounding boxes 1621 and 1622 of an object detected within the control area 1611. Additionally, Image 1630 corresponds to an image of the imaging area showing the proportion of the control area 1611 occupied by the bounding boxes 1621 and 1622.

[0203] Referring to image 1610, the ship safety control system can detect multiple objects in the control area 1611.

[0204] Referring to Figure 1620, the ship safety control system can determine bounding boxes 1621 and 1622 for each object detected within the control area 1611. Here, bounding boxes 1621 and 1622 represent the smallest rectangle containing the object.

[0205] Referring to Figure 1630, the ship safety control system can calculate the proportion of each object's bounding boxes 1621 and 1622 that occupy within the control area 1611. For example, it may be calculated that bounding box 1621 occupies 39% of the control area 1611, and bounding box 1622 occupies 32% of the control area 1611.

[0206] The ship safety control system can determine that an event associated with crowd formation has occurred if the proportion of all bounding boxes 1621 and 1622 within the control area 1611 exceeds a predetermined threshold.

[0207] Furthermore, the ship's safety control system can determine that an event associated with crowd formation has occurred if the proportion of all bounding boxes 1621 and 1622 that occupy a predetermined period exceeds a predetermined threshold for a period of time equal to or greater than a predetermined threshold.

[0208] For example, a ship safety control system can determine that a crowd-related event has occurred if the proportion of all bounding boxes over a 20-second period exceeds a predetermined threshold for more than 4 seconds.

[0209] Figure 17 is an illustrative diagram illustrating a method for detecting an event caused by intrusion into a hazardous area according to one embodiment.

[0210] Referring to Figure 17, within the pre-processed second video 1710, a person 1730 can be detected in the designated control area 1720 using an object detection algorithm.

[0211] The ship safety control system can designate control area 1720 as a hazardous area. If a person 1730 is detected in control area 1720 designated as a hazardous area, the ship safety control system can determine that an event associated with intrusion into the hazardous area has occurred.

[0212] Furthermore, the ship's safety control system can determine that an event associated with intrusion into a hazardous area has occurred if the time during which a person 1730 is detected in a control area 1720 designated as a hazardous area within a predetermined period exceeds a predetermined threshold.

[0213] For example, a ship's safety control system can determine that an event has occurred due to intrusion into a hazardous area if, within a 20-second period, a person 1730 is detected in a designated hazardous area control area 1720 for 3 seconds or more.

[0214] Figure 18 is an illustrative diagram illustrating a method for detecting a sea crash event according to one embodiment.

[0215] Referring to Figure 18, the ship safety control system can divide the area contained within the second image 1800 into a sea area 1820 and a ship area 1810. For example, the ship safety control system can divide the area into a sea area 1820 and a ship area 1810 based on image segmentation technology.

[0216] Furthermore, the ship safety control device may generate virtual lines 1830 connecting pixels corresponding to the portion where the sea area 1820 and the ship area 1810 are connected.

[0217] The ship safety control system can determine whether person 1840 is moving from the ship area 1810 to the sea area 1820.

[0218] Furthermore, the ship safety control system can determine that a fall at sea event has occurred if, after a person 1840 moves from the ship area 1810 to the sea area 1820, the time during which person 1840 is detected in the sea area 1820 exceeds a predetermined threshold.

[0219] For example, the ship safety control system can determine that a fall at sea event has occurred if, within 20 seconds after person 1840 moves from the ship area 1810 to the sea area 1820, the time during which person 1840 is detected in the sea area 1820 exceeds 4 seconds.

[0220] Figure 19 is an illustrative diagram illustrating a sorting and control method according to one embodiment.

[0221] Referring to Figure 19, Image 1900 corresponds to a screen displaying all the video footage of the ships, yards, and shipyards.

[0222] The ship safety control system can provide the user with video footage from all locations where cameras are installed. Furthermore, the ship safety control system can help the user easily identify the location of an event by highlighting and providing the user with video footage of the location where the event occurred.

[0223] Embodiments of the present invention can be implemented in the form of a computer program that can be executed on a computer via various components, and such a computer program can be recorded on a computer-readable medium. In this case, the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical recording media such as floptical disks; and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memory.

[0224] On the other hand, the computer program may be specifically designed and configured for the present invention, or it may be publicly known and available to those skilled in the field of computer software. Examples of computer programs may include not only machine code, such as that produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

[0225] According to one embodiment, the methods according to various embodiments of the present disclosure may be provided in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or through an application store (e.g., Play Store). TM It can be distributed online (e.g., by download or upload) via a network or directly between two user devices. In the case of online distribution, at least a portion of the computer program product can be temporarily stored or temporarily generated on a storage medium readable by equipment such as the memory of the manufacturer's server, the application store's server, or an intermediary server.

[0226] With respect to the steps constituting the method according to the present invention, unless otherwise stated, the steps may be performed in any order that suits them. The present invention is not necessarily limited by the order in which the steps are described. The use of all examples or exemplary terms (e.g., etc.) in the present invention is merely for the purpose of illustrating the present invention in detail, and the scope of the present invention is not limited by such examples or exemplary terms unless otherwise limited by the claims. Furthermore, those skilled in the art will understand that the claims or their equivalents can be constructed according to design conditions and factors, with various modifications, combinations, and changes.

[0227] Therefore, the concept of the present invention should not be limited to the embodiments described above, and it can be said that not only the claims described later, but also all scopes equivalent to or equivalently modified from these claims, fall within the scope of the concept of the present invention.

Claims

1. As a method of safe control of ships, A step of determining the maritime conditions in the area where the vessel is located based on the video features of a first video showing the vessel's condition, The steps include preprocessing the first video based on the aforementioned sea conditions, A step of specifying a control area within the second video based on the second video generated by the aforementioned preprocessing, The steps include detecting the occurrence of at least one object or event within the control area, Methods that include...

2. The aforementioned video feature is The method according to claim 1, comprising at least one of an illuminance feature, an RGB feature, and a dark channel feature.

3. The aforementioned pre-processing step is: If the value of the illuminance feature exceeds a first threshold, nighttime preprocessing is performed on the first video. If the value of the RGB feature exceeds the second threshold, rainfall preprocessing is performed on the first image. The method according to claim 2, wherein if the value of the dark channel feature exceeds a third threshold, pre-processing of the first image with sea fog is performed.

4. The step of determining the aforementioned maritime conditions is: The steps include calculating the brightness value of each RGB channel of the first video, The steps include determining the channel having the minimum luminance value among the RGB channels, A step of calculating the average brightness value of pixels in the first video for the channel determined above, The steps of determining the average brightness value as the value of the dark channel feature and The method according to claim 2, including the method described in claim 2.

5. The step of determining the aforementioned maritime conditions is: The steps include calculating the brightness value of each pixel included in the frame of the first video, The steps include: comparing the brightness value of each pixel in the frame with the brightness value of each pixel in an adjacent frame to calculate the change in the brightness value of each pixel; The steps include determining the number of pixels whose change in brightness value exceeds a predetermined threshold as the value of the RGB feature, The method according to claim 2, including the method described in claim 2.

6. The first threshold is, The method according to claim 3, wherein the setting is based on at least one of the background conditions of the first video, the average brightness value of the RGB channels of each frame in the first video, and a control image in which no sea fog was detected.

7. The second threshold is, The method according to claim 2, wherein the setting is based on at least one of the background conditions of the first video, the average brightness value of the RGB channels of each frame in the first video, and the control image in which no rainfall was detected.

8. The step of specifying the control area is, The steps include dividing the second image into a maritime area and a ship area, The steps include designating at least one of the aforementioned sea area and ship area as the control area, The method according to claim 1, including the method described in claim 1.

9. The step of specifying the control area is, The steps include detecting whether the shooting area of ​​the first video can be changed, If the shooting area of ​​the first video is changed, the steps include maintaining and recognizing the designated control area, The method according to claim 1, including the method described in claim 1.

10. The step of detecting the occurrence of at least one of the aforementioned objects or events is: The steps include determining the bounding box of the object, A step of recognizing the occurrence of the event based on the proportion or movement of the bounding box, The method according to claim 1, including the method described in claim 1.

11. The step of recognizing the occurrence of the aforementioned event is: The steps include determining whether at least one of the following exceeds a predetermined threshold: the aspect ratio of the bounding box, the distance between multiple bounding boxes, the movement speed of the bounding box, and the proportion of the control area occupied by the bounding box. The method according to claim 10, further comprising:

12. A computer-readable recording medium that stores a program for causing a computer to perform the method described in claim 1.

13. At least one memory, Includes at least one processor, The aforementioned at least one processor is Based on the video features of a first video showing the condition of the vessel, the maritime conditions of the area where the vessel is located are determined; the first video is preprocessed in different ways according to the maritime conditions; and at least one of the occurrence of an object or an event is detected based on the second video generated by the preprocessing. The aforementioned video feature is A computing device comprising at least one of the following: an illuminance feature, an RGB feature, and a dark channel feature.

14. The aforementioned processor, The luminance values ​​of each RGB channel in the first video are calculated, the channel having the minimum luminance value among the RGB channels is determined, and the average luminance value of the pixels in the first video for the determined channel is determined as the value of the dark channel feature. The computing device according to claim 13, wherein if the value of the dark channel feature exceeds a first threshold, the first video is preprocessed such that the luminance value of at least one of the RGB channels of the second video tends to be significantly lower than the luminance value of the other channels.

15. The aforementioned processor, The luminance value of each pixel included in the frame of the first video is calculated, the luminance value of each pixel in the frame is compared with the luminance value of each pixel in an adjacent frame to calculate the change in the luminance value of each pixel, and the number of pixels whose change in luminance value exceeds a predetermined threshold is determined as the RGB feature. The computing device according to claim 13, wherein if the value of the RGB feature exceeds a second threshold, the device compresses and restores the frames in the first image and performs rain preprocessing on the first image based on information obtained by inversely calculating the color information of pixels having a relatively high change in the average brightness value of the RGB channels.

Images