Method and system for automated ship check-in and boarding registration

The intelligent ship check-in system uses biometric and 3D verification, along with real-time monitoring, to enhance security and compliance in vessel boarding, addressing vulnerabilities in existing systems and preventing illegal boarding and accidents.

KR103046520B1Active Publication Date: 2026-11-11SGMA CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
KR1020250187217
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Priority Date
2025-02-28
Filing Date
2025-12-01
Publication Date
2026-11-11
Estimated Expiration
2045-12-01

AI Technical Summary

Technical Problem

Existing vessel boarding management systems face challenges in preventing illegal boarding, tracking safety equipment status, and ensuring compliance with maritime regulations due to limitations in biometric authentication, QR code vulnerabilities, and lack of real-time monitoring, especially in offline environments.

Method used

An intelligent ship check-in and boarding report automation system utilizing biometric authentication, 3D object verification, human skeletal detection, and real-time location tracking, combined with an encrypted token-based boarding report processing, to ensure compliance and prevent illegal boarding.

Benefits of technology

Enhances the reliability and security of biometric authentication, enforces safety regulation compliance, and prevents marine safety accidents by continuously monitoring safety equipment wearing and location, while providing legal evidence for accidents.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112025135254034-PAT00005_ABST
    Figure 112025135254034-PAT00005_ABST
Patent Text Reader

Abstract

The automated system for ship check-in and boarding reports may include memory and a processor for storing instructions. The system acquires situational awareness data in real time, including the user's biometric information, the status of safety equipment wearing, and the results of detecting ship beacon signals, through sensors and cameras of a user terminal; analyzes the acquired situational awareness data to determine whether the biometric information was acquired from a real-time subject rather than through abnormal input using a pre-acquired image or video, whether the safety equipment is located on a designated part of the body, and whether the user terminal is located within the authorized radius of the ship; and based on the fact that all verification conditions are satisfied, it generates a token in which the pattern is updated in real time by combining the hash value of the situational awareness data with time information, encrypts the generated token by electronically signing it with the user's private key, and transmits it to a ship manager terminal via near-field wireless communication to perform boarding list registration.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to the field of ship boarding management and marine safety management technology.

[0002] More specifically, the invention relates to an intelligent ship check-in and boarding report automation system and a method of operation thereof, which integrates biometric authentication, safety equipment wearing verification, and real-time location verification to fundamentally prevent illegal boarding and violations of safety regulations, and implements encrypted token-based automated boarding report processing. Background Technology

[0004] With the recent increase in the marine leisure industry and fishing activities, the importance of vessel boarding management is being highlighted. In particular, verifying the identity of passengers, checking the use of safety equipment, and maintaining accurate passenger manifests are essential for preventing safety accidents and illegal fishing. However, existing vessel boarding management systems have primarily relied on paper manifests or simple QR code scanning methods.

[0005] Existing QR code-based boarding authentication methods had a problem in that once a code was generated, it could be captured and transmitted or shared with others, making it difficult to effectively prevent proxy boarding or illegal boarding by unqualified individuals. Additionally, since QR codes contain only static information, they cannot reflect real-time situational changes, and there are limitations in that the passenger's safety equipment status or location changes cannot be tracked after a one-time authentication.

[0006] Furthermore, although there have been attempts to apply biometric authentication technology to boarding verification, its practicality has been low due to limitations unique to the maritime environment. Facial recognition failure rates were high during dawn or nighttime hours, or in maritime environments with strong backlighting, and there are issues with insufficient security as it is vulnerable to spoofing attacks using simple photos or videos. The problem to be solved

[0008] Regarding the wearing of safety equipment, violations of safety regulations frequently occurred where individuals wore life jackets during the boarding declaration but removed them after actually boarding. Existing systems performed only a one-time verification at the time of boarding and lacked a mechanism to continuously monitor the wearing status after boarding or to link the wearing status with boarding authentication. Furthermore, there was a limitation in that it could not distinguish between merely possessing safety equipment and actually wearing it correctly on the body.

[0009] For deep-sea fishing vessels or ships sailing to distant waters, situations may arise where connection to a central server is impossible due to communication dead zones. In such offline environments, the creation and management of passenger manifests had to be done manually, and there was a lack of technical means to verify the possibility of data falsification or alteration after the fact.

[0010] The present invention aims to effectively detect spoofing attacks using photos or videos even under poor lighting conditions in a marine environment and to allow authentication only from actual biometrics by applying a 3D object verification technology utilizing multi-wavelength illumination signals.

[0011] The present invention aims to enforce compliance with safety regulations by utilizing a human skeletal detection algorithm to determine whether safety equipment is correctly positioned on a specific part of the user's body and by automatically revoking the validity of the token in real time if the wearing state is not maintained.

[0012] The present invention aims to block remote hacking or signal manipulation by calculating the precise location of a user terminal using a triangulation method through the measurement of the reception strength and arrival time of a beacon signal, and by performing cross-verification with GPS coordinates.

[0013] The present invention aims to simultaneously achieve compliance with regulations and prevent safety accidents by automatically verifying closed seasons, minimum catch sizes, and licensing requirements in conjunction with a fisheries resource regulation database, and by verifying the suitability of safety equipment based on real-time weather forecast data. means of solving the problem

[0015] According to one embodiment, the system (400) can acquire situational awareness data in real time, including the user's biometric information, the status of safety equipment wearing, and the results of detecting ship beacon signals, through the sensor and camera of the user terminal. The system (400) can analyze the acquired situational awareness data to determine whether the biometric information was acquired from a real-time subject rather than through abnormal input using a pre-acquired image or video, whether the safety equipment is located on a designated part of the body, and whether the user terminal is located within the authorized radius of the ship. Based on the fact that all verification conditions are satisfied, the system (400) can generate a token in which the pattern is updated in real time by combining the hash value of the situational awareness data with time information. The system (400) can control the registration of the boarding list by electronically signing and encrypting the generated token with the user's private key, and then transmitting it to a ship manager terminal via near-field wireless communication.

[0016] According to one embodiment, the system (400) may generate a first hash value by unidirectionally encrypting situation awareness data at a time when all verification conditions are satisfied, generate a dynamic seed combining time flow and situation information by combining the calculated first hash value with timestamp information updated at a preset period, and generate a one-time random number code based on the generated dynamic seed. The system (400) may periodically compare a second hash value calculated from situation awareness data reacquired in real time after token generation with the first hash value, and determine a security violation state caused by the user removing safety equipment or deviating from the vessel's radius when the two hash values ​​do not match. Based on the determination of a security violation state, the system (400) may control the system to stop generating the random number code or to block boarding processing by discarding the validity verification key value of the previously generated token.

[0017] According to one embodiment, the system (400) can analyze joint feature points of the human body within a user image acquired through a camera to extract multiple skeletal coordinates corresponding to the left and right shoulders, elbows, wrists, and pelvis, and define an area corresponding to the user's torso by connecting the extracted left and right shoulder coordinates and pelvic coordinates. The system (400) can acquire a bounding box or mask area of ​​the safety equipment identified through object recognition, calculate an Inter-Area of ​​Units (IoU) where the area of ​​the safety equipment overlaps with the area corresponding to the torso, and determine a wearing state when the area ratio is greater than or equal to a preset threshold. The system (400) can determine a carrying state when the area ratio is less than the threshold and the area of ​​the safety equipment is located near the wrist or elbow coordinates, and determine a non-wearing state when the area of ​​the safety equipment is separated from the skeletal coordinates, thereby verifying whether the safety equipment is located in a designated area.

[0018] According to one embodiment, the system (400) receives signals transmitted from beacons distributed at multiple points within the boarding area of ​​the vessel, and measures the received signal strength (RSSI) and time of arrival (ToF) of each beacon signal in real time to calculate the relative coordinates of the user terminal using a triangulation method. The system (400) may determine that the user terminal is located within the authorized radius of the vessel only if the calculated relative coordinates are located within a safe zone within a preset threshold distance, and the error between the absolute coordinates obtained through the GPS module and the calculated relative coordinates is within an allowable range. The system (400) may control the boarding manifest registration procedure to be stopped if the error exceeds the allowable range or if the reception pattern of the beacon signal changes abnormally rapidly.

[0019] According to one embodiment, when acquiring biometric information, the system (400) emits a lighting signal that changes continuously in a set order of different brightness or different color when the ambient illuminance is below a preset threshold, and can analyze the reflectance distribution according to the change in light source by generating a difference image between a plurality of face image frames acquired before and after the emission of the lighting signal. The system (400) extracts face landmarks corresponding to major feature points of the face from a plurality of face image frames acquired while the lighting signal is being emitted, generates an expected 3D shape model based on spatial arrangement information of the extracted landmarks, and can calculate a pattern in which the brightness of each pixel within the plurality of face image frames changes according to the change in the lighting signal. The system (400) compares the calculated brightness change pattern with a reference brightness change pattern simulated based on the surface curvature of the expected 3D shape model and the direction of the light source of the lighting signal, and can determine it as an actual 3D subject if the match rate between the two patterns is greater than or equal to a preset value. The system (400) can determine that the calculated brightness change pattern is a two-dimensional image if the surface normal vector shows a tendency to match the reflection characteristics of a planar model having a constant direction, if a total reflection component exceeding a preset threshold is detected in a local area, or if pixel grid noise generated during display re-capture within the image is detected. The system (400) can determine that the input is normal if it is determined to be a three-dimensional object, and determine that the input is abnormal if it is determined to be a two-dimensional image, judging it as an attempt at spoofing.

[0020] According to one embodiment, the system (400) may obtain information on the main target fish species that are the target of the fishing trip from the vessel's departure report information, and may extract regulatory data including information on the closed season, minimum catch size, and prohibited catch zone set for the main target fish species by querying a pre-established fishery resource regulation database. If the system (400) determines that the time of boarding falls within the closed season or that the vessel's route includes a prohibited catch zone, it may obtain user consent by displaying the regulatory data as a visual pop-up on the screen of the user terminal. If a separate license or permit is required for the capture of the main target fish species, the system (400) may induce the user to activate the camera of the user terminal to photograph the fishing license or capture permit, and may verify the validity by applying an optical character recognition algorithm to the acquired permit image to extract the permit number, validity period, and owner information as text. The system (400) can control the generation of a token whose pattern is updated in real time regardless of whether the verification conditions are met, if the verification result shows that the corresponding permit is invalid or the owner information is inconsistent with the user identified through biometric information.

[0021] According to one embodiment, the system (400) can monitor the communication connection status of the user terminal and, if it is determined to be a dead zone where communication with the server is impossible, execute an offline mode that switches the ship manager terminal to a temporary verification terminal capable of confirming boarding status. The system (400) can transmit a token, which is updated in real-time and generated to prove that verification conditions including biometrics, safety equipment, and location are satisfied through the user terminal, to the ship manager terminal via short-range wireless communication. The system (400) can decrypt the electronic signature of the token received through the ship manager terminal to perform a first verification of validity, then perform boarding approval processing, and store sequentially occurring boarding history data in a chain form in which the hash value of the previous data is included in the next data, thereby configuring the entire chain link to be destroyed if the order of the stored data is changed or the content is tampered with. When communication is restored, the system (400) receives a bundle of boarding history data stored in a chain form from a ship manager terminal, and checks whether the result obtained by recalculating the received data sequentially from the beginning matches the result finally generated by the ship manager terminal, thereby controlling the verification of integrity, which is whether the boarding list stored in an offline state has been tampered with or forged.

[0022] According to one embodiment, the system (400) can receive MME-based marine weather forecast data regarding the departure route and time period of the vessel from an external weather observation server and calculate a weather risk level including wave height, wind speed, and swell index in real time. If the calculated weather risk level exceeds a preset safety threshold, the system (400) can extract equipment specification data including color information of the equipment, whether reflective material is attached, and buoyancy performance text marked on the surface by performing object recognition and optical character recognition on an image of safety equipment acquired through a camera of a user terminal. The system (400) can further perform conformity verification by querying the essential safety equipment grade criteria required under the weather conditions from a preset database in response to the calculated weather risk level and comparing and analyzing whether the extracted equipment specification data is equal to or greater than the essential safety equipment grade criteria. If the system (400) determines, based on comparative analysis results, that the specifications of the safety equipment worn by the user do not meet the essential safety equipment grade standards, it can block the generation of a token whose pattern is updated in real time regardless of the validity of biometric information and location information, and control the output of a guidance message instructing the user terminal to replace it with a higher grade safety equipment on the screen. Effects of the invention

[0024] The present invention can improve the reliability and security of biometric authentication by applying a 3D subject verification technology based on multiple lighting signals to effectively detect spoofing attacks using photos, videos, masks, etc., even in poor lighting conditions and backlighting situations in marine environments, and by allowing only authentication from actual biometrics.

[0025] This invention utilizes human skeletal detection technology to clearly distinguish between the simple possession and actual wearing of safety equipment, and by monitoring changes in the wearing status in real time and linking them with token validity, it enforces compliance with safety regulations without the intervention of an administrator, thereby having the effect of preventing marine safety accidents.

[0026] The present invention includes an electronic signature in an encrypted token, enabling all situational data at the time of boarding to be utilized as legal evidence and having the effect of clearly proving liability in the event of an accident. Brief explanation of the drawing

[0028] FIG. 1 is a diagram illustrating the overall structure of an artificial intelligence-based system according to one embodiment. FIG. 2 is a diagram illustrating the learning of a neural network according to one embodiment. FIG. 3 is a diagram illustrating the configuration of an artificial intelligence model according to one embodiment. FIG. 4 is a block diagram showing the configuration of an automated system for ship check-in and boarding reporting according to one embodiment. FIG. 5 is a flowchart illustrating the operation method of a ship boarding authentication system according to one embodiment. FIG. 6 illustrates a method for determining the wearing status of safety equipment based on skeletal detection according to one embodiment. FIG. 7 illustrates a method for verifying fishery resource regulations and a capture permit according to one embodiment. Specific details for implementing the invention

[0029] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, various modifications may be made to the embodiments, and thus the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, and substitutions to the embodiments are included within the scope of the rights.

[0030] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed forms, and the scope of this specification includes modifications, equivalents, or substitutions that fall within the technical concept.

[0031] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0032] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0033] The terms used in the embodiments are for illustrative purposes only and should not be interpreted as intended to be limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0034] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the embodiments pertain. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0035] In addition, when describing with reference to the attached drawings, identical components are assigned the same reference numeral regardless of drawing symbols, and redundant descriptions thereof are omitted. In describing the embodiments, if it is determined that a detailed description of related prior art could unnecessarily obscure the essence of the embodiments, such detailed description is omitted.

[0036] The embodiments can be implemented in various forms of products such as personal computers, laptop computers, tablet computers, smartphones, televisions, smart home appliances, intelligent automobiles, kiosks, and wearable devices.

[0037] Artificial Intelligence (AI) systems are computer systems that implement human-level intelligence; unlike existing rule-based smart systems, they are systems in which machines learn and make decisions autonomously. As AI systems improve in recognition accuracy and gain a more accurate understanding of user preferences with continued use, existing rule-based smart systems are gradually being replaced by deep learning-based AI systems.

[0038] Artificial intelligence technology consists of machine learning and component technologies utilizing machine learning. Machine learning is an algorithmic technology that autonomously classifies and learns the characteristics of input data, while component technologies are technologies that mimic the cognitive and judgmental functions of the human brain by utilizing machine learning algorithms such as deep learning, and are comprised of technological fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control.

[0039] The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding refers to technologies that recognize, apply, and process human language and text, including natural language processing, machine translation, dialogue systems, question answering, and speech recognition / synthesis. Visual understanding refers to technologies that perceive and process objects like human vision, including object recognition, object tracking, image search, people recognition, scene understanding, spatial understanding, and image enhancement. Inference and prediction refers to technologies that logically reason and predict by judging information, including knowledge / probability-based inference, optimization prediction, preference-based planning, and recommendation. Knowledge representation refers to technologies that automatically process human experiential information into knowledge data, including knowledge construction (data generation / classification) and knowledge management (data utilization). Motion control refers to technologies that control the autonomous driving of vehicles and the movement of robots, including motion control (navigation, collision, driving) and manipulation control (behavior control).

[0040] Generally, to apply machine learning algorithms to real-world situations, training is performed using a trial-and-error method due to the inherent characteristics of the fundamental methodologies. In particular, deep learning requires hundreds of thousands of iterations. Since it is impossible to execute this in a real physical external environment, training is instead performed through simulations that virtually recreate the actual physical environment on a computer.

[0041] In the present invention, Artificial Intelligence (AI) refers to a technology that imitates human learning ability, reasoning ability, and perceptual ability, and implements them on a computer, and may include concepts such as machine learning and symbolic logic. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. AI technology can analyze input data as a machine learning algorithm, learn from the results of the analysis, and make judgments or predictions based on the results of the learning. Furthermore, technologies that mimic the functions of the human brain, such as cognition and judgment, by utilizing machine learning algorithms can also be understood as falling within the category of AI. For example, technological fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control may be included.

[0042] Machine learning can refer to the process of training neural network models using experience in processing data. Through machine learning, computer software can improve its own data processing capabilities. A neural network model is constructed by modeling the correlations between data, and these correlations can be expressed by multiple parameters. A neural network model extracts and analyzes features from given data to derive correlations between them; machine learning can be defined as the process of optimizing the model's parameters by repeating this process. For example, a neural network model can learn the mapping (correlation) between inputs and outputs for data given as input-output pairs. Alternatively, even when only input data is provided, a neural network model can derive regularities between the given data and learn those relationships.

[0043] An artificial intelligence learning model or neural network model can be designed to implement the structure of the human brain on a computer and may include multiple network nodes that have weights and simulate neurons of a human neural network. The multiple network nodes may have interconnected relationships by simulating the synaptic activity of neurons, where neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes may be located in layers of different depths and exchange data according to convolutional connections. The artificial intelligence learning model may be, for example, an Artificial Neural Network (ANN) or a Convolutional Neural Network (CNN). As an embodiment, the artificial intelligence learning model may be machine learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning may include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, Ada-boost, Perceptron, Genetic Programming, and Clustering.

[0044] Among these, CNNs are a type of multilayer perceptron designed to use minimal preprocessing. CNNs consist of one or more convolutional layers and standard artificial neural network layers stacked on top, additionally utilizing weights and pooling layers. Thanks to this structure, CNNs can fully utilize two-dimensional input data. Compared to other deep learning architectures, CNNs demonstrate good performance in both image and audio fields. CNNs can also be trained using standard backpropagation. CNNs have the advantage of being easier to train than other feedforward artificial neural network techniques and using a small number of parameters.

[0045] Convolutional networks are neural networks comprising sets of nodes with bounded parameters. Many computer vision tasks have been significantly improved, driven by the increased size of available training data and the availability of computational power, combined with algorithmic advancements such as discriminative linear units and dropout training. In the case of massive datasets, such as those available for many tasks today, outfitting is not critical, and increasing the network size improves test accuracy. Optimal utilization of computing resources becomes a limiting factor. To address this, distributed, scalable implementations of deep neural networks can be employed.

[0047] FIG. 1 is a diagram illustrating the overall structure of an artificial intelligence-based system according to one embodiment.

[0048] As illustrated in FIG. 1, an artificial intelligence-based system (100) may include a plurality of user terminals (110-1 to 110-n), a server (120), and a database (130). This system adopts a distributed architecture and operates based on a client-server model, with each component playing a unique role and contributing to increasing the efficiency and scalability of the entire system. According to one embodiment, the database (130) is depicted as being configured separately from the server (120), but is not limited thereto; depending on system design and operational efficiency, the database (130) may be integrated within the server (120). This integrated configuration has the advantage of improving data access speed and reducing system complexity. For example, the server (120) may include a plurality of artificial intelligence models and processing units for performing machine learning algorithms, and these can implement various types of deep learning and machine learning technologies (e.g., CNN, RNN, Transformer, reinforcement learning, etc.) to respond to user requests and provide intelligent services. According to one embodiment, a plurality of user terminals (110-1 to 110-n), a server (120), and a database (130) can be connected to communicate with each other through a network (N), which enables real-time data exchange and smooth interaction between system components.

[0049] According to one embodiment, the network (N) may perform wireless or wired communication between a plurality of user terminals (110-1 to 110-n), a server (120), a database (130), etc., and is located in the center of FIG. 1 to serve as a hub connecting all components. For example, the network may perform wireless communication according to methods such as 5G, LTE (Long-Term Evolution), LTE-A (LTE Advanced), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), WiBro (Wireless BroadBand), WiFi (Wireless Fidelity), Bluetooth, NFC (Near Field Communication), GPS (Global Positioning System), or GNSS (Global Navigation Satellite System). The 5G network supports high-speed data transmission (up to 20Gbps), ultra-low latency (1ms or less), and large-scale connectivity, making it suitable for real-time processing of large-capacity AI models. For example, the network (N) may be configured to perform wired communication using methods such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), RS-232 (Recommended Standard 232), Ethernet, fiber optic cable, or POTS (Plain Old Telephone Service). In particular, in data center environments, high-performance network technologies such as 400Gbps Ethernet or InfiniBand can be utilized for communication between large-scale AI computing clusters.

[0050] According to one embodiment, user terminals (110-1 to 110-n) are various client devices that access the system and can be implemented in various forms, such as smartphones, tablets, desktop computers, wearable devices, and IoT devices. These terminals are responsible for transmitting service requests through a user interface and receiving and displaying results processed by a server (120). Each terminal may also run a lightweight AI model locally, which can reduce network latency and enhance privacy protection. Direct connection or peer-to-peer (P2P) communication between terminals may be possible, as indicated by the dotted line, which is particularly useful in distributed learning or edge computing scenarios.

[0051] According to one embodiment, the server (120) is the central processing unit of the system, responsible for processing user requests, executing artificial intelligence models, and managing data. The server is equipped with a high-performance processor (CPU, GPU, TPU, NPU, etc.) for large-scale computation, large-capacity memory, and a stable operating system, and may be operated in a virtualized manner on a cloud infrastructure. The server performs training and inference functions for complex deep learning models and transmits the results to user terminals or stores them in a database. In addition, the stability and reliability of the system can be ensured through functions such as load balancing, error recovery, and security management.

[0052] According to one embodiment, a database (130) is a storage facility for storing and managing various data, and is depicted in a cylindrical shape in FIG. 1. Data stored in the database (130) is data acquired, processed, or used by at least one component of a plurality of user terminals (110-1 to 110-n) or a server (120), and may include software (e.g., programs), user profiles, training datasets, model weights, log records, etc. Structured relational data may be stored in a SQL-based database (MySQL, PostgreSQL, etc.), and unstructured large-scale data may be stored in a NoSQL database (MongoDB, Cassandra, etc.) or a distributed file system (Hadoop HDFS, etc.). The database (130) may include volatile and / or non-volatile memory, where volatile memory (RAM) is used for caching or temporary data processing requiring fast data access, and non-volatile memory (SSD, HDD, tape drive, etc.) is used for permanent data storage. Databases also ensure the security and integrity of data through data encryption, access control, and backup and recovery mechanisms.

[0053] This artificial intelligence-based system (100) can be utilized in various application fields and can provide services such as user behavior analysis, recommendation systems, natural language processing, image and video recognition, and predictive analysis. In particular, the distributed architecture and scalable design of the system enable stable performance to be maintained even as the number of users and data volume increase.

[0055] FIG. 2 is a diagram illustrating the learning of a neural network according to one embodiment.

[0056] As illustrated in FIG. 2, the learning device can train a neural network (123) to classify review responses received from a plurality of user terminals (110-1 to 110-n) by item. Additionally, the learning device can train a neural network (123) to extract user stay history from user movement path information. The neural network (123) is also called an artificial neural network and is a computational model designed inspired by the neural structure of the human brain, specialized in recognizing and learning complex patterns within data. According to one embodiment, the learning device may be a separate entity from the server (120), but it may also be implemented integrated into the same system, so it is not limited thereto.

[0057] According to one embodiment, the neural network (123) includes an input layer (121) into which training samples are input and an output layer (125) that outputs training outputs, and can be learned based on the difference between the training outputs and labels (i.e., actual correct data). Here, the labels are defined based on items corresponding to review responses (e.g., service quality, price satisfaction, cleanliness, etc.) and can be defined based on user dwell history (place visited, time spent, frequency of visit, etc.) corresponding to movement path information. The neural network (123) is connected as a group of multiple nodes and is defined by weights between the connected nodes and an activation function that activates the nodes. Various forms of the activation function may be used, such as sigmoid, hyperbolic tangent (tanh), and ReLU, which enables the network to learn non-linear patterns.

[0058] According to one embodiment, the learning device can train a neural network (123) using a Gradient Descent (GD) technique or a Stochastic Gradient Descent (SGD) technique. The GD technique is a method of updating weights all at once by calculating gradients based on the entire dataset, while the SGD technique is a method of increasing computational efficiency and the possibility of escaping a local optimum by updating weights more frequently using only a randomly selected portion of data (mini-batch). The learning device can use a loss function designed by the outputs and labels of the neural network. The loss function is an important element that quantifies the difference between the model's predicted value and the actual correct answer to suggest the direction of learning.

[0059] The learning device can calculate the training error using a predefined loss function. The loss function can be predefined with labels, outputs, and parameters as input variables, where the parameters can be set by weights within the neural network (123). For example, the loss function can be designed in the form of Mean Square Error (MSE), entropy, etc. MSE is calculated as the squared mean of the difference between the predicted value and the actual value and is mainly used for regression problems, while cross-entropy is a function that measures the difference between the predicted probability distribution and the actual distribution and is suitable for classification problems. In addition, loss functions that are less sensitive to outliers, such as Huber Loss, can be utilized, and various techniques or methods can be employed in the embodiments where the loss function is designed.

[0060] According to one embodiment, the learning device can identify weights that influence the training error using a backpropagation technique. Backpropagation is a process of calculating the degree to which each weight contributes to the final error while propagating the error calculated in the output layer toward the input layer, and is performed through differential calculations using the chain rule. Here, the weights are relationships between nodes within the neural network (123). The learning device may use an SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning device may update the weights of a loss function defined based on labels, outputs, and weights using an SGD technique. This process operates by calculating the gradient (the derivative of the loss function with respect to the weights) and adjusting the weights in the direction of this gradient to gradually decrease the value of the loss function.

[0061] According to one embodiment, the learning device extracts first objects from a review response, obtains first labels which are items corresponding to the first objects, applies the first objects to a first neural network to generate first training outputs corresponding to the first objects, and can train the first neural network based on the first training outputs and the first labels. At this time, the first objects may include key keywords, sentence structures, sentiment expressions, etc. extracted from the review text, and thereby learn the ability to identify the core content of the review and classify it into the corresponding items.

[0062] According to one embodiment, the learning device extracts second objects from movement path information, obtains second labels which are user dwell history corresponding to the second objects, applies the second objects to a second neural network to generate second training outputs corresponding to the second objects, and can train the second neural network based on the second training outputs and the second labels. The second objects may consist of data such as the user's GPS coordinate sequence, movement speed, stopping point, and movement pattern, thereby learning the ability to determine where and for how long the user stayed.

[0063] According to one embodiment, a learning device can generate first training feature vectors based on constituent features (e.g., sentence structure, word frequency, part-of-speech distribution), positional features (e.g., location of important words, location of key sentences), and pattern features (e.g., repetitive expressions, usage patterns of specific phrases) of a review response. These feature vectors are generated through a process of converting raw text data into a numerical form that can be processed by a neural network, and various methods such as TF-IDF, Word2Vec, and BERT may be employed to extract features.

[0064] According to one embodiment, the learning device can generate second training feature vectors based on constituent features of movement path information (e.g., shape of the path, estimation of the means of movement), length features (e.g., total travel distance, travel time, distance between each point), and pattern features (e.g., repeated visit patterns, movement patterns by day of the week / time of the day). Since location data has continuous values ​​over time, time-series data processing techniques or spatial data analysis methodologies may be applied, and various methods may be employed to extract features.

[0065] According to one embodiment, the learning device can obtain training outputs by applying first training feature vectors to the neural network (123). This process is also called feedforward and is a process in which input data passes through each layer of the network to generate a final prediction value. The learning device can train the review item extraction algorithm of the neural network (123) based on the training outputs and first labels. The learning device can train the review item extraction algorithm of the neural network (123) by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network (123) to minimize the training errors. This process is repeated over several epochs, and in each epoch, the entire training dataset is processed and weights are updated. The server (120) can automatically extract and classify items from new review responses using the first neural network that has been trained.

[0066] According to one embodiment, the learning device can obtain training outputs by applying second training feature vectors to the neural network (123). The learning device can train the user stay history acquisition algorithm of the neural network (123) based on the training outputs and second labels. The learning device can train the user stay history acquisition algorithm of the neural network (123) by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network (123) to minimize the training errors. Since movement path data has temporal continuity, a recurrent neural network (RNN) structure such as LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit) can be applied, which enables effective learning of the patterns of the sequence data. The server (120) can obtain user stay history from movement path information using the second neural network after training is complete, and the information thus obtained can be utilized in various application fields such as improving location-based services, building customized recommendation systems, and analyzing traffic patterns.

[0068] FIG. 3 is a diagram illustrating the configuration of an artificial intelligence model according to one embodiment.

[0069] An artificial intelligence model according to one embodiment may include an input layer, a hidden layer, and an output layer. This multi-layer structure is effective for learning complex patterns and solving non-linear problems, and is utilized as a core structure, particularly in deep learning models. The input layer is a layer related to input values ​​input into the artificial intelligence model and serves to receive data coming from the outside. As illustrated in FIG. 3, the input layer consists of three nodes, and each node may represent a specific feature or attribute of the data. The number of nodes in the input layer may vary depending on the dimension or characteristics of the data to be processed, and may be determined by the number of pixels in the case of image data, the dimension of the word vector in the case of text data, etc.

[0070] According to one embodiment, a feature map can be output by performing a MAC (multiply-accumulate) operation and an activation operation on the input values ​​in the hidden layer. In FIG. 3, the hidden layer consists of four nodes, and these nodes are connected to all nodes of the input layer to form a fully connected structure. The MAC operation may be an operation that multiplies the input values ​​by their corresponding weights and sums the multiplied values, and this acts as the basic unit of information processing in a neural network.

[0071] According to one embodiment, the activation operation may be an operation that inputs the result of a MAC operation into an activation function and outputs a result value. The activation function is a key element that enables a model to learn complex patterns by introducing non-linearity into the result of a linear operation. The activation function may be of various types. For example, the activation function may include a sigmoid function (which outputs a value between 0 and 1 and is useful for expressing probability), a tangent function (which outputs a value between -1 and 1 and is suitable for balanced data), a ReLU function (which outputs 0 for negative inputs and outputs as is for positive inputs, mitigating the vanishing gradient problem), a Leaky ReLU function (which applies a linear function with a small gradient to negative inputs), a MaxOut function (which selects the maximum value among several linear functions), and / or an ELU function (which is an exponential linear unit function that provides a smooth saturation curve for negative inputs), but there are no limitations on the types thereof. Recently, new activation functions such as Swish and Mish are also being studied and are contributing to the improvement of model performance.

[0072] According to one embodiment, the hidden layer may be composed of at least one layer, and the more layers there are, the more complex patterns can be learned, resulting in a deep learning structure. For example, when the hidden layer is composed of a first hidden layer and a second hidden layer, the first hidden layer performs MAC operations and activation operations based on the input value of the input layer to output a feature map, and the feature map, which is the result value of the first hidden layer, can become the input value of the second hidden layer. The feature map is an intermediate representation representing features extracted from input data; in image processing, it may represent visual features including edges, textures, and shapes, and in text processing, it may represent semantic patterns or syntactic structures. The second hidden layer can perform MAC operations and activation operations based on the feature map, which is the result value of the first hidden layer, and through this hierarchical processing, increasingly abstract and high-dimensional features can be extracted.

[0073] According to one embodiment, the output layer may be a layer associated with the result of an operation performed in the hidden layer, and in FIG. 3, it consists of three nodes. The number of nodes in the output layer varies depending on the type of problem to be solved; for binary classification problems, it may be one node, for multi-class classification problems, it may be equal to the number of classes, for regression problems, it may be determined by the dimension of the value to be predicted, and for sequence generation problems, it may be determined by the length of the output sequence. In the output layer, a special activation function such as a softmax function or a sigmoid function is generally used to generate a final predicted value or a probability distribution.

[0074] In one embodiment, the learning model learns syllable (character) patterns that are frequently combined and used in a given corpus to automatically learn the boundaries of compound words and named entities, integrates object information from a first UI source with object information rendered in a browser to create a learning object information file, and uses the learning object information file to generate training data for training a deep learning network. Natural language processing (NLP) technology is utilized in this process, and architectures such as Word Embedding, Recurrent Neural Network (RNN), and Transformer may be applied. Additionally, the learning model receives data from various domains of a support system, standardizes the data from the various domains into an integrated format based on at least one standardization method corresponding to each of the various domains, learns and infers data from a specific domain, determines information to be transmitted for standardization from the data of the specific domain, and performs post-processing on the data from the various domains.

[0075] According to one embodiment, the first UI source includes an XML file, and the training object information file includes an input JSON file for feature learning and an output JSON file that serves as label data during training. This structured data format allows the model to process it easily and is particularly useful for analyzing UI elements of web-based applications. The output JSON file includes a file containing HTML DOM Tree information implemented in compliance with web standards, and the various domains include at least one of a RAN (radio access network), a transport, or a core. A RAN refers to a wireless access network where a mobile device connects to a cellular network; a transport refers to a network layer responsible for data transmission; and a core refers to the central part of a network responsible for central processing and routing. Post-processing may include a correlation function, which plays an important role in analyzing relationships between data collected from various domains and discovering meaningful patterns. Through correlation analysis, the model can identify the causes of anomalies or performance degradation occurring in complex systems and improve prediction accuracy.

[0077] FIG. 4 is a block diagram showing the configuration of an automated system for ship check-in and boarding reporting according to one embodiment.

[0078] A system (400) according to one embodiment may include a processor (420) and memory (430), and some of the illustrated components may be omitted or substituted. Although the system diagram briefly depicts only the core components, the actual implementation may include various additional components such as an input / output controller, a system bus, a graphics processing unit (GPU), a network interface card (NIC), and a storage device controller. A system (400) according to one embodiment may be a server or a terminal; if implemented as a server, it may be operated in a virtualized environment as part of a cloud infrastructure or implemented as a physical hardware server, and if implemented as a terminal, it may be implemented in various forms such as a desktop computer, a laptop, a tablet, a smartphone, an embedded system, or an IoT device. According to one embodiment, the processor (420) is a component capable of performing operations or data processing regarding the control and / or communication of each component of the system (400), and may be composed of one or more processors. In modern processor architectures, multi-core designs are common, which improve parallel processing performance by integrating multiple independent processor cores within a single chip. For example, configurations such as dual-core, quad-core, and octa-core are possible, and each core has independent cache memory (L1, L2), and some cache (L3) can be shared among cores.

[0079] The memory (430) can store information related to the method described above or a program in which the method described above is implemented. The memory subsystem is designed with a hierarchical structure and is composed of several levels according to access speed and capacity. The memory (430) may be volatile memory or non-volatile memory. Volatile memory is memory in which stored data is lost when the power is turned off, and is mainly implemented in the form of DRAM (Dynamic RAM) or SRAM (Static RAM). DRAM stores each bit in the form of charge in a capacitor and requires periodic refreshing, whereas SRAM uses flip-flop circuits to maintain data, so it does not require refreshing but has higher cost and power consumption due to the use of more transistors. The memory (430) can store various file data, and the stored file data can be updated according to the operation of the processor (420). The memory management unit (MMU) is responsible for converting virtual memory addresses to physical memory addresses, managing page tables, and maintaining memory protection and cache consistency.

[0080] According to one embodiment, the processor (420) can execute a program and control the device (400). The processor can operate through basic pipeline stages of instruction fetch, decode, execute, and write-back. The code of the program executed by the processor (420) can be stored in memory (430). In addition to the application, the operating system, device driver, middleware, system services, etc. are loaded into memory and executed, and these provide basic system functions such as hardware resource management, process scheduling, interrupt handling, and file system management. The operations of the processor (420) can be performed by loading instructions stored in memory (430). In this process, the program counter (PC) points to the address of the next instruction to be executed, the instruction register (IR) stores the instruction currently being executed, and various general-purpose registers store operation data and intermediate results. The system (400) can be connected to an external device (e.g., a personal computer or a network) through an input / output device (not shown in the drawing) and exchange data. The input / output system can operate on an interrupt-based or polling-based basis and can transfer data directly between memory and the input / output device without the intervention of a processor through a Direct Memory Access (DMA) controller.

[0081] According to one embodiment, data exchange between memory (430) and processor (420) is performed via a system bus, which consists of an address bus, a data bus, and a control bus. The address bus specifies the memory location that the processor intends to access, the data bus transmits actual data, and the control bus transmits control information such as read / write signals. The system (400) may be operated under an operating system that supports advanced features such as multitasking, virtual memory, memory protection, and separation of privileges, and if a real-time operating system (RTOS) is used, it may satisfy deterministic response times and strict time constraints. For artificial intelligence applications, the memory (430) stores a neural network model, weight parameters, a training dataset, intermediate calculation results, etc., and the processor (420) may perform inference and learning by executing forward propagation and backpropagation algorithms. The system (400) can ensure high availability and fault tolerance by applying technologies such as clustering, load balancing, and failover mechanisms for scalability, and can implement technologies such as data encryption, access control, secure booting, and memory protection for security.

[0083] FIG. 5 is a flowchart illustrating the operation method of a ship boarding authentication system according to one embodiment.

[0084] The operations described through FIG. 5 can be implemented based on instructions that can be stored in memory (e.g., memory (420) of FIG. 4). The order of each operation of FIG. 5 may be changed, some operations may be omitted, and some operations may be performed simultaneously.

[0085] In operation 510, the system (400) can acquire situational awareness data in real time, including biometric information, safety equipment wearing status, and ship beacon signals, through the sensors and cameras of the user terminal.

[0086] According to one embodiment, the system (400) can drive a camera module mounted on a user terminal to capture an image of the user's face and simultaneously collect location and movement information from the terminal's accelerometer, gyroscope sensor, and GPS module. The system (400) can extract the user's biometric information by applying a face detection algorithm to the image data acquired through the camera. The system (400) can identify the presence of safety equipment, such as a life vest, by executing an object recognition algorithm within the same camera image. The system (400) can scan and receive signals transmitted from a beacon device installed on a vessel by activating the Bluetooth or Wi-Fi module of the user terminal. The system (400) can determine the relative location of the terminal by measuring the identifier, signal strength, and arrival time information of the received beacon signal. The system (400) can generate a situational awareness dataset that comprehensively represents the current situation by integrating this complex sensor data with a synchronized timestamp.

[0087] In operation 520, the system (400) can analyze situation awareness data to determine whether biometric information is acquired in real time, whether the safety equipment is located in a body part, and whether the user terminal is located within the radius of the vessel.

[0088] According to one embodiment, the system (400) may perform a liveness inspection algorithm to verify whether the acquired biometric information was obtained from a real-time subject rather than being extracted from a pre-captured still image or a played video. The system (400) may determine whether it is a three-dimensional real subject or a two-dimensional image by changing the screen brightness of the user terminal or sequentially emitting lights of different colors and analyzing the resulting pattern of change in the reflectance of the face image. To verify the state of wearing safety equipment, the system (400) may extract the location coordinates of the safety equipment identified within the image and compare them with the coordinates of the user's body parts, including the shoulders, chest, and waist, extracted through a human skeleton detection algorithm. The system (400) may distinguish whether the safety equipment is actually worn or is being held in the hand by calculating whether the safety equipment area shows a sufficient overlap ratio with the user's torso area. The system (400) may calculate the current location of the user terminal using a triangulation method by analyzing the reception strength of the beacon signal and the difference in signal arrival times from multiple beacons. The system (400) can determine whether the calculated location is included within the radius designated as the boarding area of ​​the vessel and verify the location information by cross-verifying the degree of agreement with the absolute coordinates obtained by GPS.

[0089] In operation 530, the system (400) can generate a real-time update token by combining the hash value of the situation awareness data and time information when the verification condition is met, and digitally sign it with a private key and transmit it to the ship manager terminal.

[0090] According to one embodiment, when the system (400) detects that all conditions confirmed in operation 520 are satisfied, it may generate a unique hash value by applying a one-way hash function (e.g., SHA-256) to the entire situation awareness data at the current time. The system (400) may construct a time-dependent dynamic seed by combining the generated hash value with timestamp information of the current time. The system (400) may generate a one-time random number code based on the dynamic seed and include it in the payload of the boarding authentication token. Even after the token is generated, the system (400) may continuously reacquire sensor data to calculate a new hash value and periodically compare it with the hash value at the time of initial generation. If a discrepancy between the two hash values ​​is detected, the system (400) may determine that the user has removed the life vest or deviated from the boarding radius and automatically invalidate the token. The system (400) may authenticate the issuer of the token and prevent tampering with the content by electronically signing the valid token data using the user's private key. The system (400) can directly transmit an electronically signed token to a ship manager's terminal via a short-range wireless communication protocol such as Bluetooth Low Energy (BLE) communication or Wi-Fi. The system (400) can complete the registration of the boarding list through peer-to-peer (P2P) token transmission even in a maritime environment where communication with a central server is impossible.

[0091] According to one embodiment, the system (400) can acquire context awareness data in real time, including the user's biometric information, the status of safety equipment wear, and the results of detecting ship beacon signals, through sensors and cameras of the user terminal. Here, context awareness data may refer to a dynamic data stream that is continuously updated, rather than static information at a single point in time. The biometric information may include biological feature data capable of identifying the user's identity, such as at least one element of a facial image, iris pattern, or fingerprint. The status of safety equipment wear may be binary state information indicating whether essential safety equipment, such as a life vest, is actually worn on the user's body, or a continuous value indicating the degree of wear. The results of detecting ship beacon signals may include the intensity and identification information of radio waves emitted from a wireless signal transmitting device installed on the ship, which the user terminal receives.

[0092] According to one embodiment, the system (400) can analyze acquired situation awareness data to determine whether the biometric information was acquired from a real-time subject rather than through abnormal input using a prior acquired image or video. A prior acquired image or video may refer to a photograph taken in the past, a printed document, a video playback screen, etc., and an abnormal input using such may represent a spoofing attack in which a third party, not the actual person, attempts authentication by stealing another person's biometric information. The system (400) can determine whether the acquired biometric information was collected in real-time from an actual 3D subject. The system (400) can distinguish between a 2D image and a 3D real object by emitting light through a terminal screen or flash and analyzing the reflection pattern resulting therefrom, taking into account the poor lighting conditions of a marine environment.

[0093] According to one embodiment, the system (400) can verify whether safety equipment is located on a designated part of the body. The designated part of the body may vary depending on the type of safety equipment, and in the case of a life vest, it may be a torso area including the shoulders and chest. By verifying the correct wearing status rather than simply the presence of the safety equipment, the system (400) can block attempts by the user to bypass the safety equipment by holding it in their hand or placing it beside them while filming. The system (400) can quantitatively evaluate the spatial relationship between the safety equipment and the body part by combining a human skeleton detection algorithm and an object recognition algorithm. By verifying the wearing status, the system (400) can prevent safety violations such as not wearing a life vest after boarding and secure evidence to clarify legal liability in the event of a maritime accident.

[0094] According to one embodiment, the system (400) can determine whether a user terminal is located within an authorized radius of the vessel. The authorized radius of the vessel may refer to a limited spatial range around the vessel where actual boarding is physically possible, for example, within a radius of 3 meters from the boarding port of the vessel. To overcome the limitation that precise location verification is difficult due to errors of several meters occurring with GPS alone, the system (400) may use a short-range location measurement technology that measures the signal strength and arrival time of beacons installed on the vessel. The system (400) can calculate the relative location of the terminal with centimeter precision by analyzing signals received from multiple beacons using a triangulation method. Through location verification, the system (400) can detect and block hacking attempts that use authentication information generated in advance at a remote location on-site or manipulate the beacon range using signal amplification equipment.

[0095] According to one embodiment, the system (400) can generate a token in which the pattern is updated in real time by combining the hash value of the situation awareness data and time information based on the fact that all verification conditions are satisfied. The hash value may refer to the output value of a one-way encryption function that converts data of arbitrary length into a unique string of fixed length. By applying a hash function to the entire situation awareness data as input, the system (400) can utilize the characteristic that a completely different hash value is generated if any part of the data is changed. The time information refers to a timestamp at the time of token generation, and by combining this with the hash value, it is possible to generate different tokens depending on the time, even in the same situation. A token in which the pattern is updated in real time may serve as a one-time authentication means that makes it fundamentally impossible to capture and reuse tokens generated in the past. Even after token generation, the system (400) continuously remeasures the situation awareness data to calculate a new hash value and compares it with the hash value at the time of the initial token generation, thereby revoking the validity of the token if the user removes the life vest or leaves the boarding area.

[0096] According to one embodiment, the system (400) can control the registration of a boarding list by electronically signing and encrypting a generated token with the user's private key, and then transmitting it to a ship manager terminal via near-field wireless communication. The private key may refer to a secret key possessed only by the user among a pair of keys used in public-key cryptography. The electronic signature may be a technology that guarantees the integrity of data and proves that the data was generated by the private key owner by encrypting the data using the private key. Through the electronic signature, the system (400) can guarantee the identity of the token issuer and prevent the user from denying consent to compliance with safety regulations after boarding. Near-field wireless communication may include communication methods that exchange data directly between two devices without passing through a central server, such as Bluetooth, Wi-Fi, and NFC. By utilizing near-field wireless communication, the system (400) can complete the boarding list registration even in communication blind spots at sea or in the event of a central server failure.

[0098] According to one embodiment, the system (400) can calculate a first hash value by unidirectionally encrypting the situation awareness data at a point in time when all verification conditions are satisfied. Unidirectional encryption may refer to an encryption method in which it is possible to generate an encrypted result value from the original data, but it is computationally impossible to reverse-engineer the original data from the encrypted result value. The first hash value may be a fixed-length string obtained by processing the entire situation awareness data, including biometric information, safety equipment wearing status, location information, etc., as a single input. The system (400) may use standard hash algorithms such as SHA-256 or SHA-512 to control the generation of a completely different hash value even if only one bit of the data is changed. The system (400) may store the first hash value as a snapshot of the reference state so that it can be used as a comparison reference point to detect subsequent situation changes.

[0099] According to one embodiment, the system (400) can generate a dynamic seed that combines time flow and situation information by combining a calculated first hash value and timestamp information that is updated at a preset period. The timestamp information may be time information that includes year, month, day, hour, minute, second, and millisecond units based on Coordinated Universal Time (UTC), and the preset period may be set, for example, to 1 second, 5 seconds, 10 seconds, etc. A dynamic seed refers to a value used as an initial value for a random number generation algorithm, and may have the characteristic that the same random number sequence is always generated from the same seed, but a completely different random number sequence is generated when the seed is changed. The system (400) can combine the first hash value and the timestamp through methods such as string concatenation, XOR operation, or the application of an additional hash function. Since the timestamp changes automatically as time elapses, the system (400) can generate different dynamic seeds depending on time, even for the same situation awareness data.

[0100] According to one embodiment, the system (400) can generate a one-time random number code based on a generated dynamic seed. The one-time random number code may refer to an authentication code that is valid only at a specific point in time and cannot be reused, and may operate on a principle similar to that of a One-Time Password (OTP). The system (400) can generate an unpredictable random number sequence by inputting the dynamic seed into a pseudo-random number generator. The system (400) can construct a final random number code by cutting the generated random number into specific lengths or converting it to a specific base. The system (400) can represent this random number code in the form of a QR code, barcode, or string of numbers to display it on a user terminal screen or include it in token data. The system (400) can block attacks that capture and use codes generated in the past through the generation of a dynamic seed-based random number code.

[0101] According to one embodiment, the system (400) may periodically compare a second hash value calculated from situation awareness data reacquired in real time after token generation with a first hash value. Real-time reacquiring may mean continuously operating sensors and cameras to collect new situation awareness data even after the initial token generation. The second hash value may be a value calculated by applying the same hash algorithm as used when generating the first hash value to the reacquired situation awareness data. The system (400) may calculate a new second hash value at set time intervals, for example, every 3 seconds or every 5 seconds, and perform a string comparison operation with the stored first hash value. If the two hash values ​​match completely, the system (400) may determine that there is no change in the situation and maintain the validity of the token.

[0102] According to one embodiment, the system (400) may determine that a case where two hash values ​​are inconsistent constitutes a security breach state resulting from the user removing safety equipment or deviating from the vessel's radius. A hash value inconsistency may mean that one or more elements constituting the situational awareness data have changed, which may include cases where the user has removed a life vest, left the boarding area, or stopped obtaining biometric information. Since the system (400) can immediately detect even minute changes in the situation because, due to the characteristics of the hash function, even a very small change in input data causes a large change in the output value, a security breach state refers to a state where the basic prerequisites for boarding authentication are not met, and in this state, boarding should not be permitted. The system (400) may log the time of the hash value inconsistency, snapshots of the situational awareness data before and after the change, and user identification information to be used as post-audit data.

[0103] According to one embodiment, the system (400) may control the blocking of boarding processing by stopping the generation of a random number code or discarding the validity key value of a previously generated token based on the determination of a security breach state. Stopping the generation of a random number code may mean stopping the dynamic seed update process and preventing new random number codes from being generated any further. The validity key value may refer to a cryptographic key or verification code used by the ship manager terminal to verify the authenticity of a received token. By discarding the validity key value, the system (400) may cause the ship manager terminal to determine that the token is invalid even if the user submits a token that has already been generated. The system (400) may also visually clarify that the token is no longer valid by deleting the QR code displayed on the user terminal screen or replacing it with an invalidation mark. Simultaneously with blocking boarding processing, the system (400) may display a notification message to the user (e.g., "Boarding authentication has been canceled because the safety equipment wearing status has changed") to inform the user that re-authentication is required. The system (400) can have the effect of preventing marine safety accidents by systematically blocking acts that do not comply with safety regulations after boarding through this real-time validity verification mechanism.

[0105] FIG. 6 illustrates a method for determining the wearing status of safety equipment based on skeletal detection according to one embodiment.

[0106] The operations described through FIG. 6 can be implemented based on instructions that can be stored in memory (e.g., memory (420) of FIG. 4). The order of each operation of FIG. 6 may be changed, some operations may be omitted, and some operations may be performed simultaneously.

[0107] In operation 610, the system (400) can extract multiple skeletal coordinates corresponding to the left and right shoulders, elbows, wrists, and pelvis from a user image obtained through a camera.

[0108] According to one embodiment, the system (400) can apply a deep learning-based human pose estimation algorithm to RGB image data received from a camera of a user terminal. The system (400) can detect feature points corresponding to major joint points of the human body within the image through a pre-trained neural network model. The system (400) can identify and extract 2D pixel coordinates of the left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, and pelvic center point among the detected feature points. The system (400) can evaluate the accuracy of the detection results by calculating a reliability score for each skeletal coordinate. For coordinates where the reliability score is below a preset threshold, the system (400) can correct the coordinates by utilizing information from adjacent frames or an interpolation algorithm. The system (400) can protect privacy and improve response speed without server transmission by processing the skeletal coordinate extraction process in real time within the user terminal using an on-device AI processor.

[0109] In operation 620, the system (400) can define an area corresponding to the user's torso by connecting the extracted left and right shoulder coordinates and pelvic coordinates.

[0110] According to one embodiment, the system (400) may set a line segment connecting the extracted left shoulder coordinates and the right shoulder coordinates as the upper boundary and a line segment connecting the left and right endpoints of the pelvic coordinates as the lower boundary. The system (400) may define a torso area in the shape of a rectangle or a trapezoid by connecting the upper boundary and the lower boundary. The system (400) may create a more accurate torso area by analyzing the ratio of shoulder width to pelvic width considering the natural shape of the human body and adjusting the side boundary accordingly. The system (400) may store all pixel coordinates within the defined torso area as a set and utilize them for subsequent comparison operations. Considering that the shape of the torso area may be distorted depending on changes in the user's posture or camera angle, the system (400) may calculate a normalized torso area by applying perspective transformation or rotation transformation.

[0111] In operation 630, the system (400) can calculate the ratio of the overlapping area between the boundary box area and the torso area of ​​the safety equipment and determine the wearing state if it is greater than or equal to a threshold.

[0112] According to one embodiment, the system (400) can detect a life vest within an image through an object recognition algorithm and obtain the coordinates of a minimum bounding rectangle surrounding the detected life vest. When applying an instance segmentation technique, the system (400) can extract a mask area representing the accurate outline of the life vest. The system (400) can calculate an intersection area between the bounding box or mask area of ​​the safety equipment and the torso area defined in motion 620. The system (400) can calculate an Intersection over Union (IoU) value by dividing the area of ​​the intersection area by the area of ​​the union of the two areas. If the calculated IoU value is greater than or equal to a preset threshold (e.g., 0.6 or 60%), the system (400) can determine that the safety equipment is sufficiently overlapping the user's torso and determine that it is in a wearing state. Even if the IoU value exceeds the threshold, if the position of the safety equipment is significantly off from the center of the torso, the system (400) can perform additional verification to confirm whether it is in a correct wearing state.

[0113] In operation 640, the system (400) can determine that the safety equipment area is located near the wrist or elbow and the area ratio is below a threshold, and that the safety equipment is not worn if it is separated from the skeletal coordinates.

[0114] According to one embodiment, the system (400) may first determine that the safety equipment is not worn on the torso if the IoU value calculated in operation 630 is less than a threshold. The system (400) may calculate the center point coordinates of the safety equipment bounding box and calculate the Euclidean distance between this center point and each skeletal coordinate extracted in operation 610. The system (400) may determine that the user is in a holding state, holding the safety equipment in the hand or arm, if the center point of the safety equipment is located within a set proximity distance (e.g., 50 pixels) from the wrist coordinate or elbow coordinate. The system (400) may distinguish whether the safety equipment is held in the left or right hand through coordinate comparison. The system (400) may determine that the safety equipment is in an unworn state, separated from the user's body, if the center point of the safety equipment is located at a distance greater than a set separation distance from all skeletal coordinates. The system (400) can enforce compliance with safety regulations by displaying a warning message on the user terminal screen when it is determined that the user is carrying or not wearing the device, and by blocking the generation of the boarding authentication token.

[0116] According to one embodiment, the system (400) can extract multiple skeletal coordinates corresponding to the left and right shoulders, elbows, wrists, and pelvis by analyzing joint feature points of the human body within a user image acquired through a camera. Joint feature points may refer to two-dimensional pixel coordinates where major joint parts of the human body are located within the image, and these can be identified through Human Pose Estimation technology. The system (400) can detect 17 or 25 major joint points from an input image using a Convolutional Neural Network-based pose estimation model, and among these, can selectively extract the coordinates of the left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, and pelvic center point required for boarding authentication. Skeletal coordinates may include pixel location information in the form of (x, y) along with a confidence score of the corresponding coordinates, and if the confidence score is low, the system (400) determines that the detection of the corresponding joint point is uncertain and may request a re-shoot or perform correction using adjacent frame information. The system (400) can quantitatively analyze the spatial relationship between safety equipment and body parts by numerically modeling the user's body structure through skeletal coordinate extraction.

[0117] According to one embodiment, the system (400) can define an area corresponding to the user's torso by connecting the extracted left and right shoulder coordinates and the pelvic coordinates. The torso area may represent a core part of the body where safety equipment, such as a life vest, must be properly worn. The system (400) can set a line segment connecting the left shoulder coordinate and the right shoulder coordinate as the upper boundary of the torso and a line segment connecting the left and right endpoints of the pelvic coordinates as the lower boundary. The system (400) can create a polygonal area in the shape of a rectangle or a trapezoid by connecting the left and right endpoints of the upper boundary and the lower boundary, respectively. The system (400) can define all pixels inside the created polygon as the torso area and store a set of pixel coordinates included in this area. To reflect the natural shape of the human body, the system (400) can create a torso area that takes into account individual differences, such as an inverted triangle body type where the shoulder width is wider than the pelvic width, or a body type where the pelvic width is wider. The system (400) can secure a comparison target that can objectively determine whether the safety equipment is worn by clearly establishing a reference area where the safety equipment must be located through the definition of the torso area.

[0118] According to one embodiment, the system (400) can obtain a bounding box or mask area of ​​safety equipment identified through object recognition. A bounding box refers to a minimum-sized rectangle surrounding a detected object and can be represented by top-left corner coordinates (x1, y1) and bottom-right corner coordinates (x2, y2). A mask area represents the precise outline of an object in pixels and may be in the form of a binary mask where pixels corresponding to the object are 1 and pixels corresponding to the background are 0. The system (400) can identify life vests within an image and generate bounding boxes using object detection algorithms such as YOLO and Faster R-CNN, and can obtain a more precise mask area when using instance segmentation algorithms such as Mask R-CNN. The system (400) can maintain high detection accuracy even under various conditions of the marine environment by pre-training a safety equipment detection model with life vest images of various colors, shapes, and angles. The system (400) can quantify the spatial range occupied by the safety equipment within the image by acquiring a bounding box or mask area.

[0119] According to one embodiment, the system (400) can calculate the ratio of the area where the area of ​​the safety equipment overlaps with the area corresponding to the torso. The area ratio is generally expressed as IoU and may be a value obtained by dividing the intersection area of ​​the two areas by the union area. When using a bounding box, the system (400) can calculate the intersection area of ​​the two rectangles through coordinate comparison, and when using a mask, it can calculate the number of pixels of the intersection and union by performing AND and OR operations on a pixel basis. The system (400) can utilize the fact that the calculated IoU value is expressed as a real number between 0 and 1, where 0 means that the two areas do not overlap at all, and 1 means that the two areas completely match.

[0120] According to one embodiment, the system (400) may determine a wearing state when the area ratio is greater than or equal to a preset threshold. The threshold is a reference value for determining whether the safety equipment is sufficiently overlapping the torso, and may be set, for example, to 0.5 (50%), 0.6 (60%), 0.7 (70%), etc., and may vary depending on the setting. If the IoU value is greater than or equal to the threshold, the system (400) may determine that the user is wearing the safety equipment on their body because a significant portion of the safety equipment is located within the torso area. A wearing state may refer to a state in which the safety equipment is draped over the shoulders and covers the chest area, thereby enabling it to perform actual protective functions in the event of a maritime accident. The system (400) may allow the boarding certification procedure to continue only when it is determined to be in a wearing state. The system (400) may control the level of strictness of the wearing state determination by adjusting the threshold, and a high threshold may require more accurate wearing, while a low threshold may have the effect of relaxing the determination criteria.

[0121] According to one embodiment, the system (400) may determine a state of possession when the area ratio is below a threshold and the area of ​​the safety equipment is located near the wrist or elbow coordinates. A state of possession may refer to a state where the user holds the safety equipment in their hand or places it over their arm, so that it is visible only to the camera without being worn on the body. The system (400) may calculate the center point coordinates of the safety equipment boundary box and calculate the Euclidean distance between this center point and the wrist coordinates and the Euclidean distance between the center point and the elbow coordinates, respectively. The system (400) may determine that the safety equipment is located near the hand or arm if the calculated distance is within a preset proximity distance threshold (e.g., 30 pixels, 50 pixels, 100 pixels), and the proximity distance threshold may vary depending on the setting. If the system (400) determines that the safety equipment is in a state of possession, it may display a warning message (e.g., "Please wear the safety equipment on your body") on the user terminal screen and refuse boarding authentication. The system (400) can block attempts by the user to formally satisfy safety regulations by detecting the state of possession and can enforce actual safety.

[0122] According to one embodiment, the system (400) may determine a non-wearing state when the area of ​​the safety equipment is separated from the skeletal coordinates. A non-wearing state may mean a state in which the safety equipment is physically separated from the user's body, existing within the camera's field of view but not in direct contact with the user. The system (400) may determine a non-wearing state if the center point of the safety equipment area is separated from all extracted skeletal coordinates (left and right shoulders, elbows, wrists, pelvis) by a set separation distance threshold (e.g., 150 pixels, 200 pixels), and the separation distance threshold may vary depending on the setting. The system (400) may classify a situation in which the safety equipment is placed on the floor, held by a person next to it, or hung on a wall as a non-wearing state. When a non-wearing state is determined, the system (400) may display the strongest level of warning and immediately stop the boarding procedure. The system (400) can finely distinguish the relationship between the safety equipment and the user's body through a three-stage classification of 'wearing, carrying, and not wearing,' and can ensure compliance with maritime safety regulations by recognizing only the 'wearing state' as a condition for boarding.

[0124] According to one embodiment, the system (400) can receive signals transmitted from beacons distributed at multiple points within the boarding area of ​​a ship. A beacon may refer to a small transmitting device that periodically emits a signal using wireless communication technology such as Bluetooth Low Energy (BLE), Wi-Fi, or Ultra-Wideband (UWB). Distributed at multiple points means installing at least three beacons at different locations around the boarding area of ​​the ship, and may be a triangular structure in which beacons are placed on the left, right, and top of the boarding area, respectively. The system (400) can scan and receive signals transmitted by each beacon by activating the Bluetooth or Wi-Fi receiving module of the user terminal. The system (400) can distinguish which beacon the signal is from through the beacon identifier (Beacon ID) included in the received signal and query the installation location information of each beacon in a database. The system (400) can establish a basis for precise location measurement that is impossible with a single beacon alone by receiving multiple beacon signals simultaneously.

[0125] According to one embodiment, the system (400) can measure the received signal strength and signal arrival time of each beacon signal in real time. The Received Signal Strength Indicator (RSSI) is a value representing the power strength of a received wireless signal, generally expressed in units of dBm (decibel-milliwatts), and can have a negative value in the range of -30 dBm (strong) to -100 dBm (weak). The system (400) can utilize the characteristic that the RSSI value is measured to be higher (closer to 0) as the distance between the beacon and the terminal is closer, and the RSSI value is measured to be lower (larger absolute negative value) as the distance is farther. The time of flight (ToF) refers to the elapsed time from the time the beacon transmits the signal until the time the user terminal receives the signal, and can be converted into distance by considering the speed of radio waves (approximately 300,000 kilometers per second). The system (400) can calculate ToF by comparing the timestamp included in the beacon signal with the reception time of the terminal, and for this purpose, time synchronization between the beacon and the terminal can be performed. The system (400) can improve measurement accuracy by measuring RSSI and ToF together.

[0126] According to one embodiment, the system (400) can calculate the relative coordinates of a user terminal using a triangulation method. Triangulation may refer to a geometric technique for calculating the location of an unknown point based on the distance from three or more reference points. Relative coordinates are coordinates representing a relative position with a specific reference point as the origin; for example, if the center of a ship's boarding area is set to (0, 0) and the terminal is located 2 meters east and 1 meter north of it, it can be expressed as (2, 1). The system (400) can determine the location of the terminal by converting the RSSI or ToF measured from each beacon into a distance value and calculating the intersection point of the circles when drawing circles centered on each beacon with the measured distance as the radius. When using three beacons, the system (400) can determine the location of the terminal in a two-dimensional plane. Through triangulation, the system (400) can obtain precise location information that improves the error of GPS from several meters to the level of centimeters or tens of centimeters.

[0127] According to one embodiment, the system (400) can confirm that the calculated relative coordinates are located within a safety zone within a preset threshold distance. The safety zone refers to a limited area around a vessel where actual boarding is physically possible, and may be formed around the vessel's boarding port. The threshold distance may vary depending on the setting. The system (400) can calculate the Euclidean distance between the calculated relative coordinates and the center point of the safety zone and compare whether this distance is within the threshold distance. The system (400) can determine that the terminal has passed the first location verification only if it is located within the safety zone, and if it is located outside the safety zone, it can determine that it is a non-boarding location and refuse authentication.

[0128] According to one embodiment, the system (400) may determine that the user terminal is located within the authorized radius of the vessel only if the error between the absolute coordinates obtained through the GPS module and the calculated relative coordinates is within an allowable range. The absolute coordinates may refer to GPS latitude and longitude values ​​representing a unique location on the surface of the Earth. The system (400) may convert the relative coordinates calculated based on the beacon into absolute coordinates based on the vessel's GPS location and compare them with the absolute coordinates directly measured by the terminal's GPS module. The error refers to the difference in distance between the two coordinates; for example, if the beacon-based coordinates are 35.1234 degrees North latitude and 123.5678 degrees East longitude and the GPS measured coordinates are 35.1235 degrees North latitude and 123.5679 degrees East longitude, the error may be calculated to be approximately 10 meters. The allowable range is the maximum error distance allowed between the two measurement methods, which may be set to, for example, 5 meters, 10 meters, 15 meters, etc., and may vary depending on the setting. The system (400) can verify the reliability of location information by determining that two independent location measurement results match each other when the error is within an allowable range. The system (400) can compensate for the limitations of a single measurement method and detect attempts to manipulate location information through cross-verification of beacons and GPS.

[0129] According to one embodiment, the system (400) may control the boarding manifest registration procedure to be stopped if the error exceeds an allowable range. An error exceeding the limit means that the beacon-based measurement location and the GPS measurement location differ significantly beyond a reasonably explainable level, which may indicate an abnormal situation such as the following: The system (400) may suspect that there is a hacking attempt in which a user receives a beacon signal from a remote location using an amplification device, software-spoofs GPS coordinates, or remotely manipulates another person's terminal. When an error exceeding the limit is detected, the system (400) may display a message indicating a security verification failure and record the event in a security log to notify the administrator.

[0130] According to one embodiment, the system (400) may control the boarding manifest registration procedure to be stopped if the reception pattern of the beacon signal changes abnormally rapidly. The reception pattern may refer to comprehensive signal characteristics including the trend of change in RSSI values ​​over time, changes in the number of received beacons, and the frequency of signal reception. Abnormal rapid change refers to a rapid change in pattern that cannot be explained by normal user movement, and may include, for example, cases where the RSSI value drops from -50dBm to -90dBm within 1 second, or cases where two out of three received beacons are simultaneously not detected. The system (400) may set a maximum value for the expected RSSI change rate considering a normal walking speed (e.g., about 4 to 5 km / h) and determine it as abnormal if the measured change rate exceeds this value. Upon detection of an abnormal pattern, the system (400) may immediately stop the authentication procedure, send a notification to the administrator, and temporarily block further attempts by the user in question. The magnitude of the normal walking speed is merely an example and may vary depending on the settings.

[0132] According to one embodiment, when acquiring biometric information, the system (400) may emit a lighting signal in which different brightness or different colors change continuously in a set order when the ambient illuminance is below a preset threshold. Ambient illuminance is a value representing the brightness level of the environment, measured in units of lux, and can be acquired in real time through an illuminance sensor of a user terminal. The preset threshold is a threshold value for determining whether sufficient illumination is secured for biometric recognition, and may be set, for example, to 50 lux, 100 lux, etc., and may vary depending on the setting. The system (400) may detect a failure to meet the illuminance threshold in environments lacking natural light, such as dawn, night, or shaded areas of a ship. The lighting signal may refer to a structured light pattern generated using the terminal's screen or flash as a light source. Different brightness may be a pattern that increases stepwise, for example, to 20% brightness, 50% brightness, and 100% brightness, and different colors may be an RGB pattern that sequentially emits red, green, and blue. The system (400) can complete a series of lighting sequences for a total of 1 to 3 seconds by emitting each lighting signal at intervals of 0.1 to 0.5 seconds, and the time intervals may vary depending on the settings. The system (400) can compensate for poor lighting conditions by emitting lighting signals.

[0133] According to one embodiment, the system (400) can analyze the reflectance distribution according to changes in the light source by generating a difference image between a plurality of face image frames acquired before and after the emission of a lighting signal. The plurality of face image frames are a series of still images continuously captured by a terminal camera, for example, a total of 90 frames can be acquired by capturing for 3 seconds at a speed of 30 frames per second. The difference image is an image generated by calculating the difference in pixel brightness between two temporally consecutive frames, and areas with changes can be represented by high values, while areas without changes can be represented by low values. The system (400) can select a frame immediately before the emission of the lighting signal and a frame immediately after the emission and perform an operation to subtract the brightness values ​​of each corresponding pixel. The reflectance distribution is information indicating how the degree to which incident light is reflected from a surface is spatially distributed; while a 3D object has a reflectance that varies depending on the slope of the surface, a 2D plane may have the characteristic of exhibiting a uniform reflectance. The system (400) can infer the three-dimensional structure of the face by identifying areas with large brightness changes (e.g., protruding areas such as the nose and forehead) and small areas (e.g., sunken areas around the eyes) in the difference image.

[0134] According to one embodiment, the system (400) can extract face landmarks corresponding to major feature points of the face from a plurality of face image frames acquired while a lighting signal is emitted. Face landmarks are anatomical feature points of the face, including the ends of the eyes, eyebrows, the tip of the nose, the ends of the mouth, and the jawline, and the number of feature points may vary depending on the model used. The system (400) can extract landmark coordinates from each frame using a deep learning-based face landmark detection model. The system (400) can determine a stable landmark location by averaging the landmarks extracted from multiple frames or taking a median value.

[0135] According to one embodiment, the system (400) can generate an estimated 3D shape model based on spatial arrangement information of extracted landmarks. Spatial arrangement information may refer to relative positional relationships and distance information between each landmark point. The estimated 3D shape model may be a mesh or point cloud model representing the 3D geometric shape of a face inversely estimated from landmarks extracted from a 2D image. The system (400) may use a method of taking an average face shape from a 3D face model database and transforming it to fit the extracted landmark positions. The system (400) can estimate 3D features such as nose height, degree of chin protrusion, and forehead slope by analyzing the distance ratio between landmarks. The system (400) can obtain surface curvature information by calculating a normal vector (= direction perpendicular to the surface) for each surface point of the generated 3D model.

[0136] According to one embodiment, the system (400) can calculate a pattern in which the brightness of each pixel within a plurality of face image frames changes according to a change in a lighting signal. The brightness change pattern refers to a sequence of brightness values ​​recorded over time at the same pixel location, for example, if a specific pixel has a brightness value of 50 in frame 1, 80 in frame 2, and 120 in frame 3, it can be expressed as a pattern of [50, 80, 120]. The system (400) can extract this temporal brightness sequence for all pixels of the image to quantify how each pixel responds to changes in lighting. The system (400) can distinguish pixels that increase together as the brightness or color of the lighting signal increases, pixels that remain constant, or pixels that change non-linearly.

[0137] According to one embodiment, the system (400) can compare the calculated brightness change pattern with a simulated reference brightness change pattern based on the surface curvature of the expected 3D shape model and the light source direction of the lighting signal. Surface curvature is information indicating how much the surface is curved at each point of the 3D model and can be expressed as the direction of a normal vector. The light source direction may refer to a direction vector from the location of a terminal screen or flash emitting a lighting signal to the surface of the face. The system (400) can calculate the brightness expected to be observed at each surface point when a lighting signal is shone on the expected 3D shape model by applying a lighting model used in 3D computer graphics (e.g., Lambertian reflection model, Phong reflection model). The reference brightness change pattern may refer to a theoretical brightness sequence generated through such simulation. The system (400) can calculate similarity by comparing the actually measured brightness change pattern with the simulated reference pattern pixel by pixel. Similarity may be quantified as a numerical value such as the correlation coefficient between the two patterns, mean squared error, or cosine similarity.

[0138] According to one embodiment, the system (400) can determine that the object is an actual 3D subject if the match rate between two patterns is greater than or equal to a preset value. The match rate is a value expressing the similarity of the brightness change pattern as a percentage, which can be measured, for example, as 70%, 80%, 90%, etc., and the preset value can be set, for example, as 75%, 80%, etc., and may vary depending on the setting. When a high match rate is measured, the system (400) can determine that the actual photographed object has a three-dimensional structure as predicted by the 3D shape model and reacts like a 3D object to changes in lighting. The actual 3D subject refers to the face of a living person and may have the characteristic of forming shadows and highlights depending on the direction of the light source due to the three-dimensional structure of the body parts. By determining the object as a 3D subject, the system (400) can confirm that the corresponding biometric information is reliable data obtained from a real person in real time.

[0139] According to one embodiment, the system (400) can determine that the calculated brightness change pattern is a two-dimensional image if it tends to match the reflection characteristics of a planar model in which the surface normal vector has a constant direction. The planar model represents a flat surface without depth change, and may correspond to a printed photograph or a face image displayed on a smartphone screen. The fact that the surface normal vector has a constant direction may mean that all points on the surface are facing the same direction. The system (400) can analyze whether the brightness of all pixels increases or decreases at nearly the same rate with respect to changes in lighting. The system (400) can determine that it is a two-dimensional image based on the confirmation that the spatial distribution of brightness changes is uniform and the position of shadows or highlights does not shift with changes in the direction of lighting.

[0140] According to one embodiment, the system (400) may determine a two-dimensional image when a total reflection component exceeding a preset threshold is detected in a local area. A total reflection component refers to a bright spot or area that appears when a light source is reflected from a glossy surface, and may also be called specular reflection. A local area is a small sub-area within an image, for example, a block of size 10x10 pixels. The system (400) may measure the intensity of total reflection by calculating how much the maximum brightness value of each local area is greater than the surrounding average brightness. The preset threshold is a reference value for determining total reflection and may be set, for example, to 2 times or 3 times the surrounding brightness, and may vary depending on the setting. The system (400) may utilize the characteristic that strong total reflection occurring on the glossy surface of a printed photograph or the glass surface of a smartphone screen does not appear on actual human skin. The system (400) may allow weak total reflection in some areas, such as the forehead or the tip of the nose, as normal, but may determine a two-dimensional image if excessively strong total reflection appears across the entire face.

[0141] According to one embodiment, the system (400) can determine a two-dimensional image when pixel grid noise is detected during a display re-capture within the image. Pixel grid noise may refer to a pattern in which the pixel structure of the screen appears in the captured image when the display screen is captured by a camera. The system (400) can detect the presence of a periodic grid pattern by applying a high-frequency filter or a Fourier transform to the image. The system (400) can utilize the difference that while real human skin has a smooth texture, a display re-capture image shows a regular pixel structure as an interference pattern. The system (400) can determine whether to re-capture by checking whether the detected grid frequency falls within a range corresponding to the pixel density of a typical smartphone or tablet display.

[0142] According to one embodiment, the system (400) may determine that the input is normal if it is determined to be a 3D object, and determine that the input is abnormal if it is determined to be a 2D image, judging it as a spoofing attempt. Normal input refers to biometric information captured in real-time in front of a camera by the person actually intending to board, and in this case, the system (400) may continue the boarding authentication procedure. A spoofing attempt may refer to an attack attempting to bypass biometric authentication using another person's photo, video, 3D printed mask, display screen, etc. If the system (400) determines that the input is abnormal, it may display a message (e.g., "Biometric authentication failed: Real-time identity verification required") and refuse boarding. The system (400) can block proxy boarding or illegal boarding by verifying liveness with high accuracy by combining active lighting and 3D shape analysis, even under poor lighting conditions in the marine environment.

[0144] FIG. 7 illustrates a method for verifying fishery resource regulations and a capture permit according to one embodiment.

[0145] The operations described through FIG. 7 can be implemented based on instructions that can be stored in memory (e.g., memory (420) of FIG. 4). The order of each operation of FIG. 7 may be changed, some operations may be omitted, and some operations may be performed simultaneously.

[0146] In operation 710, the system (400) can obtain information on the main target fish species from the vessel's departure report information and query the fisheries resource regulation database to extract information on the closed season, minimum catch size, and zone.

[0147] According to one embodiment, the system (400) may receive departure report data of the vessel from a ship management system or a server of the marine and fisheries authority. The system (400) may parse and extract the name, scientific name, or species code of the main target fish species included in the departure report data. The system (400) may use the extracted fish species information as a key value to send a query to a pre-established fisheries resource regulation database. The system (400) may search for and receive regulatory data from the database that includes the start and end dates of the closed season designated by law for the fish species, the minimum and maximum body lengths for which capture is prohibited, the geographical boundary coordinates of the prohibited zone, and the daily allowable catch amount. If multiple fish species are the target, the system (400) may individually query and integrate regulatory information for each species to manage them. The system (400) may periodically update the fisheries resource regulation database to reflect the latest amendments to the laws.

[0148] In operation 720, the system (400) may obtain user consent by displaying regulatory data as a popup when the time of boarding corresponds to a closed season or when the navigation route includes a prohibited fishing zone.

[0149] According to one embodiment, the system (400) can determine whether the time of boarding falls within the closed season period by comparing the current date with the start and end dates of the closed season obtained from operation 710. The system (400) can represent the vessel's planned navigation route information as a sequence of GPS coordinates and check through a spatial analysis algorithm whether this route intersects with the boundary of a prohibited fishing zone. If the system (400) determines that the time of boarding falls within the closed season or that part of the navigation route passes through a prohibited fishing zone, it can display a warning pop-up on the screen of the user terminal. The system (400) can clearly state the legal basis of the relevant regulation, penalties for violation, and details to be complied with in the pop-up content. The system (400) can place a checkbox or an agreement button along with an agreement phrase at the bottom of the pop-up. The system (400) can block the boarding procedure from proceeding until the user explicitly selects the agreement button. The system (400) can record the time of user consent, displayed regulatory content, and user identification information in a log and store them as legal evidence.

[0150] In operation 730, the system (400) may drive the camera to photograph the fishing license or capture permit when a license or permit is required to capture the main target fish species.

[0151] According to one embodiment, the system (400) can determine whether a special license or permit is required for the capture of a fish species by checking the permit requirement information for each fish species stored in the fishery resource regulation database. If it is confirmed that a license or permit is required, the system (400) can display a guidance message on the user terminal screen and request the user to photograph the license or permit they possess with a camera. The system (400) can automatically launch the camera application on the user terminal and overlay shooting guidelines on the screen to guide the user to ensure that the four corners of the permit are clearly included within the frame. The system (400) can automatically analyze the focus, brightness, resolution, etc., of the captured image to evaluate whether the quality is suitable for character recognition, and if the quality is insufficient, request a retake. The system (400) can display specific guidance text (e.g., "Please take a picture so that all information on the license is clearly visible") on the permit shooting screen. When the user completes the shooting and selects the confirmation button, the system (400) can temporarily save the image and proceed to the next verification step.

[0152] In operation 740, the system (400) can apply optical character recognition to the acquired license image to extract the license number, validity period, and owner information to verify validity.

[0153] According to one embodiment, the system (400) can perform preprocessing on a permit image captured in operation 730 to remove noise and enhance contrast. The system (400) can identify the location of areas containing the permit number, validity period, owner's name, etc. by applying a text detection algorithm that detects text areas within the image. The system (400) can convert the characters within the image into digital text data by running an optical character recognition (OCR) engine on each detected text area. The system (400) can parse the permit number, the date format of the validity period, and the owner's name into structured data by analyzing the extracted text data through a regular expression or pattern matching algorithm. The system (400) can verify whether the permit has not expired by comparing the extracted validity period with the current date. The system (400) can compare the extracted owner's name with the user's identity information identified from biometric information obtained in operation 510. The system (400) can send a query request to the permit issuance database of the marine and fisheries authority to verify the validity of the permit number and check whether the number is a valid number that was actually issued. If the reliability of the OCR result is low, the system (400) can request manual input from the user or guide them to retake the photo.

[0154] In operation 750, the system (400) can forcibly block token generation regardless of the verification condition if the authorization is invalid or if the owner information does not match the user.

[0155] According to one embodiment, the system (400) may determine that a permit is invalid if, as a result of the verification of operation 740, the permit has expired, the permit number does not exist in the database, or is suspected of being forged or altered. The system (400) may determine that another person's permit is being used fraudulently if the extracted owner's name does not match the real name of the person boarding verified through biometric recognition. If the system (400) confirms a failure in permit validity verification or an owner mismatch, it may forcibly stop the process of generating a boarding authentication token, even if all other conditions, including biometric information verification, wearing of safety equipment, and location verification, have been met. The system (400) may display a clear reason for refusal (e.g., "Boarding is restricted because capture permit requirements are not met") on the user terminal screen. The system (400) may log the permit verification failure event and transmit the user's identification information, the captured permit image, the reason for failure, etc., to a management server to support an investigation by relevant authorities. The system (400) can block future boarding by registering the user on a blacklist when repeated attempts to violate the permit are detected.

[0157] According to one embodiment, the system (400) can obtain information on the main target fish species that are the target of the fishing trip from the vessel's departure report information. The departure report information is an official document submitted by the vessel to the maritime and fisheries authorities before departure, and may include the departure time, destination sea area, planned fish species, crew list, etc. The main target fish species refers to the type of fish intended to be mainly caught during the fishing trip, and may be expressed by species names such as tuna, mackerel, squid, and flounder, for example. The system (400) can receive fish species information in real time by linking with the vessel management system or the maritime and fisheries authorities' departure report database.

[0158] According to one embodiment, the system (400) may query a pre-established fishery resource regulation database to extract regulatory data including information on closed seasons, minimum catch sizes, and prohibited catch zones set for the main target fish species. The fishery resource regulation database may refer to a data repository that systematically organizes protection regulations for each fish species based on the Fisheries Resource Management Act and fisheries-related laws. A closed season refers to a period during which catching is legally prohibited because it corresponds to the spawning season or fry growth season of a specific fish species; for example, the closed season for yellow croaker may be designated from May 1 to June 30. A minimum catch size refers to the minimum or maximum body length during which catching is not permitted for resource protection; for example, blue crabs may be regulated as prohibited from catching if they are less than 6.5 cm. A prohibited catch zone refers to a geographical area where the catching of a specific fish species is restricted to protect spawning grounds or habitats, and its boundaries may be set using latitude and longitude coordinates. The system (400) can query the database using the fish species name as a search key and extract the corresponding regulatory information.

[0159] According to one embodiment, if the system (400) determines that the time of boarding falls within a closed fishing season or that the vessel's navigation route includes a prohibited fishing zone, it may obtain user consent by displaying regulatory data as a visual popup on the screen of the user terminal. The system (400) may determine whether this applies by comparing the current date with the start and end dates of the closed fishing season. The system (400) may represent the vessel's planned route as a sequence of GPS coordinates and perform spatial analysis to determine whether this route line intersects the boundary polygon of the prohibited fishing zone. The visual popup is a dialog box displayed in the center of the screen, which can clearly present the legal basis for potential violations, penalties for violations, and compliance requirements using text and icons. If user consent is not obtained, the system (400) may not proceed with the boarding authentication procedure.

[0160] According to one embodiment, the system (400) may induce the user to activate the camera of the user terminal to photograph a fishing license or capture permit when a separate license or permit is required for the capture of the main target fish species. The separate license or permit refers to public approval that must be obtained to fish for specific species or methods beyond general fishing activities, and may include, for example, a whale watching permit or an abalone harvesting permit. The system (400) may check in the regulatory database whether a flag indicating whether a license is required for the relevant fish species is set. The system (400) may automatically launch a camera application and overlay permit photography guidelines on the screen to guide the user to ensure that the four corners are within the frame.

[0161] According to one embodiment, the system (400) can verify validity by applying an optical character recognition algorithm to an acquired permit image to extract the permit number, validity period, and owner information as text. The system (400) can increase the recognition rate by performing noise removal, contrast enhancement, and gradient correction during the image preprocessing process. The permit number is a string that uniquely identifies the permit and may be in a format such as, for example, "No. 2024-1234". The validity period is the period during which the permit has legal effect and may be expressed as a start date and an end date, such as "2024.01.01 ~ 2025.12.31". The owner information may include the name and date of birth listed on the permit. The system (400) can determine whether the permit has expired by checking whether the end date of the extracted validity period is after the current date. The system (400) can verify whether the permit number is an actually issued number by querying the permit issuance database of the Ministry of Oceans and Fisheries.

[0162] According to one embodiment, the system (400) may control the generation of a token to be forcibly blocked regardless of whether the verification conditions are met, if the verification result indicates that the relevant permit is invalid or the owner information does not match the user identified through biometric information. Cases where the permit is invalid may include cases where the validity period has expired, where the permit number does not exist in the database, or where the permit has been revoked.

[0164] According to one embodiment, the system (400) may monitor the communication connection status of the user terminal and, if it is determined to be a dead zone where communication with the server is impossible, execute an offline mode that switches the ship manager terminal to a temporary verification terminal. Monitoring the communication connection status may mean that the terminal periodically sends a ping signal to the central server or calls a network status API to check whether internet access is possible. A dead zone may refer to an area where internet connection is impossible because cellular network or Wi-Fi signals do not reach, such as open sea areas or island areas. Switching to a temporary verification terminal may mean setting the ship manager terminal to temporarily take over the role of the central server to perform token verification and boarding manifest recording functions. The offline mode may refer to an operational state that allows essential boarding management functions to operate even without a network connection.

[0165] According to one embodiment, the system (400) can transmit a token generated through a user terminal to a ship manager terminal via near-field wireless communication. The system (400) may use protocols that enable direct communication between devices without an internet connection, such as Bluetooth, Wi-Fi, and NFC. The system (400) can transmit token data by directly connecting the user terminal and the ship manager terminal in a peer-to-peer (P2P) manner.

[0166] According to one embodiment, the system (400) can perform boarding approval processing after first verifying the validity by decrypting the electronic signature of a token received through a ship manager terminal. Decrypting the electronic signature may refer to a process of decrypting the encrypted signature contained in the token using the user's public key and comparing it with the hash value of the original data to check for tampering. The first verification refers to a basic validity check performed by the ship manager terminal itself without a central server, and the system (400) can check the validity of the signature, the token expiration time, and whether essential data is included.

[0167] According to one embodiment, the system (400) can store sequentially occurring boarding history data by connecting them in a chain form that includes the hash value of the previous data in the next data. The boarding history data may refer to records containing the identity, time of boarding, token information, etc. of each boarder. The chain connection may refer to a structure similar to blockchain technology, whereby the first boarding record is stored as Block 1, its hash value is calculated, the second boarding record is stored as Block 2 with the hash value of Block 1 included, and this process is repeated. The system (400) can implement a structure in which, by having each block include the hash value of the previous block, if the content of any one block is changed, the hash value of that block changes and the connection of all subsequent blocks is broken. If data in the middle of the chain is modified, all hash value verifications after that point in time fail, and the fact of tampering may be revealed.

[0168] According to one embodiment, the system (400) may receive bundles of boarding history data stored in a chain form from a ship manager terminal at the time when communication is restored. Internet connectivity may be restored when the ship approaches the coast or moves to a communication-enabled area. When internet connectivity is restored, all boarding records accumulated during the offline period can be uploaded in bulk to a central server.

[0169] According to one embodiment, the system (400) can verify whether there is a match by comparing the result obtained by recalculating the received data sequentially from the beginning with the result finally generated by the ship manager terminal. The system (400) can repeat the process of calculating the hash value of the first block and combining it with the second block to calculate the second hash value until the last block. The final result is the hash value for the last block of the entire chain and may represent a value that represents the integrity of the entire chain. If the final hash value independently calculated by the central server matches the final hash value submitted by the ship manager terminal, the system (400) can confirm that the data during the offline period has not been tampered with. In the event of a discrepancy, the system (400) can accurately identify the attempt at tampering by tracing back from when the chain was damaged.

[0170] According to one embodiment, the system (400) can receive MME-based marine weather forecast data regarding the vessel's departure route and time period from an external weather observation server and calculate a weather risk level in real time. MME (Multi-Model Ensemble) may refer to a technique that improves forecast accuracy by integrating the results of multiple weather prediction models. The marine weather forecast data may include wave height, wind speed, and swell index. The system (400) may receive expected weather conditions by time period along the vessel's planned route. The weather risk level refers to a classification into stages such as safe, caution, warning, and danger based on the received weather data. For example, if the wave height is 3m or more or the wind speed is 15m / s or more, it may be calculated as a warning level, and the criteria may vary depending on the settings.

[0171] According to one embodiment, the system (400) can extract equipment specification data by performing object recognition and optical character recognition on an image of safety equipment obtained through a camera of a user terminal when the calculated weather risk level exceeds a preset safety threshold. The safety threshold is a level of weather risk where general safety equipment is deemed insufficient, and can be set, for example, to a warning level or higher. Color information refers to the color of the life vest and can determine whether it is a highly visible color (e.g., orange, yellow). The presence or absence of reflective material may refer to whether retroreflective tape or patches that reflect light in nighttime or low-light environments are attached. Buoyancy performance text may refer to a performance level indication, such as 'buoyancy of 7.5 kg or more', printed on the surface of the life vest. The system (400) can generate structured equipment specification data by analyzing the RGB values ​​of the color using an image processing algorithm, identifying the reflective material as a bright color band or a specific texture pattern, and extracting the text using OCR.

[0172] According to one embodiment, the system (400) can perform conformity verification by querying the essential safety equipment grade criteria required for the corresponding weather conditions from a predefined database in response to the calculated weather risk grade. The essential safety equipment grade criteria are rules that define the minimum specifications of safety equipment required for each weather condition; for example, for a warning grade, they may be set to buoyancy of 10 kg or more, reflective material required, and highly visible color required, and the criteria may vary depending on the setting. Conformity verification may mean comparing each item of the extracted equipment specifications (e.g., color, reflective material, buoyancy) with the essential criteria item by item. The system (400) may convert the buoyancy value into a numerical value and perform a comparison with a threshold value, match the color with a predefined list of safety colors, and compare the presence or absence of the reflective material as a binary value.

[0173] According to one embodiment, if the system (400) determines through comparative analysis that the specifications of the safety equipment worn by the user do not meet the essential criteria, it may block the generation of a token regardless of the validity of biometric information and location information. The system (400) may disallow boarding solely due to the failure to meet safety equipment specifications, even if all other boarding conditions are met. The system (400) may display a specific guidance message (e.g., "A life vest with a buoyancy of 10 kg or more is required under current weather conditions") on the screen of the user terminal.

[0174] According to one embodiment, the system (400) can collect real-time biosignals from a heart rate sensor, a blood oxygen saturation sensor, and a body temperature sensor mounted on a user terminal, and analyze the collected biosignals to calculate the user's current health status index. For example, the system (400) can determine that a heart rate exceeds 100 beats per minute or is less than 60 beats per minute, that blood oxygen saturation is less than 95%, or that body temperature is 37.5 degrees or higher is a sign of abnormal health. When a sign of abnormal health is detected, the system (400) can additionally include the relevant information in the situation awareness data so that the health status information is encrypted together when generating a token. When the system (400) transmits the generated token to the ship manager terminal, it can also transmit a health abnormality flag to provide a notification so that the ship manager can perform additional verification or prepare emergency measures for the person on board. The system (400) can periodically monitor the biosignals even after boarding and automatically transmit a warning notification to the ship manager terminal and the shore control center if the health status deteriorates rapidly. The system (400) can support a rapid medical response by providing medical personnel with records of health status at the time of boarding and real-time monitoring data in the event of an emergency at sea. The system (400) can integrate and analyze health data of multiple passengers to detect patterns of collective health abnormalities (e.g., outbreaks of food poisoning or seasickness) at an early stage and recommend changes to the ship's operation plan.

[0175] According to one embodiment, the system (400) may query the user’s past boarding history database to check if there is a record of a maritime accident, a violation of safety regulations, or an emergency evacuation due to a health issue within the last six months. If a high-risk history is confirmed, the system (400) may calculate a risk score along with the token generation and transmit this to the ship manager to indicate that special management is required for the person on board. If the risk score exceeds a threshold, the system (400) may require the completion of additional safety training before boarding or set the designation of an accompanying guardian as a condition.

[0176] According to one embodiment, the system (400) can classify the risk level of the current sea condition into five grades by analyzing real-time wave height, wind speed, and precipitation data received from a weather observation server. If the risk level rises to grade 3 or higher, the system (400) can dynamically adjust the safety distance by automatically reducing or expanding the radius of the safe boarding zone of the vessel from the default setting. If the system (400) determines that boarding itself is dangerous due to deteriorating weather conditions, it can temporarily block all boarding authentication requests and display a message (e.g., "Boarding is restricted due to current weather conditions") on the user terminal. If the weather conditions improve and the risk level drops, the system (400) can automatically lift the boarding restriction and restore the safe zone radius to the original setting. If the system (400) predicts that the weather will rapidly deteriorate within the next two hours based on weather forecast data, it can send a pre-warning notification to those scheduled to board and recommend a change in boarding time.

[0177] According to one embodiment, the system (400) may apply differential safety thresholds for weather conditions to each vessel by considering information on the size, type, and age of the vessel. The system (400) may set standards tailored to the characteristics of the vessel, such as applying boarding restrictions when the wave height is 1.5 meters or higher for small fishing vessels and applying restrictions when the wave height is 3 meters or higher for large passenger vessels. The system (400) may compare the vessel's scheduled departure time with the weather forecast and automatically suggest a delay or cancellation of departure to the vessel manager if the weather is expected to deteriorate during the departure time. By integrating and analyzing weather data and vessel information to determine whether boarding is permitted, the system (400) can prevent maritime accidents caused by deteriorating weather conditions in advance and ensure the safety of the passengers.

[0179] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

Claims

Claim 1 In an automated system for ship check-in and boarding reports, a memory for storing instructions; The system includes a processor, and the instructions, when executed by the processor, acquire situational awareness data in real time, including the user's biometric information, the status of safety equipment wearing, and the results of detecting ship beacon signals, through the sensors and cameras of the user terminal; analyze the acquired situational awareness data to determine whether the biometric information was acquired from a real-time subject rather than through abnormal input using a pre-acquired image or video, whether the safety equipment is located on a designated part of the body, and whether the user terminal is located within the authorized radius of the ship; based on the fact that all of the verification conditions are satisfied, generate a token in which the pattern is updated in real time by combining the hash value of the situational awareness data and time information; encrypt the generated token by electronically signing it with the user's private key, and then transmit it to a ship manager terminal via near-field wireless communication to perform boarding manifest registration; and when acquiring biometric information, if the ambient illuminance is below a preset threshold, emit a lighting signal that continuously changes in a sequence of different brightness or different colors, and generate a differential image between a plurality of face image frames acquired before and after the emission of the lighting signal. Analyzes the reflectance distribution according to changes in the light source, extracts face landmarks corresponding to key feature points of the face from a plurality of face image frames acquired while the lighting signal is emitted, generates an expected 3D shape model based on spatial arrangement information of the extracted landmarks, calculates a pattern in which the brightness of each pixel within the plurality of face image frames changes according to changes in the lighting signal, and the calculated brightness change patternA system that controls the input by comparing it with a reference brightness change pattern simulated based on the surface curvature of the predicted 3D shape model and the light source direction of the lighting signal, determining it as an actual 3D object if the match rate between the two patterns is greater than or equal to a preset value, determining it as a 2D image if the calculated brightness change pattern shows a tendency to match the reflection characteristics of a planar model in which the surface normal vector has a constant direction, or if a specular reflection component exceeding a preset threshold is detected in a local area, or if pixel grid noise generated during display re-capturing within the image is detected, determining it as a normal input if it is determined to be a 3D object, and determining it as an abnormal input by judging it as an attempt at spoofing if it is determined to be a 2D image. Claim 2 In claim 1, the above instructions, when executed by the processor, the system calculates a first hash value by unidirectionally encrypting the situation awareness data at the point in time when all the above verification conditions are satisfied, generates a seed combining time flow and situation information by combining the calculated first hash value with timestamp information updated at a preset time interval, generates a one-time random number code based on the generated dynamic seed, periodically compares a second hash value calculated from situation awareness data reacquired in real time after token generation with the first hash value, determines a security violation state caused by the user removing safety equipment or deviating from the vessel radius when the two hash values ​​do not match, and controls to stop the generation of the random number code or discard the validity verification key value of the generated token to block boarding processing based on the determination of a security violation state. Claim 3 In claim 1, the above instructions are a system that controls the system to determine whether the safety equipment is located in a designated area by analyzing joint feature points of the human body within a user image acquired through the camera, extracting multiple skeletal coordinates corresponding to the left and right shoulders, elbows, wrists, and pelvis, connecting the extracted left and right shoulder coordinates and pelvic coordinates to define an area corresponding to the user's torso, acquiring a bounding box or mask area of ​​the safety equipment identified through object recognition, calculating an Intersection over Union (IoU) ratio in which the area of ​​the safety equipment overlaps with the area corresponding to the torso, determining a wearing state when the area ratio is greater than or equal to a preset threshold, determining a holding state when the area ratio is less than the threshold and the area of ​​the safety equipment is located near the wrist or elbow coordinates, and determining a non-wearing state when the area of ​​the safety equipment is separated from the skeletal coordinates. Claim 4 In claim 1, the above instructions, when executed by the processor, the system receives signals transmitted from beacons distributed at multiple points within the boarding area of ​​the vessel, measures the received signal strength (RSSI) and time of arrival (ToF) of each beacon signal in real time to calculate the relative coordinates of the user terminal using a triangulation method, determines that the user terminal is located within the authorized radius of the vessel only when the calculated relative coordinates are located within a safe zone within a preset threshold distance and the error between the absolute coordinates obtained through the GPS module and the calculated relative coordinates is within an allowable range, and controls the system to stop the boarding manifest registration procedure when the error exceeds the allowable range or the reception pattern of the beacon signals changes abnormally rapidly. Claim 5 delete Claim 6 In claim 1, when the instructions are executed by the processor, the system obtains information on the target species that are the catch target of the fishing trip from the vessel's departure report information, queries a pre-established fisheries resource regulation database to extract regulatory data including information on the closed season, minimum catch size, and prohibited catch zone set for the target species, and if it is determined that the time of boarding falls within the closed season or that the vessel's navigation route includes a prohibited catch zone, the system displays the regulatory data as a visual pop-up on the screen of the user terminal to obtain user consent, and if a separate license or permit is required for catching the target species, the system induces the user to activate the camera of the user terminal to photograph a fishing license or catch permit, applies an Optical Character Recognition (OCR) algorithm to the acquired permit image to extract the permit number, validity period, and owner information as text to verify validity, and if, as a result of the verification, the permit is invalid or the owner information does not match the user identified through the biometric information, the system determines whether the verification conditions are satisfied and A system that controls the generation of tokens that are updated in real time regardless of the above pattern. Claim 7 In claim 1, the instructions, when executed by the processor, execute an offline mode in which, if the system monitors the communication connection status of the user terminal and determines it to be a dead zone where communication with the server is impossible, the ship manager terminal is switched to a temporary verification terminal capable of confirming boarding status; transmit a token, which is updated in real-time and generated to prove that verification conditions including biometrics, safety equipment, and location are satisfied through the user terminal, to the ship manager terminal via near-field wireless communication (P2P); perform boarding approval processing after decrypting the electronic signature of the token received through the ship manager terminal to perform a first verification of validity; and configure the sequentially occurring boarding history data to be stored in a chain form in which the hash value of the previous data is included in the next data, so that if the order of the stored data is changed or the content is falsified, the link of the entire chain is destroyed; and at the time when communication is restored, receive the bundle of boarding history data stored in the chain form from the ship manager terminal, and the result obtained by recalculating the received data sequentially from the beginning and the ship manager terminal finally A system that controls the verification of integrity, specifically whether a passenger manifest stored offline has been tampered with, by comparing the generated result values ​​to check for a match.

Citation Information

Patent Citations

  • Method of delay tolerant network

    KR101958256B1

  • Embarkation management system electronic id card based on deep learning and method thereof

    KR1020210093634A

  • System for providing beacon based safe embarkation service

    KR1020220036611A

  • System and method for managing wear of safety helmets of boarding members on board ship

    KR1020240062414A