System and method for detecting and correcting anomalies in marine data

Abstract

A system and method for detecting and correcting anomalies in marine data are provided. The system and method for detecting and correcting anomalies in marine data according to an embodiment of the present invention comprises: a data collection unit that collects and validates data; a data preprocessing unit that performs OCR processing, LLM integration, and standardization on the collected data; an anomaly detection unit that identifies anomalies on the basis of the preprocessed data; a correction suggestion unit that generates correction suggestions and explanations for the identified anomalies; and a feedback unit that collects feedback on the suggested corrections and trains a model on the basis of the feedback.

Description

Marine data anomaly detection and correction system and method thereof

[0001] The present invention relates to a maritime data anomaly detection and correction system and method thereof, and more particularly, to a maritime data anomaly detection and correction system and method of using the same, which improves the accuracy of anomaly detection through an ensemble approach in maritime navigation data and provides correction suggestions and explanations through a proposed model.

[0002] In the maritime industry, operational data plays a vital role in maintaining the efficiency and safety of vessel operations. However, this data often contains anomalies, which, if not properly detected and corrected, can lead to faulty analysis and decision-making.

[0003] Conventional anomaly detection methods have limited accuracy and adaptability, and often fail to effectively adapt to changing environments. This reduces data reliability and operational safety.

[0004] Therefore, a system that integrates various methods to accurately detect anomalies and provide reliable corrections is needed.

[0005] In order to solve the problems of the prior art as described above, one embodiment of the present invention is to provide a maritime data anomaly detection and correction system and method that can improve the accuracy of anomaly detection by utilizing an ensemble model that combines rule-based, statistical methods, and machine learning models for anomaly detection in maritime navigation data, and can propose reliable corrections for detected anomalies through a proposed model that includes a heuristic algorithm and a machine learning model.

[0006] According to an aspect of the present invention for solving the above-mentioned problem, the present invention comprises: a data collection unit that collects and validates data from a designated email account, a client system, and a third-party service through sources including an email parser, a custom API (Application Programming Interface), and manual upload; a data preprocessing unit that processes the data collected by the data collection unit through OCR (Optical Character Recognition) processing, LLM (Large Language Model) integration, and standardization; an anomaly detection unit that identifies anomalies in the data preprocessed by the data preprocessing unit based on a rule-based detection technique, a statistical analysis technique, and a machine learning detection technique; a modification suggestion unit that proposes a modification to the identified anomaly based on a heuristic algorithm and a machine learning model, and generates an explanation for the proposed modification based on an LP (Natural Language Processing) model and provides the explanation through a template; and a feedback unit that collects feedback on the identified anomaly and the proposed modification through a feedback loop of a web interface and an API, and trains a machine learning model of the anomaly detection unit based on the collected feedback. A system for detecting and correcting marine data anomalies is provided.

[0007] In one embodiment, the anomaly detection unit may include a rule detection unit that detects anomalies in the preprocessed data based on threshold value checks and conditional rules for operating parameters; a statistical analysis unit that detects anomalies in the preprocessed data based on Z-score analysis, moving average, and standard deviation checks; and a machine learning detection unit that detects anomalies in the preprocessed data through anomaly pattern detection according to a supervised learning model and anomaly pattern detection in unlabeled data according to an unsupervised learning model.

[0008] In one embodiment, the modification suggestion unit may include a heuristic unit that suggests a predefined modification based on a heuristic algorithm; a machine learning unit that suggests a modification generated according to a machine learning model that has learned existing modification data; an NLP unit that generates a natural language description of the suggested modification based on decision-making logic and model interpretation; and a template unit that structures the description to ensure consistency and clarity of the description.

[0009] In one embodiment, the feedback unit uses a feedback loop that is integrated into a user interface including the web interface and capable of collecting feedback on the anomaly and the modification; trains the machine learning detection model on data points where the collected feedback occurred based on an active learning algorithm; and manages versions to track and provide changes and updates to the supervised learning model and the unsupervised learning model based on the collected feedback.

[0010] According to another aspect of the present invention, there is provided a data collection step for collecting and validating data from a designated email account, a client system, and a third-party service through sources including an email parser, a custom API, and a manual upload; a data preprocessing step for OCR processing, LLM integration, and standardization of the data collected in the data collection step; an anomaly detection step for identifying anomalies in the data preprocessed in the data preprocessing step based on a rule-based detection technique, a statistical analysis technique, and a machine learning detection technique; a modification suggestion step for proposing a modification to the identified anomaly according to a heuristic algorithm and a machine learning model, and generating an explanation for the proposed modification based on an NLP (Natural Language Processing) model and providing it through a template; a feedback collection and reflection step for collecting feedback on the identified anomaly and the proposed modification through a feedback loop of a web interface and an API, and training a machine learning model of the anomaly detection unit based on the collected feedback; and a data storage and access step for storing the data preprocessed in the data preprocessing step on a server and providing an interactive dashboard and a RESTful API to a user. A system and method for detecting and correcting anomalies in marine data including .

[0011] A system and method for detecting and correcting marine data anomalies according to one embodiment of the present invention collect data for anomaly detection using various sources such as email parsing, API integration using a custom API, and manual upload using a user's web interface, and perform an integrity check on the collected data, thereby improving the accuracy of anomaly detection and ensuring system reliability.

[0012] In addition, the system and method for detecting and correcting anomalies in maritime data according to one embodiment of the present invention can improve the quality of collected data, maintain consistency, and enhance the performance of anomaly detection models by converting documents and images into text through OCR processing and standardizing the extracted text through a custom script, thereby further improving the accuracy of anomaly detection.

[0013] In addition, the system and method for detecting and correcting marine data anomalies according to one embodiment of the present invention can effectively detect anomalies occurring in various environments by securing adaptability by combining a rule-based detection technique utilizing predefined rules, a statistical analysis technique for detecting statistical anomalies, and an ensemble detection model that integrates a machine learning technique for detecting anomaly patterns through supervised learning and unsupervised learning.

[0014] In addition, the system and method for detecting and correcting marine data anomalies according to one embodiment of the present invention utilize a heuristic algorithm and a machine learning model trained on historical correction data to provide rule-based suggestions for common anomalies, thereby generating appropriate correction suggestions for various anomalies that are difficult for a user to predict, including common anomalies, thereby enabling a quick response to the anomalies.

[0015] In addition, the system and method for detecting and correcting marine data anomalies according to one embodiment of the present invention can provide prediction-based suggestions and natural language explanations using NLP models, thereby providing explanations of anomalies and increasing the user's understanding of correction suggestions, thereby improving usability.

[0016] In addition, the system and method for detecting and correcting marine data anomalies according to one embodiment of the present invention collect feedback from system operators and users using a user interface that applies a feedback loop, and train the anomaly detection system with the collected feedback through an active learning algorithm, thereby continuously improving the anomaly detection function in various environments and providing a correction function with improved precision and reliability by reflecting the user's experience.

[0017] Figure 1 is a schematic diagram of a marine data abnormality detection and correction service system according to one embodiment of the present invention.

[0018] FIG. 2 is a block diagram of a system for detecting and correcting abnormalities in maritime data according to one embodiment of the present invention.

[0019] FIG. 3 is a detailed block diagram of an abnormality detection unit of a marine data abnormality detection and correction system according to one embodiment of the present invention.

[0020] FIG. 4 is a detailed block diagram of a modification proposal section of a maritime data anomaly detection and correction system according to one embodiment of the present invention.

[0021] Figure 5 is a flowchart of a method for detecting and correcting abnormal maritime data according to one embodiment of the present invention.

[0022] Figure 6 is a flowchart of an abnormality detection procedure of a method for detecting and correcting abnormalities in maritime data according to one embodiment of the present invention.

[0023] Figure 7 is a flowchart of a modification proposal procedure for a method for detecting and correcting abnormalities in maritime data according to one embodiment of the present invention.

[0024] Hereinafter, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention. The present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description have been omitted for clarity of description, and the same reference numerals designate identical or similar components throughout the specification.

[0025] Hereinafter, a system and method for detecting and correcting abnormalities in maritime data according to an embodiment of the present invention will be described in more detail with reference to the drawings.

[0026] Figure 1 is a schematic diagram of a marine data abnormality detection and correction service system according to one embodiment of the present invention.

[0027] Referring to FIG. 1, a marine data abnormality detection and correction service system (1) according to one embodiment of the present invention may include a marine data abnormality detection and correction system (100), a user terminal (10), and a client system (20).

[0028] The maritime data anomaly detection and correction service system (1) is a system for providing a service for detecting anomalies in operational data essential for operational efficiency and safety in the maritime industry. Based on data collected from sources such as a user terminal (10) and a client system (20), the maritime data anomaly detection and correction system (100) can detect anomalies in operational data and suggest corrections for the detected anomalies.

[0029] The maritime data anomaly detection and correction system (100) collects operational data on maritime operations by periodically monitoring user terminals (10) and client systems (20) through a communication network using tools such as email, API, and web upload to maintain the efficiency and safety of ship operations, detects anomalies through an ensemble model that improves detection accuracy by combining various techniques, and proposes corrections by applying rules and machine learning models to detected anomalies and provides explanations to the user.

[0030] For example, the maritime data abnormality detection and correction system (100) can detect anomalies such as fuel consumption that deviates from the normal range based on maritime operation data collected over the past 6 to 7 years, and can suggest corrections to the detected anomalies, such as values ​​within the normal range.

[0031] The user terminal (10) can communicate with the maritime data anomaly detection and correction system (100) via a wired or wireless communication network. For example, the user terminal (10) may be a wireless portable electronic device such as a smartphone, smart pad, or laptop. In addition, the user terminal (10) can upload navigation data and feedback via a web interface provided by the maritime data anomaly detection and correction system (100).

[0032] The client system (20) can transmit data for anomaly detection to the marine data anomaly detection and correction system (100) via a custom API. Furthermore, the client system (20) may include a third-party service system server used by the user and a system server of the user's company. In this case, the marine data anomaly detection and correction system (100) may distribute a custom API to the client system (20) to directly retrieve data.

[0033] FIG. 2 is a block diagram of a system for detecting and correcting abnormalities in maritime data according to one embodiment of the present invention.

[0034] Referring to FIG. 2, the marine data anomaly detection and correction system (100) may include a storage unit (110) and a control unit (150).

[0035] The storage unit (110) can store information necessary to execute the maritime data anomaly detection and correction system (100). For example, the storage unit (110) can store information such as maritime navigation data acquired by the maritime data anomaly detection and correction system (100) from the user terminal (10) and the client system (20), data preprocessed by the control unit (150), detected anomalies, and suggested corrections.

[0036] The storage unit (110) stores the original and preprocessed data of collected data, and can retrieve data searched by the user under the control of the control unit and provide it to the user terminal (10). For example, the storage unit (110) can store structured data such as metadata and preprocessed data through a database such as PostgreSQL. In addition, the storage unit (110) can store data of various data formats such as semi-structured and unstructured data through a database such as MongoDB. In this case, the storage unit (110) can store large files such as original data, images, and backup archives using an object storage service such as Amazon S3.

[0037] The control unit (150) can be communicatively connected to the storage unit (110) and control the overall operation of the maritime data anomaly detection and correction system (100). In addition, the control unit (150) can collect and preprocess data from the user terminal (10) and the client system (20) to detect anomalies, suggest corrections, and train a machine learning model through feedback. At this time, the control unit (150) can include a data collection unit (151), a data preprocessing unit (153), an anomaly detection unit (155), a correction suggestion unit (157), a feedback unit (158), and an application unit (159).

[0038] The data collection unit (151) can collect maritime navigation data from a user terminal (10) and a client system (20) through various sources, such as an email parser, API integration, and manual upload. Here, the data collection unit (151) can monitor a designated user's email account using an email parsing script. That is, the data collection unit (151) can extract and collect relevant information from the body and attachments of monitored emails. For example, the data collection unit (151) can automatically detect when a report file regarding maritime navigation data is attached to a specific email account and automatically extract it using the Gmail API and a custom email parsing script.

[0039] Additionally, the data collection unit (151) can acquire maritime navigation data in real time through a custom API provided in the client system (20). For example, the data collection unit (151) can collect maritime navigation data such as the operation log and real-time sensor data of the client system (20).

[0040] Additionally, the data collection unit (151) can collect data through manual upload by the user. At this time, the data collection unit (151) can receive data uploads from the user by providing a web interface.

[0041] Additionally, the data collection unit (151) can validate data collected from sources such as email, custom APIs, and manual uploads. For example, the data collection unit (151) can perform format checks and integrity checks to ensure that data meets required criteria before entering the preprocessing process.

[0042] In this way, the marine data anomaly detection and correction system (100) according to one embodiment of the present invention collects data for anomaly detection using various sources such as email parsing, API integration using a custom API, and manual upload using a user's web interface, and performs an integrity check on the collected data, thereby improving the accuracy of anomaly detection and ensuring system reliability.

[0043] The data preprocessing unit (153) can preprocess maritime navigation data collected by the data collection unit (151) and verified to meet criteria through validation. Here, the data preprocessing unit (153) can preprocess data through OCR processing. For example, the data preprocessing unit (153) can convert scanned document and image data into text and extract it using AWS (Amazon Web Services) Textract and Google Cloud Vision APIs.

[0044] In addition, the data preprocessing unit (153) can integrate text extracted through OCR processing based on LLM so that the user can understand the context, and organize and standardize it based on a custom script. At this time, the data preprocessing unit (153) can standardize the preprocessed maritime navigation data by converting it into a consistent format by processing various units and terms through LLM. For example, the data preprocessing unit (153) can parse CSV files, extract text from PDF files, and convert images through OCR using a custom script designed using a Python script.

[0045] In this way, the marine data anomaly detection and correction system (100) according to one embodiment of the present invention can improve the quality of collected data, maintain consistency, and enhance the performance of an anomaly detection model by converting documents and images into text through OCR processing and standardizing the extracted text through a custom script, thereby further improving the accuracy of anomaly detection.

[0046] The anomaly detection unit (155) can detect anomalies in preprocessed maritime navigation data using an anomaly detection model that applies an ensemble technique combining rule-based, statistical analysis, and machine learning detection models. In addition, the anomaly detection unit (155) can learn and manage the anomaly detection model to improve the anomaly detection rate. That is, the anomaly detection unit (155) can detect anomalies by applying a detection technique that combines a rule-based method, a statistical method, and a machine learning detection model to preprocessed maritime navigation data, and can continuously improve detection performance by learning feedback.

[0047] The modification suggestion unit (157) can suggest modifications to the abnormality detected by the abnormality detection unit (155). In addition, the modification suggestion unit (157) can provide an explanation of the proposed modification by converting it into natural language so that the user can understand it.

[0048] That is, the modification suggestion unit (157) can suggest modifications to detected anomalies based on historical data and domain knowledge, and provide a clear and detailed explanation for each proposed modification so that the user can understand and trust the marine data anomaly detection and correction system (100).

[0049] The feedback unit (158) can collect and reflect feedback from operators who use and operate the user terminal (10) and client system (20) to improve the performance of the marine data anomaly detection and correction system (100). Here, the feedback unit (158) can collect feedback in real time from operators of the user terminal (10) and client system (20) through a feedback loop integrated into the user interface of the marine data anomaly detection and correction system (100). For example, the feedback unit (158) can collect feedback, such as errors in response to detected anomalies, in a manner in which the operator annotates data points.

[0050] Additionally, the feedback unit (158) can train the anomaly detection model with feedback collected through an active learning algorithm. At this time, the active learning algorithm can allow the anomaly detection model to prioritize retraining on annotated data points.

[0051] The feedback unit (158) can train the anomaly detection model with feedback collected through an active learning algorithm. At this time, the active learning algorithm can induce the anomaly detection model to preferentially perform retraining on annotated data points.

[0052] Additionally, the feedback unit (158) can track and provide changes and updates to the anomaly detection model based on feedback collected from operators through version control. For example, the feedback unit (158) can configure an active learning algorithm based on Python libraries such as Scikit-learn and TensorFlow, and perform version control based on Data Version Control (DVC).

[0053] In this way, the marine data anomaly detection and correction system (100) according to one embodiment of the present invention collects feedback from system operators and users using a user interface that applies a feedback loop, and trains the anomaly detection system with the collected feedback through an active learning algorithm, thereby continuously improving the anomaly detection function in various environments and providing a correction function with improved precision and reliability by reflecting the user's experience.

[0054] The application unit (159) may provide a user interface, such as a web interface, and an API, such as a RESTful API, to allow users to access the maritime data anomaly detection and correction system (100). Here, the application unit (159) may provide an interactive dashboard and tools through the user interface, enabling users to access data, view anomaly reports, and provide feedback. For example, the application unit (159) may provide a web interface using an interactive dashboard built on a framework, such as React.js or Angular.

[0055] Additionally, the application unit (159) may provide an API that allows programmatic access to the system's data and functions, enabling integration with other corporate systems. For example, the application unit (159) may provide a RESTful API built with a framework such as Flask or Django to provide secure and efficient access to the system's functions.

[0056] FIG. 3 is a detailed block diagram of an abnormality detection unit of a marine data abnormality detection and correction system according to one embodiment of the present invention.

[0057] Referring to FIG. 3, the anomaly detection unit (155) may include a rule detection unit (1551), a statistical analysis unit (1553), and a machine learning detection unit (1555).

[0058] The rule detection unit (1551) can identify anomalies by applying predefined rules and thresholds. Here, the rule detection unit (1551) can check and identify anomalies by setting simple conditions, such as the maximum and minimum values ​​of operating parameters, as thresholds.

[0059] The rule detection unit (1551) can detect anomalies by utilizing conditional rules established based on various conditions, such as fuel consumption and travel distance. In this case, the rule detection unit (1551) can comprehensively analyze various conditions to identify anomalies that exceed operating standards.

[0060] The statistical analysis unit (1553) can identify values ​​that indicate anomalies through Z-score analysis, moving average, and standard deviation tests. Here, the statistical analysis unit (1553) can identify values ​​that indicate anomalies by measuring how many standard deviations a data point is away from the mean through Z-score analysis. In addition, the statistical analysis unit (1553) can identify values ​​that indicate anomalies that deviate from expected trends over time through moving average analysis. In addition, the statistical analysis unit (1553) can identify values ​​that indicate anomalies by flagging data points that deviate from the mean by a certain standard deviation through standard deviation tests.

[0061] The machine learning detection unit (1555) can identify abnormal patterns using machine learning models, including supervised and unsupervised learning models. For example, the machine learning detection unit (1555) can detect known abnormal patterns using supervised learning models such as random forests and gradient boosting machines trained on labeled abnormal data. Furthermore, the machine learning detection unit (1555) can detect new abnormal patterns in unlabeled data using unsupervised learning models such as isolation forests and DBSCAN to cluster outliers in unlabeled data.

[0062] The anomaly detection unit (155) implements machine learning and statistical algorithms based on Scikit-learn, develops and trains a machine learning model based on TensorFlow rules, and can detect specific anomalies through a self-developed custom algorithm.

[0063] In this way, the marine data anomaly detection and correction system (100) according to one embodiment of the present invention can effectively detect anomalies occurring in various environments by securing adaptability by combining a rule-based detection technique utilizing predefined rules, a statistical analysis technique for detecting statistical anomalies, and an ensemble detection model that integrates a machine learning technique for detecting anomaly patterns through supervised learning and unsupervised learning.

[0064] FIG. 4 is a detailed block diagram of a modification proposal section of a maritime data anomaly detection and correction system according to one embodiment of the present invention.

[0065] Referring to FIG. 4, the modification proposal unit (157) may include a heuristic unit (1571), a machine learning unit (1573), an NLP unit (1575), and a template unit (1577).

[0066] The heuristic unit (1571) can generate rule-based correction suggestions based on predefined correction actions for common anomalies through a heuristic algorithm. Since the heuristic unit (1571) can quickly find approximate correction suggestions for common anomalies, it can operate effectively even in environments with limited computational resources. For example, the heuristic unit (1571) can implement the heuristic algorithm through a rule-based script developed in Python.

[0067] The machine learning unit (1573) can predict and propose appropriate corrections to detected anomalies using a machine learning model trained on historical correction data. In other words, the machine learning unit (1573) analyzes large amounts of data and recognizes patterns through the machine learning model, enabling it to suggest appropriate response measures for potential future anomalies based on past correction cases.

[0068] Additionally, the machine learning unit (1573) utilizes the learning algorithm of the machine learning model to reflect the characteristics of data that change over time, thereby suggesting accurate and reliable modifications. At this time, the machine learning unit (1573) continuously learns new data through the machine learning model, allowing performance to improve over time.

[0069] Furthermore, the machine learning unit (1573), trained on historical correction data, can propose more complex corrections. In this process, the machine learning unit (1573) conducts multidimensional analysis, considering various variables. This allows it to identify complex anomalies that cannot be captured with a simple rule-based approach and suggest corrections. For example, the machine learning unit (1573) can simultaneously analyze multiple factors in a specific situation to derive an optimal correction plan.

[0070] The NLP unit (1575) can generate a natural language explanation of the modifications through the NLP model. At this time, the NLP unit (1575) can generate an explanation of the modifications in a form that is easy for the user to understand, based on the decision-making logic and model interpretation through the NLP model. In other words, the NLP unit (1575) can generate and provide an explanation in natural language so that the user can clearly understand the suggested corrections for the abnormalities detected by the marine data abnormality detection and correction system (100).

[0071] For example, the NLP unit (1575) can generate natural language explanations through an NLP model based on the latest transformer architecture such as GPT-3 or BERT.

[0072] Additionally, the NLP unit (1575) can provide explanations for correction suggestions to users in an intuitive and accessible manner through an NLP model that learns a large amount of text data to understand the context and generate content in natural language.

[0073] The template unit (1577) can apply predefined templates to the descriptions generated by the NLP unit (1575) to ensure consistency and clarity. At this time, the template unit (1577) can systematically organize and structure the descriptions provided to users through the templates. Furthermore, the template unit (1577) can provide descriptions using a customized description framework that combines natural language descriptions output from the NLP model with predefined templates to generate descriptions.

[0074] In this way, the marine data anomaly detection and correction system (100) according to one embodiment of the present invention utilizes a heuristic algorithm and a machine learning model that has learned historical correction data to provide rule-based suggestions for common anomalies, thereby generating appropriate correction suggestions for various anomalies that are difficult for a user to predict, including common anomalies, and thus enabling a quick response to the anomalies.

[0075] In addition, the marine data anomaly detection and correction system (100) according to one embodiment of the present invention can provide a prediction-based suggestion and a natural language explanation using an NLP model, thereby providing an explanation of the anomaly and increasing the user's understanding of the correction suggestion, thereby improving usability.

[0076] Figure 5 is a flowchart of a method for detecting and correcting abnormal maritime data according to one embodiment of the present invention.

[0077] Referring to FIG. 5, the method for detecting and correcting anomalies in maritime data (100) may include a data collection step (S210), a data preprocessing step (S220), an anomaly detection step (S230), a correction suggestion step (S240), a feedback collection and reflection step (S250), and a data storage and access step (S260).

[0078] More specifically, the maritime data anomaly detection and correction system (100) collects and validates data from designated email accounts, client systems, and third-party services through sources including an email parser, a custom API, and manual upload (step S210). At this time, the maritime data anomaly detection and correction system (100) monitors emails sent and received from the designated email account, extracts and collects data related to maritime operations from the body and attachments, and can collect data related to maritime operations sent and received from a client system (20) equipped with a custom API. In addition, the maritime data anomaly detection and correction system (100) can collect data manually uploaded by a user through a user terminal (10). At this time, the maritime data anomaly detection and correction system (100) can verify the validity of the collected data through a format check and an integrity check.

[0079] Next, the maritime data anomaly detection and correction system (100) performs OCR processing, LLM integration, and standardization on the data collected in the data collection step (S210) (step S220). At this time, the maritime data anomaly detection and correction system (100) performs OCR processing on the data collected in step S210 to extract text from documents and images, integrates the extracted text through LLM to secure contextual understanding, and organizes and standardizes the integrated text to ensure consistency in units and terms through a custom script.

[0080] Next, the maritime data anomaly detection and correction system (100) identifies anomalies in the preprocessed data based on rule-based detection techniques, statistical analysis techniques, and machine learning detection techniques (step S230). At this time, the maritime data anomaly detection and correction system (100) can detect anomalies in maritime navigation data through an ensemble model that combines rule-based detection, statistical analysis, and machine learning detection techniques.

[0081] Next, the marine data anomaly detection and correction system (100) proposes corrections to the anomalies identified in step S230 based on a heuristic algorithm and a machine learning model, and generates an explanation of the proposed corrections based on an NLP model and provides the explanations through a template (step S240). At this time, the marine data anomaly detection and correction system (100) can generate and propose corrections using a heuristic algorithm and a machine learning model trained on historical correction data, and generate an explanation of the proposed corrections in natural language using NLP.

[0082] In addition, the marine data anomaly detection and correction system (100) can provide a user with an explanation of the correction by applying an explanation framework configured based on pre-established rules to the generated natural language explanation.

[0083] Next, the marine data anomaly detection and correction system (100) collects feedback on the anomalies identified in step S230 and the corrections suggested in step S240 through a feedback loop of the web interface and API, and trains a machine learning model based on the collected feedback (step S250). At this time, the marine data anomaly detection and correction system (100) can train the machine learning model that performs anomaly detection in step S230 by applying an active learning algorithm. In addition, the marine data anomaly detection and correction system (100) can train the machine learning model by prioritizing retraining for data points where feedback has occurred.

[0084] Next, the marine data anomaly detection and correction system (100) stores the preprocessed data and original data in step S220 on a server and provides an interactive dashboard and a RESTful API to the user (step S260). At this time, the marine data anomaly detection and correction system (100) may provide an interactive dashboard configured to allow users to access and visualize the data, and a RESTful API configured to allow users to programmatically access the data.

[0085] Figure 6 is a flowchart of an abnormality detection procedure of a method for detecting and correcting abnormalities in maritime data according to one embodiment of the present invention.

[0086] Referring to FIG. 6, the anomaly detection procedure (S230) includes a rule detection step (S231), a statistical analysis step (S233), and a machine learning detection step (S235).

[0087] The marine data anomaly detection and correction system (100) detects anomalies in the preprocessed data based on Z-score analysis, moving average, and standard deviation tests (step S233). At this time, the marine data anomaly detection and correction system (100) can detect anomalies using statistical methods such as Z-score analysis, which detects outliers by measuring the standard deviation between the average and data points, moving average analysis, which identifies anomalies that deviate from the expected trend over time, and standard deviation tests, which detect anomalies that deviate from the average of data points by a certain standard deviation.

[0088] Next, the marine data anomaly detection and correction system (100) detects anomalies in the preprocessed data using supervised and unsupervised machine learning detection models (step S235). At this time, the marine data anomaly detection and correction system (100) can detect known anomaly patterns using a supervised learning model trained with labeled anomaly data, and detect anomaly patterns in unlabeled data using an unsupervised learning model.

[0089] Figure 7 is a flowchart of a modification proposal procedure for a method for detecting and correcting abnormalities in maritime data according to one embodiment of the present invention.

[0090] Referring to FIG. 7, the modification proposal procedure (S240) includes a heuristic algorithm-based modification proposal step (S241), a machine learning model-based modification proposal step (S243), an NLP model-based explanation generation step (S245), and a template application step (S247).

[0091] More specifically, the marine data anomaly detection and correction system (100) uses a heuristic algorithm to propose predefined corrections (step S241). That is, the marine data anomaly detection and correction system (100) can generate approximate correction suggestions for common anomalies through the heuristic algorithm.

[0092] Next, the marine data anomaly detection and correction system (100) proposes corrections using a machine learning model trained on existing correction data (step S243). At this time, the marine data anomaly detection and correction system (100) proposes corrections for potential anomalies using the machine learning model trained on historical correction data, and can continuously train the machine learning model with new data to improve the performance of the correction suggestions.

[0093] Next, the marine data anomaly detection and correction system (100) generates a natural language explanation of the proposed modification based on decision-making logic and model interpretation (step S245). At this time, the marine data anomaly detection and correction system (100) can generate a natural language explanation of the proposed modification in step S243, using an NLP model, so that the user can clearly understand it.

[0094] Next, the marine data anomaly detection and correction system (100) structures the natural language description by applying predefined templates and rules (step S247). At this time, the marine data anomaly detection and correction system (100) can combine the description generated in step S245 with the predefined template to create a description that ensures consistency and clarity through the description framework.

[0095] The above methods can be implemented by a marine data anomaly detection and correction system and method (100) as shown in FIG. 1, and in particular, can be implemented by a software program performing these steps, in which case these programs can be stored in a computer-readable recording medium or transmitted by a computer data signal combined with a carrier wave in a transmission medium or a communication network.

[0096] At this time, the computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored, and may be, for example, ROM, RAM, CD-ROM, DVD-ROM, DVD-RAM, magnetic tape, floppy disk, hard disk, optical data storage device, etc.

[0097] Although one embodiment of the present invention has been described above, the spirit of the present invention is not limited to the embodiments presented in this specification, and those skilled in the art who understand the spirit of the present invention will be able to easily propose other embodiments by adding, changing, deleting, or adding components within the scope of the same spirit, but this will also be considered to fall within the spirit of the present invention.

[0098] This invention can be utilized in the shipping and port logistics industries by detecting and correcting abnormalities in vessel operation data in real time. Furthermore, it can be utilized in digital forensics and cybersecurity through data integrity verification and forgery / alteration detection. Furthermore, it can be applied to AI-based data purification and correction services through data quality enhancement based on OCR and preprocessing technologies.

[0099] The present invention can be utilized in the fields of industrial monitoring and automatic control systems with its sensor-based real-time detection function, and can also be utilized in the smart marine and environmental monitoring industry fields by integrating and analyzing heterogeneous marine data to respond to environmental changes.

Claims

1. A data collection unit that collects and validates data from designated email accounts, client systems, and third-party services through sources including email parsers, custom APIs (Application Programming Interfaces), and manual uploads; A data preprocessing unit that processes the data collected from the above data collection unit through OCR (Optical Character Recognition) and integrates and standardizes the LLM (Large Language Model); An anomaly detection unit that identifies anomalies in the data preprocessed in the above data preprocessing unit based on a rule-based detection technique, a statistical analysis technique, and a machine learning detection technique; A modification suggestion unit that proposes modifications to the identified anomalies based on heuristic algorithms and machine learning models, and generates an explanation of the proposed modifications based on an NLP (Natural Language Processing) model and provides the explanation through a template; and A feedback unit that collects feedback on the identified anomalies and suggested corrections through a feedback loop of a web interface and API, and trains a machine learning model of the anomaly detection unit based on the collected feedback; A marine data anomaly detection and correction system including .

2. In paragraph 1, The above abnormality detection unit, A rule detection unit that detects anomalies in the preprocessed data based on threshold value checks and conditional rules for operating parameters; A statistical analysis unit that detects anomalies in the preprocessed data based on Z-score analysis, moving average and standard deviation tests; and A machine learning detection unit that detects anomalies in the preprocessed data by detecting anomalies in unlabeled data according to a supervised learning model and detecting anomalies in unlabeled data according to a non-supervised learning model; A marine data anomaly detection and correction system including .

3. In paragraph 1, The above amendment proposal is, A heuristic section that suggests predefined modifications based on heuristic algorithms; A machine learning unit that proposes modifications based on a machine learning model that has learned existing modification data; An NLP unit that generates a natural language explanation of the proposed modification based on decision logic and model interpretation; and A template section that structures the above description to ensure consistency and clarity of description; A marine data anomaly detection and correction system including .

4. In paragraph 2, The above feedback section, A feedback loop is used that is integrated into the user interface including the above web interface to enable the collection of feedback on the above ideal and the above modification; The supervised learning and unsupervised learning models of the machine learning detection unit are trained on the data points where the collected feedback occurs based on the active learning algorithm. A system and method for detecting and correcting marine data anomalies, which manages versions to track and provide changes and updates to the supervised learning model and the non-supervised learning model based on the collected feedback.

5. A data collection step that collects and validates data from specified email accounts, client systems, and third-party services through sources including email parsers, custom APIs, and manual uploads; A data preprocessing step that OCR-processes, LLM integrates, and standardizes the data collected in the above data collection step; An anomaly detection step that identifies anomalies in the data preprocessed in the above data preprocessing step based on a rule-based detection technique, a statistical analysis technique, and a machine learning detection technique; A modification suggestion step that proposes a modification to the identified anomaly based on a heuristic algorithm and a machine learning model, and generates an explanation for the proposed modification based on an NLP model and provides it through a template; A feedback collection and reflection step for collecting feedback on the identified anomalies and suggested corrections through a feedback loop of the web interface and API, and training the machine learning model of the anomaly detection unit based on the collected feedback; and A data storage and access step that stores preprocessed data and original data in the above data preprocessing step on a server and provides an interactive dashboard and RESTful API to users; A system and method for detecting and correcting anomalies in marine data including:

Images