Method and system for continuous learning of fish species identification ai model based on metadata-driven user-participatory label quality evaluation

Abstract

A metadata-based user-participatory label quality evaluation and continuous learning system for a fish species identification AI model may include memory and a processor for storing instructions. The system acquires and stores images including fishing photos from a user terminal, performs initial classification by calculating fish species candidates and prediction probabilities for the input images using a pre-trained fish species classification model, displays a list of top prediction results and candidates based on prediction probabilities to the user, and provides a user interface that receives confirmation, candidate selection, or direct input regarding the prediction results, generates a user action log by collecting event information including the type of user input, dwell time, and number of zooms in and outs of user inputs occurring in the user interface, scans the user action log at regular intervals to collect new user modification history, evaluates and refines label quality by analyzing metadata including user reliability based on the labeling match rate for already verified data and the degree of consensus on modification results by multiple users, generates a training data set by selecting images with refined labels, and controls the model to update by performing retraining or fine-tuning of the fish species classification model according to the cumulative amount of the training data or recent model performance indicators.

Description

Technology Field

[0001] The present invention relates to the field of artificial intelligence-based image analysis and continuous learning technology for machine learning models.

[0002] More specifically, the invention relates to a system and a method of operation thereof that collects training data through a participatory labeling method in which a user directly modifies or verifies the prediction results provided by a fish species identification AI model, and evaluates and refines the quality of labels by comprehensively analyzing the collected training data, thereby continuously retraining a fish species classification model and improving its performance based on refined, high-quality training data. Background Technology

[0004] With the recent increase in demand for automatic fish species identification technology in the fisheries and marine leisure sectors, the development of fish species identification systems utilizing deep learning-based image classification models is actively underway. These systems operate by receiving images of catches as input, automatically classifying fish species through pre-trained artificial intelligence models, and providing the identification results to the user.

[0005] However, existing fish species classification AI systems have a static structure in which they collect an initial dataset, have professional labelers or fisheries experts manually assign species labels, train a classification model using those labels, and then deploy the trained model to a service to perform predictions in a fixed state.

[0006] This approach has a problem where data drift occurs, leading to a degradation in model performance over time, if there is a discrepancy between the data distribution used for model training and the data distribution occurring in the actual service environment.

[0007] Furthermore, securing high-quality labels for new data requires separate labeling work, which places a significant burden in terms of cost and time, such as expert labor costs and the establishment of labeling tools. In addition, existing systems have limitations in that the model operates in a fixed state after initial training and lacks a structure for continuous learning that automatically incorporates new data and label information collected during service.

[0008] Furthermore, the quality of labeling information provided by users varies significantly depending on individual expertise, attention span, and input patterns; if user input is simply used as training data, there is a risk that noisy, low-quality data may be included in the model training, potentially worsening performance. The problem to be solved

[0010] The present invention aims to secure high-quality training data in real time without the cost of separate expert labeling by presenting results predicted by a fish species classification AI model to a user, systematically collecting all action information generated during the process of the user verifying or modifying the prediction results, and utilizing this as user-participatory labeling data.

[0011] Furthermore, the present invention aims to minimize noise from individual user input and derive accurate labels utilizing collective intelligence by evaluating user-specific reliability based on verified data-based historical labeling match rates, calculating a user-based quality score by combining behavioral indicators including dwell time, number of zooms, and whether direct input was performed, and calculating the degree of consensus using each user's reliability as a weight when multiple users modify the same image or the same fish species.

[0012] The present invention calculates a multidimensional metadata score including environmental suitability based on model prediction probability and fish species appearance probability according to environmental variables, cluster consistency based on fish species distribution reported in the same cell and similar time periods, and outlier detection based on image quality and metadata abnormal combinations, and determines the reliability of a label based on a comprehensive quality score integrated as a weighted sum of these scores to reflect in the generation of training data. means of solving the problem

[0014] A continuous learning system for a metadata-based user-participatory label quality evaluation and fish species identification AI model may include memory and a processor for storing instructions.

[0015] When executed by a processor, the instructions control the system to acquire and store an image including a fishing photo from a user terminal, perform initial classification by calculating fish species candidates and prediction probabilities for the input image using a pre-trained fish species classification model, display a list of top prediction results and candidates based on the prediction probabilities to the user, provide a user interface that receives confirmation, candidate selection, or direct input regarding the prediction results, generate a user action log by collecting event information including the type of user input, dwell time, and number of zooms in and outs of user input occurring in the user interface, scan the user action log at regular intervals to collect new user modification history, evaluate and refine label quality by analyzing metadata including user reliability based on the labeling match rate for already verified data and the degree of agreement on modification results by multiple users, generate a training data set by selecting images with refined labels, and update the model by performing retraining or fine-tuning of the fish species classification model according to the cumulative amount of the training data or recent model performance indicators. Effects of the invention

[0017] The metadata-based user-participatory label quality evaluation and fish species identification artificial intelligence model continuous learning system according to the present invention systematically collects and refines feedback that naturally occurs during the process of a user using the service and utilizes it as training data, thereby enabling the acquisition of large-scale, high-quality training data in real time without the cost of separate expert labeling.

[0018] In addition, the present invention can significantly improve the quality of model training by comprehensively analyzing multidimensional metadata including user reliability, consensus, model reliability, environmental suitability, cluster consistency, and outlier detection to precisely evaluate label quality, thereby effectively removing noise and bias from individual user inputs and adopting only highly reliable, refined labels as training data.

[0019] The present invention can automatically respond to changes in data distribution over time and continuously improve model performance by continuously retraining and fine-tuning a fish species classification model based on refined training data, and stably updating the model through sandbox verification and phased deployment. Brief explanation of the drawing

[0021] FIG. 1 is a diagram illustrating the overall structure of an artificial intelligence-based system according to one embodiment. FIG. 2 is a diagram illustrating the learning of a neural network according to one embodiment. FIG. 3 is a diagram illustrating the configuration of an artificial intelligence model according to one embodiment. FIG. 4 is a block diagram showing the configuration of a metadata-based user-participatory label quality evaluation and fish species identification artificial intelligence model continuous learning system according to one embodiment. FIG. 5 is a flowchart illustrating a method for continuous learning of a metadata-based user-participatory label quality evaluation and fish species identification artificial intelligence model according to one embodiment. FIG. 6 is a flowchart illustrating a metadata-based label quality evaluation method according to one embodiment. FIG. 7 is a flowchart illustrating a model distribution and user reliability management method according to one embodiment. Specific details for implementing the invention

[0022] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, various modifications may be made to the embodiments, and thus the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, and substitutions to the embodiments are included within the scope of the rights.

[0023] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0024] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0025] The terms used in the embodiments are for illustrative purposes only and should not be interpreted as intended to be limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0026] In addition, when describing with reference to the attached drawings, identical components are assigned the same reference numeral regardless of drawing symbols, and redundant descriptions thereof are omitted. In describing the embodiments, if it is determined that a detailed description of related prior art could unnecessarily obscure the essence of the embodiments, such detailed description is omitted.

[0027] The embodiments can be implemented in various forms of products such as personal computers, laptop computers, tablet computers, smartphones, televisions, smart home appliances, intelligent automobiles, kiosks, and wearable devices.

[0028] Artificial Intelligence (AI) systems are computer systems that implement human-level intelligence; unlike existing rule-based smart systems, they are systems in which machines learn and make decisions autonomously. As AI systems improve in recognition accuracy and gain a more accurate understanding of user preferences with continued use, existing rule-based smart systems are gradually being replaced by deep learning-based AI systems.

[0029] Artificial intelligence technology consists of machine learning and component technologies utilizing machine learning. Machine learning is an algorithmic technology that autonomously classifies and learns the characteristics of input data, while component technologies are technologies that mimic the cognitive and judgmental functions of the human brain by utilizing machine learning algorithms such as deep learning, and are comprised of technological fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control.

[0030] Generally, to apply machine learning algorithms to real-world situations, training is performed using a trial-and-error method due to the inherent characteristics of the fundamental methodologies. In particular, deep learning requires hundreds of thousands of iterations. Since it is impossible to execute this in a real physical external environment, training is instead performed through simulations that virtually recreate the actual physical environment on a computer.

[0031] In the present invention, Artificial Intelligence (AI) refers to a technology that imitates human learning ability, reasoning ability, perceptual ability, etc., and implements them on a computer, and may include concepts such as machine learning and symbolic logic. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. Artificial intelligence technology can analyze input data as a machine learning algorithm, learn the results of the analysis, and make judgments or predictions based on the results of the learning.

[0032] Machine learning can refer to the process of training neural network models using experience in processing data. Through machine learning, computer software can improve its own data processing capabilities. A neural network model is constructed by modeling the correlations between data, and these correlations can be expressed by multiple parameters. A neural network model extracts and analyzes features from given data to derive correlations between them; machine learning can be defined as the process of optimizing the model's parameters by repeating this process. For example, a neural network model can learn the mapping (correlation) between inputs and outputs for data given as input-output pairs. Alternatively, even when only input data is provided, a neural network model can derive regularities between the given data and learn those relationships.

[0034] FIG. 1 is a diagram illustrating the overall structure of an artificial intelligence-based system according to one embodiment.

[0035] As illustrated in FIG. 1, an artificial intelligence-based system (100) may include a plurality of user terminals (110-1 to 110-n), a server (120), and a database (130). This system adopts a distributed architecture and operates based on a client-server model, with each component playing a unique role and contributing to increasing the efficiency and scalability of the entire system. According to one embodiment, the database (130) is depicted as being configured separately from the server (120), but is not limited thereto; depending on system design and operational efficiency, the database (130) may be integrated within the server (120). This integrated configuration has the advantage of improving data access speed and reducing system complexity. For example, the server (120) may include a plurality of artificial intelligence models and processing units for performing machine learning algorithms, and these may implement various types of deep learning and machine learning technologies (e.g., CNN, RNN, Transformer, reinforcement learning, etc.) to respond to user requests and provide intelligent services. According to one embodiment, a plurality of user terminals (110-1 to 110-n), a server (120), and a database (130) can be connected to communicate with each other through a network (N), which enables real-time data exchange and smooth interaction between system components.

[0036] According to one embodiment, the network (N) may perform wireless or wired communication between a plurality of user terminals (110-1 to 110-n), a server (120), a database (130), etc., and is located in the center of FIG. 1 and serves as a hub connecting all components. For example, the network may perform wireless communication according to methods such as 5G, LTE (Long-Term Evolution), LTE-A (LTE Advanced), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), WiBro (Wireless BroadBand), WiFi (Wireless Fidelity), Bluetooth, NFC (Near Field Communication), GPS (Global Positioning System), or GNSS (Global Navigation Satellite System). The 5G network supports high-speed data transmission (up to 20Gbps), ultra-low latency (1ms or less), and large-scale connectivity, making it suitable for real-time processing of large-capacity AI models.

[0037] According to one embodiment, user terminals (110-1 to 110-n) are various client devices that access the system and can be implemented in various forms, such as smartphones, tablets, desktop computers, wearable devices, and IoT devices. These terminals are responsible for transmitting service requests through a user interface and receiving and displaying results processed by a server (120). Each terminal may also run a lightweight AI model locally, which can reduce network latency and enhance privacy protection. Direct connection or peer-to-peer (P2P) communication between terminals may be possible, as indicated by the dotted line, which is particularly useful in distributed learning or edge computing scenarios.

[0038] According to one embodiment, the server (120) is a central processing unit of the system that processes user requests, executes artificial intelligence models, and is responsible for data management. The server is equipped with a high-performance processor (CPU, GPU, TPU, NPU, etc.) for large-scale computation, large-capacity memory, and a stable operating system, and may be operated in a virtualized manner on a cloud infrastructure. The server performs training and inference functions for complex deep learning models and transmits the results to user terminals or stores them in a database.

[0039] According to one embodiment, the database (130) is a storage facility for storing and managing various data, and is depicted in a cylindrical shape in FIG. 1. The data stored in the database (130) is data that is acquired, processed, or used by at least one component of a plurality of user terminals (110-1 to 110-n) or a server (120), and may include software (e.g., programs), user profiles, training datasets, model weights, log records, etc. The database (130) may include volatile and / or non-volatile memory, where the volatile memory (RAM) is used for caching or temporary data processing requiring fast data access, and the non-volatile memory (SSD, HDD, tape drive, etc.) is used for permanent data storage.

[0041] FIG. 2 is a diagram illustrating the learning of a neural network according to one embodiment.

[0042] As illustrated in FIG. 2, the learning device can train a neural network (123) to classify review responses received from a plurality of user terminals (110-1 to 110-n) by item. Additionally, the learning device can train a neural network (123) to extract user stay history from user movement path information. The neural network (123) is also called an artificial neural network and is a computational model designed inspired by the neural structure of the human brain, and is specialized in recognizing and learning complex patterns within data.

[0043] According to one embodiment, the neural network (123) includes an input layer (121) into which training samples are input and an output layer (125) that outputs training outputs, and can be trained based on the difference between the training outputs and the labels (i.e., actual correct data). The neural network (123) is connected in groups of multiple nodes and is defined by weights between the connected nodes and an activation function that activates the nodes. Various forms of activation functions can be used, such as sigmoid, hyperbolic tangent (tanh), and ReLU, which allows the network to learn non-linear patterns.

[0044] According to one embodiment, the learning device can train a neural network (123) using a Gradient Descent (GD) technique or a Stochastic Gradient Descent (SGD) technique. The GD technique is a method of updating weights at once by calculating gradients based on the entire dataset, while the SGD technique is a method of increasing computational efficiency and the possibility of escaping a local optimum by updating weights more frequently using only a randomly selected portion of data (mini-batch). The learning device can use a loss function designed by the outputs and labels of the neural network.

[0045] The learning device can calculate the training error using a predefined loss function. The loss function can be predefined with labels, outputs, and parameters as input variables, where the parameters can be set by weights within the neural network (123). For example, the loss function can be designed in the form of Mean Square Error (MSE), entropy, etc. MSE is calculated as the squared mean of the difference between the predicted value and the actual value and is mainly used for regression problems, while cross-entropy is a function that measures the difference between the predicted probability distribution and the actual distribution and is suitable for classification problems.

[0046] According to one embodiment, the learning device can identify weights that influence the training error using a backpropagation technique. Backpropagation is a process of calculating the degree to which each weight contributes to the final error while propagating the error calculated in the output layer toward the input layer, and is performed through differential calculations using the chain rule. Here, the weights are relationships between nodes within the neural network (123). The learning device may use an SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning device may update the weights of a loss function defined based on labels, outputs, and weights using an SGD technique. This process operates by calculating the gradient (the derivative of the loss function with respect to the weights) and adjusting the weights in the direction of this gradient to gradually decrease the value of the loss function.

[0047] According to one embodiment, the learning device extracts first objects from a review response, obtains first labels which are items corresponding to the first objects, applies the first objects to a first neural network to generate first training outputs corresponding to the first objects, and can train the first neural network based on the first training outputs and the first labels. At this time, the first objects may include key keywords, sentence structures, sentiment expressions, etc. extracted from the review text, and thereby learn the ability to identify the core content of the text and classify it into the corresponding items.

[0048] According to one embodiment, the learning device extracts second objects from movement path information, obtains second labels which are user dwell history corresponding to the second objects, applies the second objects to a second neural network to generate second training outputs corresponding to the second objects, and can train the second neural network based on the second training outputs and the second labels. The second objects may consist of data such as the user's GPS coordinate sequence, movement speed, stopping point, and movement pattern, thereby learning the ability to determine where and for how long the user stayed.

[0049] According to one embodiment, a learning device can generate first training feature vectors based on constituent features (e.g., sentence structure, word frequency, part-of-speech distribution), positional features (e.g., location of important words, location of key sentences), and pattern features (e.g., repetitive expressions, usage patterns of specific phrases) of a review response. These feature vectors are generated through a process of converting raw text data into a numerical form that can be processed by a neural network, and various methods such as TF-IDF, Word2Vec, and BERT may be employed to extract features.

[0050] According to one embodiment, the learning device can generate second training feature vectors based on constituent features of movement path information (e.g., shape of the path, estimation of the means of movement), length features (e.g., total travel distance, travel time, distance between each point), and pattern features (e.g., repeated visit patterns, movement patterns by day of the week / time of the week).

[0051] According to one embodiment, the learning device can obtain training outputs by applying first training feature vectors to the neural network (123). This process is also called feedforward and is a process in which input data passes through each layer of the network to generate a final prediction value. The learning device can train the review item extraction algorithm of the neural network (123) based on the training outputs and first labels. The learning device can train the review item extraction algorithm of the neural network (123) by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network (123) to minimize the training errors. This process is repeated over several epochs, and in each epoch, the entire training dataset is processed and the weights are updated.

[0052] According to one embodiment, the learning device can obtain training outputs by applying second training feature vectors to the neural network (123). The learning device can train the user stay history acquisition algorithm of the neural network (123) based on the training outputs and the second labels. The learning device can train the user stay history acquisition algorithm of the neural network (123) by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network (123) to minimize the training errors. Since movement path data has temporal continuity, a recurrent neural network (RNN) structure such as LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit) can be applied, which enables effective learning of the patterns of the sequence data.

[0054] FIG. 3 is a diagram illustrating the configuration of an artificial intelligence model according to one embodiment.

[0055] An artificial intelligence model according to one embodiment may include an input layer, a hidden layer, and an output layer. This multi-layer structure is effective for learning complex patterns and solving non-linear problems, and is utilized as a core structure, particularly in deep learning models. The input layer is a layer related to input values ​​input into the artificial intelligence model and serves to receive data coming from the outside. As illustrated in FIG. 3, the input layer consists of three nodes, and each node may represent a specific feature or attribute of the data. The number of nodes in the input layer may vary depending on the dimension or characteristics of the data to be processed, and may be determined by the number of pixels in the case of image data, the dimension of the word vector in the case of text data, etc.

[0056] According to one embodiment, a feature map can be output by performing a MAC (multiply-accumulate) operation and an activation operation on the input values ​​in the hidden layer. In FIG. 3, the hidden layer consists of four nodes, and these nodes are connected to all nodes of the input layer to form a fully connected structure. The MAC operation may be an operation that multiplies the input values ​​by their corresponding weights and sums the multiplied values, and this acts as the basic unit of information processing in a neural network.

[0057] According to one embodiment, the activation operation may be an operation that inputs the result of a MAC operation into an activation function and outputs a result value. The activation function is a key element that enables a model to learn complex patterns by introducing non-linearity into the result of a linear operation. The activation function may include various types. For example, the activation function may include a sigmoid function (which outputs a value between 0 and 1 and is useful for expressing probability), a tangent function (which outputs a value between -1 and 1 and is suitable for balanced data), a ReLU function (which outputs 0 for negative inputs and outputs it as is for positive inputs, mitigating the vanishing gradient problem), a Leaky ReLU function (which applies a linear function with a small gradient to negative inputs), a MaxOut function (which selects the maximum value among several linear functions), and / or an ELU function (which is an exponential linear unit function that provides a smooth saturation curve for negative inputs), but there are no limitations on the types.

[0058] According to one embodiment, the hidden layer may be composed of at least one layer, and the more layers there are, the more complex patterns can be learned, resulting in a deep learning structure. For example, when the hidden layer is composed of a first hidden layer and a second hidden layer, the first hidden layer performs MAC operations and activation operations based on the input value of the input layer to output a feature map, and the feature map, which is the result value from the first hidden layer, can become the input value for the second hidden layer. The feature map is an intermediate representation representing features extracted from input data; in image processing, it can represent visual features such as edges, textures, and shapes, and in text processing, it can represent semantic patterns or syntactic structures. The second hidden layer can perform MAC operations and activation operations based on the feature map, which is the result value of the first hidden layer, and through this hierarchical processing, increasingly abstract and high-dimensional features can be extracted.

[0059] According to one embodiment, the output layer may be a layer related to the result of an operation performed in the hidden layer, and in FIG. 3, it is composed of three nodes. The number of nodes in the output layer varies depending on the type of problem to be solved, and may be one for a binary classification problem, one for a multi-class classification problem, one for the number of classes, one for a regression problem, one for the dimension of the value to be predicted, and one for a sequence generation problem, one for the length of the output sequence.

[0060] In one embodiment, the learning model learns syllable (character) patterns that are frequently combined and used in a given corpus to automatically learn the boundaries of compound words and named entities, integrates object information from a first UI source with object information rendered in a browser to create a learning object information file, and uses the learning object information file to generate training data for training a deep learning network. In this process, natural language processing (NLP) technology is utilized, and architectures such as word embedding, recurrent neural networks (RNN), and Transformers may be applied. Additionally, the learning model receives data from various domains of a support system, standardizes the data from the various domains into an integrated format based on at least one standardization method corresponding to each of the various domains, learns and infers data from a specific domain, determines information to be transmitted for standardization from the data of the specific domain, and performs post-processing on the data from the various domains.

[0061] According to one embodiment, the first UI source includes an XML file, and the training object information file includes an input JSON file for feature learning and an output JSON file that serves as label data during training. This structured data format allows the model to process it easily and is particularly useful for analyzing UI elements of web-based applications. The output JSON file includes a file containing HTML DOM Tree information implemented in compliance with web standards, and the various domains include at least one of a RAN (radio access network), a transport, or a core. A RAN refers to a wireless access network where a mobile device connects to a cellular network; a transport refers to a network layer responsible for data transmission; and a core refers to the central part of a network responsible for central processing and routing. Post-processing may include a correlation function, which plays an important role in analyzing relationships between data collected from various domains and discovering meaningful patterns. Through the analysis of correlations, the model can identify the causes of anomalies or performance degradation occurring in complex systems and improve prediction accuracy.

[0063] FIG. 4 is a block diagram showing the configuration of a metadata-based user-participatory label quality evaluation and fish species identification artificial intelligence model continuous learning system according to one embodiment.

[0064] A system (400) according to one embodiment may include a processor (420) and memory (430), and some of the illustrated components may be omitted or substituted. Although the system diagram briefly depicts only the core components, the actual implementation may include various additional components such as an input / output controller, a system bus, a graphics processing unit (GPU), a network interface card (NIC), and a storage device controller. A system (400) according to one embodiment may be a server or a terminal; if implemented as a server, it may be operated in a virtualized environment as part of a cloud infrastructure or implemented as a physical hardware server, and if implemented as a terminal, it may be implemented in various forms such as a desktop computer, laptop, tablet, smartphone, embedded system, or IoT device. According to one embodiment, the processor (420) is a component capable of performing operations or data processing regarding the control and / or communication of each component of the system (400), and may be composed of one or more processors. In modern processor architectures, multi-core designs are common, which improve parallel processing performance by integrating multiple independent processor cores within a single chip. For example, configurations such as dual-core, quad-core, and octa-core are possible, and each core has independent cache memory (L1, L2), and some cache (L3) can be shared among cores.

[0065] The memory (430) can store information related to the method described above or a program in which the method described above is implemented. The memory subsystem is designed with a hierarchical structure and is composed of several levels according to access speed and capacity. The memory (430) may be volatile memory or non-volatile memory. Volatile memory is memory in which stored data is lost when the power is turned off, and is mainly implemented in the form of DRAM (Dynamic RAM) or SRAM (Static RAM). DRAM stores each bit in the form of charge in a capacitor and requires periodic refreshing, whereas SRAM uses flip-flop circuits to maintain data, so it does not require refreshing but has higher cost and power consumption due to the use of more transistors. The memory (430) can store various file data, and the stored file data can be updated according to the operation of the processor (420). The memory management unit (MMU) is responsible for converting virtual memory addresses to physical memory addresses, managing page tables, and maintaining memory protection and cache consistency.

[0066] According to one embodiment, the processor (420) can execute a program and control the device (400). The processor can operate through basic pipeline stages of instruction fetch, decode, execute, and write-back. The code of the program executed by the processor (420) can be stored in memory (430). In addition to the application, the operating system, device driver, middleware, system services, etc. are loaded into memory and executed, and these provide basic system functions such as hardware resource management, process scheduling, interrupt handling, and file system management. The operations of the processor (420) can be performed by loading instructions stored in memory (430). In this process, the program counter (PC) points to the address of the next instruction to be executed, the instruction register (IR) stores the instruction currently being executed, and various general-purpose registers store operation data and intermediate results. The system (400) can be connected to an external device (e.g., a personal computer or a network) through an input / output device (not shown in the drawing) and exchange data. The input / output system can operate on an interrupt-based or polling-based basis and can transfer data directly between memory and the input / output device without the intervention of a processor through a Direct Memory Access (DMA) controller.

[0067] According to one embodiment, data exchange between memory (430) and processor (420) is performed via a system bus, which consists of an address bus, a data bus, and a control bus. The address bus specifies the memory location that the processor intends to access, the data bus transmits actual data, and the control bus transmits control information such as read / write signals. The system (400) may be operated under an operating system that supports advanced features such as multitasking, virtual memory, memory protection, and separation of privileges, and if a real-time operating system (RTOS) is used, it may satisfy deterministic response times and strict time constraints. For artificial intelligence applications, the memory (430) stores a neural network model, weight parameters, a training dataset, intermediate calculation results, etc., and the processor (420) may perform inference and learning by executing forward propagation and backpropagation algorithms. The system (400) can ensure high availability and fault tolerance by applying technologies such as clustering, load balancing, and failover mechanisms for scalability, and can implement technologies such as data encryption, access control, secure booting, and memory protection for security.

[0069] FIG. 5 is a flowchart illustrating a method for continuous learning of a metadata-based user-participatory label quality evaluation and fish species identification artificial intelligence model according to one embodiment.

[0070] The operations described through FIG. 5 can be implemented based on instructions that can be stored in a computer recording medium or memory (e.g., memory (430) of FIG. 4). The order of each operation of FIG. 5 may be changed, some operations may be omitted, and some operations may be performed simultaneously.

[0071] In operation 510, the system (400) can acquire a fishing photo from a user terminal and perform initial classification using a pre-trained fish species classification model. The system (400) can receive a fishing image uploaded by the user via a web page or mobile application, store it in an image storage, and assign a unique image identifier. The system (400) can input the stored image into a pre-trained deep learning-based fish species classification model to calculate multiple fish species candidates and prediction probabilities for each candidate. The system (400) can utilize object detection algorithms, such as YOLO or Mask R-CNN, to extract morphological features of fish species within the image and perform classification. The system (400) can provide the user with various options by generating not only the TOP-1 prediction result, which is predicted to have the highest probability of being the correct answer, but also a TOP-N candidate list in order of probability. Through this initial classification process, the system (400) can obtain a basic prediction result that serves as a reference point for user modification.

[0072] In operation 520, the system (400) can display prediction results and a list of candidates, receive user confirmation, selection, and direct input, and collect event information. The system (400) can display the TOP-1 prediction result most prominently through a web interface module and provide the TOP-N candidate fish species below it in the form of a list sorted by probability. The system (400) can provide a user interface to perform any one of the following actions: clicking a confirmation button when the user determines that the prediction result is accurate, selecting and modifying another candidate fish species, or searching for and directly entering a fish species name not in the candidate list. The system (400) can collect detailed event information in real time, including dwell time (the time the user stayed on the page), the number of times the image was zoomed in or out, and the number of times the fish species information popup was checked. The system (400) can generate all collected event information into a structured log record including a user identifier, image identifier, AI prediction label, user modification label, action type, and timestamp, and store it in an event log database. Through this detailed event collection, the system (400) can evaluate the reliability of the user's labeling.

[0073] In operation 530, the system (400) can periodically scan user action logs after generating them to evaluate and refine label quality based on user reliability and consensus. The system (400) can scan the event log database at regular intervals through a label quality evaluation module to extract newly collected user modification history. The system (400) can update the user reliability score by calculating the concordance rate between the past labeling results for pre-prepared verified data and the actual correct answer for each user. If multiple users modify labels for the same image or the same fish species, the system (400) can calculate the consensus as the ratio of the total number of modifications to the number of identical label modifications by applying the reliability of each user as a weight. The system (400) can calculate model reliability based on model prediction probability, environmental suitability based on fish species appearance probability according to location and season, cluster consistency based on reporting frequency of the same cell and similar time zones, and outlier detection score based on a combination of image quality and metadata.

[0074] According to one embodiment, the system (400) can calculate an overall quality score by performing a weighted sum operation by applying weights that are preset or determined through meta-model learning to each calculated score. The system (400) determines a final label candidate by comprehensively considering the overall quality score, the consensus, and the initial prediction probability, and can adopt the corresponding label as a refined label only if the consensus is above a preset threshold value. The system (400) can detect abnormal patterns, such as users who click multiple images quickly in too short a time or users who always use the same label, and assign a low weight to or filter out the corresponding label. Through this multi-layered quality evaluation mechanism, the system (400) can effectively remove noise from individual user inputs and select only highly reliable refined labels.

[0075] In operation 540, the system (400) can select refined labeled images to generate a training data set and perform model retraining or fine-tuning. The system (400) can select only refined images with confirmed labels through a training data generation module, convert the image data into a tensor structure corresponding to the input layer dimensions of a fish species classification model, and normalize pixel values. A tensor is a basic data structure that represents data and enables a computer to understand it, and can be used to represent inputs, outputs, and parameters in the field of deep learning. The structure of a tensor is a multidimensional array and can refer to a structure that extends dimensions, such as a 0-dimensional scalar (a single number), a 1-dimensional vector (an array of numbers), or a 2-dimensional matrix (an array of numbers consisting of rows and columns).

[0076] According to one embodiment, the system (400) can generate a training data set by converting the final confirmed label into a one-hot encoding or integer encoding form and combining it with image data. One-hot encoding may refer to a method of converting categorical data into a numeric vector that machine learning can understand. Integer encoding may refer to a method of performing encoding that assigns an integer to a word. The system (400) may trigger a retraining trigger when the accumulated amount of refined training data exceeds a set threshold, or when the user modification rate within a recent period exceeds a preset threshold, thereby determining that model performance has declined. In response to the retraining trigger, the system (400) may update the model by selecting either a transfer learning method that uses existing model weights as initial values ​​or a full retraining method that targets the entire data at regular intervals. The system (400) may deploy the newly trained model to a sandbox environment separated from the actual service environment and verify whether accuracy has improved by replicating actual incoming traffic and comparing the prediction results with the actual user modification values. The system (400) can safely replace the existing model with the new model by routing only a set portion of the total traffic to the verified new model and gradually increasing the ratio while monitoring for errors.

[0077] According to one embodiment, the system (400) may acquire and store an image including a fishing photo from a user terminal, and perform initial classification by calculating fish species candidates and prediction probabilities for the input image using a pre-trained fish species classification model. The system (400) may provide a user interface that displays a list of top prediction results and candidates based on the prediction probabilities to the user, and receives confirmation, candidate selection, or direct input from the user regarding the prediction results. The system (400) may generate a user action log by collecting event information including the type of user input, dwell time, and the number of zooms in and outs of user inputs occurring in the user interface. The system (400) may scan the user action log at regular intervals to collect new user modification history, and evaluate and refine label quality by analyzing metadata including user reliability based on the labeling match rate for previously verified data and the degree of agreement on modification results by multiple users. The system (400) generates a training data set by selecting refined labeled images and can update the model by retraining or fine-tuning the fish species classification model according to the cumulative amount of training data or recent model performance indicators.

[0078] According to one embodiment, the system (400) may define metadata including model prediction information for the initial classification result, location information, time information, marine environment information, user information, image quality information, and statistical consistency information with surrounding data. The system (400) may calculate a model reliability score based on the prediction probability or entropy of the initial classification, an environmental suitability score (Q_env) based on the distribution of fish species appearance probability according to environmental variables, a user-based score (Q_user) combining user reliability, dwell time, and number of magnifications, a cluster consistency score (Q_cluster) compared with the fish species distribution reported in the same cell and similar time zones, and an anomaly detection score (Q_anomaly) that detects abnormal combinations of image quality and metadata. The system (400) may determine the reliability of the label by calculating a comprehensive quality score (Q) as a weighted sum of each calculated score. The system (400) may adjust the reliability of the input by considering the complexity of the action when the user directly inputs and modifies a fish species name that is not in the candidate list. The system (400) can calculate the degree of agreement as the ratio of the number of times the same label is modified to the total number of modifications, using the user reliability of each user as a weight when multiple users modify a specific image or a specific fish species with the same label. The system (400) can adopt the corresponding label as a refined label and reflect it in the generation of training data only when the degree of agreement is above a set standard.

[0079] According to one embodiment, the system (400) may define metadata including model prediction information for the initial classification result, location information, time information, marine environment information, user information, image quality information, and statistical consistency information with surrounding data. The model prediction information may include probability values ​​for each fish species output by the fish species classification model and entropy representing the flatness of the overall probability distribution. The location information may be expressed as cell identifiers that divide the GPS coordinates of the fishing point into a grid of a preset size, and the time information may be used for seasonality and diurnal pattern analysis by classifying the shooting time into year, month, day, and hour units. The marine environment information may consist of physical environment variables including water temperature, salinity, water depth, and current speed at the corresponding location and time. The user information may include a unique identifier for an individual user and a confidence score reflecting past labeling accuracy. The image quality information may include at least one of a blur index representing resolution and sharpness or an exposure index representing brightness distribution. The statistical consistency information with surrounding data may indicate similarity with fish species distribution patterns reported by other users in the same grid cell and similar time periods.

[0080] According to one embodiment, the system (400) can calculate a model confidence score based on the prediction probability or entropy of the initial classification. A higher prediction probability may indicate that the model has high confidence in the corresponding classification result, and this can be normalized to a value between 0 and 1 and used directly as a confidence score. Entropy is an indicator representing the uncertainty of a probability distribution, and it may have a higher value as the prediction probability for all fish species is similarly distributed. The system (400) can use the value obtained by dividing the entropy by the maximum entropy value and subtracting it from 1 as the confidence score, thereby converting it so that the lower the entropy, the higher the confidence. The system (400) defines a case where the number of samples of a specific fish species within the training data set is less than a preset ratio relative to the total data as a minority class, and can adjust the model confidence score for that fish species by multiplying it by an imbalance correction factor greater than 1. This correction can mitigate the problem of the model's prediction probability for the minority class being lowered due to data imbalance.

[0081] According to one embodiment, the system (400) can calculate an environmental suitability score based on the probability distribution of fish species appearance according to environmental variables. The system (400) can statistically analyze a large amount of past fishing data to calculate the frequency of appearance of each fish species under specific location, season, water temperature, and water depth conditions, convert this into a probability distribution, and store it in a database. The system (400) can match the fish species derived from the current identification result with the environmental variables included in the metadata of the corresponding image to the database to query the probability of that fish species appearing under those conditions. The system (400) can apply the queried probability value to a normalization function, such as a log transformation or a sigmoid function, to convert it into an environmental suitability score between 0 and 1. A high environmental suitability score means that the fish species is likely to appear in that environment, which can support the validity of the label, while a low score may indicate the possibility that the label is an error.

[0082] According to one embodiment, the system (400) can calculate a user-based score by combining user reliability, dwell time, and zoom count. User reliability is a value calculated by accumulating the accuracy rate of labeling performed by the user on previously verified data, and can be normalized to a value between 0 and 1. Dwell time refers to the total time the user spent on the labeling page of a specific image, and if it is too short, it may indicate the possibility that the user entered the content hastily without sufficient review. Zoom count refers to the number of times the user zoomed in on an image to check detailed features, and a higher value may indicate that the user made a careful judgment. The system (400) can define a function that evaluates the appropriateness of the behavior by receiving dwell time and zoom count as inputs, and can be implemented, for example, by imposing a penalty if the dwell time is below a set minimum value and granting bonus points if the zoom count is above a certain level. The system (400) can calculate a final user-based score by summing the value obtained by multiplying user reliability by a specific weight and the value obtained by multiplying the result of the behavior indicator function by another weight.

[0083] According to one embodiment, the system (400) can calculate a cluster consistency score by comparing it with the distribution of fish species reported in the same cell and similar time zones. The same cell may include grids adjacent to the grid to which the fishing location belongs, and the similar time zone may refer to a certain period before and after the time of shooting. The system (400) can calculate the frequency of occurrence of each fish species by aggregating the distribution of fish species reported by other users within the cell and time zone range. The system (400) can derive a cluster consistency score by calculating the proportion of the frequency of the currently identified fish species in the total number of reports and applying this to a normalization function. If the same fish species is reported repeatedly, a high score may be assigned to strengthen the validity of the corresponding label; conversely, if it is an unusual report that is completely different from surrounding data, a low score may be assigned to indicate that additional verification is required. The cluster consistency score can serve as an indicator to evaluate consistency with patterns in collective data beyond the judgment of individual users or models.

[0084] According to one embodiment, the system (400) can calculate an anomaly detection score by detecting an abnormal combination of image quality and metadata. Image quality can be quantified by factors such as resolution, blur index, noise level, and brightness distribution, and images of lower quality than a specified level may have low reliability because it is difficult to accurately identify fish species. An abnormal combination of metadata may refer, for example, cases where GPS coordinates point to land, the time of shooting is recorded as a future point in time, or the water temperature value is within a physically impossible range. The system (400) can check whether each field of metadata is within an acceptable range through rule-based inspection and determine it as an abnormal signal if it falls outside the range. The system (400) can also utilize a machine learning-based anomaly detection model to automatically identify patterns that deviate significantly from the distribution of normal data. The system (400) calculates a comprehensive anomaly detection score by combining the image quality score and the metadata consistency score, and can lower the reliability of the data by assigning a lower score as an anomaly is detected.

[0085] According to one embodiment, the system (400) can determine the reliability of a label by calculating a comprehensive quality score as a weighted sum to which weights are applied to each calculated score. The system (400) can perform a weighted sum operation by multiplying each of the model reliability score, environment suitability score, user-based score, cluster consistency score, and outlier detection score by an individual weight and then summing them all. The weights represent the importance of each score contributing to the final quality evaluation and can be determined automatically through meta-model learning using training data with separate quality labels or pre-set by an administrator. Meta-model learning may be a process of deriving the optimal weight for each score by learning past labeling data and whether the final answer is correct. The system (400) can use the comprehensive quality score calculated as a weighted sum as the overall reliability of the label, and can determine that the higher this score, the higher the probability that the corresponding label is accurate. The comprehensive quality score can mitigate the bias of a single metric and enable a more balanced quality evaluation by integrating evaluations from various perspectives.

[0086] According to one embodiment, the system (400) can adjust the reliability of the input by considering the complexity of the action when a user directly inputs a fish species name that is not in the candidate list to modify it. Inputting a fish species not in the candidate list may mean that the user entered the fish species name by using a search function or directly typing text, without being satisfied with the presented options. Since this act of direct input requires more effort and time than a simple click, it may imply that the user is confident about the fish species. However, at the same time, there may also be a possibility that the user entered it by guesswork without knowing the exact fish species name.

[0087] According to one embodiment, the system (400) can adjust the reliability by analyzing detailed actions that occur during direct input. For example, if a user checks a fish species information popup multiple times or repeatedly zooms in on an image, the reliability can be increased as it appears to be a careful judgment; conversely, if the input is completed in a very short time (e.g., 0.5 seconds), the reliability can be lowered as it is likely a mechanical response without much thought. The system (400) can perform a more sophisticated evaluation by reflecting these action complexity indicators as additional variables when calculating user-based scores.

[0088] According to one embodiment, the system (400) can calculate the degree of consensus as the ratio of the number of times the same label is modified to the total number of modifications, using the user reliability of each user as a weight, when multiple users modify a specific image or a specific fish species with the same label. The degree of consensus is an indicator representing how consistent the labeling results of multiple users are, and can reflect the reliability of collective intelligence. When multiple users modify a label for the same image identifier, the system (400) can sum the reliability scores of the users who selected the corresponding label for each label. The system (400) can calculate the degree of consensus for a specific label by dividing the weighted sum of reliability for that label by the weighted sum of reliability of all users. For example, if a total of 6 users modified it and 4 of them selected label A, the simple ratio would be 4 / 6, but the weighted ratio may differ if the reliability of each user is different. If the sum of the reliability of the 4 users who selected label A is 3.2 and the sum of the reliability of all 6 users is 4.0, the degree of consensus can be calculated as 3.2 / 4.0 = 0.8. This weighted consensus method can produce more accurate judgments than simple majority voting by giving greater weight to the opinions of users with high credibility.

[0089] According to one embodiment, the system (400) may adopt a label as a refined label and reflect it in the generation of training data only when the consensus level is above a set threshold. The consensus level threshold can be set by a system administrator according to a quality policy and is generally set to a high value between 0.7 and 0.9 to select only labels with strong consensus. The system (400) may confirm labels with a consensus level above the threshold as refined labels and include the corresponding images and labels in the training data set. If the consensus level is below the threshold, the system (400) may classify the corresponding label as pending and wait for additional user input or move it to an expert verification queue. Pending data may be re-evaluated in the future when more user input accumulates and the consensus level rises. Through such strict quality control, the system (400) can increase the learning effect and prevent model performance degradation by excluding uncertain or controversial labels from the training data and using only data with high confidence for model training.

[0091] FIG. 6 is a flowchart illustrating a metadata-based label quality evaluation method according to one embodiment.

[0092] In operation 610, the system (400) can define metadata and calculate model reliability, environmental suitability, user base, cluster consistency, and outlier detection scores. For each discrimination result, the system (400) can define metadata including model prediction information including model prediction probability and TOP-k probability distribution, location information including location cell identifier and shooting time, time information, marine environment information including water temperature and salinity, user information including user identifier and user reliability based on historically verified data matching rate, image quality information including resolution and blur index, and statistical consistency information with surrounding data including the reporting rate of the same fish species within the same cell and similar time period. The system (400) can calculate the model reliability score by normalizing the model's prediction probability or by subtracting the value obtained by dividing the prediction entropy by the maximum entropy from 1, and can adjust for minority classes by multiplying by an imbalance correction factor.

[0093] According to one embodiment, the system (400) can calculate an environmental suitability score by pre-database the probability distribution of appearance according to environmental variables such as location, season, water temperature, and water depth for each fish species, and by applying the probability derived by matching the fish species of the current identification result with metadata to a normalization function. The system (400) can calculate a user-based score through a formula that applies weights to behavioral indicator functions with user trust, dwell time, and number of zooms as variables, and can impose a penalty if the dwell time is too short and grant bonus points if the zoom and information verification behaviors are sufficient. The system (400) can calculate a cluster consistency score by applying the frequency of the fish species reported within the same location grid and season to a normalization function, and can impose a penalty in the case of an anomalous report that is completely different from surrounding data. The system (400) can calculate a low outlier detection score through an outlier detection model or rule-based detection if the image quality is extremely low or the metadata combination is abnormal.

[0094] In operation 620, the system (400) can calculate a comprehensive quality score by applying weights to each score and determine a final label by combining the consensus and prediction probabilities. The system (400) can apply weights determined through meta-model learning using preset or separate quality labels to each of the model confidence score, environment fit score, user base score, cluster consistency score, and outlier detection score. The system (400) can derive a comprehensive quality score by summing all the products of each score and its corresponding weight, which can be expressed as a normalized value between 0 and 1.

[0095] According to one embodiment, when multiple users perform a labeling action for the same image identifier, the system (400) can calculate the degree of consensus by calculating a ratio in which the sum of the user confidence scores of users who selected a specific label is divided by the sum of the user confidence scores of all users who performed the action on the image. The system (400) can determine the final label candidate by comprehensively considering the overall quality score derived through a weighted sum operation, the degree of consensus calculated by the ratio, and the prediction probability of the initial classification. The system (400) can determine the final label by preferentially selecting the label that exhibits the highest confidence among the three indicators, or by using a voting method in which separate weights are applied to each indicator.

[0096] In operation 630, the system (400) may include only labels with a consensus level greater than or equal to a threshold value as refined labels in the training data set. The system (400) may compare the previously calculated consensus level with a preset threshold value and adopt the corresponding label as a refined label only if it is greater than or equal to the threshold value. The system (400) may set the threshold value to a high level, such as 0.8 or 0.9, to select only labels with strong consensus among multiple users.

[0097] According to one embodiment, if the level of agreement is below a threshold value, the system (400) may classify the corresponding label as pending and wait for additional user input or move it to an expert verification queue. The system (400) may create a training dataset by storing metadata including image paths, final labels, label confidence, and collection times together for data confirmed with refined labels. The system (400) may convert the generated training dataset into a standard format so that it can be used directly in the model training pipeline. By applying these strict quality standards, the system (400) can increase the training effect by feeding only high-quality training data with minimized noise into the model training.

[0098] In operation 640, the system (400) can apply a minority class correction coefficient and calculate environment, user, and cluster scores based on a mathematical formula. The system (400) can define a minority class as a case where the ratio of the number of data samples of a specific fish species to the total data within the training dataset of the fish species classification model is less than a preset threshold. The system (400) can mitigate data imbalance issues by applying an imbalance correction coefficient greater than 1 to fish species corresponding to the minority class and calculating a model confidence score.

[0099] According to one embodiment, the system (400) can pre-store in a database the probability of fish species appearing by environmental variable derived from the cumulative statistical analysis results of past collected data or a marine organism habitat prediction model, and calculate an environmental suitability score through a mathematical formula that applies the probability to a normalization function by matching it with the current environmental variable. The system (400) can calculate a user-based score through a mathematical formula that sums the value obtained by multiplying user trust by a specific coefficient and the value obtained by multiplying a behavioral indicator function, which has dwell time and expansion frequency as variables, by another coefficient. The system (400) can calculate a community consistency score through a mathematical formula that applies the frequency of the fish species reported within the same location grid and season to a normalization function.

[0100] According to one embodiment, the system (400) may perform a weighted sum operation by applying weights (w_1 to w_5) determined through meta-model learning using preset or separate quality labels to each of the model reliability score (Q_model), environment suitability score (Q_env), user-based score (Q_user), cluster consistency score (Q_cluster), and anomaly detection score (Q_anomaly) to calculate the overall quality score (Q). To calculate the consensus, the system (400) may calculate a ratio by dividing the sum of the user reliability scores (U_trust) of users who selected a specific label by the sum of the user reliability scores of all users who performed an action on the image, in the case where multiple users performed a labeling action on the same image identifier (Image ID). The system (400) determines a final label candidate by combining the overall quality score (Q) derived through a weighted sum operation, the consensus calculated by ratio, and the prediction probability of the initial classification, and can confirm the corresponding label as a refined label and include it in the training data set only if the consensus is greater than or equal to a preset threshold value.

[0101] According to one embodiment, the system (400) may define a minority class as a case where the ratio of the number of data samples of a specific fish species to the total data within the training data set of a fish species classification model is less than a preset threshold. The system (400) may apply an imbalance correction coefficient to fish species corresponding to the minority class and calculate a model reliability score through [Equation 1].

[0102] [Mathematical Formula 1]

[0103] Q_model = (1 - (H / H_max)) * C_balance

[0104] (Here, Q_model is the model confidence score, H is the prediction entropy, H_max is the maximum entropy, and C_balance is an imbalance correction factor that is set to a value greater than 1 for the minority class and 1 otherwise.)

[0105] The system (400) can calculate an environmental suitability score by [Equation 2], which stores in advance in a database the probability distribution of occurrence for each fish species according to environmental variables including location, season, water temperature, and water depth, derived from the cumulative statistical analysis results of past collected data or from a marine organism habitat prediction model, and applies the probability derived by matching the fish species and metadata that are the current identification results to the database to a normalization function.

[0106] [Mathematical Formula 2]

[0107] Q_env = f(P(s* | env*))

[0108] (Here, Q_env is the environment fit score, P(s* l env*) is the probability of the occurrence of fish species s* identified in the current environment variable env*, and f is the normalization function)

[0109] The system (400) can calculate a user-based score through [Equation 3], which applies weights to each of the behavior indicator functions, such as user trust, measured dwell time and number of enlargements when performing modification actions.

[0110] [Mathematical Formula 3]

[0111] Q_user = a * U_trust + b * g(dwell_time, zoom_cnt)

[0112] (Here, Q_user is the user-based score, U_trust is the user trust, g is the behavioral metric function, dwell_time is the dwell time, zoom_cnt is the number of zooms, and a and b are weighting coefficients)

[0113] The system (400) can calculate a cluster consistency score by applying the frequency of reported fish species within the same location grid and season to a normalization function [Equation 4].

[0114] [Mathematical Formula 4]

[0115] Q_cluster = h(frequency(s*, cell, season))

[0116] (Here, Q_cluster is the cluster consistency score, frequency is the frequency calculation function, s* is the identified fish species, cell is the location grid, season is the season, and h is the normalization function)

[0117] According to one embodiment, the system (400) may define a minority class as a case where the ratio of the number of data samples of a specific fish species to the total data within the training data set of a fish species classification model is less than a preset threshold. A minority class may refer to a class where it is highly likely that the model could not accurately learn the fish species because insufficient samples were obtained from the training data.

[0118] The system (400) can classify rare fish species, for example, that appear in less than 1% of the total data, as a minority class. The model’s predicted probability for the minority class may tend to be lower than the actual value due to data imbalance, which may result in an unfairly lowered model reliability score. To correct this bias, the system (400) can calculate the model reliability score through Equation 1 by applying an imbalance correction coefficient to fish species corresponding to the minority class. The imbalance correction coefficient is set to a value greater than 1 for the minority class to increase the model reliability score, and is set to 1 for the majority class to maintain the original score without correction. The magnitude of the correction coefficient can be set in proportion to the degree of scarcity of the corresponding class, for example, and can be calculated based on the inverse of the data ratio.

[0119] According to one embodiment, in Equation 1, H represents uncertainty as the entropy of the predicted probability distribution output by the model, and H_max may be the maximum entropy value when the same probability is assigned to all classes. The entropy ratio H / H_max has a value between 0 and 1, and the value obtained by subtracting this from 1 may represent the degree of confidence. Through this correction mechanism, the system (400) can mitigate evaluation distortion caused by data imbalance and perform a fair quality evaluation even for rare fish species.

[0120] According to one embodiment, the system (400) may store in advance in a database the distribution of fish species appearance probabilities by environmental variable derived from the cumulative statistical analysis results of past collected data or from a marine organism habitat prediction model. Environmental variables may include cell identifiers converted from location coordinates into a grid, the season determined from the time of filming, and water temperature and depth information measured or estimated at the corresponding location. The system (400) may analyze long-term accumulated fishing records to aggregate the frequency of appearance of each fish species under specific environmental conditions, convert this into a probability, and store it. For example, if a specific fish species is observed with high frequency in a specific sea area at a water temperature of 15-20 degrees, a depth of 30-50 meters, and during the spring season, the probability of appearance under those conditions may be recorded as high. The system (400) may query the probability of that fish species appearing under those conditions by matching the fish species, which is the current identification result, with the environmental variables included in the metadata in the database. The system (400) may calculate an environmental suitability score through Equation 2, which applies the queried probability value to a normalization function. In mathematical formula 2, P(s*|env*) may represent the conditional probability that the identified fish species s will appear given the current environment variable env. The normalization function f serves to convert the probability value into a score between 0 and 1, and a log transformation or a sigmoid function may be used. A higher environmental suitability score can be interpreted to mean that the fish species is more likely to appear biologically in that environment. Through this environment-based verification, the system (400) can perform a comprehensive quality assessment that considers not only image similarity but also the habitat characteristics of marine organisms.

[0121] According to one embodiment, the system (400) can calculate a user-based score through Equation 3, which applies weights to behavioral indicator functions, each having user trust and the dwell time and zoom count measured when performing a modification action as variables. In Equation 3, U_trust is a trust score calculated based on the accuracy rate of labeling performed by the user on previously verified data, and may have a value between 0 and 1. The behavioral indicator function g may be a function that takes dwell time and zoom count as inputs to evaluate how carefully the user's current labeling behavior was performed. Dwell time dwell_time may be a value measured in seconds for the total time the user stayed on the labeling page of the image, and zoom count zoom_cnt may represent the number of times the user zoomed in on the image to check detailed features. The behavioral indicator function may be designed, for example, to impose a penalty if the dwell time is below a minimum threshold value, and to grant bonus points if the zoom count is above a certain level. Weight coefficients a and b may represent the relative importance of the user's past trust and current behavioral pattern contributing to the final score. For example, if a is set to be greater than b, an evaluation system can be established that places more emphasis on past reliability, and conversely, places more emphasis on current behavior. Through this dual evaluation method, the system (400) can assign a low score to a user with high reliability if they performed a specific labeling carelessly, and assign a certain level of score to a user with low reliability if they performed it very carefully.

[0122] According to one embodiment, the system (400) can calculate a cluster consistency score through Equation 4, which applies the frequency of reported fish species within the same location grid and season to a normalization function. In Equation 4, frequency is a function that calculates the frequency of fish species reports under specific conditions and can receive the identified fish species (s*), location grid (cell), and season (season) as inputs. The location grid may include the cell to which the current fishing location belongs and adjacent cells in a grid system that divides the ocean into cells of a certain size. The season may be one of spring, summer, autumn, and winter determined from the time of filming, or a more finely subdivided time range. The frequency calculation function may aggregate and return the number of times the same fish species was reported during a past period in the corresponding cell and season combination. The normalization function h serves to convert the frequency value into a score between 0 and 1, and log scaling or max value normalization methods may be used. If the same fish species is repeatedly observed in the region and time, a high frequency value is calculated, and a high cluster consistency score may be assigned. This implies that the label aligns with regional and temporal patterns, which can strengthen its validity. Conversely, in the case of an anomalous report that is completely different from surrounding data, a low score may be assigned due to the low frequency value, indicating the need for further verification.

[0123] According to one embodiment, the system (400) can classify the nature of the data by analyzing the discrepancy pattern between the environmental suitability score (Q_env) and the consensus resulting from user modifications. The system (400) can identify a data drift candidate group, which is a phenomenon where the distribution of existing training data differs from the distribution of actual field data due to changes in the marine environment or the appearance of non-existent fish species, when the visual prediction probability of the fish species classification model is less than a first threshold value and the environmental suitability score (Q_env) is calculated to be low, but the consensus exceeds a third threshold value because multiple users with a high level of user trust (U_trust) commonly modify the label to a specific fish species. For image data identified as a data drift candidate group, the system (400) can perform emergency retraining by setting the sample weight applied to the calculation of the loss function during model training higher than that of general data so that the model reflects the change preferentially. Conversely, if the visual prediction probability of the fish species classification model is higher than the fourth threshold value but the environmental suitability score (Q_env) for the location and season is lower than the fifth threshold value, which is a level where survival is impossible, the system (400) may determine this as a hallucination or anomaly in which the model has a false conviction, withhold automatic labeling confirmation, and classify it into an expert verification queue.

[0124] According to one embodiment, the system (400) can classify the nature of the data by analyzing a pattern of discrepancy between the environmental suitability score and the level of consensus resulting from the users' modifications. A pattern of discrepancy may refer to a case where a high level of consensus is observed because multiple highly reliable users commonly modify a specific fish species, even though the environmental suitability score is low and the likelihood of ecological occurrence is assessed as low, or the opposite case. Such discrepancy may represent an exceptional situation with significant meaning rather than a simple error, and the system (400) can automatically detect this and take appropriate measures.

[0125] According to one embodiment, the system (400) can identify a data drift candidate group in cases where the visual prediction probability of the fish species classification model is below a first threshold value and the environmental suitability score is calculated to be low, below a second threshold value, but multiple users with a high level of user trust commonly modify the label to a specific fish species, thereby exceeding a third threshold value of consensus. This may mean a situation where the model is not confident and past environmental data does not support the appearance of the fish species, but expert users in the field consistently judge it to be a specific fish species. This pattern may refer to a phenomenon in which fish species that did not previously appear in the area newly appear due to rising water temperatures caused by climate change, changes in ocean currents, or the influx of alien species. Data drift is a phenomenon in which the distribution of past data used for model training differs from the distribution of current data occurring in the actual field, and if left unchecked, model performance may continuously deteriorate. The system (400) can respond quickly to environmental changes by identifying such candidate groups early.

[0126] According to one embodiment, the system (400) can perform emergency retraining for image data identified as data drift candidates by setting the sample weights applied to the loss function calculation during model training higher than those for general data. The loss function is a function that quantifies the difference between the model's predicted value and the actual correct answer, and may be an objective function to be minimized during the model training process. The sample weight is a coefficient that controls the degree to which each training data contributes to the calculation of the loss function, and samples assigned high weights may have a greater impact on model training. The system (400) may assign weights to the data drift candidates, for example, between 2 and 5 times those of general data. Through this, the model can intensively train on data reflecting new environmental changes and quickly adapt to changed field conditions. Emergency retraining may refer to a retraining process that is performed immediately when a certain number of drift candidates accumulates, without waiting for a regular retraining schedule. Through this emergency response mechanism, the system (400) can maintain model performance and prevent service quality degradation even in the event of rapid environmental changes.

[0127] According to one embodiment, the system (400) may, conversely, determine a case where the visual prediction probability of the fish species classification model is high, exceeding a fourth threshold value, but the environmental suitability score for the corresponding location and season is below a fifth threshold value as a model hallucination or an outlier. A model hallucination may refer to a phenomenon where an artificial intelligence model makes a prediction with high confidence but actually outputs results that are ecologically impossible or very rare. For example, this may apply to cases where a tropical fish species is predicted with high probability in the Arctic Ocean during midwinter. This may be because the model made a wrong judgment ignoring context by overreacting to a specific visual pattern in an image. An environmental suitability score below the fifth threshold value may mean that the fish species cannot survive in that environment, for example, indicating cases where the water temperature is significantly outside the range of survival or the habitat depth is completely unsuitable. In such cases, the system (400) may not trust the high prediction probability of the model and may determine it as a hallucination or an outlier, thereby withholding automatic labeling confirmation. The system (400) may classify the data into an expert verification queue so that fisheries experts or marine biologists can review it directly. The expert verification queue may be a queue structure that separately manages exceptional cases that are difficult to handle with an automated process, and the data included therein may be adopted as training data only after the expert's judgment is completed. Through this verification process, the system (400) can prevent erroneous data caused by the model's false confidence from being included in the training and maintain the reliability of the data quality.

[0128] According to one embodiment, data nature classification through discrepancy pattern analysis may be a technique that interprets the relative relationships and combinations between various indicators, rather than simply evaluating the high or low scores. The system (400) can capture important situations that might be missed with a single indicator alone by cross-validating signals from different sources, such as environmental suitability scores and consensus. Identifying data drift candidates acts as an early warning system for environmental changes to enhance the adaptability of the model, and hallucination detection acts as a safeguard to prevent overconfidence in the model to strengthen quality control. Through this dual analysis system, the system (400) can perform intelligent data management that simultaneously considers the changing field environment and the limitations of the model.

[0130] FIG. 7 is a flowchart illustrating a model distribution and user reliability management method according to one embodiment.

[0131] In operation 710, the system (400) can deploy a new model to a sandbox environment and verify the predicted value by replicating actual traffic. The system (400) can deploy a new model generated through retraining to a sandbox environment, which is a virtual verification space isolated from the actual service environment. The system (400) can replicate user request traffic flowing into the actual service in real time and provide the same image as input to the new model in the sandbox environment. The system (400) can verify performance without affecting the actual service by storing the prediction results of the new model only internally without returning them to the user. The system (400) can compare and analyze the predicted values ​​of the new model and the existing model collected over a certain period and track whether user modifications occurred for each prediction. The system (400) calculates the accuracy of each model by calculating the ratio of accurate predictions where no user modifications occurred, and can proceed to the next step only if the accuracy of the new model is statistically significantly higher than that of the existing model. Through this sandbox verification process, the system (400) can objectively evaluate the actual performance of the new model without compromising service stability.

[0132] In operation 720, the system (400) can verify whether accuracy has improved by comparing the new model's predicted value with the user's modified value. The system (400) can compare and analyze the prediction results of the new model collected in the sandbox environment with the actual user's final modified value for the corresponding image by matching them one-to-one. The system (400) determines the case where the new model's TOP-1 prediction matches the user's final label as the correct answer, and calculates the accuracy by calculating the ratio of the correct answer to the total verification samples.

[0133] According to one embodiment, the system (400) calculates the accuracy of the existing model in the same manner and determines that the verification is passed only when the difference in accuracy between the new model and the existing model is greater than or equal to a preset improvement threshold. The system (400) can analyze not only the overall accuracy but also detailed accuracy by fish species, image quality, and shooting environment to verify whether performance degradation occurs under specific conditions. The system (400) can record the verification results as a log and send a report to an administrator to obtain final approval for the model update. Through this systematic verification procedure, the system (400) can clearly confirm the performance improvement of the new model before actual deployment and minimize deployment risks.

[0134] In operation 730, the system (400) may route only a portion of traffic to the verified new model and monitor for errors. The system (400) may deploy the verified new model to the actual service environment, but initially route only a small amount of the total user traffic, such as 5% or 10%, to the new model. The system (400) may control traffic splitting at the load balancer or API gateway level so that some user requests are delivered to the new model and the rest to the existing model.

[0135] According to one embodiment, the system (400) can collect and monitor key performance indicators in real time, including response time, whether an error occurred, and the rate of user modification, for requests routed to a new model. If the error rate in the new model exceeds a set threshold or the response time becomes slower than a standard, the system (400) can immediately stop routing to that model and return all traffic to the existing model. If the stability of the new model is confirmed for a certain period, the system (400) can increase traffic to the next stage.

[0136] In operation 740, the system (400) can gradually increase the traffic ratio and gradually replace the existing model with a new model. The system (400) can gradually increase the traffic ratio routed to the new model after stability is confirmed during the initial deployment phase. The system (400) can gradually increase the ratio, for example, 5% -> 10% -> 25% -> 100%, and monitor performance and stability for a certain period at each stage. The mentioned ratios are merely examples and may vary depending on the settings.

[0137] According to one embodiment, the system (400) can compare and analyze the prediction accuracy, response time, system resource usage, and user modification rate of the new model with that of the existing model. If the system (400) confirms that all indicators are within the normal range and there is no performance degradation compared to the existing model, it can proceed with the next stage of traffic increase. When the new model finally handles 100% of the traffic, the system (400) can keep the existing model for backup or completely remove it. The system (400) can log the entire replacement process and visualize performance indicators for each stage on a dashboard so that an administrator can monitor them in real time. Through this phased replacement method, the system (400) can prevent a degradation in service quality caused by sudden model changes and safely complete the model update.

[0138] In operation 750, the system (400) can promote verified existing data and cross-verify user reliability in real time. The system (400) can present to the user an image of verified existing data, for which a correct answer label has already been secured, by inserting it at a random time into an image queue that the user is to identify. The system (400) can grant bonus points to the user's reliability by applying a weight proportional to the number of consecutive correct answers when the modification action entered by the user for the image promoted to verified data matches the pre-set correct answer. Conversely, the system (400) can update the reliability score in real time by granting a penalty when the user's input does not match the correct answer.

[0139] According to one embodiment, the system (400) can strengthen the verification intensity by gradually increasing the proportion of verified data images among all images presented to the user when the updated user trust level falls below a preset first threshold. The system (400) may impose a penalty that temporarily restricts the user's labeling authority or invalidates the validity of previously entered data if the user fails verification more than a certain number of times. Conversely, for excellent users whose updated user trust level exceeds a preset second threshold, the system (400) may increase the frequency of exposure by preferentially allocating images with high uncertainty due to a flat prediction probability distribution of the fish species classification model, or images presumed to be rare fish species with insufficient data samples.

[0140] According to one embodiment, the system (400) may provide differential points or grade experience points in proportion to the value evaluation score calculated by multiplying the level of model performance improvement based on the amount of uncertainty reduction of the image and the scarcity weight of the fish species when the labeling results performed by an excellent user are finally adopted as training data. The system (400) may provide additional rewards based on traffic generated whenever the data is utilized for verification when the image labeled by the excellent user is promoted to verified data for evaluating the reliability of other users. Through this real-time cross-verification and differential reward system, the system (400) can continuously manage the quality of users' labeling and provide incentives for providing high-quality labels, thereby improving the overall quality of the training data.

[0141] According to one embodiment, the system (400) may select images with refined labels, convert the image data into a tensor structure corresponding to the input layer dimension of a fish species classification model, normalize pixel values, and convert the final confirmed labels into one-hot encoding or integer encoding forms to create a combined training data set. The system (400) may analyze logs collected during real-time service operation to calculate a User Correction Rate, which is the ratio of the number of cases where a user performs a correction action to the total number of predictions, and may trigger a retraining trigger by determining that the user correction rate within a recent period exceeds a preset threshold as a decline in model performance. In response to the retraining trigger, the system (400) may deploy the newly created model to a sandbox environment, which is a virtual verification space separated from the actual service environment, while controlling the model so that the prediction results are not returned to the user by replicating the actual incoming user traffic and inputting it into the new model, and verify whether the accuracy has improved by comparing the prediction values ​​of the new model with the actual user correction values. For a new model that has been verified, the system (400) routes only a set portion of the total user traffic to the new model and processes the remaining traffic with the existing model, and while monitoring for errors, gradually increases the ratio to replace the existing model with the new model.

[0142] According to one embodiment, the system (400) may select refined labeled images, convert the image data into a tensor structure corresponding to the input layer dimension of a fish species classification model, normalize pixel values, and convert the final confirmed label into a one-hot encoding or integer encoding form to generate a combined training data set. A tensor is a data structure in the form of a multidimensional array and may be a standard input format that a deep learning model can process. The system (400) may convert the image into a 4-dimensional tensor in the form of, for example, [batch size, height, width, channels], where the channels may be three dimensions representing RGB color information. The input layer dimension is a specific image size required by the fish species classification model, for example, 224x224 or 512x512 pixels, and the original image may be resized to fit this size. Pixel value normalization may refer to a standardization process of scaling the pixel value range of the original image from 0-255 to between 0-1, or subtracting the mean and dividing by the standard deviation. Normalization can have the effect of increasing the convergence speed of model training and improving stability. One-hot encoding is a method of converting labels into a vector form, for example, when there are 50 types of fish, they can be represented as a vector of length 50 where only the positions corresponding to a specific fish type are 1 and the rest are 0. Integer encoding may be a method of representing labels as a single integer value by assigning a unique integer identifier to each fish type. The system (400) can generate a training dataset by combining the converted image tensor and the encoded label in pairs, which may be a standardized format that the model can directly use for training.

[0143] According to one embodiment, the system (400) can calculate a user modification rate, which is the ratio of the number of cases where a user performs a modification action to the total number of predictions, by analyzing logs collected during real-time service operation. The user modification rate refers to the percentage of cases where a user does not agree with the prediction results presented by the model and modifies them, and this may be an indicator that indirectly indicates the actual accuracy of the model. For example, if a total of 1,000 predictions were made during the last 24 hours and users modified 150 of them, the system (400) can calculate the modification rate as 15%. The system (400) may determine that the model performance has deteriorated if the user modification rate within a recent period exceeds a preset threshold. The threshold can be set by the system administrator according to service quality goals and may be set, for example, to 20% or 25%. An increase in the modification rate to the threshold means that the model's prediction does not match the field data and frequent user intervention is required, which may indicate performance degradation due to data drift or model aging. When the system (400) detects such performance degradation signals, it can automatically trigger a retraining process to start the model update process. Through this, the system (400) can detect performance degradation early and respond quickly to maintain service quality.

[0144] According to one embodiment, the system (400) can deploy a new model generated in response to a retraining trigger into a sandbox environment, while replicating actual incoming user traffic and inputting it into the new model, and controlling the system so as not to return the prediction result to the user. The sandbox environment is an independent test space physically or logically isolated from the actual service environment, and may be a safe verification environment that does not affect actual users even when a new model is deployed.

[0145] According to one embodiment, the system (400) can replicate user requests entering the actual service in real time to provide the same input to both the existing model and the new model. The prediction results of the existing model are normally returned to the user so that the service continues without interruption, while the prediction results of the new model are recorded internally only and may not be delivered to the user. The system (400) can calculate accuracy by comparing the prediction values ​​of the new model accumulated over a certain period with the final correction values ​​of the actual user regarding those predictions. If the ratio of the new model's TOP-1 prediction matching the user's final confirmed label is statistically significantly higher than that of the existing model, it can be determined that performance improvement has been verified. The system (400) can also perform comprehensive verification by monitoring non-functional indicators such as the response time, resource usage, and stability of the new model. Through this sandbox verification method, the system (400) can perform realistic performance evaluation based on actual traffic without affecting the actual user experience at all.

[0146] According to one embodiment, the system (400) routes only a set portion of the total user traffic to the new model for which verification has been completed and processes the remaining traffic to the existing model, and can replace the existing model with the new model by gradually increasing the ratio while monitoring for errors. Traffic routing is a process of distributing incoming user requests to different model endpoints and can be implemented at the level of a load balancer or API gateway. The system (400) can be configured to initially route, for example, only 5% of the traffic to the new model and process the remaining 95% to the existing model.

[0147] According to one embodiment, the system (400) can collect and monitor performance indicators, including response time, error rate, and user modification rate, in real time for requests routed to the new model. If the performance of the new model remains stable for a certain period and shows indicators equivalent to or better than those of the existing model, the system (400) can verify stability at each stage by gradually increasing the routing ratio. Conversely, if problems occur in the new model, such as a sudden surge in the error rate or a response time exceeding an acceptable range, the system (400) can immediately stop routing and return all traffic to the existing model. Finally, when it is confirmed that the new model handles 100% of the traffic stably, the existing model can be kept for backup or completely removed.

[0148] According to one embodiment, the system (400) may present to the user an image of verified existing data, for which a correct answer label has already been secured, by inserting it at a random time into an image queue to be determined by the user in order to cross-verify the user's labeling reliability in real time. The system (400) may update the reliability score by applying a weight proportional to the number of consecutive correct answers to the user's reliability if the modification action entered by the user for the presented verified existing data image matches the preset correct answer, and by applying a penalty if it does not match. If the updated user reliability falls below a preset first threshold, the system (400) may strengthen the verification intensity by gradually increasing the proportion of verified existing data images among all images presented to the user, and may impose a penalty such as restricting the user's labeling authority or invalidating the validity of the previously entered data if the verification fails more than a certain number of times.

[0149] According to one embodiment, the system (400) may increase the frequency of exposure by preferentially assigning images that are presumed to be rare fish species with a lack of data samples or images with high uncertainty due to a flat prediction probability distribution of the fish species classification model, in the case of a superior user whose updated user reliability exceeds a preset second threshold. When the labeling results performed by the superior user are finally adopted as training data, the system (400) may award differential points or grade experience points in proportion to the value evaluation score calculated by multiplying the level of model performance improvement based on the amount of uncertainty reduction of the image and the scarcity weight of the fish species. When the image labeled by the superior user is subsequently promoted to verified data for evaluating the reliability of other users, the system (400) may award additional rewards based on the traffic generated whenever the data is utilized for verification.

[0150] According to one embodiment, the system (400) may present to the user an image of verified existing data, for which a correct label has already been secured, by inserting it at a random time into an image queue to be identified by the user in order to cross-verify the user's labeling reliability in real time. The verified existing data is data for which a label has been confirmed by an expert or recognized as the correct answer by assigning a matching label to multiple high-reliability users, and can be set as 'verified data'. The image queue refers to a queue of images to be presented to the user sequentially and can generally be operated on a first-in, first-out basis. The system (400) may randomly mix and insert verified images along with general user images into this queue, and the process can be transparent so that the user cannot know which images are for verification. By inserting at random times, the system (400) can induce continuous and consistent quality labeling by preventing the user from predicting verification patterns or focusing only on specific times. The system (400) may, for example, configure about 10% to 20% of the total images as verification images, and this ratio may be dynamically adjusted according to the user's confidence level. Through this cross-verification mechanism, the system (400) can objectively evaluate the user's current labeling capabilities and continuously update the confidence score.

[0151] According to one embodiment, the system (400) can update the reliability score by adding points to the user's reliability if the modification action entered by the user for the presented verified existing data image matches the preset correct answer, and by deducting points if it does not match. The system (400) may not simply determine whether the answer is correct, but may apply a weight proportional to the number of consecutive correct answers. Consecutive correct answers refer to the number of times the correct answer is given consecutively without incorrect answers to the verification image, and this can be used to verify the user's consistent judgment ability. The system (400) may provide greater rewards for superior performance by, for example, applying a weight of 1.5 times the basic bonus points for 5 consecutive correct answers and 2 times the basic bonus points for 10 consecutive correct answers.

[0152] Conversely, if an incorrect answer occurs, the consecutive correct answer count is reset, and a certain score may be deducted from the reliability. The size of the deduction may be applied differentially depending on the severity of the incorrect answer; for example, a large deduction may be applied if a taxonomically completely different species is selected, while a small deduction may be applied if similar species are confused. The system (400) reflects the updated reliability score in the database in real time and can immediately apply it to the quality evaluation of all labeling performed by the user thereafter. Through this real-time update method, the system (400) can sensitively detect changes in the user's capabilities and respond dynamically.

[0153] According to one embodiment, the system (400) may strengthen the verification intensity by gradually increasing the proportion of verified existing data images among all images presented to the user when the updated user confidence falls below a preset first threshold. The first threshold serves as a baseline to ensure minimum labeling quality and may be set, for example, to 0.6 or 0.7. A decrease in confidence below this threshold may mean that the user's labeling accuracy has fallen below an acceptable level. The system (400) may gradually increase the proportion of images for verification for such users, for example, from a base 10% to 20%, 30%, and so on. Strengthening the verification intensity may have the effect of more strictly monitoring the user's labeling and preventing incorrect labels from being included in the training data. The system (400) may impose a stronger penalty if verification fails more than a certain number of times. Restricting labeling authority may mean a measure to temporarily or permanently block the user from performing labeling on new images. Invalidation of previously entered data may mean re-evaluating the reliability of labels previously entered by the user and excluding them from the consensus calculation or significantly lowering their weight. Through this multi-layered penalty system, the system (400) can suppress low-quality labeling and protect overall data quality.

[0154] According to one embodiment, the system (400) may preferentially assign images that are estimated to be rare fish species with a lack of data samples or images with high uncertainty due to a flat prediction probability distribution of the fish species classification model to excellent users whose updated user confidence exceeds a preset second threshold. The second threshold is a criterion for recognizing excellent labeling ability and may be set, for example, to 0.85 or 0.9. A flat prediction probability distribution may mean a state in which the model assigns similar levels of low probability to multiple fish species, resulting in uncertainty about any of them, which may manifest as a high entropy value. These images are high-difficulty cases that are difficult to distinguish, and labeling by a user with high expertise may be particularly important. Rare fish species are those with a very small number of samples in the training data set, and securing additional data can be of great help in improving model performance. By preferentially exposing high-difficulty images that are difficult to distinguish to excellent users, the system (400) can efficiently utilize limited human resources and intensively reinforce weaknesses in the training data. Increasing exposure frequency can be implemented by adjusting priorities in the image queue so that the corresponding images are assigned more frequently to premium users.

[0155] According to one embodiment, the system (400) may award differential points or grade experience points in proportion to the value assessment score calculated by multiplying the level of model performance improvement based on the amount of uncertainty reduction of the image and the scarcity weight of the fish species when the labeling result performed by an excellent user is finally adopted as training data. The amount of uncertainty reduction may be a value that quantifies how much the entropy or consensus variance of the label for the image has decreased before and after the user's labeling. The level of model performance improvement may represent the degree of improvement in model accuracy expected by adding the label to the training data and may be measured as an information gain or a reduction in the loss function. The scarcity weight is set in proportion to the degree of data scarcity of the fish species, and a higher weight may be assigned as the scarcer it is.

[0156] According to one embodiment, the system (400) may calculate a comprehensive value assessment score by multiplying three factors (e.g., amount of uncertainty reduction, level of model performance improvement, and scarcity weight of the fish species) and award points or experience points in proportion to this. Points may be virtual currency that can be converted into benefits when using the service in the future, and grade experience points may be used to increase the user's level to grant more authority or honor. Through a differential reward system, the system (400) may motivate excellent users to focus on generating high-value data rather than simple repetitive tasks, and increase the speed of overall data quality and model performance improvement.

[0157] According to one embodiment, the system (400) may provide additional rewards whenever the data is used for verification when the image labeled by the excellent user is promoted to verified data for evaluating the reliability of other users. Promotion to verified data may mean that the label has been recognized for its very high reliability and has become verified data that can be used as a standard for verifying the quality of other users' labeling. The system (400) may track the number of times the promoted data is actually presented to other users for verification and used for evaluating the reliability of those users. Traffic-based rewards are paid in proportion to the frequency of use of the verified data, and for example, if it is presented to 100 users for verification, a corresponding reward may be accumulated.

[0159] According to one embodiment, the system (400) may construct a biological classification database that stores hierarchical phylogenetic classification information including the order, family, genus, and species of fish. When a user modifies a first fish species predicted by a fish species classification model to a second fish species, the system (400) may refer to the database to identify the lowest common hierarchical level at which the first fish species and the second fish species are taxonomically separated, and calculate the taxonomic distance between the two fish species based on the weight assigned to that hierarchical level. If the calculated taxonomic distance is less than a preset threshold, the system (400) may identify the input image as a high-difficulty identification sample that is difficult to distinguish due to similar external features. For the image identified as a high-difficulty identification sample, the system (400) may set an image having the representative features of the second fish species designated by the user as the correct answer as the correct answer corresponding sample, and set an image having the representative features of the first fish species predicted by the model as the incorrect answer contrast sample to generate a data pair for contrast learning. When retraining a fish species classification model, the system (400) can detect local areas where visual differences between two fish species occur by comparing the feature maps of the anchor and the incorrect comparison sample in pixel units and assign a high weight to the area.The system (400) applies a weighted feature map to extract a feature vector focused on a local area where a difference exists rather than the entire image, and then learns to minimize the distance between the feature vector of the image identified as a high-difficulty discrimination sample and the feature vector of the correct answer corresponding sample, and to increase the distance between the feature vector of the image identified as a high-difficulty discrimination sample and the feature vector of the incorrect answer comparison sample, while applying a loss function that sets the margin value, which is the minimum distance to be increased from the incorrect answer comparison sample, to be inversely proportional to the calculated taxonomic distance rather than a fixed value, thereby adjusting the model's identification criteria so that taxonomically close fish species can distinguish finer feature differences more significantly.

[0160] According to one embodiment, the system (400) may construct a biological classification database that stores hierarchical phylogenetic classification information including the order, family, genus, and species of fish. Hierarchical phylogenetic classification is a system that classifies organisms hierarchically according to evolutionary relationships, and can be classified more finely as one moves from a higher level to a lower level. An order may be a higher classification unit that includes several families, a family may include several genera, and a genus may be a lower classification unit that includes several species. For example, it may have a hierarchical structure such as the order Perciformes, family Percidae, genus Percidae, and species Percidae. The system (400) may store this hierarchical information for each fish species in the database and index it to quickly query the taxonomic relationships between fish species. The database may assign weights to each hierarchical level, and may set lower weights for lower levels, such as a weight of 1.0 if diverged at the order level, 0.7 if diverged at the family level, 0.4 if diverged at the genus level, and 0.1 if diverged at the species level. Through this biological classification database, the system (400) can establish a basis for distinguishing whether the model's prediction error is a simple misclassification or confusion between similar species.

[0161] According to one embodiment, when a user modifies the first fish species predicted by a fish species classification model to the second fish species, the system (400) may refer to a database to identify the lowest common hierarchy level at which the first fish species and the second fish species are taxonomically separated and calculate the taxonomic distance. The lowest common hierarchy level may refer to the lowest hierarchy where the two fish species belong to the same taxonomic group. For example, if the two fish species belong to the same genus but are different species, the genus level may be the lowest common hierarchy. The system (400) may look up the weight assigned to this hierarchy level and use it as the taxonomic distance. A small taxonomic distance may mean that the two fish species are evolutionarily close and likely to be similar in appearance. If the calculated taxonomic distance is less than a preset threshold, the system (400) may identify the corresponding input image as a high-difficulty discrimination sample. The threshold may be set, for example, to 0.5, which may serve as a criterion for capturing confusion between similar fish species that diverge at the genus level or lower. High-difficulty discriminative samples refer to cases where visual features are very similar, making it difficult for a model to distinguish them, and special training methods may be required for such samples.

[0162] According to one embodiment, the system (400) may generate a pair of data for comparative learning by setting an image having the representative features of a second fish species designated by the user as the correct answer as the correct answer corresponding sample and an image having the representative features of a first fish species predicted by the model as the incorrect answer contrast sample for an image identified as a high-difficulty discrimination sample. The image having the representative features is an image showing the typical and clear visual characteristics of the fish species in question, and can be determined by selecting images from existing training data that had the highest model prediction probability for the fish species in question. The correct answer corresponding sample represents the typical appearance of the actual correct fish species, and the incorrect answer contrast sample represents the typical appearance of the fish species incorrectly predicted by the model. The pair of data for comparative learning may be in the form of an anchor for the high-difficulty discrimination sample, a positive sample for the correct answer corresponding sample, and a negative sample for the incorrect answer contrast sample. An anchor may refer to a specific data point that serves as a reference point for comparison. A positive sample may refer to another data point that has the same class or the same semantic identity as the anchor. Negative samples can refer to other data points that have the opposite class to the anchor. Contrast learning is a method that learns to make the distance in feature space between the anchor and positive samples closer and the distance between the anchor and negative samples farther, which can improve the ability to distinguish subtle differences between similar classes.

[0163] According to one embodiment, the system (400) can detect local areas where visual differences between two fish species occur by comparing the feature maps of the anchor and the incorrect comparison sample in pixel units during the retraining of the fish species classification model, and assign a high weight to such areas. The feature map is a multidimensional array extracted from an intermediate layer of a convolutional neural network and may spatially represent various visual features of an input image. The system (400) can spatially align the feature maps of the anchor and the incorrect comparison sample and then calculate the difference in feature vectors at each pixel location. Locations with a large difference may represent key areas that distinguish the two fish species, such as fin shape, lateral body pattern, or mouth shape. The system (400) can assign a high weight to these areas with a large difference to induce the model to focus on local areas with differentiating features rather than the entire image. The weights may be implemented in the form of an attention mechanism, and higher values ​​may be assigned as the difference increases.

[0164] According to one embodiment, the system (400) may apply a weighted feature map to extract feature vectors concentrated in local regions where differences exist, and then perform contrast learning. The system (400) may calculate the Euclidean distance or cosine similarity between the feature vector of a high-difficulty discriminant sample and the feature vector of a correct answer corresponding sample, and learn in a direction that minimizes this. At the same time, the system (400) may learn in a direction that increases the distance between the feature vector of a high-difficulty discriminant sample and the feature vector of an incorrect answer contrast sample. The system (400) may apply a loss function that dynamically sets a margin value, which is the minimum distance that must be separated from the incorrect answer contrast sample, to be inversely proportional to the calculated taxonomic distance. The margin refers to the minimum distance difference that must be secured between the anchor and the negative sample, and in a general triplet loss function, it may be set as a fixed value. The system (400) may train the model to clearly distinguish even minute differences by requiring a larger margin as the taxonomic distance decreases, that is, as the two species are closer. For example, the margin may be set to 2.0 when the taxonomic distance is 0.1, and to 0.5 when the taxonomic distance is 0.5. Through this adaptive margin setting, the system (400) can optimize learning efficiency by applying stricter distinction criteria to similar fish species that are taxonomically close, and relatively relaxed criteria to fish species that are distant. As a result, the system (400) can intensively improve the ability to identify high-difficulty pairs of fish species that are similar in appearance, and effectively resolve cases of confusion that frequently occur in the field.

[0166] According to one embodiment, the system (400) can build a normal behavior profile by learning the distribution patterns of dwell time when a user selects one of the fish species presented in a candidate list and when the user directly searches and inputs it. The system (400) can detect abnormal behavior when a specific user's current labeling behavior deviates statistically significantly from that user's past normal behavior profile. For example, if a user who usually has an average dwell time of 30 seconds suddenly performs labeling continuously within 5 seconds, the system (400) can identify this as an abnormal pattern and automatically lower the reliability of the labels entered during that time period. The system (400) can collect micro-behavioral data including the user's mouse movement trajectory, accuracy of click locations, and scroll patterns to distinguish whether the judgment was made carefully by a real person or by an automated script or random clicks. Through this behavior-based anomaly detection, the system (400) can block malicious users or careless labeling at an early stage and protect data quality.

[0167] According to one embodiment, the system (400) can utilize historical tidal data, water temperature records, and ocean current directions of a corresponding location in real-time from an external ocean database or API based on GPS coordinates included in image metadata for environmental suitability assessment. The system (400) can improve the accuracy of the environmental suitability score by learning a pattern in which a specific fish species is mainly caught in a specific current direction and comparing it with the current conditions of the current image. The system (400) can evaluate the possibility of predatory fish species appearing by utilizing satellite observation data to analyze the trend of recent chlorophyll concentration changes at a corresponding point and indirectly estimating the abundance of prey organisms. The system (400) can check atmospheric pressure changes, precipitation status, and wind speed at the time of shooting by linking with external meteorological agency data, and learn the impact of these weather conditions on the activity of a specific fish species to reflect in the environmental suitability score. The system (400) can improve the precision of environment-based label verification by comprehensively utilizing not only location and season but also real-time ocean physical data, taking into account the spatiotemporal variability of the marine environment.

[0168] According to one embodiment, the system (400) may consider the chronological order in which users modified labels when calculating the consensus, thereby reflecting the possibility that users who modified later may have referred to the opinions of previous users. The system (400) may apply a time decay function that assigns the highest independence weight to the opinion of the first user to modify and sequentially decreases the weight for later users. However, the system (400) may preserve the value of expert opinions by not applying or mitigating time decay when a user with high credibility modified later. The system (400) may discount the consensus of the labels by considering the possibility of group bias or mutual influence when the time interval for modifications to the same label is very short.

[0169] According to one embodiment, the system (400) may separately classify a certain top proportion of data with particularly high overall quality scores among the data confirmed with refined labels as premium training data. When retraining the model, the system (400) may train the entire training data over multiple epochs, but the premium training data may be trained repeatedly for more epochs or the sampling probability per batch may be increased so that the model is exposed to high-quality data more. The system (400) may set the contribution of the loss function for premium data and general data differently so that errors on premium data are penalized more severely, and errors on general data are penalized relatively less severely. Through this quality stratification training method, the system (400) can increase the influence of high-quality data within a limited training time and improve both the convergence speed and final performance of the model.

[0170] According to one embodiment, the system (400) can automatically generate and update a confusion matrix between the species confused with the same species when multiple users modify the same species with different labels, resulting in a low level of agreement. The confusion matrix is ​​a two-dimensional array of size N x N, where the rows represent species predicted by the model and the columns represent species finally confirmed by the user, and the value of each cell represents the number of times the corresponding combination occurred. The system (400) can accumulate confusion patterns in real time by increasing the count of the corresponding matrix cell by 1 whenever a new user modification event occurs. Since the diagonal elements represent correct predictions and the off-diagonal elements represent misclassification, the system (400) can identify frequent confusion patterns by extracting cells with large values ​​among the off-diagonal elements. According to one embodiment, the system (400) can identify pairs of species that appear with high frequency in the confusion matrix to identify similar species combinations that the model systematically confuses. The system (400) can classify, for example, if the frequency of confusion between flounder and halibut accounts for more than 10% of the total errors, as a high-frequency confusion pair. The system (400) can manage pairs of fish species whose confusion frequency exceeds a set threshold as a priority list and update it periodically. The system (400) can also track the direction of confusion for each confusion pair and, for example, record the frequency of mispredicting flounder as halibut and mispredicting halibut as flounder, respectively.

[0171] According to one embodiment, the system (400) can automatically generate and display a customized guide message when presenting prediction results in a user interface for high-frequency confusion pairs. For example, if the model predicts a fish as a flounder, the system (400) can generate a specific message such as, "This fish species is easily confused with a halibut, so please carefully check the position of the eyes and the lateral line." The system (400) can refer to a database of identification features for each fish species to automatically determine the distinguishing points, and the database may store key distinguishing features for each pair of fish species in text form. The system (400) can automatically infer the distinguishing points by utilizing distinguishing feature information previously entered by experts or by analyzing the magnified area patterns used by past expert users when identifying the corresponding pair of fish species.

[0172] According to one embodiment, the system (400) may automatically highlight the areas of difference in representative images of two fish species or present them side-by-side for comparison to explain the distinguishing points. The system (400) may mark the eye location area in the representative images of flounder and halibut with a red circle or arrow, and display an explanation such as "eyes are located on the left" for flounder and "eyes are located on the right" for halibut. To automatically detect the areas of difference, the system (400) may pass the representative images of the two fish species through a feature extraction layer of a convolutional neural network to generate a feature map and calculate an area with a large spatial difference between the two feature maps. The system (400) may mark the coordinates of the area with a large difference with a bounding box and enlarge the area to place side-by-side on a side panel. The system (400) may automatically display this visual guide on the right panel of the user interface so that the user can refer to it along with the image currently being distinguished.

[0173] According to one embodiment, the system (400) can periodically analyze confusion matrix data to identify fish species combinations that are weak points in the model and intensively reinforce training data for those combinations. For example, the system (400) can analyze the confusion matrix weekly to extract the top 10 high-frequency confusion pairs and calculate the training data ratio for those fish species pairs. If the training data for a specific confusion pair is less than 1% of the total data, the system (400) determines that there is a lack of data and can adjust the user queue assignment algorithm to prioritize the collection of images of that fish species pair. When images of high-frequency confusion pairs are newly labeled and confirmed as refined labels, the system (400) can classify them as premium training data and assign a higher sampling probability during retraining.

[0175] According to one embodiment, the system (400) can track which body part the user focused on observing by recording the coordinates of the area where the user zoomed in on the image. The system (400) can detect a zoom event whenever the user zooms in on the image using a mouse wheel or a pinch gesture, and log the pixel coordinates of the zoom center point, the zoom level, and the time the zoomed state is maintained. The system (400) can store all zoom events that occurred during a single labeling session in chronological order and normalize each zoomed area by converting it into coordinates based on the original image resolution. The system (400) can divide the image into a grid to determine which part of the image the zoomed area corresponds to and identify the grid cell to which each zoom center point belongs.

[0176] According to one embodiment, the system (400) may build a species-specific identification point database to determine whether an enlarged area corresponds to a major identification feature area of ​​the fish species. The identification point database may store the anatomical names and approximate locations on images of major identification areas for each fish species, and may include information such as, for example, "Flounder - Eye position: upper left area, lateral line: central horizontal line, number of fin rays: lower abdomen." The system (400) may use identification point information manually entered by an expert as an initial database and then continuously reinforce the database by automatically learning the enlargement patterns of high-reliability users. The system (400) may cluster areas commonly enlarged by users with a reliability of 0.9 or higher in images of a specific fish species and automatically register them as major identification areas. The system (400) may assign weights to the importance of each area in the identification point database, and may set higher weights for features that are decisive for species identification.

[0177] According to one embodiment, the system (400) can refine the behavior indicator function by adding the reliability of the corresponding labeling when the user zooms in on an accurate identification area and subtracting the reliability when the user zooms in on only an irrelevant area. The system (400) can calculate the spatial overlap between each area zoomed in by the user and the identification point area stored in the database using IoU or Euclidean distance. The system (400) can assign a higher score as the degree of overlap increases and calculate a final zoom quality score by multiplying by the importance weight of the identification point. For example, if the user zooms in on the location of the eye in a flounder image, the system (400) may receive a high score with a weight of 1.0, but if the user zooms in only on the tip of the tail, it may receive a low score with a weight of 0.2. The system (400) can calculate the total zoom behavior score by averaging the quality scores of all zoom events and include this as an additional variable in the behavior indicator function g of Equation 3.

[0178] According to one embodiment, the system (400) can learn the pattern of areas that experts primarily zoom in on when identifying a specific fish species and use this as an expert indicator. The system (400) can separately collect the zoom patterns of expert users with a confidence level of 0.95 or higher to generate expert zoom pattern profiles for each fish species. The system (400) can calculate the similarity between a new user's zoom pattern and an expert profile using cosine similarity or Jaccard similarity. The system (400) can determine that the higher the similarity, the more likely the user is using a discrimination method similar to that of an expert, and can add points to the user-based score. For example, the system (400) can identify that when distinguishing between a flounder and a halibut, a skilled user first checks the overall shape, then zooms in intensively on the location of the eyes, and then checks the lateral line pattern, whereas a novice user tends to zoom in on various parts randomly or zoom in on background areas that are not helpful for identification. The system (400) can also evaluate the rationality of the expansion order and may award additional sequential expertise points if it matches the typical expansion order of an expert.

[0179] According to one embodiment, the system (400) can manage user reliability scores by separating them by fish species category rather than as a single value. The system (400) can classify all fish species into multiple categories based on biological classification, habitat type, or management purpose, and can, for example, be classified into freshwater fish, marine fish, migratory fish, benthic fish, etc.

[0180] For example, the system (400) can calculate independent confidence scores for each category when a specific user shows high accuracy for freshwater fish but low accuracy for marine fish. Whenever the user performs labeling on data recognized as correct, the system (400) checks the fish species category of the correct data and can reflect the correctness only in the confidence score of that category. The system (400) can calculate the correct answer rate for each category by accumulating the number of correct answers and the total number of attempts for each category, and normalize this to a range between 0 and 1 to use as a confidence score. If the number of attempts for a specific category is insufficient, the system (400) can calculate confidence combined with prior probability by applying a Bayesian estimation method to reflect that the confidence interval is wide. The system (400) can independently record, for example, that the user correctly answers 46 out of 50 freshwater fish data points and obtains a confidence level of 0.92, but correctly answers only 13 out of 20 marine fish data points and obtains a confidence level of 0.65.

[0181] According to one embodiment, the system (400) can perform a more accurate quality evaluation by applying a reliability score corresponding to the fish species category of the image when the user performs labeling. The system (400) can identify the predicted fish species category based on the results of the initial classification by the model and retrieve the user's reliability score for that category. The system (400) can calculate a user-based score by substituting the reliability (U_trust) variable, rather than the overall reliability, into the reliability (U_trust) variable in Equation 3. Through this, the system (400) can accurately reflect the difference in expertise by category, even for the same user, by evaluating the labeling of freshwater fish images with high reliability and the labeling of marine fish images with low reliability. The system (400) can also verify the final confirmed fish species category and apply the corresponding category reliability even if the user modifies the model's prediction.

[0182] According to one embodiment, the system (400) can help the user recognize their strengths and weaknesses in specific fields by providing feedback on reliability by category. The system (400) can visualize and display reliability by category in the form of a radar chart or a bar graph on a user dashboard or profile page. The system (400) can provide the current reliability score, number of attempts, correct answer rate, and rank relative to the average for each category, enabling the user to identify their relative position.

[0183] According to one embodiment, the system (400) can optimize the overall labeling quality by matching expertise with image characteristics by preferentially assigning images corresponding to the user's strength category. The system (400) can classify the expected category of the image based on the initial prediction result of the model when a new image is added to the labeling queue. The system (400) can query the reliability of currently connected users by category and preferentially assign images to the user with the highest reliability in that category. For example, when user A with a reliability of 0.95 for freshwater fish and user B with a reliability of 0.90 for saltwater fish are connected at the same time, the system (400) can maximize the overall labeling accuracy by assigning the newly received freshwater fish image to user A and the saltwater fish image to user B.

[0185] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

Claims

Claim 1 In a continuous learning system for a metadata-based user-participatory label quality evaluation and fish species identification artificial intelligence model, a memory for storing instructions;The system includes a processor, and when the instructions are executed by the processor, the system acquires and stores an image including a catch photo from a user terminal, performs initial classification by calculating species candidates and prediction probabilities for the input image using a pre-trained species classification model, displays a list of top prediction results and candidates to the user based on the prediction probabilities, and provides a user interface that receives the user's confirmation, candidate selection, or direct input regarding the prediction results, generates a user action log by collecting event information including the type of user input occurring in the user interface, dwell time (total time spent on the labeling page of a specific image), and zoom-in counts (number of times the image was zoomed in to check detailed features), scans the user action log at regular intervals to collect new user modification history regarding the modification of labels for the species, and the metadata includes model prediction information for the initial classification result, location information, time information, marine environment information, user information, image quality information, and statistical consistency information with surrounding data, a user-based score calculated by combining the dwell time and zoom counts with user confidence based on the labeling match rate for already verified data, and a specific image Alternatively, a system that evaluates and refines label quality by analyzing metadata including the degree of agreement on the modification results of multiple users, calculated as the ratio of the number of modifications to the same label relative to the total number of modifications, using the user reliability of each user as a weight when multiple users modify a specific fish species with the same label; generates a training data set by selecting images assigned the refined labels; and controls the model to update by performing retraining or fine-tuning on the fish species classification model according to the cumulative amount of the training data or recent model performance indicators. Claim 2 In claim 1, when the instructions are executed by the processor, the system defines metadata including model prediction information for the initial classification result, location information, time information, marine environment information, user information, image quality information, and statistical consistency information with surrounding data; calculates a model reliability score based on the prediction probability or entropy of the initial classification, an environmental suitability score (Q_env) based on the distribution of fish species appearance probability according to environmental variables, a user-based score (Q_user) combining the user reliability and the dwell time and magnification count, a cluster consistency score (Q_cluster) compared with the fish species distribution reported in the same cell and similar time period including the grid to which the catch location belongs and adjacent grids, and an anomaly detection score (Q_anomaly) that detects abnormal combinations of image quality and metadata, respectively; calculates a comprehensive quality score (Q) by applying weights to each of the calculated scores to determine the reliability of the label; and if the user modifies a fish species name not in the candidate list by directly entering it, analyzes the detailed actions that occurred during the direct input and displays the fish species information popup multiple times A system that adjusts the reliability of an input by reflecting it as an additional variable in user-based score calculation, taking into account an action complexity indicator including the number of times the image was checked or repeatedly zoomed in and the time taken to complete the input; calculates the degree of consensus as the ratio of the number of times the same label is modified to the total number of modifications, using the user reliability of each user as a weight when multiple users modify a specific image or a specific fish species with the same label; and controls the system to adopt the corresponding label as a refined label and reflect it in the generation of training data only when the degree of consensus is above a set standard. Claim 3 In claim 2, when the instructions are executed by the processor, the system performs a weighted sum operation by applying weights (w_1 to w_5) determined through meta-model learning using preset or separate quality labels to each of the model confidence score (Q_model), environment suitability score (Q_env), user base score (Q_user), cluster consistency score (Q_cluster), and anomaly detection score (Q_anomaly) in order to calculate the overall quality score (Q); to calculate the consensus, when multiple users perform a labeling action on the same image identifier (Image ID), calculates a ratio by dividing the sum of the user confidence scores (U_trust) of users who selected a specific label by the sum of the user confidence scores of all users who performed an action on the image; determines a final label candidate by combining the overall quality score (Q) derived through the weighted sum operation, the consensus calculated by the ratio, and the prediction probability of the initial classification; and only when the consensus is greater than or equal to a preset threshold value A system that controls the inclusion of the corresponding label into the training data set by confirming it as a refined label. Claim 4 In claim 3, the instructions, when executed by the processor, define a minority class in which the ratio of the number of data samples of a specific fish species to the total data within the training data set of the fish species classification model is less than a preset threshold, and calculate the model reliability score through [Equation 1] by applying an imbalance correction coefficient to the fish species corresponding to the minority class; calculate the environmental suitability score through [Equation 2] by pre-storing in a database the probability distribution of occurrence for each fish species according to environmental variables including location, season, water temperature, and water depth derived from the cumulative statistical analysis results of historically collected data or a marine organism habitat prediction model, and applying the probability derived by matching the current identification result fish species and metadata to the database to a normalization function; calculate the user-based score through [Equation 3] by applying weights to a behavioral indicator function whose variables are user reliability, dwell time measured when performing a correction action, and the number of expansions; and apply the frequency of reported occurrence of the corresponding fish species within the same location grid and season to a normalization function. A system that controls the calculation of the cluster consistency score through [Mathematical Formula 4].[Equation 1] Q_model = (1 - (H / H_max)) * C_balance (where Q_model is the model confidence score, H is the prediction entropy, H_max is the maximum entropy, and C_balance is an imbalance correction coefficient that is set to a value greater than 1 for the minority class and 1 otherwise)[Equation 2] Q_env = f(P(s* | env*))(where Q_env is the environment fit score, P(s* | env*) is the probability of the appearance of fish species s* identified in the current environment variable env*, and f is the normalization function)[Equation 3] Q_user = a * U_trust + b * g(dwell_time, zoom_cnt)(where Q_user is the user base score, U_trust is the user trust, g is the behavioral indicator function, dwell_time is the dwell time, zoom_cnt is the number of zooms, and a and b are weighting coefficients)[Equation 4]Q_cluster = h(frequency(s*, cell, season))(where, Q_cluster is the cluster consistency score, frequency is the frequency calculation function, s* is the identified fish species, cell is the location grid, season is the season, and h is the normalization function). Claim 5 In Clause 4, when the instructions are executed by the processor, the system classifies the nature of the data by analyzing the discrepancy pattern between the environmental suitability score (Q_env) and the consensus resulting from user modifications; wherein if the visual prediction probability of the fish species classification model is below a first threshold value and the environmental suitability score (Q_env) is calculated to be low, below a second threshold value, but the consensus exceeds a third threshold value because multiple users with a high level of user trust (U_trust) commonly modify the label to a specific fish species, this is identified as a Data Drift candidate group, which is a phenomenon where the distribution of existing training data differs from the distribution of actual field data due to changes in the marine environment or the appearance of non-existent fish species; and for the image data identified as the Data Drift candidate group, emergency retraining is performed by setting the sample weights applied to the calculation of the loss function during model training higher than those for general data so that the model reflects the change preferentially; conversely, even if the visual prediction probability of the fish species classification model is high, exceeding a fourth threshold value, the environment regarding the location and season A system that controls the model to withhold automatic label confirmation and classify into an expert validation queue when the fitness score (Q_env) is below a fifth threshold value indicating that survival is impossible, by determining it as a hallucination or anomaly where the model has false confidence. Claim 6 In claim 1, when the instructions are executed by the processor, the system selects the refined, labeled images, converts the image data into a tensor structure corresponding to the input layer dimension of the fish species classification model, normalizes pixel values, and converts the final confirmed labels into one-hot encoding or integer encoding forms to generate a combined training data set; analyzes logs collected during real-time service operation to calculate a User Correction Rate, which is the ratio of the number of cases where a user performs a correction action to the total number of predictions; determines that if the User Correction Rate within a recent period exceeds a preset threshold, it indicates a decline in model performance and triggers a retraining trigger; deploys the newly generated model in response to the retraining trigger to a sandbox environment, which is a virtual verification space separated from the actual service environment, while controlling the system so that user request traffic containing image information including the catch photos flowing into the actual service is replicated in real-time and the same images are input into the new model in the sandbox environment, without returning the prediction results to the user; and the new model A system that verifies whether accuracy has been improved by comparing a predicted value with a modified value of an actual user, processes that for a new model that has been verified, routes only a set portion of the total user request traffic containing image information including the fishing photo to the new model and delivers the remaining user request traffic to the existing model, and controls that the existing model be replaced with the new model by gradually increasing the ratio while monitoring for errors. Claim 7 In claim 1, the instructions, when executed by the processor, present to the user an image of verified existing data with a correct answer label already secured at a random time within an image queue to be determined by the user, so that the system cross-verifies the user's labeling reliability in real time; if the modification action entered by the user regarding the presented verified existing data image matches the preset correct answer, a weight proportional to the number of consecutive correct answers is applied to the user's reliability to grant bonus points, and if it does not match, a penalty is applied to update the reliability score; if the updated user reliability falls below a preset first threshold, the ratio of the verified existing data image among all images presented to the user is gradually increased to strengthen the verification intensity, and if verification fails more than a certain number of times, a penalty is imposed to restrict the user's labeling authority or invalidate the validity of the previously entered data; and in the case of a superior user whose updated user reliability exceeds a preset second threshold, an image or data with high uncertainty where the prediction probability distribution of the fish species classification model is flat A system that prioritizes the allocation of images presumed to be rare fish species with insufficient samples to increase exposure frequency, and, when the labeling results performed by the excellent user are finally adopted as training data, provides differential points or grade experience points in proportion to the value evaluation score calculated by multiplying the level of model performance improvement based on the amount of uncertainty reduction of the image and the scarcity weight of the fish species, and controls the system to provide additional rewards based on traffic generated whenever the data is utilized for verification when the image labeled by the excellent user is subsequently promoted to verified data for evaluating the reliability of other users. Claim 8 In claim 1, the instructions, when executed by the processor, construct a biological classification database storing hierarchical phylogenetic classification information including the order, family, genus, and species of fish in the system; when the user modifies the 'first fish species' predicted by the fish species classification model to 'second fish species', refer to the database to identify the lowest common hierarchical level at which the first fish species and the second fish species are taxonomically separated, and calculate the taxonomic distance between the two fish species based on the weight assigned to that hierarchical level; if the calculated taxonomic distance is less than a preset threshold, identify the input image as a 'high-difficulty identification sample' that is difficult to distinguish due to similar external features; for the image identified as the high-difficulty identification sample, set an image having the representative features of the second fish species designated by the user as the correct answer as the correct answer corresponding sample, and set an image having the representative features of the first fish species predicted by the model as the incorrect answer comparison sample to generate data pairs for comparison learning; and retrain the fish species classification model. Then, by comparing the feature map of the aforementioned incorrect comparison sample with an anchor serving as a reference point for comparison at the pixel level, local areas where visual differences occur between the two fish species are detected, and a high weight is assigned to such areas; and by applying the weighted feature map to extract feature vectors concentrated in the local region where differences exist rather than the entire image, the system is trained to minimize the distance between the feature vector of the image identified as the high-difficulty discrimination sample and the feature vector of the correct answer corresponding sample, and to increase the distance between the feature vector of the image identified as the high-difficulty discrimination sample and the feature vector of the incorrect comparison sample, while applying a loss function that sets the margin value—the minimum distance required to be separated from the aforementioned incorrect comparison sample—to be inversely proportional to the calculated taxonomic distance rather than as a fixed value, therebyA system that adjusts the model's identification criteria to distinguish more subtle characteristic differences among taxonomically closely related fish species.

Images