Product weight prediction device and product weight prediction method

Abstract

A product weight prediction device disclosed in the present document comprises: a communication interface for receiving user input data related to an order of a product; and at least one processor for extracting product information of the product from a web page on the basis of the user input data, generating preprocessed data by embedding the product information, and predicting a weight of the product by inputting the preprocessed data to a learning model, an input value of which is the preprocessed data and an output value of which is a predicted weight of the product.

Description

Product weight prediction device and product weight prediction method

[0001] The embodiments disclosed in this document relate to a product weight prediction device and a product weight prediction method.

[0002] A purchasing agency service is a service that purchases specific products or services on behalf of customers when purchasing overseas products or specialized products that are difficult for customers to access. In a purchasing agency service, accurately predicting the weight of a product can provide customers with an accurate purchase cost, thereby improving the predictability of product purchases.

[0003] To do this, you need to be able to collect product information from a website, but existing scraping technology requires a process of analyzing the HTML tags of the website from which you want to collect data in advance, and has the limitation that you cannot scrape websites that have not been analyzed.

[0004] In addition, since the process of predicting weight was previously performed manually by purchasing agency service providers, which hindered efficiency, there is a need for a technology that can collect product information and accurately predict the weight of a product based on machine learning.

[0005] According to one embodiment disclosed in this document, a product weight prediction device and a product weight prediction method are provided that can provide an accurate shipping cost to a consumer using a purchasing agent by predicting the weight of the product based on machine learning.

[0006] The technical problems of the embodiments disclosed in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the descriptions below.

[0007] A product weight prediction device according to one embodiment includes a communication interface for receiving user input data related to an order of a product, and at least one processor for extracting product information of the product based on a web page based on the user input data, generating preprocessing data by embedding the product information, and inputting the preprocessing data into a learning model that uses the preprocessing data as an input value and the predicted weight of the product as an output value to predict the weight of the product.

[0008] The at least one processor may input text information and image information included in the product information into a text embedding model and an image embedding model, respectively, to generate the preprocessing data.

[0009] The at least one processor may determine the format of a word embedded in the text information according to a text embedding technique, and input a word processed to correspond to the format into the text embedding model to generate the preprocessed data.

[0010] The at least one processor may generate encoding data by assigning a unique index to the number of products included in the word, and input the encoding data into the text embedding model to generate the preprocessing data.

[0011] The at least one processor may extract a high-dimensional vector from a pre-selected layer of an image classification model included in the image embedding model, and determine the high-dimensional vector as the preprocessing data.

[0012] The at least one processor can generate the preprocessing data by expressing the embedding data, which is an output value of the text embedding model and the image embedding model, as a ragged tensor.

[0013] The at least one processor may extract the product information from the tag included in the web page based on an ensemble model based on supervised learning that combines a first learning model and a second learning model.

[0014] The at least one processor can predict the weight of the product based on the learning model that forms a category tree according to the attributes of the product included in the preprocessed data and classifies the preprocessed data according to the category tree.

[0015] The at least one processor may generate a plurality of category trees, determine a category tree with the highest similarity to the user's input as a final category tree, and predict the weight of the product based on the final category tree.

[0016] The at least one processor can predict the weight of the product based on the learning model learned to minimize an error between the preprocessed data and the label data classified according to the category tree.

[0017] The at least one processor may derive the preprocessed data classified based on the category tree as an intermediate output value, and output the category of the product together with the weight of the product as a predicted value.

[0018] A method for predicting a weight of a product according to one embodiment includes receiving user input data related to an order of a product, extracting product information of the product based on a web page based on the user input data, generating preprocessing data by embedding the product information, and inputting the preprocessing data into a learning model that uses the preprocessing data as an input value and the predicted weight of the product as an output value, thereby predicting the weight of the product.

[0019] According to a weight prediction device according to one embodiment, meaningful product information can be automatically extracted from product data, and the weight of the product can be predicted based on a learning model, thereby improving accuracy and significantly reducing the weight prediction time.

[0020] FIG. 1 is a block diagram showing the configuration of a product weight prediction device according to one embodiment.

[0021] Figure 2 schematically illustrates a learning process of a product weight prediction device according to one embodiment.

[0022] FIG. 3 is a schematic flowchart illustrating a method for predicting the weight of a product by a product weight prediction device according to one embodiment.

[0023] Figure 4 illustrates a category tree constructed by a product weight prediction device according to one embodiment.

[0024] FIG. 5 illustrates an LSTM model utilized by a product weight prediction device according to one embodiment.

[0025] FIG. 6 illustrates a seq2seq model utilized by a product weight prediction device according to one embodiment.

[0026] FIG. 7 illustrates an unstructured tensor utilized in a product weight prediction device according to one embodiment.

[0027] FIG. 8 illustrates a result of classifying categories based on one category tree by a product weight prediction device according to one embodiment.

[0028] FIG. 9 illustrates a control flowchart of a method for predicting product weight according to one embodiment.

[0029] Figure 10 illustrates a method for predicting product volume / weight and calculating international shipping costs according to one embodiment.

[0030] Figure 11 is a drawing comparing the conventional cross-border purchasing and delivery agency procedures with the simplified procedures of the present invention.

[0031] Figure 12 illustrates a comparison between a manual quotation flow according to a conventional method and a real-time quotation flow according to the present invention.

[0032] Hereinafter, various embodiments disclosed in this document will be described in detail with reference to the attached drawings. In this document, identical components in the drawings are designated by the same reference numerals, and redundant descriptions of identical components are omitted.

[0033]

[0034] FIG. 1 is a block diagram showing the configuration of a product weight prediction device according to one embodiment.

[0035] FIG. 1 is a block diagram showing the configuration of a product weight prediction device (1) according to one embodiment. The product weight prediction device (1) includes a control unit (100), a communication interface (200), and may be connected with a product information collection device (2) and an external device (3) wired or wirelessly as needed, and may also include the product information collection device (2). The product weight prediction device (1) may be implemented as a user terminal or a server device, and includes all electronic devices capable of providing the functions described below without limitation on hardware configuration.

[0036] The control unit (100) includes a processor (110) and a memory (120).

[0037] The processor (110) may include one or more CPUs, GPUs, and optionally NPUs, and may be implemented as a plurality of logic gate arrays or a combination of a general-purpose microprocessor and a memory storing a program. The processor (110) outputs control signals to control the device as a whole, and performs learning and inference of a machine learning model. The learning algorithm may include one or more of supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning, and the artificial neural network may include, but is not limited to, CNN, DNN, RNN, LSTM / GRU, RBM, DBN, BRDNN, DQN, etc.

[0038] The memory (120) stores the operating system, programs, and data required for device operation. In one embodiment, the memory (120) may include a product information DB and a machine learning learning model DB. To improve performance, the memory (120) may adopt an in-memory database (IMDB) structure, and store preprocessing, embedding vectors, category trees, and learning samples in a column-oriented manner to support high-speed retrieval. To process unstructured inputs (variable number of products per order), the embedding results may be managed as ragged tensors, and may be stored in a sparse compression format to improve storage efficiency (not shown). When using GPU / NPU acceleration, ragged operations may be optimized for parallel processing through batch sorting and packing (not shown).

[0039] For security / integrity, the processor (110) may include a security encryption module. End-to-end encryption (E2EE) is performed when transmitting and receiving data with the product information collection device (2) or external device (3). Session keys are exchanged using ECC / RSA, and body data is encrypted and decrypted using a symmetric key algorithm such as AES-256. The integrity of received data is verified using SHA-3 hashing, etc. For audit trail purposes, collection / prediction / transmission logs can be optionally recorded in a blockchain-based ledger to prevent forgery and alteration (not shown). To protect personal information, a differential privacy technique may be applied during the learning phase (not shown).

[0040] For model management / enhancement, the processor (110) may include a module that combines text embeddings and image embeddings in a cross-attention fusion layer for multimodal processing. Hyperparameters and architectures are automatically explored and tuned by AutoML, and training samples can be augmented with text paraphrasing / image augmentation for data-insufficient categories. Furthermore, the device adopts a plug-in AI module architecture, enabling the seamless addition and replacement of new embedding / prediction models (not shown).

[0041] The communication interface (200) includes a wireless communication interface (210) and a wired communication interface (220). The wireless interface (210) may include at least one of a short-range (e.g., Wi-Fi, Bluetooth, Zigbee) and a long-range (e.g., 4G / 5G, LTE-M, NB-IoT) interface. The wired interface (220) may include an Ethernet / optical link, etc. This interface transmits and receives programs and data with the product information collection device (2) and the external device (3).

[0042] In relation to QoS / fault response, the communication interface (200) monitors network conditions (RTT, loss rate, bandwidth) to perform QoS control and adaptively adjusts compression ratio, batch cycle, and transmission priority. In the event of a fault or quality degradation, the processor (110) automatically activates a local prediction model to generate a temporary result, and compares and updates the result with the server-side result after recovery through a synchronization protocol (including a roll-forward / rollback policy).

[0043] The product information collection device (2) collects product data through web crawling or downloading public datasets, and can extract product information from HTML / images, etc. The product information collection device (2) can be implemented as a separate hardware that communicates with the product weight prediction device (1), or as a software tool within the product weight prediction device (1). Data transmitted from the product information collection device (2) is encrypted and transmitted using the aforementioned E2EE.

[0044] The external device (3) receives the predicted product weight and product information from the product weight prediction device (1) and provides them to the user. The external device (3) may be implemented as a shipping cost calculation server, WMS / OMS, e-commerce back office, mobile app server, etc. In addition, the external device (3) may be another device that outputs international shipping costs or measures the volume / weight of a product, and may include various devices other than the product weight prediction device.

[0045] The product weight prediction device (1) may further include an input / output interface (not shown) for transmitting and receiving data between an input device such as a keyboard, mouse, or touch panel, an output device such as a display, and a processor (110).

[0046] In the user input reception step, the processor (110) receives order-related inputs such as product URL, product name, quantity, price, options, and shipping / payment information.

[0047] In the data collection / extraction step, the processor (110) obtains product data based on a web page based on input, and extracts meaningful product information from tags / contents by ensembling first and second learning models (e.g., LSTM-based text analysis, GRU / seq2seq-based structural analysis, etc.).

[0048] In the embedding / preprocessing step, the processor (110) generates preprocessed data as an intermediate layer vector of a text embedding model (e.g., KoBART, single / split input, one-hot encoding of counts) for text information and an image embedding model (e.g., tf_efficientnet_l2_ns) for image information. The embeddings are expressed as unstructured tensors to preserve variable lengths per order.

[0049] In the model input and prediction stage, the processor (110) inputs preprocessed data into a supervised learning-based prediction model to calculate product weights, and can derive category tree classification results as intermediate outputs. Multiple category trees can be generated, and the final tree can be selected by reflecting dynamic weights such as accuracy, data reliability, and recent sales volume, or the final weight can be calculated by weighted ensembling the results of multiple trees.

[0050] In the result provision step, the processor (110) transmits the predicted weight and category to the external device (3). If the network quality is low, the local results are provided first, and then synchronized with the server results.

[0051] The above flow is linked to the detailed operations below, and the product weight prediction device (1) enables accurate shipping cost calculation based on data, thereby improving the user cost predictability.

[0052] The product weight prediction device (1) can be implemented as a user terminal (PC, terminal, mobile phone, smartphone, handheld / wearable, etc.) or a server device (workstation, cloud, data station, etc.). The server can be configured as a distributed prediction cluster of multiple servers physically / logically separated by function or data criteria, and performs parallel inference under load balancing. For scalability, the device adopts a modular structure, allowing for hot-swapping of additional AI accelerators or storage devices via standard interfaces such as PCIe / NVLink / M.2.

[0053] Preprocessed data and training samples are stored column-oriented in an In-Memory Database (IMDB), and order-to-product mapping can be managed using a ragged index. The preprocessing pipeline can include noise removal, dimensionality reduction, and smoothing / filtering. Change history is backed up to non-volatile storage using Write-Ahead Logging (WAL), and training / inference logs can optionally be written to the blockchain ledger.

[0054] The product weight prediction device (1) can automatically restart or roll back to a previously stable version if it detects model / process abnormalities through a health check / watchdog mechanism. Time synchronization (NTP / PTP) is performed to compensate for network delays, and prediction request / response packets can be sent with priority based on QoS policies.

[0055] According to the above configuration, the product weight prediction device (1) can (i) safely collect, preprocess, and predict text / image multimodal information, (ii) process variable length data and large-scale datasets in real time, (iii) ensure continuity even in the event of network environment changes and failures, and (iv) provide long-term expandability through module replacement and expansion.

[0056] In addition, dynamic selection / ensemble based on category trees, GPU optimization of ragged tensors, IMDB, E2EE, QoS, local fallback, and blockchain logs are differentiating factors that simultaneously improve accuracy, processing speed, security, and reliability compared to existing technologies.

[0057]

[0058] Figure 2 illustrates a learning step of a product weight prediction device according to one embodiment.

[0059] Referring to FIG. 2, in this embodiment, the processor (110) collects and refines a large amount of product data to construct preprocessed data and label data (actual weights), which are then used to train a machine learning / deep learning-based prediction model. The learning process may include supervised learning (weight regression / category classification) and data-based category generation using unsupervised learning (clustering).

[0060] With regard to data collection and storage (building a learning information DB), according to one embodiment, the processor (110) can collect various product attributes from the product information collection device (2). Specifically, in addition to its own data, product data supplied from a DB of a web-based platform (e.g., a standard DB for distributed products) can be utilized as learning data, and a large amount of product information (approximately 300,000 items) can be collected and learned through website-based crawling. The collected learning information can be stored in the learning information DB.

[0061] Data governance can be implemented throughout the data collection phase. This includes deduplication, outlier detection, unit / currency unification (e.g., g↔kg, KRW↔USD), missing value handling, and image resolution / ratio standardization. Storage adopts a column-oriented structure based on an In-Memory Database (IMDB), supporting high-speed retrieval of large-scale data. Durability is ensured through a Write-Ahead Logging (WAL) policy during writes. Differential privacy and anonymization can be applied throughout the collection, storage, and learning processes to protect sensitive information.

[0062] For label acquisition and quality assurance, label data consists of actual weights extracted from actual shipping / inspection records. To enhance data reliability, weight values ​​from multiple sources are cross-validated and outliers (e.g., missing packaging, abnormal entries) are removed. Rule-based normalization and statistical corrections can be applied to account for differences in product options (quantity, set composition, packaging inclusion).

[0063] In relation to the embedding / preprocessing pipeline, the processor (110) embeds the collected product information to generate preprocessing data and inputs it into the prediction model.

[0064] In terms of text, text such as product name / option / description is converted into a text embedding model after tokenization and normalization (e.g., single input <[product name][configuration][quantity]> or split input [product name], [option name] model format; the quantity is given a unique index by applying one-hot encoding).

[0065] In terms of images, product images are resized / normalized / augmented (rotated, cropped, changed in brightness, etc.), and then high-dimensional vectors are extracted from the intermediate layer of the image embedding model and used as preprocessing data.

[0066] In relation to ragged tensors, sequences with different lengths for each order / product can be preserved in the form of ragged tensors, so that they can be used for learning without information loss due to excessive padding.

[0067] Text / image embeddings can be combined through multimodal fusion layers (e.g., cross-attention), and feature selection / dimensionality reduction (smoothing / filtering, PCA / UMAP) during training can be used to remove noise and enhance generalization. Data-sparse categories can be supplemented with on-demand augmentation (text paraphrasing / synthetic images).

[0068] Regarding training data composition (splitting, balancing, and batching), preprocessed and labeled data are split into training / validation / test sets, and data bias is mitigated through stratified sampling by category and weight distribution. Training batches are organized into ragged collations to efficiently group samples of different sequence lengths, and packing / masking is applied to ensure GPU / NPU acceleration.

[0069] In relation to model learning (supervised learning + unsupervised clustering), the processor (110) can input preprocessed data (a) and label data (actual weight) into an RNN-based learning model (b), and perform learning using the classified product categories and predicted product weights (c) as output values. In addition, unsupervised learning-based clustering can be applied to the preprocessed data to generate data-based product categories. The clustering algorithm utilizes K-means or DBSCAN depending on the implementation, and K-means has the characteristic of separating noise through a centroid iterative update method, and DBSCAN has the characteristic of separating noise through density-based clustering. Through this, the present device can perform weight prediction through supervised learning while simultaneously performing category classification through unsupervised learning.

[0070] For multi-task learning, the processor (110) simultaneously learns weight regression (primary target) and category classification (sub-target) in one network to maximize representation sharing. The loss is can be optimized for multi-objectives. L is the loss, and L weight is the product weight regression loss, MAE (Mean Absolute Error) is the mean absolute error, RMSE (Root Mean Square Error) is the root mean square error, and L clsis the auxiliary classification loss (e.g., category, hazardous material flag, packaging type, etc.), CE is the cross-entropy value, Focal is the imbalance correspondence value, and λ is the weighting coefficient of the two losses (adjusted to match the relative importance / scale of regression and classification).

[0071] For AutoML / hyperparameter optimization, the processor (110) automatically searches for learning rate, batch size, number of fusion layers, regularization strength, etc., and prevents overfitting through k-fold cross-validation.

[0072] To stabilize learning, the processor (110) applies regularization techniques such as mixed precision (AMP), gradient clipping / accumulation, early stopping, and weight decay / dropout.

[0073] For semi-supervised / self-supervised expansion, the processor (110) applies pseudo-labeling and contrastive pretraining to the label-deficient category to enhance performance.

[0074] Regarding the utilization of category trees and model linking, the category trees formed or updated during the learning phase are used for preprocessing data classification and weight prediction during subsequent inference. In this embodiment, tree selection / decision rules can be continuously updated by reflecting past prediction errors, data reliability, and recency as weights for each category tree node. If necessary, an embodiment is also possible that weights the results of multiple trees to produce a final weight.

[0075] For performance verification and calibration, the product weight prediction device (1) uses MAE (Mean Absolute Error), RMSE (Root Mean Squared Error), and MAPE (Mean Absolute Percentage Error) as evaluation indicators after learning is completed, and Accuracy / F1 is also used for category classification. Prediction reliability is corrected through calibration (Temperature Scaling, MC-Dropout) and can be provided to an external device (3) along with a confidence interval or confidence score. Data distribution drift monitoring and model drift alerts are implemented to detect performance degradation due to changes in data time / source.

[0076] In terms of distributed learning and operations (MLOps), large-scale training is performed in distributed data parallelism and model parallelism, and the pipeline minimizes GPU idle time through asynchronous data loaders and caches. Models, data, and environment settings are tracked through a model registry / versioning, and performance is verified through A / B testing before and after deployment. Training outputs can be stored in a machine learning model database. Retraining is automatically triggered based on schedule and performance thresholds, forming a continuous learning model.

[0077] Regarding security / integrity and reliability (applied during the learning phase), logs from the entire process—collection → storage → learning → deployment—are optionally recorded on a blockchain ledger to prevent tampering. Data and model artifacts are digitally signed. Checkpoints and retry policies are implemented to ensure that learning / verification operations are not interrupted even in the event of network failures.

[0078] The processor (110) collects various product attributes from the product information collection device (2), and learns by obtaining a large amount of product information (e.g., 300,000 items) through an external DB (e.g., standard DB of distributed products) or crawling.

[0079] The collected learning information is stored in the learning information DB, and the preprocessing embedding (preprocessing data) and the actual weight label (label data) are input to the RNN-based learning model (b) to learn to output the category and predicted product weight (c).

[0080] Unsupervised learning-based clustering (K-means / DBSCAN) can be applied to preprocessed data to create data-based categories.

[0081] As a result, the product weight prediction device (1) can predict weight based on supervised learning and classify categories based on unsupervised learning.

[0082] According to the above learning process, the present invention (i) reduces information loss through multimodal embedding and fusion, (ii) processes large-scale / unstructured data at high speed using ragged tensor, IMDB, and distributed learning, (iii) improves accuracy and stability through AutoML, clustering, and calibration, and (iv) provides a reliable learning pipeline with security, integrity, and version control. This is closely linked to the inference pipeline below (preprocessing → prediction → transmission), enabling accurate and consistent weight prediction in real-world service environments such as overseas purchasing agents.

[0083]

[0084] FIG. 3 is a schematic flowchart illustrating a method for predicting the weight of a product by a product weight prediction device according to one embodiment.

[0085] Referring to Fig. 3, the configurations of 101 to 104 can be implemented in the form of software blocks, stored in a memory (120), and executed by a processor (110). In this embodiment, the product weight prediction device (1) operates in the order of input reception, data collection / extraction, embedding / preprocessing, prediction, and result provision, and each step can include reinforcement logic for security, quality, and efficiency.

[0086] In connection with input reception and initial verification, the processor (110) receives order-related input from the user, such as the product URL, product name, product quantity, product price, options, shipping information, payment method, and other requirements (user input reception). The input is received via the communication interface (200) and may be transmitted from a user terminal or an external device (3).

[0087] Data transmission and reception are protected by end-to-end encryption (E2EE), and packet integrity can be verified using hashes (e.g., SHA-3) (enhancing security and integrity). Input formats, units, and currencies are normalized to a standard schema (e.g., g↔kg, KRW↔USD).

[0088] Rate limiting to prevent spam and duplicate requests, and queuing / priority (QoS) to distribute system load can be applied (request control).

[0089] The processor (110) collects URL-based product information and stores it in a product information DB (121), and extracts product information from web tags using the first and second learning model ensembles.

[0090] In relation to product information collection (data acquisition step), the processor (110) can acquire product data based on a web page based on input (URL, etc.) and store it in a product information DB (web page-based collection).

[0091] In addition to our own data, we can accumulate a large amount of product data (e.g., approximately 300,000 items) in advance through external platform DBs (e.g., standard DB for distributed products) or crawling (collection source).

[0092] Dynamically rendered pages are processed using a headless browser, and deduplication, ad / banner removal, and multilingual support are performed (improving collection quality). Retry and alternative source lookup procedures may be applied in case of response failure.

[0093] In relation to product information extraction (ensemble-based), the processor (110) can extract product information from tags included in a web page based on a supervised learning-based ensemble model that combines a first learning model and a second learning model (extraction logic).

[0094] The first learning model (such as LSTM) can learn meaningful text and structural patterns from the HTML sequence of a web page and extract text-based attributes such as product name, price, and description.

[0095] The second learning model (seq2seq / GRU, etc.) can sequentially extract attribute sequences of multiple products from a list-type page using an encoder-decoder structure.

[0096] Ensembles can be implemented using weighted voting, averaging, or metarunner, and weights can be dynamically adjusted based on accuracy and reliability (combined method).

[0097] The embedding data generation unit (101) of the control unit (100) can input extracted product information into a text embedding model and an image embedding model to generate preprocessing data.

[0098] The embedding data generation unit (101) generates preprocessing data using text embedding (format, split input, one-hot encoding) and image embedding (tf_efficientnet_l2_ns intermediate layer vector).

[0099] In relation to text embedding, the embedding data generation unit (101) can use a Korean-specific language model such as KoBART. The embedding data generation unit (101) supports a single input <[Product Name][Configuration][Quantity]> or a split input ([Product Name], [Option Name]) as an input format, and the number of products can be assigned a unique index using one-hot encoding). The embedding data generation unit (101) can generate sentence / token embedding vectors (fixed / variable length) as output values.

[0100] In relation to image embedding, the embedding data generation unit (101) can use tf_efficientnet_l2_ns based on the timm library as a model, and can extract a high-dimensional vector from a pre-selected intermediate layer and determine it as preprocessing data.

[0101] The embedding data generation unit (101) can perform resizing, normalization, and optional augmentation (rotation, cropping, etc.) as preprocessing.

[0102] In relation to the unstructured tensor configuration, the embedding data generation unit (101) configures sequences with different lengths, such as order and bundle units, into unstructured tensors (ragged tensors) to reduce padding overhead and preserve the original structure.

[0103] In relation to multimodal fusion, the embedding data generation unit (101) can combine text / image embeddings in a cross-attention fusion layer to enhance mutually complementary features.

[0104] The preprocessing data generation unit (102) performs preprocessing such as noise removal.

[0105] The preprocessing data generation unit (102) can emphasize meaningful features (noise removal) through outlier removal, smoothing / filtering, normalization (scale / unit / currency), dimension reduction (PCA / UMAP), etc.

[0106] Additionally, the preprocessing data generation unit (102) configures the batch with ragged collation / packing for real-time inference (batch configuration).

[0107] In addition, the preprocessing data generation unit (102) can shorten the delay by utilizing the feature cache for the same URL and same image (cache / reuse).

[0108] The machine learning model learning unit (103) learns and saves a model with selected features and hyperparameters, and the product weight derivation unit (104) predicts the product weight along with the category tree classification results.

[0109] In the machine learning model learning unit (103), features and hyperparameters selected in the learning stage are version-managed together with model artifacts, and the same preprocessing pipeline is applied during inference to minimize data shift (learning-inference consistency).

[0110] The machine learning model learning unit (103) can optimize the delay-accuracy tradeoff by selecting a lightweight local model or a server high-precision model depending on the situation (model selection).

[0111] The product weight derivation unit (104) inputs preprocessing data into a supervised learning-based prediction model to predict the product weight (prediction).

[0112] The product weight derivation unit (104) can derive preprocessed data classified according to the category tree as an intermediate output and output the product category together with the predicted value (category output).

[0113] The product weight derivation unit (104) can calculate the final weight by weight-based selection or multi-tree ensemble that reflects past errors, data reliability, recency, etc. from multiple category trees (using category trees).

[0114] The product weight derivation unit (104) can correct the results through calibration (temperature scaling, etc.) and calculate the confidence score / interval together (calibration / reliability).

[0115] In relation to combined delivery and cluster processing, the product weight prediction device (1) can calculate the total weight by summing the weight prediction values ​​of each product in the case of combined delivery of multiple products (summing method). The product weight prediction device (1) can predict the bundle weight by considering the bundle as a single product using the average value of sentence embeddings (representative embedding method). For example, the product weight prediction device (1) can configure a model based on nine items and perform zero-value padding if the number of items is less than nine (padding-based model). Examples of bundle processing, such as combined delivery summing, average embedding, and zero-padding models, are possible.

[0116] With regard to result transmission and feedback, the product weight prediction device (1) can transmit the predicted product weight and product category to an external device (3) (external device transmission). During transmission, the communication interface (200) can prioritize transmitting the predicted response through QoS control, provide local results in the event of a network failure, and then synchronize with the server results after recovery. The results are provided to the external device (3).

[0117] The product weight prediction device (1) can prevent tampering by selectively recording the collection-inference-transmission history in a blockchain-based ledger. The storage layer can ensure durability (log / integrity) through a Write-Ahead Logging (WAL) policy.

[0118] The product weight prediction device (1) retrieves the actual delivery weight from an external device and returns it as label data, which can be used for model retraining and category tree weight update (continuous improvement).

[0119] The flowchart in Figure 3 of this embodiment (i) enhances accuracy through ensemble extraction and multimodal embedding, (ii) reduces processing latency with ragged tensors, cache, and QoS, (iii) enhances practical reliability through calibration and reliability provision, and (iv) strengthens operational reliability through security and integrity logs and retraining feedback. Therefore, it can simultaneously achieve accurate and consistent weight predictions and rapid responses in real-world service environments, such as overseas purchasing agencies.

[0120]

[0121] Figure 4 illustrates a category tree constructed by a product weight prediction device according to one embodiment.

[0122] Referring to FIG. 4, in this embodiment, the processor (110) selects one or more category trees based on preprocessed data (text / image embeddings and additional attributes) or, if necessary, combines the results of multiple trees to produce a final weight. Each category tree has a hierarchical structure (parent-child category relationship) for all products, and each node may correspond to product group-specific statistics or learned parameters, such as "product name, weight, and other information."

[0123] Describes the structure and number of category trees.

[0124] In one embodiment, a category tree comprises a data structure hierarchically structuring multiple products into categories. Each node may include a category identifier, name, upper and lower connection information, the center / variance of product embeddings within the category, weight distribution (mean, standard deviation, quantiles, etc.), and quality indicators (recency, data reliability, label matching rate, etc.).

[0125] The product weight prediction device (1) may include multiple (e.g., thousands to billions) category trees (e.g., first, second, …, nth category trees), and even for the same category, different values ​​may be provided depending on the implementation, data source, and weight configuration. The processor (110) may determine the tree with the highest accuracy among these as the final category tree.

[0126] Explains the first prediction and label comparison.

[0127] In one embodiment, the processor (110) assigns preprocessed data to an arbitrary category tree (e.g., a first tree) and performs a primary weight prediction using "product, weight, and other information" stored in the tree (or predicted based on tree conditions). The derived primary predicted weight is compared with label data (actually measured weight) to determine an error.

[0128] Thereafter, the processor (110) inputs the same preprocessed data into another category tree (e.g., the second tree) to perform a secondary prediction, which is then compared with the labels. This process can be repeated multiple times (tens to tens of thousands of times), and if further iterations do not result in a smaller error (convergence or early termination conditions are met), the final category tree is determined. As a result, the final weight is calculated based on the final tree.

[0129] For example, if a consumer selects "T-shirt," the processor (110) uses the first tree, which is highly associated with clothing, to perform a first prediction and compare it with the label to determine the error. Then, a second prediction and comparison is performed using the second tree, which is associated with clothing. This process is repeated until the tree with the smallest error is selected as the final category tree.

[0130] Describes category tree-based prediction.

[0131] In this device, weight prediction is performed by linking the derived product information as input and a supervised learning-based prediction model trained to minimize the difference from the actual weight (label) with the classification results of the category tree. In other words, the preprocessed data undergoes a hierarchical classification of the tree, and category-conditional parameters (or statistics, correction values) are applied, resulting in a predicted weight. In addition, the processor (110) can output the classified category as an intermediate output value or output the predicted value together with the weight.

[0132] Describes tree grouping and representative trees (for accelerated search).

[0133] According to one embodiment, the processor (110) groups category trees with identical or similar characteristics and sets a representative category tree for each group. The group with the representative tree closest to the feature values ​​(text / image embeddings, additional attributes) extracted from preprocessed data is selected as a priority candidate, and multiple trees within the group are searched in a priority order to shorten the prediction time. At this time, information on trees or groups with low accuracy for a specific product group in the past can be stored and excluded from subsequent searches or processed with a low priority. This can reduce the overall prediction time and improve the average accuracy.

[0134] Describes dynamic scoring and selection policies.

[0135] According to another embodiment, the processor (110) can produce a dynamic score S(T|x) for tree selection.

[0136] * Input: Preprocessed data x, candidate tree T

[0137] * Example scores:

[0138] S(T|x)=αA T +βR T +γD T +δsim(x,C T )-ηC(T)

[0139] Here A T : Past prediction accuracy (e.g. 1-error), R T : Recency index (weighted time decay), D T : Data confidence (label agreement rate·number of samples), sim(x,C T ) : Similarity between input embedding and tree-centered embedding, C(T): Computational cost (depth, number of nodes, number of parameters), α, β, γ, δ, η: System parameters.

[0140] The processor (110) can select a tree with the largest S(T|x) first, or apply a bandit policy (ε-greedy, UCB, etc.) for search-exploitation balance to perform new tree search and superior tree utilization in a balanced manner.

[0141] Describes the multi-tree ensemble transformation.

[0142] In another embodiment, instead of selecting a single optimal tree, the predictions of the top K trees can be weighted and ensembled to produce the final weight. The weights are set based on factors such as S(T|x), the category matching probability, and the inverse of the variance (uncertainty). Ensembling has the effect of mitigating extreme bias and increasing robustness in the face of data drift.

[0143] Explains tree creation, update, and version management.

[0144] According to one embodiment, a category tree can be created or updated by the following procedure.

[0145] The processor (110) accumulates label weights, product categories, embedding vectors, etc. (data accumulation).

[0146] The processor (110) establishes a tree hierarchy (initial tree construction) based on an external standard classification (e.g., HS code, industry classification) or unsupervised clustering (K-means / DBSCAN) results.

[0147] The processor (110) calculates weight distribution, correction values, embedding center, quality indicators, etc. for each node (node ​​parameter learning / estimation).

[0148] The processor (110) performs time-weighted (decaying) updates when the latest data is ingested, and performs node split / merge when necessary (update during operation).

[0149] The processor (110) manages the tree structure and parameters as versions to enable reproducibility and rollback (version management).

[0150] The processor (110) monitors error trends by node (quality monitoring) and can utilize alarm / retraining triggers when thresholds are exceeded. Optionally, creation / update history and prediction logs can be recorded in a blockchain-based ledger to support tamper-proofing and audit trails.

[0151] Describes data structures, indexes, and performance optimization.

[0152] Tree / node metadata and statistics are stored column-oriented in an in-memory database (120) to support high-speed retrieval (storage). An approximate nearest neighbor search (e.g., LSH / IVF-PQ) index is constructed for category-centric embeddings to quickly find candidates for nodes / trees similar to the input embedding (indexing). The search cost is bounded by depth limits, width limits, and early termination conditions (error reduction rate thresholds). For repeated requests for the same input, the delay is reduced by using a result / feature cache (complexity control).

[0153] Describes multilingual / domain adaptation and exception handling.

[0154] Tree node names and keywords are normalized using multilingual dictionaries / translation embeddings to ensure language-independent matching (multilingual). For nodes with sparse data, hierarchical smoothing is applied using parent node statistics, or meta-learning-based correction is performed (cold start / sparse data). If the input does not fit into any tree, it is routed to the "Other / Unknown" tree, and similar category candidates are returned or the search is retried (exception handling).

[0155] In one embodiment, the "T-shirt" preprocessing data is input into the first tree to predict the first weight and compare it with the label. Then, the second tree performs the second prediction and comparison, and after repeating up to the nth iteration, the tree with the minimum error is selected as the final result.

[0156] In one embodiment, the processor (110) groups clothing group trees into a single group and establishes a representative tree. The group with the representative tree most similar to the input features is first searched, and only the trees within that group are searched in depth to reduce search time. Trees and groups with low accuracy are excluded from the search.

[0157] The product weight prediction device (1) can perform weight prediction after generating preprocessed data and then going through a tree selection step. The product weight prediction device (1) can visualize prediction accuracy by tree / category and guide tree selection based on indicators such as "Fashion → Fashion 0.98" and "Leisure → Leisure 0.65." The product weight prediction device (1) can reflect differences in composition and quantity during tree classification by preserving variable-length features for each order as a ragged tensor.

[0158] According to the above configuration, the present invention (i) improves prediction accuracy by determining the optimal tree (or ensemble) through multiple tree iterative selection or dynamic scoring / bandit policies; (ii) reduces search time through grouping, representative trees, indexing, and caching; (iii) provides reliability / traceability through version control and blockchain logs; and (iv) ensures versatility through hierarchical smoothing and multilingual normalization. Consequently, it enables accurate and rapid weight prediction in real-world service environments, such as overseas purchasing agencies.

[0159]

[0160] FIG. 5 illustrates an LSTM model utilized by a product weight prediction device according to one embodiment.

[0161] Referring to FIG. 5, the processor (110) can extract product information from tags included in a web page based on a supervised learning-based ensemble model combined with an LSTM model as a first learning model (text / structure-based analysis) and a second learning model (e.g., GRU-based seq2seq). LSTM is a type of recurrent neural network (RNN) that effectively learns patterns and long-term dependencies in sequence data, making it suitable for processing web page crawling results.

[0162] In one embodiment, the LSTM model included in the first learning model can collect HTML tags and text content from web pages using a web crawling tool such as BeautifulSoup or Scrapy. For dynamically rendered pages, the Document Object Model (DOM) can be collected using a headless browser (not shown). The collected raw HTML is normalized through preprocessing rules (not shown), such as removing script / style / ad blocks, normalizing whitespace, and unifying character encoding.

[0163] Input value X of Fig. 5 t-1 , X t , X t+1 is data tokenized with HTML tags and contents, which can be used as input to LSTM cells, for example. <title>, < / title> , , Text between tags and tags such as . may be included as part of the input sequence.

[0164] In one embodiment, each token may be composed of one or more feature embeddings (not shown) among tag embeddings (tag name, attribute name, attribute value), text embeddings (word / morpheme / subword), DOM location embeddings (depth, sibling index, section ID), visual hint embeddings (alt / aria-label, caption, etc.), and normalized numeric features (price pattern, currency / unit identification, length / count, etc.). For variable-length document processing, sequences may be packed into batches in a ragged tensor representation (not shown).

[0165] This section describes the LSTM architecture and gate operation. LSTM consists of an input gate, a forget gate, an output gate, and a cell state, and preserves and utilizes the long-term context of web documents.

[0166] The forget gate can remove unnecessary information (e.g., repeated headers / footers that are not product information) from the previous cell state. The input gate determines the degree to which new inputs (e.g., product name, price, description) are introduced. It can generate candidate values ​​and add them to the cell state by including sigmoid and tangent (tanh) layers. The cell state acts as a repository of important information across the entire document and can continuously track and update the composition and context of product information. The output gate determines which information to output based on the current cell state and can be used as input for subsequent tasks (e.g., product information extraction).

[0167] Hidden state h t-1 , h t , h t+1 The current time step output is passed to the next LSTM cell for sequential processing. For example, if a product description is divided into multiple tags, the hidden state can add new information while preserving previous information.

[0168] In another embodiment, the short-term and long-term dependencies and generalization performance can be improved by combining bidirectional (Bi-LSTM), attention (not shown), residual connection (not shown), and dropout / regularization (not shown).

[0169] This paper describes the task design and output layer. In one embodiment, the final output of the LSTM can be configured in the form of sequence labeling or interval extraction. Regarding sequence labeling, BIO tags (e.g., B-PRICE, I-PRICE, B-TITLE, etc.) are predicted for each token to identify price / product name / option / description intervals (not shown). Regarding interval extraction / pointer, the start and end locations of regions of interest are predicted to extract accurate field values ​​(not shown). The extracted fields are then converted to currency and unit, noise is removed, and duplicates are merged in a subsequent normalization module (not shown) to determine product information.

[0170] This section describes the training data construction and loss function. In one embodiment, labels are generated manually or semi-automatically from actual product pages, and weighted cross-entropy or focal loss (not shown) may be used to mitigate class imbalance. The evaluation is based on accuracy / precision / recall / F1 at the token / sequence level, as well as field-level span matching accuracy (not shown). Training may include mixed-precision optimization (AMP, not shown), gradient clipping / accumulation (not shown), and early stopping (not shown).

[0171] This describes the ensemble with the inference pipeline. The processor (110) determines product information by ensembling the outputs of LSTM (the first learning model) and GRU-based seq2seq (the second learning model). The outputs are combined using one of weighted voting, average, and a meta-learner (not shown), and dynamic weights can be adjusted to reflect confidence (e.g., logit temperature, predicted variance) and recent performance (not shown). In case of conflict, a conclusion is drawn using rule-based prioritization (e.g., a price field is weighted upwards by the LSTM) or inverse uncertainty weighting (not shown).

[0172] This section describes noise suppression and exception handling. The forget gate can be trained to underestimate repeated headers / footers, navigation, and ad blocks. In exceptional cases where the HTML structure is abnormal, replacement extraction is performed using postprocessing rules / backoff models (not shown). Non-English and multilingual pages can be normalized using language identification and multilingual embeddings (not shown).

[0173] This section describes optimizations for variable-length and high-volume processing. Because document lengths vary, batches are organized using ragged packing (not shown), and mask operations are applied to facilitate GPU / NPU parallelization. Latency is reduced by caching features (not shown) for identical URLs / images.

[0174] The present invention can be modified as follows.

[0175] i) Tree-LSTM / Hierarchical LSTM (not shown): Tree-LSTM that directly traverses the DOM tree to reflect hierarchical context.

[0176] ii) CNN-LSTM hybrid (not shown): combines local patterns (CNN) and global context (LSTM).

[0177] iii) Character / subword LSTM (not shown): Improved robustness against typos and unstructured text.

[0178] iv) Domain adaptation / continuous learning (not shown): Fast fine-tuning with small amounts of data when introducing new sites.

[0179] According to the above LSTM-based first learning model, (i) it can precisely combine scattered product information with long-term context preservation, (ii) forget / input / output gates suppress irrelevant content while emphasizing key fields to improve accuracy and robustness, (iii) it covers list-type pages / compound documents in ensemble with seq2seq models, and (iv) it reduces delay with ragged processing, caching, and parallelization.

[0180] As a result, the present invention enables accurate and consistent product information extraction from various types of e-commerce pages, thereby substantially improving the performance of subsequent steps (embedding, category tree classification, and weight prediction).

[0181]

[0182] FIG. 6 illustrates a seq2seq model utilized by a product weight prediction device according to one embodiment.

[0183] Referring to Fig. 6, the processor (110) uses a Gated Recurrent Unit (GRU)-based neural network with an encoder-decoder structure as a second learning model. This model receives the HTML structure of a web page as a sequence input, generates context information (encoder state) that summarizes the input sequence, and a decoder using this as an initial input sequentially outputs structured information for each product. Therefore, even for a page where multiple products are listed in a list format, each product information can be processed and extracted as each element of the sequence. For example, it can be trained to sequentially output product name, price, image URL, etc.

[0184] Describes the input sequence structure (HTML → token).

[0185] According to one embodiment, the encoder inputs X1, X2, …, X n is a tokenized sequence of a specific part of a web page (HTML tags and their contents). A token may consist of one or more of the following embeddings (not shown):

[0186] i) Tag embedding: Tag name ( , , , etc.), attributes (class, id, alt, aria-label)

[0187] ii) Text embeddings: word / subword (BPE) embeddings, number / currency pattern flags

[0188] iii) DOM location embedding: depth, sibling index, section / block ID

[0189] iv) Embedding visual hints: presence of images / links, presence of captions / labels

[0190] For variable length document processing, the batch sequence can be packed into a ragged tensor (not shown).

[0191] Describes the encoder configuration.

[0192] The encoder consists of multilayer GRUs (either unidirectional or bidirectional), with each GRU cell integrating the previous hidden state and the current input and passing it to the next cell. The output of the final encoder step is considered a context vector (the final encoder state) that summarizes the information of the entire input sequence. Optionally, the hidden states of the encoder at all time points can be buffered as attention keys / values ​​for reference in the decoder (not shown). Regularization (dropout / layer norm) and gradient clipping can be applied to stabilize learning (not shown).

[0193] Describes the decoder configuration and output system.

[0194] The decoder consists of GRUs, and sets the initial state to the final state (context) of the encoder.

[0195] In the first step, a special starting token ( <bos>) and context and generates the first output Y1. Y1 usually contains a specific field (e.g., product name) of the first product in the list or a product record start token.

[0196] In subsequent steps, at each point in time, the previous output token is used as the next input (teacher forcing / scheduled sampling is possible) and combined with the hidden state to produce the next output Y. t ) is created.

[0197] Output sequence Y1, Y2, … Y n is interpreted as structured information of a product unit, for example, [<ITEM_START> , <name> , <price>, <IMG_URL>, <desc>,<ITEM_END> ] You can describe multiple products sequentially by repeating the pattern multiple times. The sequence ends at <eos>It is expressed as .

[0198] Additionally, the decoder pays attention to the encoder's hidden state at each step to select input segments relevant to the current prediction (improving robustness to variable layouts). The decoder penalizes accumulated attention to prevent overreferencing / duplicate extraction of the same segment. Furthermore, the decoder can directly copy original text tokens, such as price and URL, using a pointer generator.

[0199] Describes output format constraints and post-processing.

[0200] The output token space is a dictionary of fields (e.g. <name> , <price> , <currency>,<IMG_URL> etc.) and value tokens (text / number / URL). Field-value rule masking (e.g. <price>Illegal combinations are prevented through the use of a post-processing feature (only allowing numeric / currency tokens) (not shown). Post-processing includes numeric, currency, and unit normalization, space / HTML entity restoration, and merging of duplicates (not shown).

[0201] Describe the learning process.

[0202] The processor (110) generates a product unit sequence label for each HTML document and specifies field / value boundaries (labeling).

[0203] Then, the processor (110) basically uses token-wise cross entropy, and optionally, coverage loss and label smoothing (target function).

[0204] Then, the processor (110) can apply Teacher forcing (rate scheduling), mixed precision (AMP), gradient clipping, and early termination (learning techniques).

[0205] Then, the processor (110) uses field exact match rate, span F1, and item unit EM in addition to token accuracy (accuracy indicators).

[0206] In real-time inference, the decoder generates sequences using greedy or beam search (with length compensation and repetition penalty). This avoids duplicate items or infinite loops.<ITEM_END> / <eos>Perform inspection. Batch processing is streamlined with ragged packing.

[0207] For processing multiple product lists (its core purpose), this model sequentially generates product-specific information by considering the context of each product block on a list-type page. By leveraging reference context from previous products to predict the next product, it reflects interdependencies such as repeating structures, shared descriptions, and price tables. For example, it learns the recurring patterns of product names, prices, and image URLs on a single page and sequentially outputs them.

[0208] In connection with the ensemble linkage with the LSTM-based first model, the processor (110) determines the final product information as an ensemble of the first learning model (LSTM; FIG. 5) and the second learning model (seq2seq; FIG. 6). The linkage is a weighted voting / meta-learning method and may include dynamic weight adjustments based on recent accuracy and uncertainty. In case of conflict, a conclusion is derived by field-specific priority rules or inverse uncertainty weighting.

[0209] For system integration (linking with subsequent pipelines), the structured product sequences produced by the decoder are passed to the embedding / preprocessing pipeline for text and image embedding and noise removal. The variable number of orders / products is preserved as an unstructured tensor and utilized in the subsequent category tree classification and weight prediction stages.

[0210] The present invention can be modified as follows.

[0211] i) Attention Expansion: Bahdanau / Luong Attention, Multi-Head Attention

[0212] ii) Bidirectional encoder: Simultaneously reflecting forward and backward contexts with Bi-GRU

[0213] iii) Hybrid structure: Only the encoder is a Transformer, and the decoder maintains a GRU (within the scope of the main embodiment's GRU base).

[0214] iv) Scheduled sampling / data augmentation: Improved robustness against layout changes by shuffling HTML tags and randomizing whitespace / attribute order.

[0215] According to the above configuration, the second learning model (i) summarizes and utilizes the global context of the HTML sequence with a GRU encoder-decoder, (ii) reliably extracts product-level information through sequential generation using previous outputs and hidden states, (iii) can process multiple products in a list-type page with a single decoding sequence, and (iv) is trained to sequentially output key fields such as product name, price, and image URL, thereby greatly improving versatility and accuracy compared to rule-based scraping.

[0216] As a result, the present invention secures structured information without manual tag analysis from various e-commerce pages, thereby simultaneously improving the accuracy and processing efficiency of the subsequent weight prediction step.

[0217]

[0218] FIG. 7 illustrates an unstructured tensor utilized in a product weight prediction device according to one embodiment.

[0219] Referring to FIG. 7, a structure for storing, managing, and calculating embedding and attribute values ​​extracted from different numbers of products for each order without information loss is illustrated in a product weight prediction device (1) according to one embodiment, using a ragged tensor representation.

[0220] According to one embodiment, the processor (110) may utilize an unstructured tensor in the process of generating preprocessed data by embedding extracted product information. An unstructured tensor refers to a tensor that includes subdimensions of different lengths. This enables efficient storage and management of variable-length data, which differs from typical structured tensors (fixed length, requiring padding).

[0221] In this embodiment, unstructured tensors are used to construct sequences using only actual values, without unnecessary padding, when the number of products per order ID varies (e.g., some orders have 3, others 10) or when the data length is irregular (e.g., different numbers of options or SKUs). This improves memory and computational efficiency, reduces padding-induced noise, and enhances the model's generalization performance.

[0222] Referring to Figure 7 (a), a value array is shown. This is a one-dimensional array that sequentially lists the individual attribute values ​​of a product. For example, in Figure 7 (a), [3, 1, 4, 1, 5, 9, 2] can represent a single attribute value (numeric or categorical encoding) such as price, weight, category code, or inventory.

[0223] Referring to (b) of Fig. 7, the group index array (Index / Row-ids Array) is shown. This is an identifier sequence indicating which order number each element in (a) belongs to. For example, [0, 0, 0, 0, 2, 2, 3] means that the first four values ​​belong to order 0, the next two values ​​belong to order 2, and the last value belongs to order 3.

[0224] Referring to (c) of Figure 7, a ragged tensor structure is revealed. By integrating (a) and (b), a data structure is formed that preserves sequences of different lengths for each order. For example, order 0 is [3,1,4,1], order 1 is an empty sequence, order 2 is [5,9], and order 3 is [2].

[0225] An unstructured tensor constructed in this way allows orders containing different numbers of product information to be treated as input values ​​for the same learning model.

[0226] We describe the definition and representation of an unstructured tensor. In one embodiment, an unstructured tensor can be implemented using at least one of the following two equivalent representations.

[0227] 1) values / rowids expression:

[0228] i) values: A one-dimensional array of length NNN (value array of Fig. 7 (a)).

[0229] ii) rowids: A one-dimensional integer array of length NNN (order index array of Fig. 7 (b)).

[0230] iii) Invariant: rowids are non-decreasing, and a slice of values ​​in the same rowid range corresponds to one order.

[0231] 2) row_splits representation (sparse CSR-like):

[0232] i) row_splits: An integer array of length R+1R+1R+1, which stores the start and end offsets of each order.

[0233] ii) In the previous example, the order length is [4,0,2,1][4,0,2,1][4,0,2,1], so row_splits = [0,4,4,6,7].

[0234] iii) The two representations are mutually convertible and can be selected based on memory access patterns or GPU kernel optimization needs (not shown).

[0235] Describes the procedure for generating an unstructured tensor.

[0236] The process of generating an unstructured tensor according to one embodiment can be performed in the following sequential steps.

[0237] <Data extraction step>

[0238] The processor (110) collects attribute information (numeric / categorical, such as price, weight, category code, and inventory quantity) of each product from web pages, databases, and external APIs. The collected data may undergo schema normalization, unit / currency conversion, and outlier filtering.

[0239] <Attribute Array Stage>

[0240] The processor (110) sequentially lists all extracted product attributes to form a one-dimensional array. For example, each value represents a single product attribute value, such as [3, 1, 4, 1, 5, 9, 2] in (a) of FIG. 7.

[0241] <Group Index Mapping Step>

[0242] The processor (110) identifies the order number (orderID) to which each attribute belongs and constructs a corresponding group index array. For example, (b) [0, 0, 0, 0, 2, 2, 3] of FIG. 7 indicates that the first four attributes belong to order 0, the next two attributes belong to order 2, and the last attribute belongs to order 3.

[0243] <Unstructured tensor construction steps>

[0244] The processor (110) combines the attribute array and the group index array to generate an unstructured tensor having sequences of different lengths for each order. As illustrated in (c) of Fig. 7, order 0 is mapped to [3, 1, 4, 1], order 1 is mapped to [ ] (an empty sequence), order 2 is mapped to [5, 9], and order 3 is mapped to [2].

[0245] <Framework Conversion Step>

[0246] The generated unstructured tensor can be converted into a data structure supported by the deep learning framework.

[0247] i) TensorFlow: Create a RaggedTensor object using tf.ragged.constant().

[0248] ii) PyTorch: Convert to NestedTensor (depending on the supported version) or list of tensors and process with collate_fn in DataLoader.

[0249] iii) Storage / Serialization (optional): Residing in column-oriented manner in IMDB (In-Memory Database), durability is secured with WAL (Write-Ahead Logging), and snapshots can be performed in parallel.

[0250] Additionally, the processor (110) can store text / image embeddings or numeric features as a multidimensional vector of values ​​(e.g., [N,d][N,d][N,d]) to form a ragged[*, d] structure (multidimensional embedding). In addition, the processor (110) can perform segment_mean / sum / max and segment attention to produce a representative vector of each order (segment operation). In addition, the processor (110) can apply unified unit / currency normalization and outlier smoothing to each segment within the order (normalization / scaling). In addition, the processor (110) can provide a sequence mask to the model input for variable length processing to exclude padding contributions (masking).

[0251] For integration with the learning and inference pipeline, the unstructured tensor is provided as input to the category tree classification and weight regression models (see Figs. 4 and 8) by aggregating the sequence embeddings derived from the LSTM-based first learning model of Fig. 5 and the GRU-based seq2seq second learning model of Fig. 6 by order. Even within the same mini-batch, different sample lengths can be processed by the same pipeline, thereby improving the throughput in the real-time inference pipeline.

[0252] The following examples may be added to optimize performance:

[0253] i) GPU / NPU optimization: Accelerate segment aggregation with coalesced reads and warp-level reduction based on row_splits. Apply mixed-precision (AMP), but maintain accumulation in FP32.

[0254] ii) Bucket batching: Warp divergence is mitigated by forming bucketized batching of similar length orders.

[0255] iii) Cache / Reuse: Embeddings of the same URL and image are reused in the feature cache to reduce delay.

[0256] The following examples may be added for exception / boundary handling.

[0257] i) Empty order: A segment of length 0 is denoted as row_splits[k]==row_splits[k+1] and is automatically ignored in subsequent operations.

[0258] ii) Maximum length exceeded: Dynamic truncation or sliding window can be applied when the system threshold is exceeded.

[0259] iii) Integrity verification: If rowids are reversed or negative, or if row_splits violate monotonicity, it is isolated and logged as an error.

[0260] The unstructured tensor-based preprocessing method of the present invention (1) reduces memory usage by removing unnecessary padding (memory efficiency). Furthermore, (2) improves learning and inference speed by performing computations only in areas where actual data exists (computational efficiency). Furthermore, (3) allows processing within the same pipeline even if sample lengths within mini-batches vary (flexible batch configuration). Furthermore, (4) reduces noise introduced by padding, improving generalization performance (improved model performance). Furthermore, (5) by combining IMDB+WAL-based storage, GPU / NPU optimization, and segment operations and masking, accurate and rapid product weight prediction is possible for large-scale, variable-length e-commerce data.

[0261]

[0262] FIG. 8 illustrates the result of classifying categories based on one category tree by a product weight prediction device according to an embodiment.

[0263] Referring to FIG. 8, an example of a product weight prediction device (1) according to one embodiment visualizing the results of classifying product categories based on a single category tree in the form of an accuracy heatmap is illustrated. Each category tree has a structure in which multiple products are hierarchically organized by category, and each node can have detailed additional information about the product (e.g., node representative embedding, weight statistics, reliability index, etc.) along with the relationship between upper and lower categories. The category tree may be configured to encompass all products, but may also be constructed to include a dataset specialized for a specific topic or product group (e.g., clothing, leisure).

[0264] In Fig. 8, the horizontal axis (X-axis) represents the predicted category, and the vertical axis (Y-axis) represents the actual category. Each cell numerically represents the accuracy or normalized frequency for the corresponding (actual, predicted) combination. The color scale, as shown on the right side of the figure, is such that bright colors (closer to 1.0) represent high values, and dark red / dark colors (closer to 0) represent low values. For example, if the value of the "Fashion → Fashion" cell is 0.98, it means that the Fashion item in the tree shows a 98% matching accuracy. On the other hand, if the value of the "Leisure → Leisure" cell is 0.65, it means that the Leisure item in the same tree shows a relatively low classification performance.

[0265] If the user inputs "T-shirt," the processor (110) may first select the first category tree, which is rich in clothing-related information. As shown in the results of FIG. 8, the "Fashion→Fashion" value of the first tree is very high at 0.98, so the processor (110) may select the first tree as the category tree for weight prediction. Conversely, if the user inputs leisure goods, the "Leisure→Leisure" value of the first tree is low at 0.65, so the second category tree, specialized for leisure, is likely to be selected.

[0266] In this embodiment, the processor (110) selects the optimal category tree through the following procedure.

[0267] The processor (110) receives and normalizes product name, candidate category, and detailed attributes (brand / option, etc.) from the user (input analysis stage).

[0268] Then, the processor (110) configures multiple trees with high relevance as candidates based on category, keyword, and embedding similarity (candidate tree selection step).

[0269] Then, the processor (110) calculates the accuracy / precision / recall / F1 or cell-by-cell average accuracy based on past verification data (ground truth labels) for each candidate tree and generates an accuracy matrix as shown in FIG. 8 (accuracy evaluation step).

[0270] Then, the processor (110) determines the tree with the highest accuracy for the target category (or likely candidate) of the input product as the final category tree (final tree selection step). If necessary, in case of a tie / close match, uncertainty (variance) inverse weighting, recency weighting, and data reliability are additionally considered.

[0271] <Quantitative example: Tree T's score>

[0272] S(T|x)=αA T +βR T +γD T +δsim(x,C T )-ηC(T)

[0273] Here A T : Target category accuracy (or cell value), R T : Recent, D T : Data confidence (label agreement rate, number of samples), sim(x,C T ) : Input embedding x and tree-centered embedding C T Similarity of, C(T): Computational cost. α, β, γ, δ, η are system parameters. The search-exploitation balance can be achieved by selecting the largest S(T|x) tree or by using a bandit policy (ε-greedy / UCB).

[0274] Accuracy is basically calculated as "Acc = number of correct predictions / total number of predictions", and class-specific normalization or weighted average can be applied to correct class imbalance. Element M of matrix M ij represents the frequency (or accuracy) of "actual i → predicted j", and row normalization / column normalization modes can be supported. The visualization is displayed as in Figure 8 by applying a color map in the range of 0 to 1.

[0275] The final selected category tree is linked to the weight prediction step as follows:

[0276] i) Use node-based statistics: Apply weight statistics (mean / quantile / corrected value) to the node (or subnode).

[0277] ii) Node-wise learning model: Calls a lightweight regression model (linear / tree / small DNN) at the node / subtree level.

[0278] iii) Nearest neighbor search: The KNN weights of similar products are weighted and added together using the representative embedding index within the tree (approximate nearest neighbor: IVF-PQ / LSH, etc.).

[0279] iv) Hierarchical smoothing: When lower node data is sparse, robustness is achieved by smoothing using upper node statistics.

[0280] In multi-tree ensembles, instead of a single tree, the weight predictions of the top K trees can be weighted ensembled. The weights are calculated based on factors such as the above score S(T|x), the category matching probability, and the inverse of the prediction variance. Ensembles improve robustness against data drift and address temporary quality degradation in specific categories.

[0281] In relation to tree creation, update, and version management, the processor (110) manages the tree as follows.

[0282] i) Initial construction: Create hierarchy using external standard classification (HS / industry classification) or clustering (K-means / DBSCAN).

[0283] ii) Parameter estimation: Calculation of node-specific weight distribution, representative embedding, and quality indicators.

[0284] iii) Operational refresh: Time-weighted updates when collecting the latest labels, splitting / merging nodes.

[0285] iv) Version control / audit: Tree structures and parameters are saved as versions, enabling rollback when necessary. Logs are optionally recorded in the blockchain ledger to prevent tampering.

[0286] For example, in the case of a clothing input ("T-shirt"), since the "fashion→fashion" value of the first tree is 0.98, the processor (110) selects the first tree as the final tree. In the case of a leisure input, since the "leisure→leisure" value of the first tree is low at 0.65, the leisure-specific second tree is selected.

[0287] In terms of visualization interpretation, the X-axis can represent the predicted value, the Y-axis the actual value, and the color bar on the right can represent the value size (brighter means larger). The closer to 1.0, the higher the accuracy, and the closer to 0.0, the lower the accuracy.

[0288] The product weight prediction device (1) compares values ​​stored in multiple category trees with actual data, determines the tree with the highest accuracy as the final category tree, and then calculates the weight of the input product using the product-specific weights stored or predicted in the final tree. This process is repeated, and the quality of the prediction model and tree is continuously improved as new labels are collected.

[0289] To enhance quality and reliability, the following may be included:

[0290] i) Calibration: Probability correction with temperature scaling / flattening, ECE / Brier score monitoring.

[0291] ii) Providing reliability: Transmitting scores / sections together to external devices (3) to improve transparency in cost estimation.

[0292] iii) Operation policy: QoS-based response priority, provisional results provided as a local model in case of network failure and synchronization upon recovery.

[0293] According to the above configuration, the product weight prediction device (1) (i) selects a category tree based on a quantitative basis of an accuracy heatmap to increase the reliability of weight prediction, (ii) mitigates domain bias with a specialized tree / multi-tree ensemble, and (iii) ensures continuous performance improvement and traceability with tree update and version management.

[0294] Especially in overseas purchasing, weight prediction errors directly affect shipping cost calculations, so this technology enables accurate shipping cost calculations.

[0295]

[0296] FIG. 9 illustrates a control flowchart of a method for predicting product weight according to one embodiment.

[0297] Referring to FIG. 9, the processor (110) operates in the following order: (i) receiving user input data related to an order (900), (ii) collecting web-based product information based on the input (910), (iii) extracting product information using a supervised learning-based scraping ensemble (920), (iv) preprocessing text / image embedding after determining data type (930, 940, 950, 960), (v) calculating product weight using a prediction model (970), and (vi) transmitting the prediction result to an external device (3) (980). The accuracy of the above series of operations can be improved through a subsequent iterative learning loop.

[0298] The processor (110) can receive user input data related to a product order (900). The processor (110) can receive user input data including product name, product options, quantity, seller information, product identification code (SKU), product page URL, etc. The input path can include at least one of direct user input, an API of an external ordering system, and linkage with a shopping mall back office. Optionally, the communication interface (200) can perform end-to-end encryption (E2EE) and request signing / integrity verification, and the processor (110) can perform schema normalization of unit, currency, and date formats and verification of required fields in advance.

[0299] The processor (110) can collect product information based on web pages based on received user input data (910). The processor (110) parses the HTML structure, image links, text descriptions, prices, seller information, etc. of the web page using a predefined crawler / scraper. In one embodiment, the collection module can analyze the Document Object Model (DOM) to remove advertisement / notice / recommendation areas and extract only the core product description area. In the case of dynamically rendered pages, the DOM can be secured after rendering using a headless browser.

[0300] At this time, the processor (110) can utilize a supervised learning-based scraper, and can extract product information based on the ensemble model of the first learning model and the second learning model as described above (920). The processor (110) can extract product information through a supervised learning-based ensemble model that combines the first learning model (text / structure-based analysis) and the second learning model (structured seq2seq). The first learning model (text / structure-based analysis) can perform sequence labeling / key-value extraction (category, brand, specification, color, etc.) for DOM text using LSTM, etc. The second learning model (structured seq2seq) uses GRU-based seq2seq (encoder-decoder) to input a DOM path / token string and generate a field sequence (product name, price, image URL, etc.), and sequentially (Y1…Y) records of multiple products in a list-type page. n ) can be produced. The combination method can be to integrate the outputs of the two models by weighted voting, weighted average, or meta-learner (lightweight MLP / GBM) to produce normalized primary extraction results (text fields, image URLs / bytes, confidence scores). The ensemble stage is specialized for product information extraction and can be operated as a module separate from the subsequent embedding / weight prediction stages.

[0301] Additionally, the processor (110) can determine whether the extracted product information is text (930). If the extracted product information is text (e.g., 930), the processor can input the text into a text embedding model to generate preprocessed data (950). The embedding model can use a pre-trained language model (e.g., BERT, Word2Vec, FastText) or a product domain-specific language model. The text preprocessing process can include stopword removal, tokenization, title extraction, and word embedding conversion.

[0302] For example, a text embedding model can be implemented based on KoBART / KoBERT. KoBART (open source, MIT licensed), an evolved Korean generation and recognition model from KoBERT, can be used as the text embedding backbone. KoBERT, a similar model, can also be adopted as an alternative / auxiliary backbone.

[0303] Additionally, text embedding models can be implemented using a single-string template approach. Text input is combined into a single sentence and fed into the model. For example, the format is 〈[Product Name] [Option / Specification] [Quantity]〉, and an example could be 〈[Pureumi Peekaboo Picture Book] [40 Volumes] [1 Unit]〉.

[0304] Additionally, text embedding models can be implemented using a segmented embedding method. Fields such as [Product Name] and [Option Name] are separated, each producing an embedding. Then, the sentence embedding can be formed by combining the fields (connection / weighted average). The product count (quantity) can be expressed using one-hot encoding or categorical embedding to ensure stable input to the classifier.

[0305] Additionally, text embedding models can produce sentence embeddings. After generating word / token embeddings, sentence embeddings can be produced using mean pooling. Alternatively, CLS tokens, max pooling, and weighted averaging (attention pooling) are also available.

[0306] In terms of output dimension, the embedding vector can be generated with a fixed length, such as 768 dimensions, and used as an input feature for a subsequent weight / volume weight prediction model.

[0307] According to the above configuration, text inputs, whether in the form of a single sentence or in the form of field segmentation, are all converted into consistent preprocessing vectors, and structural information such as quantity can be safely included as one-hot (or category embedding) to improve prediction accuracy.

[0308]

[0309] On the other hand, if the extracted product information is not text (No in 930), the processor (110) can determine whether the extracted product information is an image (940), and if the extracted product information is an image (Yes in 940), the processor can input the image into an image embedding model to generate preprocessed data (960). The image embedding model can be based on a convolutional neural network (CNN), a vision transformer (ViT), ResNet, an efficient net, etc., and can extract key visual features from the product image. The preprocessing process can include resizing, normalization, and data augmentation.

[0310] The image embedding model can implement an image embedding pipeline based on the PyTorch Image Models (timm) library. Furthermore, the image embedding model can use the tf_efficientnet_l2_ns (EfficientNet-L2, Noisy Student) backbone with pretrained weights. Other timm backbones (ResNet, ConvNeXt, ViT, etc.) can be substituted.

[0311] Additionally, the image embedding model can be used as a fixed-length embedding by removing the classifier (class header) through embedding extraction and normalizing the feature vector (pre-logit / penultimate feature) input to the final classification layer after global average pooling (feature extraction-based embedding). If necessary, the pre-logit can be used as is or dimensionally adjusted using a shallow projection layer (MLP).

[0312] Additionally, the image embedding model can be resized and normalized by applying the model's default input size and ImageNet mean / std of timm, and lightweight augmentation such as random cropping and horizontal flipping can be applied during learning / inference.

[0313] Additionally, the image embedding model can be used as a fixed embedding model by freezing the backbone learned from a large amount of general domains, or by partially fine-tuning it with product domain data to improve its expressiveness.

[0314] Optionally, if both text and images are present, they can be integrated into a single representation using a multimodal fusion layer (cross-attention / weighted combination). When the number of products per order varies, they can be packed into a ragged tensor (as shown in Figure 7) for batch processing without unnecessary padding.

[0315] Thereafter, the processor (110) can input the generated preprocessed data into a supervised learning-based prediction model to predict the weight of the product (970). The prediction model may include a regression-based deep learning network, a gradient boosting machine (GBM), or an ensemble tree model such as a random forest. During training, the model's prediction accuracy can be improved by utilizing past sales data, delivery data, and product attribute data.

[0316] Optionally, node statistics (mean / quantile / corrected value), node-specific submodels, or approximate nearest neighbor search (KNN) weighted sums can be applied in conjunction with the category trees of Figures 4 and 8. Performance metrics such as Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE) can be used.

[0317] The processor (110) can transmit the predicted product weight to an external device (3) (980). External devices may include, for example, a shipping cost calculation server, a warehouse management system (WMS), or an international shipping cost estimation system. Transmission methods may include HTTP API calls, message queue transmission (Kafka, RabbitMQ), or database updates. Quality of Service (QoS) control can be applied to minimize network delays, and in the event of a failure, a local prediction model can be used to provide temporary results, which can then be synchronized with the server results upon recovery.

[0318] Additionally, the product weight prediction device (1) may further include the following:

[0319] i) Security / Integrity: Enhanced audit trail with E2EE in input / transmission sections, request signing, WAL (Write-Ahead Logging)-based journaling and snapshots, and optional blockchain ledger recording.

[0320] ii) Performance optimization: Improved throughput with IMDB (In-Memory Database) resident cache, unstructured tensor-based segment operations, GPU / NPU acceleration, and bucketized batching.

[0321] iii) Operational availability: circuit breaker / retry / fallback policies, model / pipeline versioning, real-time quality monitoring.

[0322] iv) Model quality: calibration (temperature scaling), confidence score / interval calculation, data drift monitoring, and retraining trigger.

[0323] v) Extensibility: When new data types (e.g. 3D models, videos) appear, dedicated embedding modules can be added as plugins.

[0324] Additionally, the product weight prediction device (1) may further include the following for exception handling and branching.

[0325] i) Missing / mismatched essential data: Rule-based correction, re-collection attempt, request for user feedback.

[0326] ii) Multilingual / abnormal DOM: Apply multilingual embedding after language identification, apply backoff rules or fallback parser in case of DOM error.

[0327] iii) Combined delivery of multiple products: Individual predictions are combined, or processed using a combined delivery model based on the average of sentence embeddings (not shown).

[0328] Additionally, the product weight prediction device may further include the following for iterative learning and feedback loops: By feeding back actual delivery weights (labels) retrieved from an external device (3) or a logistics system as learning data, the prediction model / category tree parameters can be updated periodically / event-based, thereby improving accuracy and robustness in the long term.

[0329] The product weight prediction method according to this embodiment has the following benefits. First, it improves accuracy. By applying a customized embedding model according to data type, feature extraction performance can be maximized. Second, it improves processing efficiency. Parallel processing of text and image data can accelerate prediction speed. Third, it improves scalability. Even when new data types (e.g., 3D models, videos) emerge, dedicated embedding models can be added to address them. Fourth, it facilitates practical application. Its integration with external systems allows immediate application in a variety of tasks, including logistics, delivery, and inventory management.

[0330]

[0331] Figure 10 illustrates a method for predicting product volume / weight and calculating international shipping costs according to one embodiment.

[0332] Referring to FIG. 10, the processor (110) forms a closed loop structure that continuously improves performance by (i) crawling / extracting and formatting data from an online product page (S1010, S1020), (ii) performing AI volume / weight prediction learning (S1030), (iii) performing real-time prediction calculation (S1040) and calculating / outputting international shipping costs (S1050, S1060), (iv) performing on-site measurement and storing the measurement DB (S1070, S1080), and (v) verifying the predicted value (S1090). In particular, the present embodiment is characterized in that, when a user link is input, inference is immediately made with the latest stable version of the model that is resident or stored in memory without going through the learning step (S1030).

[0333] The communication interface (200) can receive single or multiple product page links (URLs) from a user or an external system.

[0334] The processor (110) can verify the integrity of received data. For example, the processor (110) can verify URL canonicalization (protocol / host / query cleanup), deduplication, and whitelist / blacklist policies (normalization). Furthermore, the processor (110) can configure robots.txt, cookies, sessions, headers, etc., and prepare a headless browser if necessary (accessibility check).

[0335] The processor (110) can crawl HTML data and extract data (S1010).

[0336] The processor (110) obtains HTML DOM, text descriptions, image URLs / binaries, prices, seller / shipping information, review meta, etc. from "online product pages" using a crawler / scraper. In the case of dynamically rendered pages, the DOM can be obtained after rendering with a headless browser.

[0337] For example, the processor (110) can remove advertisements, notices, and recommendations and extract only the core product description area (area refinement). Furthermore, communication and storage can be protected with end-to-end encryption (E2EE) and request signing / integrity verification. Furthermore, if the latest extract for the same URL exists in the cache, this step can be skipped and the process can proceed directly to S1040 (prediction operation).

[0338] The processor (110) can analyze and format data (S1020).

[0339] The processor (110) can generate a structured schema through unit, currency, and date standardization, option and quantity parsing, and category code mapping. Text is tokenized / stopword removed / entityified, and images are resized, normalized, and augmented as needed. If there are multiple links, they are treated as the same order and a product list is constructed. If the number of products is variable, they are packed into a ragged tensor or packed / nested sequence (padding is unnecessary). Depending on the operating policy, a fixed K+ padding can be used in conjunction, and anything exceeding K is summarized into a "tail bundle slot" (including the average, number, and total price of excess product embeddings).

[0340] The processor (110) can learn volume or weight using an artificial intelligence model (S1030).

[0341] According to one embodiment, the processor (110) may execute a learning module that includes the following four learning paths in parallel or selectively to support a combined delivery (multiple product) situation.

[0342] The first path is a single shipment weight prediction-based path (A). The processor (110) trains individual weight predictors (regression heads) for each product, and for the combined shipment label, the individual predicted values ​​are combined to calculate a loss to match the order weight label.

[0343] The second path is the sentence embedding average-based path (B). The processor (110) generates text / image embeddings (e.g., 768 dimensions) for each product, averages them to create a single representative vector

[0768] , and learns order weights using a single regressor.

[0344] For example, the processor (110) generates a text embedding (e.g., KoBART / BERT) and an image embedding (e.g., EfficientNet / ViT) for each product i (i=1…N) included in each order, and combines the two embeddings (linear projection or weighted sum after concatenation) if necessary to generate a product-specific representative vector v. i ∈R 768 Afterwards, average pooling is performed as follows to obtain the order unit representative vector ∈R 768 , and input it into a single regressor (MLP head) to learn / predict the order weight.

[0345] The input may be in the form [N,768], and the output may be in the form

[0768] . For example, if there are 9 products, the embeddings of each product can be averaged and considered as a single representative vector. As in "[9,768]⇒

[0768] ", the processor (110) can predict the weight with a single regressor.

[0346] The processor (110) may be configured to be a ragged tensor or to be padded to a maximum length of K and then masked by averaging, even if the number of products N per order is different. The effect of unnecessary padding can be removed by applying (variable count processing). m i is a mask value indicating whether the i-th position is an actual product (=valid), and v i is the embedding vector of the ith product.

[0347] In addition, the processor (110) weights the price, quantity, category reliability, and attention scores instead of a simple average. i Use as can be applied. w i is the weight indicating the importance of the i-th product, and v i is the embedding vector of the ith product. This improves accuracy by increasing the reflection of high-price and high-weight items (weighted average).

[0348] Additionally, the processor (110) can stabilize the variance by selectively applying L2 normalization, layer normalization, and dropout before / after the average.

[0349] In addition, the processor (110) may include a multilayer perceptron (MLP)-based regressor that receives a preprocessed embedding vector (e.g.,

[0768] ) as input and calculates a weight. The regressor may be composed of, for example, an input layer-a hidden layer-an output layer, and the number of nodes in the hidden layer may be set to 256, and the output layer may be composed of 1 node that outputs a scalar value (

[0768] →

[0256] → [1]). At least one of ReLU or GeLU may be applied as an activation function in the hidden layer, and dropout may be further applied to prevent overfitting (the dropout ratio may be set to 0.1 to 0.3, etc., depending on the implementation). Batch normalization, residual linkage, weight decay (L2 regularization), etc. may be additionally applied as needed. The learning loss is optimized based on MAE / RMSE, and MAPE is monitored as an auxiliary indicator when necessary.

[0350] This second path is permutation-invariant, computationally lightweight, and memory-efficient, making it suitable for combined shipping (multiple items). However, because subtle differences between individual items can be diluted, performance can be improved when combined with key item weighting or attention pooling.

[0351] The third path is a fixed K input (e.g., 9 items) + padding path (C). For example, the processor (110) sorts an order into the shape [K, 768] (in the example of K=9). If the number of items is less than K, 0 padding is applied, and if it exceeds K, the top K items in importance (quantity, price, and category weighting) are selected or summarized into a 'tail bundle slot'. The processor (110) trains a predictor using one of RNN / Transformer / MLP.

[0352] The fourth path is a variable-length path (D) based on ragged tensors. The processor (110) inputs a sequence with different numbers of products per order as an unstructured tensor (ragged / packed), summarizes it using LSTM / GRU or Attention Pooling, and trains it using a regression head. No padding is used and a length mask is applied. The processor (110) inputs and uses the embedding results for each product, and can dynamically use LSTM inputs depending on the number of products belonging to a single order ID.

[0353] The processor (110) can integrate the outputs of the above paths. The processor (110) learns to select or weight one or more of A to D according to order length, quality, and category uncertainty.

[0354] The processor (110) minimizes padding overhead even in variable length / multiple product orders, and secures a balance between accuracy and delay by selecting the optimal path for each situation.

[0355]

[0356] The processor (110) can perform the following operations to predict the volume or weight of a product in real time using the latest model that has been pre-learned and distributed (S1040).

[0357] The processor (110) receives a single or multiple product page links (URLs) and auxiliary information (quantity, options, category hints, etc.) as input, and performs URL normalization, accessibility check, and duplication removal.

[0358] The processor (110) collects HTML / text / image / table structure data from a web page when a cache miss occurs, and then performs advertisement / recommendation area removal, unit / currency / language normalization, and token cleansing to generate a structured input feature.

[0359] The processor (110) generates a language model embedding for text and a CNN / ViT embedding for images, and combines them with auxiliary features such as quantity, price, category, seller / brand, etc.

[0360] The processor (110) sorts the input into a ragged / packed sequence or a [K,768] fixed array (including padding / summary) according to the irregularity of the number of products per order.

[0361] The processor (110) uses the meta selector learned in S1030 to select or weight-combine one or more of the prediction paths A to D (individual summation, average embedding, fixed K, ragged) based on order characteristics (number of products, embedding quality score, category uncertainty, etc.).

[0362] The processor (110) calculates the weight of a product (or order) through a weight regression head using the representation of the selected path as input, and calculates the external dimensions in parallel through a dimension (L / W / H) head. The text / image results are integrated into a quality / uncertainty-based adaptive ensemble.

[0363] The processor (110) adds up the additional weights by reflecting the packaging rules (buffering material, box specifications, multi-item packaging rules), calculates the volumetric weight using the predicted L / W / H, and compares it with the actual weight.

[0364] The processor (110) applies the calibration parameters stored in S1030 to calculate the confidence interval (e.g., 95%) and confidence score of the predicted value.

[0365] The processor (110) calculates the international shipping cost (S1050).

[0366] The processor (110) generates a prediction result meta including predicted weight / volume weight, reliability, usage model / meta version, selected path, and category tree identifier, transmits it to S1050 (international shipping cost calculation), and provides it to an external device (3) in the form of an API.

[0367] The processor (110) can calculate international shipping costs by applying carrier / route / service policies (volumetric weight calculation, fuel / regional surcharges, insurance / quarantine). Furthermore, the processor (110) can perform parallel calculations for multiple carriers and then sort and filter candidates based on cost / expected arrival date / reliability.

[0368] The processor (110), external device (3), or user terminal can output international shipping costs (S1060).

[0369] The processor (110) can provide predicted weight / volume weight, confidence (e.g., 95% interval), model version / selected category tree, etc. to an external device (3) or a user terminal. Transmission is performed using at least one of HTTP API / message queue / DB update, and in case of a failure, temporary results can be provided using QoS control and a local fallback model, and can be synchronized with server results after recovery.

[0370] The processor (110), external device (3), or user terminal can measure the volume / weight of the predicted product (S1070).

[0371] The processor (110) can obtain actual measurements and, if possible, actual L / W / H from a logistics base. The data may include a timestamp, equipment ID, and quality flag.

[0372] The processor (110) or the product weight prediction device (1) can store the actual volume / weight data of the product in the DB (S1080).

[0373] Real-world data is stored in a persistent database using Write-Ahead Logging (WAL), and category / node-specific statistics are loaded into the IMDB cache and reused as initial and correction values ​​for the next prediction. The prediction time, model version, input URL, and order ID are recorded together to ensure traceability.

[0374] The processor (110) can verify the predicted value and improve the closed loop (S1090).

[0375] The processor (110) can compare the predicted value (S1040) with the measured value (S1070 / S1080) to calculate the Mean Absolute Error (MAE), the Root Mean Square Error (RMSE), the Mean Absolute Percentage Error (MAPE), and the bias. When the threshold is exceeded, the processor can continuously improve the model by automatically executing (a) calibration re-estimation, (b) ensemble weight re-adjustment, (c) category tree conversion / re-weighting, and (d) re-training trigger (calling S1030).

[0376] The processor (110) can record predicted input / output and usage versions in IMDB (in-memory DB) and log storage, and can generate a WAL-based integrity log so that it can be linked to the S1070 (actual measurement collection) and S1090 (verification) steps.

[0377] According to this step, the processor (110) (i) performs low-latency multimodal prediction with only single / multiple link inputs, (ii) supports both ragged / fixed K methods to stably process multiple product (combined delivery) scenarios, (iii) provides reliability applicable to field applications through packaging correction, volumetric weight calculation, and calibration, and (iv) can complete a cyclic improvement loop with S1070 and S1090 through log and IMDB records.

[0378]

[0379] Figure 11 is a drawing comparing the conventional cross-border purchasing and delivery agency procedures with the simplified procedures of the present invention.

[0380] Referring to Figure 11, the conventional procedure requires the following steps: (a) individual shopping mall membership registration and mobile phone authentication, (b) order / payment, (c) delivery request and combined delivery request, (d) receipt at logistics center, (e) weight / volume measurement, (f) confirmation of delivery fee at distribution center, (g) payment of international delivery fee, (h) shipment, (i) customs clearance, (j) domestic logistics delivery, and (k) receipt confirmation.

[0381] A simplified procedure according to the present invention is described below.

[0382] The processor (110) can receive a purchase request via a link from a user terminal. The processor (110) can receive one or more product links and options / quantities from the user terminal. The processor (110) can perform URL normalization, deduplication, and accessibility verification.

[0383] The processor (110) can then calculate the total shipping cost and process a single payment. The processor (110) can extract product information from the link's text and images and use predictive calculations to calculate the actual weight / volumetric weight. The processor (110) can then apply carrier-specific billing rules, exchange rates, taxes, and service fees to generate a total price estimate and charge the user the total purchase price, including shipping costs.

[0384] The processor (110) can authorize one payment from a user.

[0385] Once payment is completed, an order is automatically placed on the domestic shopping mall, and after the seller ships the product, it is measured and inspected upon arrival at the logistics center.

[0386] During the customs clearance stage, the processor (110) can automatically generate customs clearance documents using product / price information, issue a carrier label, and instruct international shipment. The product can then be shipped domestically.

[0387] The processor (110) can notify the user of the status value of each step via push notification / email. After the final domestic delivery is completed, the processor (110) provides a receipt / invoice and a tracking log, and the actual measurement data is stored for future verification and re-learning.

[0388] The present invention can provide the following benefits: (i) From a user perspective, the present invention can significantly reduce the difficulty and time of purchasing by condensing 7 to 10 steps into 3 steps. (ii) The present invention can remove entry barriers by enabling purchases through a legal intermediary mechanism without requiring mobile phone authentication or website registration. (iii) The present invention minimizes billing errors by using AI weight / volume prediction in total calculations, and simplifies payment through a single payment, preventing repayments and additional charges. (iv) The present invention can improve accuracy in the long term by forming a self-improving closed loop through automatic loading of actual data and prediction verification. (v) The present invention can simultaneously improve costs, risks, and customer service by incorporating bundle optimization, status tracking, and risk management.

[0389]

[0390] Figure 12 illustrates a comparison between a manual quotation flow according to a conventional method and a real-time quotation flow according to the present invention.

[0391] Referring to Figure 12, the conventional method involves (i) a user selecting a product from a shopping mall, copying the URL link, and (ii) submitting the product link to the shopping mall operator to place an order. (iii) The operator manually verifies the price, options, domestic shipping costs, and availability. (iv) The operator notifies the user of the verification results, (v) the user makes a payment, and (vi) the purchase of the requested product is processed.

[0392] The conventional method involves a repeated process of confirmation, response, and re-inquiry, causing delays from order request to payment.

[0393] According to the present invention, when a link and option are received, the processor (110) checks whether a recent extraction or prediction result for the same link exists in the cache (request reception). If a hit is found, an estimate is immediately generated; if a miss is found, the processor proceeds to the next step.

[0394] Then, the processor (110) uses an AI data scraper to parse text / image / structured tags (JSON-LD / OG, etc.) after page rendering (headless browser if necessary) to extract product names, options, prices, image URLs, and category candidates. Unnecessary areas are removed and units, currencies, and languages ​​are standardized (automatic extraction / standardization).

[0395] Then, the processor (110) matches the extraction results with the stored data and embedding similarity to reinforce missing attributes (e.g., category code, brand, detailed specifications) (matching / reinforcement).

[0396] Then, the processor (110) uses a multimodal model with text / image embedding and auxiliary features (quantity, price, category, etc.) as input to predict the actual weight or L / W / H (Length / Width / Height), and can calculate the final weight or volumetric weight by reflecting the packaging rules (weight / dimension prediction).

[0397] Then, the processor (110) can calculate the international shipping cost and total purchase cost in real time by applying the charging rules, taxes, and exchange rates of each carrier (total amount estimate calculation).

[0398] Then, the processor (110) can immediately present the total amount to the user and accept the order through a single payment (display / payment). Upon receiving the order, an automatic order is placed on the domestic shopping mall.

[0399] Then, the processor (110) notifies the status of all steps in real time, and the logistics center actual measurement data is stored and can be used for verification and re-learning (tracking and information accumulation).

[0400] The processor (110) of the present invention automatically recognizes product information required for calculating international shipping costs simply by entering a link, estimates weight / volume weight by considering combined shipping (multiple products), and then calculates the final chargeable weight by reflecting packaging rules. The processor (110) performs the following procedures.

[0401] In terms of product recognition / matching, the processor (110) can rerank candidate products using embeddings (e.g., 768 dimensions) generated from text and images and category tree meta. Furthermore, the processor (110) can apply a semi-automated operation policy that transfers the product to the operator review queue if the calculated confidence score falls below a preset threshold.

[0402] In terms of weight prediction (combined delivery response), the processor (110) can predict the order weight (or weight per product) by automatically selecting the optimal path or weighted combination among ragged input, fixed K (+padding), individual prediction summation, average embedding, and attention pooling based on the variability of the number of products in the order.

[0403] In terms of volumetric weight calculation and packaging correction, the processor (110) can calculate the volumetric weight using (i) the L / W / H (Length / Width / Height) predicted by the model or (ii) the item-specific dimensional specification table. For items that are prone to handling, such as glass and batteries, the processor (110) can apply an additional weight that reflects the cushioning material and double packaging rules to determine the billing standard weight (the larger value between the actual weight and the volumetric weight).

[0404] In terms of performance policy, the processor (110) can guarantee quotation generation within seconds by combining a pre-trained model with a large number of product / order cases and cache / in-memory processing (IMDB).

[0405] The product weight prediction device (1) of this invention can provide the following effects.

[0406] (i) The product weight prediction device (1) can provide an instant quote by presenting international shipping costs and total amounts in real time (or within seconds) with just a link input, thereby eliminating the payment decision-making delay that used to take 1-2 days. (ii) The product weight prediction device (1) can reduce billing errors through an integrated pipeline of product recognition, weight prediction, packaging correction, and volumetric weight calculation to improve accuracy. (iii) The product weight prediction device (1) can minimize human intervention through automatic extraction, matching, and payment / order orchestration to improve operational efficiency, and can switch only exceptional cases such as those below the reliability threshold or prohibited items to human inspection. (iv) The product weight prediction device (1) can utilize a self-improvement loop to continuously accumulate logistics measurement and customs clearance results, periodically retrain and recalibrate the model, and automatically adjust the category tree / meta weight. (v) The product weight prediction device (1) flexibly responds to multiple links, multi-item combined delivery, and multi-national carrier rules to increase expandability and continuity, and can ensure service continuity with a local fallback model and QoS control in the event of a network failure.

[0407]

[0408] In this way, the product weight prediction device (1) according to one embodiment can calculate accurate shipping costs based on data in an overseas purchasing agency service, thereby enabling cost optimization and providing transparent cost information to consumers, thereby improving consumer satisfaction and promoting overseas purchasing.

[0409] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0410] < / eos> < / price> < / currency> < / price> < / name> < / eos> < / desc> < / price> < / name> < / bos>

Claims

A communication interface for receiving user input data related to an order for a product; and A product weight prediction device comprising at least one processor for extracting product information of the product based on the user input data on a web page basis, embedding the product information to generate preprocessed data, and inputting the preprocessed data into a learning model that uses the preprocessed data as an input value and the predicted weight of the product as an output value to predict the weight of the product. In claim 1, At least one processor, A product weight prediction device that generates the preprocessing data by inputting text information and image information included in the above product information into a text embedding model and an image embedding model, respectively. In claim 2, At least one processor, A product weight prediction device that determines the format of words embedded in the text information according to a text embedding technique, and inputs words processed to correspond to the format into the text embedding model to generate the preprocessed data. In claim 3, At least one processor, A product weight prediction device that generates encoding data by assigning a unique index to the number of products included in the above word, and inputs the encoding data into the text embedding model to generate the preprocessing data. In claim 4, At least one processor, A product weight prediction device that extracts a high-dimensional vector from a pre-selected layer of an image classification model included in the image embedding model and determines the high-dimensional vector as the preprocessing data. In claim 5, At least one processor, A product weight prediction device that generates the preprocessing data by expressing the embedding data, which is the output value of the text embedding model and the image embedding model, as a ragged tensor. In claim 1, At least one processor, A product weight prediction device that extracts product information from tags included in the web page based on an ensemble model based on supervised learning that combines a first learning model and a second learning model. In claim 1, At least one processor, A product weight prediction device that predicts the weight of the product based on the learning model that forms a category tree according to the attributes of the product included in the preprocessed data and classifies the preprocessed data according to the category tree. In claim 8, At least one processor, A product weight prediction device that generates multiple category trees, determines a category tree with the highest category prediction accuracy based on the user's input as a final category tree, and predicts the weight of the product based on the final category tree. In claim 9, At least one processor, Predict the weight of the product based on the learning model learned to minimize the error between the preprocessed data and the label data classified according to the final category tree, A product weight prediction device that derives the preprocessed data classified based on the above category tree as an intermediate output value and outputs the category of the product along with the weight of the product as a predicted value.

Images