Machine learning model and neural network for predicting object characteristics from digital image similarity

Abstract

Systems, methods, and other embodiments for predicting future characteristics of a target object / product based on a digital target image are described. In one embodiment, the method includes a machine learning model identifying a set of similar known product images by comparing a target product image to a group of known product images. For each similar known product image, a product attribute is retrieved that includes historical characteristic / event data associated with each similar known product image. A predictive characteristic model of the target product is generated based on a similarity score combined with the historical characteristic / event data associated with each similar known product image to generate the predictive characteristics of the target product.

Description

Background Art

[0001] Background of the Invention Analyzing and / or characterizing digital images is a highly complex task. This is especially true when an image database may contain millions of images of objects and may include millions of different versions of the objects. There may also be many images of the same object from different image views (e.g., front view, back view, side view, etc.). Extrapolating this to a situation where there are thousands of different objects in the image database results in an enormous amount of images. Needless to say, this enormous amount makes it difficult for humans to attempt to characterize them in any useful manner and further to use the images. Image analysis systems can be used to identify physical features in images (e.g., face recognition), but they do not make predictions beyond the physical features in the images.

Summary of the Invention

[0002] Summary In one aspect, a method implemented by a computing system includes inputting a digital target object image representing a target object into a machine learning model, comparing, by the machine learning model, at least digital pixel data of the target object image with digital pixel data from a group of known object images, and generating, by the machine learning model, a similarity score between the target object image and one or more known object images from the group of known object images based on at least the digital pixel data.

[0003] The machine learning model is configured to identify a set of similar object images based at least in part on similarity scores of one or more known object images. The method includes, for each similar object image in the set of similar object images, extracting an object attribute including historical event data associated with each similar object image, and generating a prediction property model including predicted properties of a target object represented in a target object image based on the historical event data of a given similar object combined with at least the similarity score of the given similar object, generating an electronic message having the predicted properties of the target object, and further including transmitting the electronic message to a remote computer.

[0004] In another aspect, a non-transitory computer-readable medium storing computer-executable instructions, which when executed by a computer including a processor, cause the computer to perform functions configured by the computer-executable instructions, and the computer inputting a digital target product image representing a target product into a machine learning model, and by the machine learning model, comparing the digital pixel data of the target product image with the digital pixel data of known product images to generate a similarity score between the target product image and each of the similar product images, thereby identifying a set of similar product images, and for each similar product image in the set of similar product images, extracting a product attribute including historical event data associated with each similar product image, and generating predicted properties of the target product represented in the target product image based on the historical event data of a given similar product combined with at least the similarity score of the given similar product, a non-transitory computer-readable medium is disclosed.

[0005] In another aspect, a computing system at least one processor connected to at least one memory, a non-transitory computer-readable medium storing stored instructions, which when executed by at least one processor, cause at least one processor Inputting a digital form target product image representing a target product into a machine learning model; comparing, by the machine learning model, at least digital pixel data of the target product image with digital pixel data from a group of known product images; generating, by the machine learning model, a similarity score between the product image and one or more known product images from the group of known product images based at least on the at least digital pixel data; identifying, by the machine learning model, a set of similar product images based at least in part on the similarity scores of the one or more known product images; extracting, for each similar product image in the set of similar product images, product attributes including historical event data associated with each similar product image; and generating a predictive characteristic model including predictive characteristics of the target product represented in the target product image based on the historical event data of a given similar product combined with at least the similarity score of the given similar product. A computing system is disclosed that includes a non-transitory computer-readable medium on which the above operations are performed.

[0006] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various systems, methods, and other embodiments of the present disclosure. It will be understood that the boundaries of the elements shown in the drawings (e.g., boxes, groups of boxes, or other shapes) represent one embodiment of the boundaries. In some embodiments, one element may be implemented as multiple elements, or multiple elements may be implemented as one element. In some embodiments, an element shown as an internal component of another element may be implemented as an external component, and vice versa. Further, the elements may not be drawn to scale.

Brief Description of the Drawings

[0007]

Fig. 1A

Fig. 1B

Fig. 2

Fig. 3

Embodiments for Carrying Out the Invention

[0008] Detailed Description Systems and methods for implementing a machine learning (ML) model configured using a neural network to predict future characteristics of a new or unknown object / product based on digital images of other similar objects / products are described herein. In one embodiment, the technology trains a machine learning model (including a neural network) using a set of digital images of known objects / products. The known images can include associated characteristics of each image (e.g., the orientation of the object or the image view, etc.), and the characteristics of its corresponding object / product shown in the image (e.g., the type of object, size, color, historical event data, etc.). There may be multiple images of the same object / product with different characteristics such as different orientations / views, colors, sizes, shapes, etc. of the object (e.g., dozens, hundreds, thousands, or more images), and there may be multiple images of multiple different versions of the same object / product with slightly different features. Since the machine learning model can learn from its own implementation and correct its own predictions, in one embodiment, the machine learning model is used.

[0009] In one embodiment, for a given image of a new object / product, the machine learning model is configured to identify a set of similar images from known images of known objects / products. The machine learning model can generate a similarity score between the new object / product image and the set of known images. The set of similar images from the set of known images is identified by the machine learning model by comparing the new object / product image with the known object / product images and determining which images are similar based at least on a threshold similarity score between each of the new object / product image and the known object / product images. In one embodiment, the similarity focuses on the similarity of the object / product itself within the image rather than any background or incidental items (e.g., people) within the image. Image portions associated with the background and incidental items can be removed to identify the actual object / product within the image before determining the similarity.

[0010] In one embodiment, the technology generates a predictive property model that predicts one or more future properties of a new object / product within a new object / product image based on known objects / products from the identified set of similar images. For example, the predictive property model may be generated based on the similarity scores of the similar images combined with historical property / event data associated with the known objects / products found in the identified similar images.

[0011] In one embodiment, the historical event data of a known object / product includes data on previous events or activities that have occurred to and / or in relation to the known object / product. Thus, the historical property data goes beyond the range of physical attributes of the object / product and provides increased technical and predictive improvements. Thus, the system generates a predictive model that predicts future events or activities that may occur to or be associated with the target object / product based at least on historical events / activities that have occurred to the similar objects / products identified within the identified similar images. Such predictive results were not previously made possible by prior systems.

[0012] In one embodiment, the ML model can generate an electronic message having predicted future characteristics of a new product and transmit the electronic message to a remote computer via network communication, such that the predicted future characteristics and / or other ML model outputs are made available and / or accessible to a user or other system.

[0013] Training of the Machine Learning Model Referring to FIGS. 1A and 1B, one embodiment of a method 100 for training a machine learning model using digital images of known objects / products is shown. The method 100 is implemented by a processor of a computer system that accesses and interacts with at least a memory and / or a data storage device. For example, the processor accesses at least the memory, reads and writes data thereto, and processes network communication to perform the operations of FIG. 1.

[0014] Focus on FIGS. 1A and 1B to train a neural network model, such as a machine learning model, to classify images of objects / products, such as clothing. In one embodiment, the neural network is trained to identify an image of an object / product based on a set of known images of known objects / products. The machine learning model is further trained and configured to analyze an input image (e.g., a digital target image of a target object / product to be analyzed) and generate predicted characteristics of the target object / product based on the identified similar images from the known images of the object / product. The similarity is determined based in part on a similarity score that reflects the degree to which any of the known images of the known products match the target image of the new product.

[0015] As shown in FIG. 1, in block 105, method 100 begins when multiple digital images of known products are input into a machine learning model. The digital images may be retrieved from a database or from memory in a computer system. Typically, tens of thousands (and in some cases, even millions) of images of known products can be input into the machine learning model as a training data set. In one embodiment, the images may be stored in the database as image pixel data, but the system is not limited to such a format. Other image formats may be used.

[0016] As will be discussed in more detail later, the known product images are used to train the neural network of the machine learning model. At this time, the model can be tested using a test set of new product images to evaluate how accurately the neural network has learned to classify the known product images and how accurately the neural network has identified the products found in the new product images.

[0017] In one embodiment, each of the images of the known and new products shows an individual product (e.g., a photo of a shirt, handbag, shoe, bag, car, etc.). Any background or incidental items (e.g., people / models) in the image are removed so that the actual object / product in the image can be identified.

[0018] In one embodiment, the ML model includes one or more digital image analysis algorithms configured to extract information from digital images. The digital images may be processed by pixel analysis. Object-based image analysis can also be applied to group pixels from the digital images into uniform objects that can be used to classify objects. For example, object statistics associated with the image objects, such as the size, geometric shape, texture, and context of the image objects, can be determined. The object statistics can also be used to classify the corresponding image objects. Thus, in one embodiment, the machine learning model can analyze the digital pixel data of the digital images and can include object-based image analysis for grouping pixels to identify products within the digital product images.

[0019] Furthermore, each of the images of the known products and the new products includes data (e.g., metadata) associated with the image that identifies physical attributes / characteristics or features related to that particular known product. The physical attributes may include one or a combination of the category / type of the product in the image, the subcategory of the product in the image, and the descriptive attributes of the product in the image (e.g., product size, color, length, brand, material, and / or other features). In one embodiment, the known product attributes can be enhanced using other information such as the brand or manufacturer of the product, the country of production, etc., which may be powerful for making future predictions about the product.

[0020] The known images can also include associated image characteristics of each image, such as the orientation of the object or the image view (e.g., front view / pose of the product, back view / pose of the product, side view / pose of the product, etc.). In this context, "pose" refers to an image showing a model wearing a clothing product, and the pose of the model represents the view of the clothing product.

[0021] In one embodiment, one or more images of a known product can include historical characteristic data, such as historical event data associated with the known product within the image. The historical event data of a known object / product can include data of previous events or activities that occurred to and / or in relation to the known object / product. Thus, the historical characteristic event data goes beyond the range of physical attributes (e.g., statistical attributes) of the object / product, such as size, shape, color, weight, etc.

[0022] In one embodiment, the historical event data can include the historical demand of the known product. For example, the historical demand may include the known unit quantity ordered / sold within a previous time period, the location / region where there is demand, etc. The historical demand may be defined based on seasonality, location, the name of the product associated with the known image, the historical unit price of the product associated with that known image, the brand, and / or other known characteristics associated with the product.

[0023] In block 110, a training set of known images (including the image pixel data of the known product, the image orientation data, and the associated characteristic data) is pre - processed before training the neural network to verify that the data is in the specified format of the machine - learning model.

[0024] In block 115, the pre - processed images are verified to ensure that the data is in the correct format.

[0025] In block 120, in one embodiment, first, a neural network (machine learning model) is constructed by creating a layer that converts a known product image from a two-dimensional pixel array to a one-dimensional pixel array. In this way, the two-dimensional pixel array is "unstacked" and then the pixels are "lined up". This step simply re-formats the data. After the pixels are flattened (lined up), the neural network includes a sequence consisting of two layers of "lined up" pixels. These pixel layers are fully connected neural network layers.

[0026] In block 125, the neural network (machine learning model ML model) is compiled. This can include specifying an optimizer for fitting the model and a loss function used for optimization. The compilation of the neural network is beyond the scope of the present disclosure and will not be described in detail.

[0027] In block 130, the training of the neural network is started by feeding training data (data from known product images) into the ML model. In one embodiment, the neural network can be trained using known product images. In another embodiment, the neural network (e.g., RESNET-50) can be used and trained by machine learning with respect to a random set of known product images.

[0028] In block 135, the ML model learns to associate training images. In one embodiment, the ML model is trained to associate images having the same image view / pose of a particular product. Thus, the ML mode can ignore or be different from the pose / view of a particular product in other known product images and can ignore the pose / view of a particular product in known images.

[0029] In one embodiment, if the pose / view in the image of a product of a known brand X is a front pose / view, the ML model will ignore other images of the same brand X product that have different poses / views, such as side poses and / or back poses (not associated as similar). Thus, an ML model is trained where a front view image of a product is not considered similar to a side view image of the same product.

[0030] Thus, the ML model learns and defines a set of similar product images that have the same pose / view of the same product. For example, a known product may have several images showing 4 - 5 poses (e.g., front, back, side) of the known product. This information can be used to exclude images with dissimilar poses from known product images with similar poses, thereby improving the accuracy of the model (and the speed of image processing) by removing unwanted poses from consideration.

[0031] In block 140, the ML model can be tested on how accurately it can identify known images of products similar to the input test image of a product and generate a similarity score for it. As used herein, a known image is "similar" to the input test image if the product in the known image is similar or the same as the product in the input test image. For example, if the input test image contains a photo of a red, patternless men's short - sleeved shirt, the known images identified as similar by the ML model should have photos of red, patternless men's short - sleeved shirts. The ML model is expected to generate a high similarity score for such closely matching product images.

[0032] To begin the test, for example, a request to perform a similarity prediction regarding a test set of product images and known product images is sent to the ML model. For example, the test set may include digital images of new products (and data regarding each image). The machine learning model can analyze the digital pixel data of the digital images and can include object-based image analysis to group pixels to identify products within the digital product images.

[0033] In one embodiment, the similarity prediction can include generating a similarity score that represents the statistical similarity between each of the test product image and the known product images.

[0034] In one embodiment, when using a neural net trained by machine learning on a random set of product images, the k-nearest neighbor algorithm (k-NNA) is performed on the deep features of the known images (features within one pixel layer before the last pixel layer). Based on the similarity score or k-NNA, the top number k (e.g., k = 5) of known images identified as being similar to the input test image are identified by the ML model.

[0035] Referring to FIG. 1B, the process from FIG. 1A continues. At block 145, the prediction from the ML model is verified to determine if the prediction of the similar image matches the test product within the input test image. This can include displaying the identified top similar images on a display device along with the corresponding test images. Thereafter, the displayed images can be visually compared to verify the accuracy of the identified similar images. To make appropriate learning adjustments to the similarity determination, the accuracy or inaccuracy of any identified image can be marked and fed back to the ML model.

[0036] In block 150, as the model is being trained, the loss and accuracy of the model can be determined until the desired accuracy of the model is reached. In one embodiment, the ML model is evaluated with respect to a loss function equal to the tolerance entropy loss and the triplet loss. For example, the tolerance entropy loss function is to evaluate the prediction results with respect to the training set. The triplet loss is used to incorporate information obtained from different poses / views of known images. In one embodiment, loss = tolerance entropy loss + triplet loss.

[0037] In block 155, with the ML model trained to identify similar images, additional test images of other new products can be input into the ML model to make predictions (similarity scores) regarding the similarity between the test images of other new products and the images of known products. At the end of block 155, the ML model should have been initially trained to identify similar images of similar products.

[0038] In block 160, the ML model is further trained to generate predicted future events or characteristics of the input product image based at least on the identified historical characteristics and / or events associated with the set of identified similar products from the identified similar images. In one embodiment, the known product images used by the ML model include (or are associated / linked with) the historical characteristic / event data of the known products shown in the known product images.

[0039] For example, the historical event data of a known object / product can include data on previous events or activities that occurred to and / or in relation to the known object / product. Thus, the historical event data goes beyond the range of physical attributes (e.g., statistical attributes) of the object / product such as size, shape, color, weight, etc.

[0040] Accordingly, for an input target object / product, the ML model is trained to generate a prediction model for predicting future characteristics / events that can occur to or be associated with the target object / product based on historical events / activities that occurred for similar objects / products identified within similar images identified by the ML model.

[0041] In one embodiment, the historical characteristics / events data includes historical demand data of known objects / products. The prediction model generated for the target object / product includes a predicted demand model representing the predicted future demand of the target object / product. Thus, for example, when a new product is developed, an image of the new product can be input into the ML model. The ML model generates a predicted demand for the new product based at least on the historical demand from a set of similar products identified from similar images by the ML model. The predicted future demand can then be used as guidance for the new product for actions such as determining the order quantity per product unit, the locations where there is demand, the time period (seasonality) when there is demand, and / or the assignment of an initial price for the new product. For example, the initial price of a new product can be inferred and predicted from the historical data of similar products.

[0042] In one embodiment, known product attributes can be enhanced using information such as brand, country of production, and / or other attributes that are considered powerful in the prediction and assignment of the initial price of a new product.

[0043] Operation of the machine learning model Upon receiving that the machine learning model was first trained, attention is hereby directed to FIG. 2. Referring to FIG. 2, one embodiment of method 200 for generating predicted characteristics of an unknown object is shown. In one embodiment, the predicted characteristics are described as the predicted demand for a product, but this is not limited thereto as the machine learning model can be trained based on other types of characteristics. Method 200 is implemented by a trained ML model that includes at least a processor of a computer system that accesses and interacts with a memory and / or a data storage device. For example, the processor accesses at least the memory, reads and writes data thereto, and processes network communications to perform the operations of FIG. 2.

[0044] Referring to FIG. 2, at block 210, method 200 begins when a digital product image of a new product is input into the machine learning model. The input image of the product is also referred to as a target image or a target product image. In one embodiment, the machine learning model can analyze the digital pixel data of the target image and include object-based image analysis for grouping pixels to identify a new product within the digital product image.

[0045] Next, at block 220, the machine learning model identifies a set of similar product images from a database of known product images by comparing the digital image data of the target product image with the digital image data of the known product images in the database. In one embodiment, pixels and other digital information from the target product image are analyzed to identify the digital object statistics of the target product, and as a result, the target product features can be identified.

[0046] The ML model compares the new product in the target image with the products in the known product images and attempts to match them based at least on the digital object statistics. The ML model generates a similarity score for each known product image based on how well the known product characteristics / attributes of the known product images statistically match the product characteristics / attributes of the target image.

[0047] As described above, the similarity prediction is a similarity score representing the statistical similarity between the target product image and the known product images. The final set of the best-matching known images is selected as the similar product images. The best-matching known images are, for example, the known images that received a similarity score exceeding a threshold similarity score.

[0048] In one embodiment, when using a neural net trained on a random set of product images, the k-nearest neighbor algorithm (k-NNA) is performed on the deep features of the images (e.g., features within one pixel layer before the last pixel layer). Based on the threshold similarity score or k-NNA, the top k (e.g., k = 5) known product images similar to the target product image can be identified.

[0049] Furthermore, as described above, in one embodiment, known images that include products the same as or similar to the target product image but have different poses / views of the products can be excluded from the final set of similar images. This reduces computing resources and processor time.

[0050] Returning to FIG. 2, in block 230, the data associated with each identified similar image from the final set of similar images is retrieved. For example, the machine learning model retrieves the historical characteristics / events (described above) including the historical demand data (in one embodiment) associated with each similar product image. In one embodiment, other product attributes / characteristics of the similar images can also include the product image category, product image subcategory, and the descriptive attributes of the product image.

[0051] In one embodiment, for example, the historical demand data of similar known products can include one or a combination of one or more other types of historical data related to the product. For example, the historical demand data may include the number of units ordered / sold within a certain time period, the locations where there is demand, seasonality, historical price, price elasticity, and / or base demand, etc.

[0052] In block 240, a machine learning model generates a predictive characteristic model of a target product within a target product image based at least on historical demand data of similar products associated with the identified similar images. In one embodiment, the predictive characteristic model may be a predictive demand model of the target product. The predictive characteristic / demand model may include a predicted demand based on the type and amount of demand that has occurred in the past for each known product within a final set of known products identified as being similar to the target product.

[0053] In one embodiment, the predictive demand model is generated by using a similarity score of each identified similar known product image combined with (e.g., weighted thereby) the historical demand data associated with the corresponding similar known product. For example, historical demand from more similar known products (e.g., better / higher similarity scores) is given more weight in the predicted demand than historical demand from less similar known products (e.g., lower similarity scores). One example is shown in Equation 1 below.

[0054] In this way, the predictive demand model of the target product predicts future / anticipated demand for the target product, including, for example, any one or combination of the number of orders per product unit, the regional location per product unit, the seasonality of demand, and / or the initial product price. Thus, the predictive demand model can be used as a guide for planning, ordering for, and adjusting the time, place, and number of product units ordered, shipped, and / or placed in different regions (per location), and / or the estimated initial price for a new target product. Thus, the initial price of a new target product without known market data can be inferred and predicted from the historical data of the identified similar products by the ML model.

[0055] In one embodiment, the ML model can be configured to generate a predicted demand model as follows. Using the similarity scores of known products associated with the identified similar known product images, the predicted demand model is trained to evaluate the predictive ability of the similarity scores using the known product images associated with the known products.

[0056] For example, assume that a new product (i) associated with a new product image is determined to be similar to known products (j1, j2,... jk) associated with known product images. The similarity score between the new product i and the known product j1 can be expressed as sim_i_j1. Each similarity score can be used to weight the corresponding historical data (historical demand) of the similar known products. Therefore, a larger similarity score increases the influence of the corresponding historical demand data, and vice versa. An example of the predicted demand model is expressed in Equation 1.

[0057] Equation 1 Demand for new product (i) at store s = function (sim_i_j1 * demand for known product j1), (sim_i_j2 * demand for known product j2),... (sim_i_jk * demand for known product jk) In one embodiment, the predicted demand model can be generated for specific selected demand characteristics. For example, one predicted characteristic model may be generated to predict the initial price of a new product when the "demand for known products" is based on the historical unit price associated with the historical demand. Another prediction model may be generated to predict the demand volume when the "demand for known products" is the historical demand volume. The predicted demands can be combined into a final predicted demand model for identifying multiple predicted future characteristics of the target product.

[0058] In one embodiment, the predicted demand model can be enhanced or otherwise adjusted using other product features of similar known products such as category, seasonality, etc. In one embodiment, the generated demand model for the target product may be a linear additive model.

[0059] In another embodiment, selected brands of products identified as known products similar to the new target product (e.g., competing brand products) can be considered and used to adjust the predicted demand model. For example, the predicted demand model (e.g., from Equation 1) can be adjusted by adding a negative number of the combined / multiplied value of the historical demand of the competing product to the similarity score of the competing product. In other words, the predicted demand model can be negatively adjusted by the similarity score and the historical demand combination of the similar images associated with the selected brand (e.g., competing brand).

[0060] In block 250, after the predicted demand model is generated, the machine learning model can output a prediction model for the new product. The ML model can also generate an electronic message containing the content of the predicted demand for the new product. As described above, the predicted demand model can be used as a guideline for planning, ordering for the new target product, and adjusting the time, location, and quantity when product units are ordered, shipped, and / or placed in different regions (per location), and / or for the estimated initial price. Thus, in one embodiment, the initial price of a new target product without known market data can be predicted in the predicted demand model by the ML model as described above.

[0061] In block 260, the machine learning model can send the prediction model and / or the electronic message to a remote computer via network communication across the network, and as a result, the prediction model becomes available and / or accessible by the user or other systems. The machine learning model can also generate and display the prediction model on a display device.

[0062] As another example, the method of FIG. 2 is described when the new target product is a new shirt design or style. Referring to FIG. 2, at block 210, method 200 begins when a product image of the new shirt design or style is input into the machine learning model.

[0063]

[0062] Next, at block 220, the machine learning model identifies a set of known shirt images that are similar by comparing the new shirt image to a database of known images. The database of known images can be filtered based on the product category so that the comparison step and the generation of the similarity score are limited to only known images having a shirt category. Further, as described above, in one embodiment, known images of shirts having a pose / view different from the pose / view of the input shirt image can also be excluded.

[0064] For the remaining known shirt images, the ML model generates a respective similarity score as compared to the new shirt image. As described above, in one embodiment, each similarity score represents the statistical similarity between the new shirt image and one known shirt image.

[0065] The similarity scores of the known shirt images can be filtered based at least on a threshold similarity score. Using the threshold, a top set consisting of N (e.g., 4, 5, or 6, etc.) similar images can be selected from the shirt images that are most closely similar (e.g., the highest / maximum similarity score). The top set forms the identified set of similar images.

[0066] Returning to FIG. 2, in block 230, the machine learning model extracts product attributes that include historical demand data associated with each image in the identified set of similar images. In one embodiment, the product attributes can also include a shirt category (i.e., tops), a product item image subcategory (i.e., blouses), and descriptive attributes of the shirt image (i.e., sleeve length, sleeveless, etc.). In another embodiment, the images of known shirts can also include data related to the demand for the shirts associated with the known shirt images, based on seasonality, the name or brand of the shirt associated with the known shirt image, the historical unit price of the shirt associated with the known shirt product image, and the like.

[0067] In block 240, the machine learning model generates a predicted demand model for the new shirt in the new shirt image. In one embodiment, the predicted demand model is generated using a similarity score of similar known images (from the identified similar images) combined with the historical demand data associated with those similar known shirt images.

[0068] For example, assume that a new shirt (i) shown in the shirt image input to the ML model is determined to be similar to known shirts (j1, j2,... jk) associated with known shirt images. The similarity score between the new shirt (i) and the known shirt j1 can be expressed as sim_i_j1. An example of the predicted demand model for the new shirt is expressed in Equation 2.

[0069] Equation 2 Demand for new shirt (i) at store s = function(sim_i_j1 * demand for known shirt j1), (sim_i_j2 * demand for known shirt j2),... (sim_i_jk * demand for known shirt jk) In one embodiment, the predicted demand model can be enhanced using other product features of similar shirts, such as category, seasonality, and the like. In one embodiment, the generated demand model for the target product can be a linear additive model.

[0070] In block 250, after the predicted demand model is generated, a machine learning (ML) model can output the prediction model of the new shirt to a remote computer and / or a display device, and as a result, the prediction model becomes available and / or accessible by the user or other systems. The ML model can also generate an electronic message containing the content of the predicted demand for the new shirt. The electronic message can then be sent via a communication network to a destination (e.g., an email address, text, SMS message, online account, etc.) accessible by a remote computer / device, and as a result, the predicted demand / model becomes available and / or accessible by the user or other systems.

[0071] Prediction Model for Inventory Management - Robot Mechanism In one embodiment, the predicted demand model can be used to manage the inventory of related products. For example, after the predicted demand model and / or the electronic message is received at a remote computer, the predicted demand model can be configured such that instructions for assigning predicted characteristics (e.g., predicted initial price) to data records associated with a target product (e.g., a new shirt) as identified in the predicted demand model are sent to one or more inventory management systems and / or inventory databases.

[0072] In another embodiment, the predicted demand model can be configured to generate instructions for an order and / or cause a robotic mechanism to retrieve a certain quantity of the target product. For example, the order can be generated to include the quantity of the target product as defined in the predicted demand model of the target product. The order can then be prepared and fulfilled by retrieving that quantity of the target product and transporting that quantity to a destination as identified in the predicted demand model (if applicable). The remote computer can be associated with a warehouse (or fulfillment center) or a distribution channel (retail store) and manage the inventory.

[0073] Referring to an example of new shirts, after a predicted demand model and / or an electronic message is received from a remote computer, an order is fulfilled by retrieving a certain quantity of new shirts. If the order is sent to a warehouse (or fulfillment center), the warehouse management software receives the order, and the system in the warehouse retrieves the unit of new shirts from the unit storage location, packages the new shirts, and prepares them for shipping to fulfill the order. If the order is sent to a distribution channel (store), the order may be fulfilled at the store. In one embodiment, the system is at least partially controlled by instructions within the order.

[0074] In one embodiment, a system for processing an order can include, for example, an automated robotic machine or mechanism configured to locate and retrieve a target product from a warehouse or storage location based on the order. The retrieved target product can then be delivered by the robotic mechanism to an automated packaging mechanism that packages the retrieved new shirts within the warehouse.

[0075] In one embodiment, the robotic mechanism can include one or more robots configured to navigate throughout a given warehouse or store, locate and retrieve items, and transport the items to a destination. Each robot can include at least a body structure, a power source, a control interface, a wired / wireless communication interface, a drive device for moving the robot, a navigation device, one or more sensors, and / or a balancing device. Of course, the robots may be configured in various manners, and multiple different types of robots may operate together within the robotic mechanism of the warehouse or store.

[0076] In one or more embodiments, the robotic mechanism may include one or more of the following systems. An automated guided vehicle (AGV) for transporting materials, supplies, and inventory within a warehouse or store facility. The AGV can be configured to automatically navigate a warehouse or store facility by following a defined path marked by wires embedded in the floor, magnetic tapes, tracks, sensors, or other physical guides. The AGV can also be navigated by a defined map of the warehouse or store based on a coordinate system and the tracked location of the AGV. Cameras can also be used to navigate the AGV.

[0077] Another robotic mechanism can be an automated storage and retrieval system (AS / RS) that includes a group of computer control systems that automate inventory management and store / retrieve products from storage locations within a warehouse or store upon request. The AS / RS can operate as either a crane or a shuttle on fixed tracks and can traverse product aisles and vertical heights to remove or deposit items from storage locations. Another robotic mechanism is an articulated robotic arm that is a type of pick-and-place robot. These arms can move, rotate, grip / release, lift, and move items using, for example, multi-jointed limbs that are used to manipulate products.

[0078] After the retrieved product (shirt) is transported to the packaging area of the warehouse (block 270), the product (shirt) related to the order is packaged by an automated system. This may include boxing software that determines the amount, size, and type of container required to package the order, and / or a bagging machine that helps accelerate the packaging operation. As described above, after the retrieved product (shirt) is transported to the distribution channel (retailer's store), the retrieved product (shirt) can then be transported to the display area of the store, and the retrieved product (shirt) can be placed in the designated display area of the store.

[0079] The packaged order is transferred to the shipper, or if the retailer has its own shipping process, the retailer's own truck picks up the shipment. The status of the order is changed to "in transit" within the inventory management system. The order reaches the designated store. Finally, the computerized management system software then updates the inventory position of the retrieved product (shirt) in the distribution channel.

[0080] On the other hand, if the order is sent to the manufacturer, the sequence of steps is similar, except that the manufacturer serves as the fulfillment center and takes on the role of fulfilling the order.

[0081] Computing Device Embodiments FIG. 3 shows an exemplary computing device configured and / or programmed as a dedicated computing device having one or more of the exemplary systems and methods described herein and / or equivalents. The exemplary computing device may be a computer 300 including at least one hardware processor 302, a memory 304, and an input / output port 310 operably connected by a bus 308. In one example, the computer 300 is implemented using a prediction system or logic 330 configured to facilitate a machine learning (ML) model configured to predict future characteristics of new or unknown objects / products as described with reference to FIGS. 1A, 1B, and / or 2.

[0082] In different examples, the logic 330 may be implemented in hardware, a non-transitory computer-readable medium 337 storing instructions, firmware, and / or combinations thereof. Although the logic 330 is shown as a hardware component attached to the bus 308, it should be understood that in other embodiments, the logic 330 may be implemented in the processor 302, stored in the memory 304, or stored on the disk 306.

[0083] In one embodiment, the logic 330 or computer is a means (e.g., structure, hardware, non-transitory computer-readable medium, firmware) for performing the described operations. In some embodiments, the computing device may be a server operating within a cloud computing system, a server configured in a software-as-a-service (SaaS) architecture, a smartphone, a laptop, a tablet computing device, or the like.

[0084] The means may be implemented, for example, as an ASIC programmed to predict future characteristics such as product demand as described herein. The means may also be implemented as stored computer-executable instructions temporarily stored in the memory 304 and then presented to the computer 300 as data 316 to be executed by the processor 302.

[0085] The logic 330 may also provide means (e.g., hardware, non-transitory computer-readable medium storing executable instructions, firmware) for predicting the characteristics of new or unknown objects as described herein.

[0086] To generally describe an exemplary configuration of the computer 300, the processor 302 may be a variety of processors, including dual microprocessors and other multiprocessor architectures. The memory 304 may include volatile memory and / or non-volatile memory. The non-volatile memory may include, for example, ROM, PROM, and the like. The volatile memory may include, for example, RAM, SRAM, DRAM, and the like.

[0087] The memory disk 306 can be operably connected to the computer 300, for example, via an input / output (I / O) interface (such as a card, device) 318 and an input / output port 310 that are at least controlled by an I / O controller 340. The disk 306 can be, for example, a magnetic disk drive, a solid state disk drive, a floppy disk drive, a tape drive, a Zip drive, a flash memory card, a memory stick, etc. Further, the disk 306 can be a CD-ROM drive, a CD-R drive, a CD-RW drive, a DVD ROM, etc. The memory 304 can store, for example, a process 314 and / or data 316. The disk 306 and / or the memory 304 can store an operating system that controls and allocates the resources of the computer 300.

[0088] The computer 300 can interact with, control, and / or be controlled by an input / output (I / O) device via an I / O controller 340, an I / O interface 318, and an input / output port 310. The input / output device can include, for example, one or more displays 370, a printer 372 (such as an inkjet, laser, or 3D printer), an audio output device 374 (such as a speaker or headphones), a text input device 380 (such as a keyboard), a cursor control device 382 for pointing and selection input (such as a mouse, trackball, touch screen, joystick, pointing stick, electronic stylus, electronic pen tablet, etc.), an audio input device 384 (such as a microphone or external audio player), a video input device 386 (such as a video and still camera, external video player), an image scanner 388, a video card (not shown), the disk 306, a network device 320, etc. The input / output port 310 can include, for example, a serial port, a parallel port, and a USB port.

[0089] Computer 300 can operate within a network environment and can thus be connected to network device 320 via I / O interface 318 and / or I / O port 310. Through network device 320, computer 300 can interact with network 360. Through the network, computer 300 can be logically connected to remote computer 365. Networks with which computer 300 can interact include, but are not limited to, LANs, WANs, and other networks.

[0090] Definitions and Other Embodiments In another embodiment, the disclosed methods and / or their equivalents may be implemented using computer-executable instructions. Thus, in one embodiment, a non-transitory computer-readable / storage medium is configured with computer-executable instructions stored thereon that, when executed by a machine, cause the machine (and / or associated components) to implement the algorithms / executable applications for implementing the present methods. Exemplary machines include, but are not limited to, processors, computers, servers operating within a cloud computing system, servers configured within a service-type software (SaaS) architecture, smartphones, etc. In one embodiment, a computing device is implemented using one or more executable algorithms configured to implement any of the disclosed methods.

[0091] In one or more embodiments, the disclosed methods or their equivalents are implemented by either computer hardware configured to implement the present methods or computer instructions embodied within modules stored on a non-transitory computer-readable medium, the instructions being configured as executable algorithms that, when executed by at least a processor of a computing device, implement the present methods.

[0092] For the purpose of brevity of description, although the methods shown in the drawings are illustrated and described as a series of blocks of an algorithm, it should be understood that the methods are not limited by the order of the blocks. Some blocks can be performed in a different order than that illustrated and described, and / or concurrently with other blocks. Moreover, the blocks used to implement the exemplary methods may be fewer than all of the blocks illustrated. The blocks may be combined or separated into multiple operations / components. Further, additional and / or alternative methods may utilize additional operations not shown in the blocks.

[0093] The following includes definitions of selected terms used herein. The definitions include various examples and / or forms of components that fall within the scope of the terms and can be used for embodiments. The examples are not intended to be limiting. Both the singular and plural forms of the terms may be within the definitions.

[0094] References to "one embodiment", "an embodiment", "one example", "an example", etc. indicate that the embodiment or example so described may include a particular feature, structure, characteristic, property, element, or limitation, but not all embodiments or examples necessarily include that particular feature, structure, characteristic, property, element, or limitation. Further, when the phrase "in one embodiment" is used repeatedly, this does not necessarily refer to the same embodiment, although it may.

[0095] "Data structure", as used herein, is an organization of data within a computing system stored in memory, a storage device, or other computerized systems. A data structure can be, for example, any one of a data field, a data file, a data array, a data record, a database, a data table, a graph, a tree, a linked list, etc. A data structure can be formed from, and can include, many other data structures (e.g., a database includes many data records). According to other embodiments, other examples of data structures are possible.

[0096] "Computer-readable medium" or "computer storage medium", as used herein, refers to a non-transitory medium that stores instructions and / or data configured to perform one or more of the functions disclosed when executed. The data can, in some embodiments, function as instructions. A computer-readable medium can take forms including, but not limited to, non-volatile media and volatile media. Non-volatile media can include, for example, optical disks, magnetic disks, etc. Volatile media can include, for example, semiconductor memory, dynamic memory, etc. Common forms of computer-readable media include, but are not limited to, floppy disks, flexible disks, hard disks, magnetic tapes, other magnetic media, application specific integrated circuits (ASICs), programmable logic devices, compact disks (CDs), other optical media, random access memory (RAM), read only memory (ROM), memory chips or cards, memory sticks, solid state storage devices (SSDs), flash drives, and other media that can function together with a computer, a processor, or other electronic devices. Each type of media can include stored instructions of an algorithm configured to perform one or more of the functions disclosed and / or claimed when selected for implementation in one embodiment.

[0097] "Logic", as used in this specification, represents components implemented using computer or electrical hardware, non-transitory media storing instructions of executable applications or program modules, and / or combinations thereof, for performing any of the functions or operations disclosed herein and / or for enabling functions or operations from another logic, method, and / or system to be performed as disclosed herein. Equivalent logic may include firmware, microprocessors programmed using algorithms, discrete logic (e.g., ASIC), at least one circuit, analog circuits, digital circuits, programmed logic devices, memory devices containing instructions of algorithms, etc., any of which can be configured to perform one or more of the disclosed functions. In one embodiment, the logic may include one or more gates, combinations of gates, or other circuit components configured to perform one or more of the disclosed functions. Where multiple logics are described, it may be possible to incorporate the multiple logics into one logic. Similarly, where a single logic is described, it may be possible to distribute the single logic among multiple logics. In one embodiment, one or more of these logics is the corresponding structure associated with the implementation of the disclosed and / or claimed functions. The choice of the type of logic to implement may be based on the desired system conditions or specifications. For example, if higher speed is a consideration, hardware is selected to implement the function. If lower cost is a consideration, stored instructions / executable applications are selected to implement the function.

[0098] A "operable connection" or the connection by which an entity is "operably connected" is a connection through which signals, physical communication, and / or logical communication can be transmitted and / or received. An operable connection may include a physical interface, an electrical interface, and / or a data interface. An operable connection may include different combinations of interfaces and / or connections sufficient to enable operable control. For example, two entities can be operably connected to communicate signals with each other directly or through one or more intermediate entities (e.g., a processor, an operating system, logic, a non-transitory computer-readable medium). An operable connection can be created using logical and / or physical communication channels.

[0099] "User", as used herein, includes, but is not limited to, one or more persons, computers or other devices, or combinations thereof.

[0100] Although the disclosed embodiments have been described in considerable detail, the Applicant does not intend to limit the scope of the appended claims to such detail or in any way. Of course, it is not possible to describe all conceivable combinations of components or methods for the purpose of explaining various aspects of the subject matter. Accordingly, the present disclosure is not limited to the specific details or examples illustrated and described. Accordingly, the present disclosure is intended to encompass such alternative forms, modifications, and variations as fall within the scope of the appended claims and their equivalents.

[0101] To the extent that the term "comprising" or "including" is used in the detailed description or the claims, the term is intended to be inclusive in the same manner as the term "comprises" is construed when used as a transitional term in the claims.

[0102] The term "or" is intended to mean "A or B or both" within the scope of use in the detailed description or the claims (e.g., A or B). When the applicant intends to indicate "only A or B, but not both", the phrase "only A or B, but not both" will be used. Therefore, the use of the term "or" in this specification is inclusive and not exclusive.

Claims

1. A method implemented by a computing system comprising at least one processor, the method comprising: inputting a digital target object image representing a target object into a machine learning model; comparing, by the machine learning model, at least digital pixel data of the target object image with digital pixel data from a group of known object images; generating, by the machine learning model, a similarity score between the target object image and one or more known object images from the group of known object images based at least on the digital pixel data; identifying, by the machine learning model, a set of similar object images based at least in part on the similarity score of the one or more known object images; extracting, for each similar object image in the set of similar object images, an object attribute including historical event data associated with each similar object image; generating a prediction property model including predicted properties of the target object represented in the target object image based on the historical event data of a given similar object combined with at least the similarity score of the given similar object; generating an electronic message having the predicted properties of the target object; transmitting the electronic message to a remote computer A method comprising the steps above.

2. The machine learning model is configured using a neural network model trained to classify images of objects, and the method according to claim 1, wherein the machine learning model adjusts the predicted properties of the target object using additional object attributes from the set of similar object images before generating the electronic message.

3. The method according to claim 1 or claim 2, further comprising analyzing the digital pixel data of the object image by the machine learning model, including object-based image analysis for grouping pixels to identify the target object within the target object image.

4. The method according to any one of the preceding claims, wherein generating the prediction property model of the target object includes weighting the historical event data of a given similar object using the similarity score of the given similar object.

5. The target object is a target product, and generating the predicted properties includes When the target product (i) is associated with a target product image determined to be similar to known products (j1, j2,... jk) associated with the known product images, functions (sim_i_j1 * demand for known product j1), (sim_i_j2 * demand for known product j2) ,... (sim_i_jk * demand for known product jk) to generate the prediction characteristics so as to configure the prediction characteristic model as the predicted demand for the target product (i) in the first store (S). sim_i_j1, sim_i_j2,... and sim_i_jk represent the similarity scores between the target product (i) and each of the known products (j1, j2,... jk) from the set of similar product images, according to the method of claim 1.

6. The step of identifying further includes removing from the set of similar object images a similar object image that does not include an image orientation similar to the orientation of the image within the target object image, according to any of the preceding claims.

7. further including causing a robotic mechanism to retrieve a certain amount of the target object from a storage location based at least on the prediction characteristic model, according to any of the preceding claims.

8. A non-transitory computer-readable medium storing computer-executable instructions, which when executed by a computer including a processor, cause the computer to perform functions configured by the computer-executable instructions, and the instructions cause the computer to input a digital form target product image representing a target product into a machine learning model; by the machine learning model, compare the digital pixel data of the target product image with the digital pixel data of known product images, and identify a set of similar product images by generating a similarity score between the target product image and each of the similar product images; for each similar product image in the set of similar product images, retrieve product attributes including historical event data associated with each similar product image; generate prediction characteristics of the target product represented in the target product image based on the historical event data of the given similar product combined with at least the similarity score of the given similar product A non-transitory computer-readable medium that causes the above to be performed. **Claim 9** When at least executed by the above processor, further causes the processor to The non-transitory computer-readable medium according to claim 8, further comprising instructions for adjusting the predicted characteristics of the target product using additional product attributes from the set of similar product images. **Claim 10** When at least executed by the above processor, further causes the processor to The non-transitory computer-readable medium according to claim 8 or claim 9, further comprising instructions for analyzing the digital pixel data of the product image, including object-based image analysis for grouping pixels to identify the target product within the product image by the machine learning model. **Claim 11** When at least executed by the above processor, further causes the processor to The non-transitory computer-readable medium according to claim 10, further comprising instructions for negatively adjusting the predicted characteristics by the similarity score and a historical event combination of similar images associated with a selected brand of a known product. **Claim 12** When at least executed by the above processor, further causes the processor to When the target product (i) is associated with a target product image determined to be similar to known products (j1, j2,... jk) associated with the known product images, Functions (sim_i_j1 * demand for known product j1), (sim_i_j2 * demand for known product j2) ,... (sim_i_jk * demand for known product jk) to generate the predicted characteristics as the predicted demand for product (i) that is the target product in the first store (S), The non-transitory computer-readable medium according to any one of claims 8 to 11, wherein sim_i_j1, sim_i_j2,... and sim_i_jk represent the similarity scores between the target product (i) and each of the known products (j1, j2,... jk). **Claim 13** When at least executed by the above processor, further causes the processor to The non-transitory computer-readable medium according to any one of claims 8 to 12, further comprising instructions for removing from the set of similar product images similar product images that do not include an image orientation similar to the image orientation within the target product image. **Claim 14** When at least executed by the processor, cause the processor to The non-transitory computer-readable medium according to any one of claims 8 to 13, further comprising an instruction for causing the database to generate an instruction for assigning the prediction characteristic to a data record associated with the target product. **Claim 15** A computing system, At least one processor connected to at least one memory, and A non-transitory computer-readable medium including stored instructions Comprising When the instructions are executed by the at least one processor, the at least one processor is caused to Input a digital target product image representing the target product into a machine learning model; Compare at least the digital pixel data of the target product image with digital pixel data from a group of known product images by the machine learning model; Generate a similarity score between the product image and one or more known product images from the group of known product images based at least on the digital pixel data by the machine learning model; Identify a set of similar product images by the machine learning model based at least in part on the similarity score of the one or more known product images; For each similar product image in the set of similar product images, retrieve product attributes including historical event data associated with each similar product image; Generate a prediction characteristic model including prediction characteristics of the target product represented in the target product image based on the historical event data of the given similar product combined with at least the similarity score of the given similar product The computing system to perform. **Claim 16** When at least executed by the processor, cause the processor to The computing system according to claim 15, wherein the instructions further include instructions for causing the machine learning model to analyze the digital pixel data of the product image, including object-based image analysis for grouping pixels to identify the target product within the product image. **Claim 17** When at least executed by the processor, cause the processor to Generate an electronic message having the prediction characteristics of the target product sending the electronic message to a remote computer The computing system according to claim 15 or claim 16, further comprising an instruction to cause the above to be performed. **Claim 18** When the instruction for generating the prediction characteristic model is executed by at least the processor, the processor is caused to The computing system according to any one of claims 15 to 17, further comprising an instruction to cause the processor to combine the similarity score with the historical event data of a given similar product by weighting the historical event data using the similarity score. **Claim 19** When the instruction is executed by at least the processor, the processor is caused to The computing system according to any one of claims 15 to 18, further comprising an instruction to cause the processor to remove from the set of similar product images a similar product image that does not include an image orientation similar to the image orientation within the target product image. **Claim 20** When the instruction is executed by at least the processor, the processor is caused to The computing system according to any one of claims 15 to 19, further comprising an instruction to cause the processor to negatively adjust the prediction characteristic by the similarity score and a historical event combination of similar images associated with a selected brand of a known product.

Images