Machine learning model and neural network for predicting object characteristics according to digital image similarity
Through machine learning models, identify similar image sets in a huge image database and predict future characteristics, the problem of object/product characteristic prediction in the image database is solved, and efficient image analysis and prediction capabilities are achieved.
Patent Information
- Application Number
- CN202380035822.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-05-20
- Filing Date
- 2023-04-25
- Publication Date
- 2024-12-06
AI Technical Summary
In a huge image database, it is difficult to identify and predict physical features and future characteristics in images, especially when the number of images is huge and the same object has multiple views, it is difficult to effectively characterize and analyze the prior art.
Through machine learning models, train with digital images of known objects/products, generate similarity scores, identify similar image sets, and predict future characteristics of target objects/products based on historical event data of similar images.
It realizes effective representation and future characteristics prediction of objects/products in huge image databases, improves analysis accuracy and processing efficiency, and exceeds the physical attribute prediction capabilities of traditional systems.
Smart Images

Figure CN119096274A_ABST
Abstract
Description
Background Art
[0001] Analyzing and / or characterizing digital images is a very complex task. This is especially true when the image database includes millions of images of objects and may contain millions of different versions of the objects. The same object may also have many images from different image views (e.g., front view, back view, side view, etc.). Now extrapolate that there are thousands of different objects in the image database, and this becomes a huge amount of images. Such a large number makes it impossible for a human to even use these images, let alone try to characterize them in any useful way. Image analysis systems can be used to identify physical features in an image (e.g., facial recognition), but they do not make predictions beyond the physical features in the image. Summary of the invention
[0002] In one aspect, a method performed by a computing system is disclosed, comprising: inputting a target object image in digital form representing a target object into a machine learning model; comparing, by the machine learning model, at least digital pixel data of the target object image with digital pixel data of a set of known object images; and generating, by the machine learning model, a similarity score between the target object image and one or more known object images in the set of known object images based at least on the digital pixel data.
[0003] The machine learning model is configured to identify a set of similar object images based at least in part on the similarity scores of the one or more known object images. The method also includes: for each similar object image in the set of similar object images, retrieving object attributes including historical event data associated with each similar object image; generating a predictive characteristic model including a predictive characteristic for a target object represented in the target object image based at least on the historical event data of a given similar object in combination with the similarity score of the given similar object; generating an electronic message having the predictive characteristic of the target object, and transmitting the electronic message to a remote computer.
[0004] In another aspect, a non-transitory computer-readable medium is disclosed that stores computer-executable instructions that, when executed by a computer including a processor, cause the computer to perform functions configured by the computer-executable instructions, wherein the computer is caused to:
[0005] A target product image in digital form representing the target product is input into a machine learning model; the machine learning model identifies a set of similar product images by comparing digital pixel data of the target product image with digital pixel data of known product images and generating a similarity score between the target product image and each similar product image; for each similar product image in the set of similar product images, product attributes including historical event data associated with each similar product image are retrieved; and a predicted feature is generated for the target product represented in the target product image based at least on the historical event data of a given similar product in combination with the similarity score of the given similar product.
[0006] In another aspect, a computing system is disclosed, comprising:
[0007] at least one processor coupled to at least one memory;
[0008] A non-transitory computer-readable medium comprising instructions stored thereon, which, when executed by at least one processor, causes the at least one processor to:
[0009] A target product image in digital form representing the target product is input into a machine learning model; the machine learning model at least compares digital pixel data of the target product image with digital pixel data of a set of known product images; the machine learning model generates a similarity score between the product image and one or more known product images in the set of known product images based at least on the digital pixel data; the machine learning model identifies a set of similar product images based at least in part on the similarity scores of the one or more known product images; for each similar product image in the set of similar product images, retrieves product attributes including historical event data associated with each similar product image; and generates a predictive characteristic model including predictive characteristics for the target product represented in the target product image based at least on the historical event data of a given similar product in combination with the similarity score of the given similar product. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The accompanying drawings that are incorporated into and form a part of this specification illustrate various systems, methods and other embodiments of the present disclosure. It will be appreciated that the element boundaries (e.g., boxes, frame groups or other shapes) shown in the figures represent an embodiment of boundaries. In some embodiments, an element can be implemented as multiple elements, or multiple elements can be implemented as one element. In some embodiments, an element that is shown as an internal component of another element can be implemented as an external component, and vice versa. In addition, the element may not be drawn to scale.
[0011] Figure 1A and 1B One embodiment of a method for training a machine learning model using known digital images is illustrated.
[0012] Figure 2 One embodiment of a method for generating a machine learning model for predictive properties of an object / product is illustrated.
[0013] Figure 3 An embodiment of a special purpose computing system configured with the disclosed example systems and / or methods is illustrated. DETAILED DESCRIPTION
[0014] Described herein are systems and methods for implementing a machine learning (ML) model configured with a neural network to predict future characteristics of a new or unknown object / product based on digital images of other similar objects / products. In one embodiment, the present technology uses a collection of digital images of known objects / products to train a machine learning model (including a neural network). The known images may include associated characteristics of each image (e.g., object orientation or image view, etc.) and characteristics of its corresponding object / product displayed in the image (e.g., type of object, size, color, historical event data, etc.). The same object / product may have multiple images (e.g., tens, hundreds, thousands, or more images) with different characteristics, such as different object orientations / views, colors, sizes, shapes, etc., and multiple images of multiple different versions of the same object / product may have slightly different features. In one embodiment, a machine learning model is used because the machine learning model can learn from its own performance and modify its own predictions.
[0015] In one embodiment, for a given image of a new object / product, the machine learning model is configured to identify a set of similar images from known images of known objects / products. The machine learning model can generate a similarity score between the new object / product image and the set of similar known images. The machine learning model identifies the set of similar images from a collection of known images by comparing the new object / product image to the known object / product images and determining which images are similar based on at least a threshold similarity score between the new object / product image and each of the known object / product images. In one embodiment, the similarity focuses on the similarity of the object / product itself in the image, rather than any background or incidental items (e.g., people) in the image. Prior to determining the similarity, portions of the image associated with the background and incidental items can be removed to identify the actual object / product in the image.
[0016] In one embodiment, the present technology generates a predictive property model based on known objects / products from the identified set of similar images, which predicts one or more future properties of a new object / product in a new object / product image. For example, the predictive property model can be generated based on similarity scores of (one or more) similar images combined with historical property / event data associated with known objects / products appearing in the identified (one or more) similar images.
[0017] In one embodiment, the historical characteristic data of a known object / product includes data of previous events or activities that occurred on and / or associated with the known object / product. Thus, the historical characteristic data goes beyond the physical attributes of the object / product and provides enhanced techniques and prediction improvements. Thus, the present system generates a prediction model that predicts future events or activities that may occur on or associated with a target object / product based at least on historical events / activities that occurred on similar objects / products identified in identified similar images. Such prediction results were previously not possible with previous systems.
[0018] In one embodiment, the ML model may generate an electronic message having predicted future properties of a new product and transmit the electronic message to a remote computer via network communications so that the predicted future properties and / or other ML model outputs are available and / or accessible to a user or other system.
[0019] Training of machine learning models
[0020] refer to Figure 1A and 1B , illustrates one embodiment of a method 100 for training a machine learning model using digital images of known objects / products. The method 100 is performed by at least a processor of a computer system, which accesses and interacts with a memory and / or data storage device. For example, the processor at least accesses and reads / writes data to the memory and handles network communications to perform the actions of FIG. 1 .
[0021] To train a neural network model such as a machine learning model to classify images of objects / products such as clothing, note that Figure 1A and 1BIn one embodiment, a neural network is trained to recognize images of an object / product based on a collection of known images of the known objects / products. A machine learning model is further trained and configured to analyze an input image (e.g., a digital target image of a target object / product being analyzed) and generate predicted characteristics of the target object / product based on similar images identified from the known images of the object / product. Similarity is determined in part based on a similarity score that reflects how closely any known image of the known product matches the target image(s) of the new product.
[0022] As shown in FIG. 1 , at box 105, method 100 is initiated when a plurality of digital images of known products are input to a machine learning model. The digital images may be retrieved from a database or memory of a computer system. Typically, tens of thousands (or even potentially millions) of images of known products may be input to a machine learning model as a training data set. In one embodiment, the images may be stored in a database as image pixel data, but the system is not limited to this format. Other image formats may be used.
[0023] As will be discussed in detail later, known product images are used to train the neural network of the machine learning model. The model can then be tested using a test set of new product images to assess the accuracy with which the neural network has learned to classify known product images and the accuracy with which the neural network has identified the product(s) that appear in new product images.
[0024] In one embodiment, each image of known and new products shows a single product (e.g., a photo of a shirt, wallet, shoe, bag, car, etc.) Any background or incidental items (e.g., a person / model) in the image can be identified and removed so that the actual object / product in the image is identified.
[0025] In one embodiment, the ML model includes one or more digital image analysis algorithms configured to extract information from digital images. The digital images can be processed by pixel analysis. Object-based image analysis can also be applied, which groups pixels in the digital images into similar objects that can be used to classify the objects. Object statistics associated with the image objects can be determined, such as the size, geometry, texture, and background of the image objects. The object statistics can also be used to classify the corresponding image objects. Therefore, in one embodiment, the machine learning model can analyze the digital pixel data of the digital image and include object-based image analysis to group pixels to identify products in the digital product image.
[0026] In addition, each image of the known and new products includes data (e.g., metadata) associated with the image that identifies physical attributes / characteristics or features associated with that particular known product. The physical attributes may include one or more combinations of: category / type of product in the image, subcategory of product in the image, and descriptive attributes of the product in the image (e.g., product size, color, length, brand, material, and / or other features). In one embodiment, known product attributes may be augmented with other information (such as the brand or manufacturer of the product, country of manufacture), which may be influential in making future predictions about the product.
[0027] The known images may also include associated image characteristics of each image, such as object orientation or image view (e.g., front view / pose of a product, rear view / pose of a product, side view / pose of a product, etc.). In this context, "pose" refers to an image showing a model wearing an apparel product, and the pose of the model represents a view of the apparel product.
[0028] In one embodiment, one or more images of a known product may include historical characteristic data, such as historical event data associated with the known product in the image. The historical event data for a known object / product may include data of previous events or activities that occurred on and / or in relation to the known object / product. Thus, the historical characteristic event data goes beyond the physical attributes (e.g., static attributes) of the object / product, such as size, shape, color, weight, etc.
[0029] In one embodiment, the historical event data may include historical demand for a known product. For example, the historical demand may include known unit quantities ordered / sold in a previous time period, location / region of demand, etc. The historical demand may be defined based on seasonality, location, name of the product associated with the known image, historical unit price of the product associated with the known image, brand, and / or other known characteristics associated with the product, etc.
[0030] At box 110, before training the neural network, a training set of known images (including image pixel data, image orientation data, and associated characteristic data of known products) is pre-processed to verify whether the data is in the format specified by the machine learning model.
[0031] At block 115, the pre-processed image is verified to ensure that the data is in the correct format.
[0032] At box 120, in one embodiment, a neural network (machine learning model) is initially built by creating layers that transform a known product image from a two-dimensional array of pixels into a one-dimensional array of pixels. In this way, the two-dimensional array of pixels is "unstacked" and then the pixels are "lined up". This step simply reformats the data. After the pixels are flattened (lined), the neural network includes two layers of sequences of "lined" pixels. These pixel layers are fully connected neural network layers.
[0033] At block 125, the neural network (machine learning ML model) is compiled. This may include specifying an optimizer to fit the model and a loss function for optimization. Compiling the neural network is beyond the scope of this disclosure and is therefore not described in detail.
[0034] At block 130, training of the neural network is initiated by feeding training data (data from known product images) to the ML model. In one embodiment, the neural network may be trained using known product images. In another embodiment, a neural network (e.g., RESNET-50) may be used and trained on a random set of known product images through machine learning.
[0035] At block 135, the ML model learns to correlate the training images. In one embodiment, the ML model is trained to correlate images having the same image view / pose of a particular product. Thus, the ML model may ignore poses / views of a particular product in a known image that are not correlated or different than poses / views of the particular product in other known product images.
[0036] In one embodiment, if the pose / view in an image of a known brand X product is a front pose / view, then the ML model will ignore (and not associate as similar) other images of the same brand X product with different poses / views (such as side poses and / or back poses). Thus, the ML model is trained that a front view image of a product is not considered similar to a side view image of the same product.
[0037] From this, the ML model learns and defines sets of similar product images with the same pose / view of the same product. For example, a known product may have many images showing 4-5 poses (e.g., front, back, side) of the known product. This information can be used to exclude images with dissimilar poses from known product images with similar poses, which should improve the model's accuracy (and image processing speed) by removing unnecessary poses from consideration.
[0038] At block 140, the accuracy of the ML model in identifying and generating similarity scores for known product images that are similar to the input test image of the product can be tested. As used herein, a known image is "similar" to an input test image if the product in the known image is similar or identical to the product in the input test image. For example, if the input test image includes a photo of a men's short-sleeved shirt that is red in color and unlabeled, then a known image identified as similar by the ML model should have a photo of a men's short-sleeved shirt that is red in color and unlabeled. The ML model is expected to generate a high similarity score for such closely matching product images.
[0039] For example, to begin testing, a request is sent to the ML model to make similarity predictions for a test set of product images and known product images. For example, the test set may include digital images of new products (and data about each image). The machine learning model may analyze the digital pixel data of the digital images and include object-based image analysis to group pixels to identify products in the digital product images.
[0040] In one embodiment, similarity prediction may include generating a similarity score representing the statistical similarity between each test product image and the known product images.
[0041] In one embodiment, when using a neural network that has been trained on a random set of product images through machine learning, a k-nearest neighbor algorithm (k-NNA) is performed on the deep features of the known images (features in the layer of pixels before the last layer of pixels). Based on the similarity score or k-NNA, the ML model identifies the top k (e.g., k=5) known images that are identified as similar to the input test image.
[0042] refer to Figure 1B , Figure 1A Processing continues in . At box 145, the predictions from the ML model are verified to determine that the predictions of similar images match the test product in the input test image. This may include displaying the top few similar images identified along with the corresponding test image on a display device. The displayed images may then be visually compared to verify the accuracy of the identified similar images. The accuracy or inaccuracy of any identified image may be flagged and fed back to the ML model to make appropriate learning adjustments when determining similarity.
[0043] At block 150, as the model is trained, the loss and accuracy of the model may be determined until the desired accuracy of the model is reached. In one embodiment, the ML model is evaluated on a loss function that is equal to the cross entropy loss and the triplet loss. For example, the cross entropy loss function is used to evaluate the prediction results for the training set. The triplet loss is used to incorporate information obtained from different poses / views of a known image. In one embodiment, loss = cross entropy loss + triplet loss.
[0044] At box 155, after the ML model is trained to recognize similar images, additional test images of other new products can be input into the ML model to make predictions (similarity scores) about the similarity between the test images of other new products and images of known products. At the end of box 155, the ML model should initially be trained to recognize similar images of similar products.
[0045] At block 160, the ML model is further trained to generate predicted future events or characteristics for the input product image based at least on historical characteristics / events associated with the identified set of similar products from the identified similar images. In one embodiment, the known product images used by the ML model include (or are associated / linked to) historical characteristic / event data for the known products shown in the known product images.
[0046] For example, historical event data of a known object / product may include data of previous events or activities that occurred on and / or in connection with the known object / product. Thus, historical event data goes beyond the physical attributes (e.g., static attributes) of the object / product, such as size, shape, color, weight, etc.
[0047] Thus, for an input target object / product, the ML model is trained to generate a predictive model that predicts future characteristics / events that may occur on or be associated with the target object / product based on historical events / activities that occurred with similar objects / products identified in similar images identified by the ML model.
[0048] In one embodiment, the historical characteristic / event data includes historical demand data for a known object / product. The predictive model generated for the target object / product includes a predictive demand model that represents the predicted future demand for the target object / product. Thus, for example, when developing a new product, an image of the new product can be input to the ML model. The ML model generates a predicted demand for the new product based at least on the historical demand for a set of similar products identified by the ML model from similar images. The predicted future demand can then be used as a guide for the new product to determine actions such as the number of product units to order, the location of the demand, the time period of the demand (seasonality), and / or assigning an initial price to the new product. For example, the initial price of a new product can be inferred and predicted from historical data for similar products.
[0049] In one embodiment, known product attributes may be augmented with information such as brand, country of manufacture, and / or other attributes that are believed to be influential in predicting and assigning an initial price for a new product.
[0050] Operations of machine learning models
[0051] Once the machine learning model has been initially trained, attention is now turned to Figure 2 . refer to Figure 2 , illustrates one embodiment of a method 200 for generating predictive characteristics of an unknown object. In one embodiment, the predictive characteristics are described as predicted demand for a product, but are not limited thereto, as the machine learning model can be trained based on other types of characteristics. The method 200 is performed at least by a trained ML model, including a processor of a computer system that accesses and interacts with a memory and / or data storage device. For example, the processor at least accesses and reads / writes data to the memory, and handles network communications to perform Figure 2 action.
[0052] refer to Figure 2 At block 210, method 200 is initiated when a digital product image of a new product is input into a machine learning model. The input image of the product will also be referred to as a target image or a target product image. In one embodiment, the machine learning model may analyze digital pixel data of the target image and include object-based image analysis to group pixels to identify new products in the digital product image.
[0053] At block 220, the machine learning model then identifies a set of similar product images from the database of known product images by comparing the digital image data of the target product image with the digital image data of known product images in the database. In one embodiment, pixels and other digital information from the target product image are analyzed to identify digital object statistics of the target product so that target product features can be identified.
[0054] The ML model compares and attempts to match new products in the target image with products in the known product images based on at least the digital object statistics. The ML model generates a similarity score for each known product image based on how well known product features / attributes of the known product image statistically match the product features / attributes of the target image.
[0055] As discussed above, the similarity prediction is a similarity score that represents the statistical similarity between the target product image and the known product images. The final best matching set of known images is selected as the similar product images. The best matching known images are, for example, known images that receive a similarity score higher than a threshold similarity score.
[0056] In one embodiment, when using a neural network that has been trained on a set of random product images, a k-nearest neighbor algorithm (k-NNA) is performed on the deep features of the image (e.g., features in a pixel layer before the last pixel layer). Based on a threshold similarity score or k-NNA, the top k (e.g., k=5) known product images similar to the target product image can be identified.
[0057] Furthermore, as discussed above, in one embodiment, known images containing the same or similar product as the target product image but with a different pose / view of the product may be excluded from the final set of similar images. This will reduce computing resources and processor time.
[0058] Return to Figure 2 At block 230, data associated with each identified similar image is retrieved from the final set of similar images. For example, the machine learning model retrieves historical characteristics / events (described above) including historical demand data (in one embodiment) associated with each similar product image. In one embodiment, other product attributes / characteristics of the similar images may also include product image categories, product image subcategories, and descriptive attributes of the product images.
[0059] In one embodiment, the historical demand data for similar known products may include, for example, one or more combinations of other types of historical data related to the product. For example, the historical demand data may include the number of units ordered / sold in a time period, the location of demand, seasonality, historical prices, price elasticity, and / or base demand, etc.
[0060] At block 240, the machine learning model generates a predictive characteristic model for the target product in the target product image based at least on historical demand data for similar products associated with the identified similar images. In one embodiment, the predictive characteristic model may be a predictive demand model for the target product. The predictive characteristic / demand model may include predictive demand based on the type and amount of demand that has historically occurred for each known product in the final set of known products identified as similar to the target product.
[0061] In one embodiment, the predicted demand model is generated by using the similarity score of each identified similar known product image in combination with historical demand data associated with the corresponding similar known product (e.g., by weighting). For example, historical demand from more similar known products (e.g., better / higher similarity scores) is given greater weight in predicting demand than historical demand from less similar known products (e.g., lower similarity scores). An example is shown in Equation 1 below.
[0062] In this manner, the predictive demand model for the target product predicts future / forecasted demand for the target product, for example, including any one or more of the following: number of product units to be ordered, regional location of product units, seasonality of demand, and / or initial product price. Thus, the predictive demand model can be used as a guide for planning the target new product, ordering and coordinating when, where, and how many product units will be ordered, shipped and / or located in different regional ranges (location by location), and / or estimated initial prices, etc. Therefore, the ML model can infer and predict the initial price of a new target product for which no market data is known based on historical data of identified similar products.
[0063] In one embodiment, the ML model can be configured to generate a predictive demand model as follows: Using similarity scores of known products associated with identified similar known product images, the predictive demand model is trained using known product images associated with the known products to evaluate the predictive power of the similarity scores.
[0064] For example, assume that a new product (i) associated with a new product image is determined to be similar to a known product (j1, j2, ... jk) associated with a known product image. The similarity score between the new product i and the known product j1 can be expressed as sim_i_j1. Each similarity score can be used to weight the corresponding historical data (historical demand) of similar known products. Therefore, the greater the similarity score, the greater the impact of the corresponding historical demand data, and vice versa. An example of a predictive demand model is shown in Equation 1:
[0065] Equation 1
[0066] Store s's demand for new product (i) = function (sim_i_j1*demand for known product j1), (sim_i_j2*demand for known product j2), ... (sim_i_jk*demand for known product jk).
[0067] In one embodiment, a predictive demand model may be generated for a particular selected demand characteristic. For example, a predictive characteristic model may be generated to predict the initial price of a new product, where the "known product demand" is based on historical demand and associated historical unit prices. Another predictive model may be generated to predict demand volume, where the "known product demand" is historical demand volume. Each predicted demand may be combined into a final predictive demand model to identify multiple predicted future characteristics of the target product.
[0068] In one embodiment, the forecast demand model may be enhanced or otherwise adjusted using other product characteristics of similar known products, such as category, seasonality, etc. In one embodiment, the demand model generated for the target product may be a linear additive model.
[0069] In another embodiment, products of the selected brand (e.g., competitor brand products) that are identified as known products similar to the new target product can be considered and used to adjust the predictive demand model. For example, the predictive demand model (e.g., in Equation 1) can be adjusted by adding the negative of the similarity score of the competitor product and combining / multiplying it with the historical demand for the competitor product. In other words, the predictive demand model can be negatively adjusted by combining the similarity scores and historical demand for similar images associated with the selected brand (e.g., competitor brand).
[0070] At box 250, after generating the predictive demand model, the machine learning model can output the predictive model for the new product. The ML model can also generate an electronic message including the predicted demand content for the new product. As previously described, the predictive demand model can be used as a guide for planning the target new product, ordering and coordinating when, where, and how many product units will be ordered, shipped and / or located in different regional ranges (location by location), and / or estimating initial prices, etc. Therefore, in one embodiment, the initial price of a new target product without known market data can be predicted by the ML model in the predictive demand model, as described above.
[0071] At block 260, the machine learning model may transmit the predictive model and / or electronic message via network communication to a remote computer over a network so that a user or other system may use and / or access the predictive model. The machine learning model may also generate a predictive model and display it on a display device.
[0072] As another example, describing Figure 2 The method wherein the new target product is a new shirt design or style. Figure 2 , at box 210, method 200 is initiated when a product image of a new shirt design or style is input into the machine learning model.
[0073] At block 220, the machine learning model then identifies a set of similar known shirt images by comparing the new shirt image to a database of known images. The database of known images may be filtered based on product category so that the comparison step and generation of similarity scores are limited to known images of the shirt category. Additionally, as discussed above, in one embodiment, known images of shirts having poses / views that are different from the pose / view of the input shirt image may also be excluded.
[0074] For the remaining known shirt images, the ML model generates a similarity score for each image compared to the new shirt image. As discussed above, in one embodiment, each similarity score represents a statistical similarity between the new shirt image and one of the known shirt images.
[0075] The similarity scores of the known shirt images may be filtered based at least on a threshold similarity score. Using the threshold, a top N (e.g., 4, 5, or 6, etc.) similar image set may be selected from the most closely similar shirt images (e.g., highest / largest similarity scores). The top image set forms the identified similar image set.
[0076] Return to Figure 2 At block 230, the machine learning model retrieves product attributes, including historical demand data associated with each image in the identified set of similar images. In one embodiment, the product attributes may also include shirt categories (i.e., tops), product item image subcategories (i.e., women's shirts), and descriptive attributes of the shirt images (i.e., sleeve length, sleeveless, etc.). In another embodiment, the images of known shirts may also include data related to demand for shirts associated with the known shirt images based on seasonality, the name or brand of the shirt associated with the known shirt image, historical unit prices of shirts associated with the known shirt product images, etc.
[0077] At block 240, the machine learning model generates a predictive demand model for the new shirt in the new shirt image. In one embodiment, the predictive demand model is generated using similarity scores of similar known images (from the identified similar images) in conjunction with historical demand data associated with the similar known shirt images.
[0078] For example, assume that a new shirt (i) shown in a shirt image input to the ML model is determined to be similar to known shirts (j1, j2, ... jk) associated with the known shirt image. The similarity score between the new shirt (i) and the known shirt j1 can be expressed as sim_i_j1. An example of a predictive demand model for a new shirt is shown in Equation 2:
[0079] Equation 2
[0080] The demand for new shirts (i) at store s = function (sim_i_j1*demand for known shirts j1), (sim_i_j2*demand for known shirts j2), ... (sim_i_jk*demand for known shirts jk).
[0081] In one embodiment, other product characteristics of similar shirts may be used to enhance the predictive demand model, such as category, seasonality, etc. In one embodiment, the demand model generated for the target product may be a linear additive model.
[0082] At block 250, after generating the predictive demand model, the machine learning (ML) model may output the predictive model for the new shirt to a remote computer and / or display device so that the user or other system can use and / or access the predictive model. The ML model may also generate an electronic message containing the predictive demand content for the new shirt. The electronic message may then be transmitted via a communication network to a destination accessible to the remote computer / device (e.g., an email address, text, SMS message, online account, etc.) so that the user or other system can use and / or access the predictive demand / model.
[0083] Inventory Control Prediction Model - Robotic Mechanism
[0084] In one embodiment, the predictive demand model can be used to control inventory of associated products. For example, upon receipt of the predictive demand model and / or electronic message at a remote computer, the predictive demand model can cause instructions to be sent to one or more inventory control systems and / or inventory databases to assign predictive characteristics (e.g., a predicted initial price) to data records associated with a target product (e.g., a new shirt) as identified in the predictive demand model.
[0085] In another embodiment, the predictive demand model may result in instructions for generating an order and / or causing a robotic mechanism to retrieve a quantity of a target product. For example, an order may be generated to include a quantity of a target product as defined in the predictive demand model for the target product. The order may then be prepared and fulfilled by retrieving the quantity of the target product and (if applicable) shipping the quantity to a destination as identified in the predictive demand model. The remote computer may be associated with a warehouse (or fulfillment center) or a sales channel (retail store) and control inventory.
[0086] Referring to the new shirt example, after receiving the forecast demand model and / or electronic message at the remote computer, the order is fulfilled by retrieving a certain number of new shirts. If the order is sent to a warehouse (or fulfillment center), then the warehouse management software receives the order, and the system at the warehouse fulfills the order by retrieving a certain number of new shirts from a unit storage location, packaging the new shirts, and preparing them for shipping. If the order is sent to a sales channel (store), then the order can be fulfilled at the store. In one embodiment, the system is controlled at least in part by the instructions in the order.
[0087] In one embodiment, the system for processing orders may include, for example, an automated robotic machine or mechanism configured to locate and retrieve the target product from a warehouse or store location based on the order. The robotic mechanism may then deliver the retrieved target product to an automated packaging mechanism that packages the retrieved new shirts in the warehouse.
[0088] In one embodiment, the robotic mechanism may include one or more robots configured to navigate, locate and retrieve items in a given warehouse or store, and transport the items to a destination. Each robot may include at least one main structure, a power source, a control interface, a wired / wireless communication interface, a drive device for moving the robot, a navigation device, one or more sensors, and / or a balancing device. Of course, the robots may be configured in different ways, and multiple different types of robots may operate together in a robotic mechanism in a warehouse or store.
[0089] In one or more embodiments, the robotic mechanism may include one or more of the following systems. Automated guided vehicles (AGVs) are used to transport materials, supplies, and inventory within a warehouse or store facility. The AGV can be configured to autonomously navigate a warehouse or store facility by following a defined route marked by wires, magnetic strips, tracks, sensors embedded in the floor, or other physical guides. The AGV can also navigate through a defined map of the warehouse or store based on a coordinate system and the tracked position of the AGV. Cameras can also be used to navigate the AGV.
[0090] Another robotic mechanism is an automated storage and retrieval system (AS / RS), which can include a set of computer-controlled systems that automate inventory management and store / retrieve goods as needed in storage locations in a warehouse or store. The AS / RS can operate as a crane or a shuttle on fixed tracks and can traverse product aisles and vertical heights to remove items or drop items off from storage. Another robotic mechanism is an articulated robotic arm, which is a type of pick and place robot. These arms can move, turn, grab / release, lift and move items, for example, using multi-jointed limbs for manipulating products.
[0091] After the retrieved products (shirts) are delivered to the packing area of the warehouse (block 270), the ordered products (shirts) are packed by an automated system. This may include cartoning software that determines the number, size, and type of containers required to pack the order, and / or a bagging machine that helps speed up the packing operation. As discussed above, after the retrieved products (shirts) are delivered to the sales channel (retailer's store), the retrieved products (shirts) may be delivered to the store's display area and placed on a designated display area in the store.
[0092] The packed order is handed over to the shipper, or if the retailer has its own shipping operation, it is shipped by the retailer's own truck. The status of the order in the inventory management system is changed to "in transit". The order arrives at its destination store. Finally, the computerized management system software then updates the inventory location of the retrieved product (shirt) at the sales channel.
[0093] On the other hand, if the order is sent to a manufacturer, then the sequence of steps is similar, except that the manufacturer is responsible for acting as a fulfillment center and fulfilling the order.
[0094] Computing Device Embodiments
[0095] Figure 3 An example computing device is illustrated that is configured and / or programmed as a special purpose computing device having one or more of the example systems and methods described herein and / or equivalents. The example computing device may be a computer 300 that includes at least one hardware processor 302, a memory 304, and an input / output port 310 operably connected via a bus 308. In one example, the computer 300 is implemented with a prediction system or logic 330 that is configured to facilitate a machine learning (ML) model that is configured to predict future characteristics of a new or unknown object / product, such as a reference Figure 1A , 1B and / or 2.
[0096] In various examples, the logic 330 may be implemented in hardware, a non-transitory computer-readable medium 337 having stored instructions, firmware, and / or a combination thereof. Although the logic 330 is shown as a hardware component attached to the bus 308, it should be appreciated that in other embodiments, the logic 330 may be implemented in the processor 302, stored in the memory 304, or stored in the disk 306.
[0097] In one embodiment, logic 330 or computer is a means (e.g., structure: hardware, non-transitory computer-readable medium, firmware) for performing the described actions. In some embodiments, the computing device may be a server operating in a cloud computing system, a server configured in a software as a service (SaaS) architecture, a smart phone, a laptop, a tablet computing device, etc.
[0098] The component may be implemented, for example, as an ASIC programmed to predict future characteristics such as product demand described herein. The component may also be implemented as stored computer executable instructions that are presented to the computer 300 as data 316 temporarily stored in the memory 304 and then executed by the processor 302.
[0099] Logic 330 may also provide means (eg, hardware, non-transitory computer-readable media storing executable instructions, firmware) for predicting characteristics of new or unknown objects as described herein.
[0100] Generally describing an example configuration of a computer 300, processor 302 may be a variety of different processors, including dual microprocessors and other multi-processor architectures. Memory 304 may include volatile memory and / or non-volatile memory. Non-volatile memory may include, for example, ROM, PROM, etc. Volatile memory may include, for example, RAM, SRAM, DRAM, etc.
[0101] Storage disk 306 can be operably connected to computer 300 via input / output (I / O) interface (e.g., card, device) 318 and input / output port 310, which are controlled by at least one input / output (I / O) controller 340, for example. Disk 306 can be, for example, a magnetic disk drive, a solid state disk drive, a floppy disk drive, a tape drive, a Zip drive, a flash memory card, a memory stick, etc. In addition, disk 306 can be a CD-ROM drive, a CD-R drive, a CD-RW drive, a DVD ROM, etc. Memory 304 can store, for example, processes 314 and / or data 316. Disk 306 and / or memory 304 can store an operating system that controls and allocates resources of computer 300.
[0102] The computer 300 can interact with, control, and / or be controlled by input / output (I / O) devices via an input / output (I / O) controller 340, an I / O interface 318, and an input / output port 310. The input / output devices may include, for example, one or more displays 370, a printer 372 (such as an inkjet printer, a laser printer, or a 3D printer), an audio output device 374 (such as a speaker or headphones), a text input device 380 (such as a keyboard), a cursor control device 382 for pointing and selecting input (such as a mouse, a trackball, a touch screen, a joystick, a pointing stick, an electronic stylus, an electronic handwriting tablet), an audio input device 384 (such as a microphone or an external audio player), a video input device 386 (such as a video and still camera or an external video player), an image scanner 388, a video card (not shown), a disk 306, a network device 320, and the like. The input / output port 310 may include, for example, a serial port, a parallel port, and a USB port.
[0103] The computer 300 can operate in a network environment and therefore can be connected to a network device 320 via the I / O interface 318 and / or the I / O port 310. Through the network device 320, the computer 300 can interact with a network 360. Through the network, the computer 300 can be logically connected to a remote computer 365. The networks with which the computer 300 can interact include, but are not limited to, LANs, WANs, and other networks.
[0104] Definitions and Other Examples
[0105] In another embodiment, the described methods and / or their equivalents may be implemented with computer executable instructions. Thus, in one embodiment, a non-transitory computer readable / storage medium is configured with stored computer executable instructions of an algorithm / executable application, which when executed by (one or more) machines causes the (one or more) machines (and / or associated components) to perform the described methods. Example machines include, but are not limited to, processors, computers, servers operating in a cloud computing system, servers configured with a software as a service (SaaS) architecture, smart phones, and the like. In one embodiment, a computing device is implemented with one or more executable algorithms configured to perform any of the disclosed methods.
[0106] In one or more embodiments, the disclosed methods or their equivalents are performed by any of the following: computer hardware configured to perform the methods; or, computer instructions implemented in a module stored in a non-transitory computer-readable medium, wherein the instructions are configured as an executable algorithm that is configured to perform the methods when executed by at least one processor of a computing device.
[0107] Although for the purpose of simplifying the description, the method illustrated in the figure is shown and described as a series of frames of the algorithm, it should be appreciated that these methods are not limited by the order of the frames. Some frames may appear in a different order than shown and described and / or appear simultaneously with other frames. Moreover, the example method can be implemented using frames less than all the illustrated frames. Frames can be combined or divided into multiple actions / components. In addition, additional and / or alternative methods can adopt additional actions not illustrated in the frame.
[0108] The following includes definitions of selected terms used herein. The definitions include various examples and / or forms of components that fall within the scope of the terms and can be used to implement. The examples are not intended to be limiting. Both singular and plural forms of the terms may be within the definitions.
[0109] References to "one embodiment," "an embodiment," "an example," "an example," etc. indicate that the embodiment(s) or examples described herein may include particular features, structures, characteristics, properties, elements, or limitations, but not every embodiment or example must include the particular features, structures, characteristics, properties, elements, or limitations. Furthermore, repeated use of the phrase "in one embodiment" does not necessarily refer to the same embodiment, but may.
[0110] As used herein, a "data structure" is an organization of data stored in a memory, storage device, or other computerized system in a computing system. A data structure may be, for example, any of a data field, a data file, a data array, a data record, a database, a data table, a graph, a tree, a linked list, and the like. A data structure may be formed of and include many other data structures (e.g., a database includes many data records). Other examples of data structures are possible according to other embodiments.
[0111] As used herein, "computer-readable medium" or "computer storage medium" refers to a non-transitory medium that stores instructions and / or data configured to perform one or more of the disclosed functions when executed. In some embodiments, data can be used as instructions. Computer-readable media can take the form of, but not limited to, non-volatile media and volatile media. Non-volatile media can include, for example, optical disks, magnetic disks, etc. Volatile media can include, for example, semiconductor memories, dynamic memories, etc. Common forms of computer-readable media can include, but are not limited to, floppy disks, flexible disks, hard disks, magnetic tapes, other magnetic media, application-specific integrated circuits (ASICs), programmable logic devices, compact disks (CDs), other optical media, random access memories (RAMs), read-only memories (ROMs), memory chips or cards, memory sticks, solid-state storage devices (SSDs), flash drives, and other media that computers, processors, or other electronic devices can work with. If each type of medium is selected for implementation in one embodiment, it can include storage instructions for algorithms configured to perform one or more of the disclosed and / or claimed functions.
[0112] As used herein, "logic" means a component implemented using a computer or electrical hardware, a non-transitory medium with instructions of an executable application or program module stored, and / or a combination of these, to perform any function or action as disclosed herein, and / or to cause a function or action from another logic, method, and / or system to be performed as disclosed herein. Equivalent logic may include firmware, a microprocessor programmed with an algorithm, discrete logic (e.g., ASIC), at least one circuit, an analog circuit, a digital circuit, a programmed logic device, a memory device containing instructions of an algorithm, etc., any of which may be configured to perform one or more of the disclosed functions. In one embodiment, logic may include one or more gates, a combination of gates, or other circuit components configured to perform one or more of the disclosed functions. In the case of describing multiple logics, it is possible to merge multiple logics into one logic. Similarly, in the case of describing a single logic, it is possible to distribute that single logic between multiple logics. In one embodiment, one or more of these logics are corresponding structures associated with performing the disclosed and / or claimed functions. Choosing which type of logic to implement may be based on desired system conditions or specifications. For example, if higher speed is considered, hardware will be selected to implement the function. If lower cost is a consideration, stored instructions / executable applications will be chosen to implement the functionality.
[0113] An "operable connection," or a connection through which entities are "operably connected," is a connection that can send and / or receive signals, physical communications, and / or logical communications. An operable connection may include a physical interface, an electrical interface, and / or a data interface. An operable connection may include different combinations of interfaces and / or connections sufficient to allow operable control. For example, two entities may be operably connected to transmit signals to each other directly or through one or more intermediate entities (e.g., a processor, an operating system, logic, a non-transitory computer-readable medium). Logical and / or physical communication channels may be used to create an operable connection.
[0114] As used herein, a "user" includes, but is not limited to, one or more persons, computers or other devices, or a combination of these.
[0115] Although the disclosed embodiments have been illustrated and described in considerable detail, it is not intended that the scope of the appended claims be limited or in any way restricted to such details. Of course, it is not possible to describe every expected combination of components or methods for the purpose of describing various aspects of the subject matter. Therefore, the present disclosure is not limited to the specific details or illustrative examples shown and described. Therefore, the present disclosure is intended to cover changes, modifications and variations that fall within the scope of the appended claims.
[0116] To the extent the term "includes" / "including" is employed in the detailed description or the claims, it is intended to be inclusive in a manner similar to how the term "comprising" is interpreted when employed as a transition word in a claim.
[0117] To the extent the term "or" is employed in the detailed description or claims (e.g., A or B), it is intended to mean "A or B or both." When the applicants intend to indicate "only A or B but not both," then the phrase "only A or B but not both" will be used. Thus, the use of the term "or" herein is inclusive, rather than exclusive.
Claims
1. A method performed by a computing system including at least one processor, the method comprising: inputting a target object image in digital form representing the target object into a machine learning model; comparing, by the machine learning model, at least digital pixel data of the target object image with digital pixel data of a set of known object images; generating, by the machine learning model, a similarity score between the target object image and one or more known object images in the set of known object images based at least on the digital pixel data; identifying, by the machine learning model, a set of similar object images based at least in part on the similarity scores of the one or more known object images; for each similar object image in the set of similar object images, retrieving object attributes including historical event data associated with each similar object image; generating a predictive characteristic model including predictive characteristics for a target object represented in the target object image based at least on historical event data for a given similar object in combination with a similarity score for the given similar object; generating an electronic message having a predictive characteristic of a target subject; as well as Transmit electronic messages to a remote computer.
2. The method according to claim 1, wherein the machine learning model is configured with a neural network model trained to classify images of objects; and Prior to generating the electronic message, additional object attributes from the set of similar object images are used by the machine learning model to adjust the predicted characteristics of the target object.
3. The method according to claim 1 or 2, further comprising: Digital pixel data of an object image is analyzed by a machine learning model, including object-based image analysis, to group pixels to identify a target object in the target object image.
4. The method of any preceding claim, wherein generating a predictive property model for a target object comprises: The historical event data of a given similar object is weighted using the similarity score of the given similar object.
5. The method of claim 1 , wherein the target object is a target product, and wherein generating the predicted characteristics further comprises: The predictive characteristic model is configured to generate predictive characteristics as the predicted demand for the target product (i) at the first store (S) based on the function (sim_i_j1*demand for known product j1), (sim_i_j2*demand for known product j2), ... (sim_i_jk*demand for known product jk), wherein a target product (i) is associated with a target product image, the target product image being determined to be similar to known products (j1, j2, ... jk) associated with the known product images, and sim_i_j1, sim_i_j2, ..., and sim_i_jk represent the similarity scores between the target product (i) and each of the known products (j1, j2, ...jk) in the similar product image set.
6. A method as claimed in any preceding claim, wherein the identifying step further comprises: Similar object images that do not include image orientations similar to the image orientation in the target object image are removed from the similar object image set.
7. The method of any preceding claim, further comprising: The robotic mechanism is caused to retrieve a number of target objects from a storage location based at least on the predicted characteristic model.
8. A non-transitory computer-readable medium storing computer-executable instructions, which, when executed by a computer comprising a processor, cause the computer to perform functions configured by the computer-executable instructions, wherein the instructions cause the computer to: inputting a target product image in digital form representing the target product into a machine learning model; identifying, by the machine learning model, a set of similar product images by comparing digital pixel data of the target product image with digital pixel data of known product images and generating a similarity score between the target product image and each similar product image; For each similar product image in the set of similar product images, retrieving product attributes including historical event data associated with each similar product image; and A predicted characteristic is generated for a target product represented in the target product image based at least on the historical event data for a given similar product in combination with a similarity score for the given similar product.
9. The non-transitory computer readable medium of claim 8, further comprising instructions that, when executed by at least the processor, cause the processor to: Additional product attributes from the set of similar product images are used to adjust predicted characteristics of the target product.
10. The non-transitory computer readable medium of claim 8 or 9, further comprising instructions that, when executed by at least the processor, cause the processor to: The digital pixel data of the product images is analyzed by a machine learning model, including object-based image analysis, to group pixels and thereby identify target products in the product images.
11. The non-transitory computer readable medium of claim 10, further comprising instructions that, when executed by at least the processor, cause the processor to: The predictive features are negatively adjusted by a combination of similarity scores and historical events of similar images associated with the selected brand of known products.
12. The non-transitory computer readable medium of any one of claims 8 to 11, further comprising instructions that, when executed by at least the processor, cause the processor to: Generate forecast characteristics as the forecast demand for the target product, product (i), at the first store (S), based on the function (sim_i_j1*demand for known product j1), (sim_i_j2*demand for known product j2), ... (sim_i_jk*demand for known product jk), wherein a target product (i) is associated with a target product image, the target product image being determined to be similar to known products (j1, j2, ... jk) associated with the known product images, and sim_i_j1, sim_i_j2, ..., and sim_i_jk represent the similarity scores between the target product (i) and each of the known products (j1, j2, ...jk).
13. The non-transitory computer readable medium of any one of claims 8 to 12, further comprising instructions that, when executed by at least the processor, cause the processor to: Similar product images that do not include an image orientation similar to that in the target product image are removed from the similar product image set.
14. The non-transitory computer readable medium of any one of claims 8 to 13, further comprising instructions that, when executed by at least the processor, cause the processor to: Instructions are generated to cause the database to assign predictive characteristics to data records associated with the target product.
15. A computing system comprising: at least one processor coupled to at least one memory; A non-transitory computer-readable medium comprising instructions stored thereon, which, when executed by the at least one processor, cause the at least one processor to: inputting a target product image in digital form representing the target product into a machine learning model; comparing, by the machine learning model, at least digital pixel data of the target product image with digital pixel data of a set of known product images; generating, by the machine learning model, a similarity score between the product image and one or more known product images in the set of known product images based at least on the digital pixel data; identifying, by a machine learning model, a set of similar product images based at least in part on the similarity scores of the one or more known product images; For each similar product image in the set of similar product images, retrieving product attributes including historical event data associated with each similar product image; and A predictive characteristic model including predictive characteristics is generated for the target product represented in the target product image based at least on the historical event data of the given similar product in combination with the similarity score of the given similar product.
16. The computing system of claim 15, wherein the instructions further comprise instructions that, when executed by at least the processor, cause the processor to: The digital pixel data of the product images is analyzed by a machine learning model, including object-based image analysis, to group pixels and thereby identify target products in the product images.
17. The computing system of claim 15 or 16, wherein the instructions further comprise instructions that, when executed by at least the processor, cause the processor to: generating an electronic message having the predicted characteristics of the target product; and Transmit electronic messages to a remote computer.
18. The computing system of any one of claims 15 to 17, wherein the instructions for generating a predictive characteristic model further comprise instructions that, when executed by at least the processor, cause the processor to: The similarity score and the historical event data for the given similar product are combined by weighting the historical event data for the given similar product using the similarity score for the given similar product.
19. The computing system of any one of claims 15 to 18, wherein the instructions further comprise instructions that, when executed by at least the processor, cause the processor to: Similar product images that do not include an image orientation similar to that in the target product image are removed from the similar product image set.
20. The computing system of any one of claims 15 to 19, wherein the instructions further comprise instructions that, when executed by at least the processor, cause the processor to: The predictive features are negatively adjusted by a combination of similarity scores and historical events of similar images associated with the selected brand of known products.
