Method and apparatus for grading collectible images using image segmentation and image analysis

Abstract

In some embodiments, the method may include augmenting the set of images of the collectible to generate a set of composite images of the collectible. The method may further include combining the set of images of the collectible and the set of composite images of the collectible to result in a training set. The method may further include training a set of machine learning models based on the training set. Each machine learning model from the set of machine learning models may generate a grade for an image attribute from the set of image attributes. The set of image attributes may include an edge, a corner, a center, or a surface. The method may further include running the set of machine learning models after training to generate a set of grades for images of the collectible not included in the training set.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and benefit of U.S. Provisional Patent Application No. 63 / 150,793, filed February 18, 2021, and entitled "METHODS AND APPARATUS FOR GRADING IMAGES OF COLLECTABLES USING MACHINE LEARNING MODELS," the entire contents of which are incorporated herein by reference.

[0002] Technical Field This disclosure relates to image analysis of images depicting "real" objects, and more particularly to apparatus and methods for performing image analysis on one or more segments of an image for grading images of collectibles. [Background technology]

[0003] background

[0003] Grading images of collectibles can be useful, for example, to assess the value of an asset. Grading images of collectibles can include grading various segments of the image, such as segments that represent corners or edges of the collectible. However, known grading methods can be labor intensive and expensive. Thus, there is a need for devices and methods for grading collectibles accurately and efficiently. Summary of the Invention [Means for solving the problem]

[0004] overview

[0004] In some embodiments, a method may include receiving a set of images of a collection of items. Each image of the set of images is associated with at least one defect type label and at least one of a first grading classification label for a surface condition of a collection from the collection of items, a second grading classification label for an edge condition of the collection, a third grading classification label for a corner condition of the collection, or a fourth grading classification label for a centering condition of the collection. The method may further include generating a set of preprocessed images based on the set of images by detecting, for each image of the set of images, a boundary defining the collection in the image, performing a perspective warp transformation on the image from the set of images for which the boundary for the image does not have a predetermined shape, and removing portions of the image that are not within the boundary defining the collection. The method may further include training at least one model based on each preprocessed image from the set of preprocessed images, the at least one defect type label associated with the preprocessed image, and at least one of (1) a first grade classification label associated with the preprocessed image, (2) a second grade classification label associated with the preprocessed image, (3) a third grade classification label associated with the preprocessed image, or (4) a fourth grade classification label associated with the preprocessed image. The method may further include applying the at least one model to a new image of a new collectible not included in the collection. The method may further include displaying an output indicating that the new collectible includes a defect, an approximate location of the defect, and a defect type associated with the defect.

[0005]

[0005] In some embodiments, a non-transitory processor-readable medium stores code representing instructions executed by a processor. The instructions include code for causing the processor to preprocess an image of a collectible to generate a preprocessed image by detecting a boundary in the image that defines the collectible, perform a perspective warp transformation to cause the boundary to have a predetermined shape, and remove portions of the image that are not within the boundary that defines the collectible. The instructions may further include code for causing the processor to apply a machine learning (ML) model to the preprocessed image to generate a set of defect confidence levels. Each defect confidence level from the set of defect confidence levels (1) is associated with a unique portion of the preprocessed image from a set of unique sub-groups of the preprocessed image, and (2) indicates a likelihood that there is at least one defect in the unique portion of the preprocessed image. The instructions may further include code for causing the processor to display the preprocessed image on a display. The instructions may further include code for causing the processor to display each unique portion of the preprocessed image from the set of unique sub-groups that is associated with a defect confidence level from the set of defect confidence levels that is outside a predetermined range.

[0006]

[0006] In some embodiments, the device includes a memory and a processor operably coupled to the memory. The processor may be configured to augment the set of images of the collectible to generate a set of composite images of the collectible. The processor may be further configured to combine the set of images of the collectible and the set of composite images of the collectible to provide a training set. The processor may be further configured to train a set of machine learning models based on the training set. Each machine learning model from the set of machine learning models is configured to generate a grade for an image attribute from the set of image attributes. The set of image attributes includes at least one of an edge, a corner, a center, or a surface. The processor may be further configured to execute the set of machine learning models after training to generate a set of grades for images of the collectible that are not included in the training set. [Brief description of the drawings]

[0007] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1] 1 is a schematic block diagram of a grading device, according to one embodiment; [Diagram 2]

[0008] 1 is a flow diagram of a method for training a grading device, according to one embodiment. [Diagram 3]

[0009] 1 is a flow diagram of a method of using a grading device, according to one embodiment. [Figure 4]

[0010] 1 is a flow diagram of a method for training a grading device, according to one embodiment. [Diagram 5]

[0011] 1 is a schematic depiction of a machine learning model used for grading, according to one embodiment. [Figure 6]

[0012] 1 is a flow diagram of a method for training and using a model based on a set of preprocessed images, according to one embodiment. [Figure 7]

[0013] 1 is a flow diagram of a method for using a model to generate and use a defect confidence level, according to one embodiment. [Figure 8]

[0014] 1 is a flow diagram of a method for training a model using a training set that includes a set of synthetic images, according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Detailed Description

[0015] Non-limiting examples of various aspects and modifications of the embodiments are described herein and illustrated in the accompanying drawings.

[0009]

[0016] The methods and devices described herein can generate gradings for assets such as trading cards (eg, sports cards, game cards, etc.), coins, currency, and the like.

[0010]

[0017] 1 is a schematic block diagram of a grading device 101 according to one embodiment. The grading device 101 (also referred to herein as a "rating device") may be or include a hardware-based computing device and / or multimedia device, such as, for example, a computer, desktop, laptop, smartphone, etc. The grading device 101 includes a memory 102, a communication interface 103, and a processor 104. The grading device 101 may operate a set of grader models 105 that may collectively generate a grade for an image of a collectible (e.g., trading cards, sports cards, collectible cards, coins, currency, fine art, stamps, antiques, comic books, toys, jewelry, etc.).

[0011]

[0018] The memory 102 of the grading device 101 may be, for example, a memory buffer, a random access memory (RAM), a read only memory (ROM), a hard drive, a flash drive, etc. The memory 102 may store, for example, a set of images of collectibles (e.g., a set of images of trading cards, a set of images of collectible cards, a set of images of coins, a set of images of stamps, a set of images of artworks, etc.), a set of grades (e.g., a set of numerical values), and / or code (e.g., a program written in C, C++, Python, etc.) including instructions for causing the processor 104 to perform one or more processes or functions (e.g., a set of grader models 105).

[0012]

[0019] The communication interface 103 of the grading device 101 may be a hardware component of the grading device 101 for facilitating data communication between the grading device 101 and an external device (e.g., a network, a computing device, and / or a server, not shown). The communication interface 103 may be operatively coupled to and used by the processor 104 and / or memory 102. The communication interface 103 may be, for example, a network interface card (NIC), a Wi-Fi® module, a Bluetooth® module, an optical communication module, and / or any other suitable wired and / or wireless communication interface.

[0013]

[0020] The processor 104 may be, for example, a hardware-based integrated circuit (IC) or any other suitable processing device configured to run or execute a set of instructions or a set of code. For example, the processor 104 may include a general purpose processor, a central processing unit (CPU), an application specific integrated circuit (ASIC), a graphics processing unit (GPU), etc. The processor 104 is operatively coupled to the memory 102 by a system bus (e.g., an address bus, a data bus, and / or a control bus, not shown). The processor 104 includes a set of grader models 105. Each grader model from the set of grader models 105 may be configured to grade attributes of a collectible or a portion of an image from a set of images of the collectible and may include software stored in the memory 102 and executed by the processor 104. In some examples, a grader model from the set of grader models 105 may include a collectible and / or card type predictor (not shown) and / or a tile defect predictor (not shown). Each of the collectibles and / or card type predictors or tile defect predictors may include software stored in memory 102 and executed by processor 104.

[0014] Generating a trained model

[0021] FIG. 4 is a flow diagram of a method for training a grading device (e.g., grading device 101) according to an embodiment. In some implementations, the method discussed with respect to FIG. 4 may be executed by a processor (e.g., processor 104 of FIG. 1). In step 1, pre-process training images. Pre-processing may cause the training images and / or portions of the training images (e.g., images of collectibles only) to have a standardized format in at least one aspect. Pre-processing may include, for example, cropping, trimming, filtering, distorting, and / or resizing the training images and / or portions thereof. The training images and / or pre-processed training images may be associated with one or more labels and / or grades, such as centering grades, tile defect labels, collectible and / or card type labels, player information, character information, edge grades, corner grades, etc. Additionally, as discussed in more detail herein, synthetic images may be generated for training in some examples.

[0015]

[0022] In step 2, homography values ​​are generated by comparing collectibles (e.g., cards, stamps, artwork, etc.) having various centering grades. In step 7, a centering grade regression model (e.g., a neural network) can be trained using the homography values, centering grade labels, and / or preprocessed training images to generate a trained centering model. In some implementations, the homography values ​​and / or preprocessed images can be used as input training data for the centering grade regression model, and the centering grade labels can be used as target training data for the centering grade regression model.

[0016]

[0023] In step 3, NxN surface tiles are generated for each of the preprocessed training images. In step 8, a defect classification model may be trained using the surface tiles, defect labels that may be associated with the surface tiles, collectible and / or card type labels, player information, and / or character information to identify defects (e.g., generate defect confidence levels for the surface tiles). In step 11, the trained defect classification model may then be run using the surface tiles, defect labels, collectible and / or card types, player information, and / or character information to generate a defect confidence level for each surface tile. In step 12, a regression model of surface grades may be trained using the preprocessed training images, the transfer weights from the defect classification model trained in step 8, and the defect confidence levels generated in step 11.

[0017]

[0024] In step 4, the preprocessed training images are used to generate edge images. The edge images and edge grade labels can be used in step 9 to train and generate an edge grade regression model. In some implementations, the edge images can be used as input training data for the edge grade regression model, and the edge grade labels can be used as target training data for the edge grade regression model.

[0018]

[0025] In step 5, the preprocessed training images are used to generate corner images. The corner images and corner grade labels can be used in step 10 to train and generate a corner grade regression model. In some implementations, the corner images can be used as input training data for the corner grade regression model, and the corner grade labels can be used as target training data for the corner grade regression model.

[0019]

[0026] In step 6, the pre-processed training images can be used to train and generate (e.g., using computer vision) a collectibles and / or card type / player / character classification model. The collectibles and / or card type / player / character classification model can be trained to identify collectibles type, card type (e.g., set, year, etc.), stamp type (e.g., year, issue, etc.), coin type, player information, character information, and / or any other information related to the collectibles. In some implementations, step 6 is performed before step 8 and step 11, and the trained collectibles and / or card type / player / character classification model outputs collectibles type, card type, player information, stamp type (e.g., year, issue, etc.), coin type, character information, and / or other information related to the collectibles used in steps 8 and step 11. In some implementations, steps 1-12 can be performed in any order. In some implementations, steps 1-12 can be performed sequentially, in parallel, or any combination thereof.

[0020] Image preprocessing

[0027] The set of images of the collectibles (e.g., images of sports cards, game cards, collector cards, coins, stamps, artwork, etc.) used to train the set of grader models 105 and / or images of the collectibles (not in the set of images of the collectibles) used when running the set of grader models 105 after training may be taken using an imaging device (e.g., a camera, scanner, etc. (not shown)) of the grading device or a device operatively coupled to the grading device 101. For example, the set of images of the collectibles and / or the images of the collectibles may be taken by a smartphone camera or scanner. Thus, the images for processing by the processor 104 of the grading device 101 may be taken at slightly different angles, under various lighting conditions, and / or may also include additional background surrounding the actual collectibles (e.g., cards). Thus, image pre-processing may be used to generate pre-processed images that are based on the images (e.g., the set of images of the collectibles) used to train the set of grader models 105 and / or the images graded by the set of grader models 105 (e.g., to normalize the images of the collectibles). In some examples, to use images of collections for training and / or grading purposes, one or more of the following pre-processing steps may be performed using any suitable technique (e.g., step 1 of FIG. 4): 1. Boundary detection of collectibles in images of collectibles. 2. Perspective warp transformation to convert collectible images taken from imperfect angles into rectangles. 3. Removal of background by cropping the outer area surrounding the boundary of the collectible in the collectible image. 4. Resizing images of collections to a uniform size and resolution suitable for processing by machine learning models from a set of grader models 105.

[0021]

[0028] In some implementations, a desired shot of an image of a collectible can be selected from a live camera video feed and / or recording (e.g., from an imaging device) by applying a border detection algorithm in near real time and selecting a frame with a detected border that is closest to the rectangle. Doing so can minimize the amount of perspective warping transformation and improve image quality and overall grading accuracy. Additionally or alternatively, a desired shot of an image of a collectible can be selected from a live camera video and / or recording by applying resizing and / or resolution adjustments and selecting a frame with a size and / or resolution that is closest to the desired size and / or resolution. In some implementations, if multiple images (e.g., frames from a video and / or multiple still images) exist for a collectible, the selected desired shot can be used to generate a grade for the collectible.

[0022]

[0029] In some implementations, for images of collectibles with shiny surfaces, additional pre-processing steps can be performed to detect and skip frames of the video that contain distracting reflections, saturation in the image, or white spots. Additional machine learning models can be trained to detect distracting reflections, saturation in the image, or white spots, and filter out undesired frames, pixels, etc. from the images of the collectibles. In some implementations, the final image of the collectibles can be brought about by stitching unaffected portions of the collected images or video frames of the collectibles. For example, if a first image of the collectibles in a first frame contains a reflection in the top half of the first image, and a second image (different from the first image) of the collectibles in a second frame (different from the first frame) contains a reflection in the bottom half of the second image, the bottom half of the first image can be combined (e.g., stitched) with the top half of the second image to form a final image without distracting reflections (e.g., the top half does not overlap with the bottom half, or the top half partially overlaps with the bottom half).

[0023] Grading

[0030] In some examples, for example, a grading of an asset (e.g., a collectible such as a sports card) may include four scores (or grades) within a numerical range (e.g., on a scale of 1 to 10). The scores may represent the condition of the asset's surface, the condition of the asset's edges, the condition of the asset's corners, and / or the condition of the asset's centering. In some examples, a higher score (or grade) may mean better condition (e.g., of the surface, edges, corners, or centering). Each side of the asset (or collectible) may be graded separately and may have its own set of grades. Although four scores are shown in this example, other implementations may identify and use any number of scores relating to different aspects and / or attributes of the asset.

[0024]

[0031] Asset grading may be accomplished by training an ensemble of machine learning models (e.g., artificial neural networks, convolutional neural networks, recurrent neural networks, self-organizing maps, Boltzmann machines, autoencoders, etc.) designed to handle specific types of grades. Each type of grade may have one or more designated machine learning models (e.g., neural network models). In some implementations, two machine learning models may be used to identify a grade (e.g., a surface grade). In some implementations, each grade may be identified using any number of machine learning models.

[0025]

[0032] In some implementations, a first machine learning model may be configured and / or trained to detect types of surface defects on a surface image of a set of images of a collectible divided into several approximately equal subdivisions (e.g., tiles). In some implementations, a surface image of a collectible may be divided into approximately equal subdivisions, and one or more defect labels may be assigned (e.g., by a human, by a machine learning model, by a computer vision algorithm, etc.) to (1) each subdivision (including those with or without defects) or (2) only the subdivisions that have defects. A defect label may be a letter, word, number, etc. that indicates that there is a defect and / or the type of defect (e.g., wrinkle, crease, etc.) in a given subdivision. The defect labels and the various equal subdivisions may be used to train the first machine learning model. For example, each subdivision may be used as input training data for a neural network, and the defect label associated with that subdivision may be used as output training data for the neural network. A second machine learning model may use the entire surface image of the collectible as well as the tile defect information from the first machine learning model for the collectible to regress a final grade. The machine learning model can be trained using a training dataset that includes existing images (e.g., thousands) of collectibles that may have been previously graded by expert human graders. For surface grading, for example, the training dataset may include grade labels assigned to each collectible photograph as well as defect type labels assigned to individual tiles.

[0026]

[0033] Input for the training process includes providing a set of images of the collectible that are each associated with a first grading classification label for the surface (e.g., an integer value from 1 to 10), a second grading classification label for the edges (e.g., an integer value from 1 to 10), a third grading classification label for the corners (e.g., an integer value from 1 to 10), and / or a fourth grading classification label for the centering (e.g., an integer value from 1 to 10). Grading classification labels may be assigned to each photo separately for the surface and the back of each image from the set of images of the collectible.

[0027]

[0034] For surface grades, in addition to the grade labels, the training set may include classification labels for various defect types that are assigned to individual surface tiles of the collectible image outlined by the NxN grid of tiles. Each tile in the grid of tiles may have multiple types of defect labels, e.g., wrinkles, creases, printing defects, stains, ink, etc. In some implementations, a set of surface flaw codes (e.g., represented as letters, numbers, etc.) may represent the types of tile flaws. In some examples, the set of surface flaw codes may include, for example, the following: C - Wrinkles, creases, bends, crushing, etc. H-hole, pinhole, punch, etc. I - Traces, scratches, etc. M-Miscut P-Printing, stains, alignment, etc. S-stain T-Tears, tears, bends, etc. W - Writing, ink, etc. Corner / Edge Scratch Code X-angle(1~4) E-edge (1~N) A- Damage (curling and / or minor damage to paper or surface) B- Creases or Lifting (creases / bends or lifting on corner surfaces) Y-Impact (marks, dents, or other indentations not indicated by the surface damage code)

[0028]

[0035] In some implementations, additional models can be trained to detect collectible and / or card type (e.g., which set it is in, year of production, manufacturer, etc.), player and / or character information, stamp type (e.g., year, issue, etc.), coin type (e.g., year, coin identification, etc.), and / or any other information related to the collectible. This information can be used in the underlying grading model (e.g., the machine learning model described above) to reduce the number of false positives specific to a particular collectible and / or card type. For example, some cards may contain creases on a player's clothing that may be erroneously identified as a defect. Adding card type and / or player / character information (and / or other information specific to the collectible) to the input of the grading model can help eliminate such false positives by training the model with exceptions specific to the collectible. The additional models can be, for example, machine learning models, artificial intelligence models, analytical models, or mathematical models. In some implementations, the additional model can be trained to use computer vision to detect collectibles and / or card types, player information, character information, stamp types (e.g., year, issue, etc.), coin types (e.g., year, coin identification, etc.), and / or any other information about the collectibles. In some implementations, the additional model can be trained using supervised learning. In some implementations, the additional model can be trained using unsupervised learning. In some implementations, the additional model is a neural network (e.g., a convolutional neural network) that is trained using images of the collectibles (e.g., cards) as input training data and card types, player information, character information, and / or characteristics specific to the collectibles as output training data.

[0029]

[0036] The resulting trained machine learning model can be used to grade the collectible (as shown in step 301 of FIG. 3). During the grading process, the same image pre-processing steps (in step 302) as in the training phase can be applied to an input image of the collectible (e.g., a new collectible not related to the training data) to generate a pre-processed image. The pre-processed image can then be input to a set of grader models 105 (e.g., including a trained regression model of the surface grade) for predicting a grade to predict a grade. The set of grader models 105 can predict (in step 303) card type and / or player / character information (or other information about the collectible). The set of grader models 105 can further predict (in step 304) tile defects. The set of grader models 105 can further predict (in step 305) a grade based on information about the collectible (e.g., card type, player / character information), tile defects, and / or other grades generated by the set of grader models 105. In some implementations, after the grade is calculated by the set of grader models 105, an additional overlay image can be constructed from the weights of the convolutional layers of the grader models. The overlay image can be used to highlight specific image regions where defects are identified from the set of images of the collectible (e.g., images of cards). Additionally or alternatively, the overlay image can be used to highlight specific image regions where defects are not identified from the set of images of the collectible.

[0030] Surface Grader

[0037] The set of grader models 105 may include a surface grader model. In some implementations, the surface grader model may be or may include an ensemble of two separate models: Tile defect classification model Surface grade regression model In some implementations, both the tile defect classification model and the surface grade regression model can be generated using an existing machine learning model (e.g., using transfer learning). For example, the machine learning model can be a pre-trained neural network model that is trained using a large dataset of generic images (e.g., the ImageNet dataset (a publicly available dataset containing over 14 million images of real-world objects)). Using a pre-trained neural network model can add existing knowledge of various object shapes to the machine learning model, making the set of grader models 105 (e.g., the tile defect classification model and / or the surface grade regression model) more effective in distinguishing between known object shapes and surface defects. In some implementations, a pre-trained ImageNet-based model can be used, such as VGGNet, ResNet, Inception, Xception, etc.

[0031]

[0038] A tile defect classification model can be trained using a small subset of training images that are classified by surface flaw code (as shown in steps 3 and 8 of FIG. 4). After the tile defect classification model is trained, it can be used to classify tiles in the training set (as shown in step 11 of FIG. 4) and generate a confidence level for possible defects on each tile.

[0032]

[0039] The structure of layers in a tile defect classification model (e.g., neural network) is shown in FIG. 5 according to one embodiment. Additional layers can be added to a base model (e.g., trained with generic images) responsible for classifying tile defects in images of collectibles. Dropout layers can be used to reduce overfitting of the model and provide better generalization for the neural network. In some implementations, the size of the final output layer can be determined by the number of defect types supported. For example, in an application of the grading device 101 on a certain type of collectibles, there can be N (e.g., 7, 10, 100, etc.) types of defects that can occur for that particular type of collectibles. Thus, the size of the final output layer of the tile defect classification model can be N (e.g., an integer number N). For example, the output of the tile defect classification model can include N confidence levels in the range of 0.0 to 1.0, where N is the number of defect types supported.

[0033]

[0040] The structure of the layers in the surface grade regression model may be similar to the structure of the layers in the tile defect classification model. The difference in the structure of the layers may be the size of the last output layer of the surface grade regression model. Since the surface grade regression model is a regression model, in some implementations the surface grade regression model has one output representing a continuous value of the surface grade. Similarly, in contrast to the tile defect classification model, where the output is a label, in such implementations the grade may be represented as a decimal number (e.g., 5.0, 8.5, 9.99).

[0034]

[0041] In addition, weights from the tile defect classification model (FIG. 5) can be transferred to the surface grade regression model, allowing the surface grade regression model to recognize the defect patterns learned by the tile defect classification model (steps 8 and 12 in FIG. 4).

[0035] Edge and corner graders

[0042] The set of grader models 105 may include dedicated models for edges and corners. In some implementations, the dedicated models for edges and corners may have the same and / or similar layer structure. In some cases, the dedicated models for edges and corners may be similar to the tile defect classification models, the difference being the number of outputs in the last layer (the output layer). In some implementations, the dedicated models for edges and corners have one output that represents a continuous grade value (e.g., values ​​from 0 to 10). The grade may represent the condition of the edge and / or corner and may be used to determine whether corrective action should be taken. For example, if the grade is outside a predetermined tolerance, the edge and / or corner may be indicated as defective.

[0036]

[0043] In some implementations, separate input images for the edge grader model and the corner grader model can be extracted from the pre-processed image. Similar to the tile defect classification model and the surface grade regression model, the edge grader model and / or the corner grader model may provide the ability to generate overlay images to highlight edge and / or corner defects.

[0037] Grader for centering

[0044] The set of grader models 105 may include a dedicated model for the center of the image of the collectible to determine how centered the collectible is. In some implementations where the collectible includes an image (e.g., of a player or character) printed on card stock, the centering grade may refer to how centered the image is on the card stock. In some implementations, the centering grade of the collectible image may be calculated by a centering regression grader model that takes as input a set of homography matrices. The set of homography matrices (as shown in step 2 of FIG. 4) may be calculated by comparing the preprocessed images in the training set with several other collectibles (e.g., cards) with different centering grades. Such an approach may be similar to triangulation, where the homography distance between various centering grades is considered (e.g., using computer vision). The grade may represent the state of centering for the collectible image and may be used to determine whether corrective action should be taken. For example, if the grade is outside of a predetermined tolerance, the centering of the image of the collectible may be indicated as defective (eg, by text or any other label indicating that the centering condition is undesirable).

[0038]

[0045] In some examples, in addition to homography values, the central regression grader model can take collectible specific information (e.g., card type and / or player / character information) as input, which can ensure that collectible and / or card type specific biases are avoided.

[0039] Dealing with imbalanced training data

[0046] In some examples, the difficult part of making a set of grading models accurate is the problem of overfitting the training set when using a limited imbalanced training data set. The grading device 101 of FIG. 1 can train an accurate predictive model using a training set that does not cover a large number of samples for each collectible's specific information (e.g., card type, player / character, and grade combination). In other words, the set of grader models 105 of the grading device 101 is developed to generalize the generation of grades based on collectible images. Therefore, without maintaining an extremely large training set (e.g., billions of images), the same set of grader models trained based on the training set can successfully grade a wide variety of collectible images (e.g., a wide variety of sports cards and / or player sets, a wide variety of stamps, a wide variety of artworks, etc.) based on some reusable images in the training set (e.g., thousands of images).

[0040]

[0047] In some implementations, generalization methods to avoid or reduce some of the problems with imbalanced training data may include, for example, one or more of the following: 1. Upsampling and Downsampling of the Training Set. The purpose of this step is to adjust the training set so that it has a relatively equal distribution of some samples across all grades. For grades with an above average number of samples, the dataset can be reduced by randomly pruning excess samples from the dataset (downsampling). For grades with a below average number of samples, additional synthetic images of the collection can be generated and added to the training set (upsampling). 2. Dropout Layer. Using a dropout layer allows for a very computationally cheap and effective regularization method to reduce overfitting of the grading model and improve the generalization error. 3. Layer weight regularizers. Similar to dropout layers, weight regularizers reduce the chance of overfitting a machine learning model (e.g., a neural network) by constraining the range of weight values ​​in the network. In some cases, weight regularizers can be added to individual layers of a network, including layers in a base model that is trained on generic image data. 4. K-fold validation can be used to improve generalization and reduce overfitting. 5. Additional image augmentation by generating synthetic training data.

[0041]

[0048] In some examples, the number of dropout layers, the dropout rate, and / or the number of weight regularizers may be determined during a hyperparameter optimization stage. The hyperparameter optimization stage may improve, tune, and / or optimize the hyperparameters of a model (e.g., a model from the set of grader models 105 of FIG. 1). Further details regarding hyperparameter optimization are discussed below.

[0042] Synthetic training images

[0049] In some implementations, the grading device 101 can generate synthetic images (in addition to the set of images of the collection) to further improve the accuracy of the set of grader models 105 trained on a moderate data set. In some examples, a set of image augmentation techniques can be randomly applied to the set of images of the collection to augment the training set with additional synthetic images. The set of image augmentation techniques may include rotation, vertical and / or horizontal shift, scaling, brightness and contrast adjustment, vertical and / or horizontal flip, etc., for one or more images from the set of images of the collection to generate a set of synthetic images. The set of synthetic images can be used to train or retrain one or more grader models from the set of grader models 105 in addition to the set of images of the collection. In some implementations, the set of synthetic images is pre-processed (e.g., perspective warp transformation, resizing, background cropping, etc.) before being used to train one or more grader models from the set of grader models 105.

[0043]

[0050] A set of augmentation techniques may ensure consistent grading accuracy for images of collections taken using cameras with different capabilities (e.g., resolution, zoom, filters, depth, etc.) and / or under different lighting conditions (e.g., angles). Using augmentation can also significantly expand the number of samples in the training set, which may improve the generalization of the set of grader models 105.

[0044] Hyperparameter tuning

[0051] The hyperparameters of the set of grader models 105 can be optimized using one of the following tuning algorithms: random search, hyperband, Bayesian optimization, etc. The effectiveness of a particular tuning algorithm may vary based on the training set and other factors. Thus, tuning algorithms can be individually evaluated to achieve the best accuracy for a particular model and a particular training set. The set of adjustable parameters and / or hyper-parameters for the grader model 105 may include, for example: 1. Neural network parameters Layer size Number of dropout layers Dropout rate · Weight regularizer type Regularization coefficient Types of ImageNet-based models 2. Image Augmentation Parameters Rotation angle, shift, brightness, scaling, and flip ranges 3. Training parameters Optimizer type Learning Rate Batch size Number of epochs

[0045] Defect Visualization

[0052] The defects identified by the set of grader models 105 can be visualized as an overlay on the image of the original collection. The overlay can be constructed from the weights of the last convolutional layer of the model trained using the generic image data. For example, if the model trained using the generic image data is a Visual Geometry Group Network (VGGNet) model, the last convolutional layer would be block5_conv3. A large weight value represents a high confidence that a defect is detected in the corresponding pixel or group of pixels.

[0046]

[0053] The range of weight values ​​can be represented using various overlay colors or pixel intensities, effectively creating a heatmap representation. Other visual cues can be achieved by displaying contours around high intensity clusters whose weight values ​​are above a certain threshold or highlighting areas around such clusters. Such visual representations can be presented and / or displayed to a user by a user device (e.g., the grading device 101 and / or a device operably coupled to the grading device).

[0047]

[0054] In some implementations, the grading device 101 may be operatively coupled to a computing device (not shown) and / or a server (not shown) over a network to transmit and / or receive data (e.g., images of collectibles) and / or analytical models over the network. In some examples, the computing device and / or server may provide training data to the grading device 101. In some examples, the computing device and / or server may execute trained machine learning models to grade assets, such as collectibles.

[0048]

[0055] 6 is a flow diagram of a method 600 for training and using a model based on a set of preprocessed images, according to one embodiment. In some implementations, the method 600 may be performed by a processor (e.g., the processor 104 of FIG. 1). For example, instructions for causing the processor 104 to perform the method 600 may be stored in the memory 102 of FIG.

[0049]

[0056] At 602, a set of images of a group of collectibles (e.g., one collectible, two collectibles, three collectibles, etc.) is received. Each image from the set of images is associated with at least one defect type label and at least one of a first grading classification label for a surface condition of a collectible from the group of collectibles, a second grading classification label for an edge condition of the collectible, a third grading classification label for a corner condition of the collectible, or a fourth grading classification label for a centering condition of the collectible. In some implementations, the group of collectibles can include only trading cards, only coins, only currency, only art, only stamps, only antiques, only comic books, only toys, only jewelry, or combinations thereof. In some implementations, the set of images is of a common side (e.g., front) of the group of collectibles. In some implementations, the set of images is of various different sides (e.g., front and back) of the group of collectibles. In some implementations, collectibles refer to items that are of interest to collectors. In some implementations, collectibles refer to things that are collectable.

[0050]

[0057] At 604, a set of pre-processed images is generated based on the set of images by detecting, for each image of the set of images, a boundary that defines a collectible in the image, performing a perspective warp transformation on the image from the set of images for which the boundary for the image does not have a predetermined shape (e.g., a square, a rectangle, a parallelogram, etc.), and removing portions of the image that are not within the boundary that defines the collectible. In some implementations, step 604 is performed automatically (e.g., without requiring human input) in response to receiving the set of images. In some implementations, generating the set of pre-processed images further includes resizing each image from the set of images that has a dimension that is not the predetermined dimension to have the image have the predetermined dimension. In some implementations, generating the set of pre-processed images further includes resizing each image from the set of images that has a resolution that is not within a predetermined resolution range to have the image have a resolution within a predetermined resolution range.

[0051]

[0058] At 606, at least one model (e.g., the set of grader models 105 shown in FIG. 1 ) is trained based on each preprocessed image from the set of preprocessed images, the at least one defect type label associated with the preprocessed image, and at least one of (1) the first grade classification label associated with the preprocessed image, (2) the second grade classification label associated with the preprocessed image, (3) the third grade classification label associated with the preprocessed image, or (4) the fourth grade classification label associated with the preprocessed image. In some implementations, the at least one model includes at least one dropout layer to reduce overfitting. In some implementations, the at least one model includes: (1) a first model trained using (a) each preprocessed image from the set of preprocessed images and (b) a first grade classification label associated with the preprocessed image; (2) a second model trained using (a) each preprocessed image from the set of preprocessed images and (b) a second grade classification label associated with the preprocessed image; (3) a third model trained using (a) each preprocessed image from the set of preprocessed images and (b) a third grade classification label associated with the preprocessed image; (4) a fourth model trained using (a) each preprocessed image from the set of preprocessed images and (b) a fourth grade classification label associated with the preprocessed image; and (5) a fifth model trained using (a) each preprocessed image from the set of preprocessed images and (b) at least one defect type label associated with the preprocessed image.

[0052]

[0059] At least one model is applied to a new image of a new collectible that is not included in the collection of collectibles at 608. In some implementations, the at least one model is automatically applied to the new image in response to a representation of the new image being received (e.g., by processor 104 of FIG. 1).

[0053]

[0060] At 610, an output is displayed indicating that the new collection includes defects, the approximate locations of the defects, and a type of defect associated with the defects. In some implementations, 610 is performed automatically (e.g., without requiring human input) in response to applying the at least one model to the new images at 608. In some implementations, the output is displayed by a processor (e.g., processor 104) sending at least one electrical signal to a display (not shown in FIG. 1 ) that is operably coupled to the processor by a wired and / or wireless connection to cause the display to indicate (e.g., by text, symbols, color coding, highlighting, etc.) that the new collection includes defects, indicate (e.g., by text, symbols, color coding, highlighting, etc.) the approximate locations of the defects, and indicate (e.g., by text, symbols, color coding, highlighting, etc.) a type of defect associated with the defects (e.g., bend, wrinkle, etc.).

[0054]

[0061] In some implementations of method 600, a first image from the set of images is captured or taken under a first lighting condition and a second image from the set of images is captured or taken under a second lighting condition that is different from the first lighting condition. The lighting condition may be, for example, a quantity of luminance.

[0055]

[0062] In some implementations of method 600, a first image from the set of images is captured or taken at a first angle relative to a first collection from a collection group, and a second image from the set of images is taken at a second angle relative to one of the first collection or a second collection from a collection group different from the first collection. The second angle is different from the first angle. The first image and the second image can be captured or taken using the same imaging device (e.g., a single common camera) or different imaging devices (e.g., two different cameras).

[0056]

[0063] In some implementations of method 600, a first image from the set of images is taken with a first background and a second image from the set of images is taken with a second background that is different from the first background, e.g., the first background and the second background may be of different color, texture, pattern, shape, orientation, scenery, etc.

[0057]

[0064] In some implementations, the method 600 further includes optimizing and / or improving hyper-parameters associated with the at least one model using at least one of a random search algorithm, a hyperband algorithm, or a Bayesian optimization algorithm.

[0058]

[0065] 7 is a flow diagram of a method 700 for using a model to generate and use a defect confidence level, according to one embodiment. In some implementations, the method 700 may be performed by a processor (e.g., the processor 104 of FIG. 1). For example, instructions for causing the processor 104 to perform the method 700 may be stored in the memory 102 of FIG.

[0059]

[0066] At 702, an image of a collectible is preprocessed to generate a preprocessed image by detecting a boundary that defines the collectible in the image, performing a perspective warp transformation to make the boundary have a predetermined shape (e.g., rectangle, square, parallelogram, etc.), and removing portions of the image that are not within the boundary that defines the collectible. The collectible may be, for example, trading cards (e.g., baseball cards, basketball cards, football cards, Pokemon cards, etc.), coins, currency, fine art, stamps, antiques, comic books, toys, jewelry, etc. The image may be collected by an imaging device such as a camera or scanner.

[0060]

[0067] At 704, a machine learning (ML) model (e.g., a set of grader models 105 of FIG. 1 ) is applied to the preprocessed image to generate a set of defect confidence levels. Each defect confidence level from the set of defect confidence levels (1) is associated with a unique portion of the preprocessed image from a set of unique portions of the preprocessed image, and (2) indicates a likelihood that there is at least one defect within that unique portion of the preprocessed image. In some implementations, 704 is performed automatically (e.g., without requiring human input) in response to generating the preprocessed image at 702. In some implementations, each of the defect confidence levels is associated with a numerical value (e.g., from 0 to 100, from 0% to 100%, from 1 to 10, etc.). In some implementations, each of the defect confidence levels is associated with a text label (e.g., like new, very good, fine, near fine, excellent, good, poor, etc.). In some implementations, each unique portion from the set of unique portions does not overlap with any other unique portion from the set of unique portions (e.g., one unique portion in the top half and another unique portion in the bottom half). In some implementations, at least one unique portion from the unique portion set (e.g., one to all unique portions from the unique portion set) overlaps with another unique portion from the unique portion set (e.g., a first unique portion in the upper half, a second unique portion in the lower half, and a third unique portion in the central portion that includes subsections of the upper and lower halves).

[0061]

[0068] At 706, the preprocessed image is displayed on a display. In some implementations, 706 is performed automatically (e.g., without requiring human input) in response to generating the set of confidence levels at 704. In some implementations, the output is displayed on a display by a processor (e.g., processor 104) sending at least one electrical signal to a display (not shown in FIG. 1 ), which is operably coupled to the processor by a wired and / or wireless connection for displaying the preprocessed image on the display.

[0062]

[0069] At 708, each unique portion of the preprocessed image from the set of unique portions associated with a defect confidence level from the set of defect confidence levels outside of a predetermined range is caused to be shown on the display. In some implementations, 708 is performed automatically (e.g., without requiring human input) in response to causing 706 to display the preprocessed image. In some implementations, a defect confidence level within the predetermined range indicates that the unique portion associated with the defect confidence level is in a desired (or "good enough") condition (e.g., like new, very good, excellent, etc.), and a defect confidence level outside the predetermined range indicates that the unique portion associated with the defect confidence level is not in the desired condition (e.g., poor, low quality, etc.). In some implementations, the predetermined range can be adjusted (e.g., by instructions entered by a user and received at a processor) for a particular use case (i.e., based on what is considered an acceptable condition by a user, customer, organization, population, etc.).

[0063]

[0070] In some implementations, the ML model is a first ML model, and the method 700 further includes applying a second ML model to the pre-processed image to generate a first score indicative of a surface condition of the collectible, applying a third ML model to the pre-processed image to generate a second score indicative of an edge condition of the collectible, applying a fourth ML model to the pre-processed image to generate a third score indicative of a corner condition of the collectible, and applying a fifth ML model to the pre-processed image to generate a fourth score indicative of a centering condition of the collectible. The method 700 may further include assigning at least one label to the collectible that indicates an overall condition of the collectible based on the first score, the second score, the third score, and the fourth score. In some implementations, the at least one label may indicate an overall condition of one of mint, very good, very good / very good, very good, near good / very good, near good, excellent / near good, good, or poor. In some implementations, a numerical value that is a function (e.g., sum, average, weighted average, etc.) of the first score, the second score, the third score, and / or the fourth score corresponds to (e.g., is within a numerical range associated with) the at least one label, and thus the numerical value can be calculated and used to determine the at least one label. Method 700 may further include displaying each defect confidence level from the group of defect confidence levels as superimposed on a unique portion of the preprocessed image associated with that defect confidence level. For example, if the preprocessed image includes N unique portions (e.g., tiles), then N defect confidence levels may be displayed, with each confidence level associated with (superimposed on) a different unique portion.

[0064]

[0071] In some implementations, the method 700 may further include applying a computer vision model to the pre-processed image to identify at least one of a card type, player information, character information, and / or other information associated with the collectible, where the at least one of the card type, player information, character information, and / or other information is used by at least one of a first ML model for generating the set of defect confidence levels, a second ML model for generating the first score, a third ML model for generating the second score, a fourth ML model for generating the third score, or a fifth ML model for generating the fourth score. In some implementations, applying the first ML model occurs before applying the second through fifth ML models, and at least two of applying the second ML model, applying the third ML model, applying the fourth ML model, or applying the fifth ML model occur in parallel. In some implementations, the first through fifth ML models may be applied sequentially, in parallel, or any combination thereof.

[0065]

[0072] In some implementations, the pre-processing at 702 further includes resizing the image to a predetermined dimension. In some implementations, the pre-processing at 702 further includes resizing the image to have a resolution within a predetermined resolution range.

[0066]

[0073] In some implementations, the method 700 further includes determining at least one of a card type, player information, character information, and / or other information associated with the collectible for the pre-processed image. The ML model can be further applied to at least one of the card type, player information, character information, and / or other information associated with the collectible to generate a set of defect confidence levels. Stated similarly, the set of defect confidence levels can be generated by the ML model based at least in part on at least one of the card type, player information, character information, and / or other information associated with the collectible.

[0067]

[0074] 8 is a flow diagram of a method 800 for training a model using a training set including a set of synthetic images, according to one embodiment. In some implementations, the method 800 may be performed by a processor (e.g., the processor 104 of FIG. 1). For example, instructions for causing the processor 104 to perform the method 800 may be stored in the memory 102 of FIG. 1.

[0068]

[0075] At 802, the set of images of the collectible is augmented to generate a set of composite images of the collectible (e.g., only trading cards, only coins, only currency, a combination of cards, coins, and / or currency, etc.). In some implementations, the augmentation at 802 may include at least one of rotating a first image from the set of images, shifting the first image vertically, shifting the first image horizontally, scaling the first image, adjusting the brightness of the first image, adjusting the contrast of the first image, flipping the first image vertically, or flipping the first image horizontally. At 804, the set of images of the collectible and the set of composite images of the collectible are combined to result in a training set. At 806, a set of machine learning models (e.g., the set of grader models 105 of FIG. 1) are trained based on the training set. Each machine learning model from the set of machine learning models is configured to generate a grade for an image attribute from the set of image attributes. The set of image attributes includes at least one of an edge, a corner, a center, or a surface. At 808, the set of machine learning models is trained and executed to generate a set of grades for images of collectibles not included in the training set. In some implementations, the set of grades can be used to determine that collectibles not included in the training set are defective and a signal can be sent to cause at least one corrective action to be taken (e.g., flagging the image, flagging the collectible, notifying a user, etc.). In some implementations, at least one image from the set of images is captured using at least one first camera setting, and images of collectibles not included in the training set are captured using a second camera setting different from the at least one first camera setting.

[0069]

[0076] It should be understood that the disclosed embodiments do not represent all innovations as claimed. As such, certain aspects of the disclosure are not discussed herein. Alternative embodiments may not be presented for certain parts of the innovation, or that further undescribed alternative embodiments may be available for certain parts, and should not be considered as excluding those alternative embodiments. Thus, it should be understood that other embodiments may be utilized, and functional, logical, operational, organizational, structural, and / or topological modifications may be made without departing from the scope of the disclosure. As such, all examples and / or embodiments throughout this disclosure are considered non-limiting.

[0070]

[0077] Some embodiments described herein relate to methods. It should be understood that such methods may be computer-implemented methods (e.g., instructions stored in a memory and executed on a processor). Although the methods described above show certain events occurring in a certain order, the order of certain events may be modified. In addition, certain of the events may be performed repeatedly and simultaneously in parallel processes where possible, as well as sequentially as described above. Furthermore, certain embodiments may omit one or more of the described events.

[0071]

[0078] Some embodiments described herein relate to computer storage products having a non-transitory computer-readable medium (which may also be referred to as a non-transitory processor-readable medium) having thereon instructions or computer code for performing various computer-implemented operations. The computer-readable medium (or processor-readable medium) is non-transitory in the sense that it does not include a transitory propagating signal itself (e.g., a propagating electromagnetic wave that carries information over a transmission medium such as space or a cable). The medium and computer code (which may also be referred to as code) may be designed and constructed for a specific purpose. Examples of non-transitory computer-readable media include, but are not limited to, magnetic storage media such as hard disks, floppy disks, and magnetic tapes, optical storage media such as compact disks / digital video disks (CD / DVDs), compact disk read-only memory (CD-ROM), and holographic devices, magneto-optical storage media such as optical disks, carrier wave signal processing modules, and hardware devices that are specifically configured to store and execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), read-only memory (ROM), and random access memory (RAM) devices. Other embodiments described herein relate to computer program products, which may include, for example, instructions and / or computer code discussed herein.

[0072]

[0079] To address various problems and advance the art, the entirety of this application (including the cover page, title, headings, background, summary, brief description of drawings, detailed description, claims, abstract, drawings, appendices, etc.) illustrates various embodiments in which the embodiments may be practiced. The advantages and features of this application are merely representative samples of embodiments and are not intended to be exhaustive and / or exclusive. They are presented to aid in the understanding and to teach the principles recited in the claims.

[0073]

[0080] Examples of computer code include, but are not limited to, microcode or microinstructions, machine instructions such as produced by a compiler, code used to create web services, and files containing higher level instructions that are executed by a computer using an interpreter. For example, embodiments may be implemented using Python, Java, JavaScript, C++, and / or other programming languages, packages, and software development tools.

[0074]

[0081] The drawings are primarily for illustrative purposes and are not intended to limit the scope of the subject matter described herein. The drawings are not necessarily drawn to scale, and in some instances various aspects of the subject matter disclosed herein may be exaggerated or enlarged in the drawings to aid in understanding various features. In the drawings, like reference characters generally refer to like features (e.g., functionally similar and / or structurally similar elements).

[0075]

[0082] Acts performed as part of the disclosed methods may be ordered in any suitable manner. Thus, embodiments may be constructed in which processes or steps are performed in a different order than shown, and even if an exemplary embodiment shows acts as sequential, such embodiments may include performing some steps or processes simultaneously. In other words, it should be understood that such features may not necessarily be limited to a particular order of execution, but rather any number of threads, processes, services, servers, etc. may execute sequentially, asynchronously, in parallel, in parallel, simultaneously, synchronously, etc. in a manner consistent with the present disclosure. As such, some of the features may be inconsistent with each other in that they cannot exist together in a single embodiment. Similarly, some features may be applicable to some aspects of the innovation and not to other aspects.

[0076]

[0083] As used herein and in the embodiments, the phrase "and / or" should be understood to mean "either or both" of the elements so conjoined, i.e., elements that are conjunctive in some cases and disjunctive in other cases. Multiple elements listed with "and / or" should be interpreted similarly, i.e., as "one or more" of the elements so conjoined. Elements other than the elements specifically identified by the "and / or" clause may optionally be present, whether or not related to those elements specifically identified. Thus, as a non-limiting example, when used in conjunction with open-ended language such as "comprises," a reference to "A and / or B" may refer in one embodiment to only A (optionally including elements other than B), in another embodiment to only B (optionally including elements other than A), in yet another embodiment to both A and B (optionally including other elements), etc.

[0077]

[0084] As used herein and in the embodiments, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as being inclusive, i.e., including at least one but also more than one of the elements or list, and optionally including additional items not listed. Only terms clearly indicating the contrary, such as "only one of" or "exactly one of," or when used in the embodiments, "consisting of," refer to the inclusion of exactly one element of the elements or list. In general, as used herein, the term "or" should only be interpreted to indicate exclusive alternatives (i.e., "one or the other but not both") when preceded by an exclusive term, such as "either," "one of," "only one of," or "exactly one of." When used in the embodiments, "consisting essentially of" shall have its ordinary meaning as used in the field of patent law.

Claims

1. receiving a set of images of a plurality of collectibles, each image of the set of images associated with at least one defect type label, a first grading label related to a surface condition of a collectible from the plurality of collectibles, a second grading label related to an edge condition of the collectible, and a third grading label related to a corner condition of the collectible; generating a set of pre-processed images based on the set of images by, for each image in the set of images, detecting a boundary defining the collection in that image, performing a perspective warp transformation on that image from the set of images for which the boundary does not have a predetermined shape, and removing portions of that image that are not within the boundary defining the collection; training at least one model based on each preprocessed image from the set of preprocessed images, at least one defect type label associated with the preprocessed image, and at least one of (1) the first grade classification label associated with the preprocessed image, (2) the second grade classification label associated with the preprocessed image, or (3) the third grade classification label associated with the preprocessed image, wherein the at least one model: (1) a first model trained using (a) each preprocessed image from the set of preprocessed images, and (b) the first grade classification label associated with that preprocessed image; (2) a second model trained using (a) each preprocessed image from the set of preprocessed images, and (b) the second grade classification label associated with the preprocessed image; and (3) training a third model including: (a) each preprocessed image from the set of preprocessed images; and (b) the third grade classification label associated with that preprocessed image; applying the at least one model to a new image of a new collectible not included in the plurality of collectibles; and displaying an output indicating that the new collection contains a defect, an approximate location of the defect, and a defect type associated with the defect. A method comprising:

2. The method of claim 1 , wherein a first image from the set of images is taken under a first lighting condition and a second image from the set of images is taken under a second lighting condition that is different from the first lighting condition.

3. 2. The method of claim 1, wherein a first image from the set of images is taken at a first angle relative to a first collection from the plurality of collections, and a second image from the set of images is taken at a second angle relative to one of the first collection or a second collection from the plurality of collections different from the first collection, the second angle being different from the first angle.

4. The method of claim 1 , wherein a first image from the set of images is taken with a first background and a second image from the set of images is taken with a second background different from the first background.

5. 2. The method of claim 1, wherein said generating the set of pre-processed images further comprises resizing each image from the set of images having a dimension other than a predetermined dimension so that the image has the predetermined dimension.

6. 2. The method of claim 1 , wherein said generating the set of pre-processed images further comprises resizing each image from the set of images having a resolution not within a predetermined resolution range to cause that image to have the resolution within the predetermined resolution range.

7. The method of claim 1 , wherein the at least one model includes at least one dropout layer to reduce overfitting.

8. improving hyperparameters associated with the at least one model using at least one of a random search algorithm, a hyperband algorithm, or a Bayesian optimization algorithm; The method of claim 1 further comprising:

9. The method of claim 1, wherein each image from the set of images is further associated with a fourth grade classification label indicating a centering state of the collectible, and the at least one model further includes: (1) a fourth model trained using (a) each preprocessed image from the set of preprocessed images and (b) the fourth grade classification label associated with the preprocessed image; and (2) a fifth model trained using (a) each preprocessed image from the set of preprocessed images and (b) the at least one defect type label associated with the preprocessed image.

10. A non-transitory processor-readable medium storing code representing instructions for execution by a processor, the instructions comprising: generating a preprocessed image by preprocessing an image of a collectible to detect a boundary in the image that defines the collectible, performing a perspective warp transformation on the boundary to have a predetermined shape, and removing portions of the image that are not within the boundary that defines the collectible; applying a machine learning (ML) model to the pre-processed image to generate a plurality of defect confidence levels, each defect confidence level from the plurality of defect confidence levels (1) associated with a unique portion of the pre-processed image from a plurality of unique portions of the pre-processed image and (2) indicative of a likelihood that at least one defect is present within that unique portion of the pre-processed image, the ML model being trained using a training dataset including at least one image and at least one synthetic image generated by adjusting a brightness of the at least one image; displaying the preprocessed image on a display; and showing on the display each unique portion of the preprocessed image from the plurality of unique portions associated with a defect confidence level from the plurality of defect confidence levels that is outside a predetermined range.

23. A non-transitory processor-readable medium comprising code for causing the processor to:

11. the ML model is a first ML model, and the code comprises: applying a second ML model to the pre-processed image to generate a first score indicative of a surface condition of the collectible; applying a third ML model to the pre-processed image to generate a second score indicative of an edge condition of the collection; applying a fourth ML model to the pre-processed image to generate a third score indicative of a corner condition of the collectible; applying a fifth ML model to the pre-processed image to generate a fourth score indicative of a centering condition of the collection; assigning at least one label to the collectible indicating an overall condition of the collectible based on the first score, the second score, the third score, and the fourth score; and Displaying each defect confidence level from the plurality of defect confidence levels superimposed on a unique portion of the preprocessed image associated with that defect confidence level.

11. The non-transitory processor-readable medium of claim 10, further comprising code for causing the processor to:

12. The code, applying a computer vision model to the pre-processed images to identify at least one of a card type, player information, or character information associated with the collectible; and further comprising code for causing the processor to: At least one of the card type, the player information, or the character information is used by at least one of the first ML model for generating the plurality of defect confidence levels, the second ML model for generating the first score, the third ML model for generating the second score, the fourth ML model for generating the third score, or the fifth ML model for generating the fourth score.

12. The non-transitory processor-readable medium of claim 11.

13. The code, determining at least one of a card type, player information, or character information for the preprocessed image; and further comprising code for causing the processor to: The ML model is further applied to at least one of the card type, the player information, or the character information to generate the plurality of defect confidence levels.

11. The non-transitory processor-readable medium of claim 10.

14. 11. The non-transitory processor-readable medium of claim 10, wherein the pre-processing further comprises resizing the image to have a resolution within a predetermined resolution range.

15. The non-transitory processor-readable medium of claim 10 , wherein the predetermined shape is a rectangle.

16. the applying of the first ML model occurs prior to the applying of the second ML model, the applying of the third ML model, the applying of the fourth ML model, and the applying of the fifth ML model; At least two of the applying of the second ML model, the applying of the third ML model, the applying of the fourth ML model, or the applying of the fifth ML model are performed in parallel.

12. The non-transitory processor-readable medium of claim 11.

17. Memory and a processor operatively coupled to the memory; 11. An apparatus comprising: dilating a set of images of the collection to generate a set of composite images of the collection, where at least one image from the set of images is shifted to generate at least one composite image included in the set of composite images; combining a set of images of the collection with a set of synthetic images of the collection to produce a training set; training a set of machine learning models based on the training set, each machine learning model from the set of machine learning models configured to generate a grade for an image attribute from a set of image attributes, the set of image attributes including at least one of an edge, a corner, a center, or a surface; and Post-training and executing the set of machine learning models to generate a set of grades for images of collectibles not included in the training set. configured to: device.

18. 18. The device of claim 17, wherein the expanding comprises at least one of rotating a first image from the set of images, vertically shifting the first image, horizontally shifting the first image, scaling the first image, adjusting a brightness of the first image, adjusting a contrast of the first image, vertically flipping the first image, or horizontally flipping the first image.

19. 20. The apparatus of claim 17, wherein at least one image from the set of images is captured using at least one first camera setting, and the images of the collection not included in the training set are captured using a second camera setting different from the at least one first camera setting.

20. 20. The device of claim 17, wherein the collectible is at least one of a trading card, a coin, or a currency.

Images