Systems and methods associated with neural networks based on the physical state evaluation of electronic devices

Abstract

To provide systems and methods for evaluating the physical and / or cosmetic condition of electronic devices using machine learning techniques.SOLUTION: In one example aspect, an example system includes a kiosk that comprises an inspection plate configured to hold an electronic device, one or more light sources arranged above the inspection plate configured to direct one or more light beams towards the electronic device, and one or more cameras configured to capture at least one image of a first side of the electronic device. The system also includes one or more processors in communication with the one or more cameras configured to extract a set of features of the electronic device and determine, via a first neural network, a condition of the electronic device based on the set of features.SELECTED DRAWING: None

Description

Technical Field

[0001] (Cross - reference to Related Applications) This patent application claims the priority and benefit of U.S. Provisional Patent Application No. 62 / 807,165, filed on February 18, 2019, entitled "NEURAL NETWORK BASED PHYSICAL CONDITION EVALUATION OF ELECTRONIC DEVICES, AND ASSOCIATED SYSTEMS AND METHODS". The entire content of the patent applications described above is hereby incorporated by reference in its entirety into this specification.

[0002] This technology generally aims to evaluate the condition of such devices, such as evaluating the presence, quantity, and / or distribution of surface scratches or cracks in mobile phones and / or other electronic devices based on machine learning techniques.

Background Art

[0003] Consumer electronic devices such as mobile phones, laptop computers, notebooks, tablets, MP3 players, etc. are ubiquitous. Currently, more than 6 billion mobile devices are used worldwide, and the number of these devices is growing rapidly, with more than 1.8 billion mobile phones sold in just 2013. Currently, more mobile devices are being used than there are people on Earth. Part of the reason for the rapid growth in the number of mobile phones and other electronic devices is the rapid pace at which these devices are evolving and the increasing use of such devices in third - world countries.

[0004] As a result of the rapid pace of development, a relatively high percentage of electronic devices are replaced annually as consumers continuously upgrade their mobile phones and other electronic devices to acquire the latest features or better operational plans. According to the U.S. Environmental Protection Agency, 370 million mobile phones, PDAs, tablets, and other electronic devices are discarded in the United States alone every year. Hundreds of other outdated or broken mobile phones and other electronic devices are simply tossed into junk drawers or stored until a more suitable disposal solution arises.

[0005] While many electronics retailers and mobile phone stores now offer trade-in or buyback programs for older phones, many older phones still end up in landfills or are improperly dismantled and discarded in developing countries. Unfortunately, however, mobile phones and similar devices typically contain substances that can be harmful to the environment, such as arsenic, lithium, cadmium, copper, lead, mercury, and zinc. If not properly disposed of, these toxic substances can seep from decomposing landfills into groundwater, contaminating the soil with potentially harmful consequences for humans and the environment.

[0006] As an alternative to retailers' trade-in or buyback programs, consumers can now recycle and / or sell their used mobile phones using self-service kiosks located in shopping malls, retail stores, or other publicly accessible areas. Such kiosks are operated by ecoATM, LLC, the applicant of this application, and are disclosed, for example, in U.S. Patents 8,463,646, 8,423,404, 8,239,262, 8,200,533, 8,195,511, and 7,881,965 (jointly owned by ecoATM, LLC and incorporated herein by reference as a whole).

[0007] In many cases, it is necessary to visually assess the physical and / or cosmetic condition of electronic devices. For example, pricing an electronic device, assessing it for possible repairs, and evaluating it for warranty coverage may all require identifying scratches, cracks, water damage, or other cosmetic defects on the device's screen and / or non-screen areas. Individualized manual inspection of devices can be slow, cumbersome, and lead to inconsistent results between devices. There remains a need for more efficient technologies for assessing the physical and / or cosmetic condition of electronic devices. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] U.S. Patent No. 8,463,646 [Overview of the Initiative] [Means for solving the problem]

[0009] This disclosure describes various embodiments of systems and methods for evaluating the appearance and / or physical condition of mobile phones and / or other electronic devices using machine learning techniques. In some embodiments, as will be described in more detail below, these systems and methods can be implemented by a consumer-operated kiosk, for example, to evaluate whether the display screen of a mobile phone is cracked or otherwise damaged.

[0010] Efficiently and consistently evaluating the outward appearance of electronic devices can be challenging. For example, manual identification of defects, such as those shown in images of electronic devices, is expensive, cumbersome, and susceptible to variability even between different or the same inspector. Manual processes can also be inaccurate in many cases. For instance, when a device's screen is on, a human inspector may not be able to distinguish between outward defects and the background image displayed on the device. In another example, a screen protector or case attached to the device can make manual inspection difficult. In this regard, certain feature or rule-based automated pattern recognition methods may not be able to provide satisfactory and consistent evaluation results. In addition, the evaluation of outward appearance cannot be limited to the identification of a predetermined set of defects (e.g., scratches, cracks, dents, water damage, and / or defective pixels). Rather, the evaluation may address the comprehensive and overall "look and feel" of an electronic device, such as identifying whether a device is counterfeit. Therefore, predetermined feature or rule-based methods may be inefficient and / or insufficient to handle various outward appearance evaluation scenarios.

[0011] An aspect of this technology is that it uses machine learning techniques (particularly artificial neural networks (ANNs)) to perform visual condition assessments based on images of electronic devices without pre-determined features or rules. In particular, the use of ANNs as described herein contributes to various advantages and improvements (e.g., computational efficiency, detection accuracy, system robustness, etc.) when processing images of electronic devices. As those skilled in the art will understand, an ANN is generally a computing system that "learns" a task (i.e., gradually improves its performance) by considering examples without task-specific programming. For example, in image recognition, an ANN can learn to identify images containing cats by analyzing exemplary images manually labeled "cat" or "no cat" and using the results to identify cats in other images.

[0012] An ANN is typically based on a set of connected units or nodes called artificial neurons. Each connection between artificial neurons can transmit signals from one artificial neuron to another. The receiving artificial neuron can process the signal and then signal to the artificial neurons it connects to. Typically, in an ANN implementation, the signals in the connections between artificial neurons are real numbers, and the output of each artificial neuron is calculated by a nonlinear function of the sum of its inputs. Artificial neurons and connections typically have weights that adapt as learning progresses. The weights increase or decrease the intensity of the signals in the connections. Artificial neurons can have thresholds such that a signal is transmitted only when the aggregated signal exceeds a threshold. Typically, artificial neurons are organized in layers. Different layers can perform different kinds of transformations on their inputs. Signals can potentially traverse the layers multiple times, progressing from the first layer (input) to the last (output) layer.

[0013] In some embodiments, one or more ANNs used by the technique include convolutional neural networks (CNNs or ConvNets). Typically, CNNs use a variation of a multilayer perceptron designed to require minimal preprocessing. CNNs can also be shift-invariant or spatial-invariant artificial neural networks (SIANNs) based on their shared weighting architecture and translation-invariant properties. Illustratively, CNNs are inspired by biological processes in that the connectivity patterns between neurons are analogous to the structure of an animal's visual cortex. Individual cortical neurons respond only to stimuli within a limited area of ​​the visual field, known as their receptive field. The receptive fields of different neurons partially overlap so that they cover the entire visual field. The present invention provides, for example, the following items: (Item 1) A system for evaluating the state of electronic devices, It's a kiosk, An inspection plate configured to hold the aforementioned electronic device, One or more light sources arranged above the inspection plate, configured to direct one or more light beams toward the electronic device, One or more cameras configured to capture at least one image of a first side of the electronic device based on at least one illumination condition generated by the one or more light sources, Including kiosks, One or more processors that communicate with the one or more cameras, the one or more processors Extracting a set of features of the electronic device based on the at least one image of the electronic device, The state of the electronic device is determined based on the extracted set of features via a first neural network. One or more processors configured to perform the following: A system that includes these features. (Item 2) The system according to item 1, wherein the one or more light sources comprises a first subset of light sources and a second subset of light sources, and the light beams of the first subset of light sources and the light beams of the second subset of light sources are arranged to be orthogonal to each other. (Item 3) The aforementioned kiosk further, An upper chamber positioned above the inspection plate, wherein one or more light sources are arranged within the upper chamber, A lower chamber located below the inspection plate, A second set of light sources positioned within the lower chamber, configured to direct a light beam toward the electronic device through the inspection plate, and The system described in item 1, including the system described in item 1. (Item 4) The aforementioned kiosk further, An inversion mechanism configured to invert the electronic device and enable the one or more cameras to capture at least another image of a second side of the electronic device The system according to item 1, including (Item 5) The system according to item 1, wherein at least one of the one or more light sources is configured to generate a collimated light beam. (Item 6) The system according to item 1, wherein the angle between the light beam from one of the one or more light sources and the first side of the electronic device is equal to or less than 60 degrees. (Item 7) The system according to item 1, wherein the one or more cameras are configured to capture a plurality of images corresponding to a plurality of sides of the electronic device under different illumination conditions, and the one or more processors are configured to process the plurality of images and combine them into a single input image. (Item 8) The system according to item 1, wherein the first neural network is configured to output an indicator indicating the state of the electronic device. (Item 9) The system according to item 1, wherein the one or more processors are further configured to determine an estimated price of the electronic device based on the state. (Item 10) The system according to item 1, wherein the state includes a physical state or an appearance state. (Item 11) A system for evaluating the state of an electronic device, comprising A capture device comprising at least one light source and at least one camera, wherein the at least one camera is configured to capture a plurality of images of the electronic device based on one or more predetermined settings, and each of the one or more predetermined settings is (1) an angle at which the capture device is positioned with respect to the electronic device, (2) the light intensity of the at least one light source, (3) an exposure setting of the at least one camera, or (4) a white balance setting of the at least one camera, and a capture device that defines at least one of them, One or more processors communicating with the capture device, wherein the one or more processors are Processing the plurality of images to generate a single input image, Extracting a set of features of the electronic device based on at least one image of the electronic device, Determining the state of the electronic device via a first neural network One or more processors configured to perform, A system comprising. (Item 12) The system according to item 11, wherein the state comprises a physical state or an appearance state. (Item 13) A computer-implemented method for evaluating the state of an electronic device, comprising: Capturing at least one image of a first side of the electronic device by at least one camera of a kiosk, wherein the kiosk includes a plurality of light sources, Extracting a set of features of the electronic device based on the at least one image of the electronic device, Determining the state of the electronic device based on the set of features by a neural network A method comprising. (Item p) The method according to item 13, comprising capturing an image of at least one second side of the electronic device, different from the first side, via the at least one camera, based on at least one illumination condition generated by the plurality of light sources. (Item 15) The method of item 14, comprising inverting the electronic device so that the light beams of the plurality of light sources are directed toward the second side of the electronic device, prior to capturing the at least one image of the second side of the electronic device. (Item 16) Processing multiple images of multiple sides of the electronic device such that the multiple images have a uniform size, The process of combining the aforementioned multiple images into a single image to be provided to the neural network. The method described in item 13, including the method described in item 13. (Item 17) The method according to item 13, comprising adjusting one of the plurality of light sources such that the angle between a light beam from one of the plurality of light sources and the first side of the electronic device is equal to or less than 60 degrees. (Item 18) Determining the model of the electronic device based in part on at least one of the aforementioned images, To identify the apparent defects on the electronic device that are specific to the aforementioned model. The method described in item 13, including the method described in item 13. (Item 19) Receiving input from the user indicating acceptance or rejection of the proposed price, Training the neural network based in part on the at least one image and the input from the user. The method described in item 13, including the method described in item 13. (Item 20) The method according to item 13, wherein the state comprises a physical state or an external state. [Brief explanation of the drawing]

[0014] [Figure 1A] Figure 1A is a schematic diagram of a typical operating environment having elements configured according to several embodiments of the present technology.

[0015] [Figure 1B] Figures 1B-1E are a series of isometric views of the kiosk shown in Figure 1A, with the housing removed to illustrate selected internal components, configured according to several embodiments of the present technology. [Figure 1C] Figures 1B-1E are a series of isometric views of the kiosk shown in Figure 1A, with the housing removed to illustrate selected internal components, configured according to several embodiments of the present technology. [Figure 1D] Figures 1B-1E are a series of isometric views of the kiosk shown in Figure 1A, with the housing removed to illustrate selected internal components, configured according to several embodiments of the present technology. [Figure 1E] Figures 1B-1E are a series of isometric views of the kiosk shown in Figure 1A, with the housing removed to illustrate selected internal components, configured according to several embodiments of the present technology.

[0016] [Figure 2] Figure 2 is a flowchart illustrating a method for evaluating the external state of an electronic device according to several embodiments of this technology.

[0017] [Figure 3] Figure 3 illustrates an exemplary neural network that can be implemented according to several embodiments of this technology.

[0018] [Figure 4A] Figure 4A illustrates an example of a pre-processed image showing the front of a smartphone according to several embodiments of the present technology.

[0019] [Figure 4B]Figure 4B illustrates another example of a pre-processed image showing the front of a smartphone, according to several embodiments of the present technology.

[0020] [Figure 5] Figure 5 is a flowchart illustrating a method for training a neural network to evaluate the external state of an electronic device, according to several embodiments of this technology.

[0021] [Figure 6] Figure 6 is a block diagram illustrating an exemplary architecture for a computer system that can be used to implement various parts of this technology.

[0022] [Figure 7] Figure 7 is a flowchart representing a method for evaluating the physical state of an electronic device according to several embodiments of this technology.

[0023] [Figure 8] Figure 8 illustrates exemplary architectures of a system for examining consumer devices and providing a proposed price, according to several embodiments of the present technology.

[0024] [Figure 9A] Figure 9A shows a side view of an exemplary arrangement of light sources 901a and 901b in the upper chamber according to one or more embodiments of the present technology.

[0025] [Figure 9B] Figure 9B illustrates exemplary arrangements of two sets of light sources according to several embodiments of the present technology.

[0026] [Figure 10] Figure 10 illustrates an example of evaluating an electronic device using another mobile device, according to several embodiments of the present technology.

[0027] [Figure 11]Figure 11 illustrates exemplary architectures for training neural networks according to several embodiments of this technology. [Modes for carrying out the invention]

[0028] Figures 1A–E illustrate details of kiosk models according to several embodiments of the present technology. Figure 1A illustrates an exemplary kiosk 100 for the resale, sale, and / or other processing of mobile phones and other consumer electronic devices according to several embodiments of the present technology. In some embodiments, at least some parts of the technology described herein can be performed using a kiosk that includes an imaging device therein. For example, the kiosk can process and evaluate images received from the imaging device. The kiosk may include, for example, a processing component (e.g., including one or more physical processors) and a memory for storing instructions that, when performed by the processing component, perform at least some of the operations described herein. The term “processing” is used herein for ease of reference to generally refer to any kind of service and operation that can be performed or facilitated by the kiosk 100 on, using, or otherwise in relation to an electronic device. Such services and operations may include, for example, selling, reselling, recycling, donating, exchanging, identifying, appraising, pricing, auctioning, decommissioning, transferring data to or from, reconfiguring, and modifying mobile phones and other electronic devices. While many embodiments of the Technology are described herein in relation to mobile phones, aspects of the Technology are not limited to mobile phones and can generally be applied to other consumer electronic devices. Such devices, in non-limiting embodiments, include all types of mobile phones, smartphones, handheld devices, personal digital assistants (PDAs), MP3 or other digital music players, tablets, notebooks, ultrabooks, and laptop computers, all types of e-readers, GPS devices, set-top boxes, universal remote controls, wearable computers, and the like.In some embodiments, it is considered that the kiosk 100 may facilitate the sale and / or disposal of larger consumer electronic devices such as desktop computers, televisions, and game consoles, and smaller electronic devices such as Google® Glass™, smartwatches (e.g., Android Wear™ devices such as Apple Watch™, Moto 360®, or Pebble Steel™ watches). Embodiments of the kiosk 100 and its various features are incorporated as a whole by reference to the following patents and patent applications, namely, U.S. Patent Nos. 10,127,647, 10,055,798, 10,032,140, ​​9,904,911, 9,881,284, 8,200,533, 8,195,511, 8,463,646, 8,423,404, 8,239,262, 8,200,533, 8,195,511, and 7,881,965, U.S. Patent Application No. 12 / 573,089, No. 12 / 727,624, No. 13 / 113,497, No. 12 / 785,465, No. 13 / 017,560, No. 13 / 438,924 , No. 13 / 753,539, No. 13 / 658,825, No. 13 / 733,984, No. 13 / 705,252, No. 13 / 487,299 Issues 13 / 492,835, 13 / 562,292, 13 / 658,828, 13 / 693,032, 13 / 792,030, 13 / 794,814, 13 / 794,816, 13 / 862,395, and 13 / 913,408, "METHODS AND SYSTEMS FOR PRICING AND PERFORMING OTHER PROCESSES ASSOCIATED WITH RECYCLING MOBILE PHONES AND OTHER." U.S. Patent Application No. 14 / 498,763, titled "ELECTRONIC DEVICES," filed by the applicant on September 26, 2014, "MAINTAINING SETS OF CABLE COMPONENTS USED FOR WIRED U.S. Patent Application No. 14 / 500,739, titled "Analysis, Charging, or Other Interaction with Portable Electronic Devices," filed by the applicant on September 29, 2014, for "Wireless-Enabled Kiosk for Recycling" U.S. Patent Application No. 14 / 873,158, titled "CONSUMER DEVICES," filed by the applicant on October 1, 2015; U.S. Patent Application No. 14 / 506,449, titled "SYSTEM FOR ELECTRICALLY TESTING MOBILE DEVICES AT A CONSUMER-OPERATED KIOSK, AND ASSOCIATED DEVICES AND METHODS," filed by the applicant on October 3, 2014; U.S. Patent Application No. 14 / 925,357, titled "SYSTEMS AND METHODS FOR RECYCLING CONSUMER ELECTRONIC DEVICES," filed by the applicant on October 28, 2015; "METHODS AND SYSTEMS FOR FACILITATING PROCESSES ASSOCIATED WITH INSURANCE SERVICES AND / OR OTHER SERVICES FOR ELECTRONIC U.S. Patent Application No. 14 / 925,375, filed by the applicant on October 28, 2015, titled "METHODS AND SYSTEMS FOR EVALUATING AND RECYCLING ELECTRONIC DEVICES", U.S. Patent Application No. 14 / 934,134, filed by the applicant on November 5, 2015, titled "METHODS AND SYSTEMS FOR PROVIDING INFORMATION REGARDING COUPONS / PROMOTIONS AT KIOSKS FOR RECYCLING MOBILE PHONES AND OTHER ELECTRONIC DEVICES", U.S. Patent Application No. 14 / 964,963, filed by the applicant on December 10, 2015, titled "METHODS AND SYSTEMS FOR IDENTIFYING MOBILE PHONES AND OTHER ELECTRONIC U.S. Patent Application No. 14 / 568, titled “DEVICES”, was filed by the applicant on December 11, 2014.U.S. Patent Application No. 14 / 966,346, filed by the applicant on December 11, 2015, titled "SYSTEMS AND METHODS FOR RECYCLING CONSUMER ELECTRONIC DEVICES"; U.S. Patent Application No. 14 / 598,469, filed by the applicant on January 16, 2015, titled "METHODS AND SYSTEMS FOR DYNAMIC PRICING AND PERFORMING OTHER PROCESSES ASSOCIATED WITH RECYCLING MOBILE PHONES AND OTHER ELECTRONIC DEVICES"; U.S. Patent Application No. 14 / 660,768, filed by the applicant on March 17, 2015, titled "SYSTEMS AND METHODS FOR INSPECTING MOBILE DEVICES AND OTHER CONSUMER ELECTRONIC DEVICES WITH A LASER"; "DEVICE RECYCLING U.S. Patent Application No. 14 / 663,331, filed by the applicant on March 19, 2015, titled "SYSTEMS WITH FACIAL RECOGNITION," U.S. Provisional Application No. 62 / 169,072, filed by the applicant on June 1, 2015, titled "METHODS AND SYSTEMS FOR VISUALLY EVALUATING ELECTRONIC DEVICES," U.S. Provisional Application No. 62 / 202,330, filed by the applicant on August 7, 2015, titled "METHODS AND SYSTEMS FOR INSPECTING MOBILE DEVICES AND OTHER CONSUMER ELECTRONIC DEVICES WITH ROBOTIC ACTUATION," and "METHODS AND SYSTEMS FOR INTERACTIONS WITH A SYSTEM FOR PURCHASING, The systems, methods, and corresponding features described in U.S. Patent Application No. 15 / 057,707, titled “MOBILE PHONES AND OTHER ELECTRONIC DEVICES,” filed by the applicant on March 1, 2016, and in U.S. Patent Application No. 15 / 176,975, titled “METHODS AND SYSTEMS FOR DETECTING SCREEN COVERS ON ELECTRONIC DEVICES,” filed by the applicant on June 8, 2016, may be at least generally similar in structure and function to those described in the U.S. Patent Application No. 15 / 176,975, titled “METHODS AND SYSTEMS FOR DETECTING SCREEN COVERS ON ELECTRONIC DEVICES.” In some embodiments, the kiosk 100 is based on U.S. Patent Application No. 16 / 719,699, filed December 18, 2019, titled "SYSTEMS AND METHODS FOR VENDING AND / OR PURCHASING MOBILE PHONES AND OTHER ELECTRONIC DEVICES", U.S. Patent Application No. 16 / 788,169, filed February 11, 2020, titled "KIOSK FOR EVALUATING AND PURCHASING USED ELECTRONIC DEVICES", U.S. Patent Application No. 16 / 788,153, filed February 11, 2020, titled "CONNECTOR CARRIER FOR ELECTRONIC DEVICE KIOSK", and "SYSTEMS AND METHODS FOR VENDING AND / OR PURCHASING MOBILE PHONES AND OTHER ELECTRONIC Many or all of the features of the kiosk may be shared with those disclosed and described in U.S. Provisional Application No. 62 / 950,075, titled “DEVICES,” filed on December 18, 2019. All patents and patent applications listed in the preceding sentences, and any other patents or patent applications identified herein, are incorporated herein by reference as a whole.

[0029] In the illustrated embodiment, kiosk 100 is a floor-standing self-service kiosk configured for use by a user 101 (e.g., a consumer, customer, etc.) to recycle, sell, and / or perform other operations using a mobile phone or other consumer electronic device. In other embodiments, kiosk 100 may be configured for use on a counter or similar raised surface. Although kiosk 100 is configured for consumer use, in various embodiments, kiosk 100 and / or various parts thereof may also be used by other operators, such as retail staff or kiosk assistants, to facilitate the sale or other processing of mobile phones and other electronic devices.

[0030] In the illustrated embodiments, the kiosk 100 includes a housing 102 which is approximately the size of a conventional vending machine. The housing 102 may be a conventional product made from, for example, a metal plate, a plastic panel, etc. Multiple user interface devices are provided on the front portion of the housing 102 to provide instructions and other information to the user and / or to receive user input and other information from the user. For example, the kiosk 100 may include a display screen 104 (e.g., a liquid crystal display (LCD) or light-emitting diode (LED) display screen, a projection display (head-up display or head-mounted device, etc.)) for providing information, prompts, etc. to the user. The display screen 104 may include a touchscreen for receiving user input and responses to displayed prompts. In some embodiments, the kiosk 100 may include a separate keyboard or keypad for this purpose. The kiosk 100 may also include an ID reader or scanner 112 (e.g., a driver's license scanner), a fingerprint reader 114, and one or more cameras 116a-c (e.g., digital still and / or video cameras, individually identified as cameras). In addition, the kiosk 100 may include an output device such as a label printer having an exit 110, and an ATM having an exit 118. Not identified in Figure 1A-1E, the kiosk 100 may further include a speaker and / or headphone jack for communicating information audibly to the user, one or more lights for visually communicating signals or other information to the user, a handset or microphone for receiving oral input from the user, a card reader (e.g., a credit / debit card reader, a loyalty card reader, etc.), a receipt or voucher printer and dispenser, and other user input and output devices. Input devices may include pointing devices such as a touchpad or mouse, a joystick, a pen, a gamepad, a motion sensor, a scanner, a gaze direction monitoring system, etc.In addition, the kiosk 100 may also include a barcode reader, a QR code® reader, a bag / package dispenser, a digital signature pad, etc. In the illustrated embodiment, the kiosk 100 also includes a header 120 having a display screen 122 for displaying marketing advertisements and / or other video or graphical information to attract users to the kiosk. In addition to the user interface devices described above, the front portion of the housing 102 also includes an access panel or door 106 located directly below the display screen 104. The access door may be configured to automatically retract so that a user 101 can place an electronic device (e.g., a mobile phone) into the inspection area 108 for automated inspection, evaluation, and / or other processing by the kiosk 100.

[0031] The side walls of the housing 102 may include several conveniences to help the user recycle or otherwise dispose of their mobile phone. For example, in the illustrated embodiment, the kiosk 100 includes an accessory sorting container 128 configured to receive mobile device accessories that the user wishes to recycle or otherwise dispose of. In addition, the kiosk 100 may provide a free charging station 126 with multiple electrical connectors 124 for charging a wide variety of mobile phones and other consumer electronic devices.

[0032] Figures 1B-1E illustrate a series of isometric views of the kiosk 100 with the housing 102 removed to show selected internal components configured according to several embodiments of the present technology. Referring first to Figure 1B, in the illustrated embodiment, the kiosk 100 includes a connector carrier 140 and an inspection plate 144 operably positioned behind the access door 106 as shown in Figure 1A. In the illustrated embodiment, the connector carrier 140 is a rotatable carousel configured to rotate about a substantially horizontal central axis and carrying a plurality of electrical connectors 142 (e.g., about 25 connectors) distributed around its outer circumference. In other embodiments, other types of connector carrying devices (including both fixed and movable arrays) may also be used. In some embodiments, the connectors 142 include a plurality of interchangeable USB connectors configured to provide power and / or exchange data with various different mobile phones and / or other electronic devices. During operation, the connector carrier 140 is configured to automatically rotate around its axis, positioning the appropriate connector 142 adjacent to an electronic device such as a mobile phone 150 placed on an inspection plate 144 for recycling. The connector 142 can then be manually and / or automatically pulled out from the connector carrier 140 and connected to a port on the mobile phone 150 for electrical analysis. Such analysis may include, for example, an evaluation of the manufacturer, model, configuration, condition, etc.

[0033] In the illustrated embodiment, the inspection plate 144 is configured to move back and forth (for example, on a parallel mounting track) to move an electronic device such as a mobile phone 150 between a first position directly behind the access door 106 and a second position between the upper chamber 130 and the lower chamber 132 facing it. Furthermore, in this embodiment, the inspection plate 144 is transparent or at least partially transparent (e.g., formed from glass, plexiglass, etc.) and allows the mobile phone 150 to be photographed from all or at least most of the field of view (e.g., top, bottom, sides, etc.) and / or otherwise optically evaluated using an imaging device 190 (e.g., one or more cameras) mounted in or otherwise associated with the upper and lower chambers 130 and 132. When the mobile phone 150 is in the second position, the upper chamber 130 moves downward so that the mobile phone 150 can generally be enclosed between the upper chamber 130 and the lower chamber 132. The upper chamber 130 is operably coupled to a gate 138 that moves up and down in coordination with the upper chamber 130.

[0034] In some embodiments, the imaging device 190 may include one or more cameras positioned within both the upper chamber 130 and the lower chamber 132 to capture images of the top and bottom surfaces of the mobile device 150 in order to detect cracks and / or scratches on the screen. The upper chamber 130 and / or the lower chamber 132 may include one or more light sources (e.g., spotlights) to enable the imaging device 190 to capture high-quality images demonstrating cosmetic defects on the mobile device 150.

[0035] In some embodiments, one or more light sources are arranged within an upper chamber 130 and / or a lower chamber 132. Figure 9A illustrates a side view of an exemplary arrangement of light sources 901a,b in the upper chamber according to one or more embodiments of the Art. The light beams 911a,b from the light sources 901a,b form a small angle (e.g., equal to or less than 60 degrees) with respect to the display of the mobile phone 150 to avoid direct reflection of light from the highly reflective display of the mobile phone 150. The relative positions between one or more light sources 901a,b and one or more cameras 921a,b of the imaging device 190 can be adjusted to ensure that reflected light beams 913a,b from the mobile phone 150 can reach the cameras 921a,b. In some embodiments, the kiosk can perform self-calibration, adjusting the angles of the light sources to ensure that the correct angles are formed. In some embodiments, technicians can be dispatched periodically or upon request to perform kiosk calibration.

[0036] In some embodiments, one or more light sources include two sets of light sources arranged orthogonally to each other. Having two sets of light sources arranged orthogonally allows the camera to capture various combinations of cracks and / or scratches, as cracks and / or scratches on the mobile device 150 may extend in different directions (e.g., both horizontally and / or vertically). For example, a first angle between the light beam from one set of lights and the upper side of the inspection plate 144 may be 30 to 60 degrees (e.g., preferably 45 degrees), while a second angle between the light beam from a second set of lights and the left side of the inspection plate 144 may be 30 to 60 degrees (e.g., preferably 45 degrees). The two sets of lights are positioned orthogonally to each other. Figure 9B illustrates an exemplary arrangement of two sets of light sources according to some embodiments of the Art. A first set of light sources 931a,b is arranged orthogonally to a second set of light sources 941a,b. The light beams 951a,b from the first set of light sources 931a,b are at approximately 45 degrees from both sides of the inspection plate 144 (e.g., the X-axis and / or Y-axis). Similarly, the light beams 961a,b from the second set of light sources 941a,b are at approximately 45 degrees from both sides of the inspection plate 144 (e.g., the X-axis and / or Y-axis). Such an arrangement may help reduce or eliminate image noise or shadows from other components of the kiosk 100 arranged along the sides of the inspection plate. In some embodiments, additional sets of light sources may be arranged within the upper and / or lower chambers to expose damage that would not be visible from the orthogonal arrangement of light sources.

[0037] In some embodiments, light beams from one or more light sources can be collimated to produce clearer shadows of cracks and / or scratches. In some embodiments, one or more light sources support a wide range of brightness so that multiple sets of images can be captured at different light intensities with varying exposure times. For example, different devices may have different background colors (e.g., a white phone or a black phone) which may affect the processing of the captured images. Capturing at least two sets of images at different camera exposures, different light intensities, and / or different white balance settings can enable more accurate processing of the device's visual characteristics.

[0038] Because the mobile phone 150 is positioned on the transparent plate 144, the light beam from the light source located in the lower chamber 132 undergoes additional reflection within the transparent plate 144 before reaching the mobile phone 150, thereby affecting the quality of the captured image. Therefore, in some embodiments, all cameras and light sources of the imaging device 190 are located only within the upper chamber 130. The kiosk 100 may include a reversing mechanism 148 (e.g., a robotic arm) for reversing the mobile phone 150 so that images of both the top and bottom surfaces of the mobile phone 150 can be captured without any reflection between the camera and the mobile phone 150.

[0039] Furthermore, to improve the quality of the captured image, the color of the upper chamber 130 and the lower chamber 132 may be an intermediate gray, such as 18% gray, for calibrating the exposure meter. The appropriate color of the chambers provides sufficient contrast for the flash across the display and the shadows of the fine cracks on the mobile phone 150.

[0040] Images captured by kiosk 100 can be transmitted to a qualified human operator to examine the image quality as a measure to ensure input quality for computer-implemented visual analysis. Alternatively, captured images can be transmitted to another neural network model to automatically determine image quality and provide feedback to the kiosk. If the operator or neural network model determines that images captured by a particular kiosk routinely exhibit certain defects (e.g., images are too dark, images are overexposed, etc.), a technician can be dispatched to recalibrate the kiosk and ensure that consistent input images are acquired by different kiosks.

[0041] In some embodiments, the upper chamber 130 and / or lower chamber 132 may also include one or more magnifying tools, scanners (e.g., barcode scanners, infrared scanners, etc.), or other imaging components (not shown) and arrays of mirrors (similarly not shown) for viewing, photographing, and / or otherwise visually evaluating the mobile phone 150 from multiple viewpoints. In some embodiments, one or more of the cameras and / or other imaging components discussed above may be movable to facilitate device evaluation. For example, as described above with respect to Figure 1A, the imaging device 190 may be attached to a movable mechanical component such as an arm, which may be moved using a belt drive, rack and pinion system, or other preferred drive system coupled to an electronic controller (e.g., a computing device). The inspection area 108 may also include weighing scales, thermal detectors, UV or infrared readers / detectors, and equivalents for further evaluation of electronic devices installed therein. For example, information from weighing scales, UV or infrared readers / detectors may provide accurate information and facilitate the determination of the model of the mobile phone 150. The kiosk 100 may further include an angled sorting plate 136 for directing electronic devices into a collection sorting container 134 located within the lower part of the kiosk 100, separated from the transparent plate 144.

[0042] The kiosk 100 can be used in several different ways to efficiently facilitate the recycling, sale, and / or other processing of mobile phones and other consumer electronic devices. Referring together to Figures 1A-1E, in one embodiment, a user 101 who wishes to sell a used mobile phone, such as a mobile phone 150, approaches the kiosk 100 and, in response to prompts on the display screen 104, identifies the type of device the user wishes to sell (e.g., mobile phone, tablet, etc.). The user may then be prompted to remove any cases, stickers, or other accessories from the device so that it can be accurately assessed. In addition, the kiosk 100 can print and distribute unique identification labels (e.g., small adhesive backing stickers with Quick Response Codes ("QR Code®"), barcodes, or other machine-readable markings, etc.) from the label outlet 110 for the user to attach to the back of the mobile phone 150. After this is complete, the door 106 retracts and opens, allowing the user to place the mobile phone 150 on the transparent plate 144 in the inspection area 108, as shown in Figure 1B. The door 106 then closes, and the transparent plate 144 moves the mobile phone 150 under the upper chamber 130, as shown in Figure 1C. The upper chamber 130 then moves downward, generally enclosing the mobile phone 150 between the upper chamber and the lower chambers 130 and 132, and the cameras and / or other imaging components in the upper chamber and the lower chambers 130 and 132 perform a visual inspection of the mobile phone 150. In some embodiments, the visual inspection of the mobile phone 150 includes the step of performing at least part of method 200 (as shown in Figure 2), at least part of method 500 (as shown in Figure 5), and / or at least part of method 600 (as shown in Figure 6) to evaluate the physical and / or outward condition of the mobile phone 150.In some embodiments, the visual inspection includes a computer-implemented visual analysis (e.g., three-dimensional (3D) analysis) performed by a processing device in the kiosk to verify the identification of the mobile phone 150 (e.g., manufacturer, model, and / or submodel) and / or to evaluate or assess the condition and / or functionality of the mobile phone 150 and / or its various components and systems. For example, the visual analysis may include a computer-implemented evaluation (e.g., digital comparison) of images of the mobile phone 150 taken from top, side, and end view perspectives to determine the length, width, and / or height (thickness) dimensions of the mobile phone 150. The visual inspection may further include a computer-implemented inspection of the display screen and / or other surfaces of the mobile phone 150 to check, for example, glass cracks and / or other damage or defects in the LCD (e.g., defective pixels, etc.).

[0043] Next, referring to Figure 1D, after a visual analysis is performed and the device is identified, the upper chamber 130 returns to its upper position, and the transparent plate 144 returns the mobile phone 150 to its initial position near the door 106. The display screen 104 can also provide an estimated price or estimated price range that the kiosk 100 may suggest to the user regarding the mobile phone 150, based on the visual analysis and / or user input (e.g., input regarding the type, status, etc., of the phone 150). If the user indicates (e.g., via input via the touchscreen) that they wish to proceed with the transaction, the connector carrier 140 automatically rotates the appropriate connector 142 to a position adjacent to the transparent plate 144, and the door 106 is opened again. The user can then be instructed (e.g., via the display screen 104) to pull out the selected connector 142 (and its associated wires) from the carousel 140, plug the connector 142 into the corresponding port on the mobile phone 150 (e.g., a USB port), and reposition the mobile phone 150 within the inspection area on the transparent plate 144. After doing so, the door 106 closes again, and the kiosk 100 (e.g., kiosk CPU) performs an electrical inspection of the device via the connector 142 to further evaluate the condition of the phone and specific components, and operating parameters such as memory, mobile phone carrier, etc. In some embodiments, the electrical inspection may include determining phone manufacturer information (e.g., vendor identification number or VID) and product information (e.g., product identification number or PID). In some embodiments, the kiosk 100 may perform electrical analysis using one or more of the methods and / or systems described in detail in the jointly owned patents and patent applications identified herein and incorporated as a whole by reference.

[0044] After visual and electronic analysis of the mobile phone 150, the user 101 is presented with the phone purchase price via the display screen 104. If the user declines the price (e.g., via the touchscreen), a retraction mechanism (not shown) automatically disconnects the connector 142 from the mobile phone 150, the door 106 opens, and the user can reach in and retrieve the mobile phone 150. If the user accepts the price, the door 106 remains closed, and the user may be prompted to place their identification document (e.g., driver's license) inside the ID scanner 112 and provide a thumbprint via the fingerprint reader 114. As a fraud prevention measure, the kiosk 100 can be configured to transmit an image of a driver's license to a remote computer screen, where an operator on the remote computer can visually compare the photograph (and / or other information) on the driver's license with an image of a person standing in front of the kiosk 100, as viewed by one or more of the cameras 116a-c as shown in Figure 1A, to confirm that the person attempting to sell the phone 150 is indeed the person identified by the driver's license. In some embodiments, one or more of the cameras 116a-c may be movable to facilitate the viewing of the kiosk user and other individuals in close proximity to the kiosk 100. In addition, the person's fingerprints may be checked against a record of known fraudsters. If any of these checks indicate that the person selling the phone presents a risk of fraud, the transaction may be declined and the mobile phone 150 may be returned. After the user's identity has been verified, the transparent plate 144 moves back toward the upper and lower chambers 130 and 132. As shown in Figure 1E, when the upper chamber 130 is in the lower position, the gate 138 allows the transparent plate 144 to slide downwards, rather than the electronic device supported on it. As a result, the gate 138 drops the mobile phone 150 from the transparent plate 144 onto the sorting plate 136 and into the sorting container 134. The kiosk can then offer the user payment for the purchase price. In some embodiments, payment can be made in the form of cash distributed from a cash outlet 118.In other embodiments, the user may receive rewards to the mobile phone 150 in various other useful ways. For example, the user may be paid via redeemable cash vouchers, coupons, electronic certificates, prepaid cards, wired or wireless deposits to electronic accounts (e.g., credit accounts, accounts receivable, points accounts, online trading accounts, mobile wallets, etc.), Bitcoin, etc.

[0045] As those skilled in the art will understand, the aforementioned routines are merely some embodiments of the methods by which the kiosk 100 may be used to recycle or otherwise process consumer electronic devices such as mobile phones. Although the embodiments described above are in relation to mobile phones, it should be understood that the kiosk 100 and its various embodiments may also be used in a similar manner to recycle virtually any consumer electronic device, such as MP3 players, tablet computers, PDAs, and other portable devices, as well as other relatively non-portable electronic devices such as desktop computers, printers, and devices for implementing games, entertainment, or other digital media on CDs, DVDs, Blu-rays, etc. Furthermore, although the embodiments described above are in relation to consumer use, the kiosk 100 in its various embodiments may similarly be used by others, such as store clerks, to assist consumers in the recycle, sell, exchange, etc., of their electronic devices.

[0046] Figure 8 illustrates exemplary architectures of a system 800 for examining consumer devices and providing a suggested price, according to several embodiments of the present technology. The system 800 includes a capture module 801 that captures information about the consumer device. The capture module 801 can be implemented on a kiosk, as described in relation to Figures 1A-1E. The capture module 801 can capture device information 811, such as the device identifier (ID) of the consumer device, the time and / or location where the consumer device is examined. The capture module 801 can also capture images 813 of various surfaces of the device, revealing various features of the consumer device, such as cosmetic defects, which may indicate the condition of the device. For example, images may be captured to show the side of the device, the location or presence of buttons on the device, or the light emitted from the screen to indicate the integrity of the LCD panel. In some embodiments, images may be captured while the device is moving to capture the nature and extent of damage. Images may also show the depth of scratches and / or cracks to facilitate estimation of the impact on the underlying electronics. In some embodiments, the entire system 800 can be implemented on the kiosk 100.

[0047] The input information captured by the capture module 801 is transmitted to a price prediction model 803, which is configured to determine a candidate price for the input consumer device. The price prediction model 805 can extract features (e.g., scratches, fine cracks, water damage) from the input information and determine a candidate price based on the number of cosmetic defects on the device. Alternatively, and / or in addition, the capture module 801 can extract features from the input information, transmit the extracted features to the price prediction model 803, and determine a candidate price based on the number of cosmetic defects on the device.

[0048] System 800 also includes a pricing policy model 805 that accepts input from both the capture module 801 and the pricing prediction model 805. The pricing policy model 805 can leverage various submodels to generate the final proposed price. The submodels may include, at a minimum, a submodel for predicting resale value, a submodel for predicting the quantity of consumer devices arriving, a submodel for predicting the processing costs associated with the devices, and / or other submodels to facilitate the prediction process. Additional features that may influence the final proposed price include the kiosk location, the time the device was examined, the age of the device, predicted repair costs, the quantity of devices in similar conditions, the risk of counterfeiting or fraud, the expected demand for the device, predicted resale channels, and other electrical information read from the device. These submodels may be centrally located in conjunction with the pricing policy model. The submodels may also be distributed across different locations in the network as part of a cloud-based computing service. Each model and / or submodel can be implemented using neural networks such as CNNs and / or ConvNets. Compared to human operators, neural networks can generate more consistent analysis results across different geographical locations and are further scalable when a large number of consumer devices need to be evaluated.

[0049] Depending on whether the customer accepts or rejects the final proposed price, relevant data for the consumer device can be fed back into the price prediction model for further training and improvement. As described above, the capture module 801 can be deployed within the kiosk, while the other components of the system are located in a distributed manner within a remote server. In some embodiments, the entire system can be deployed within the kiosk, as described in detail with reference to Figures 1A-1E.

[0050] In some embodiments, instead of finding a kiosk to perform evaluations of used consumer devices (as discussed in relation to Figures 1A-1E), the customer may download and install a software implementation of the capture module 801 onto another device (e.g., another mobile phone, tablet, wearable device, etc.). The software implementation of the capture module 801 may provide the customer with a user interface to define device information 811 (e.g., device ID, brand, model, etc.) and to capture an image 813 of the target consumer device. Figure 10 illustrates an embodiment of evaluating an electronic device 1005 using another mobile device 1003 according to some embodiments of the technology. The customer 1001 may download a software application to their current mobile device 1003 (also referred to as the capture device). The software application is configured to control one or more of the light sources (e.g., flashlight) and / or cameras of the mobile device 1003 and to capture at least one image of the target electronic device 1005. Customer 1001 may also be prompted to provide additional information about the target device 1005, such as the device manufacturer, model, purchase date, general condition, and device characteristics, via the user interface.

[0051] Referring back to Figure 8, input data (e.g., captured images and / or additional device information provided by the customer) can be transmitted via the network to a remote server that hosts the price prediction model 803 and the price determination policy model 805, and determines the status of the target device and / or the final proposed price. Once the final proposed price is determined, the capture module 801 can display the final proposed price of the target device on the user interface of the captured device, and the customer can decide whether to accept or reject the proposed price. Depending on whether the customer accepts or rejects the final proposed price, relevant data for the consumer device can be fed back to the price prediction model 803 for further training of the model. If the customer accepts the proposed price, the capture module 801 can provide further instructions for packaging the device and mailing it to the corresponding recycling and processing center.

[0052] To ensure the quality of the captured images, in some embodiments, the capture module 801 can control the light source of the capture device and generate various lighting conditions. The capture model 801 can further provide a set of predetermined settings or templates to guide the customer to capture images of the target consumer device. Each setting or template can specify at least a desired angle for holding the capture device relative to the used consumer device, a desired exposure level, a desired light intensity, a desired white balance level, brightness, contrast, and / or other parameters. The predetermined templates help the user capture uniform input data, enabling the system to produce consistent analysis results.

[0053] In some cases, network bandwidth limitations can cause delays when large amounts of input data (e.g., large sets of images) need to be transmitted to a remote server. To address such problems, some of the computational logic (e.g., preprocessing of captured images) can be deployed locally on the capture device. For example, a neural network that performs feature extraction and extracts cosmetic defects (e.g., scratches, cracks, water stains, etc.) can be deployed on the capture device as part of the capture module. Once features are extracted, only the extracted features and information about the device (e.g., device ID, model, release date) are transmitted over the network to the prediction and policy models, thereby reducing the bandwidth requirements for transmitting relevant data.

[0054] In some embodiments, image preprocessing also includes operations such as filtering, scrubbing, normalization, or equivalents to generate preliminary features as input to be fed into a neural network. As discussed above, preprocessing captured images can, in some embodiments, mitigate network bandwidth limitations for transmitting data. Image preprocessing can also be particularly useful for capture modules deployed on customer-owned devices, as customers generally do not have precise control over the device's camera and position, unlike kiosks. For example, preprocessing can employ object detection algorithms to remove images that cannot contain any consumer device. Image preprocessing can also generate a uniform input, which is suitable for visual analysis by neural networks to produce consistent results. For example, based on image segmentation techniques, an image of an electronic device can be cropped to show one side of the electronic device (e.g., front, rear, top, bottom, or equivalent). Cropped images showing different sides of the same device can be combined into a single image.

[0055] Figure 2 is a flowchart illustrating a method 200 for evaluating the outward appearance of an electronic device according to several embodiments of the present technology. Referring to Figure 2, the method includes the step of feeding one or more images of the electronic device to a preprocessing module 210. In some embodiments, images can be acquired by various cameras and / or other imaging components of a kiosk 100, as described with reference to Figures 1A-1E, or by a customer-owned capture device. As described above, the images can be preprocessed to generate preliminary features. In some embodiments, preprocessing can be performed by the processing components of the kiosk 100 or by the capture device. In other embodiments, the images can be transmitted to a remote system or device (e.g., a cloud-based computing service), and at least some or all of the preprocessing operations can be performed remotely. Exemplarily, an image of an electronic device can be cropped to show one side of the electronic device (e.g., front, rear, top, bottom, or equivalent). Alternatively, or in addition, images can be taken under natural and / or controlled lighting. Furthermore, images can be taken while the device is powered on or off. Images of the same device, including cropped images showing different aspects, images taken under different lighting conditions, images taken while the device is on or off, and / or images taken under other controlled / uncontrolled conditions, can be combined into a single image.

[0056] The preprocessing may further include a step of resizing the image (either the original image, a combined image, or an otherwise processed image) to a predetermined size. The image is resized to provide a uniform input to the appearance evaluation neural network. The predetermined size for the neural network input can generally be determined in a manner that does not affect the ability to detect appearance defects. For example, the predetermined size must be large enough so that any damage or defect shown in the original image still appears in the resized image. Exemplarily, each image may be resized to 299 × 299 pixels. In some embodiments, if the image is a color image, the technique can separate the red, green, and blue color spaces and transform the image into a three-dimensional integer matrix.

[0057] In some embodiments, if the image is a color image, the technique can separate the image into various color spaces (e.g., red, green, and blue color spaces) and convert the image into a multidimensional (e.g., 3D) integer matrix. For example, as used in standard RGB encoding, each value in the matrix is ​​an integer ranging from 0 to 255. In some embodiments, the matrix can be scaled by dividing by 255 to generate decimal values ​​between 0 and 1 for each matrix input.

[0058] Figures 4A and 4B illustrate examples of pre-processed images 400a-h for input to a neural network according to several embodiments of the present technology. Figure 4A illustrates a combined image showing the front of the smartphone 402 and the back of the smartphone 400d under three different scenarios: lighting 400a with the screen on and a first white balance setting; lighting 400b with the screen on and a second white balance setting; and lighting 400c with the screen off. The images do not show any obvious scratches or fine cracks, and therefore the smartphone 402 can be considered to be in "good appearance" condition. Figure 4B illustrates a combined image showing the front of the smartphone 404 and the back of the smartphone 400h under three different scenarios: lighting 400e with the screen on and a first white balance setting; lighting 400f with the screen on and a second white balance setting; and lighting 400g with the screen off. The combined images show scratches on the screen of smartphone 404, and therefore smartphone 404 can be considered to be in a "defective appearance" condition.

[0059] Referring back to Figure 2, Method 200 includes the step of feeding preliminary features 212 (e.g., original images, pre-processed images, or 3D matrices, depending on whether and how preprocessing is performed) into the neural network 220. The neural network may include a price prediction model and a price decision policy model, as shown in Figure 1F. Method 200 further includes the step of obtaining an output 222 from the neural network 220.

[0060] In some embodiments, the output of the neural network may include integers 0 or 1. 0 may represent "good appearance" (e.g., no cracks, no significant scratches, or equivalent), and 1 may represent "poor appearance" (e.g., cracked, significant scratches, or equivalent). In these embodiments, scaling the input to a range of 0 to 1 may help train the network more consistently as the input and output become more closely matched. In some embodiments, instead of binary values, the output of the neural network may be a range of scores indicating the severity of damage on the consumer device. The output of the neural network may also include at least one of the following: appearance rating or category, type of defect detected, orientation of the defect detected, location of the defect detected, size of the defect detected, associated confidence level, or other appearance rating indicators. In some embodiments, the output of the neural network may further include the brand, model, and / or type of the electronic device shown in the input image. The experimental results demonstrated that the neural network's accuracy in determining physical defects can reach 91%, exceeding the average human ability (approximately 89.9% accuracy).

[0061] As discussed above, the neural network 220 can be implemented as part of a processing component or user device of a kiosk 100, as described above with reference to Figures 1A-E. In other embodiments, at least a portion of the neural network 220 can be implemented on a remote system or device (e.g., a cloud-based computing service). In these cases, a complete set of input data (e.g., images from the electronic device 202), preliminary features 212, and / or some intermediate data (e.g., inputs / outputs between neural network layers) can be transmitted to the remote system or device for processing.

[0062] Figure 3 illustrates an exemplary neural network 300 that can be implemented according to several embodiments of the present technology. The exemplary neural network 300 may be a CNN or a modified CNN. The exemplary neural network 300 may include two main types of network layers, namely convolutional layers and pooling layers. Convolutional layers can be used to extract various features from the input to the convolutional layer. In particular, different kernel sizes may be applied within the convolutional layer for feature extraction, taking into account the fact that scratches and / or fine cracks have varying sizes. Pooling layers may be used to compress the features input to the pooling layer, thereby reducing the number of training parameters for the neural network and mitigating the degree of model overfitting. The exemplary neural network 300 may include multiple cascaded convolutional and pooling layers connected to each other in various structural arrangements (e.g., series connections). In some embodiments, the final layer of the network may include a layer of densely connected nodes, a dropout layer to mitigate overfitting, and / or one or more sigmoid activations to derive the final classification. In some embodiments, a sigmoid activation may be used for binary prediction (e.g., outputting values ​​0 and 1 indicating whether the device state is acceptable). In some embodiments, other types of activations (e.g., softmax activation) may be used so that the neural network can output predictions for different categories (e.g., "scam - do not buy", "fake", etc.).

[0063] Figure 11 illustrates an exemplary architecture 1100 for training a neural network according to several embodiments of the present technology. As shown in Figure 11, the neural network can be trained using pre-collected images 1101 labeled by inspectors 1103 (e.g., human inspectors, electronic labeling systems, etc.). In some embodiments, each image in the training set is associated with an appearance evaluation indication (e.g., "good appearance" or "bad appearance") agreed upon by at least a threshold number of inspectors (e.g., two human inspectors). Thus, the training set includes representative images of an electronic device at a specific appearance status agreed upon by a threshold number of inspectors, and the appearance status can be reasonably determined by visual inspection without requiring the presence of the device phone on site.

[0064] The training set, once expanded, may include images preprocessed in the same manner as the images contributing to the input of the machine learning system 1105 (e.g., a neural network). The training set may include subsets of images of equal or substantially equal size (e.g., within a 5%, 10%, or 15% size difference) associated with each distinctly different visual evaluation indication. For example, with respect to approximately 700,000 images used for training, approximately 350,000 may be associated with “good visual” indications and the other 350,000 with “bad visual” indications. Dividing the training set in this way can prevent or mitigate a “random guess” effect in a trained neural network, where the output may be biased to favor what is reflected by the majority of the training set. In some embodiments, at least some of the images in the training set may be mirrored, rotated, or otherwise repositioned to generate additional images to be included in the training set.

[0065] The trained neural network 1105 can be validated using other pre-collected images labeled by a human inspector 1103. Similar to the training set, the validation set may include a subset of images associated with each distinctly different visual evaluation indication. In contrast to the training set, the relative size of the subsets is more consistent or reflects differently the real-world statistics of pre-evaluated electronic devices. For illustrative purposes, approximately 300,000 images are used to validate the trained neural network.

[0066] In some embodiments, the machine learning system 1105 (e.g., a neural network) is deployed after successful validation (e.g., the false positive and / or false negative rate of the network's outputs across the validation set does not exceed a predetermined threshold). Additional data, such as some of the captured images 1107 and / or associated outputs validated by a human inspector, can be collected for further training of the neural network. In some embodiments, at each stage of further training, layers closer to the input layer (e.g., within a threshold number) can be frozen, while the parameters of layers closer to the output can be adjusted. Doing so may help to reserve specific basic aspects already learned by the network (e.g., representing very slight cracks to different orientations), while allowing the network to adjust parameters targeting more generalized higher-level features, which can efficiently adapt to newer models of devices, different lighting, and / or other modified scenarios. For example, specific basic features learned when training on cracks in relation to an iPhone® 8 may still be applicable to detecting cracks on a Galaxy 9, even if the phone differs in size, shape, color, etc. In some embodiments, as shown in Figure 11, a portion of the captured images can be directed to a human inspector 1103 to perform a manual evaluation and / or generate further training data for a machine learning system 1105.

[0067] Figure 5 is a flowchart illustrating a method 500 for training a neural network to evaluate the appearance state of an electronic device, according to several embodiments of the present technology. In various embodiments, the method 500 can be carried out by a remote system or device associated with a kiosk 100, as described with reference to Figures 1A-1E. Referring to Figure 5, block 510 includes the step of generating a training set, which includes subsets of images of equal or similar size, associated with each distinctly different appearance evaluation indication.

[0068] The training set may include pre-collected images (e.g., those acquired by kiosk 100) labeled by human inspectors. In some embodiments, each image in the training set is associated with an appearance evaluation indication (e.g., "good appearance" or "bad appearance") agreed upon by at least two human inspectors. Once unpacked, the images in the training set may be pre-processed in the same manner as images contributing to the input of the neural network. The training set may include subsets of images of equal or substantially equal size (e.g., within a 5%, 10%, or 15% size difference) associated with each distinctly different appearance evaluation indication. In some embodiments, at least some of the images in the training set may be mirrored, rotated, or otherwise repositioned to generate additional images to be included in the training set.

[0069] In addition, the training set may include information about the devices (e.g., brand, model, release date) so that the model can be trained to identify damage specific to a particular set of devices.

[0070] In block 520, method 500 includes the step of training at least a portion of a neural network based on a training set. Exemplarily, the training set provides “ground truth” samples of network inputs and associated outputs (e.g., sample images of electronic devices and associated appearance evaluation indications), and the components of the neural network can be trained in various ways as deemed appropriate by those skilled in the art. The parameters of the neural network can be learned through a sufficiently large number of training samples in the training set.

[0071] In block 530, method 500 includes the step of generating a proof set, which includes a subset of images associated with each distinctly different appearance rating indication, which are generally consistent in relative size as reflected in real-world statistics. Similar to the training set, the proof set may include a subset of images associated with each distinctly different appearance rating indication. In contrast to the training set, the relative size of the subsets may be more consistent or otherwise reflect the real-world statistics of pre-evaluated electronic devices.

[0072] In block 540, method 500 includes the step of validating the trained neural network and, if successful, deploying the neural network. As described above, in some embodiments, each category of output is represented equally (or substantially equally) during training, and the ratios between output categories are more in line with field statistics during validation. Such an arrangement may be a basis for determining that the trained network does not, on the whole, classify all inputs in a particular direction (e.g., specific appearance evaluation indications), but is still capable of effectively extracting appearance states that are not so well represented in the dataset.

[0073] The neural network can be deployed (for example, to run on kiosk 100 or as part of a capture module on a customer's device) after successful validation (e.g., the false detection and / or failure rate of the network's outputs across the validation set does not exceed a predetermined threshold). In some embodiments, method 500 includes a step of collecting additional data (e.g., inputs to the deployed network and associated outputs validated by a human inspector) for further training of the neural network. This can be achieved by looping back to block 510 of the method. In some embodiments, at each stage of further training, layers closer to the input layer (e.g., within a threshold number) can be frozen, while the parameters of layers closer to the output can be tuned. Doing so may help to reserve specific fundamental aspects already learned by the network (e.g., representing very slight cracks into different orientations), while allowing the network to tune parameters targeting more generalized higher-level features, which can efficiently adapt to newer models of the device, different lighting, and / or other modified scenarios.

[0074] Figure 6 is a block diagram illustrating an embodiment of the architecture for a computer system 600 that may be used to implement various parts of this technology. In Figure 6, the computer system 600 includes one or more processors 605 and memory 610, connected via interconnect 625. Interconnect 625 can represent one or more separate physical buses, point-to-point connections, or both, connected by appropriate bridges, adapters, or controllers. Interconnect 625 can therefore include, for example, a system bus, a Peripheral Component Interconnection (PCI) bus, a HyperTransport or Industry Standard Architecture (ISA) bus, a Small Computer System Interface (SCSI) bus, a Universal Serial Bus (USB), an IIC (I2C) bus, or the Institute of Electrical and Electronics Engineers (IEEE) standard 674 bus, sometimes referred to as "Firewire".

[0075] The processor 605 may include, for example, a central processing unit (CPU) for controlling the overall operation of the host computer. In one embodiment, the processor 605 does this by executing software or firmware stored in memory 610. The processor 605 may be or include one or more programmable general-purpose or dedicated microprocessors, digital signal processors (DSPs), programmable controllers, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or equivalents, or a combination of such devices.

[0076] Memory 610 may be or may contain the main memory of a computer system. Memory 610 represents any preferred form of random access memory (RAM), read-only memory (ROM), flash memory, or equivalent, or a combination of such devices. When in use, memory 610 may, among other things, contain a set of machine instructions that, when executed by processor 605, cause processor 605 to perform operations to implement embodiments of the present technology. In some embodiments, memory 610 may contain an operating system (OS) 630 that manages computer hardware and software resources and provides common services for computer programs.

[0077] Also connected to the processor 605 through interconnection 625 is an (optional) network adapter 615. The network adapter 615 provides the computer system 600 with the ability to communicate with remote devices such as storage clients and / or other storage servers, and may be, for example, an Ethernet® adapter or a Fibre Channel adapter.

[0078] The techniques described herein can be implemented, for example, by programmable circuits (e.g., one or more microprocessors) programmed using software and / or firmware, or by entirely dedicated wiring circuits, or a combination of such forms. Dedicated wiring circuits may take the form of, for example, one or more application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), etc. Systems implemented using the disclosed techniques can be deployed either centrally (e.g., kiosks) or in a distributed configuration (e.g., client devices and remote servers) depending on network resources, bandwidth costs, desired performance, etc.

[0079] Software or firmware used to implement the techniques introduced herein may be stored on a machine-readable storage medium and executed by one or more general-purpose or dedicated programmable microprocessors. “Machine-readable storage medium” as used herein includes any mechanism capable of storing information in a machine-accessible form (the machine may be, for example, a computer, a network device, a mobile phone, a personal digital assistant (PDA), a manufacturing tool, or any device with one or more processors). For example, machine-accessible storage mediums include recordable / non-recordable media (e.g., read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.). The term “logic” as used herein may include, for example, programmable circuits, dedicated wiring circuits, or combinations thereof, programmed using specific software and / or firmware.

[0080] Figure 7 is a flowchart representation of a method 700 for evaluating the state of an electronic device according to several embodiments of the present technology. Method 700 includes, in operation 710, capturing at least one image of a first side of the electronic device by at least one camera of a kiosk, the kiosk including multiple light sources. Method 700 includes, in operation 720, extracting a set of features of the electronic device based on at least one image of the electronic device by a neural network. Method 700 also includes, in operation 830, determining the state of the electronic device based on the set of features.

[0081] In some embodiments, the method includes capturing at least one image of a second side of an electronic device, different from a first side, via at least one camera, based on at least one illumination condition generated by a plurality of light sources. Using different settings of light sources and / or cameras to generate different illumination conditions can facilitate imaging of scratches and / or fine cracks. To image the second side of the electronic device, the method may include, prior to capturing at least one image of the second side of the electronic device, the step of inverting the electronic device so that a light beam is directed toward the second side of the electronic device. In some embodiments, the images are preprocessed as described in reference to Figures 4A-B. The method includes processing a plurality of images of a plurality of sides of an electronic device so that the plurality of images have a uniform size, and combining the plurality of images into a single image to be provided to a neural network.

[0082] The angle of the light beam and the arrangement of the light sources can affect the final captured image, as discussed in relation to Figures 9A-B. In some embodiments, the method includes the step of adjusting one of several light sources such that the angle between the light beam from the light source and a first side of the electronic device is equal to or less than 60 degrees.

[0083] In some embodiments, the method includes the steps of determining a model of an electronic device based in part on at least one image, and identifying cosmetic defects on the electronic device that are specific to the model. In some embodiments, the method includes the step of determining the price of the electronic device based in part on an initial estimated price via a second neural network. The final proposed price can further be determined based on at least (1) the predicted resale value of the electronic device, (2) the predicted arrival quantity of the model of the electronic device, or (3) the predicted processing cost of the electronic device. In some embodiments, the method includes the steps of receiving input from a user indicating acceptance or rejection of the final price, and training a neural network based in part on at least one image and input from the user.

[0084] Some embodiments of the disclosed technique are further described below. [Examples]

[0085] (Example 1) A system for evaluating the state of an electronic device, comprising a kiosk including an inspection plate configured to hold the electronic device, one or more light sources arranged above the inspection plate and configured to direct one or more light beams toward the electronic device, and one or more cameras configured to capture at least one image of a first side of the electronic device based on at least one illumination condition generated by the one or more light sources. The system also includes one or more processors communicating with the one or more cameras, configured to extract a set of features of the electronic device based on at least one image of the electronic device, and to determine the state of the electronic device based on the set of features via a first neural network.

[0086] (Example 2) The system according to Example 1, wherein one or more light sources comprise a first subset of light sources and a second subset of light sources, and the light beams of the first subset of light sources and the light beams of the second subset of light sources are arranged to be orthogonal to each other.

[0087] (Example 3) The system according to Example 1 or 2, further comprising an upper chamber positioned above the inspection plate, wherein one or more light sources are arranged within the upper chamber; a lower chamber positioned below the inspection plate; and a second set of light sources positioned within the lower chamber, configured to direct a light beam toward an electronic device through the inspection plate.

[0088] (Example 4) The system according to one or more of Examples 1-3, wherein the kiosk further includes a reversing mechanism configured to invert the electronic device so that one or more cameras can capture at least another image of a second side of the electronic device.

[0089] (Example 5) The system according to one or more of Examples 1-4, wherein at least one of the one or more light sources is configured to produce a collimated light beam.

[0090] (Example 6) The system according to one or more of Examples 1-5, wherein the angle between a light beam from one or more light sources and a first side of an electronic device is equal to or less than 60 degrees.

[0091] (Example 7) The system according to one or more of Examples 1-6, wherein one or more cameras are configured to capture multiple images corresponding to multiple sides of an electronic device under different lighting conditions, and one or more processors are configured to process the multiple images and combine them into a single input image.

[0092] (Example 8) The system according to one or more of Examples 1-7, wherein the first neural network is configured to output an indicator indicating the state of an electronic device.

[0093] (Example 9) The system according to one or more of Examples 1-8, wherein one or more processors are further configured to determine the estimated price of an electronic device based on its state.

[0094] In some embodiments, the kiosk is configured to provide information about an electronic device, and one or more processors are configured to invoke a second neural network to determine the final price of the electronic device based on the estimated price and the information about the electronic device.

[0095] (Example 10) The system described in one or more of Examples 1-9, wherein the state is a physical state or an external state.

[0096] (Example 11) A system for evaluating the state of an electronic device, comprising a capture device comprising at least one light source and at least one camera, wherein the at least one camera is configured to capture multiple images of the electronic device based on one or more predetermined settings, each of which defines at least one of (1) the angle at which the capture device is positioned relative to the electronic device, (2) the light intensity of at least one light source, (3) the exposure setting of at least one camera, or (4) the white balance setting of at least one camera. The system also includes one or more processors communicating with the capture device, configured to process multiple images, generate a single input image, extract a set of features of the electronic device based on at least one image of the electronic device, and determine the state of the electronic device via a first neural network.

[0097] In some embodiments, the capture device is configured to provide information about the electronic device, and one or more processors are further configured to invoke a second neural network to determine the price of the electronic device based on the state and information about the electronic device.

[0098] (Example 12) The system according to Example 11, wherein the state is a physical state or an external state.

[0099] (Example 13) A computer implementation method for evaluating the state of an electronic device, comprising the steps of: capturing an image of at least one first side of the electronic device by at least one camera of a kiosk, wherein the kiosk includes a plurality of light sources; extracting a set of features of the electronic device based on the image of the electronic device using a neural network; and determining the state of the electronic device based on the set of features.

[0100] (Example 14) The method according to Example 13, further comprising the step of capturing an image of at least one second side of an electronic device, different from a first side, via at least one camera, based on at least one lighting condition generated by a plurality of light sources.

[0101] (Example 15) The method according to Example 14, further comprising the step of inverting an electronic device so that a light beam is directed toward the second side of the electronic device, prior to capturing an image of at least one side of the electronic device.

[0102] (Example 16) The method of one or more of Examples 13-14, comprising the steps of processing multiple images of multiple sides of an electronic device such that the multiple images have a uniform size, and combining the multiple images into a single image to be provided to a neural network.

[0103] (Example 17) The method according to one or more of Examples 13-16, further comprising the step of adjusting one of a plurality of light sources such that the angle between a light beam from a light source and a first side of an electronic device is equal to or less than 60 degrees.

[0104] (Example 18) The method according to one or more of Examples 13-17, comprising the steps of determining a model of an electronic device based in part on at least one image, and identifying a cosmetic defect on the electronic device that is specific to the model.

[0105] In some embodiments, the method includes a step of determining a proposed price for an electronic device, partially based on its state, via a second neural network, the proposed price further determined based on at least (1) the predicted resale value of the electronic device, (2) the predicted arrival quantity of the electronic device model, or (3) the predicted processing cost of the electronic device.

[0106] (Example 19) The method of one or more of Examples 13-18, comprising the steps of receiving input from a user indicating acceptance or rejection of a proposed price, and training a neural network based in part on at least one image and input from the user.

[0107] (Example 20) The method according to one or more of Examples 13-19, wherein the state is a physical state or an external state.

[0108] Some embodiments of this disclosure also have other aspects, elements, features, and / or steps in addition to, or instead of, those described above. These potential additions and substitutions are described throughout the remainder of this specification. Any reference in this specification to “various embodiments,” “a certain embodiment,” or “several embodiments” means that any particular feature, structure, or characteristic described in relation to an embodiment is included in at least one embodiment of this disclosure. These embodiments, even alternative embodiments (for example, also referred to as “other embodiments”), are not mutually exclusive with the other embodiments. Furthermore, various features that may be exhibited by some embodiments but not by other embodiments are described. Similarly, various requirements that may be required for some embodiments but not by other embodiments are described. As used herein, the phrase “and / or,” such as “A and / or B,” refers to A only, B only, and both A and B.

[0109] In other instances, well-known structures, materials, operations, and / or systems, often associated with smartphones and other handheld devices, consumer electronic devices, computer hardware, software, and network systems, etc., are not shown or described in detail in the following disclosure to avoid unnecessarily obscuring the description of various embodiments of the Art. However, those skilled in the art will recognize that the Art can be practiced without one or more of the details described herein, or using other structures, methods, components, etc. The terms used below are used in conjunction with the detailed description of certain embodiments of the Art, but should be interpreted in their broadest and most reasonable form. Indeed, any term that may even be emphasized below, but is intended to be interpreted in any restricted form, will be specifically defined in sections such as the Modes for Carrying Out the Invention.

[0110] The accompanying figures illustrate embodiments of the present invention and are not intended to limit the scope of the present invention. The sizes of the various elements depicted are not necessarily drawn to a constant scale, and these elements may be arbitrarily enlarged to improve readability. Component details may be abstracted in the figures to exclude details such as the location of components and any precise connections between such components when such details are unnecessary for a complete understanding of how the present invention is made and used.

[0111] In the diagram, the same reference number can identify the same or at least generally similar elements. To facilitate discussion of any particular element, the most significant digit of any reference number can refer to the diagram in which that element is first introduced.

Claims

1. A system for evaluating the state of an electronic device, wherein the system is It's a kiosk, One or more cameras configured to capture at least one image of the electronic device, One or more processors that communicate with the one or more cameras, the one or more processors The method involves applying a first machine learning model to the at least one image of the electronic device, wherein the first machine learning model is trained to output the brand and / or model of the electronic device, and an appearance rating of the electronic device specific to the brand and / or model, based on the analysis of the at least one image. Based on the aforementioned visual assessment, the state of the electronic device is determined, The proposed price of the electronic device is determined based on the determined state via a second machine learning model, wherein the second machine learning model is different from the first machine learning model. One or more processors configured to perform the following: Kiosks equipped with A system equipped with these features.

2. The system according to claim 1, wherein the one or more processors are configured to determine that at least one image captured by the one or more cameras is defective.

3. The system according to claim 1, wherein the second machine learning model is configured to use a submodel for predicting the resale value of the electronic device in order to provide the proposed price.

4. The aforementioned kiosk is An inversion mechanism configured to invert the electronic device, allowing one or more cameras to capture at least one image of the electronic device. The system according to claim 1, including the following:

5. The system according to claim 1, wherein the second machine learning model is configured to use a submodel for predicting the number of electronic devices arriving.

6. The system according to claim 1, wherein the second machine learning model is configured to use a submodel for predicting the processing cost associated with the electronic device.

7. The system according to claim 1, wherein one or more cameras are configured to capture multiple images corresponding to multiple sides of the electronic device under different lighting conditions, and one or more processors are configured to process and combine the multiple images into a single input image.

8. The proposed price is based on the location of the kiosk, the time the electronic device was examined, and / or the age of the electronic device, according to claim 1.

9. The system according to claim 1, wherein the proposed price is based on the estimated repair cost of the electronic device, the risk of counterfeiting or fraud, and / or the expected demand for the electronic device.

10. The system according to claim 1, wherein the second machine learning model is configured to use submodels distributed across different locations in the network according to a cloud-based computing service.

11. A computer implementation method for evaluating the state of an electronic device, wherein the method is The kiosk captures an image of at least one of the first sides of the electronic device using at least one camera, The method involves applying a first machine learning model to the at least one image of the electronic device, wherein the first machine learning model is trained to output the brand and / or model of the electronic device, and an appearance rating of the electronic device specific to the brand and / or model, based on the analysis of the at least one image. Based on the aforementioned visual assessment, the state of the electronic device is determined, The proposed price of the electronic device is determined based on the determined state via a second machine learning model, wherein the second machine learning model is different from the first machine learning model. Methods that include...

12. The method according to claim 11, comprising capturing an image of at least one second side of the electronic device, which is different from the first side, via the at least one camera.

13. The method according to claim 12, comprising inverting the electronic device so that the light beams from the kiosk's multiple light sources are directed toward the second side of the electronic device, prior to capturing the at least one image of the second side of the electronic device.

14. Processing the multiple images such that multiple images of multiple sides of the electronic device have a uniform size, The process involves combining the aforementioned multiple images to create a single image to be provided to the first machine learning model. The method according to claim 11, including the method described in claim 11.

15. To determine whether the at least one image captured by the at least one camera is acceptable or defective. The method of claim 11, further comprising applying the first machine learning model to the at least one image only when it is determined that the at least one image captured by the at least one camera is acceptable.

16. The method according to claim 11, further comprising using a submodel for predicting the resale value of the electronic device in order to provide the proposed price, by the second machine learning model.

17. Receiving input from the user indicating acceptance or rejection of the aforementioned proposed price, Training the second machine learning model based in part on the at least one image and the input from the user. The method according to claim 11, including the method described in claim 11.

18. The method according to claim 11, further comprising using a submodel for predicting the number of electronic devices to arrive in order to provide the proposed price, using the second machine learning model.

19. The method according to claim 11, further comprising using a submodel for predicting the processing costs associated with the electronic device in order to provide the proposed price, using the second machine learning model.

20. The method according to claim 11, wherein the proposed price is based on the estimated repair cost of the electronic device, the risk of counterfeiting or fraud, and / or the expected demand for the electronic device.

Images