Device screen damage detection

Abstract

To provide methods for identifying a condition of one or more screens of an electronic device.SOLUTION: An image including a device screen is received (operation 410). A selected portion of the received image is identified (operation 420). The selected portion of the image is adjusted (operation 430). The adjusted image is divided into parts (operation 440). A determination of whether one or more part(s) of the image shows damage is made (operation 450). A determination of whether the device screen is damaged is made based on the determination of whether a part shows damage and / or whether an adjacent part shows damage (operation 460).SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Non-Provisional Patent Application No. 15 / 452,707, filed March 7, 2017, entitled "SCREEN DAMAGE DETECTION FOR DEVICES," and U.S. Provisional Patent Application No. 62 / 304,729, filed March 7, 2016, entitled "SCREEN DAMAGE DETECTION FOR MOBILE DEVICES," all of which are incorporated herein by reference.

[0002] Technical Field The present invention relates to determining the state of one or more device screens. [Background technology]

[0003] background Devices such as smartphones, watches, and tablets are often sold back to manufacturers or third parties when consumers upgrade their devices. These used devices may have value in the resale market based on the condition of the device. For example, a user may provide a used phone to a reseller, who may evaluate the phone's condition and offer a price based on the evaluation. However, the evaluation of the device's condition is performed by a human and is therefore often time-consuming and subjective. Furthermore, the user must wait for the evaluation (e.g., at a kiosk, store, etc.) to receive the offered price, which may reduce the user's willingness to resell the used device and / or reduce the user's satisfaction with the process. Summary of the Invention [Problem to be solved by the invention]

[0004] overview In various embodiments, the condition of one or more screens on a device is determined (e.g., whether or not there is screen damage). A user can request an assessment of the condition of the screen and / or device via a return application on the device. The return application can determine the condition of the device (e.g., for determining value, resale, insurance claim, warranty claim). The return application can display a first graphic on the device's screen and prompt the user to position the device in front of a reflective surface, such as a mirror (e.g., to allow the device's reflected image in the mirror to be captured by the device itself). The return application can guide the user to position the device in a predetermined position (e.g., closer to the mirror) to increase the probability of capturing an image that can be used to accurately assess the condition of the screen. An image of the device's screen can be obtained (e.g., an image can be automatically captured and / or taken by the user). For example, a photo of the screen's reflection in a mirror can be captured. The image of the screen can be processed and / or analyzed, and a determination of whether the screen is damaged can be made based on the analyzed image. One or more notifications and / or flags can be generated, sent, and / or displayed based on the analysis of the screen damage.

[0005] In some implementations, the second device can be utilized to facilitate identification of the state of the device's screen. For example, the first device may have a broken and / or damaged camera and / or the screen of the first device may be so damaged that it is no longer possible to use the second device. It may not be possible to interact with the return application on one device (e.g., a crack in the screen could injure the user's fingers), so the return application on the second device can be utilized to identify the state of the screen on the first device. [Means for solving the problem]

[0006] In various embodiments, the state of one or more screens of a device (e.g., an electronic device such as a mobile device, laptop, etc.) can be identified. A request for an evaluation of the state of a screen or portion thereof of a first device can be received via a return application on the first device. A first graphic including a first identification code can be presented on a screen (e.g., a display component) of the first device. The return application can cause the first graphic to be displayed on the screen of the first device. At least a portion of a first image of the first graphic can be captured by a camera of the first device. The first image can include a reflection of the first graphic on a reflective surface, such as a mirror. One or more second graphics can be presented on the screen of the first device (e.g., via the return application), and at least a portion of one or more second images of at least one of the second graphics can be captured (e.g., via the camera of the first device). One or more of the second images can include a reflection of at least one of the second graphics on a reflective surface, such as a mirror. The return application can be capable of controlling and / or allowing a user to control a camera component of the first device. The return application can access images captured by the camera of the first device. One or more of the second images can be processed to determine a screen condition of the first device. Processing the second image(s) can include dividing the second image into portions, determining whether one or more of the portions of the second image include damage, and identifying portions adjacent to one or more of the portions including damage. The screen condition of the first device can be determined based on whether one or more of the one or more portions of the second image are determined to include damage and whether one or more of the portions adjacent to one of the portions determined to include damage also include damage.

[0007] Implementations may include one or more of the following features: The identity of the first device may be verified based on analysis of the first identification code. A second captured image including the second graphic may be embedded or otherwise tagged with the first identification code, or a portion thereof, from the first graphic. Capturing at least a portion of a second image(s) of the second graphic(s) may include determining an orientation of the device based on the captured image of the first graphic and providing guidance for adjusting the orientation of the device based on the determined orientation. At least a portion of additional images of the first graphic may be captured via a camera of the first device. In some implementations, the orientation of the device may be determined based on the captured image of the first graphic, and guidance for adjusting the orientation of the device may be provided based on the determined orientation. At least a portion of additional images of the second graphic may be captured via a camera of the first device, each of the additional images including a reflection of at least one of the second graphics on a reflective surface. If it is determined that the first image(s) being captured and / or the second image(s) being captured are not processable images, the first device may be allowed to change orientation to capture a processable image. To capture a processable image, in some implementations, the orientation of the device may be determined based on the captured first image or one or more captured second images, and guidance may be provided for adjusting the orientation of the device based on the determined orientation. The captured second image(s) may tag at least a portion of the captured first image. In some implementations, for example, one or more processes of the first device may be restricted (e.g., pop-ups, alerts, banners, etc.) while the image is being captured and / or the return application is running. Identifying the screen of the first device or a portion thereof in the second image may include identifying the screen of the first device in the second image using corner detection and / or edge detection.Processing one of the second images may include identifying a screen or portion thereof of the first device in the second image and generating a third image, where portions of the second image not identified as a screen or portion thereof in the second image are constrained not to be included in the third image. The third image may be divided into portions (e.g., non-overlapping with the second image and / or in addition to the second image), and it may be determined whether one or more of the portions of the third image include damage. Portions adjacent to one or more of the portions that include or do not include damage may be identified. Determining the state of the screen of the first device may be based on whether one or more of the one or more portions of the third image are determined to include damage and whether one or more of the portions adjacent to one of the portions determined to include damage include damage. Generating the third image may include modifying the second image such that portions of the second image not identified as a screen or portion thereof are removed. Identifying the screen or portion thereof may include identifying an active area of ​​the screen of the first device.

[0008] In various embodiments, the state of a screen(s) of a device (e.g., an electronic device such as a mobile device) can be identified. For example, the state of a first device can be identified using a second device. At least one of the first device or the second device can include a camera (e.g., an external image capture component). The first device and the second device may or may not be the same device. A request for an assessment of the state of a screen, or a portion thereof, of the first device can be received via a return application on the second device. The first device may include the return application. A first graphic including a first identification code can be enabled to be presented on the screen of the first device via the return application on the first device. At least a portion of the first graphic presented on the first device may be captured via a camera of the second device. One or more second graphics can be enabled to be presented on the screen of the first device, and at least a portion of the second graphic(s) presented on the first device may be captured via a camera of the second device. One or more of the second images can be processed (e.g., preprocessed and / or processed) to determine the condition of the screen of the first device. Processing the second image can include dividing the second image into portions and determining whether one or more of the portions of the second image include damage. In some implementations, a neural network can perform operations of the return application, such as processing the image. Portions adjacent to one or more of the portions including damage can be identified. The adjacent portions may or may not include damage. The condition of the screen of the first device can be determined based on whether one or more of the portions of the second image are determined to include damage and whether one or more of the adjacent portions include damage.

[0009] In various embodiments, if the state of the first device is determined to be damaged, damage information can be determined. A flag can be generated to identify one or more portions of the second image determined to contain damage based on the damage information. If the state of the first device is determined to be damaged, the touchscreen of the first device can be tested (e.g., according to known touchscreen tests). The brightness of the screen of the first device can be captured (e.g., to facilitate image processing and / or accuracy). The calibration may be based on the first image. Enabling presentation of a second graphic on the screen of the first device may include enabling presentation of a set of burst images on the first device. The set of burst images includes at least one of the second graphics at a plurality of brightness levels. Capturing at least a portion of the one or more second graphics presented on the first device may include capturing a set of burst images presented on the first device and selecting one of the captured burst images by determining which color of the captured burst image set is most similar to a reference color. The selected captured burst image may be identified as one of the captured second graphics (e.g., for preprocessing and / or processing). Capturing at least a portion of one or more second images of at least one of the second graphics may include determining an orientation of the device based on the captured image of the first graphic and providing guidance for adjusting the orientation of the device based on the determined orientation. At least a portion of an additional image of the first graphic may be captured via a camera of the first device. Enabling the presentation of one or more second graphics on the screen of the first device can include enabling two or more second graphics to be presented sequentially on the screen of the first device. Capturing at least a portion of one or more of the second graphics can include capturing at least one image of each of the second graphics presented sequentially on the screen of the first device. Enabling the presentation of one or more second graphics on the screen of the first device can include enabling two or more second graphics to be presented simultaneously on the screen of the first device.

[0010] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the embodiments will become apparent from the following description and drawings. [Brief explanation of the drawings]

[0011] For a more complete understanding of the present disclosure and its features, reference is now made to the following description taken in conjunction with the accompanying drawings. [Figure 1] FIG. 1 illustrates an exemplary system implementation. [Figure 2] FIG. 1 illustrates an implementation of an exemplary process for determining the state of a device's screen. [Figure 3A] 10A-10C show an exemplary positioning implementation of the device in front of a mirror. [Figure 3B] FIG. 1 illustrates an implementation of an exemplary capture of an image including a device screen. [Figure 3C] FIG. 10 illustrates an implementation of an exemplary notification displayed by a return application. [Figure 4] 1 illustrates an implementation of an exemplary process for determining whether a device screen is damaged. [Figure 5A] FIG. 1 illustrates an exemplary image implementation. [Figure 5B] FIG. 1 illustrates an exemplary image implementation. [Figure 6A] FIG. 1 illustrates an implementation of an exemplary image before processing. [Figure 6B] FIG. 10 illustrates an implementation of an exemplary image after processing. [Figure 7] FIG. 1 illustrates an embodiment of a portion of an image segmentation. [Figure 8A] FIG. 1 illustrates an implementation of an exemplary captured image of a device screen. [Figure 8B] 8B illustrates an embodiment of an interface generated as a result of determining whether the device screen shown in FIG. 8A is damaged. [Figure 8C] FIG. 1 illustrates an implementation of an exemplary captured image of a device screen. [Figure 8D] 8D illustrates an embodiment of an interface generated as a result of determining whether the device screen shown in FIG. 8C is damaged. [Figure 8E]FIG. 1 illustrates an implementation of an exemplary captured image of a device screen. [Figure 8F] 8F illustrates an embodiment of an interface generated as a result of determining whether the device screen shown in FIG. 8E is damaged. [Figure 9A] FIG. 1 illustrates an implementation of an exemplary learning tool. [Figure 9B] FIG. 10 illustrates an exemplary second image implementation from which the learning tool is derived. [Figure 9C] 9C illustrates an implementation of exemplary processing of the exemplary second image shown in FIG. 9B. [Figure 10A] FIG. 1 illustrates an implementation of an exemplary learning tool. [Figure 10B] FIG. 10 illustrates an exemplary second image implementation from which the learning tool is derived. [Figure 10C] 10C illustrates an implementation of exemplary processing of the exemplary second image shown in FIG. 10B. [Figure 11A] FIG. 1 illustrates an implementation of an exemplary learning tool. [Figure 11B] FIG. 10 illustrates an exemplary second image implementation from which the learning tool is derived. [Figure 11C] 9C illustrates an implementation of exemplary processing of the exemplary second image shown in FIG. 9B. [Figure 12A] FIG. 1 illustrates an embodiment of exemplary accuracy results for an exemplary neural network. [Figure 12B] FIG. 1 illustrates an implementation of exemplary cross-entropy results for an exemplary neural network. DETAILED DESCRIPTION OF THE INVENTION

[0012] Like reference numbers in the various drawings indicate like elements. Detailed Description In various embodiments, a device (e.g., an electronic device such as a smartphone, watch, tablet, e-reader, laptop, portable game console, etc.) may be evaluated for the state of one or more screens of the device. The device may include a component capable of capturing external images, such as a camera (as opposed to a component capable of saving an image of a graphical user interface displayed on the screen, such as a screenshot).

[0013] The condition of the screen can affect aesthetics and / or usability (e.g., because a crack may require repair before use and / or may or may not affect use and / or viewing on the device), and therefore may change the price of the device when it is offered for sale. Human evaluation can cause variability because the evaluation can be subjective. Human evaluation can create a time lag between when a user offers a device for sale and when a price is quoted for that device. These factors can reduce the willingness to resell the device and reduce user satisfaction with the process, thereby driving good devices off the market, increasing resale prices (e.g., because supply is more limited), and / or reducing device recycling (e.g., because devices may be stored or discarded). In some embodiments, automatic determination of screen condition can reduce fraud. For example, screen condition can be verified for insurance purposes (e.g., policy issuance and / or claims) and / or reuse. Automatically determining the screen status as described can reduce the occurrence of fraud, which can reduce insurance costs (e.g., because the status can be verified and / or objectively determined), and / or increase user satisfaction (e.g., because the need to take the device to a store to verify the status can be eliminated and / or because the objective status of the device can be obtained for possible reuse). ) Therefore, there is a need for automatic detection of the state of a device or its components, such as whether the device has screen damage or not.

[0014] As shown in FIG. 1 , one or more devices 110 may be coupled to a server 120 (e.g., via a network such as the Internet or other communications network). Server 120 may be any suitable server. In some implementations, server 120 may include a neural network. The neural network may be implemented on the server via software and / or hardware (e.g., some implementations of neural networks are commercially available and / or may be built using convolutional neural networks from IBM®, CogniMem® via the FANN library, and / or on the Google® TensorFlow framework). In some implementations, a convolutional neural network may be utilized because, in some implementations, a convolutional neural network may be more capable of image processing (e.g., faster, more accurate, etc.) than other neural networks.

[0015] In some implementations, the neural network can self-tune and / or adjust based on system updates to improve the accuracy of screen damage detection. For example, learning tools such as captured images and / or marked damage (e.g., captured images in which portions identified as damaged are flagged, e.g., by a change in color and / or pattern) can be provided to the neural network to encourage and / or enable the neural network to further learn to identify the state of the device. To develop the neural network (e.g., so that it can identify the state of the device or portion thereof), the neural network can analyze the learning tools, associated captured images, process the associated captured images, identify damage through processing, and / or identify differences and / or similarities between the portions with identified damage and the learning tools. In some implementations, the neural network can organize its own learning of perceived damage to pixels and / or other portions of the device (e.g., based on the learning tools and / or processing of the captured images provided to the neural network).

[0016] Device 110 may include one or more device screens 115 (e.g., a monitor, an LCD screen, glass, Gorilla Glass, etc.). Device 110 may include a camera (e.g., a front-facing camera or a camera on the same side of the device as the device screen) capable of capturing an image reflection of the device screen. Device 110 may include a return application stored in the device's memory and executable by the device's processor. The return application may enable device 110 to communicate with server 120. The return application may receive input from a user, prompt the user to provide input and / or position the device, send images and / or notifications, generate graphics, capture images, instruct device components such as the camera to capture images, restrict device components (e.g., flash), restrict device operation (e.g., pop-ups, banners, alerts, reminders, etc. during operation of the return application and / or image capture), and / or communicate with other devices such as server 120. In some implementations, the return application allows the sale of the device via the application, determines the condition of the device or its components, enables the value of the device to be determined based on the condition of the device, determines the condition of the device or its components for reuse of the device, determines the condition of the device or its components for insurance claims (e.g., allowing a user to submit a device insurance policy claim), and The return application may determine the status of the device or component if desired), determine the status of the device for warranty claims, etc.

[0017] The server and device 110 can perform one or more of the described operations independently and / or in cooperation with other devices. In some implementations, a server may not be utilized, and a return application (e.g., rather than a server) may perform one or more of the described operations. In some implementations, to increase the speed of operations, a server may perform one or more operations of the described processes. In some implementations, one or more of device 110 and / or the server may perform one or more of the operations in cooperation with each other. In some implementations, the server may be cloud-based, and the device(s) may communicate with the server to perform operations such as image processing and / or analysis (e.g., condition identification).

[0018] In some implementations, when a user decides to sell a device (e.g., when upgrading a device, switching devices, etc.), the user can select a return application on the device. The return application can determine information about the device (e.g., device condition, device characteristics such as model, market resale value, etc.) and display a price for the device. The condition of the screen can adjust the price offered for the device via the application because screen damage can be aesthetically displeasing to some users, may indicate damage to other components, and / or may be expensive to replace and / or repair. Because the screen condition can be automatically determined by the application and / or server, evaluations can be made more quickly and / or provide greater consistency (e.g., because a human is not determining the condition). Additionally, because the screen condition can be automatically determined, the possibility of a user intentionally inaccurately reporting the screen condition can be reduced (e.g., because devices with good screen condition have a higher market price, a user may be tempted to report the screen condition as good even when it has a cracked or broken screen; automatic detection can prevent fraud common to self-reporting and / or human-based analysis systems). Additionally, the screen condition (e.g., large cracks, deep cracks, etc.) may prompt the system to determine whether other components of the device may also be damaged. Once the device screen and / or device condition is determined, an offer price for the device can be determined (e.g., by the server and / or return application). In some implementations, a base cost for the device can be determined (e.g., based at least in part on resale prices, recycling prices, market information, supply of similar devices, and / or demand for the device), and the base price can be adjusted based on the condition of the device (e.g., reduced for damage to the screen, reduced for damage to other components, and / or increased for new in the box). The user can then make a decision as to whether to sell the device based on the price received.The entity submitting the price can do so with knowledge of the screen state without significant time delay between the user initiating the process for resale and receiving the price, thus, in some implementations, increasing the satisfaction of the process for users and / or entities purchasing or validating claims or insurability of used devices.

[0019] In some implementations, in addition to and / or instead of resale purposes, the condition of the device may be determined (e.g., by the return application and / or server) for insurance claims (e.g., device insurance). For example, the device may be damaged and / or may be suspected of being damaged. The condition of the device or its components may be determined and / or reported to submit and / or verify insurance claims. In some implementations, the return application can be utilized to determine the condition of the device or portions thereof when purchasing device insurance. Thus, the condition of the device or its components can be determined via the return application on the device rather than relying on self-reporting and / or bringing the device to a physical store.

[0020] In some implementations, the condition of the device can be determined (e.g., by the return application and / or server) for device warranty purposes. For example, a manufacturer, reseller, repairer, and / or refurbisher may warrant a device. Warranties can be complex, and a user may have difficulty understanding what parts are covered and / or what types of damage are covered. The condition of the device and / or its components and / or a determination of whether a damaged component is covered by warranty can be determined using the return application. In some implementations, the condition determined via the return application can be used to submit and / or verify a warranty claim.

[0021] In some implementations, the condition of the device can be determined to determine whether the device can be reused (e.g., by another user). For example, a second user may be able to obtain the status of the first user's device via a return application on the device (e.g., the return application can send a notification with the device's status to the second user). The second user can then obtain an assessment of the device's condition that is less susceptible to fraud, subjectivity, or error (e.g., than a human assessment). The second user can use the damaged and / or undamaged device (e.g., with or without repairs to damage identified by the system). The return application can be used to identify what repairs can be performed and / or whether the device can be used without repairs.

[0022] FIG. 2 shows an implementation of an example process 200 for determining the status of a device's screen. Image(s) of the device's device screen may be received via a return application on the device (operation 210). For example, a user may open the return application on the device. In some implementations, the return application may initiate communication with a server (e.g., to obtain the device's most recent base price, obtain software updates, etc.). The user may request a price quote for the device, determine whether the device is reusable, submit an insurance claim, verify the device status for insurance claims and / or policies, and / or other suitable reasons for determining the device's status. The return application may prompt the user (e.g., via visual, audio, and / or tactile notifications) to capture an image of the device screen. For example, the return application may prompt the user to position the device so that the device screen is facing a mirror. Because the quality of the image may affect the ability to determine the status of the device screen (e.g., whether the device screen is damaged), the return application may prompt the user to adjust the position of the device. For example, the return application may prompt the user (e.g., via visual and / or audio notifications) with instructions to adjust the position of the device, such as moving the device closer or farther away, tapping the image to refocus the image, adjusting the angle at which the device is held, etc. When the device is in a predetermined position, one or more images may be captured. After the return application determines that the device is in a predetermined position (e.g., an optimal position for capturing an image), the image may be captured automatically. The return application may detect the device in an image and automatically focus the camera on the device. Additionally, the return application may also crop the device in an image. The captured image may be a reflection of the device in a mirror and may include the device screen. Rather than using the device's "screen capture" feature to obtain an image of the outside of the screen, rather than an image of the interface presented on the device screen, a reflection of the device in a mirror may be captured.

[0023] In some implementations, uploading images to the server and / or the return application that are not captured via the return application may be prohibited (operation 220). When screen image status is determined based on images, the potential for fraud may exist. For example, a user may take an image of a similar, undamaged device to increase the price offered for a damaged device. To reduce costs associated with fraudulent or incomplete / misrepresentation of device screen(s), uploading or selecting images that are not captured via the return application may not be accepted. For example, the return application may process images captured by the application (e.g., the application may access a camera application on the device, process images captured by the camera application, and tag the captured image with device information, such as identification information, date, and / or timestamp). The return application may not accept images from the device or cloud storage photo library (e.g., a user may not be allowed to take an image of the device screen and select an image from the device's photo library).

[0024] It may be determined whether the device screen is damaged based on the received image(s) (operation 230). The return application may send the received image(s) to a server for analysis. The server may process and / or analyze the image(s) to determine whether the device screen is damaged. For example, the server may include a neural network that is trained to identify damaged screens and / or the probability that a screen or portion thereof is damaged. In some implementations, the neural network may be trained to identify damaged screens by processing a set of images including screens with known cracks, breaks, and / or other damage and / or undamaged images. The neural network may learn to identify patterns associated with screen damage from the set of images. The server may determine whether the device screen is damaged based on analysis of the received image(s) by a neural network trained to recognize screen damage. A neural network (e.g., residing on a server) can have a first or outer layer that can be trained to identify (e.g., preprocess) typical screen images that are not associated with damage, such as, for example, reflection(s), logo(s), shadow(s), and / or other artifact(s) that may be found in images of a device. Training the outer layer neural network to identify typical screen images that are not associated with damage can reduce the occurrence of erroneous assessments that some damage (e.g., cracks, chips, scrapes) is present when no damage is actually present on the device and / or increase accurate assessments of damage. Additionally, training the outer layer of the neural network to identify typical screen images that are not associated with damage can allow for recapture of the screen image (e.g., to obtain more accurate processing of the damage image).

[0025] Process 200 may be implemented by various systems, such as system 100. Additionally, various operations may be added, removed, and / or modified. For example, the return application may perform one or more operations in determining whether the device screen is damaged. The return application may, for example, perform at least a portion of the analysis of the image. In some implementations, two or more images may be captured and processed to determine whether the device screen is damaged. In some implementations, if it is determined that the screen of the device is damaged, the base cost of the device may be adjusted. For example, if the device screen is damaged, the base cost may be reduced by a screen replacement cost, a screen repair cost, a processing time cost, and / or a labor cost.

[0026] In some implementations, a return application (e.g., on the server and / or device) can preprocess the images. For example, the return application can identify captured low-quality images (e.g., low quality) (e.g., one or more of the images captured via the return application). The return application (e.g., via an outer layer of the neural network and / or via the return application on the user device) can preprocess the images to identify poor images (e.g., via an external classifier on the neural network that is configured and trained to detect conditions that may "hide" cracks and / or defects). For example, the preprocessing can identify poor details (e.g., via an external classifier), such as fingers and / or other obstructions on the screen, objects in the image that are not the phone, the entire screen is not in the image, and / or reflections that may be hiding cracks. In some implementations, the outer layer on the neural network can identify (via training) other defects that cause poor images. In some implementations, the preprocessing can filter images based at least in part on other factors that can be calculated by analyzing the images, such as blurriness and / or poor color. Blur associated with poor image quality can be calculated based on the rate of change of color on edges to determine whether the image is in focus. Poor color in an image that may be associated with poor image quality can be detected by examining the intensity of the color.

[0027] In some implementations, the state of the first device may be determined using a second device (e.g., a device different from the first device). For example, the first screen of the first device may be damaged, making it impossible or undesirable for the user to use the device (e.g., fingers may be traumatized by a crack in the screen, tips may come loose from the screen, the screen may be further damaged by use, etc.). Thus, the second device may be utilized to capture an image of the first device. The first device and the second device may include a return application (e.g., one or more operations of the return application may be executed by processors of the first device and the second device). A request for evaluation of the state of the screen of the first device, or a portion thereof, via the return application on the second device may be received. The return applications on the first device and the second device may communicate (e.g., directly and / or indirectly via a return application on a server). For example, a return application on the second device can communicate with a return application on the first device to enable a graphic to be presented on the screen of the first device via the return application. The return application can present an image (e.g., including a first graphic and / or a second graphic) to the first device and enable the presented image to be captured by the second device. The captured image can be preprocessed and / or processed to determine the condition of the screen of the first device. Thus, even if the first device is in limited use (e.g., due to damage), an assessment of the condition of the screen of the first device can be obtained.

[0028] In some embodiments, the device can automatically adjust the position of the device. The device can be balanced on a substantially flat surface, so that the device can be placed on a surface in front of a mirror. The device can automatically reposition (e.g., rotate) the device. ) may automatically trigger one or more vibrations to adjust the position. In some implementations, if the automatic adjustment is unable to position the device in a predetermined position for image capture, the return application may prompt the user via a notification (e.g., audible, haptic, and / or visual) to adjust the position.

[0029] The notification(s), in some implementations, can send instructions to the user on how to reposition the device (e.g., move closer, farther away, etc.). In some implementations, the return application can generate positioning aids for display on the device. The positioning aids can indicate (e.g., via visual, auditory, and / or tactile signals) whether the device is in a predetermined location, how close the device is to the predetermined location, and / or in which direction the device is in and / or out of position. For example, the positioning aids may include an electronically generated bubble level (e.g., an accelerometer in the device; and / or a GPS can facilitate determining the orientation of the device, calculate where the device is detected in an image, and / or provide real-time feedback of changes in the position(s) and / or angle(s) at which the device is being held). In some implementations, the instructions can include instructions on how to change the environment (e.g., a room) in which the device is positioned. For example, the instructions may include instructions to increase and / or decrease lighting, instructions to close window(s) (e.g., to reduce glare), and / or other suitable instructions.

[0030] In some implementations, the return application can use cues (e.g., audio, tactile, and / or visual) to facilitate image capture. For example, the graphical user interface of the return application can create the impression of a "tunnel" within the graphical user interface (e.g., via a 3D square formation). For example, the graphical user interface can generate the appearance of a tunnel with an identification code (e.g., a QR code) at the end of the tunnel with size and / or shape matchers to the required aligned squares. This can guide the user to align and position the device at the correct angle and distance from the mirror. The return application can include other visual and / or audio cues. For example, the graphical user interface can include pointing arrows (e.g., 2D and / or 3D) (e.g., via an overlay, pop-up, embedded image, etc.) to instruct the user to reorient the device (e.g., tilt the phone sideways and / or upside down, move it forward or backward).

[0031] In some implementations, upon successful capture of the image(s), a notification may be generated by the return application for display on the device. For example, the captured image(s) may be pre-processed. In some implementations, if it is determined during pre-processing that the image(s) cannot be processed (e.g., the device image is cut off and / or does not show the entire screen, preventing the server from determining the screen state), the user may receive one or more notifications and / or the user may be prompted to restart the process or part thereof. The neural network of the return application may perform one or more of the pre-processing operations. For example, the neural network (e.g., the outer layer of a multi-layer neural network) may be capable (e.g., through training) of acting as a filter that rejects images during pre-processing that have problems, such as, but not limited to, a finger blocking the screen and / or window light reflections. The neural network may be capable of providing a notification to the user that includes at least part of the reason the captured image is of poor quality. The reason for this rejection may facilitate (e.g., for the user) the correction process (e.g., determining the reason for the rejection of the captured image). (This reduces the guesswork for the user when reselling.) In some implementations, the return application may prompt the user to begin the reselling process and / or may begin the reselling process automatically when the return application is opened.

[0032] In various implementations, to determine the screen state, the return application can capture image(s) of the device screen (e.g., automatically and / or manually via selection from the user). The application, in some implementations, can generate one or more graphics (e.g., pictures, patterns, monochrome displays, and / or graphical user interfaces) for display on the device screen. The graphics generated by the return application can include one or more colors (e.g., black, white, green, purple, etc.), one or more patterns, photographs, pictures, identifiers (e.g., QR codes, barcodes, etc.), other suitable graphics, and / or combinations thereof. Generating the graphics can include retrieving graphics from memory (e.g., of the device and / or coupled to the device) and / or generating identifiers (e.g., based on device information such as IMEI information, user information, time information such as date and time, and / or verification of such information in real time within a specified threshold to exclude reuse of a previously taken good device image or an image of another device). Some graphics can facilitate detection of the screen state. For example, the graphics generated by the application may include a monochromatic green, black, and / or white graphic that covers at least a portion of the display screen (eg, an active portion or portion of the display screen).

[0033] In some implementations, the application can generate a first graphic and one or more second graphics including the identifier. The first graphic and / or the second graphic(s) are analyzed to determine the device screen state. For example, the first graphic can include an identifier such as a QR code. The identifier may be generated by the return application. For example, device information (e.g., IMEI information, device age, device model, memory capacity, etc.) and / or user information can be encoded into the identifier. FIG. 3A shows an example of positioning a device in front of a mirror, and the return application generates an identifier for display on the device screen. When the return application prompts the user to position the device so that the device screen is reflected in the mirror, the return application can generate an identifier for display on the device screen. The return application can then analyze the reflection of the identifier displayed in the mirror via a camera (e.g., the device's front-facing camera) to determine whether the device's position should be adjusted. For example, if the identifier is blurred in the reflection of the identifier in the mirror, the user can be notified and prompted to adjust the device's position. In some implementations, once the device is in the appropriate position, the return application may or may not capture an image of the device screen's reflection in the mirror that includes the identifier code. In some implementations, the return application and / or server may verify that the captured QR code is associated with the device (e.g., may decode the QR code by comparing it with known device information, etc.).

[0034] The return application can then generate one or more second graphics (e.g., a green screen, a white screen, and / or other graphics) to enable capture of a reflection of the second image(s) in the mirror. Figure 3B shows an example of positioning the device in front of a mirror, where the return application on the device generates the second graphics. In some implementations, the return application may generate a notification once capture of the image(s) is complete, as shown in FIG. 3C. The captured second image(s) may be analyzed to determine the state of the screen. For example, the captured second image(s) may be sent to a server for analysis by a neural network on the server.

[0035] In some implementations, the return application can automatically generate the second graphic once the identifier code is focused, captured, and / or processed (e.g., verified). In some implementations, the second graphic can be utilized to more easily identify the state of the device screen once the identifier code is captured and / or authenticated. For example, detection of the screen state can be more easily determined using a graphic that is easily identifiable by a neural network. In some implementations, the second graphic can be generated quickly just prior to capturing an image of the screen device (e.g., by photographing the reflection of the device screen) and / or within a short period of time after the return application determines that the device is in the appropriate position (e.g., the identifier is in focus on the captured image of the first graphic) to prevent fraud. In some implementations, the second graphic can be generated sequentially to enable sequential capture of related second images. In some implementations, image capture can be automated to coordinate graphic generation and image capture.

[0036] In some implementations, a first image may be captured and / or processed such that a second image is tagged (e.g., attached and / or encoded) with the first image, a portion thereof, and / or a decoded identifier. In some implementations, the second image may be captured before and / or after the first image. By tagging the second image with the first image or a portion thereof (e.g., information obtained by decoding the identifier), fraudulent activity may be reduced. For example, a user may be prohibited from uploading an image of a different device screen (e.g., a device screen not of that device) because the uploaded image may not include the encoded portion. In some implementations, the return application and / or server may be able to identify an untagged second image to identify fraudulent images of the device screen(s).

[0037] In some implementations, the distance at which an image is captured by the device may also be managed. For example, the focal length of the camera may be set by an application, and the position of the device may be adjusted until an identifier graphic on the device screen is in focus in the image captured by the camera. In some implementations, the size of the identifier (e.g., a QR code) in the image may determine the appropriate distance at which a user should position the device from the mirror. In some implementations, corner angle detection may facilitate determining whether the device is positioned in a predetermined position for image capture with respect to the angle at which the device is positioned proximate to the mirror. For example, the predetermined position for image capture may include positioning the device parallel to the surface of the mirror; therefore, corner angle detection may identify corner angles in the image and determine the angle at each corner to determine whether the device was parallel to the surface of the mirror during image capture.

[0038] In some implementations, the return application may adjust one or more components of the device to facilitate capture of the device screen image. For example, the return application may turn off the flash (e.g., to avoid glare). As another example, the return application may adjust the display of the graphics generated by the return application to display additional graphical user interfaces. Device notifications (e.g., banners, alerts, etc.) can be blocked (e.g., temporarily) to allow them to be generated without the interface and / or overlay.

[0039] After the image(s) are captured by the return application, one or more of the images may be analyzed to determine the state of the device screen. For example, the analyzed image(s) may include second graphic(s). FIG. 4 shows an implementation of an example process 400 for determining the state of a device screen (e.g., determining whether the screen is damaged). An image including the device screen may be received (operation 410). For example, a server may receive the image via the return application. The image may be automatically uploaded to the server and / or may be selected for upload by a user via the return application. The image may include one or more graphics generated by the return application and displayed on the device screen.

[0040] The image may be processed by the server. A selected portion of the received image may be identified (operation 420). For example, the image being captured by the return application may include the device screen and an area proximate to the device screen. A selected portion of the image may be identified, such as the device screen and / or an active portion of the device screen (e.g., an illuminated portion and / or a portion that responds to touch). In some implementations, the selected portion of the received image may be selected to reduce the size of the image to a portion relevant to analyzing the state of the device screen (e.g., analysis of an area proximate to the device may not indicate whether the screen is damaged). FIG. 5A illustrates an implementation of an exemplary image 500 including a device screen 510. As shown, the image includes the device screen 510 and an area 520 proximate to the device screen. The device screen 510 may be detected using corner detection 535, as shown in image 530 of FIG. 5B. FIG. 6B illustrates an implementation of an exemplary image 600 including a device screen 610. As shown, the image includes the device screen 510 and an area 620 proximate to the device. In some implementations, edge detection of the edge 630 of the device in the image can be used to identify components of the device in the image. As shown, the device screen 610, microphone, speaker, and case can be identified in the image. Identifying one or more components can facilitate determining the condition of the device screen outside the active area of ​​the device screen and / or can facilitate identification of damage to other components (e.g., a crack on the microphone can indicate that the microphone is also damaged).

[0041] In some implementations, the image size may be altered (e.g., cropped or otherwise reduced) so that only selected portions of the image are shown in the modified image. In some implementations, the selected portions of the received image may be labeled or otherwise identified within the image.

[0042] Selected portions of the image may be adjusted (operation 430). For example, the server may adjust the contrast, brightness, color, sharpness, exposure, blur, alignment, image transformation, size, and / or other suitable aspects of the image. In some implementations, noise may be reduced to facilitate identification of screen damage as opposed to markings that should be attributed more to noise. FIG. 6A shows an implementation of an example image 600 including a device screen 610 before the image has been adjusted, and FIG. 6B shows an example of an adjusted image 650. The adjusted image 650 has reduced noise 640 within the device (e.g., lines, shadows, and / or other features that should not be attributed to screen damage).

[0043] The adjusted image may be divided into portions (ACT 440). For example, the adjusted image may be divided into multiple portions. FIG. 7 shows an example implementation of an adjusted image including a device screen 710 and a resulting portion 720. The division may be performed by cropping the image into smaller portions, identifying regions of the image as portions, and / or otherwise appropriately dividing the image. Processing of each selected portion of the image may be processed more quickly by the server (e.g., a neural network on the server) than if the entire image were processed by the server. Thus, dividing the image may increase the speed at which the image or adjusted image is processed. For example, each portion of the adjusted image may be analyzed by a node of the neural network. Thus, because each node is analyzing a separate portion of the image, the analysis may be performed more quickly than if the entire image were analyzed by the server. In some implementations, dividing the image into portions may increase the probability that a screen condition is detected and / or reduce the probability that a screen condition is erroneously identified. For example, because screen damage may span more than one portion of an image, a high-probability identifier of screen damage in one or more adjacent portions may increase the probability of screen damage in the first portion. In some implementations, the size and / or shape of the damage and whether adjacent portions contain a predetermined probability of damage can be analyzed to determine whether the portion, and therefore the device, contains damage. For example, a crack of a selected shape and / or size may be known to extend across multiple adjacent portions, and if the multiple adjacent portions do not contain the predetermined probability of damage, the overall probability of screen damage may be reduced. As another example, a chip of a selected shape and / or size may not extend across multiple adjacent portions of an image, and the absence of a predetermined probability of damage in the adjacent portions may not adjust the overall probability of screen damage.

[0044] A determination may be made whether one or more portion(s) of the image exhibit damage (operation 450). For example, the server may analyze the portions to determine whether screen damage is present (e.g., cracks, dents, chips, pixel damage, etc.). The server's neural network may perform a tailored analysis of the one or more portions of the image based on patterns and / or identifier techniques learned from previous device screen images and / or sets of known screen images (e.g., known to have or not have screen damage). In some implementations, allowing the portions to be analyzed by the server's neural network may allow for easy upgrades and maintenance of the neural network and may improve accuracy (e.g., because previous device screen images from multiple devices may have been analyzed).

[0045] A determination of whether the device screen is damaged may be made based on whether a portion exhibits damage and / or whether adjacent portions exhibit damage (operation 460). If a portion is determined to be damaged (e.g., a binary determination such as yes or no damage, where the probability of damage exceeds a predetermined probability, such as a 50% probability of damage), the server may identify adjacent portion(s). Whether one or more adjacent portions are damaged may be determined and utilized by the server (e.g., a neural network) to determine whether the screen should be identified as damaged. For example, if one portion has a 20% probability of damage and four adjacent portions have a 50% probability of damage, the screen may be determined to be damaged. As another example, if one portion has a 50% probability of damage and none of the adjacent portions have a probability of damage greater than 25%, the screen may be determined (e.g., by the server) to be undamaged. In some implementations, the overall probability of damage to the device screen may not be reduced based on the probability of damage in adjacent portions based on the characteristics (e.g., location, size, and / or shape) of the damage. In some implementations, the neural network As the accuracy of the neural network improves, the predetermined range of probabilities associated with damaged screen(s) may decrease. For example, the system may indicate that it can determine that screen damage exists if the probability of screen damage is greater than 70% within a portion, and as the accuracy of the neural network improves, the system may indicate that it can determine that screen damage exists if the probability of screen damage is greater than 50% within a portion.

[0046] Notification(s) may be sent based on a determination of whether the device screen is damaged (operation 470). For example, if the screen is determined to be damaged or not damaged, the user may receive a notification based on this determination. In some implementations, if the device screen is determined to be damaged, the user may be allowed to dispute the determination by restarting one or more actions of the process (e.g., repositioning the device and / or retaking the image). In some implementations, a notification of whether the device screen is damaged and / or a price based on the condition of the device screen may be sent to the application for presentation to the user via the return application.

[0047] The notification may be sent based on a determination that the neural network is unable to accurately assess the state of the device screen. For example, during image processing, the server may identify that the full screen is not present in the image. This determination may be made by calculating a predicted screen aspect ratio, a predicted angle value of the screen, predicted dimensions of the model, etc. The server may instruct the user via the return application to capture another image. The server may provide instructions to the user via the return application to position the device in real time as best it can.

[0048] Process 400 may be implemented by various systems, such as system 100. Additionally, various operations may be added, deleted, and / or modified. In some embodiments, process 400 may be performed in combination with other processes, such as process 200. For example, the condition of a device may be determined for resale of the device. Because the resale price of a device and / or the demand for the device in the resale market may be based at least in part on the condition of the device, automatically determining the condition of the device and / or its components may facilitate determining the price to offer for the device (e.g., to be resold). In some embodiments, the condition of the device may be determined to facilitate determining whether the device can be reused (e.g., by another user). The condition may be determined, and other users may determine the price to offer for the device, whether the device is reusable, whether the device should be used as is, and / or whether the device should be repaired. In some embodiments, the condition of the device may be determined for use in a device insurance policy. For example, if a user wishes to obtain device insurance, a return application may be utilized to determine the condition of the device and / or its components. The condition of the device and / or the history of the device's condition (e.g., multiple screen cracks being repaired) can be used to determine whether to offer a device insurance policy, the price to set for the device insurance policy, and / or to validate the condition of the device provided by the user. In some implementations, the user may wish to submit an insurance claim, and the condition of the device or components thereof can be determined by the return application to submit with the insurance claim and / or to validate parts of the insurance claim.

[0049] In some embodiments, rather than and / or instead of the first graphic and / or identification code, an IMEI and / or other device and / or operating system specific code is obtained and / or utilized to facilitate device identification. For example, the IMEI and / or other code may be associated with a screen image captured via a second graphic to identify the particular device. The user may be directed to a settings page for the device where the IMEI is displayed (e.g., via a prompt on the graphical user interface of the return application), and / or the user may be prompted to dial a code on an autodialer to display the IMEI. The user may capture a screenshot of the IMEI (e.g., via a camera on the device being used to capture the image and / or on a second device). The return app may process the screenshot (e.g., via OCR) to identify the IMEI. In some embodiments, the user may install a profile from a server configuration similar to an MDM server. The profile may provide the IMEI to the server. The server may pass the IMEI to the return application. In some embodiments, the profile may reduce the number of steps a user must take to provide the IMEI to the return application. In some embodiments, one or more of these functions may be performed automatically by the return application. The obtained IMEI may be utilized by the return app to tag the captured image (e.g., a second graphic) and / or ensure authenticity (e.g., via the obtained IMEI).

[0050] In some implementations, the state of the first device may be determined using a second device. For example, the screen of the first device may be damaged, making it impossible or undesirable for the user to use the device (e.g., fingers may be traumatized by a crack in the screen, a tip may come loose from the screen, the screen may be further damaged by use, etc.). Thus, the second device may be utilized to capture an image of the first device. The first device and the second device may include a return application (e.g., one or more operations of the return application may be executed by processors of the first device and the second device). A request for evaluation of the state of the screen of the first device, or a portion thereof, via the return application on the second device may be received. The return applications on the first device and the second device may communicate (e.g., directly and / or indirectly via a return application on a server). For example, the return application on the second device may communicate with the return application on the first device to enable a graphic to be presented on the screen of the first device via the return application. A first graphic may be presented on a screen of the first device via a return application on the first device, and the graphic may include a first identification code. At least a portion of the first graphic presented on the first device may be captured via a camera of the second device. The return application may have access to the device's camera functionality, and thus the return application may be able to enable image capture. One or more second graphics may be presented on the screen of the first device (e.g., via the return application on the first device), and at least a portion of one or more of the second graphics presented on the first device may be captured via the camera of the second device.One or more of the second images may be pre-processed and / or processed to determine the state of the screen of the first device. The first graphic image may be utilized to decode an identification code of the first graphic (e.g., to verify the identity of the first device). In some implementations, the identification code may include a code unique to the first device, such as an IMEI number, that may be utilized by the return application to verify the identity of the first device and / or images captured from the screen of the first device.

[0051] In some implementations, the first graphic may or may not be presented and / or captured when utilizing a second device to determine the status of the first device. The level of security and / or authentication provided by the return application capturing images on a device to be evaluated by the return application may not be as strong as when a second device is utilized to capture images from the first device being analyzed. Accordingly, in some implementations, insurance claims, appraisals, underwriting, and / or reimbursements may be adjusted to account for the increased risk of fraudulent user behavior.

[0052] In some implementations, processing the second image may include dividing the second image into portions, determining whether one or more of the portions of the second image include damage, identifying portions adjacent to one or more of the portions including damage, and determining a screen status of the first device based on whether one or more of the portions of the second image are determined to include damage and whether one or more of the portions adjacent to one of the portions determined to include damage also include damage. The screen and / or portions thereof (active areas) may or may not be identified from the second image prior to dividing the image into portions. For example, a neural network may, in some implementations, be trained to identify damage in an image even if active areas are not isolated.

[0053] In some embodiments, one or more of the above operations can be performed with one or more additional images. The determination of whether the device screen is damaged may, in some embodiments, be based on an analysis of the image and the additional images. In some embodiments, the return application can process, or at least partially process, the image before sending it to the server. In some embodiments, the image may not be segmented before analysis. In some embodiments, the image may not be processed before analyzing the screen condition. The determination of whether the device screen is damaged, in some embodiments, can be made based on a determination of whether a portion exhibits damage, even if adjacent portions do not exhibit damage. For example, if a portion exhibits a 100% probability of damage and / or deep cracks, the device screen can be determined to be damaged. In some embodiments, the return application captures, receives, and / or stores images (e.g., instead of and / or in addition to the server). In some embodiments, the return application can perform (e.g., instead of or in cooperation with the server) or more operations.

[0054] In some implementations, multiple images can be utilized as the second graphic. For example, the return application can present and / or capture a set of second images. The set of second images can have different coloring. The set of second images can be captured by a burst of captures (e.g., automatically capturing multiple images in a short period of time, as opposed to manually capturing multiple images). The burst of captures can use the same or different capture settings (e.g., flash, exposure, focal length, etc.). The return application (e.g., via a neural network on the device and / or on a server) can compare one or more of the captured images to predetermined reference colors and / or images containing predetermined reference colors to identify captured second images for processing (e.g., to determine screen health). Images with poor image quality, such as coloring and / or blurring, may or may not be identified as captured second images in some implementations.

[0055] In some implementations, images can be captured using one or more other automatic adjustments, for example, capture bursts to obtain better consistency in the captured images. The brightness, color, orientation, focal length, etc. can be varied (e.g., to more accurately determine health, as consistency can facilitate analysis by a neural network). For example, the brightness of a second image can be varied during a capture burst to provide a set of captured images. Images in the set can be compared to a reference color to identify the captured second image for further processing (e.g., to determine the condition of the screen). In some implementations, the return application can prompt the user to reorient the device to obtain a set of captured images (e.g., to allow images to be captured at different angles, so multiple images can be taken to vary the perspective of a crack that may not be visible at one angle).

[0056] In some implementations, having multiple images captured in the same session with different screen settings can increase the visibility of cracks to the neural network (e.g., the images may highlight different types and / or locations of screen cracks). The return application can utilize a second graphic of a different color to facilitate identification of damaged screens. In some implementations, iii. Multiple tilt angles via the UI to guide proper positioning - In some implementations, the set of captured images can be compared to a reference image (e.g., color, intensity, etc.) to select which images to further process to determine the state of the device. For example, the return application can select images that most closely resemble the brightness / color / intensity of the images used for training to improve accuracy results. In some implementations, using the return application's operations, a color is carried over to areas of the image known to be a screen. The color is matched to a reference color known to best display crack visibility while minimizing other defects such as reflections, poor color, etc. Images with colors closest to the reference color are sent to a neural network for processing. Improved image consistency can make the analysis (e.g., performed by the return application's neural network) less dependent on lighting conditions.

[0057] In some implementations, burst capture of images can facilitate the capture of one or more images that are not at the same brightness level (e.g., have the same or different graphics). One or more of the colors in the image can then be matched to a reference color to identify the captured image (e.g., from the set of burst-captured images) that most closely matches the reference color. Utilizing burst capture with different brightness levels can allow for greater consistency and / or reduce variability in the image capture process and / or the captured images sent to the neural network for analysis. This, in some implementations, can reduce error in the analysis.

[0058] In some implementations, the capture of the first graphic can provide an initial exposure setting(s). For example, an image of the first graphic can be acquired and analyzed to identify an initial exposure setting (e.g., if the image is too bright, blurry, etc.). Generating an initial exposure setting can improve image capture.

[0059] In various embodiments, the state of one or more screens of a device (e.g., an electronic device such as a mobile device, laptop, etc.) can be identified. The state of a device can be determined using the operation of the device and / or the operation of other devices. For example, if a first device lacks a camera and / or an operational camera, a second device can be utilized to capture an image of the first device or a portion thereof (e.g., screen, front, etc.). As another example, if a component of the first device renders the first device at least partially inoperable (e.g., a cracked screen, a non-functioning touchscreen, stuck pixels that could injure a user and / or further damage the device upon use, etc.), a second device can be utilized to An image of the first device or a portion thereof (eg, screen, front, back, etc.) may be captured.

[0060] In some implementations, the first device can be utilized to capture an image of the first device by positioning the device so that an image of the device can be captured on a reflective surface, such as a mirror. Using the device's own camera to capture the image (e.g., so that an image of a similar device is not presented instead) can reduce the risk associated with fraud. A request for an assessment of the condition of a component of the first device, or a portion thereof, such as a screen, can be received via a return application on the first device. The return application may be resident on the first device and / or accessible by the first device (e.g., stored remotely). The return application can present one or more graphical user interfaces to facilitate communication with the user and / or present graphics on the device's screen.

[0061] The return application may present one or more graphics on a screen of the first device (e.g., via a graphical user interface generated by the return application) for capture by a camera of the first device. For example, a first graphic may be generated and / or presented (e.g., by the return application) on a screen (e.g., a display component) of the first device. The first graphic may include one or more first identification codes, such as an IMEI associated with the device, a code number generated by the return application and associated with the device, a QR code, etc. The identification code may be analyzed (e.g., decoded, compared to a list of codes, etc.) to verify the identity of the first device. At least a portion of a first image of the first graphic may be captured by the camera of the first device. The first image may include a reflection of the first graphic on a reflective surface, such as a mirror. The first graphic and / or the identification code of the first graphic may be tagged or otherwise embedded in another captured image (e.g., a second image including a second graphic). The first graphic may be utilized by the return application to determine initial settings (e.g., brightness, contrast, orientation, distance to reflective surfaces, etc.) used for presenting and / or capturing subsequent graphics. In some implementations, the first graphic may not be utilized (e.g., generated and / or captured).

[0062] Other images may be generated and / or presented by the return application to facilitate identification of damage to the device or portions thereof (e.g., the screen). One or more second graphics may be generated and / or presented on the screen of the first device (e.g., via the graphical user interface of the return application). The second graphics may include graphics configured to facilitate identification of damage (e.g., cracks, dents, chips, etc.) by the return application (e.g., using a trained neural network). For example, the second graphics may include a solid color, a color variation, pattern(s), an image, etc. In some implementations, the second graphics may include a set of graphics in which the image displayed within the graphic changes and / or the settings used to present the second graphic on the screen change. For example, the brightness of the presented image may be changed to present the second graphic differently for capture. In some implementations, a single second graphic may be generated and / or presented (e.g., a solid green graphic). At least a portion of one or more of the second images of at least one of the second graphics may be captured (e.g., via a camera of the first device). One or more of the second images may be mirror-like. The return application may include a reflection of at least one of the second graphics on a reflective surface. The captured image may include more than the screen (e.g., the front, an area proximate to the device, etc.). The return application may be able to control and / or allow a user to control a camera component of the first device. The return application may access images being captured by the camera of the first device.

[0063] In some implementations, the captured image (e.g., the first image and / or the second image being captured) may be preprocessed. The preprocessing may be performed by a return application on the user device and / or on a server (e.g., using a trained neural network). The preprocessing may identify low-quality images, for example, by identifying portions within the captured image that are not related to damage to the presented image and / or screen (e.g., obstructions, flash reflections, etc.). The preprocessing may identify partial and / or blurry images. In some implementations, a determination by the return application in preprocessing that the captured image is of poor quality may cause the return application to reject the image and / or request recapture of the image. Upon recapturing the image, the return application may regenerate and / or present graphics on the screen of the first device. The return application may modify graphics, device settings, and / or prompt the user for adjustments upon recapture (e.g., limiting flash, adjusting orientation, etc.). In some implementations, the low-quality images may be processed to identify the condition of device components.

[0064] One or more of the second images can be processed to determine the state of a component, such as a screen, of the first device. Processing the second image(s) can include dividing the second image into portions and determining whether one or more of the portions of the second image contain damage. The second image can be divided into portions to allow for faster processing (e.g., when compared to whole image processing) and improved accuracy (e.g., by allowing analysis of adjacent regions in determining the probability of damage). In some embodiments, portions adjacent to one or more of the portions containing damage can be identified as adjacent portions. The adjacent portions may or may not contain damage.

[0065] The state of the screen of the first device can be determined based on whether one or more of the one or more portions of the second image are determined to include damage, and whether one or more of the portions adjacent to one of the portions determined to include damage (e.g., portions adjacent to a particular portion) also include damage. For example, a neural network can be trained to identify common damage patterns and can use information about adjacent portions (e.g., whether adjacent portions are damaged) to determine whether a portion is damaged.

[0066] In some implementations, the determination of whether a component, such as the screen of the first device, is damaged may include additional damage information, such as a rating (e.g., a severity rating, a rating of the type of damage, etc.), a location of the damage, etc. The additional damage information and / or the determination of whether a component of the device is damaged may be presented to the user and / or may be utilized in other operations of the return application (e.g., a reduction in rating for an insurance claim, a warranty claim, etc.).

[0067] The described process can be implemented by various systems, such as system 100. Additionally, various operations may be added, deleted, and / or modified. In some implementations, the process, or portions thereof, may be performed in combination with operations from other processes, such as processes 200 and / or 400. For example, capturing the second images may include burst capture of the second images. When burst capture is permitted, device settings may be changed. The captured images may be compared to a reference image to identify which set of captured images to process. For example, an image having a color closest to the reference color may be selected (e.g., brightness and / or contrast may be adjusted on the device to obtain different colors in the second image when a burst of image capture occurs). The set of captured images may be preprocessed to identify which images may be processed. As another example, processing the captured second images may include generating a third image in which portions of the image not associated with the device's screen are removed (e.g., cropped). This third image may be analyzed (e.g., segmented and / or analyzed) by at least a portion of a neural network to identify whether the screen is damaged. In some embodiments, lower quality images can be processed to identify the condition of device components. For example, blurry images may be processed, and the neural network can take image blur into account in its analysis (e.g., by reducing the sensitivity of damage detection to avoid over-identifying damage).

[0068] In some implementations, one or more of the operations may be performed by the second device to obtain the status of the first device. For example, the second device may include a camera for capturing images to be presented to the first device by the return application. The return application on the second device may enable the capture of the images and / or coordinate the presentation of the images on the first device, the processing (e.g., pre-processing and / or processing) of the captured images, and / or the identification of the status of the first device. In some implementations, increased changes in fraud associated with capturing images using a device different from the device whose status is being determined may be taken into account in insurance underwriting, security measures (e.g., physical inspection upon receipt of the device for trade-in, sale, and / or return), and / or discounts (e.g., lowering the determined value and / or selling price).

[0069] In some embodiments, image capture may be at least partially automated. An image may be acquired when the image meets an initial exposure setting. For example, a user may move the phone, and an image may be automatically captured when the phone is in an optimal position (e.g., when the initial exposure setting is met). The initial exposure setting may include criteria related to camera placement relative to the screen and / or mirror, tilt angle, flash setting, brightness setting, etc. In some embodiments, the phone screen brightness for the initial exposure setting may be calibrated before positioning the phone identification code. In some embodiments, the brightness may be adjusted during a calibration period with different brightness and exposure settings. A brightness for a predetermined visibility of an identification code, such as a QR code, may be selected as a reference brightness for the current lighting conditions. In some embodiments, this reference brightness may be used as the median value of multiple image captures with different brightnesses.

[0070] In some implementations, the captured image may be processed by the neural network at full resolution or at a lower resolution. Images sent to different levels may or may not change. For example, a captured image may be passed through all levels of the neural network at full resolution and remain as a PNG file. In some implementations, when an image is sent to the first level of the neural network, the image may be downscaled (e.g., to 128x256). The downsampled image may be sent to one or more layers of the neural network as an array of color intensities. For example, Byte(red) Byte(green) Byte(blue), Byte(red), Byte(green), Byte(blue) may represent pixel (0,0) and pixel (0, 1) are two pixels. In some implementations, the first layer of the neural network may be preprocessing (e.g., returning a result that the image is of poor quality and cannot be processed and / or that the image is processable). In some implementations, when the captured image is sent to the final layer, the captured image may be sampled by patch and slide (e.g., a patch may be 32, whereby a tile is 32x32, and a slide may be 17, whereby the network takes a tile from 1,1, then the next tile from 1,17, and / or other suitable patch and / or slide). There may or may not be overlapping tiles sent to the inner layers of the neural network. Samples (e.g., 32x32 tiles) may be sent to the final neural network layer as an array of RGB byte values ​​representing color intensity. In some implementations, this can be done lengthwise and / or widthwise. The neural network can start at any suitable point in the captured image. In some implementations, starting near the center may have a speed advantage in processing, since the screen is in the center of the image and contains a background where edges may be ignored.

[0071] In some implementations, the preprocessing can identify whether the captured image is of poor quality. The preprocessing can be performed by a neural network (e.g., the first layer of the neural network) in some implementations. For example, the neural network can identify poor images such as grainy images, blurry images, poor contrast (e.g., dark), poor color (e.g., brightness), mismatched images (no phone when a phone is expected), obstructions such as fingers, phone cases, partial screens, etc.

[0072] In some implementations, the return application (e.g., a neural network of the return application) can identify the condition of the device or portion thereof as damaged or undamaged. The return application can identify the type of damage, the degree of damage, etc. For example, a neural network can be trained to identify the type of damage and / or the degree of damage. The neural network can rate the severity of the damage. For example, the output of the neural network can provide details about the condition, such as cracks, dents, stuck pixels, minor defects, good, etc. In some implementations, the condition and / or the output of the neural network may be provided in an output format, such as a simple integer 0 to x, a binary 000001, and / or a percentage that sums to 1.0.

[0073] In some implementations, the neural network can have a zero level (e.g., prior to processing). In some implementations, processing can be expedited and / or accuracy can be improved by utilizing a neural network with two or more levels (e.g., including a final processing layer). The return application can be customized based on desired parameters. For example, identifying blurry images may be easier and / or faster than determining obstacles, and therefore a return application that preprocesses only to identify blurry images may have fewer layers than a return application that preprocesses for obstacles and / or other poor image quality. In some implementations, the guidance provided by the return application can enable better image capture, and a single-layer (e.g., final layer) neural network can be utilized to identify defects.

[0074] In some implementations, the return application can process poor quality captured images. For example, rather than rejecting images based on color and / or blur, the rating and / or color intensity of the portion can be processed by the return application. The return application may or may not prohibit processing of other portions. For example, the rating may be a value, color, and / or other indicator, such as a rating of 0 indicating that the image is not blurred and a rating of 255 indicating that the image is very blurred. The rating scale may be linear or non-linear. The return application (e.g., a neural network) may adjust (e.g., increase and / or decrease sensitivity) based on the rating. For example, the return application may reduce sensitivity / intensity in identifying cracks in captured images rated 255. Thus, various defect images may be processed based on lower quality images and / or may be processed nearly accurately.

[0075] In some implementations, other portions of the device may be captured and / or processed by the return application as described. For example, the return application may facilitate assessment of cracks on the cell phone casing (e.g., front and / or back).

[0076] In some embodiments, the determination of whether a device screen is damaged may be a determination of the degree of damage, whether the damage is associated with one or more damage categories (e.g., complete, cracked, scratched, reflective), the probability that damage is present, and / or any other suitable damage classification. For example, the determination of whether a device screen is damaged may be a determination of whether the screen is cracked or not. In some embodiments, the determination may result in a result between, for example, −1 and 1, where a value less than 0 is associated with a device screen without a crack and a value greater than 0 is associated with a device screen with a crack. For example, in some embodiments, a value may be associated with the probability that the device screen is damaged, and the device screen may be identified as damaged or cracked if the probability is greater than a predetermined value. For example, if the certainty that the device screen has a crack is greater than 90% (e.g., >.80), the device and / or device screen may be identified as cracked. In some embodiments, the analysis may be performed for each portion, a set of adjacent portions, all device portions, and / or the entire device.

[0077] In some implementations, the location, extent (e.g., how deep, in which layer(s), etc.) of damage to the device screen, and / or whether the damage affects other components can be determined (e.g., by the system's neural network). For example, because damage may be detected in discrete areas, in some implementations, the approximate location of the damage can be determined and / or transmitted to the user. In some implementations, the location of the damage may adjust the base price offered for the device. For example, a small crack in the screen that is not in the active area of ​​the device screen may result in a smaller price reduction than a crack that is in the active area of ​​the device screen. In some implementations, the user may be notified of the location of the screen damage. This information can help the user determine whether the identified screen damage is actually screen damage or a mark on the device that can be removed. In some implementations, the extent of the damage can be determined to facilitate identification (e.g., when repairing the device) of whether the screen is repairable or whether replacement is more appropriate.

[0078] In some implementations, the location of the portion of the device screen where damage is detected may be flagged (e.g., with a different color, pattern, flag, and / or other suitable indicia). FIG. 8A shows an implementation of an exemplary captured image of a device screen. As shown, the device screen includes a crack visible in the captured image. The captured image is divided into multiple portions and analyzed to determine whether one or more of the portions have a probability of damage greater than a predetermined probability. In some implementations, whether the device screen and / or portions thereof are damaged is determined based on whether adjacent portions include damage. The system can use the flag to identify portions that have damage greater than a predetermined probability. FIG. 8B illustrates an embodiment of an interface generated as a result of determining whether the device screen illustrated in FIG. 8A is damaged. The interface may be generated and presented to a user (e.g., via a website coupled to the return application and / or the server). As illustrated, darker (e.g., red) flags indicate portions that have been identified as damaged. The user can view the flags and dispute flags and / or verify damaged portions that the user believes are not damaged portions of the device screen. FIG. 8C illustrates an embodiment of an exemplary captured image of a device screen. As illustrated, the device screen is not damaged. Therefore, no flag is generated when the image of the device screen is analyzed. FIG. 8D illustrates an embodiment of an interface generated as a result of determining whether the device screen illustrated in FIG. 8C is damaged. As illustrated, the automatic determination of the state of the device screen does not detect any damage, and therefore no portion of the device screen is flagged as having damage. As another example, FIG. 8E illustrates an embodiment of an exemplary captured image of a device screen. As illustrated, the device screen includes damage. 8F illustrates an embodiment of an interface generated as a result of determining whether the device screen shown in FIG. 8E is damaged. As a result of determining the state of the device screen, a value app may flag damaged portions of the device screen. Darker flags (e.g., red flags) indicate portions of the device that have been labeled by the system as damaged.

[0079] In various implementations, determining the condition of the device screen (e.g., whether the device screen is damaged or undamaged) can be used to determine the condition of the device and / or to further test the device (e.g., according to commonly used and / or described techniques). For example, if there is a crack in the screen in close proximity to a component of the device and / or if there is a crack with a size (e.g., depth and / or width) greater than a predetermined maximum size, further tests can be performed to determine whether one or more other components (e.g., microphone, speaker, touchscreen layer, and / or case) are damaged. For example, the operation of the components can be tested (e.g., automatically, semi-automatically, and / or manually). The condition of the device (e.g., components including the screen), market data, current resale prices, and / or other information can be utilized to determine the price. The price can be transmitted to the user and / or displayed via the return application. The user, in some implementations, can sell the device based on the price posted in the return application.

[0080] In some implementations, if the device screen is determined to be damaged, a touchscreen test can be performed. The touchscreen test can be performed via a return application. For example, the return application can prompt the user to provide input based on prompts from the return application, and a determination can be made regarding the condition of the touchscreen (e.g., damaged or undamaged, location of damage and / or extent of damage) based on the input provided by the user. The results of the touchscreen test can be utilized to determine the depth of damage to the device screen and / or damage to one or more other components of the device.

[0081] In some implementations, the grade of a device may be based at least in part on a determination of whether the device screen is damaged, the location of the damage on the screen if damage is present, the magnitude of the damage to the screen if damage is present, whether one or more other components of the device are damaged, resale value, recycle / scrap value, and / or other suitable criteria. For example, if the device does not suffer from screen damage, the device may be rated a first grade. If the device has screen damage, the device may be assigned a second grade lower than the first grade. If the device has screen damage and touchscreen damage, the device may be assigned a third grade lower than the second grade. The grading of the device may relate to the price at which a user is offered to sell the device and / or the price at which the device is resold (e.g., in a marketplace, to a third party, etc.).

[0082] In some implementations, the assessment of damage to a device screen can be less subjective (e.g., because damage can be determined automatically) and / or more consistent (e.g., location and / or size), allowing for a more detailed overall assessment of the device and / or grading across more possible levels, thereby providing smaller differences between device conditions more consistently and more quickly. For example, screen damage that does not overlap with the active area of ​​the screen may be graded as a damaged area of ​​the screen, but with a higher grading than a screen with damage in the active area of ​​the screen.

[0083] In some implementations, the image(s) may be stored in the device's memory and / or in memory coupled to the device (e.g., cloud storage and / or server memory). The return application can manage the upload of images to the server based on the device's network connectivity (e.g., LTE, Wi-Fi, etc.).

[0084] In some implementations, screen damage can be verified by a human (e.g., a quality control operator), and this feedback can be provided to the neural network to improve accuracy and / or allow adjustments to the analysis provided by the server's neural network.

[0085] While the screen condition is described in terms of crack damage, other damage, such as damage to pixels (e.g., broken, stuck, etc.), dents, etc., can also be determined via the return application. For example, the application can cause one or more graphics to be displayed on the device screen and can cause images of the graphics on the device screen to be captured (e.g., via an on-device camera). For example, the graphics can include a single color presented on the screen, graphics having various colors, graphics having pattern(s), and / or graphics designed to facilitate identification of screen damage. The colors presented in the images can be analyzed (e.g., by a neural network on the server) to determine if one or more pixels are not accurately presenting the color. In some implementations, k-means can be used to recognize features of approximately the same color within the image. Thus, damage to pixels can be identified based at least in part on analysis of the captured image(s).

[0086] While implementations can include commercially available neural networks, in some embodiments, the neural network can be a customized neural network capable of learning patterns, recognizing cracks, assigning a probability of damage being present on the device, and / or having other suitable functionality. In some embodiments, the neural network can include a cloud-based system accessible by a return application. In some embodiments, the neural network can be stored and operational on the device.

[0087] In some implementations, the neural network uses a training tool that allows the neural network to learn how to identify the screen state of the device. The neural network can be trained by using the neural network. FIGS. 9A, 10A, and 11A show an embodiment of the learning tool, and FIGS. 9B, 10B, and 11B show associated second images. FIGS. 9C, 10C, and 11C show examples of processing of second images by the neural network, and FIGS. 12A-12B show examples of accuracy and cross-entropy achieved by the neural network. The neural network can include zero or more layers. In some implementations, the neural network may be multi-layered to facilitate processing and identification of the state of the device's screen from captured images. The neural network can be trained by providing example images, such as the example captured images shown in FIGS. 9B-11B and the corresponding example images in which damage is identified, shown in FIGS. 9A-11A. As shown in FIGS. 9B-11B, damage is identified (e.g., by different colors and / or patterns). The neural network can process example captured images, such as images 9B-11B, according to one or more of the described operations. For example, a captured image may be divided into multiple portions, and a neural network can determine which of those portions contain damage. The results can be compared to a training tool, allowing the neural network to learn and become more accurate, as shown in Figures 12A-12B.

[0088] Although the mirror has been described as providing a reflective surface that reflects the image presented on the device's screen, any reflective surface can be used in place of and / or in conjunction with the mirror. For example, metal and / or plastic reflective pieces can be used to capture the image of the device's screen.

[0089] Although users are described as humans, a user may be a single person, a group of people, one or more people interacting with one or more computers, and / or computer systems (e.g., devices, robots).

[0090] In various embodiments, a device is described. The device may be any suitable device, such as a smartphone, tablet, laptop, game console, portable media player (e.g., an electronic reader and / or video player), wearable (e.g., a watch, jewelry, etc.), or video camera capable of running an application and / or taking pictures of the device. The device may include a memory, a processor, and a camera (e.g., a component capable of capturing images). The device may store a return application in the memory, and the processor may execute the return application to perform one or more of the described operations. In some embodiments, the device may perform one or more of the described operations as performed by the server on behalf of or in cooperation with the server.

[0091] In various embodiments, a server is described. The server 110 may include a memory and a processor that executes instructions and manipulates data to perform the server's operations. The server may, in some embodiments, be cloud-based and / or support cloud-based processing and storage. As described, a neural network may be stored in the server's memory, and the processor may perform the neural network's functions. The memory may include a repository of data (e.g., a database). The data may include data for teaching and / or configuring the neural network (e.g., a set of images, feedback regarding correctly and / or incorrectly identified damage, patterns, etc.), resale price, a price to offer for the device, screen repair and / or replacement costs, location of image supplement information, market information, reuse information, insurance information, information to verify the device's identity, and / or other suitable information.

[0092] Additionally, various software may be stored in the memory of the server. For example, the software may be capable of communicating with the device, performing one or more operations to determine the state of the device screen, running tests on one or more components of the device, etc. In various implementations, one or more of the captured images may be stored in the memory of the device or the server and / or transmitted (e.g., from the user device to the server and / or vice versa).

[0093] The software on the server and / or return application may include a graphical interface to facilitate user interaction. A communications interface may enable the server to communicate with other repositories and / or devices over a network. The communications interface may transmit data to and / or from the server and / or receive data from devices and / or coupled repositories and / or other computer systems via network protocols (e.g., TCP / IP, Bluetooth, and / or Wi-Fi) and / or buses (e.g., serial, parallel, USB, and / or FireWire).

[0094] A graphical user interface (GUI) of the server and / or return application can be displayed on a presentation interface, such as a device screen. The GUI may be operable to allow a device user to interact with the repository and / or server. Generally, a GUI provides a device user with an efficient and user-friendly presentation of data provided by the server and / or return application. A GUI can include multiple displays with interactive fields, pull-down lists, and buttons operated by the user. As an example, a GUI presents a search-type interface and receives commands from the user. It should be understood that the term graphical user interface may be used in the singular or plural to describe one or more graphical user interfaces within each of a particular graphical user interface display. Furthermore, a GUI contemplates any graphical user interface, such as a common web browser, that processes information within the server and / or device and efficiently presents that information to the user. In some implementations, the GUI can present a web page that embeds content from the return application and / or server. The server can receive data from the device via a web browser (e.g., Microsoft Internet Explorer or Safari) and return an appropriate Hypertext Markup Language (HTML) or Extensible Markup Language (XML) response.

[0095] While FIG. 1 provides an example of a server that can be used in the present disclosure, the server can be implemented using computers other than servers and server pools. For example, the server can include a general-purpose personal computer (PC), a Macintosh®, a workstation, a UNIX®-based computer, a server computer, or other suitable device. According to one embodiment, the server can include a web server and / or a cloud-based server. The server may be adapted to run any operating system, including UNIX®, Linux®, Windows®, or other suitable operating systems. In short, the server can include any combination of software and / or hardware suitable for providing access to data and / or converting data into an appropriate compatible format.

[0096] Although embodiments describe a single processor in a server and / or device, multiple processors may be used depending on particular needs, and references to a processor are intended to include multiple processors where appropriate. A processor may include a programmable logic device, a microprocessor, or other suitable device for logically manipulating information.

[0097] Although the embodiments discuss the use of a neural network to perform at least a portion of the analysis of the return application, analysis frameworks implemented by other computing devices can be used as appropriate. Although the embodiments describe the neural network as being included on a server(s) (e.g., a physical server and / or a virtual server), the neural network may be housed on other devices. For example, the neural network may execute and / or at least partially execute on a user device, such as a first device and / or a second device, such as a mobile device. The neural network, in some embodiments, may be cloud-based and accessed by a server and / or a user device (e.g., a first device and / or a second device).

[0098] Although the embodiments describe a single memory of a server and / or device, multiple memories can be used as appropriate. For example, the memory can include an SQL database, a relational database, an object-oriented database, a distributed database, an XML database, cloud-based memory, device memory, and / or a web server repository. Furthermore, the memory can include one or more forms of memory, such as volatile memory (e.g., RAM) or nonvolatile memory, such as read-only memory (ROM), optical memory (e.g., CD, DVD, or LD), magnetic memory (e.g., hard disk drive, floppy disk drive), NAND flash memory, NOR flash memory, electrically erasable programmable read-only memory (EEPROM), ferroelectric random access memory (FeRAM), magnetoresistive random access memory (MRAM), nonvolatile random access memory (NVRAM), nonvolatile static random access memory (nvSRAM), and / or phase-change memory (PRAM).

[0099] It is understood that the embodiments are not limited to the particular systems or processes described, which may, of course, vary. It is also understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" include plural referents unless the content clearly dictates otherwise. Thus, for example, a reference to "an image" includes a combination of two or more images, and a reference to "a graphic" includes a combination of multiple different types and / or graphics.

[0100] Although the present disclosure has been described in detail, it should be understood that various changes, substitutions, and alterations can be made herein without departing from the spirit and scope of the present disclosure, as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the processes, machines, manufacture, compositions of matter, means, methods, and steps described herein. As those skilled in the art will readily appreciate from this disclosure, any now-existing or later-developed process, machine, manufacture, composition of matter, means, method, or step that performs substantially the same function or achieves substantially the same result as the corresponding embodiment described herein can be utilized in accordance with the present disclosure. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.

Claims

1. 1. A method for identifying a state of a component of an electronic device, comprising: receiving, at a preprocessing layer of a trained multi-layer neural network, from the electronic device, one or more first images of the electronic device, the preprocessing layer of the trained multi-layer neural network being trained with a preprocessing layer training dataset including a plurality of component images not associated with component damage, the preprocessing layer of the trained multi-layer neural network being configured to preprocess the one or more first images and output at least one adjusted component image; the method further comprising:

10. A method for determining a state of a component of an electronic device, the method comprising: inputting the adjusted component images into a processing layer of the trained multi-layer neural network; the processing layer of the trained multi-layer neural network being trained using a processing layer training dataset including component images known to have component damage and component images known to not have component damage; and the processing layer of the trained multi-layer neural network being configured to output a component damage probability corresponding to the determined state of the component of the electronic device based on an analysis of the adjusted component images.

2. 10. The method of claim 1, wherein the preprocessing layer of the trained multi-layer neural network is configured to identify one or more of a reflection, a logo, a shadow, an obstacle, a non-electronic device image, a partial image, or a blurred image in the one or more first images.

3. The preprocessing layer of the trained multi-layer neural network comprises: determining that at least one of the one or more first images is not a processable image; The method of claim 1 or 2, configured to send a notification to the electronic device that includes at least part of the reason why the at least one first image is not a processable image.

4. determining an orientation of the electronic device based on the at least one of the one or more first images; and providing guidance for adjusting the orientation of the electronic device based on the determined orientation.

5. Pre-processing the one or more first images and outputting the at least one adjusted component image includes: identifying the components of the electronic device in the one or more first images; and modifying the one or more first images such that all portions of the one or more first images that are not identified as the components are removed.

6. 6. The method of claim 1, wherein pre-processing the one or more first images and outputting the at least one adjusted component image comprises adjusting at least one parameter of the one or more first images.

7. The method of claim 6 , wherein the parameters include one or more of contrast, brightness, coloration, sharpness, exposure, blur, alignment, image transformation, or size.

8. outputting the component damage probability corresponding to the determined state of the component of the electronic device based on the analysis of the adjusted component image, Dividing the adjusted component image into a plurality of portions; analyzing each of the plurality of portions of the adjusted component image by a separate node of the processing layer; determining whether one or more of the portions of the adjusted component image include a defect; identifying portions of the adjusted component image adjacent one or more of the portions including a defect; 8. A method according to claim 1, further comprising determining the component damage probability for the component of the electronic device based on whether one or more of the plurality of portions of the adjusted component image are determined to contain an image and whether one or more of the plurality of portions of the adjusted component image adjacent to one of the plurality of portions of the adjusted component image determined to contain damage also contain damage.

9. The method of any of claims 1 to 8, wherein the processing layers of the trained multi-layer neural network are fine-tuned using previously processed component images from multiple electronic devices.

10. Pre-processing the one or more first images includes: rating the probability of the one or more first images; and adjusting an analysis sensitivity level of the analysis of the adjusted component images by the processing layer of the trained multi-layer neural network.

11. Pre-processing the one or more first images includes:

11. The method of claim 1, comprising determining blur of the one or more first images, wherein the blur is calculated based on a rate of change of color on edges of the one or more first images.

12. The method according to any of the preceding claims, wherein the one or more first images are preprocessed at full resolution in the preprocessing layer of the trained multi-layer neural network.

13. The method of any of claims 1 to 12, wherein the one or more first images are pre-processed at a lower resolution in the pre-processing layer of the trained multi-layer neural network.

14. 14. The method of any of claims 1 to 13, wherein the component is a front casing of the electronic device, and the processing layer training dataset includes at least a set of images including a front casing with known cracks, breaks, or other types of damage.

15. The method of claim 14 , wherein the preprocessing layer training data set includes images of a forward casing that are not associated with damage.

16. 16. The method of any of claims 1 to 15, wherein the component is a screen of the electronic device, and the processing layer training dataset includes at least a set of images including a screen with known cracks, breaks, or other types of damage.

17. The method of claim 16 , wherein the preprocessing layer training data set includes images of a screen that is not associated with a lesion.

18. A program for causing a server to execute the method according to any one of claims 1 to 17.

19. a memory for storing the program according to claim 18; and a server including a processor for executing the program.

Images