Device screen damage detection

Abstract

This provides a method for determining the state of one or more device screens. [Solution] The method includes: the user selecting a return application that facilitates capturing an image of the device screen; capturing an image of the screen using the camera of the same device or another device; positioning the device near a mirror so that the device can capture an image of one or more of its screens; processing and / or analyzing the captured images to determine whether the device screen is damaged; sending a notification based on the condition of the device screen; and determining the price of the device based on the condition of the screen.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims priority to U.S. Non - Provisional Application No. 15 / 452,707, filed on March 7, 2017, entitled "SCREEN DAMAGE DETECTION FOR DEVICES", and U.S. Provisional Application No. 62 / 304,729, filed on March 7, 2016, entitled "SCREEN DAMAGE DETECTION FOR MOBILE DEVICES", which are hereby incorporated by reference in their entirety herein.

[0002] Technical Field The present invention relates to determining the state of one or more device screens.

Background Art

[0003] Background Devices such as smartphones, watches, and tablets are often sold back to the manufacturer or a third party when consumers upgrade their devices. These used devices may have value in the resale market based on their condition. For example, a user may provide a used phone to a reseller, and the reseller may evaluate the condition of the phone and offer a price based on the evaluation. However, the evaluation of the device's condition is performed by humans and is therefore time - consuming and often subjective. Further, the user has to wait for the evaluation (e.g., at a kiosk, store, etc.) to receive the offered price, which may reduce the user's willingness to resell the used device and / or lower the user's satisfaction with the process.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Summary In various embodiments, the state of one or more screens on the device is determined (e.g., whether or not screen damage is present). A user can request an evaluation of the screen and / or device state via a return application on the device. The return application can determine the device state (e.g., for valuation, resale, insurance claims, warranty claims). The return application can display a first graphic on the device screen and prompt the user to position the device in front of a reflective surface such as a mirror (e.g., to allow the device's reflected image in a mirror to be captured by the device itself). The return application can guide the user to position the device in a predetermined location (e.g., closer to a mirror) to increase the probability of capturing an image that can be used to accurately evaluate the screen state. Images of the device screen can be obtained (e.g., images can be automatically taken and / or taken by the user). For example, a photograph of the screen's reflection in a mirror can be captured. The screen images can be processed and / or analyzed, and based on the analyzed images, a determination can be made as to whether or not the screen is damaged. Based on the analysis of the screen damage, one or more notifications and / or flags can be generated, transmitted, and / or displayed.

[0005] In some embodiments, a second device can be used to facilitate the identification of the state of the device's screen. For example, the first device may have a damaged and / or broken camera, and / or the screen of the first device may be so damaged that the second device cannot be identified. It may not be possible to interact with the return application on device 1 (for example, a cracked screen may injure the user's finger). Therefore, the return application on device 2 can be used to identify the screen state of device 1. [Means for solving the problem]

[0006] In various embodiments, the state of one or more screens of a device (e.g., an electronic device such as a mobile device or laptop) can be identified. A request for evaluation of the state of the screen or part of a first device can be received via a return application on the first device. A first graphic containing a first identification code may be displayed on the screen of the first device (e.g., a display component). The return application can cause the first graphic to be displayed on the screen of the first device. At least a portion of the first image of the first graphic may be captured by the camera of the first device. The first image may include a reflection of the first graphic on a reflective surface such as a mirror. One or more second graphics may be displayed on the screen of the first device (e.g., via a return application), and one or more portions of the image of at least one of the second graphics may be captured (e.g., via the camera of the first device). One or more of the second images may include a reflection of at least one of the second graphics on a reflective surface such as a mirror. The return application may control a camera component of the first device and / or allow a user to control said camera component. The return application can access images captured by the camera of the first device. One or more of the second images can be processed to determine the state of the screen of the first device. Processing the second image(s) includes dividing the second image into parts, determining whether one or more of the parts of the second image contain damage, and identifying parts adjacent to one or more of the damaged parts. The state of the screen of the first device can be determined based on whether one or more of the one or more parts of the second image are determined to contain damage, and whether one or more of the parts adjacent to one of the parts determined to contain damage also contain damage.

[0007] Embodiments may include one or more of the following features: The identity of a first device can be verified based on an analysis of a first identification code. A second captured image containing a second graphic may be tagged with the first identification code or a portion thereof from the first graphic. Capturing at least a portion of a second image(s) of the second graphic(s) may include determining the orientation of a device based on the captured image of the first graphic and providing guidance for adjusting the orientation of the device based on the determined orientation. At least a portion of additional images of the first graphic may be captured via a camera of the first device. In some embodiments, the orientation of a device can be determined based on the captured image of the first graphic, and guidance for adjusting the orientation of the device can be provided based on the determined orientation. At least a portion of additional images(s) of the second graphic may be captured via a camera of the first device, and each of the additional images may include a reflection of at least one of the second graphics on a reflective surface. If it is determined that the captured first image(s) and / or the captured second image(s) are not processable images, the first device may be allowed to change orientation in order to capture a processable image. In order to capture a processable image, in some embodiments, the orientation of the device can be determined based on the captured first image or one or more captured second images, and guidance can be provided for adjusting the orientation of the device based on the determined orientation. ) can tag at least a portion of the captured first image. In some embodiments, one or more processes of the first device may be restricted (e.g., pop-ups, alerts, banners, etc.) while the image is being captured and / or while the return application is running. Identifying the screen or a portion of the first device in the second image may include identifying the screen of the first device in the second image by utilizing corner detection and / or edge detection. Processing one of the second images may include identifying the screen or a portion of the first device in the second image and generating a third image, wherein any portion of the second image that is not identified as the screen or a portion thereof in the second image is constrained not to be included in the third image. The third image may be divided into parts (e.g., parts that do not overlap with the second image and / or in addition to the second image), and it may be determined whether one or more of the parts of the third image contain damage. Parts adjacent to one or more parts that contain or do not contain damage can be identified. Determining the state of the screen of the first device may be based on whether one or more of one or more parts of the third image are determined to contain damage, and whether one or more parts adjacent to one of the parts determined to contain damage contain damage. Generating the third image may include modifying the second image so that multiple parts of the second image that are not identified as the screen or part thereof are removed. Identifying the screen or a part thereof may include identifying the active area of ​​the screen of the first device.

[0008] In various embodiments, the state of a screen(s) of a device (e.g., an electronic device such as a mobile device) can be identified. For example, the state of a first device can be identified using a second device. At least one of the first or second device may include a camera (e.g., an external image capture component). The first and second devices may or may not be the same device. Requests for evaluation of the state of the screen or part of the first device can be received via a return application on the second device. The first device may include a return application. A first graphic including a first identification code can be made to be displayed on the screen of the first device via a return application on the first device. At least a portion of the first graphic displayed on the first device may be captured via the camera of the second device. One or more second graphics can be made to be displayed on the screen of the first device, and at least a portion of the second graphics(s) displayed on the first device may be captured via the camera of the second device. To determine the screen state of the first device, one or more of the second image may be processed (e.g., preprocessed and / or processed). Processing the second image may include dividing the second image into parts and determining whether one or more of the parts of the second image contain damage. In some embodiments, a neural network may perform actions of a return application, such as image processing. Parts adjacent to one or more parts containing damage can be identified. Adjacent parts may or may not contain damage. The screen state of the first device may be determined based on whether one or more of the parts of the second image are determined to contain damage, and whether one or more of the adjacent parts contain damage.

[0009] In various embodiments, if it is determined that the device in state 1 is damaged, damage information can be determined. Based on the damage information, a flag can be generated to identify one or more portions of the second image that are determined to contain damage. If it is determined that the state of the first device is damaged, the touchscreen of the first device can be tested (for example, according to a known touchscreen test). The brightness can be calibrated based on a captured first image (for example, to facilitate image processing and / or accuracy). Enabling the presentation of a second graphic on the screen of the first device may include enabling the presentation of a set of burst images on the first device. The set of burst images includes at least one of a plurality of brightness levels of the second graphic. Capturing at least a portion of one or more second graphics presented on the first device may include capturing a set of burst images presented on the first device and selecting one of the captured burst images by determining which color in the captured set of burst images is most similar to a reference color. The selected captured burst image may be identified as one of the captured second graphics (for example, for preprocessing and / or processing). Capturing at least a portion of one of the second images of at least one of the second graphics may include determining the orientation of the device based on the captured image of the first graphic and providing guidance for adjusting the orientation of the device based on the determined orientation. At least an additional portion of the image of the first graphic may be captured via the camera of the first device. Enabling the display of one or more second graphics on the screen of the first device may include enabling the sequential display of two or more second graphics on the screen of the first device. Capturing one or more of the second graphics may include capturing at least one image of each of the second graphics sequentially displayed on the screen of the first device. Enabling the display of one or more second graphics on the screen of the first device may also include enabling the simultaneous display of two or more second graphics on the screen of the first device.

[0010] Details of one or more embodiments are described in the accompanying drawings and the following description. Other features, purposes, and advantages of the embodiments will become apparent from the following description and drawings. [Brief explanation of the drawing]

[0011] For a more complete understanding of this disclosure and its features, the following description is provided below in conjunction with the attached drawings. [Figure 1] This is a diagram showing an exemplary embodiment of the system. [Figure 2] This figure shows an exemplary embodiment of a process for determining the state of a device's screen. [Figure 3A] This figure shows an exemplary embodiment of the positioning of a device in front of a mirror. [Figure 3B] This figure shows an exemplary embodiment of image capture, including a device screen. [Figure 3C] This figure shows an exemplary notification embodiment displayed by the return application. [Figure 4] This document illustrates an exemplary process for determining whether a device screen is damaged. [Figure 5A] This figure shows an exemplary embodiment of an image. [Figure 5B] This figure shows an exemplary embodiment of an image. [Figure 6A] This figure shows an exemplary embodiment of an image before processing. [Figure 6B] This figure shows an exemplary image embodiment after processing. [Figure 7] This figure shows a partial embodiment of the image division. [Figure 8A] This figure shows an exemplary embodiment of a captured image on a device screen. [Figure 8B] This figure shows an embodiment of the interface generated as a result of determining whether or not the device screen shown in Figure 8A is damaged. [Figure 8C] This figure shows an exemplary embodiment of a captured image on a device screen. [Figure 8D] FIG. 8C is a diagram showing an embodiment of an interface generated as a result of determining whether or not the device screen shown is damaged. [Figure 8E] FIG. 4 is a diagram showing an embodiment of an exemplary captured image of a device screen. [Figure 8F] FIG. 8E is a diagram showing an embodiment of an interface generated as a result of determining whether or not the device screen shown is damaged. [Figure 9A] FIG. 10 is a diagram showing an embodiment of an exemplary learning tool. [Figure 9B] FIG. 13 is a diagram showing an embodiment of an exemplary second image obtained by a learning tool. [Figure 9C] FIG. 9B is a diagram showing an embodiment of an exemplary process of an exemplary second image shown in FIG. 9B. [Figure 10A] FIG. 10 is a diagram showing an embodiment of an exemplary learning tool. [Figure 10B] FIG. 13 is a diagram showing an embodiment of an exemplary second image obtained by a learning tool. [Figure 10C] FIG. 10B is a diagram showing an embodiment of an exemplary process of an exemplary second image shown in FIG. 10B. [Figure 11A] FIG. 10 is a diagram showing an embodiment of an exemplary learning tool. [Figure 11B] FIG. 13 is a diagram showing an embodiment of an exemplary second image obtained by a learning tool. [Figure 11C] FIG. 9B is a diagram showing an embodiment of an exemplary process of an exemplary second image shown in FIG. 9B. [Figure 12A] FIG. 37 is a diagram showing an embodiment of an exemplary accuracy result of an exemplary neural network. [Figure 12B] FIG. 40 is a diagram showing an embodiment of an exemplary cross-entropy result of an exemplary neural network. DETAILED DESCRIPTION OF THE INVENTION

[0012] Like reference numerals in the various drawings indicate like elements. Detailed Description In various embodiments, a device (e.g., an electronic device such as a smartphone, watch, tablet, e-reader, laptop, or portable game console) may be evaluated for the state of one or more of its screens. The device may include components that can capture external images, such as a camera (as opposed to components that can save images of a graphical user interface displayed on the screen, such as a screenshot).

[0013] The condition of the screen can affect its appearance and / or usability (for example, cracks may need to be repaired before use and / or may or may not affect use and / or viewing on the device), and therefore the price of the device may change when it is put up for sale. Human evaluations can cause variability because evaluations can be subjective. Human evaluations can create a time lag between when a user puts a device up for sale and when a price is offered for that device. These factors can reduce the willingness to resell the device, reduce user satisfaction with the process, thereby potentially removing good devices from the market, potentially increasing resale prices (for example, because supply becomes more limited), and / or reducing the recycling of the device (for example, because the device may be stored or discarded). In some embodiments, automated determination of screen condition can reduce fraudulent activity. For example, screen condition can be verified for insurance purposes (e.g., for issuing insurance certificates and / or claims) and / or for reuse. As described, automatically determining the screen state can reduce the occurrence of fraudulent activity, thereby lowering insurance costs (for example, because the state can be verified and / or determined objectively), and / or increasing user satisfaction (for example, because it eliminates the need to bring the device to the store for state verification). (and / or, so that the objective state of the device can be obtained for possible reuse). Therefore, automatic detection of the state of the device or its components, such as whether or not screen damage is present, is necessary.

[0014] As shown in Figure 1, one or more devices 110 may be coupled to a server 120 (for example, via a network such as the Internet or other communication network). The server 120 can be any suitable server. In some embodiments, the server 120 may include a neural network. The neural network may be implemented on the server via software and / or hardware (for example, several embodiments of neural networks are available on the market and / or can be built using convolutional neural networks via the FANN library from IBM®, CogniMem®, and / or on the Google® tensorflow framework). In some embodiments, a convolutional neural network may be utilized because, in some embodiments, it may have greater image processing capabilities than other neural networks (e.g., faster, more accurate, etc.).

[0015] In some embodiments, the neural network may be able to self-tune and / or adjust based on system updates to improve the accuracy of screen damage detection. For example, the neural network may be provided with learning tools such as captured images and / or marked damage (e.g., captured images in which the identified damaged portion is flagged, for example, by changes in color and / or pattern) to facilitate and / or enable the neural network to further learn to identify the state of the device. To develop the neural network (for example, so that the neural network can identify the state of the device or a part thereof), the neural network may be able to analyze the learning tools, the relevant captured images, process the relevant captured images, identify damage through processing, and / or identify differences and / or similarities between the portion having the identified damage and the learning tools. In some embodiments, the neural network may systematize its own learning of perceived damage to pixels and / or other parts of the device (for example, based on the learning tools and / or processing of captured images provided to the neural network).

[0016] Device 110 may include one or more device screens 115 (e.g., monitors, LCD screens, glass, Gorilla Glass, etc.). Device 110 may include cameras (e.g., front-facing cameras, or cameras on the same side of the device as the device screen) that can capture reflections of images on the device screens. Device 110 may include a return application that is stored in the device's memory and can be executed by the device's processor. The return application may enable Device 110 to communicate with Server 120. The return application may receive input from the user, prompt the user to provide input and / or position the device, send images and / or notifications, generate graphics, capture images, instruct device components such as cameras to capture images, restrict device components (e.g., flash), restrict device behavior (e.g., pop-ups, banners, alerts, reminders, etc. between the return application and / or image capture), and / or communicate with other devices such as Server 120. In some embodiments, the return application allows the sale of the device through the application, determines the condition of the device or its components, allows the value of the device to be determined based on the condition of the device, determines the condition of the device or its components for reuse of the device, and allows the device or its components for insurance claims. The system can determine the status of components (for example, if a user wishes to submit a claim for a device insurance policy, the return application can determine the status of the device or components), and determine the status of the device for warranty claims.

[0017] The server and device 110 can perform one or more of the described operations separately and / or in conjunction with other devices. In some embodiments, the server may not be used, and a return application (e.g., instead of the server) may perform one or more of the described operations. In some embodiments, the server may perform one or more of the described operations in order to increase the speed of the operation. In some embodiments, one or more of the device 110 and / or the server may work together to perform one or more of the operations. In some embodiments, the server may be cloud-based, and the device(s) may communicate with the server to perform operations such as image processing and / or analysis (e.g., state identification).

[0018] In some embodiments, when a user decides to sell a device (e.g., when upgrading a device, switching to a new device), the user can select a return application on the device. The return application can determine information about the device (e.g., device condition, device characteristics such as model, market resale price, etc.) and display the device's price. Screen condition may adjust the price presented for the device via the application, as screen damage may be aesthetically unsightly to some users, may indicate damage to other components, and / or may be costly to replace and / or repair. Screen condition can be determined automatically by the application and / or server, allowing for faster evaluation and / or greater consistency (e.g., since the condition is not determined by a human). Also, because screen condition can be determined automatically, the possibility of a user intentionally reporting an inaccurate screen condition can be reduced (e.g., a user might try to report a screen as being in good condition even if it has cracks or damage, because devices with good screen condition command a higher market price. Automated detection can prevent fraudulent behavior common to self-reporting and / or human-based analysis systems). In addition, the condition of the screen (e.g., large cracks, deep cracks, etc.) may prompt the system to determine whether other components of the device may also be damaged. Once the condition of the device screen and / or the device itself has been determined, the offering price for the device can be determined (e.g., by a server and / or return application). In some embodiments, the base cost of the device can be determined (e.g., based at least in part on resale price, recycling price, market information, supply of similar devices, and / or demand for the device), and the base price can be adjusted based on the condition of the device (e.g., reduced for screen damage, reduced for damage to other components, and / or increased for new and boxed). The user can then decide whether or not to sell the device based on the price received.The entity presenting the price can make the offer based on its knowledge of the screen state, without creating a significant time delay between the user initiating the resale process and receiving the price. Therefore, in some embodiments, this can increase satisfaction with the process for users and / or entities purchasing used devices or verifying their claimability or insuranceability.

[0019] In some embodiments, in addition to and / or instead of resale purposes, the condition of the device can be determined (e.g., by a return application and / or server) for insurance claims (e.g., device insurance). For example, the device may be damaged and / or suspected to be damaged. The status of the components can be determined and / or reported, and an insurance claim can be submitted and / or verified. In some embodiments, a return application can be used to determine the status of the device or a part thereof when purchasing device insurance. Thus, to determine the status of the device or its components, it is possible to rely on self-reporting and / or determine the status via a return application on the device, rather than bringing the device to a physical store.

[0020] In some embodiments, the condition of a device can be determined for device warranty purposes (e.g., by a return application and / or server). For example, a manufacturer, reseller, repairer, and / or refurbisher may provide a warranty for a device. Warranties can be complex, and it may be difficult for users to understand which parts and / or types of damage are covered. The condition of the device and / or its components and / or whether a damaged component is covered can be determined using a return application. In some embodiments, the condition determined via the return application can be used to submit and / or verify a warranty claim.

[0021] In some embodiments, the state of a device can be determined to determine whether the device is reusable (for example, by another user). For example, a second user may be able to obtain the state of the first user's device via a return application on the device (for example, the return application may send a notification to the second user with the device's state). The second user can then obtain an assessment of the device's state that is less susceptible to fraud, subjectivity, or error (for example, than human assessment). The second user can then use the damaged and / or undamaged device (for example, whether the damage identified by the system has been repaired or not). The return application may be used to identify what repairs can be performed and / or whether the device can be used without repairs.

[0022] Figure 2 shows an embodiment of an exemplary process 200 for determining the state of a device screen. Images(s) of the device screen can be received via a return application on the device (operation 210). For example, a user can open a return application on the device. In some embodiments, the return application can initiate communication with a server (for example, to obtain the device's recent base price, to obtain software updates, etc.). The user can request a price quote for the device to determine if the device is reusable, to submit an insurance claim, and to verify the device state for insurance claims and / or contracts, and / or for other appropriate reasons to determine the device's state. The return application can prompt the user to obtain images of the device screen (for example, via visual, audio, and / or haptic notifications). For example, the return application can prompt the user to position the device, such as holding the device screen in a mirror. Because the quality of the images may affect the ability to determine the state of the device screen (for example, whether the device screen is damaged), the return application can prompt the user to adjust the device's position. For example, the return application can prompt the user (e.g., via visual and / or auditory notifications) to adjust the device's position, such as moving the device closer or further away, tapping an image to refocus, or adjusting the angle at which the device is held. When the device is in a predetermined position, it can capture one or more images. After the return application determines that the device is in a predetermined position (e.g., the optimal position for capturing an image), it can automatically capture an image. The return application can detect the device in the image and automatically focus the camera on the device. This is possible. Furthermore, the return application can also crop the device within the image. The captured image may be a reflection of the device in a mirror and may include the device screen. Rather than using the device's "screen capture" function to obtain an image outside the screen, rather than an image of the interface presented on the device screen, a reflection of the device in a mirror can be captured.

[0023] In some embodiments, uploading images to the server and / or the return application that are not captured via the return application may be prohibited (Action 220). When screen image status is determined based on the image, there may be a possibility of fraudulent activity. For example, a user may take a picture of a similar, undamaged device in order to increase the price offered for a damaged device. To reduce the costs associated with fraudulent or incomplete / misleading display of device screens, the return application may not accept uploads or selections of images that are not captured via the return application. For example, the return application may process images captured by the application (e.g., the application accesses a camera application on the device, processes images captured by the camera application, and tags the captured images with device information such as identification information, date, and / or timestamp). The return application may not accept images from the device or cloud storage photo library (e.g., a user taking a picture of the device screen and selecting an image from the device's photo library may not be permitted).

[0024] Based on the received image(s), it can be determined whether or not the device screen is damaged (operation 230). The return application can send the received image(s) to the server for analysis. The server can process and / or analyze the image(s) to determine whether or not the device screen is damaged. For example, the server may include a neural network trained to identify damaged screens and / or the probability that a screen or part thereof is damaged. In some embodiments, the neural network can be trained to identify damaged screens by processing a set of images containing screens with known cracks, breaks, and / or other damage, as well as a set of undamaged images. The neural network can learn to identify patterns associated with screen damage from the set of images. Based on the analysis of the received image(s) by the neural network trained to recognize screen damage, the server can determine whether or not the device screen is damaged. A neural network (e.g., residing on a server) may have a first or outer layer that can be trained to identify (e.g., pre-process) typical screen images unrelated to damage, such as reflections (single or multiple), logos (multiple), shadows (multiple), and / or other artifacts (multiple) that may be found in the device's image. By training the outer layer neural network to identify typical screen images unrelated to damage, it is possible to reduce the occurrence of false assessments that any damage (e.g., cracks, chips, scratches) is present when there is actually no damage on the device, and / or increase the accuracy of damage assessment. In addition, by training the outer layer of the neural network to identify typical screen images unrelated to damage, it is possible to enable recapture of the screen image (e.g., to obtain more accurate processing of the image of damage).

[0025] Process 200 can be implemented by various systems, such as system 100. Furthermore, various operations may be added, removed, and / or modified. For example, the return application may perform one or more operations when determining whether the device screen is damaged. The return application may, for example, perform at least part of image analysis. In some embodiments, two or more images The device can capture and process data to determine whether the device screen is damaged. In some embodiments, if it is determined that the device screen is damaged, the base cost of the device can be adjusted. For example, if the device screen is damaged, the base cost may be reduced by screen replacement costs, screen repair costs, processing time costs, and / or labor costs.

[0026] In some embodiments, a return application (e.g., on a server and / or device) can preprocess images. For example, a return application can identify captured low-quality images (e.g., low-quality) (e.g., one or more of the images captured via the return application). A return application can preprocess images (e.g., via an outer layer of a neural network and / or via a return application on a user device) to identify bad images (e.g., via an external classifier on a neural network that is configured and trained to detect conditions that may "hide" cracks and / or defects). For example, preprocessing can identify low-quality details (e.g., via an external classifier) ​​such as fingers and / or other obstructions being present on the screen, an object in the image not being a phone, the entire screen not being in the image, and / or cracks being hidden by reflections. In some embodiments, an outer layer on a neural network can identify other defects that cause bad images (through training). In some embodiments, preprocessing can filter images based at least in part on other factors that can be computed by analyzing the image, such as blur and / or poor color. Blurry associated with low image quality can be calculated based on the rate of color change at the edges to determine whether the image is in focus. Poor color in an image that may be associated with low image quality can be detected by examining the color intensity.

[0027] In some embodiments, the state of the first device may be determined using a second device (e.g., a device different from the first device). For example, the first screen of the first device may be damaged, thereby preventing the user from using the device or making it undesirable to use the device (e.g., a finger could be injured by a crack in the screen, a chip could come loose from the screen, the screen could be further damaged by use, etc.). Therefore, the second device can be used to capture an image of the first device. The first and second devices may include a return application (e.g., one or more operations of the return application may be performed by the processors of the first and second devices). A request may be received to request an evaluation of the state of the screen or part thereof of the first device via the return application on the second device. The return applications on the first and second devices may communicate (e.g., directly and / or indirectly via the return application on a server). For example, a return application on a second device can communicate with a return application on a first device to enable graphics to be displayed on the screen of the first device via the return application. The return application can present an image (including, for example, a first graphic and / or a second graphic) to the first device and enable the presented image to be captured by the second device. The captured image can be pre-processed and / or processed to determine the state of the screen of the first device. Thus, an evaluation of the state of the screen of the first device can be obtained even if the first device is limited in use (for example, due to damage).

[0028] In some embodiments, the device can automatically adjust its position. The device can maintain equilibrium on a nearly flat surface, and therefore the device can It can be placed on a surface in front of a mirror. The device can automatically trigger one or more vibrations to automatically reposition (e.g., rotate) the device. In some embodiments, if the auto-adjustment fails to position the device in a given position for image capture, the return application can prompt the user to adjust the position via notification (e.g., acoustic, tactile, and / or visual).

[0029] In some embodiments, notifications may send instructions to the user on how to reposition the device (e.g., move closer or further away). In some embodiments, the return application may generate positioning aids for display on the device. Positioning aids may indicate (e.g., via visual, auditory, and / or tactile signals) whether the device is in a given position, how close the device is to the given position, and / or in which direction the device is in and / or out of a given position. For example, positioning aids may include electronically generated bubble levels (e.g., accelerometers in the device; and / or GPS, which can facilitate the determination of the device's orientation, calculate where the device is detected in an image, and / or provide real-time feedback on changes in the position(s) and / or angle(s) in which the device is held). In some embodiments, instructions may include instructions on how to change the environment in which the device is positioned (e.g., a room). For example, instructions may include instructions to increase and / or decrease lighting, instructions to close a window(s) (e.g., to reduce glare), and / or other appropriate instructions.

[0030] In some embodiments, the return application can use cues (e.g., acoustic, tactile, and / or visual) to facilitate image capture. For example, the graphical user interface of the return application can give the impression of a "tunnel" within the graphical user interface (e.g., via 3D square formation). For example, the graphical user interface can generate the appearance of a tunnel having identification codes (e.g., QR Code®) at the ends of the tunnel, which have size and / or shape matchers s to the required aligned squares. This can guide the user to align and position the device at the correct angle and distance from the mirror. The return application can also include other visual and / or acoustic cues. For example, the graphical user interface can include arrows (e.g., 2D and / or 3D) (e.g., via overlays, pop-ups, embedded images, etc.) that point to instruct the user to change the orientation of the device (e.g., tilt the phone sideways and / or up and down, move it back and forth).

[0031] In some embodiments, when an image(s) is successfully captured, the return application can generate a notification to be displayed on the device. For example, the captured image(s) may be preprocessed. In some embodiments, if the image(s) are determined to be unprocessable during preprocessing (e.g., the image on the device is cut off and / or does not show the entire screen, so the server cannot determine the screen state), the user may receive one or more notifications and / or be prompted to resume the process or part thereof. The neural network of the return application can perform one or more of the preprocessing operations. For example, the neural network (e.g., the outer layer of a multilayer neural network) may (e.g., be trained) act as a filter during preprocessing that rejects images with problems such as fingers covering the screen and / or light reflections from windows, but is not limited to these. The neural network may provide the user with a notification that includes at least part of the reason that the captured image is of poor quality. The reason for this rejection may (e.g., to the user) This can facilitate the correction process (for example, by reducing the user's guesswork when determining the reason for rejecting a captured image). In some embodiments, the return application can prompt the user to start the resale process, and / or can start the resale process automatically when the return application is opened.

[0032] In various embodiments, to determine the screen state, the return application may capture (for example, automatically and / or manually by user selection) an image(s) of the device screen. In some embodiments, the application may generate one or more graphics (e.g., pictures, patterns, monochrome displays, and / or graphical user interfaces) for display on the device screen. Graphics generated by the return application may include one or more colors (e.g., black, white, green, purple, etc.), one or more patterns, photographs, pictures, identifiers (e.g., QR codes®, barcodes, etc.), other appropriate graphics, and / or combinations thereof. Graphic generation may include retrieving graphics from memory (e.g., of the device and / or linked to the device) and / or generating identifiers (e.g., based on real-time verification of such information within a defined threshold to exclude the reuse of previously captured good device images or images of another device). Some graphics can facilitate the detection of the screen state. For example, graphics generated by an application may include a single-color graphic of green, black, and / or white that covers at least a portion of the display screen (e.g., the active portion or part of the display screen).

[0033] In some embodiments, the application can generate a first graphic and one or more second graphics containing an identifier. The first graphic and / or second graphics (or more) are analyzed to determine the device screen state. For example, the first graphic may include an identifier such as a QR code (registered trademark). The identifier may be generated by the return application. For example, device information (e.g., IMEI information, device usage period, device model, memory capacity, etc.) and / or user information can be encoded into the identifier. Figure 3A shows an example of positioning a device in front of a mirror, where the return application generates an identifier to be displayed on the device screen. Once the return application prompts the user to position the device so that the device screen is reflected in the mirror, the return application can generate an identifier to be displayed on the device screen. The return application can then analyze the reflection of the identifier displayed in the mirror via a camera (e.g., a camera facing the front of the device) to determine whether the device position should be adjusted. For example, if the identifier is blurred in the reflection of the identifier in the mirror, the user can be notified and prompted to adjust the device position. In some embodiments, once the device is positioned appropriately, the return application may or may not capture an image of the device screen's reflection in the mirror containing an identifier code. In some embodiments, the return application and / or server can verify that the captured QR code is associated with the device (for example, by decrypting the QR code by comparing it with known device information).

[0034] The return application can then generate one or more second graphics (e.g., a green screen, a white screen, and / or other graphics) to enable the capture of the reflection of the second image(s) in the mirror. Figure 3B shows the device in front of the mirror. An example of positioning is shown, where the return application on the device generates a second graphic. In some embodiments, as shown in Figure 3C, once the capture of the image(s) is complete, the return application can generate a notification. The captured second image(s) may be analyzed to determine the state of the screen. For example, the captured second image(s) may be sent to a server for analysis by a neural network on the server.

[0035] In some embodiments, once an identifier code is focused, captured, and / or processed (e.g., verified), the return application can automatically generate a second graphic. In some embodiments, once the identifier code is captured and / or authenticated, the state of the device screen can be more easily identified by utilizing the second graphic. For example, screen state detection can be more easily determined using a graphic that is easily identifiable by a neural network. In some embodiments, the second graphic can be rapidly generated to prevent fraudulent activity, either immediately before capturing an image of the screen device (e.g., by capturing a reflection of the device screen) and / or in a short time after the return application determines that the device is in the correct position (e.g., the identifier is in focus on the captured image of the first graphic). In some embodiments, the second graphics can be generated sequentially to enable the sequential capture of related second images. In some embodiments, image capture can be automated to coordinate graphic generation and image capture.

[0036] In some embodiments, the first image may be captured and / or processed such that the second image is tagged (e.g., attached and / or encoded) with the first image, a portion thereof, and / or a decoded identifier. In some embodiments, the second image may be captured before and / or after the first image. Tagging the second image with the first image or a portion thereof (e.g., information obtained by decoded identifiers) can reduce fraudulent activity. For example, a user may be prohibited from uploading images of different device screens (e.g., device screens not belonging to that device) because the uploaded image may not contain the encoded portion. In some embodiments, the return application and / or server may be able to identify an untagded second image to identify a fraudulent image of a device screen(s).

[0037] In some embodiments, the distance at which the image is captured by the device may also be controlled. For example, the camera's focal length may be set by the application, and the device's position may be adjusted until an identifier graphic on the device screen is in focus within the image captured by the camera. In some embodiments, the size of an identifier (e.g., a QR code®) in the image can determine the appropriate distance at which the user should position the device relative to the mirror. In some embodiments, corner angle detection can facilitate the determination of whether the device is positioned in a predetermined position for image capture with respect to the angle at which the device is positioned close to the mirror. For example, a predetermined position for image capture may include positioning the device parallel to the surface of the mirror, and thus corner angle detection can identify corner angles in the image and determine the angle at each corner to determine whether the device was parallel to the surface of the mirror at the time of image capture.

[0038] In some embodiments, the return application can adjust one or more components of the device to facilitate the capture of the device screen image. For example, the return application can turn off the flash (for example, to avoid glare). As another example, the return application can adjust the return application Device notifications (e.g., banners, alerts, etc.) can be blocked (e.g., temporarily) to allow the graphic images generated by the application to be produced without additional graphical user interfaces and / or overlays.

[0039] After images(s) are captured by the return application, one or more of the images may be analyzed to determine the state of the device screen. For example, the images(s) to be analyzed may include a second(s) graphic(s). Figure 4 shows an exemplary embodiment of process 400 for determining the state of the device screen (for example, determining whether the screen is damaged). Images containing the device screen may be received (operation 410). For example, a server may receive images via the return application. Images may be automatically uploaded to the server and / or selected for upload by a user via the return application. Images may include one or more graphics generated by the return application and displayed on the device screen.

[0040] The image may be processed by the server. Selected portions of the received image can be identified (operation 420). For example, an image captured by a return application may include the device screen and areas adjacent to the device screen. Selected portions of the image can be identified, such as the device screen and / or the active portion of the device screen (e.g., the illuminated portion and / or the touch-responsive portion). In some embodiments, the selected portions of the received image can be selected so as to reduce the size of the image to a portion relevant to the analysis of the state of the device screen (e.g., analysis of areas adjacent to the device cannot indicate whether the screen is damaged or not). Figure 5A shows an embodiment of exemplary image 500 including a device screen 510. As shown, the image includes the device screen 510 and areas adjacent to the device screen 520. The device screen 510 can be detected using corner detection 535, as shown in image 530 of Figure 5B. Figure 6B shows an embodiment of exemplary image 600 including a device screen 610. As shown, the image includes the device screen 510 and areas adjacent to the device 620. In some embodiments, edge detection of the device's edges 630 in the image can be used to identify components of the device in the image. As illustrated, the device screen 610, microphone, speaker, and case can be identified in the image. Identifying one or more components can facilitate the determination of the state of the device screen outside the active area of ​​the device screen and / or facilitate the identification of damage to other components (for example, a crack on the microphone may indicate that the microphone is also damaged).

[0041] In some embodiments, the image size can be changed (e.g., by cropping or otherwise reducing) so that only a selected portion of the image is shown in the modified image. In some embodiments, a selected portion of the received image may be labeled or otherwise identified within the image.

[0042] Selected portions of the image can be adjusted (operation 430). For example, the server can adjust contrast, brightness, color, sharpness, exposure, blur, alignment, image transformation, size, and / or other appropriate aspects of the image. In some embodiments, noise can be reduced to facilitate the identification of screen damage, as opposed to markings that should be attributed more to noise. Figure 6A shows an exemplary embodiment of image 600, including the device screen 610 before the image is adjusted, and Figure 6B shows an example of the adjusted image 650. The adjusted image 650 has reduced noise 640 in the device (e.g., lines, shading, and / or other features that should not be attributed to screen damage).

[0043] The adjusted image can be divided into parts (operation 440). For example, the adjusted image can be divided into multiple parts. Figure 7 shows an embodiment of an example of an adjusted image including a device screen 710 and the resulting parts 720. Dividing can be done by cropping the image into smaller parts, identifying areas of the image as parts, and / or dividing the image appropriately in other ways. Processing each selected part of the image can be done by a server (e.g., a neural network on the server) faster than if the entire image were processed by the server. Therefore, dividing the image can increase the speed at which the image or adjusted image is processed. For example, each part of the adjusted image may be analyzed by a node in a neural network. Therefore, since each node is analyzing a distinct part of the image, the analysis can be performed faster than if the entire image were analyzed by the server. In some embodiments, dividing the image into parts can increase the probability of detecting a screen condition and / or decrease the probability of misidentifying a screen condition. For example, since screen damage may extend to two or more parts of an image, a high-probability identifier for screen damage in one or more adjacent parts can increase the probability of screen damage in the first part. In some embodiments, the size and / or shape of the damage, as well as whether adjacent portions contain a predetermined probability of damage, can be analyzed to determine whether a portion, and therefore the device, contains damage. For example, a crack of a selected shape and / or size may be known to extend across multiple adjacent portions, and if multiple adjacent portions do not contain a predetermined probability of damage, the overall probability of screen damage may decrease. As another example, a chip of a selected shape and / or size may not extend across multiple adjacent portions of the image, and the absence of a predetermined probability of damage in adjacent portions cannot adjust the overall probability of screen damage.

[0044] The system can determine whether one or more parts of an image indicate damage (operation 450). For example, the server can analyze a part to determine whether screen damage is present (e.g., cracks, dents, chips, pixel damage, etc.). The server's neural network can perform the analysis of one or more parts of an image, adjusted based on pattern and / or identifier techniques learned from previous device screen images and / or a set of known screen images (e.g., those known to have or do not have screen damage). In some embodiments, allowing the server's neural network to analyze parts allows for easy upgrade and maintenance of the neural network and can improve accuracy (e.g., by having analyzed previous device screen images from multiple devices).

[0045] Whether a device screen is damaged can be determined based on whether a portion is damaged and / or whether an adjacent portion is damaged (operation 460). If a portion is determined to be damaged (e.g., a binary determination such as yes or no damage, where the probability of damage exceeds a predetermined probability, such as a 50% probability of damage), the server can identify the adjacent portion(s). One or more adjacent portions can be determined to be damaged and can be used by the server (e.g., a neural network) to determine whether the screen should be identified as damaged. For example, if one portion has a 20% probability of damage and four adjacent portions have a 50% probability of damage, the screen can be determined to be damaged. As another example, if one portion has a 50% probability of damage and none of the adjacent portions have a probability of damage exceeding 25%, the screen can be determined (e.g., by the server) to be undamaged. In some embodiments, the overall probability of damage to the device screen is determined based on the characteristics of the damage (e.g., location, size, and / or shape) and the probability of damage in adjacent portions. The probability may not decrease proportionally. In some embodiments, as the accuracy of the neural network improves, a predetermined range of probabilities associated with a damaged screen(s) may decrease. For example, the system may be able to determine that screen damage is present if the probability of screen damage is greater than 70% in a portion, and as the accuracy of the neural network improves, the system may be able to determine that screen damage is present if the probability of screen damage is greater than 50% in a portion.

[0046] Notifications(s) can be sent based on a determination of whether or not the device screen is damaged (operation 470). For example, if it is determined that the screen is damaged or not, the user may receive a notification based on this determination. In some embodiments, if it is determined that the device screen is damaged, the user may be allowed to contest the determination by restarting one or more operations of the process (e.g., repositioning the device and / or retaking the image). In some embodiments, a notification of whether or not the device screen is damaged, and / or a price based on the condition of the device screen, may be sent to the application to be presented to the user via the return application.

[0047] Notifications may be sent based on the determination that the neural network cannot accurately assess the state of the device screen. For example, during image processing, the server may identify that the image does not contain a full screen. This determination can be made by calculating the predicted screen aspect ratio, the predicted screen angle, the model's predicted dimensions, etc. The server may instruct the user via the return application to capture another image. The server may instruct the user via the return application to position the device in real time in the best possible way.

[0048] Process 400 can be implemented by various systems, such as System 100. Furthermore, various operations may be added, removed, and / or modified. In some embodiments, Process 400 may be executed in combination with other processes, such as Process 200. For example, the status of a device may be determined for the resale of the device. Since the resale price of a device and / or the demand for a device in the resale market can be obtained based at least in part on the status of the device, automatically determining the status of the device and / or its components can facilitate the determination of a price to offer for a device (e.g., one that will be resold). In some embodiments, determining the status of a device can facilitate the determination of whether a device can be reused (e.g., by another user). The status can be determined so that other users can determine the price to offer for the device, whether the device is reusable, whether the device should be used as is, and / or whether the device should be repaired. In some embodiments, the status of a device may be determined for use in a device insurance contract. For example, if a user wishes to obtain device insurance, a return application can be used to determine the status of the device and / or its components. The device's status and / or history of its status (e.g., multiple screen cracks that have been repaired) can be used to determine whether a device insurance contract should be offered, what price to set for the device insurance contract, and / or to verify the status of the device provided by the user. In some embodiments, the user may wish to submit an insurance claim, and the status of the device or its components can be determined by the return application to be submitted with the insurance claim and / or to verify portions of the insurance claim.

[0049] In some embodiments, instead of the first graphic and / or identification code, and / or instead, the IMEI and / or other device and / or operator Device identification can be facilitated by obtaining and / or utilizing rating system-specific codes. For example, the IMEI and / or other codes may be associated with a screen image captured via a second graphic to identify a particular device. The user may be directed to a device settings page where the IMEI is displayed (e.g., via a prompt on the graphical user interface of the return application), and / or the user may be prompted to dial a code on an auto-dial device to display the IMEI. The user may capture a screenshot of the IMEI (e.g., via the camera of the device and / or a second device used to capture the image). The return application may process the screenshot to identify the IMEI (e.g., via OCR). In some embodiments, the user may install a profile from a server configuration similar to that of an MDM server. The profile can provide the IMEI to the server. The server can pass the IMEI to the return application. In some embodiments, the profile can reduce the number of steps the user has to perform to provide the IMEI to the return application. In some embodiments, one or more of these functions may be performed automatically by the return application. The obtained IMEI may be used by the return application to tag the captured image (e.g., the second graphic) and / or to guarantee its authenticity (e.g., via the obtained IMEI).

[0050] In some embodiments, the state of the first device may be determined using a second device. For example, the screen of the first device may be damaged, thereby preventing the user from using the device or making them unwilling to use it (e.g., a finger could be injured by a crack in the screen, a chip could come loose from the screen, or the screen could be further damaged by use). Therefore, the second device can be used to capture an image of the first device. The first and second devices may include a return application (e.g., one or more operations of the return application may be performed by the processors of the first and second devices). A request may be received to evaluate the state of the screen or part thereof of the first device via the return application on the second device. The return applications on the first and second devices may communicate with each other (e.g., directly and / or indirectly via the return application on a server). For example, the return application on the second device may communicate with the return application on the first device to enable graphics to be presented on the screen of the first device via the return application. A first graphic can be displayed on the screen of the first device via a return application on the first device, and the graphic may include a first identification code. At least a portion of the first graphic displayed on the first device may be captured via the camera of the second device. The return application may have access to the camera function of the device, and therefore the return application may be able to enable image capture. One or more second graphics can be displayed on the screen of the first device (for example, via a return application on the first device), and one or more portions of the second graphics displayed on the first device may be captured via the camera of the second device.One or more of the second images can be preprocessed and / or processed to determine the screen state of the first device. The first graphic image may be used to decode the identification code of the first graphic (for example, to verify the identity of the first device). In some embodiments, the identification code can be used by a return application to verify the identity of the first device and / or the image captured from the screen of the first device. It may include a code unique to the first device, such as an IMEI number.

[0051] In some embodiments, the first graphic may or may not be presented and / or captured when the second device is used to determine the state of the first device. The level of security and / or authentication provided by the return application that captures images on the device to be evaluated by the return application may not be as strong as when the second device is used to capture images from the first device being analyzed. Therefore, in some embodiments, insurance claims, assessments, acceptances, and / or refunds may be adjusted to account for the increased risk in fraudulent user behavior.

[0052] In some embodiments, processing a second image may include dividing the second image into parts, determining whether one or more of the parts of the second image contain damage, identifying parts adjacent to one or more of the damaged parts, and determining the state of the screen of the first device based on whether one or more of the parts of the second image have been determined to contain damage, and whether one or more of the parts adjacent to one of the parts determined to contain damage also contain damage. The screen and / or part thereof (active region) may or may not be identified from the second image before dividing the image into parts. For example, in some embodiments, a neural network may be trained to identify damage to the image even if the active region is not separated.

[0053] In some embodiments, one or more of the above operations can be performed by one or more additional images. In some embodiments, the determination of whether the device screen is damaged may also be made based on an analysis of the image and additional images. In some embodiments, the return application may process the image, or at least partially process it, before sending the image to the server. In some embodiments, the image may not need to be split before analysis. In some embodiments, the image may not need to be processed before the screen condition is analyzed. In some embodiments, the determination of whether the device screen is damaged can be made based on a determination of whether a portion shows damage, even if adjacent portions do not show damage. For example, if a portion shows a 100% probability of damage and / or deep crack, it can be determined that the device screen is damaged. In some embodiments, the return application captures, receives, and / or stores the image (e.g., on behalf of and / or in addition to the server). In some embodiments, the return application may perform (e.g., on behalf of or in conjunction with the server) or more operations.

[0054] In some embodiments, multiple images can be used as a second graphic. For example, a return application may present and / or capture a set of second images. The set of second images may differ in coloring. The set of second images may be captured by a capture burst (e.g., automatically capturing multiple images in a short period of time, as opposed to manually capturing multiple images). The capture burst may use the same or different capture settings (e.g., flash, exposure, focal length, etc.). A return application (e.g., via a neural network on the device and / or on a server) may identify the captured second image for processing (e.g., for determining screen health) by comparing one or more captured images with a predetermined reference color and / or images containing the predetermined reference color. Images of low image quality, such as colored and / or blurred images, may or may not be identified as the captured second image in some embodiments.

[0055] In some embodiments, the image is captured using one or more other auto-adjustments. This is possible. For example, a capture burst can change brightness, color, orientation, focal length, etc., to obtain better consistency in the captured images (for example, to more accurately determine health, as consistency can facilitate analysis by neural networks). For example, the brightness of a second image can be changed during a capture burst to provide a set of captured images. Images in the set can be compared to a reference color to identify the captured second image for further processing (for example, to determine the state of the screen). In some embodiments, the return application can prompt the user to change the orientation of the device to obtain a set of captured images (for example, to enable capturing images at different angles, as multiple images can be taken to change the viewpoint of a crack that may not be visible from one angle).

[0056] In some embodiments, the visibility of cracks to the neural network can be enhanced by obtaining multiple images captured in the same session with different screen settings (for example, images may highlight different types and / or locations of screen cracks). The return application may utilize a second graphic of a different color to facilitate the identification of the damaged screen. In some embodiments, iii. Multiple tilt angles by the UI to guide proper positioning - In some embodiments, a set of captured images can be compared to a reference image (e.g., color, intensity) to select which images to process further to determine the state of the device. For example, a return application can select images that are closest to the brightness / color / intensity of the images used for training to improve accuracy results. In some embodiments, the return application's behavior is used to carry over color to areas of the image known to be screens. The color is matched to a reference color known to best display the visibility of cracks while minimizing other defects such as reflections and bad color. Images with colors closest to the reference color are sent to a neural network for processing. Improved image consistency can make the analysis less dependent on lighting conditions (e.g., performed by the neural network in the return application).

[0057] In some embodiments, burst capture of images can facilitate the capture of one or more images that do not have the same brightness levels (e.g., have the same or different graphics). Then, one or more colors in the images can be matched against a reference color to identify the captured image that is closest to the reference color (e.g., from the set of burst-captured images). Utilizing burst capture with different brightness levels allows for greater consistency of captured images sent to a neural network for the image capture process and / or analysis, and / or reduces variability. This, in some embodiments, can reduce the error in the analysis.

[0058] In some embodiments, the first graphic capture can provide an initial exposure setting(s). For example, the image of the first graphic can be acquired and analyzed to identify an initial exposure setting (e.g., if the image is too bright or blurry). By generating an initial exposure setting, image capture can be improved.

[0059] In various embodiments, the state of one or more screens of a device (e.g., an electronic device such as a mobile device or laptop) can be identified. The state of the device can be determined using the operation of the device and / or the operation of other devices. For example, if the first device lacks a camera and / or a working camera, the second device can be used to capture an image of the first device or a part of it (e.g., the screen, front, etc.). Another example is when a component of the first device renders the first device at least partially inoperable (e.g., a cracked screen, a touchscreen, etc., which could injure the user and / or further damage the device). If it doesn't work (e.g., stacked pixels prevent use), a second device can be used to capture an image from the first device or a part of it (e.g., screen, front, back, etc.).

[0060] In some embodiments, an image of the first device can be captured by positioning the first device so that an image of the first device can be captured on a reflective surface such as a mirror. By using the camera of the device itself to capture an image (for example, so that an image of a similar device is not presented instead), the risk associated with fraudulent activity can be reduced. Requests for evaluation of the state of a component of the first device, such as a screen, or a part thereof, can be received via a return application on the first device. The return application may reside on the first device and / or be accessible by the first device (for example, it may be stored remotely). The return application may present one or more graphical user interfaces to facilitate communication with the user and / or to present graphics on the device's screen.

[0061] The return application may present one or more graphics on the screen of the first device (e.g., via a graphical user interface generated by the return application) for capture by the camera of the first device. For example, the first graphic may be generated and / or presented on the screen of the first device (e.g., a display component) (e.g., by the return application). The first graphic may include one or more first identification codes, such as an IMEI associated with the device, a code number generated by the return application and associated with the device, or a QR code®. The identification codes may be analyzed to verify the identity of the first device (e.g., decoded, compared to a list of codes, etc.). At least a portion of the first image of the first graphic may be captured by the camera of the first device. The first image may include a reflection of the first graphic on a reflective surface such as a mirror. The first graphic and / or the identification code of the first graphic may be tagged or otherwise embedded in other captured images (e.g., a second image containing a second graphic). The first graphic may be used by the return application to determine initial settings (e.g., brightness, contrast, orientation, distance to reflective surfaces, etc.) used for the presentation and / or capture of subsequent graphics. In some embodiments, the first graphic may not be used (e.g., generated and / or captured).

[0062] To facilitate the identification of damage to the device or a part thereof (e.g., the screen), the return application may generate and / or present other images. One or more second graphics may be generated and / or presented on the screen of the first device (e.g., via the graphical user interface of the return application). The second graphics may include graphics configured to facilitate the identification of damage (e.g., cracks, dents, chips, etc.) by the return application (e.g., using a trained neural network). For example, the second graphics may include a single color, a change of color, a pattern(s), an image, etc. In some embodiments, the second graphics may include a set of graphics in which the image displayed within the graphics changes and / or the settings used to present the second graphics on the screen change. For example, the brightness of the presented image may be changed to present the second graphics differently for capture. In some embodiments, a single second graphic may be generated and / or presented (e.g., a single-color green graphic). At least one of the second images of the second graphics may be generated and / or presented on the screen of the first device (e.g., the first device The images may be captured (via the camera of the first device). One or more of the second images may include a reflection of at least one of the second graphics on a mirror-like reflective surface. The captured images may include more than just the screen (e.g., the front, areas close to the device). The return application may allow control of the camera components of the first device and / or allow the user to control said camera components. The return application may have access to the images captured by the camera of the first device.

[0063] In some embodiments, the captured images (e.g., the first and / or second captured images) may be preprocessed. Preprocessing may be performed by a return application on the user device and / or on a server (e.g., using a trained neural network). Preprocessing can identify low-quality images, for example, by identifying parts of the captured image that are not related to the presented image and / or screen damage (e.g., obstacles, flash reflections, etc.). Preprocessing can identify partial and / or blurred images. In some embodiments, the return application may reject the image and / or request recapture of the image based on the return application's determination in preprocessing that the captured image is of poor quality. Upon recapture of the image, the return application can regenerate and / or present the graphics on the screen of the first device. The return application may modify the graphics, device settings, and / or prompt the user for adjustments in recapture (e.g., limiting flash, adjusting orientation, etc.). In some embodiments, low-quality images can be processed to identify the state of the device's components.

[0064] One or more of the second images can be processed to determine the state of a component such as the screen of the first device. Processing the second image(s) may include dividing the second image into parts and determining whether one or more of the parts of the second image contain damage. The second image may be divided into parts to allow for faster processing (for example, compared to the entire image processing) and to improve accuracy (for example, by allowing analysis of adjacent areas when determining the probability of damage). In some embodiments, parts adjacent to one or more parts containing damage can be identified as adjacent parts. Adjacent parts may or may not contain damage.

[0065] The state of the screen of the first device can be determined based on whether one or more of one or more parts of the second image are determined to contain damage, and whether one or more parts adjacent to one of the parts determined to contain damage (e.g., parts adjacent to a particular part) also contain damage. For example, a neural network may be trained to identify common damage patterns and may use information about adjacent parts (e.g., whether adjacent parts are damaged) to determine whether a part is damaged.

[0066] In some embodiments, the determination of whether a component of the first device, such as the screen, is damaged may include additional damage information such as a rating (e.g., severity rating, damage type rating, etc.) and the location of the damage. The additional damage information and / or the determination of whether a component of the device is damaged may be presented to the user and / or used in other operations of the return application (e.g., reduction of evaluation for insurance claims, warranty claims, etc.).

[0067] The processes described can be implemented by various systems, such as System 100. Furthermore, various operations may be added, removed, and / or modified. In several embodiments, the process or part thereof may be performed in combination with operations from other processes, such as process 200 and / or 400. For example, the capture of a second image may include a burst capture of the second image. When a burst capture is permitted, device settings may be changed. The captured images can be compared to a reference image to identify which set of captured images should be processed. For example, an image with a color closest to the reference color may be selected (e.g., brightness and / or contrast can be adjusted on the device to obtain different colors in the second image when a burst of image capture occurs). The set of captured images may be preprocessed to identify which images can be processed. As another example, processing the captured second image may include generating a third image from which multiple parts of the image not associated with the device screen have been removed (e.g., cropped). This third image may be analyzed by at least part of a neural network to identify whether the screen is damaged (e.g., it may be split into parts and / or analyzed). In some embodiments, low-quality images can be processed to identify the state of the device components. For example, blurred images may be processed, and the neural network can take the image blur into account in its analysis (for example, by reducing the sensitivity of damage detection to avoid over-identifying damage).

[0068] In some embodiments, one or more operations may be performed by a second device to obtain the state of a first device. For example, the second device may include a camera for capturing an image presented to the first device by a return application. The return application on the second device may enable image capture and / or coordinate the presentation of the image on the first device, processing of the captured image (e.g., preprocessing and / or processing), and / or identification of the state of the first device. In some embodiments, the increased variation in fraudulent activity associated with capturing an image using a device different from the device whose state is being determined may be taken into account in insurance underwriting, security measures (e.g., physical inspection upon receipt of the device for trade-in, sale, and / or return), and / or discounts (e.g., reduction of determined value and / or sale price).

[0069] In some embodiments, image acquisition may be at least partially automated. An image can be obtained when the image meets the initial exposure settings. For example, the user may move the phone, and when the phone is in the optimal position (e.g., when the initial exposure settings are met), an image may be automatically acquired. The initial exposure settings may include criteria regarding the camera's position relative to the screen and / or mirror, tilt angle, flash settings, brightness settings, etc. In some embodiments, the phone screen brightness of the initial exposure settings can be calibrated before positioning the phone identification code. In some embodiments, the brightness may be adjusted during the calibration period by different brightness and exposure settings. The brightness of a predetermined visibility of an identification code, such as a QR code (registered trademark), may be selected as a reference brightness in the current lighting conditions. In some embodiments, this reference brightness may be used as the median of multiple image acquisitions with different brightness levels.

[0070] In some embodiments, the capture image may be processed by the neural network at full resolution or a lower resolution. The image sent to different levels may or may not be changed. For example, the capture image may be passed through all levels of the neural network at full resolution and remain as a PNG file. In some embodiments, once the image is sent to the first level of the neural network, the image may be reduced (e.g., to 128x256). The downsampled image may be sent to one or more layers of the neural network as an array of color intensities. For example, Byte(red) Byte(green) Byte(blue), Byte(red), Byte e(green), Byte(blue) are two pixels, (0,0) and (0,1). In some embodiments, the first layer of the neural network may be pre-processing (e.g., returning results that the image quality is poor and unprocessable, and / or that the image is processable). In some embodiments, when the captured image is sent to the final layer, the captured image may be sampled by patches and slides (e.g., there may be 32 patches, so that the tiles are 32x32, and there may be 17 slides, so that the network takes a tile from 1,1, then the next tile from 1,17, and / or other suitable patches and / or slides). There may or may not be overlaps for the tiles sent to the inner layers of the neural network. The samples (e.g., 32x32 tiles) may be sent to the final neural network layer as an array of RGB byte values ​​representing color intensity. In some embodiments, this can be done in the longitudinal and / or widthwise directions. The neural network can be started at any suitable point in the captured image. In some embodiments, the screen is centered on the image and includes a background where edges can be ignored, so starting near the center may be speed-advantageous in processing.

[0071] In some embodiments, preprocessing can identify whether the captured image is of poor quality. In some embodiments, preprocessing may be performed by a neural network (e.g., a first layer of the neural network). For example, the neural network can identify poor images such as grainy images, blurry images, poor contrast (e.g., dark), poor color (e.g., brightness), mismatched images (e.g., no phone when a phone is expected), obstacles such as fingers or phone cases, and partial screens.

[0072] In some embodiments, a return application (e.g., a neural network for the return application) can identify the state of a device or part thereof as damaged or undamaged. The return application can identify the type of damage, the extent of the damage, and so on. For example, a neural network can be trained to identify the type and / or extent of damage. The neural network can rate the severity of the damage. For example, the output of the neural network can provide details about the state, such as crack, dent, stuck pixel, minor defect, or good. In some embodiments, the state and / or the output of the neural network may be provided in an output format such as a simple integer 0 to x, binary 000001, and / or percentages that sum to 1.0.

[0073] In some embodiments, the neural network may have a zero level (e.g., before processing). In some embodiments, processing can be accelerated and / or accuracy improved by utilizing a neural network having two or more levels (e.g., including a final processing layer). The return application can be customized based on desired parameters. For example, identifying blurred images may be easier and / or faster than determining obstacles, and therefore a return application that preprocesses only for identifying blurred images may have fewer layers than a return application that preprocesses for obstacles and / or other low image quality. In some embodiments, the guidance provided by the return application may enable better image capture, and a single-layer (e.g., final layer) neural network may be utilized to identify defects.

[0074] In some embodiments, the return application can process poor quality captured images. For example, instead of rejecting images based on color and / or blur. The rating and / or color intensity of a portion of an image can be processed by a return application. The return application may or may not prohibit the processing of other portions. For example, the rating may be a value, color, and / or other metric, such that a rating of 0 indicates that the image is not blurry, and a rating of 255 indicates that the image is very blurry. The rating scale may be linear or nonlinear. The return application (e.g., a neural network) may be adjusted based on the rating (e.g., increasing and / or decreasing sensitivity). For example, the return application may decrease sensitivity / intensity when identifying cracks in a captured image with a rating of 255. Thus, various defective images can be processed based on low-quality images, and / or processed with near accuracy.

[0075] In some embodiments, other parts of the device may be captured and / or processed by the return application as described. For example, the return application can facilitate the evaluation of cracks on the mobile phone casing (e.g., the front and / or back).

[0076] In some embodiments, the determination of whether a device screen is damaged may involve determining the extent of the damage, whether the damage relates to one or more damage categories (e.g., complete, cracked, scratched, reflective), the probability of the damage being present, and / or other appropriate damage classifications. For example, the determination of whether a device screen is damaged may involve determining whether the screen is cracked or not. In some embodiments, the determination may result in a value between, for example, -1 and 1, where a value less than 0 is associated with a device screen without cracks, and a value greater than 0 is associated with a device screen with cracks. For example, in some embodiments, the value may be associated with the probability that the device screen is damaged, and the device screen may be identified as damaged or cracked if the probability is greater than a predetermined value. For example, if the certainty that the device screen is cracked is greater than 90% (e.g., >.80), the device and / or device screen can be identified as cracked. In some embodiments, the analysis may be performed on each part, a set of adjacent parts, all device parts, and / or the entire device.

[0077] In some embodiments, the location and extent of damage to the device screen (e.g., how deep, in which layer(s) it is located), and / or whether the damage affects other components can be determined (e.g., by the system's neural network). For example, since damage may be detected in separate parts, in some embodiments, the approximate location of the damage can be determined and / or transmitted to the user. In some embodiments, the location of the damage may adjust the base price presented for the device. For example, a small crack in the screen that is not in the active area of ​​the device screen may result in a smaller price reduction than a crack that is in the active area of ​​the device screen. In some embodiments, the user may be notified of the location of the screen damage. This information can help the user determine whether the identified screen damage is actually screen damage or a mark on the device that can be removed. In some embodiments, the extent of the damage may be determined to facilitate the identification of whether the screen is repairable or whether replacement is more appropriate (e.g., when repairing the device).

[0078] In some embodiments, the location of the damaged portion of the device screen can be flagged (e.g., by a different color, pattern, flag, and / or other appropriate mark). Figure 8A shows an exemplary embodiment of a captured image of a device screen. As illustrated, the device screen includes a visible crack in the captured image. The captured image is divided into several parts and analyzed to determine whether one or more of the parts have a probability of damage higher than a predetermined probability. In some embodiments, adjacent parts may contain damage. Whether or not a flag is present can determine whether the device screen and / or a part thereof is damaged. The system can use flags to identify parts that have a higher probability of being damaged than a predetermined probability. Figure 8B shows an embodiment of an interface generated as a result of determining whether the device screen shown in Figure 8A is damaged. The interface may be generated and presented to the user (e.g., via a return application and / or a website coupled to a server). As illustrated, darker (e.g., red) flags indicate parts identified as damaged. The user can visually inspect the flags and contest any flags that the user believes are not on a damaged part of the device screen, and / or verify the damaged part. Figure 8C shows an embodiment of an exemplary capture image of the device screen. As illustrated, the device screen is not damaged. Therefore, no flags are generated when the image of the device screen is analyzed. Figure 8D shows an embodiment of an interface generated as a result of determining whether the device screen shown in Figure 8C is damaged. As illustrated, the automatic determination of the device screen state does not detect any damage and therefore no part of the device screen is flagged as damaged. As another example, Figure 8E shows an embodiment of an exemplary capture image of a device screen. As illustrated, the device screen includes damage. Figure 8F shows an embodiment of an interface generated as a result of determining whether the device screen shown in Figure 8E is damaged or not. As a result of determining the state of the device screen, the value app can flag the damaged portion of the device screen. A darker flag (e.g., a red flag) indicates the portion of the device that the system has labeled as damaged.

[0079] In various embodiments, the condition of the device can be determined and / or the device can be further tested (e.g., according to commonly used techniques and / or the techniques described) by determining the condition of the device screen (e.g., whether the device screen is damaged or not). For example, if there is a crack in the screen adjacent to a component of the device, and / or if there is a crack that is larger than a predetermined maximum size (e.g., depth and / or width), further tests can be performed to determine whether one or more other components (e.g., microphone, speaker, touchscreen layer, and / or case) are damaged. For example, the operation of components can be tested (e.g., automatically, semi-automatically, and / or manually). The price can be determined using the condition of the device (e.g., components including the screen), market data, current resale price, and / or other information. The price can be sent to the user and / or displayed via a return application. In some embodiments, the user can sell the device based on the price presented in the return application.

[0080] In some embodiments, a touchscreen test can be performed if the device screen is determined to be damaged. The touchscreen test can be performed via a return application. For example, the return application can prompt the user to provide input based on instructions from the return application, and a determination can be made regarding the state of the touchscreen (e.g., whether it is damaged or not, the location and / or extent of the damage) based on the input provided by the user. The results of the touchscreen test can be used to determine the depth of the damage to the device screen and / or damage to one or more other components of the device.

[0081] In some embodiments, the device grade is determined at least in part by whether the device screen is damaged, the location of the damage on the screen if present, the extent of the damage to the screen if present, whether one or more other components of the device are damaged, the resale price, the recyclable / scrap value, and / or other appropriate criteria. The grading may be related to the price at which the user offers the device for sale and / or the price at which the device is resold (e.g., in the market, to a third party).

[0082] In some embodiments, the assessment of damage to a device screen can be made more detailed and / or graded across more possible levels, as subjectivity can be reduced (e.g., damage can be determined automatically) and / or consistency can be increased (e.g., location and / or size). This allows for a more consistent and faster delivery of smaller differences between device conditions. For example, screen damage that does not overlap with the active area of ​​the screen may be graded as a damaged area of ​​the screen, but may receive a higher grading than a screen with damage in the active area.

[0083] In some embodiments, the image(s) may be stored in the device's memory and / or memory coupled to the device (e.g., cloud storage and / or server memory). The return application can manage the uploading of images to the server based on the device's network connectivity (e.g., LTE, Wi-Fi, etc.).

[0084] In some embodiments, screen damage can be verified by a human (e.g., a quality control operator), and this feedback can be provided to a neural network to improve accuracy and / or to adjust the analysis provided by the server's neural network.

[0085] While screen condition is described in terms of damage due to cracks, other damage such as pixel damage (e.g., breakage, sticking, etc.) and dents can also be determined via a return application. For example, the application may cause one or more graphics to be displayed on the device screen, and an image of the graphics on the device screen may be captured (e.g., via a camera on the device). For example, the graphics may include a single color presented on the screen, a graphic with various colors, a graphic with a pattern(s), and / or a graphic designed to facilitate the identification of screen damage. The colors presented in the image may be analyzed (e.g., by a neural network on a server) to determine whether one or more pixels are not presenting the color accurately. In some embodiments, k-means can be used to recognize nearly identical color features within the image. Thus, pixel damage can be identified at least in part based on the analysis of the captured image(s).

[0086] The embodiments may include commercially available neural networks, but in some embodiments, the neural network may be a customized neural network capable of learning patterns, recognizing cracks, assigning probabilities of damage present on the device, and / or having other suitable functions. In some embodiments, the neural network may include a cloud-based system accessible by the return application. In some embodiments, the neural network may be stored and operational on the device.

[0087] In some embodiments, the neural network is used by the neural network. The neural network can be trained using a learning tool that enables it to learn how to identify the state of the chair screen. Figures 9A, 10A, and 11A show embodiments of the learning tool, and Figures 9B, 10B, and 11B show relevant second images. Figures 9C, 10C, and 11C show examples of processing the second image by the neural network, and Figures 12A–12B show examples of accuracy and cross-entropy achieved by the neural network. The neural network may include zero or more layers. In some embodiments, the neural network may be multilayered to facilitate processing and identification of the state of the device screen from the capture image. The neural network can be trained by providing exemplary images, such as exemplary capture images shown in Figures 9B–11B and corresponding exemplary images in which damage is identified, as shown in Figures 9A–11A. Damage is identified (e.g., by different color and / or pattern), as shown in Figures 9B–11B. The neural network can process exemplary capture images such as images 9B–11B according to one or more of the operations described. For example, the captured image may be divided into multiple parts, and a neural network can determine which part contains damage. This result can be compared with a training tool, and as shown in Figures 12A and 12B, the neural network can learn and become more accurate.

[0088] While mirrors have been described as providing a reflective surface that reflects the image presented on the device's screen, any reflective surface can be used instead of, and / or in conjunction with, a mirror. For example, a metal reflective piece and / or a plastic reflective piece can be used to capture the image on the device screen.

[0089] Although the user is described as a human being, the user may be a single person, a group of people, one or more people interacting with one or more computers, and / or a computer system (e.g., a device, a robot).

[0090] Devices are described in various embodiments. A device may be any suitable device, such as a smartphone, tablet, laptop, game console, portable media player (e.g., e-reader and / or video player), wearable (e.g., watch, jewelry, etc.), or video camera capable of running applications and / or taking pictures of the device. A device may include memory, a processor, and a camera (e.g., a component capable of capturing images). A device may store a return application in memory, and the processor may execute the return application to perform one or more of the operations described. In some embodiments, a device may perform one or more of the operations described to be performed by a server, either on behalf of or in conjunction with a server.

[0091] Servers are described in various embodiments. Server 110 may include memory and a processor that executes instructions and manipulates data to perform the server's operations. In some embodiments, the server may be cloud-based and / or support cloud-based processing and storage. As described, the neural network may be stored in the server's memory, and the processor may perform the functions of the neural network. The memory may include a data repository (e.g., a database). The data may include data for teaching and / or setting up the neural network (e.g., a set of images, feedback on accurately and / or incorrectly identified damage, patterns, etc.), resale prices, prices for presenting about the device, screen repair and / or replacement costs, location of supplementary image information, market information, reuse information, insurance information, information for verifying device identification, and / or other appropriate information. It can include...

[0092] Furthermore, various software may be stored in the server's memory. For example, the software may be able to communicate with the device, perform one or more actions to determine the state of the device screen, perform tests on one or more components of the device, etc. In various embodiments, one or more of the captured images may be stored in the device or server's memory and / or transmitted (for example, from the user device to the server and / or vice versa).

[0093] Software and / or return applications on the server may include a graphical interface to facilitate interaction with the user. A communication interface may enable the server to communicate with other repositories and / or devices over a network. The communication interface may send data to and from the server, and / or receive data from devices and / or linked repositories and / or other computer systems, via network protocols (e.g., TCP / IP, Bluetooth®, and / or Wi-Fi) and / or buses (e.g., serial, parallel, USB, and / or FireWire).

[0094] The graphical user interface (GUI) of a server and / or return application can be displayed on a presentation interface, such as a device screen. The GUI may be operable to allow the device user to interact with the repository and / or server. Generally, the GUI provides the device user with an efficient and user-friendly presentation of data provided by the server and / or return application. The GUI may include multiple displays having interactive fields, pull-down lists, and buttons that are manipulated by the user. As an example, the GUI presents an exploratory interface and receives commands from the user. It should be understood that the term graphical user interface may be used in the singular or plural form to describe one or more graphical user interfaces within each of a particular graphical user interface display. Furthermore, the GUI envisions any graphical user interface, such as a common web browser, that processes information within the server and / or device and presents that information efficiently to the user. In some embodiments, the GUI may present a web page embedding content from the return application and / or server. The server may receive data from the device via a web browser (e.g., Microsoft Internet Explorer or Safari) and return a response in the appropriate hypertext markup language (HTML) or extensible markup language (XML).

[0095] Figure 1 provides an example of a server that can be used in this disclosure, but the server can be implemented using computers other than the server and server pools. For example, the server can include a general-purpose personal computer (PC), Macintosh®, workstation, UNIX®-based computer, server computer, or other suitable device. According to one embodiment, the server can include a web server and / or a cloud-based server. The server may be adapted to run any operating system, including UNIX®, Linux®, Windows®, or other suitable operating systems. In short, the server can include any combination of software and / or hardware suitable for providing access to data and / or converting data into a suitable compatible format.

[0096] While the embodiments describe a single processor within a server and / or device, multiple processors may be used depending on specific needs, and references to processors are intended to include multiple processors where appropriate. Processors may include programmable logic devices, microprocessors, or other suitable devices for logically manipulating information.

[0097] While the embodiments discuss the use of neural networks to perform at least part of the analysis of return applications, analytical frameworks implemented by other computing devices may be used as appropriate. Although the embodiments describe the neural network as being contained in a server(s) (e.g., a physical server and / or a virtual server), the neural network may reside on other devices. For example, the neural network may run and / or at least partially run on user devices such as a first device and / or a second device, such as a mobile device. In some embodiments, the neural network may be cloud-based and may be accessed by the server and / or user devices (e.g., a first device and / or a second device).

[0098] The embodiment describes a single memory in a server and / or device, but multiple memories may be used as appropriate. For example, the memory may include an SQL database, a relational database, an object-oriented database, a distributed database, an XML database, cloud-based memory, device memory, and / or a web server repository. Furthermore, the memory may include one or more forms of memory, such as read-only memory (ROM), optical memory (e.g., CD, DVD, or LD), magnetic memory (e.g., hard disk drive, floppy disk drive), NAND flash memory, NOR flash memory, electrically erasable programmable read-only memory (EEPROM), ferroelectric random access memory (FeRAM), magnetoresistive random access memory (MRAM), non-volatile random access memory (NVRAM), non-volatile static random access memory (nvSRAM), and / or volatile memory (e.g., RAM) or non-volatile memory such as phase-change memory (PRAM).

[0099] It should be understood that the embodiments are not limited to the specific systems or processes described and can, of course, be modified. It should also be understood that the terms used herein are merely descriptive and not limiting to specific embodiments. Where used herein, the singular forms “a,” “an,” and “the” refer to multiple objects unless the content clearly indicates otherwise. For example, a reference to “one image” includes a combination of two or more images, and a reference to “one graphic” includes a combination of multiple different types and / or graphics.

[0100] While this disclosure has been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the spirit and scope of this disclosure as defined by the appended claims. Furthermore, the scope of this application is not intended to be limited to specific embodiments of the processes, machines, manufactures, compositions, means, methods, and steps described herein. As will be readily apparent to those skilled in the art from this disclosure, existing or future-developed processes, machines, manufactures, compositions, means, methods, or steps that perform substantially the same function or achieve substantially the same results as the corresponding embodiments described herein can be utilized in accordance with this disclosure. Accordingly, the appended claims are intended to include such processes, machines, manufactures, compositions, means, methods, or steps within their scope.

Claims

1. A method for identifying the state of one or more screens of an electronic device, Receiving a request to evaluate the state of the screen of the first device or a part of the screen via a return application on the first device, The method includes enabling the display of a first graphic on the screen of the first device, wherein the first graphic includes a first identification code, and the method is The method includes capturing at least a portion of the first image of the first graphic with the camera of the first device, wherein the first image includes a reflection of the first graphic on a reflective surface, and the method is To enable the display of one or more second graphics on the screen of the first device, The method includes capturing one or at least a portion of one of the second images of the second graphics via the camera of the first device, each of the second images including a reflection of at least one of the second graphics on the reflective surface, The process includes processing one or more of the second images in order to determine the state of the screen of the first device, and processing one or more of the second images is Dividing the aforementioned second image into parts, Determining whether one or more of the parts of the second image are damaged, Identifying parts adjacent to one or more parts containing damage, A method comprising determining the state of the screen of the first device based on whether one or more of the one or more portions of the second image are determined to be damaged, and whether one or more of the portions adjacent to one of the portions determined to be damaged are also damaged.

2. The method according to claim 1, further comprising verifying the identity of a first device based on an analysis of the first identification code.

3. Capturing one or more of at least a portion of at least one second image among the second graphics is: The orientation of the device is determined based on the capture image of the first graphic, To provide guidance for adjusting the orientation of the device based on the determined orientation, The method according to claim 1, including the method described in claim 1.

4. The method according to claim 3, further comprising capturing at least a portion of an additional image of the first graphic via the camera of the first device.

5. The orientation of the device is determined based on the capture image of the first graphic, To provide guidance for adjusting the orientation of the device based on the determined orientation, Capture one or at least a portion of at least one additional image of the second graphic via the camera of the first device, wherein each of the additional images includes a reflection of at least one of the second graphic on the reflective surface. The method according to claim 3, further comprising:

6. Determining that at least one of the captured first image or the captured second image is not a processable image, The following further includes enabling the first device to reorient itself to capture a processable image: Performing the above and subsequent actions is: The orientation of the device is determined based on the captured first image or the one or more captured second images. To provide guidance for adjusting the orientation of the device based on the determined orientation. The method according to claim 3, including the method described in claim 3.

7. The method according to claim 1, further comprising tagging one or more of the captured second images with at least a portion of the captured first image.

8. The method according to claim 1, further comprising restricting one or more processes of the first device.

9. The method according to claim 1, wherein identifying the screen of the first device or a portion of the screen in the second image includes identifying the screen of the first device in the second image by utilizing at least one of corner detection or edge detection.

10. Processing one of the second images mentioned above is Identifying the screen or a portion of the screen of the first device within the second image, The process includes generating a third image, wherein any portion of the second image that is not identified as a screen or part of the screen within the second image is constrained not to be included in the third image. Dividing the aforementioned third image into parts, Determining whether one or more of the parts of the third image contain damage, Identifying a portion adjacent to one or more of the portions including the damage, Includes, The method according to claim 1, wherein determining the state of the screen of the first device is based on whether one or more of the one or more portions of the third image are determined to be damaged, and whether one or more of the adjacent portions are damaged.

11. The method according to claim 10, wherein generating a third image includes modifying the second image such that any portion of the second image that is not identified as part of the screen is removed.

12. The method according to claim 10, wherein identifying a screen or a portion of the screen includes identifying an active area of ​​the screen of the first device.

13. A method for identifying the state of one or more screens of an electronic device, The first device includes receiving a request via a return application on a second device to evaluate the state of the screen or part of the screen of the first device, wherein the first device includes the return application. The first graphic is made available on the screen of the first device via the return application on the first device, the graphic includes a first identification code, Capturing at least a portion of the first graphic presented on the first device via the camera of the second device, To enable the display of one or more second graphics on the screen of the first device, Capturing one or at least a portion of the second graphics presented on the first device via the camera of the second device, This includes processing one or more of the second images in order to determine the state of the screen of the first device, Processing one of the second images mentioned above is Dividing the aforementioned second image into parts, To determine whether one or more of the parts of the second image are damaged. Identifying parts adjacent to one or more parts containing damage, A method comprising determining the state of the screen of the first device based on whether one or more of the parts of the second image are determined to be damaged, and whether one or more of the parts adjacent to one of the parts determined to be damaged are also damaged.

14. The method according to claim 13, further comprising determining damage information if the device in state first is determined to be damaged, and generating a flag to identify one or more of the portions of the second image determined to contain damage based on the determined damage information.

15. The method according to claim 13, further comprising testing the touchscreen of the first device if it is determined that the state of the first device is damaged.

16. The method according to claim 13, further comprising calibrating the brightness of the screen of the first device based on the captured first image.

17. Making it possible to display one or more of the second graphics on the screen of the first device is: This includes enabling the presentation of a set of burst images on the first device, the set of burst images comprising at least one of the second graphics of a plurality of brightness levels, Capturing one or at least a portion of the second graphics presented on the first device is: Capturing the set of burst images presented on the first device, Selecting one of the captured burst images by determining which color in the captured burst image set is most similar to the reference color, Identifying the selected capture burst image as one of the captured second graphics, The method according to claim 13, including the method described in claim 13.

18. Capturing one or more parts of at least one second image among the second graphics is: The orientation of the device is determined based on the capture image of the first graphic, To provide guidance for adjusting the orientation of the device based on the determined orientation. Toto, Capturing at least a portion of the additional images of the first graphic via the camera of the first device, The method according to claim 13, including the method described in claim 13.

19. The method according to claim 13, wherein enabling the display of one or more second graphics on the screen of the first device includes enabling the sequential display of two or more second graphics on the screen of the first device, and capturing one or more of the second graphics includes capturing at least one image of each of the second graphics sequentially displayed on the screen of the first device.

20. The method according to claim 13, wherein enabling the display of one or more second graphics on the screen of the first device includes enabling the display of two or more second graphics simultaneously on the screen of the first device.

Images