Screen failure detection method and electronic device

Abstract

A screen fault detection method and an electronic device. The method can be applied to a first electronic device with image processing capability, such as a notebook computer, a tablet computer, etc. The method comprises: receiving, by the first electronic device, a plurality of video frames from a high-speed camera, the plurality of video frames being obtained by the high-speed camera continuously capturing an image displayed by a second electronic device; performing grayscale processing on the plurality of video frames respectively to obtain a plurality of grayscale processed video frames; determining a similarity between each pair of adjacent video frames in the plurality of grayscale processed video frames; and determining, based on the similarity between each pair of adjacent video frames, whether a screen of the second electronic device displaying the image has a screen flashing fault. By using the method, the rate of missed detection can be reduced when detecting the screen fault of the second electronic device displaying the image.

Description

Technical Field

[0001] This application relates to the field of terminals, and more particularly to a screen fault detection method and electronic device. Background Technology

[0002] The display screen (or screen) of electronic devices is the direct medium for users to interact with content. However, when displaying images, electronic device screens often experience problems such as black screens, distorted images, and flickering, resulting in a poor user experience. Therefore, it is necessary to test the display screen to detect and resolve display problems in a timely manner.

[0003] Currently, when the testing equipment (referred to as the first electronic device) performs fault detection on the screen of the device under test (referred to as the second electronic device), it can first acquire an image displayed by the second electronic device (referred to as image 1) captured by a high-speed camera, and an image of the second electronic device's own display captured by screen capture software (referred to as image 2). Then, image 1 and image 2 are compared to determine whether the screen of the second electronic device has a flickering fault. However, due to the low screen capture rate of the second electronic device, the first electronic device may have a high rate of missed detection when detecting screen flickering faults in the second electronic device.

[0004] Therefore, reducing the false negative rate has become an urgent problem to be solved when detecting screen flickering faults in second electronic devices. Summary of the Invention

[0005] This application provides a screen fault detection method and electronic device, which can reduce the missed detection rate when performing screen flicker fault detection on the screen of a second electronic device.

[0006] In a first aspect, embodiments of this application provide a screen fault detection method applied to a first electronic device. The method includes: receiving multiple video frames from a high-speed camera, wherein the multiple video frames are obtained by the high-speed camera continuously capturing images displayed by a second electronic device; performing grayscale processing on the multiple video frames respectively to obtain multiple grayscale processed video frames; determining the similarity between each pair of adjacent video frames in the multiple grayscale processed video frames; and determining whether the screen of the image displayed by the second electronic device has a screen flickering fault based on the similarity between each pair of adjacent video frames.

[0007] After implementing the method provided in the first aspect, when the first electronic device performs screen flicker fault detection on the screen displaying the image of the second electronic device, it can determine whether the screen displaying the image of the second electronic device has a flickering phenomenon based on the similarity between two adjacent video frames in multiple video frames obtained by continuously capturing the image displayed by the high-speed camera. In this way, compared with the method of the first electronic device determining whether the second electronic device has a screen flickering fault by comparing the image displayed by the second electronic device captured by the high-speed camera with the image displayed by the second electronic device itself captured by the screenshot software, there is no need for the second electronic device to capture the displayed image, thereby reducing the false negative rate when performing screen flicker fault detection on the screen displaying the image of the second electronic device.

[0008] In conjunction with the first aspect, in an optional implementation, determining the similarity between any two adjacent video frames in a plurality of grayscale processed video frames includes: determining the hash value of each video frame in the plurality of grayscale processed video frames; and determining the similarity between any two adjacent video frames based on the hash value of each video frame.

[0009] In conjunction with the first aspect, in an optional implementation, determining whether the screen of the second electronic device displaying the image has a flickering fault based on the similarity between each pair of adjacent video frames includes: selecting the first two similarities from a plurality of adjacent video frames according to the similarity between each pair of adjacent video frames in descending order of similarity; determining that the screen of the second electronic device displaying the image has a flickering fault if it is determined that the first two similarities are both greater than a preset similarity threshold and the absolute value of the difference between the ratio of the first two similarities and 1 is less than or equal to the preset threshold; and determining that the screen of the second electronic device displaying the image has not a flickering fault if it is determined that the similarity between each pair of adjacent video frames is both less than or equal to the preset similarity threshold, or the absolute value of the difference between the ratio of the first two similarities and 1 is greater than the preset threshold.

[0010] In conjunction with the first aspect, in an optional implementation, the method further includes: determining the normalized luminance value of each video frame in the plurality of grayscale processed video frames; and determining, based on the normalized luminance value of each video frame, whether the screen of the second electronic device displaying the image has a black screen fault.

[0011] After implementing the method provided in this embodiment, the first electronic device can perform black screen fault detection on the screen of the second electronic device based on the normalized brightness value of each video frame in multiple video frames after grayscale processing. Compared with the method where the first electronic device determines whether the second electronic device has a black screen fault by comparing the image of the second electronic device displayed by the high-speed camera with the image of the second electronic device displayed by the second electronic device captured by the screenshot software, there is no need for the second electronic device to capture the displayed image, thereby reducing the false negative rate when detecting black screen faults on the screen of the second electronic device.

[0012] In conjunction with the first aspect, in one optional implementation, determining whether the screen of the second electronic device displaying the image has a black screen fault based on the normalized brightness value of each video frame includes: determining that the screen of the second electronic device displaying the image has a black screen fault if any normalized brightness value in each video frame is less than a preset normalized brightness threshold; and determining that the screen of the second electronic device displaying the image has not a black screen fault if all normalized brightness values ​​of each video frame are greater than or equal to the preset normalized brightness threshold.

[0013] In conjunction with the first aspect, in an optional implementation, the method further includes: determining the No Reference Result Sharpness (NRSS) value of each video frame among multiple video frames after grayscale processing; and determining, based on the NRSS value of each video frame, whether the screen of the second electronic device displaying the image has a screen distortion fault.

[0014] After implementing the method provided in this embodiment, the first electronic device can perform screen distortion fault detection on the screen of the second electronic device displaying the image based on the NRSS value of each video frame in multiple video frames after grayscale processing. Compared with the method of the first electronic device detecting screen distortion faults on the screen of the second electronic device displaying the image based on the block variance of each video frame, this method can reduce the influence of non-distortion content, such as edge textures, on the block variance of video frames, thereby reducing the false detection rate when detecting screen distortion faults on the screen of the second electronic device displaying the image.

[0015] In conjunction with the first aspect, in one optional implementation, determining whether the screen of the second electronic device displaying the image has a screen distortion fault based on the NRSS value of each video frame includes: determining that the screen of the second electronic device displaying the image has a screen distortion fault if any NRSS value in each video frame is greater than a preset NRSS threshold; and determining that the screen of the second electronic device displaying the image has no screen distortion fault if all NRSS values ​​in each video frame are less than or equal to the preset NRSS threshold.

[0016] In conjunction with the first aspect, in an optional implementation, the method further includes: determining the edge line of each video frame in a plurality of grayscale processed video frames; and determining, based on the edge line of each video frame, whether the screen of the second electronic device has a garbled line fault.

[0017] After implementing the method provided in this embodiment, the first electronic device can perform garbled line fault detection on the screen of the second electronic device displaying the image based on the edge line of each video frame in multiple video frames after grayscale processing.

[0018] In conjunction with the first aspect, in one optional implementation, determining whether the screen of the second electronic device has a garbled line fault based on the edge lines of each video frame includes: determining that the screen of the second electronic device has a garbled line fault if the brightness value of any edge line in each video frame is greater than a preset edge line brightness value; and determining that the screen of the second electronic device does not have a garbled line fault if the brightness value of the edge lines in each video frame is less than or equal to the preset edge line brightness value.

[0019] In a second aspect, embodiments of this application provide a first electronic device, including: a touch screen, a camera, one or more processors, and one or more memories; the one or more processors are coupled to the touch screen, the camera, and the one or more memories, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when the one or more processors execute the computer instructions, cause the first electronic device to perform the method as described in the first aspect or any optional implementation thereof.

[0020] Thirdly, this application provides a chip system applied to a first computing device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the first computing device to perform the method as described in the first aspect or any optional implementation thereof.

[0021] Fourthly, embodiments of this application provide a computer program product containing instructions that, when executed on a first electronic device, cause the first electronic device to perform the method as described in the first aspect or any optional implementation thereof.

[0022] Fifthly, this application provides a computer-readable storage medium including instructions that, when executed on a first electronic device, cause the first electronic device to perform the method as described in the first aspect or any optional embodiment of the first aspect. Attached Figure Description

[0023] Figure 1This is a schematic diagram illustrating an application scenario of a screen fault detection method provided in an embodiment of this application;

[0024] Figure 2 This is a schematic flowchart of a screen fault detection method provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram illustrating the process by which a first electronic device determines the hash value of video frame 1 among multiple video frames after grayscale processing.

[0026] Figure 4a This is a schematic diagram of a set of consecutive video frames after grayscale processing provided in an embodiment of this application;

[0027] Figure 4b This is provided by the embodiments of this application. Figure 4a The diagram shows the Hamming distance between any two adjacent video frames in a series of consecutive video frames.

[0028] Figure 5 This is a schematic diagram of another screen fault detection method provided in an embodiment of this application;

[0029] Figure 6 This is a schematic diagram illustrating the process by which a first electronic device determines the normalized luminance value of video frame 2 among multiple video frames after grayscale processing.

[0030] Figure 7a This is a schematic diagram of a set of consecutive video frames after grayscale processing provided in an embodiment of this application;

[0031] Figure 7b This is provided by the embodiments of this application. Figure 7a The diagram shows the normalized luminance values ​​of each video frame in a series of video frames.

[0032] Figure 8 This is a schematic diagram of another screen fault detection method provided in the embodiments of this application;

[0033] Figure 9 This is a schematic diagram illustrating the process by which a first electronic device determines the NRSS value of video frame 3 among multiple video frames after grayscale processing.

[0034] Figure 10a This is a schematic diagram of a set of consecutive video frames after grayscale processing provided in an embodiment of this application;

[0035] Figure 10b This is provided by the embodiments of this application. Figure 10a The diagram shows the NRSS values ​​of each video frame in a series of video frames.

[0036] Figure 11This is a flowchart illustrating another screen fault detection method provided in an embodiment of this application;

[0037] Figure 12 This is a schematic diagram illustrating the process by which a first electronic device determines the edge line of video frame 4 among multiple video frames after grayscale processing.

[0038] Figure 13a This is a schematic diagram showing the result obtained by the first electronic device provided in this application performing a garbled line fault detection on a continuous video frame;

[0039] Figure 13b This is a schematic diagram of a video frame marked with decorative lines, provided in an embodiment of this application;

[0040] Figure 14 This is a schematic diagram of the software architecture of a first electronic device provided in an embodiment of this application;

[0041] Figure 15 This is a schematic diagram of the hardware structure of a first electronic device provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. The term "embodiment" as used herein means that a specific feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The appearance of this phrase in different places in the specification does not necessarily indicate the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0043] In this application, the first electronic device may include, but is not limited to, mobile phones, tablets, desktop computers, laptop computers, handheld computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices (such as smartwatches), in-vehicle devices, smart home devices, and / or smart city devices, etc. The embodiments of this application do not impose any special limitations on the specific type of the first electronic device.

[0044] In this application, the second electronic device may include, but is not limited to, mobile phones with displays, tablets, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices (such as smartwatches), in-vehicle devices, smart home devices, and / or smart city devices. This application does not impose any special limitations on the specific type of the second electronic device.

[0045] Taking a mobile phone as an example as the second electronic device, please refer to Table 1. Table 1 is a summary table of common display problems of different series of mobile phone screens provided in the embodiments of this application.

[0046] Table 1. Summary of common display problems in mobile phone screens of different series

[0047]

[0048]

[0049] As shown in Table 1 above, black-and-white flickering is the most serious display problem among common screen issues. Black-and-white flickering is often a probabilistic occurrence, and the problem scene is difficult to preserve, making reproduction and analysis challenging. Furthermore, display problems negatively impact the user experience. Therefore, screen testing is necessary to detect and resolve display issues promptly.

[0050] Currently, when the first electronic device performs screen fault detection on the second electronic device, it can first acquire an image of the second electronic device's display captured by a high-speed camera (denoted as Image 1), and an image of the second electronic device's own display captured by screen capture software (denoted as Image 2). Then, Image 1 and Image 2 are compared to determine whether the second electronic device has a screen flickering fault. However, this method has the potential for a high false negative rate when the first electronic device performs screen flickering fault detection on the second electronic device due to the low capture rate of the second electronic device.

[0051] To address the aforementioned problems, this application provides a screen fault detection method. This screen fault detection method can be applied to a first electronic device with image processing capabilities.

[0052] By implementing the screen fault detection method provided in this application, when the first electronic device performs screen flicker fault detection on the screen of the second electronic device displaying an image, it can determine whether the screen of the second electronic device displaying an image has a flickering phenomenon based on the similarity between two adjacent video frames in multiple video frames obtained by continuously capturing the image of the second electronic device displaying the image using a high-speed camera.

[0053] For details, please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of a screen fault detection method provided in an embodiment of this application. For example... Figure 1 As shown, the screen fault detection system includes a first electronic device 101, a high-speed camera 102, and a second electronic device 103. The first electronic device 101 first receives multiple video frames from the high-speed camera 102, which are obtained by the high-speed camera 102 continuously capturing images displayed on the second electronic device 103. Next, the multiple video frames are processed into grayscale, resulting in grayscale-processed video frames. Then, the first electronic device 101 determines the similarity between any two adjacent video frames in the grayscale-processed video frames. Finally, based on the similarity between any two adjacent video frames, it determines whether the screen displaying the image on the second electronic device 103 has a screen distortion fault.

[0054] Compared with the method where the first electronic device determines whether the second electronic device has a screen flickering fault by comparing the image of the second electronic device displayed by the first electronic device captured by the high-speed camera with the image of the second electronic device itself captured by the screenshot software, the above screen fault detection method does not require the second electronic device to capture the displayed image. This can reduce the false negative rate when detecting screen flickering faults on the screen of the second electronic device.

[0055] The screen fault detection method provided in the embodiments of this application will be described in detail below.

[0056] Please see Figure 2 , Figure 2 This is a flowchart illustrating a screen fault detection method provided in an embodiment of this application. The method can be implemented by a first electronic device (such as...) Figure 1 The first electronic device 101 in the process executes the command. For example... Figure 2 As shown, the screen fault detection method may include, but is not limited to, the following steps:

[0057] S201. The first electronic device receives multiple video frames from a high-speed camera, wherein the multiple video frames are obtained by the high-speed camera continuously capturing images displayed by the second electronic device.

[0058] S202, The first electronic device performs grayscale processing on multiple video frames respectively to obtain multiple grayscale processed video frames.

[0059] Optionally, the first electronic device performs grayscale processing on multiple video frames separately, which may involve performing histogram mapping (or grayscale mapping) on ​​multiple video frames separately.

[0060] In an optional implementation, before step S202, the first electronic device further performs cropping and correction processing on the multiple video frames respectively to obtain cropped and corrected video frames. Optionally, after obtaining the cropped and corrected video frames, the first electronic device may further perform grayscale processing on the cropped and corrected video frames respectively to obtain grayscale processed video frames.

[0061] Optionally, the first electronic device performs grayscale processing on the multiple cropped and corrected video frames separately, which may involve performing histogram mapping on the multiple cropped and corrected video frames separately.

[0062] S203, The first electronic device determines the similarity between any two adjacent video frames in the multiple video frames after grayscale processing.

[0063] In one alternative implementation, the first electronic device may use a perceptual hash algorithm (pHash) to determine the similarity between any two adjacent video frames in a plurality of grayscale processed video frames. The similarity between any two adjacent video frames is the Hamming distance between them.

[0064] In this embodiment, the first electronic device can first determine the hash value of each video frame in the multiple video frames after grayscale processing; based on the hash value of each video frame, the similarity between each pair of adjacent video frames is determined.

[0065] The following example illustrates how a first electronic device determines the hash value of any one of the multiple grayscale video frames (e.g., video frame 1). Figure 3 The process of determining the hash value of each video frame by the first electronic device is described. Figure 3 This is a schematic diagram illustrating the process by which a first electronic device determines the hash value of video frame 1 among multiple video frames after grayscale processing.

[0066] like Figure 3 As shown, the first electronic device can first perform a discrete cosine transform (DCT) on the grayscale processed video frame 1 to convert the video frame from the spatial domain to the frequency domain, obtaining the frequency domain information matrix corresponding to video frame 1. Next, from the frequency domain information matrix corresponding to video frame 1, a 16*16 matrix recording the low-frequency information in video frame 1 is selected. Then, the mean of the aforementioned 16*16 matrix is ​​determined. Finally, based on the aforementioned 16*16 matrix and its mean, the hash value of video frame 1 is determined. Specifically, when the first computing device determines the hash value of video frame 1 based on the aforementioned 16*16 matrix and its mean, it can first convert values ​​in the 16*16 matrix that are greater than or equal to the matrix's mean to 1, and convert values ​​in the 16*16 matrix that are less than the matrix's mean to 0. Then, a binary array is generated based on the converted values, and the generated binary array is used as the hash value of video frame 1. In this way, binarization can compress low-frequency information into a 16*16 list of 0s and 1s (hash value), which greatly compresses the video frame and preserves the low-frequency information of the video frame. The hash value can be used to compare the similarity between two adjacent video frames.

[0067] It is understandable that the first electronic device can be accessed via, for example... Figure 3 The method shown determines the hash value of each video frame in multiple video frames after grayscale processing.

[0068] S204. The first electronic device determines whether the screen of the second electronic device displaying the image has a screen flickering fault based on the similarity between each two adjacent video frames.

[0069] In one optional implementation, the first electronic device determines whether the screen of the second electronic device displaying the image has a flickering fault based on the similarity between every two adjacent video frames. This may include: selecting the first two similarities from a plurality of similarity values ​​between two adjacent video frames, in descending order of similarity; determining that the screen of the second electronic device displaying the image has a flickering fault if the first two similarities are both greater than a preset similarity threshold and the absolute value of the difference between the ratio of the first two similarities and 1 is less than or equal to the preset threshold; and determining that the screen of the second electronic device displaying the image does not have a flickering fault if the similarity between every two adjacent video frames is less than or equal to the preset similarity threshold, or the absolute value of the difference between the ratio of the first two similarities and 1 is greater than the preset threshold. Optionally, the preset similarity threshold is determined by the first electronic device based on historical flickering faults of the screen of the second electronic device displaying the image; the preset threshold is determined based on empirical values.

[0070] In this context, if the absolute value of the difference between the ratio of the first two similarities and 1 is less than or equal to the preset threshold, it can be understood that the ratio of the first two similarities is close to 1; if the absolute value of the difference between the ratio of the first two similarities and 1 is greater than the preset threshold, it can be understood that the ratio of the first two similarities is much greater than 1 or much less than 1.

[0071] Optionally, the above similarity can refer to the Hamming distance.

[0072] For example, the following is combined with Figure 4a and Figure 4b The process by which the first electronic device determines whether the screen of the second electronic device displays an image has a screen flickering fault based on the similarity between every two adjacent video frames is explained. Figure 4a This is a schematic diagram of a set of consecutive video frames after grayscale processing provided in an embodiment of this application; Figure 4b This is provided by the embodiments of this application. Figure 4a The diagram shows the Hamming distance between any two adjacent video frames in a series of consecutive video frames.

[0073] like Figure 4a As shown, it includes 7 consecutive video frames. Figure 4b As shown, the horizontal axis represents adjacent video frames, and the vertical axis represents the Hamming distance. Figure 4b It can be seen that, Figure 4aIn the seven consecutive video frames shown, the Hamming distance between video frame 1 and video frame 2 (frame(1-2)) is 0.0156; the Hamming distance between video frame 2 and video frame 3 (frame(2-3)) is 0.4219; the Hamming distance between video frame 3 and video frame 4 (frame(3-4)) is 0.0391; the Hamming distance between video frame 4 and video frame 5 (frame(4-5)) is 0.4063; the Hamming distance between video frame 5 and video frame 6 (frame(5-6)) is 0.0781; and the Hamming distance between video frame 6 and video frame 7 (frame(6-7)) is 0.0625. At this time, the first electronic device can select the first two Hamming distances from the Hamming distances between the two adjacent video frames in descending order of Hamming distance, namely 0.4219 (Hamming distance of frame (2-3)) and 0.4063 (Hamming distance of frame (4-5)), and the first electronic device can determine that the ratio of 0.4219 and 0.4063 is 0.4219 / 0.4063≈1.038. Assuming the preset Hamming distance threshold is 0.1 and the preset distance threshold is 0.04, the first electronic device can determine that the Hamming distance of frame (2-3) 0.4219 is greater than 0.1, the Hamming distance of frame (4-5) 0.4063 is greater than 0.1, and the absolute value of the difference between the ratio (1.038) of the Hamming distance of frame (2-3) 0.4219 and the Hamming distance of frame (4-5) 0.4063 and 1, |1.038-1|=0.038, is less than the preset distance threshold of 0.04. In this case, the first electronic device can determine that the screen of the image displayed by the second electronic device has a flickering fault. Furthermore, since the Hamming distances of frames (2-3) and (4-5) are relatively large, the first electronic device can determine that there are screen jumps between frames (2-3) and between frames (4-5). Therefore, the first electronic device can determine that the flickering abnormal video frames in the seven consecutive video frames are video frames 3 and video frames 4.

[0074] In an optional implementation, after step S204, the first electronic device may further store multiple received video frames to generate fault video and fault log information if it is determined that the screen displaying the image of the second electronic device has a screen flickering fault.

[0075] In this embodiment, when the first electronic device performs screen flicker fault detection on the screen displaying the image of the second electronic device, it can determine whether the screen displaying the image of the second electronic device exhibits a flickering phenomenon based on the similarity between two adjacent video frames obtained from multiple video frames continuously captured by a high-speed camera. This method, compared with the first electronic device's method of comparing the image of the second electronic device displayed by the high-speed camera with the image of the second electronic device itself captured by the screenshot software, eliminates the need for the second electronic device to capture the displayed image, thereby reducing the false negative rate when detecting screen flicker faults on the screen displaying the image of the second electronic device.

[0076] Please see Figure 5 , Figure 5 This is a schematic diagram of another screen fault detection method provided in an embodiment of this application. Figure 2 The difference in the screen fault detection method shown is that, Figure 5 The screen fault detection method shown can also determine whether the screen of the second electronic device displaying the image has a black screen fault. For example... Figure 5 As shown, the screen fault detection method may include, but is not limited to, the following steps:

[0077] S501, the first electronic device receives multiple video frames from a high-speed camera, wherein the multiple video frames are obtained by the high-speed camera continuously capturing images displayed by the second electronic device.

[0078] S502, The first electronic device performs grayscale processing on multiple video frames to obtain multiple grayscale processed video frames.

[0079] In one alternative implementation, after step S502, the first electronic device may execute both steps S503a and S504a, as well as steps S503b and S504b; or, the first electronic device may execute only steps S503b and S504b.

[0080] S503a, The first electronic device determines the similarity between any two adjacent video frames in the multiple video frames after grayscale processing.

[0081] S504a. The first electronic device determines whether the screen displaying the image on the second electronic device has a screen flickering fault based on the similarity between each two adjacent video frames.

[0082] In an optional implementation, the specific processes of steps S503a and S504a can be found in the relevant descriptions of steps S203 and S204 above, and will not be repeated here.

[0083] S503b, the first electronic device determines the normalized luminance value of each video frame in the multiple video frames after grayscale processing.

[0084] The following example uses the first electronic device to determine the normalized luminance value of any one video frame (e.g., video frame 2) among multiple video frames after grayscale processing. Figure 6 The process by which the first electronic device determines the normalized luminance value for each video frame is described. Figure 6 This is a schematic diagram illustrating the process by which a first electronic device determines the normalized luminance value of video frame 2 among multiple video frames after grayscale processing.

[0085] like Figure 6 As shown. The first electronic device can first calculate the grayscale mean of video frame 2 after grayscale processing, and then divide the grayscale mean of video frame 2 by 255 to obtain the normalized brightness value of video frame 2.

[0086] It is understandable that the first electronic device can be accessed via, for example... Figure 6 The method shown determines the normalized luminance value of each video frame in multiple video frames after grayscale processing.

[0087] S504b: The first electronic device determines whether the screen of the second electronic device displaying the image has a black screen fault based on the normalized brightness value of each video frame.

[0088] In one optional implementation, the first electronic device determines whether the screen of the second electronic device displaying the image has experienced a black screen fault based on the normalized brightness value of each video frame. This may include: determining that the screen of the second electronic device displaying the image has experienced a black screen fault if any normalized brightness value in each video frame is less than a preset normalized brightness threshold; and determining that the screen of the second electronic device displaying the image has not experienced a black screen fault if all normalized brightness values ​​in each video frame are greater than or equal to the preset normalized brightness threshold. Optionally, the preset normalized brightness threshold is determined by the first electronic device based on historical black screen faults of the screen of the second electronic device displaying the image.

[0089] For example, the following is combined with Figure 7a and Figure 7b The process by which the first electronic device determines whether the screen of the second electronic device is experiencing a black screen fault based on the normalized brightness value of each video frame is explained. Figure 7a This is a schematic diagram of a set of consecutive video frames after grayscale processing provided in an embodiment of this application; Figure 7b This is provided by the embodiments of this application. Figure 7a The diagram shows the normalized luminance values ​​for each video frame in a series of video frames.

[0090] like Figure 7aAs shown, it includes 6 consecutive video frames. Figure 7b As shown, the horizontal axis represents video frames, and the vertical axis represents normalized luminance values. (From...) Figure 7b It can be seen that, Figure 7a In the six consecutive video frames shown, the normalized luminance value of video frame 1 is 0.7354; the normalized luminance value of video frame 2 is 0.7353; the normalized luminance value of video frame 3 is 0.7348; the normalized luminance value of video frame 4 is 0.0344; the normalized luminance value of video frame 5 is 0.7346; and the normalized luminance value of video frame 6 is 0.7347. Assuming a preset normalized luminance threshold of 0.1, the first electronic device can determine that the normalized luminance value of frame 4 (0.0344) is less than 0.1. In this case, the first electronic device can determine that the screen displayed by the second electronic device is black. Furthermore, the first electronic device can determine that the black screen among the six consecutive video frames is video frame 4.

[0091] In an optional implementation, after step S504b, the first electronic device may further store multiple received video frames to generate fault video and fault log information if it is determined that the screen of the second electronic device displaying the image has a black screen fault.

[0092] In this embodiment, the first electronic device can perform black screen fault detection on the screen of the second electronic device based on the normalized brightness value of each video frame in multiple video frames after grayscale processing. Compared to the method where the first electronic device determines whether the second electronic device has a black screen fault by comparing the image displayed by the second electronic device captured by a high-speed camera with an image of the second electronic device itself captured by screenshot software, this method eliminates the need for the second electronic device to capture the displayed image, thereby reducing the false negative rate when detecting black screen faults on the screen of the second electronic device.

[0093] Please see Figure 8 , Figure 8 This is a schematic diagram of yet another screen fault detection method provided in an embodiment of this application. Figure 2 and Figure 5 The difference in the screen fault detection method shown is that, Figure 8 The screen fault detection method shown can also determine whether the screen displaying the image on the second electronic device is experiencing a flickering fault. For example... Figure 8 As shown, the screen fault detection method may include, but is not limited to, the following steps:

[0094] S801, the first electronic device receives multiple video frames from a high-speed camera, wherein the multiple video frames are obtained by the high-speed camera continuously capturing images displayed by the second electronic device.

[0095] S802, the first electronic device performs grayscale processing on multiple video frames respectively to obtain multiple grayscale processed video frames.

[0096] In one alternative implementation, after step S802, the first electronic device may execute steps S803a and S804a, steps S803b and S804b, and steps S803c and S804c; or, the first electronic device may execute only steps S803a and S804a, and S803c and S804c; or, the first electronic device may execute only steps S803c and S804c.

[0097] S803a, The first electronic device determines the similarity between any two adjacent video frames in the multiple video frames after grayscale processing.

[0098] S804a. The first electronic device determines whether the screen displaying the image on the second electronic device has a screen flickering fault based on the similarity between each two adjacent video frames.

[0099] In an optional implementation, the specific processes of steps S803a and S804a can be found in the relevant descriptions of steps S203 and S204 above, and will not be repeated here.

[0100] S803b, the first electronic device determines the normalized luminance value of each video frame in the multiple video frames after grayscale processing.

[0101] S804b: The first electronic device determines whether the screen of the second electronic device displaying the image has a black screen fault based on the normalized brightness value of each video frame.

[0102] In an optional implementation, the specific processes of steps S803b and S804b can be found in the relevant descriptions of steps S503b and S504b above, and will not be repeated here.

[0103] S803c, the first electronic device determines the No Reference Result Sharpness (NRSS) value of each video frame in multiple video frames after grayscale processing.

[0104] The NRSS value can be used to measure high-frequency information in each video frame, and the NRSS value has an adjustable window size, making it a good measure of content with screen tearing. The higher the NRSS value, the clearer the video frame.

[0105] The following example uses the first electronic device to determine the NRSS value of any one video frame (e.g., video frame 3) among multiple grayscale processed video frames. Figure 9 The process by which the first electronic device determines the NRSS value for each video frame is described. Figure 9 This is a schematic diagram illustrating the process by which a first electronic device determines the NRSS value of video frame 3 among multiple video frames after grayscale processing.

[0106] like Figure 9 As shown, the first electronic device can first use a low-pass filter to perform low-pass filtering on the grayscale processed video frame 3, filtering out the high-frequency information in video frame 3, and obtaining the low-frequency information image corresponding to the grayscale processed video frame 3. Then, the structural similarity (SSIM) between the low-frequency information image corresponding to video frame 3 and video frame 3 is calculated to measure the lost high-frequency information in video frame 3. Finally, the NRSS value of video frame 3 is obtained by subtracting the above SSIM value from 1.

[0107] It is understandable that the first electronic device can be accessed via, for example... Figure 9 The method shown determines the NRSS value of each video frame in multiple video frames after grayscale processing.

[0108] S804c: The first electronic device determines whether the screen displaying the image on the second electronic device has a screen distortion fault based on the NRSS value of each video frame.

[0109] In one optional implementation, the first electronic device determines whether the screen of the second electronic device displaying the image has a screen tearing fault based on the NRSS value of each video frame. This can include: determining that the screen of the second electronic device displaying the image has a screen tearing fault if any NRSS value in each video frame is greater than a preset NRSS threshold; and determining that the screen of the second electronic device displaying the image has not a screen tearing fault if the NRSS values ​​of each video frame are all less than or equal to the preset NRSS threshold. Optionally, the preset NRSS threshold is determined by the first electronic device based on historical screen tearing faults of the screen of the second electronic device displaying the image.

[0110] For example, the following is combined with Figure 10a and Figure 10b The process by which the first electronic device determines whether the screen of the second electronic device displays a distorted image based on the NRSS value of each video frame is explained. Figure 10a This is a schematic diagram of a set of consecutive video frames after grayscale processing, provided in an embodiment of this application. Figure 10b This is provided by the embodiments of this application. Figure 10a The diagram shows the NRSS values ​​for each video frame in a series of video frames.

[0111] like Figure 10a As shown, this includes five consecutive video frames. Figure 10b As shown, the horizontal axis represents video frames, and the vertical axis represents NRSS values. Figure 10b It can be seen that, Figure 10a In the five consecutive video frames shown, the NRSS value of video frame 1 is 0.3755; the NRSS value of video frame 2 is 0.3752; the NRSS value of video frame 3 is 0.3750; the NRSS value of video frame 4 is 0.7921; and the NRSS value of video frame 5 is 0.3308. Assuming a preset NRSS threshold of 0.75, the first electronic device can determine that the NRSS value of frame 4 (0.7921) is greater than 0.75. In this case, the first electronic device can determine that the screen of the image displayed by the second electronic device is exhibiting a distorted display. Furthermore, the first electronic device can determine that the distorted display occurs in video frame 4 out of the five consecutive video frames.

[0112] In an optional implementation, after step S804c, the first electronic device may further store multiple received video frames to generate fault video and fault log information if it is determined that the screen displaying the image of the second electronic device has a screen distorting fault.

[0113] In this embodiment, the first electronic device can perform screen distortion fault detection on the screen of the second electronic device based on the NRSS value of each video frame in multiple video frames after grayscale processing. Compared with the method of the first electronic device detecting screen distortion faults on the screen of the second electronic device based on the block variance of each video frame, this method can reduce the influence of non-distortion content, such as edge textures, on the block variance of video frames, thereby reducing the false detection rate when detecting screen distortion faults on the screen of the second electronic device.

[0114] Please see Figure 11 , Figure 11 This is a flowchart illustrating another screen fault detection method provided in an embodiment of this application. Figure 2 , Figure 5 or Figure 8 The difference in the screen fault detection method shown is that, Figure 11 The screen fault detection method shown can also determine whether the screen displaying the image in the second electronic device has a distorted line fault. For example... Figure 11 As shown, the screen fault detection method may include, but is not limited to, the following steps:

[0115] S1101, The first electronic device receives multiple video frames from a high-speed camera, wherein the multiple video frames are obtained by the high-speed camera continuously capturing images displayed by the second electronic device.

[0116] S1102, The first electronic device performs grayscale processing on multiple video frames respectively to obtain multiple grayscale processed video frames.

[0117] In one optional implementation, after step S802, the first electronic device may execute steps S1103a and S1104a, steps S1103b and S1104b, steps S1103c and S1104c, and steps S1103d and S1104d; or, the first electronic device may execute only steps S1103a and S1104a, and steps S1103d and S1104d; or, the first electronic device may execute steps S1103a and S1104a, S1103b and S1104b, and steps S1103d and S1104d; or, the first electronic device may execute only steps S1103d and S1104d.

[0118] S1103a, The first electronic device determines the similarity between any two adjacent video frames in the multiple video frames after grayscale processing.

[0119] S1104a. The first electronic device determines whether the screen of the second electronic device displaying the image has a flickering fault based on the similarity between each two adjacent video frames.

[0120] In an optional implementation, the specific processes of steps S1103a and S1104a can be found in the relevant descriptions of steps S203 and S204 above, and will not be repeated here.

[0121] S1103b: The first electronic device determines the normalized luminance value of each video frame in the multiple video frames after grayscale processing.

[0122] S1104b: The first electronic device determines whether the screen of the second electronic device displaying the image has a black screen fault based on the normalized brightness value of each video frame.

[0123] In an optional implementation, the specific processes of steps S1103b and S1104b can be found in the relevant descriptions of steps S503b and S504b above, and will not be repeated here.

[0124] S1103c, The first electronic device determines the No Reference Result Sharpness (NRSS) value of each video frame in the multiple video frames after grayscale processing.

[0125] S1104c: The first electronic device determines whether the screen displaying the image on the second electronic device has a screen distortion fault based on the NRSS value of each video frame.

[0126] In an optional implementation, the specific processes of steps S1103c and S1104c can be found in the relevant descriptions of steps S803c and S804c above, and will not be repeated here.

[0127] S1103d, the first electronic device determines the edge line of each video frame in multiple video frames after grayscale processing.

[0128] The following example illustrates how a first electronic device determines the edge line of any one of the multiple grayscale video frames (e.g., video frame 4). Figure 12 The process of determining the edge line of each video frame in multiple video frames after grayscale processing by the first electronic device is described. Figure 12 This is a schematic diagram illustrating the process by which a first electronic device determines the edge line of video frame 4 among multiple video frames after grayscale processing.

[0129] like Figure 12 As shown, the first electronic device can first use a low-pass filter to perform low-pass filtering on the grayscale video frame 4 to filter out noise in the video frame 4, obtaining the filtered video frame 3. Then, the Canny operator is used to extract the edge information of the filtered video frame 3. Finally, a Hough transform is performed on the extracted edge information to obtain the edge lines of video frame 3.

[0130] Among them, the Canny operator is an edge detection algorithm that is not easily affected by noise. It can identify weak and strong edges in an image and combine the positional relationship between strong and weak edges to comprehensively determine the overall edge information of the image.

[0131] The Hough transform is a feature extraction algorithm that is mainly used to separate geometric shapes (such as lines) that share certain characteristics from an image.

[0132] It is understandable that the first electronic device can be accessed via, for example... Figure 12 The method shown determines the edge line of each video frame in multiple video frames after grayscale processing.

[0133] S1104d: The first electronic device determines whether the screen of the second electronic device has a garbled line fault based on the edge line of each video frame.

[0134] In one optional implementation, the first electronic device determines whether the screen of the second electronic device has a garbled line fault based on the edge lines of each video frame. This may include: determining that the screen of the second electronic device has a garbled line fault if the brightness value of any edge line in each video frame is greater than a preset edge line brightness value; and determining that the screen of the second electronic device does not have a garbled line fault if the brightness value of the edge lines in each video frame is less than or equal to the preset edge line brightness value.

[0135] For example, suppose the first electronic device performs a garbled line fault detection on a consecutive video frame (e.g., 5 video frames), and the result is as follows: Figure 13a As shown, Figure 13a This is a schematic diagram illustrating the result obtained by the first electronic device provided in this application during the detection of spline faults in a continuous video frame segment. For example... Figure 13a As shown, this includes 5 video frames, with video frame 3 exhibiting obvious edge information. At this point, the first electronic device can perform a Hough transform on the edge information in video frame 3 to determine the edge lines of video frame 3, and based on these edge lines, determine that the screen of the second electronic device has a streaking fault. For example, as... Figure 13b As shown. Figure 13b This is a schematic diagram of a video frame marked with decorative lines, provided in an embodiment of this application.

[0136] In an optional implementation, after step S1104d, the first electronic device may further store multiple received video frames to generate fault video and fault log information if it is determined that the screen displaying the image of the second electronic device has a garbled line fault.

[0137] In this embodiment of the application, the first electronic device can perform edge detection on multiple received video frames, and based on the edge detection results, determine whether the screen of the image displayed by the second electronic device has a garbled line fault, thereby improving the accuracy of detecting garbled line faults on the screen of the image displayed by the second electronic device.

[0138] Please see Figure 14 , Figure 14 This is a schematic diagram of the software architecture of a first electronic device provided in an embodiment of this application.

[0139] The software system of the first electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of the first electronic device.

[0140] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), and the kernel layer.

[0141] The application layer can include a series of application packages. For example... Figure 14 As shown, the application package can include applications such as camera, gallery, video, music, navigation, calendar, map, and WLAN.

[0142] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 14 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0143] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0144] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0145] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0146] A phone manager is used to provide communication functions for the primary electronic device. This includes managing call status (including connection and disconnection).

[0147] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0148] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0149] The hardware abstraction layer can include multiple functional modules, such as a receiving module, a processing module, and a determination module.

[0150] The receiving module can be used to receive multiple video frames from a high-speed camera, which are obtained by the high-speed camera continuously capturing images displayed by the second electronic device.

[0151] The processing module can be used to perform grayscale processing on multiple video frames separately to obtain multiple grayscale processed video frames;

[0152] The determination module can be used to determine the similarity between any two adjacent video frames in multiple grayscale processed video frames;

[0153] The determination module can also be used to determine whether the screen of the second electronic device displaying the image has a flickering fault based on the similarity between each two adjacent video frames.

[0154] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0155] Figure 15 This is a schematic diagram of the hardware structure of a first electronic device provided in an embodiment of this application.

[0156] like Figure 15As shown, the first electronic device may include a processor 110, an external memory interface 120A, an internal memory 120B, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 140A, a battery 140B, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0157] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the first electronic device. In other embodiments of this application, the first electronic device may include... Figure 15 This may involve more or fewer components, or combining certain components, or splitting certain components, or different component arrangements. Figure 15 The components shown can be implemented in hardware, software, or a combination of both.

[0158] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0159] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0160] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0161] In an optional implementation, the processor 110 can be used to perform the operation of the first electronic device in the above-described screen fault detection method, specifically as follows: receiving multiple video frames from a high-speed camera, the multiple video frames being obtained by the high-speed camera continuously capturing images displayed by the second electronic device; performing grayscale processing on the multiple video frames respectively to obtain multiple grayscale processed video frames; determining the similarity between each pair of adjacent video frames in the multiple grayscale processed video frames; and determining whether the screen displaying the image of the second electronic device has a screen flickering fault based on the similarity between each pair of adjacent video frames.

[0162] In addition, the processor 110 can also be used to: determine the normalized luminance value of each video frame in a plurality of video frames after grayscale processing; and determine, based on the normalized luminance value of each video frame, whether the screen of the second electronic device displaying the image has a black screen fault.

[0163] In addition, the processor 110 can also be used to: determine the No Reference Result Sharpness (NRSS) value of each video frame in a plurality of video frames after grayscale processing; and determine, based on the NRSS value of each video frame, whether the screen of the second electronic device displaying the image has a screen distortion fault.

[0164] In addition, the processor 110 can also be used to: determine the edge line of each video frame in a plurality of video frames after grayscale processing; and determine, based on the edge line of each video frame, whether the screen of the second electronic device has a garbled line fault.

[0165] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0166] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the first electronic device.

[0167] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0168] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0169] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0170] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to achieve the shooting function of the first electronic device. The processor 110 and the display screen 194 communicate via the DSI interface to achieve the display function of the first electronic device.

[0171] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0172] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge a primary electronic device, or to transfer data between the primary electronic device and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0173] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the first electronic device. In other embodiments of this application, the first electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0174] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the first electronic device. While charging the battery 140B, the charging management module 140 can also supply power to the electronic device via the power management module 140A.

[0175] The power management module 140A connects to the battery 140B, the charging management module 140, and the processor 110. The power management module 140A receives input from the battery 140B and / or the charging management module 140, providing power to the processor 110, internal memory 120B, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 140A can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 140A may also be located within the processor 110. In other embodiments, the power management module 140A and the charging management module 140 may be housed in the same device.

[0176] The wireless communication function of the first electronic device can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0177] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the first electronic device can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0178] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to a first electronic device. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0179] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0180] The wireless communication module 160 can provide solutions for wireless communication applications in the first electronic device, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0181] In some embodiments, antenna 1 of the first electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the first electronic device to communicate with a network and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0182] The first electronic device implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0183] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized display, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the first electronic device may include one or N displays 194, where N is a positive integer greater than 1.

[0184] The first electronic device can achieve shooting functions through an ISP, camera 193, video codec, GPU, display 194, and application processor.

[0185] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimizations on image noise, brightness, etc. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0186] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the first electronic device may include one or N cameras 193, where N is a positive integer greater than 1.

[0187] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when a first electronic device selects a frequency, a DSP can perform Fourier transforms on the frequency energy.

[0188] Video codecs are used to compress or decompress digital video. A first electronic device may support one or more video codecs. Thus, the first electronic device can play or record video in various encoded formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0189] An NPU (Neural Processing Unit) is a neural network (NN) computing processor that, by borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0190] The internal memory 120B may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0191] Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, fifth generation DDR SDRAM is generally called DDR5 SDRAM), etc.

[0192] Non-volatile memory can include disk storage devices and flash memory.

[0193] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multimedia card (eMMC), etc.

[0194] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0195] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0196] The external memory interface 120A can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the first electronic device. The external non-volatile memory communicates with the processor 110 through the external memory interface 120A to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0197] The first electronic device can implement audio functions, such as music playback and recording, through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor.

[0198] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0199] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The first electronic device can listen to music or make hands-free calls through the speaker 170A.

[0200] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the first electronic device answers a telephone call or voice message, it can listen to the voice by bringing the receiver 170B close to the ear.

[0201] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. The first electronic device may have at least one microphone 170C. In some embodiments, the first electronic device may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, the first electronic device may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0202] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0203] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. A first electronic device determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, the first electronic device detects the intensity of the touch operation based on pressure sensor 180A. The first electronic device can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0204] The gyroscope sensor 180B can be used to determine the motion posture of a first electronic device. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the first electronic device around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the first electronic device, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the first electronic device by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0205] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the first electronic device calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0206] The magnetic sensor 180D includes a Hall effect sensor. The first electronic device can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the first electronic device is a flip phone, the first electronic device can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0207] The 180E accelerometer can detect the magnitude of acceleration in various directions (typically three axes) of a first electronic device. When the first electronic device is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices, and is applicable to screen orientation switching, pedometers, and other applications.

[0208] A distance sensor 180F is used to measure distance. The first electronic device can measure distance via infrared or laser. In some embodiments, during scene capture, the first electronic device can utilize the distance sensor 180F to measure distance for rapid focusing.

[0209] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The first electronic device emits infrared light outward through the LED. The first electronic device uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that an object is near the first electronic device. When insufficient reflected light is detected, the first electronic device can determine that no object is near it. The first electronic device can use the proximity sensor 180G to detect when a user holds the device close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0210] The ambient light sensor 180L is used to sense ambient light intensity. The first electronic device can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light intensity. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the first electronic device is in a pocket to prevent accidental touches.

[0211] The fingerprint sensor 180H is used to collect fingerprints. The first electronic device can utilize the characteristics of the collected fingerprint to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0212] Temperature sensor 180J is used to detect temperature. In some embodiments, the first electronic device uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, the first electronic device reduces the performance of a processor located near temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is below another threshold, the first electronic device heats battery 140B to prevent abnormal shutdown of the first electronic device due to low temperature. In still other embodiments, when the temperature is below yet another threshold, the first electronic device boosts the output voltage of battery 140B to prevent abnormal shutdown due to low temperature.

[0213] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of the first electronic device, in a different position than display screen 194.

[0214] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be incorporated into headphones to form bone conduction headphones. The audio module 170 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 180M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 180M to realize heart rate detection functionality.

[0215] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. The first electronic device can receive button input and generate key signal inputs related to user settings and function control of the first electronic device.

[0216] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0217] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0218] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the first electronic device. The first electronic device can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The first electronic device interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the first electronic device uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the first electronic device and cannot be separated from the first electronic device.

[0219] It should be noted that, for the sake of simplicity, the above method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the present invention. The various embodiments of this application can be arbitrarily combined to achieve different technical effects.

[0220] As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the listed items. As used in the above embodiments, depending on the context, the term “when” can be interpreted as meaning “if…” or “after…” or “in response to determining…” or “in response to detecting…”. Similarly, depending on the context, the phrase “when…” or “if (the stated condition or event) is interpreted as meaning “if…” or “in response to determining…” or “when (the stated condition or event) is detected” or “in response to detecting (the stated condition or event)”.

[0221] In the specification, claims, and accompanying drawings of this application, the terms "first," "second," etc., are used to distinguish different objects and not to describe a particular order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, it may include a series of steps or units, or optionally, steps or units not listed, or other steps or units inherent to these processes, methods, products, or devices.

[0222] The accompanying drawings show only the portions relevant to this application, not all of them. Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts depict operations (or steps) as sequential processes, many of these operations may be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0223] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0224] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0225] In summary, the above description is merely an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made according to the disclosure of the present invention should be included within the scope of protection of the present invention.

Claims

1. A screen fault detection method, characterized in that, Applied to a first electronic device, the method includes: Receive multiple video frames from a high-speed camera, the multiple video frames being obtained by the high-speed camera continuously capturing images displayed by a second electronic device; The multiple video frames are processed into grayscale to obtain multiple grayscale processed video frames. Determine the similarity between any two adjacent video frames in the multiple grayscale processed video frames; Following the selection order of similarity from largest to smallest, based on the similarity between each pair of adjacent video frames, the first two similarities are selected from the similarity between multiple pairs of adjacent video frames; If it is determined that the first two similarities are both greater than a preset similarity threshold, and the absolute value of the difference between the ratio of the first two similarities and 1 is less than or equal to the preset threshold, it is determined that the screen of the second electronic device displaying the image has a screen flickering fault. If the similarity between any two adjacent video frames is less than or equal to the preset similarity threshold, or if the absolute value of the difference between the ratio of the first two similarities and 1 is greater than the preset threshold, it is determined that the screen displaying the image on the second electronic device has not experienced a screen flickering fault.

2. The method according to claim 1, characterized in that, The determination of the similarity between any two adjacent video frames in the multiple grayscale processed video frames includes: Determine the hash value of each video frame in the multiple video frames after grayscale processing; Based on the hash value of each video frame, the similarity between any two adjacent video frames is determined.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Determine the normalized luminance value of each video frame in the multiple video frames after grayscale processing; Based on the normalized brightness value of each video frame, it is determined whether the screen of the second electronic device displaying the image has a black screen fault.

4. The method according to claim 3, characterized in that, Determining whether the screen displaying the image on the second electronic device has a black screen fault based on the normalized brightness value of each video frame includes: If any normalized brightness value in each video frame is found to be less than a preset normalized brightness threshold, it is determined that the screen displaying the image on the second electronic device has a black screen fault. If the normalized brightness value of each video frame is greater than or equal to the preset normalized brightness threshold, it is determined that the screen of the second electronic device displaying the image has not experienced a black screen fault.

5. The method according to claim 1 or 2, characterized in that, The method further includes: Determine the No Reference Result Sharpness (NRSS) value for each video frame among the multiple video frames after grayscale processing; Based on the NRSS value of each video frame, it is determined whether the screen displaying the image on the second electronic device has a screen distortion fault.

6. The method according to claim 5, characterized in that, Determining whether the screen displaying the image on the second electronic device exhibits a screen flickering fault based on the NRSS value of each video frame includes: If it is determined that any NRSS value in each video frame is greater than a preset NRSS threshold, it is determined that the screen displaying the image on the second electronic device has a screen flickering fault. If the NRSS value of each video frame is determined to be less than or equal to the preset NRSS threshold, it is determined that the screen displaying the image on the second electronic device has not experienced a screen distortion fault.

7. The method according to claim 1 or 2, characterized in that, The method further includes: Determine the edge line of each video frame in the multiple video frames after grayscale processing; Based on the edge lines of each video frame, it is determined whether the screen of the second electronic device displaying the image has a garbled line fault.

8. The method according to claim 7, characterized in that, The step of determining whether the screen displaying the image on the second electronic device has a streaking fault based on the edge lines of each video frame includes: If it is determined that the brightness value of any edge line in each video frame is greater than the preset edge line brightness value, it is determined that the screen of the second electronic device displaying the image has a garbled line fault. If the brightness value of the edge line in each video frame is less than or equal to the preset edge line brightness value, it is determined that the screen of the second electronic device displaying the image has not experienced a streaking fault.

9. A first electronic device, characterized in that, include: Memory, processor, and touchscreen; among which: The touchscreen is used to display content; The memory is used to store computer programs, the computer programs including program instructions; The processor is used to invoke the program instructions to cause the first electronic device to perform the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images