Display anomaly detection method, device and system

Abstract

The application discloses a screen display abnormality detection method, device and system. The method comprises the following steps: an electronic device acquires collection data, the collection data is obtained by shooting a to-be-detected display screen of a to-be-detected device and the to-be-detected device displays example data through the to-be-detected display screen during shooting; the example data comprises one or more example images, and the collection data comprises one or more collection images; the electronic device acquires N detection images from the collection data; the detection image is an image of the example image in the collection image, and N is a positive integer; and the electronic device performs abnormality detection on the to-be-detected display screen based on the N detection images. The above method does not rely on manual detection of the display screen, can realize detection of the display screen by the electronic device, and can improve the accuracy and efficiency of detection.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a display anomaly detection method, device and system. Background Art

[0002] During industrial production or user use, display devices may experience various display abnormalities. Common display abnormalities include: screen flickering, screen stripes, screen noise, blue screen, black screen, etc.

[0003] To address these display anomaly issues, methods are needed to detect them. Current anomaly detection methods rely on manual inspection of display screens, which suffers from the following drawbacks: In industrial production, full manual inspections are prohibitively labor-intensive, forcing only spot checks to be used. The results are inaccurate and time-consuming; Furthermore, the technical capabilities of inspectors vary widely, resulting in inconsistent results, prone to errors, and low accuracy. Therefore, methods that rely on manual inspections are subject to inaccuracy and low efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a display anomaly detection method, device, and system, which can detect anomalies on a display screen through an electronic device without relying on manual detection of the display screen, thereby improving detection accuracy and efficiency.

[0005] In a first aspect, an embodiment of the present application provides a display abnormality detection method, which may include: an electronic device acquiring collected data, the collected data being obtained by photographing a display screen to be detected of a device to be detected, and the device to be detected displays example data through the display screen to be detected during the photographing; the example data includes one or more example images, and the collected data includes one or more collected images; the electronic device acquires N detection images from the collected data; the detection image is an image of the example image in the collected image, and N is a positive integer; the electronic device performs abnormality detection on the display screen to be detected based on the N detection images.

[0006] The above method does not rely on manual inspection of the display screen, but can be implemented by electronic equipment to inspect the display screen, thereby improving the accuracy and efficiency of the inspection.

[0007] In combination with the first aspect, in a possible implementation, before the electronic device obtains the collected data, the method further includes: the electronic device sends an instruction to the device to be detected, where the instruction carries sample data and is used to instruct the device to be detected to display the sample data.

[0008] In combination with the first aspect, in some embodiments, the N detection images are obtained from images in which images have changed in the collected data.

[0009] In one possible implementation, the electronic device may obtain N detection images from the collected data in the following manner: first, the electronic device obtains N images with image changes from the collected data; and second, the electronic device captures an image of an example image from each of the N images to obtain N detection images.

[0010] In another possible implementation, the electronic device captures an image of the example image from each image in the collected data to obtain M images, where M is a positive integer not less than N; then, the electronic device obtains N images with image changes from the M images to obtain N detection images.

[0011] In combination with the first aspect, in some embodiments, the one or more example images include N example images, which correspond one-to-one to N detection images. The electronic device can perform abnormality detection on the display screen to be detected based on the N detection images in the following manner: in the first step, the electronic device calculates the difference frame value between the detection image and the corresponding example image, wherein the difference frame value includes the difference between the pixel values ​​of the pixel points at the same position on the detection image and the corresponding example image; in the second step, the electronic device determines the difference between the detection image and the corresponding example image based on the difference frame value between the detection image and the corresponding example image, wherein the difference degree represents the degree of difference between the detection image and the corresponding example image; in the third step, the electronic device performs abnormality detection on the display screen to be detected based on the difference degree.

[0012] In combination with the first aspect, in some embodiments, the electronic device calculates the difference frame value between the detection image and the corresponding example image, including: the electronic device performs a difference frame value on each of the N detection images and the N example images to obtain N difference frame values; wherein the example image corresponding to the minimum difference frame value among the N difference frame values ​​is the example image corresponding to each detection image.

[0013] In combination with the first aspect, in some embodiments, the difference is determined by the variance and mean of the pixel values ​​of all pixels on the difference frame values ​​between the detection image and the corresponding example image.

[0014] In a possible implementation, the specific calculation method of the above difference can be referred to the following formula:

[0015] R=S 2 / M 2

[0016] Among them, R difference, S 2 represents the variance, and M represents the mean.

[0017] In a possible implementation, the above difference is calculated based on the difference frame value between the detection image and the corresponding example image.

[0018] In one possible implementation, the electronic device performs abnormality detection on the display screen to be detected based on the difference, including: the electronic device determines a target difference based on the differences corresponding to N detection images; the electronic device performs abnormality detection on the display screen to be detected based on a comparison result between the target difference and a preset difference.

[0019] Among them, the above-mentioned target difference is the maximum difference among the differences corresponding to the N detection images; when the target difference is greater than the preset difference, there is an abnormality in the display screen to be detected; when the target difference is not greater than the preset difference, there is no abnormality in the display screen to be detected.

[0020] The above method performs abnormality detection on the display screen to be detected based on the comparison result between the target difference and the preset difference, which can improve the accuracy of detection.

[0021] In combination with the first aspect, in some embodiments, the above-mentioned example data includes an example image, and the electronic device performs abnormality detection on the display screen to be detected based on N detection images, including: the electronic device determines the brightness of the N detection images respectively; the electronic device detects backlight flickering of the display screen to be detected according to the brightness of the N detection images.

[0022] Among them, when the brightness change rate or brightness difference between two adjacent detection images in N detection images is greater than the brightness threshold, the display screen to be detected has backlight flickering; or, when the brightness change rate or brightness difference between two adjacent images in N detection images is not greater than the brightness threshold, the display screen to be detected does not have backlight flickering.

[0023] In conjunction with the first aspect, in some embodiments, the sample data includes multiple sample images, and the electronic device performs abnormality detection on the display screen to be detected based on the N detection images, including: the electronic device calculates a correlation coefficient between the brightness of the N detection images and a benchmark; the benchmark includes the brightness of each sample image in the sample data; the correlation coefficient indicates the degree of similarity between the N detection images and the sample data; and the electronic device detects content flickering on the display screen to be detected based on the correlation coefficient;

[0024] In a possible implementation, when the correlation coefficient is greater than the correlation threshold, content flickering exists on the display screen to be detected; or when the correlation coefficient is not greater than the correlation threshold, content flickering does not exist on the display screen to be detected.

[0025] This method can realize the classification detection of backlight flicker and content flicker, which can improve the accuracy and specificity of anomaly detection.

[0026] In a possible implementation, the correlation coefficient is determined by the covariance of the brightness of the N detection images and the reference and the standard deviation of the brightness of the N detection images and the reference.

[0027] In an embodiment of the present application, the computing device determines whether there is content flickering on the display screen to be detected by comparing the correlation between the brightness of N detection images and the changing trend of the benchmark. This method can reduce the influence of external factors such as image acquisition equipment, shooting parameters and shooting environment on the brightness of the detection image, and improve the accuracy of detecting content flicker on the display screen to be detected.

[0028] In a second aspect, an embodiment of the present application provides a display abnormality detection method, which may include: an electronic device acquiring first acquired data, the first acquired data being obtained by the electronic device photographing the display screen to be detected of the device to be detected during the power-on phase of the device to be detected, and the device to be detected displays first example data through the display screen to be detected during the photographing; the first example data includes one or more example images, and the first acquired data includes one or more acquired images; the electronic device performs abnormality detection on the display screen to be detected during the power-on phase based on the first acquired data; the electronic device acquires second acquired data, the second acquired data being obtained by the electronic device photographing the display screen to be detected of the device to be detected after the device to be detected is powered on, and the device to be detected displays second example data through the display screen to be detected during the photographing; the electronic device detects display flickering of the display screen to be detected based on the second acquired data.

[0029] The above method does not rely on manual inspection of the display screen, but can be implemented by electronic equipment to inspect the display screen, thereby improving the accuracy and efficiency of the inspection.

[0030] Moreover, the above method detects abnormalities of the display screen in stages, first performing abnormality detection in the startup stage, and then monitoring display flickering after startup, which can further improve detection efficiency.

[0031] In conjunction with the second aspect, in some embodiments, the one or more example images include N1 example images, the N1 example images correspond one-to-one to N1 detection images, the N1 detection images are obtained from images in which images have changed in the first acquired data, N1 is a positive integer, and the electronic device performs abnormality detection on the display screen to be detected during the power-on phase based on the first acquired data, including:

[0032] The electronic device calculates the difference frame value between the detection image and the corresponding example image; the difference frame value includes the difference in pixel values ​​of the pixel points at the same position on the detection image and the corresponding example image; the electronic device determines the difference between the detection image and the corresponding example image based on the difference frame value between the detection image and the corresponding example image, and the difference degree indicates the degree of difference between the detection image and the corresponding example image; the electronic device performs abnormality detection on the display screen to be detected during the power-on phase based on the difference degree.

[0033] In conjunction with the second aspect, in some embodiments, the electronic device calculates a frame difference value between a detection image and its corresponding example image, including:

[0034] The electronic device performs frame difference calculation between each of the N1 detection images and the N1 example images to obtain N1 frame difference values; the example image corresponding to the minimum frame difference value among the N1 frame difference values ​​is the example image corresponding to the detection image.

[0035] In one possible implementation, the electronic device performs abnormality detection on the display screen to be detected during the power-on phase based on the difference, including: the electronic device determines a target difference based on N1 differences; the electronic device performs abnormality detection on the display screen to be detected during the power-on phase based on a comparison result between the target difference and a preset difference.

[0036] Among them, the above-mentioned target difference is the maximum difference among the differences corresponding to the N1 detection images; when the target difference is greater than the preset difference, there is an abnormality in the display screen to be detected during the power-on stage; when the target difference is not greater than the preset difference, there is no abnormality in the display screen to be detected during the power-on stage.

[0037] In combination with the second aspect, in some embodiments, the second acquired data includes first sub-acquisition data, the second example data includes a target example image, the first sub-acquisition data is obtained by the electronic device photographing the display screen to be detected of the device to be detected after the device to be detected is turned on, and the device to be detected displays the target example image through the display screen to be detected during the photographing, and the electronic device detects display flicker of the display screen to be detected based on the second acquired data, including: the electronic device obtains N2 detection images from the first sub-acquisition data, where N2 is a positive integer; the electronic device determines the brightness of the N2 detection images respectively; and the electronic device detects backlight flicker of the display screen to be detected according to the brightness of the N2 detection images.

[0038] In one possible implementation, when the brightness change rate or brightness difference between two adjacent detection images in the above-mentioned N2 detection images is greater than the brightness threshold, there is backlight flickering on the display screen to be detected; or, when the brightness change rate or brightness difference between two adjacent images in the N2 detection images is not greater than the brightness threshold, there is no backlight flickering on the display screen to be detected.

[0039] In combination with the second aspect, in some embodiments, the second acquired data includes second sub-acquisition data, the second example data includes a target example video, the second sub-acquisition data is obtained by the electronic device shooting the display screen to be detected of the device to be detected after the device to be detected is turned on, and the device to be detected displays the target example video through the display screen to be detected during shooting, and the electronic device detects display flicker of the display screen to be detected based on the second acquired data, including: the electronic device obtains N3 detection images from the second sub-acquisition data, where N3 is a positive integer; the electronic device calculates the correlation coefficient between the brightness of the N3 detection images and the benchmark; the benchmark includes the brightness of each image in the target example video; the correlation coefficient indicates the degree of similarity between the N3 detection images and the target example video; the electronic device detects the flickering of the content of the display screen to be detected according to the correlation coefficient.

[0040] In a possible implementation, when the correlation coefficient is greater than a correlation threshold, content flickering exists on the display screen to be detected; when the correlation coefficient is not greater than the correlation threshold, content flickering does not exist on the display screen to be detected.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code comprising computer instructions, the one or more processors being used to call the computer instructions so that the electronic device executes the method described in the first aspect and any possible implementation of the first aspect and the method described in the second aspect and any possible implementation of the second aspect.

[0042] In a fourth aspect, an embodiment of the present application provides a chip, which is applied to an electronic device, and the chip includes one or more processors, which are used to call computer instructions to enable the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect and the method described in the second aspect and any possible implementation of the second aspect.

[0043] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when the above-mentioned computer program product is run on an electronic device, enables the above-mentioned electronic device to execute the method described in the first aspect and any possible implementation of the first aspect and the method described in the second aspect and any possible implementation of the second aspect.

[0044] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect, and the method described in the second aspect and any possible implementation of the second aspect.

[0045] It is understandable that the electronic device provided in the third aspect, the chip provided in the fourth aspect, the computer program product provided in the fifth aspect, and the computer storage medium provided in the sixth aspect are all used to execute the methods provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagrams of several images involved in the embodiments of this application;

[0047] Figure 2A A schematic diagram of a display anomaly detection system provided in an embodiment of the present application;

[0048] Figure 2B A schematic diagram of another display anomaly detection system provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0050] Figure 4 A flowchart of a first embodiment of a display anomaly detection method provided in an embodiment of the present application;

[0051] Figure 5 A flowchart of a second embodiment of a display anomaly detection method provided in an embodiment of the present application;

[0052] Figure 6A A schematic diagram of a process flow of a third embodiment provided in the present application;

[0053] Figure 6B A schematic diagram of an image acquisition window provided in an embodiment of the present application;

[0054] Figure 6C A schematic diagram of a Gaussian filter involved in an embodiment of the present application;

[0055] Figure 6D A schematic diagram of a connected area topology structure involved in an embodiment of the present application;

[0056] Figure 7 A flowchart of a method for calculating a preset difference provided in an embodiment of the present application;

[0057] Figure 8A A schematic diagram of a flow chart of a fourth embodiment provided in the present application;

[0058] Figure 8B A schematic diagram of a flow chart of a method for detecting backlight flickering provided in an embodiment of the present application;

[0059] Figure 8C A flowchart of a method for detecting content flickering provided in an embodiment of the present application;

[0060] Figure 9 A flowchart of embodiment 5 provided in the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0062] To facilitate understanding of the embodiments of the present application, some terms involved in the present application are introduced below.

[0063] The following combination Figure 1 , several images involved in this application are described:

[0064] (1) Sample data

[0065] The sample data can be a sample image or a sample video. The sample image is the original image displayed on the display screen during the inspection. Typically, the sample image is sent by the computing device to the device to be inspected, and the device to be inspected displays the sample image on its display screen.

[0066] The sample video is the original video displayed by the display screen to be detected when the display screen to be detected is detected. Generally, the sample video can be sent by the computing device to the device to be detected, and the device to be detected displays the sample video through its display screen to be detected.

[0067] It should be understood that the example image may also be an image included in the device to be detected or the computing device, such as a startup display image.

[0068] (2) Collect images and videos

[0069] The device to be tested displays a sample image on the display screen to be tested, and the image acquisition device captures the display screen to be tested using a camera. The captured image is referred to as a captured image, and the captured video is referred to as a captured video. It is understood that the captured image includes an image of the sample image. In the embodiments of the present application, the captured image and the captured video are collectively referred to as captured data.

[0070] (3) Detection images and videos

[0071] The detection image is an image of an example image intercepted from the collected image. It can be understood that the detection image is an image of the example image in the collected image, and the abnormality of the display screen can be analyzed through the detection image.

[0072] A video consisting of multiple continuous detection images is also called a detection video.

[0073] (4) Normal display of images and videos

[0074] A display screen without any abnormalities (also referred to as a normal display screen) displays a sample image. The image acquisition device captures the normal display screen via a camera. An image of the sample image captured from the captured image is referred to as a normal display image. It is understood that the normal display image is the sample image in the captured image. By comparing and analyzing the normal display image with the test image, abnormalities in the display screen to be tested can be determined.

[0075] Similarly, a display screen without any abnormalities (also called a normal display screen) displays an example video. The image acquisition device uses a camera to shoot the normal display screen, and the example video captured from each frame of the shot video can be called a normal display video.

[0076] This application is intended to detect abnormalities in the display screen of a device to be tested (also referred to as the display screen to be tested). For example, whether the display screen is abnormal; another example, whether the display screen has flickering abnormalities, such as backlight flickering and content flickering; another example, whether the display screen has other abnormalities, such as stripes, noise, ghosting, no dynamic logo, blue screen, black screen, etc.

[0077] An embodiment of the present application provides a display anomaly detection method, which includes: first, the device to be detected displays sample data through the display screen to be detected, and the display screen to be detected can be photographed to obtain collected data; the above-mentioned sample data can include one or more sample images; the above-mentioned collected data can include one or more collected images. Then, based on the above-mentioned collected data, N detection images are obtained, and the above-mentioned detection images are images of the sample images in the collected images, where N is a positive integer. Next, based on the above-mentioned N detection images, anomaly detection is performed on the display screen to be detected. The specific implementation of the display anomaly detection method provided by the present application can be referred to the following embodiments one to five, which will not be repeated here.

[0078] The above method does not rely on manual inspection of the display screen, but can be implemented by electronic equipment to inspect the display screen, thereby improving the accuracy and efficiency of the inspection.

[0079] In some embodiments, the abnormality of the display screen can be detected in stages. First, abnormality detection is performed during the power-on stage, and then display flickering is monitored after power-on, which can improve detection efficiency.

[0080] The following describes the devices and systems involved in the embodiments of the present application.

[0081] The display abnormality detection method in the embodiment of the present application can be implemented solely by an electronic device including a camera, or can be implemented by a system consisting of an image acquisition device and a computing device.

[0082] In the present application, the connection between the computing device and the device to be detected can be established through short-range wireless communication such as Bluetooth, WLAN, etc., or through mobile communication or wired connection.

[0083] Figure 2A and Figure 2B A display anomaly detection system provided in an embodiment of the present application is a system diagram including a device to be detected 10 , an image acquisition device 11 and a computing device 12 .

[0084] In one scenario, Figure 2A As shown, the image acquisition device 11 and the computing device 12 establish a communication connection. For example, the device to be detected 10 is a computer, mobile phone, tablet computer, television, etc. that includes a display screen to be detected and needs to be detected, the image acquisition device 11 is an electronic device with a camera, such as a camera, a webcam, a mobile phone, a tablet computer, etc., and the computing device 12 can be a server, or it can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc. The image acquisition device 11 or the computing device 12 can cooperate with each other to run the display anomaly detection method provided in this application. The device to be detected 10 can establish a communication connection with the image acquisition device 11 or the computing device 12, and the image acquisition device 11 or the computing device 12 performs display control on the device to be detected 10 so that the device to be detected 10 displays an example image through the display screen to be detected. The device to be detected 10 can also be controlled by the user to display the example image.

[0085] The device to be tested 10 includes a display screen to be tested, which displays a sample image on the display screen to be tested. In this application, it is necessary to detect whether the display screen to be tested is abnormal and what kind of abnormality it is. When testing the display screen to be tested, it is required that the display screen to be tested is in an operating state, that is, displaying a sample image or sample video.

[0086] The image acquisition device 11 includes at least one camera, which captures the device to be detected 10 in real time through the camera to obtain a captured image. The captured image includes a captured image of the display screen of the device to be detected, and the captured image of the display screen includes a captured image of the example image. The captured image of the example image can be obtained from each captured image, which is referred to as a detection image. In other words, the detection image is a captured image of the example image in the captured image.

[0087] In one implementation, the image acquisition device 11 can send the captured image to the computing device 12. The computing device 12 has computing capabilities and can obtain a detection image based on the captured image, and then identify abnormal conditions of the display screen to be detected based on the detection image.

[0088] In another implementation, the image acquisition device 11 may acquire a detection image based on the captured image, and then send the detection image to the computing device 12 . The computing device 12 may identify abnormal conditions of the display screen to be detected based on the detection image.

[0089] In another implementation, the image acquisition device 11 may also acquire a detection image based on the captured acquisition image and independently execute the display abnormality detection method provided in this application.

[0090] In addition, the image acquisition device 11 can perform display control on the device to be detected 10 through a communication connection. It should be understood that the display control includes the image acquisition device 11 controlling the device to be detected 10 to display an example image or an example video.

[0091] Optionally, when an abnormality is detected in the display screen to be detected, the computing device 12 or the image acquisition device 11 may also record, extract, and export abnormal information, and further, may upload the abnormal information to the background server.

[0092] It should be noted that the image acquisition device 11 and the computing device 12 may be different components of the same device or different devices.

[0093] In another scenario, Figure 2B As shown, a communication connection is established between the device to be tested 10 and the image acquisition device 11. For example, the device to be tested 10 is a computer, mobile phone, tablet computer, television, etc. that needs to be tested, and the image acquisition device 11 is an electronic device with a camera, such as a camera, mobile phone, tablet computer, etc. The device to be tested 10 can execute some or all of the steps of the display abnormality detection method provided in this application.

[0094] In some embodiments, the device to be detected 10 can establish a communication connection with the computing device 12. The device to be detected 10 can obtain a captured image of the display screen of the device to be detected 10 through the image acquisition device 11, and then send the captured image or the detection image obtained by processing the captured image to the computing device 12. The computing device 12 performs abnormality detection on the display screen of the device to be detected based on the above-mentioned received image.

[0095] Optionally, after the device to be detected 10 acquires the captured image through the image acquisition device 11, it can also independently execute the display abnormality detection method provided in this application.

[0096] Optionally, when an abnormality is detected in the display screen to be detected, the computing device 12 or the device to be detected 10 may also record, extract, and export abnormal information, and further, may upload the abnormal information to the background server.

[0097] It should be noted that the communication connection between the above-mentioned image acquisition device 11 and the computing device 12, the communication connection between the device to be detected 10 and the image acquisition device 11, and the communication connection between the device to be detected 10 and the computing device 12 can all be achieved through short-range wireless communication such as Bluetooth, WLAN, etc., or through mobile communication, or through wired connection, such as USB connection.

[0098] In some embodiments, the background server may be an independent server or the computing device 12 .

[0099] It should also be noted that the specific implementation of the display abnormality detection method provided in this application can be found in the following embodiments 1 to 5, which will not be repeated here.

[0100] Figure 3 This is a hardware structure diagram of an electronic device 100 provided in an embodiment of the present application. It can also be the aforementioned device to be detected 10, image acquisition device 11, and computing device 12.

[0101] The electronic device 100 may be a portable electronic device, such as a mobile phone, a tablet computer, a smart watch, a smart mirror, or the like. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The portable electronic device may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touchpad). It should also be understood that, in some embodiments, the electronic device may also be a desktop computer.

[0102] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0103] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0104] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0105] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0106] It is understood that the processor 110 may also include an AE system. The AE system may be specifically provided in the ISP. The AE system may be used to automatically adjust exposure parameters. Optionally, the AE system may also be integrated into other processor chips. This embodiment of the present application is not limited to this.

[0107] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0108] In some embodiments, the processor 110 may include one or more interfaces. The USB interface 130 is an interface that complies with USB standards and specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect to other electronic devices 100, such as AR devices.

[0109] The charging management module 140 is configured to receive charging input from a charger. While charging the battery 142 , the charging management module 140 can also provide power to the electronic device 100 through the power management module 141 .

[0110] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0111] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0112] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0113] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via the antenna 1.

[0114] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium- or high-frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs sound signals through an audio device (including but not limited to the speaker 170A, the receiver 170B, etc.) or displays images or videos through the display screen 194.

[0115] The wireless communication module 160 can provide wireless communication solutions including Wireless Local Area Networks (WLAN) (such as Wireless Fidelity (Wi-Fi) network), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0116] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0117] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0118] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), organic light-emitting diode (OLED), active-matrix organic light-emitting diode or active-matrix organic light-emitting diode (AMOLED), flexible light-emitting diode (FLED), Mini LED, Micro LED, Micro-OLED, quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0119] The electronic device 100 can implement the acquisition function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0120] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image or video. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0121] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then passed to the ISP for conversion into a digital image or video signal. The ISP outputs the digital image or video signal to the DSP for processing. The DSP converts the digital image or video signal into an image or video signal in a standard format such as RGB or YUV.

[0122] The digital signal processor is used to process digital signals. In addition to processing digital images or video signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0123] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0124] The NPU is a neural network (NN) computing processor that rapidly processes input information and continuously self-learns by drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain. The NPU enables intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0125] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0126] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image and video playback function, etc.). The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.).

[0127] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0128] The audio module 170 is used to convert digital audio information into analog audio signals for output, and is also used to convert analog audio input into digital audio signals.

[0129] The speaker 170A, also called a "horn", is used to convert audio electrical signals into sound signals.

[0130] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals.

[0131] The microphone 170C, also called a "microphone" or "speaker", is used to convert sound signals into electrical signals. The electronic device 100 may be provided with at least one microphone 170C.

[0132] The headphone jack 170D is used to connect a wired headphone.

[0133] The sensor module 180 may include one or more sensors, which may be of the same type or of different types. Figure 1 The sensor module 180 shown is only an exemplary division method. There may be other division methods, which are not limited in this application.

[0134] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation using pressure sensor 180A. Electronic device 100 can also calculate the location of the touch based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different intensities can correspond to different operation instructions.

[0135] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can also be used for image stabilization.

[0136] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0137] The magnetic sensor 180D includes a Hall sensor, and the electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case.

[0138] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic device 100, enabling applications such as switching between landscape and portrait modes and pedometers.

[0139] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0140] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100.

[0141] The ambient light sensor 180L is used to sense the brightness of the ambient light.

[0142] The fingerprint sensor 180H is used to obtain fingerprints.

[0143] The temperature sensor 180J is used to detect temperature.

[0144] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0145] The bone conduction sensor 180M can acquire vibration signals.

[0146] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0147] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0148] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0149] The SIM card interface 195 is used to connect a SIM card. A SIM card can be connected to and disconnected from the electronic device 100 by inserting or removing it from the SIM card interface 195. The electronic device 100 may support one or N SIM card interfaces, where N is a positive integer greater than 1. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, or embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0150] Several display anomaly detection methods involved in the embodiments of the present application are introduced below.

[0151] The display anomaly detection method provided in the embodiments of the present application can be implemented individually or in combination by the above-mentioned device to be detected, image acquisition device, or computing device. The method can be used to detect display anomalies during the production process, and can also be used to detect display anomalies during user use. The method can be deployed on different devices and can detect display anomalies in different scenarios in stages and categories. The following five embodiments are introduced separately:

[0152] Example 1

[0153] The staged display anomaly detection can include the power-on stage and the post-power-on stage. Among them, the interface changes of the device to be detected during the power-on process are fixed, and it usually displays the power-on logo (logo), which can also be called the power-on logo stage. The post-power-on stage can be divided into a static picture stage and a dynamic picture stage based on the type of abnormality to be detected. Among them, the static picture stage detects whether there is an abnormality of backlight flickering by letting the display screen to be detected display an example image of a static image; the dynamic picture stage detects whether there is an abnormality of content flickering by letting the display screen to be detected display a dynamic example video.

[0154] Specifically, if Figure 4 As an example, a flowchart of an anomaly detection method is shown. The method may include but is not limited to some or all of the following steps:

[0155] S401: When the device to be detected is powered on and started, a startup interface is displayed on the display screen to be detected.

[0156] S402: The computing device performs display anomaly detection on the display screen to be detected during the startup phase based on the captured startup interface.

[0157] When the device to be detected is powered on and turned on, the computing device or image acquisition device can capture its boot interface and obtain the captured image, so as to perform display anomaly detection during the boot phase based on the captured image.

[0158] The display abnormality detection method in the booting stage may adopt the following embodiment 3 or embodiment 5. For details, please refer to the embodiments described in the following embodiment 3 or embodiment 5, which will not be repeated here.

[0159] When the boot interface is a static image, the display abnormality detection method during the boot phase may also adopt the method for detecting backlight flickering in the following fourth embodiment. For details, please refer to the relevant description in the following fourth embodiment, which will not be repeated here.

[0160] When the boot interface is a dynamic image, the display abnormality detection method during the boot phase may also adopt the method of detecting content flickering in the following fourth embodiment. For details, please refer to the relevant description in the following fourth embodiment, which will not be repeated here.

[0161] S403: The computing device determines whether the display screen to be detected is abnormal during the startup phase.

[0162] When there is an abnormality in the display screen to be detected during the power-on phase, S410 is executed, and a prompt message indicating that the display screen to be detected is abnormal can also be output; otherwise, when there is no abnormality, display abnormality detection can be performed in the post-power-on phase, and S405 or S408 can be executed.

[0163] S404: After the device to be detected is powered on, a sample image is displayed on the display screen to be detected.

[0164] The computing device or the image acquisition device can capture the display screen to be inspected to obtain a captured image.

[0165] S405: The computing device performs display anomaly detection in the static screen stage based on the collected image.

[0166] After the device to be tested is turned on, a sample image can be displayed on the display screen to be tested. When the display screen to be tested displays the sample image, the image remains static, so this stage can be called the static image stage.

[0167] Among them, the display abnormality detection method in the static picture stage can adopt the method for detecting backlight flicker in the following embodiment 4 or embodiment 5. For details, please refer to the embodiments described in the following embodiment 4 or embodiment 5, which will not be repeated here.

[0168] S406: The computing device determines whether the display screen to be detected is abnormal in the static image stage.

[0169] When there is an abnormality in the display screen to be detected in the static picture stage, S410 is executed, and a prompt message indicating that the display screen to be detected is abnormal can also be output; conversely, when there is no abnormality, display abnormality detection can be performed in the dynamic picture stage, S408 is executed, and the process can also be ended.

[0170] S407: After the device to be tested is powered on, a sample video is displayed on the display screen to be tested.

[0171] The computing device or image acquisition device can capture the display screen to be detected to obtain a captured video.

[0172] S408: The computing device performs display anomaly detection in the dynamic picture stage based on the collected video.

[0173] After the device is powered on or after the static screen phase, the device can display a sample video on the display screen. When the display screen displays the sample video, the image changes dynamically, so this phase can be called the dynamic image phase.

[0174] Among them, the display abnormality detection method in the dynamic picture stage can adopt the method for detecting content flickering in the following embodiment 4 or embodiment 5. For details, please refer to the embodiments described in the following embodiment 4 or embodiment 5, which will not be repeated here.

[0175] S409: The computing device determines whether the display screen to be detected is abnormal during the dynamic image stage.

[0176] When there is an abnormality in the display screen to be detected in the dynamic picture stage, S410 is executed, and a prompt message indicating that the display screen to be detected is abnormal can also be output; conversely, when there is no abnormality, display abnormality detection can be performed in the static picture stage, S406 is executed, and the process can also be ended.

[0177] S410: The computing device records abnormal information.

[0178] The abnormality information may include at least one of the type of abnormality of the display screen to be detected, an abnormal image, an abnormal video, etc. The abnormal image is a captured image or a detection image in which the abnormality of the display screen to be detected is detected. The abnormal video is a sample image or a detection image before and after the captured image in which the abnormality of the display screen to be detected is detected, or it may be a detection video in which the abnormality of the display screen to be detected is detected.

[0179] In one implementation, when the computing device detects an abnormality in the display screen to be detected, it can record, extract, export, and upload the abnormality information, wherein the uploading can be to a backend server.

[0180] The computing device can send abnormal information to the device to be detected, and can also send control instructions to the device to be detected. The device to be detected responds to the control instructions and stores a log of the display screen to be detected. The log can also be sent to the computing device or a backend server. The log can be abnormal information of the device to be detected.

[0181] The first embodiment described above can detect display anomalies in stages. On the one hand, this method does not rely on manual testing of the display device; the electronic device can perform the testing of the display screen. On the other hand, it can promptly detect display anomalies in the display device, and when the computing device detects an anomaly in the display device, it can report the necessary data information to the anomaly information processing center. This can solve the problems of inconvenience, inaccuracy, and low efficiency.

[0182] Example 2

[0183] The display abnormality detection by category may include preliminary detection of display abnormality, detection of backlight flickering, detection of content flickering, and detection of other types of abnormalities, etc. For different types of display abnormalities, the electronic device may adopt different detection methods.

[0184] Specifically, if Figure 5 As an example, a flowchart of an anomaly detection method is shown. The method may include but is not limited to some or all of the following steps:

[0185] S501: The computing device preliminarily determines whether there is an abnormality on the display screen to be detected.

[0186] The computing device may use the following third or fifth embodiment to preliminarily detect whether the display screen to be detected is abnormal. For details, please refer to the embodiments described in the following third or fifth embodiment, which will not be repeated here.

[0187] If the display screen to be tested has an abnormality, the computing device executes S504 to record the abnormality information. Alternatively, the computing device may detect backlight flickering or content flickering and execute S502 or S503. Alternatively, a prompt message indicating that the display screen to be tested has an abnormality may be output. Conversely, if the display screen to be tested does not have an abnormality, a prompt message indicating that the display screen to be tested has no abnormality may be output, or the process may end.

[0188] S502: The computing device detects whether there is backlight flickering on the display screen to be detected.

[0189] Among them, the computing device can use the method for detecting backlight flicker in the following embodiment 4 or embodiment 5 to detect whether there is backlight flicker on the display screen to be detected. For details, please refer to the embodiments described in the following embodiment 4 or embodiment 5, which will not be repeated here.

[0190] If the display screen to be tested has backlight flickering, the computing device executes S504, where the computing device records abnormality information. Alternatively, the computing device may detect content flickering and execute S503. Furthermore, the computing device may output a prompt indicating abnormality in the display screen to be tested. Conversely, if the display screen to be tested does not have backlight flickering, the computing device may detect content flickering and execute S503. Alternatively, the computing device may output a prompt indicating that the display screen to be tested does not have backlight flickering abnormality. Alternatively, the computing device may terminate the process.

[0191] S503: The computing device detects whether there is content flickering on the display screen to be detected.

[0192] The computing device may use the method for detecting content flickering in the following embodiment 4 or embodiment 5 to detect whether content flickering exists on the display screen to be detected. For details, please refer to the embodiments described in the following embodiment 4 or embodiment 5, which will not be repeated here.

[0193] If the display screen to be tested has content flickering, the computing device executes S504, where the computing device records abnormal information. Alternatively, backlight flicker detection may be performed, and S502 may be performed. Furthermore, a prompt indicating abnormality in the display screen to be tested may be output. Conversely, if the display screen to be tested does not have content flickering, backlight flicker detection may be performed, and S502 may be performed. Alternatively, a prompt indicating that the display screen to be tested has no content flickering abnormality may be output. Alternatively, the process may end.

[0194] S504: The computing device records abnormal information.

[0195] The specific implementation of recording abnormal information by the computing device can refer to the embodiment described in S410 of the above embodiment 1, which will not be repeated here.

[0196] In some embodiments, the above steps S501 to S504 may be executed after the device to be detected is powered on. In other embodiments, S501 may be executed during the power-on process of the device to be detected, and S502 and S503 may be executed after the device to be detected is powered on.

[0197] The second embodiment described above can categorize display anomalies and detect them, achieving more accurate detection of backlight flicker and content flicker on the display screen. On the one hand, this method does not rely on manual testing of the display device; instead, electronic devices can perform the testing of the display screen. On the other hand, it can promptly detect display anomalies on the display device, and when the computing device detects an anomaly on the display device, it can report the necessary data to the anomaly information processing center. This can solve the problems of inconvenience, inaccuracy, and low efficiency.

[0198] Example 3

[0199] like Figure 6A As shown in the flowchart of Example 3, the method may include but is not limited to some or all of the following steps:

[0200] S601: The device to be tested displays sample data on the display screen to be tested. The sample data may be a sample image or a sample video. For example, the sample data may be a video displayed during the startup logo phase.

[0201] S602: The computing device obtains collected data, where the collected data is obtained by photographing the display screen to be inspected.

[0202] For the sample data displayed on the display screen to be inspected, the captured data can be a captured image or a captured video. If the sample data is a sample image, the captured data is a captured image; if the sample data is a sample video, the captured data is a captured video.

[0203] S603: The computing device obtains one or more detection images from the collected data.

[0204] When the example data is an example image, the collected data is an collected image. In this case, an image of the example image can be captured from the collected image to obtain a detection image.

[0205] When the sample data is a sample video, the collected data is the collected video. In this case, multiple images with image changes can be found from the collected video, referred to as key data frames. Then, images of the sample image are captured from each key data frame to obtain multiple detection images. In another implementation, an image of the sample video can be captured from each frame of the collected video to obtain multiple continuous images. Multiple images with image changes can then be obtained from the multiple continuous images, which are the multiple detection images required for subsequent detection.

[0206] Similarly, when the example data is an example image, the example image is a key data frame of the example data, and can also be called an example image corresponding to the detection image.

[0207] When the sample data is a sample video, multiple images with image changes can be found from the sample video, which are a set of key data frames of the sample video. The key data frames correspond one-to-one to the above-mentioned detection images.

[0208] The following describes a specific method for obtaining key data frames from a sample video. The method may include but is not limited to some or all of the following steps:

[0209] 1. Establish a keyframe pool.

[0210] 2. Put the first frame data in the sample video into the key frame pool.

[0211] 3. Select the next frame data of the sample video and make a difference between it and the key data frame in the key frame pool.

[0212] Specifically, subtracting two data frames means subtracting the pixel values ​​of the pixels at the same location in the two frames. This means the subtracted value includes the pixel value differences corresponding to each pixel. For example, if A1 and B1 are two pixels on one frame, and A2 and B2 are two pixels on the other frame, where A1 and A2 have the same coordinates in both frames, and B1 and B2 have the same coordinates in both frames, the subtracted value for the two frames includes the pixel value differences corresponding to pixel A1 / A2 and pixel value differences corresponding to pixel B1 / B2.

[0213] 4. If the keyframe pool includes multiple keyframes, the calculation in step 3 yields a set of difference frame values. The difference frame with the smallest average pixel value difference is taken as the minimum difference frame value. The maximum pixel value difference within this minimum difference frame value is then searched. If this maximum pixel value difference is greater than a threshold, such as 16, the frame data is placed in the keyframe pool. The value 16 is an empirical value indicating that the image of this data frame has changed from the previous frame data and can be determined to be a keyframe.

[0214] 5. Repeat step 3 above until you reach the last frame of the sample video.

[0215] Similarly, the specific method for obtaining key data frames from the captured video is the same as the specific method for obtaining key data frames from the sample video, which will not be repeated here. The key data frames of the sample data and the key data frames of the captured video can be obtained by the above-mentioned frame difference method.

[0216] In some embodiments of the present application, when the example data is an example image, the step of obtaining a key data frame may not exist.

[0217] S604: The computing device respectively calculates the difference frame values ​​between the detection image and its corresponding example image.

[0218] When the sample data is a sample image, a test image can be obtained from the collected data. The sample image corresponds to the test image. The corresponding sample data and the test image are subjected to a frame difference, that is, the pixel values ​​of the corresponding pixels on the two images are respectively subtracted.

[0219] When the sample data is a sample video, multiple detection images can be obtained from the collected data. In this case, a set of key data frames, also referred to as multiple sample images, can also be obtained from the sample video. For example, N detection images and N sample images are used as an example, where N is a positive integer. Next, for a detection image, frame difference values ​​are calculated between the detection image and the multiple sample images to obtain a set of frame difference values. The minimum frame difference value in this set of frame difference values ​​is then selected, and the sample image corresponding to the minimum frame difference value is the sample image corresponding to the detection image.

[0220] The above method for calculating the difference frame value can be referred to formula (1). Since the difference value can be positive or negative, the embodiment of the present application determines the absolute value of the difference value as the difference frame value, as shown in formula (1):

[0221]

[0222] Among them, F i is the i-th detection image among N detection images; is the jth example image among N example images; i, j are both positive integers, not greater than N. ΔFi，j is the frame difference between the i-th detection image and the j-th example image.

[0223] S605: The computing device determines the difference between the detection image and its corresponding example image based on the difference frame value corresponding to the detection image.

[0224] The difference degree is used to represent the degree of difference between the detected image and its corresponding example image. In some embodiments, the difference degree can be represented by an R value.

[0225] For example, if the sample data includes N sample images, N detection images can be obtained, where N is a positive integer. A difference frame value can be obtained from the detection image and its corresponding sample image, and an R value can be calculated based on the pixel values ​​of all pixels in this difference frame value. Thus, N R values ​​can be obtained from the N detection images.

[0226] The calculation method of R value can be referred to formula (2) and formula (3). In the embodiment of the present application, the difference between the detection image and its corresponding example image is represented by R value. The specific calculation method of R value is shown in formula (2):

[0227] R=S 2 / M 2 (2)

[0228] Among them, S 2 It represents the variance of the pixel values ​​of all pixels in a difference frame value. The calculation formula is as follows:

[0229]

[0230] Among them, x i is the pixel value of the i-th pixel in the difference frame value, M is the average pixel value of all pixels in the difference frame value, and N is the number of pixels in the difference frame value. The variance mainly describes the degree of dispersion of the pixel values ​​of all pixels in the difference frame value relative to their average value. That is, the larger the variance, the greater the data fluctuation; the smaller the variance, the smaller the data fluctuation.

[0231] M 2 It is the square of the average value of the pixel values ​​of all pixels in the difference frame value, which can reflect the overall difference of all pixels in the difference frame value and represent its average level.

[0232] S606: The computing device determines whether there is an abnormality in the display screen to be detected based on a comparison result of the difference degree with a preset difference degree.

[0233] For example, if the sample data includes N sample images, N detection images can be obtained, where N is a positive integer. N R values ​​can be obtained from the N detection images.

[0234] In some implementations, the largest R value selected from the N R values ​​is the maximum difference. Further, based on the comparison result of the maximum difference and the preset difference, it is determined whether the display screen to be inspected has an abnormality.

[0235] In other implementations, the average value, minimum value or any one value of the N R values ​​may be used as the target difference, and then, based on the comparison result between the target difference and the preset difference, it is determined whether the display screen to be inspected has an abnormality.

[0236] The preset difference degree can be a fixed value, for example, 0.0025. Alternatively, different preset difference degrees can be assigned to different models of devices to be detected, image acquisition devices, and shooting environments. In this case, the image acquisition device is required to send information such as the model of the device to be detected, the model of the image acquisition device, and the shooting environment to the computing device. Shooting environments may include daytime, nighttime, indoor, and outdoor environments.

[0237] The following uses the maximum difference as an example to introduce several specific implementation methods for determining whether the display screen to be inspected has an abnormality based on the maximum difference and the preset difference. It should be understood that the target difference can also be determined using the following methods:

[0238] Implementation method 1

[0239] In some embodiments, the rate of change of the maximum difference relative to the preset difference can be calculated to determine whether the display screen to be detected has an abnormality. Specifically, as shown in formula (4),

[0240] P=|R1-R2| / R2 (4)

[0241] Here, R1 represents the maximum difference, R2 represents the preset difference, and P represents the rate of change of the maximum difference relative to the preset difference. A change threshold, O, can be set to represent the difference between the maximum difference and the preset difference. If P is greater than O, the display under test is considered abnormal; otherwise, the display under test is not abnormal.

[0242] Implementation method 2

[0243] In other embodiments, the maximum difference may be directly compared with a preset difference. If the maximum difference is greater than the preset difference, it is determined that the display screen to be detected has an abnormality; otherwise, no abnormality exists.

[0244] Implementation method three

[0245] In some other embodiments, it is also possible to determine whether the difference between the maximum difference and the preset difference is greater than a preset value. If the difference between the maximum difference and the preset difference is greater than the preset value, it is determined that there is an abnormality in the display screen to be detected. Otherwise, there is no abnormality. In some embodiments, the preset difference can be the same or different for different devices. For example, the computing device can also obtain an identifier of the device to be detected, which is used to indicate the model of the device to be detected. The computing device can store preset differences corresponding to multiple models. When determining whether a display abnormality is present based on the maximum difference and the preset difference, the preset difference can use the preset difference corresponding to the model of the device to be detected.

[0246] Taking the example of obtaining a detection image from a captured image, the present invention performs edge detection on the captured image to obtain an edge image, finds the largest connected region in the edge image, and then uses the image within the largest connected region in the captured image as the detection image.

[0247] like Figure 6B As shown in the figure, A and C are the background areas of the captured image, and B and D are the display areas of the display screen to be tested. The captured image should include the display area of ​​the display screen to be tested, but there are no strict requirements on the placement of the display screen to be tested.

[0248] In some embodiments, the edge detection algorithm may be a Canny edge detection algorithm. The implementation of the Canny edge detection algorithm may include the following four steps:

[0249] Step 1: Gaussian filtering.

[0250] Step 2: Calculate the gradient strength and direction.

[0251] Step 3: Non-maximum suppression.

[0252] Step 4: Double threshold algorithm detects and connects edges.

[0253] The following explains them separately.

[0254] Gaussian filtering:

[0255] Gaussian filtering is to perform weighted averaging based on the pixel values ​​of the pixel to be filtered and the surrounding pixels through a Gaussian filter, and replace the original pixel value with the weighted average pixel value of each pixel to obtain a new pixel value distribution. Figure 6C The figure below shows a commonly used Gaussian filter. The numbers in the Gaussian filter indicate the weight of the pixel at that location. Overall, pixels in the center have a higher weight, while pixels in the periphery have a lower weight. The weights indicate that the farther pixels are from the center, the less they contribute to the pixel value at the center.

[0256] In this embodiment, a pixel to be filtered is selected from the captured image. The pixel values ​​of the pixel to be filtered and its surrounding pixels are weighted averaged through a Gaussian filter. The resulting weighted average pixel value for each pixel is the new pixel value for that pixel. The edge region can be determined based on the pixel weights in the Gaussian filter.

[0257] Compute the gradient magnitude and gradient direction:

[0258] In the embodiment of the present application, the purpose of calculating the gradient of the captured image is to highlight the features of the edge portion of the captured image, and the biggest feature of the edge portion of the image is the drastic change in pixel values.

[0259] In actual calculations, different operators are usually used to perform convolution operations on the image to calculate the image gradient. The operator in the Canny edge detection algorithm can be the Sobel operator. It should be understood that the operator here can be a set of values ​​in different directions, and the Sobel operator is a set of discrete difference values. The specific calculation of the gradient strength and direction is shown in Formula (5) and Formula (6):

[0260]

[0261]

[0262]

[0263] Among them, S x Represents the Sobel operator of the acquired image in the X direction. S y Represents the Sobel operator of the acquired image in the Y direction. G x Indicates the gradient intensity of the collected image in the X direction, which can be represented by S x The convolution operation is performed on the collected image; G y Indicates the gradient intensity of the acquired image in the Y direction, which can be represented by S y Get; G (x，y) Represents the gradient strength of the acquired image. (x，y) Indicates the gradient direction of the acquired image.

[0264] The gradient image of the acquired image can be determined based on the calculated gradient strength and gradient direction of the acquired image. At this time, the obtained gradient image may have problems such as rough image edges and unclear edge details. Therefore, further non-maximum suppression is required to make the edge lines thinner and the edge details more obvious.

[0265] Non-maximum suppression:

[0266] The purpose of non-maximum suppression is to suppress pixels with weak gradients in the gradient image corresponding to the acquired image, retaining only the maximum gradient, thereby achieving a thin edge. Non-maximum suppression is achieved using a non-maximum suppression algorithm, which considers the calculated gradient strength and gradient direction of the acquired image. If only the maximum value of the center pixel obtained by Gaussian filtering is considered, image noise may be introduced.

[0267] Dual threshold algorithm to detect and connect edges:

[0268] The dual-threshold algorithm first identifies strong edges, weak edges, and non-edge pixels in the image after non-maximum suppression. For weak edges, the algorithm calculates the number of strong edge pixels within their area and determines whether to convert them to strong edges for edge complementation. The resulting detected image is the edge image corresponding to the acquired image obtained after Canny edge detection.

[0269] After obtaining the edge image, the maximum connected area in the edge image can be obtained; then, the image in the largest connected area in the acquired image is used as the detection image. The following describes the specific implementation of obtaining the maximum connected area from the edge image:

[0270] In the edge image obtained by Canny edge detection, the pixel values ​​corresponding to the edge pixels are generally greater than 0, indicating that they belong to the non-background area. The pixel values ​​corresponding to the background pixels are 0. In this case, a border with a pixel value of 0 can be added to the outermost area of ​​the edge image. This is usually described as padding the outermost area of ​​the edge image. The added border belongs to the background area of ​​the edge image.

[0271] Furthermore, we search for connected regions in the edge image, such as Figure 6D , exemplarily showing various connected areas in the edge image, including connected area A, connected area B, connected area C, and connected area D.

[0272] Then, determine the connectivity between each connected area. The connectivity can include surrounding and non-surrounding. If each pixel point in a connected area (also called the first connected area) needs to pass through another connected area (also called the second connected area) on the path from all directions to the border, and one of the two connected areas belongs to the background area and the other belongs to the non-background area, then the second connected area surrounds the connected area. When two connected areas are adjacent, the surround is also called a straight surround. Figure 6D As shown, connected region A directly surrounds connected region B, and connected region B directly surrounds connected region C. Connected region B can be regarded as a hole in connected region A, and the boundary between connected region A and connected region B can also be called a hole boundary.

[0273] Furthermore, the parent boundary of each connected area can be determined based on the connectivity between the connected areas; and then, the connected area with the largest circumscribed rectangular frame area among the connected areas whose parent boundary is the frame is determined as the largest connected area.

[0274] Among them, the parent boundary of a connected region is the outer boundary of the connected region that directly surrounds the region; the circumscribed rectangle of a connected region is the minimum rectangle that surrounds the connected region, which can be determined based on the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate, and minimum vertical coordinate of each vertex on the connected region. Figure 6D As shown, the parent boundaries of connected regions D and A are both borders. The parent boundary of connected region C is the hole boundary between connected regions A and B, that is, the outer boundary of connected region B. The parent boundary of connected region B is the outer boundary of connected region A. It should be understood that among the above parent boundaries, the border is the largest parent boundary, and the parent boundaries of connected regions A and D are borders. In addition, the circumscribed rectangle area of ​​connected region A is larger than the circumscribed rectangle area of ​​connected region D. Therefore, connected region A can be determined to be the largest connected region.

[0275] The following describes a method for calculating a preset difference degree according to an embodiment of the present application.

[0276] like Figure 7 As shown in the example, a flowchart of a method for calculating a preset difference provided in an embodiment of the present application is provided. The method may include but is not limited to some or all of the following steps:

[0277] S701: The normal display device displays sample data on a normal display screen. The sample data is the same as the sample data in the above-mentioned embodiment 3. For example, the sample data includes T* sample images.

[0278] S702: The computing device captures a normal display screen to obtain T normal display images.

[0279] The sample images correspond one-to-one to the normal display images.

[0280] It should be understood that the method of obtaining a normal display image from a captured image is the same as the method of obtaining a detection image from a captured image, and will not be described in detail here.

[0281] S703: The computing device respectively calculates the difference frame values ​​between the normal display image and the corresponding example image.

[0282] It should be understood that the method for calculating the difference frame value between the normal display image and its corresponding example image is the same as the above-mentioned method for calculating the difference frame value between the detection image and its corresponding example image, and will not be repeated here.

[0283] S704: The computing device determines a preset difference degree based on a frame difference value between the normal display image and its corresponding example image.

[0284] In some embodiments, the degree of difference may be represented by an R value.

[0285] The calculation method of R value can refer to the calculation method of R value corresponding to the detection image above, which will not be repeated here. R value represents the difference between the normal display image and its corresponding example image.

[0286] In some embodiments, one or more computing devices or image acquisition devices can be used to capture images of a normal display device. Here, M* image acquisition devices are used as an example, where M* is a positive integer. By capturing the normal display device with M* image acquisition devices, M*×T normal display images can be obtained. Furthermore, M*×T R values ​​can be obtained, and the maximum R value, the average of the M*×T R values, or the most probable R value can be used as the preset difference.

[0287] In some embodiments, the N normal display images may be images captured by a same model of image acquisition device on a normal display screen of a device of the same or different models. In this case, the preset difference is the preset difference corresponding to the image acquisition device.

[0288] In other embodiments, the T normal display images may be images captured by multiple models of image acquisition devices on normal display screens of the same or different models of devices, so as to improve the accuracy of display anomaly detection.

[0289] The above-mentioned third embodiment can improve the accuracy of display screen detection.

[0290] Example 4

[0291] The fourth embodiment can detect the display flicker of the display screen, which may include backlight flicker and content flicker. Among them, the backlight flicker is a display flicker problem caused by a hardware failure of the display device, such as an error in the input signal of the display card, an error in the voltage of the backlight lamp, a damage to the display screen, etc. The content flicker is a display flicker problem caused by a software setting failure. The embodiment of the present application provides a detection strategy for categorizing the display flicker problem, that is, using different detection methods for the backlight flicker and content flicker in the display flicker. Figure 8A As shown in the flowchart of Example 4, the method may include but is not limited to some or all of the following steps:

[0292] S801: The device to be detected displays a sample image or a sample video through the display screen to be detected.

[0293] The computing device or image acquisition device can capture the display screen to be detected in real time to obtain captured images or captured videos. The duration of the capture can be 2s, 5s, 10s, etc. The embodiment of the present application takes the capture of W captured images as an example for explanation.

[0294] The computing device may send an instruction for instructing the device to be detected to display an example image, and the device to be detected may display the example image in response to the instruction.

[0295] S802: The computing device determines whether the display screen to be detected has backlight flicker based on the captured image. It should be understood that the captured image can also be called a captured video, which includes multiple frames of images.

[0296] In one implementation, Figure 8B As shown, S802 may include some or all of steps S8021-S8023:

[0297] S8021: The computing device obtains a detection image from the collected image.

[0298] The method of obtaining the detection image from the collected image can be found in the relevant description of the above embodiment 3, which will not be repeated here.

[0299] S8022: The computing device determines the brightness of the detection image.

[0300] Specifically, when the detection image is an RGB image, the detection image is converted from the RGB format to the YUV format, and the Y component in the YUV format is the brightness.

[0301] Usually, the detection image is in RGB format, but the RGB format image cannot identify the brightness information of the image. Therefore, the above detection image needs to be converted from RGB format to YUV format, as shown in formula (7):

[0302] Y=0.299×R+0.387×G+0.114×B (7)

[0303] The Y component represents the brightness data of the image in YUV format, and 0.299, 0.587, and 0.114 represent the conversion coefficients of the R, G, and B components, respectively.

[0304] S8023: The computing device determines whether there is backlight flickering on the display screen to be detected based on the brightness change rate of two adjacent detection image frames.

[0305] It should be understood that W captured images can produce W detection images. There can be W-1 pairs of adjacent detection images in the W detection images. To calculate the brightness change rate of two adjacent detection images, refer to formula (8):

[0306] P * =|Y j -Y i | / Y i (8)

[0307] Among them, i and j refer to two adjacent frames of images, i refers to the previous frame of image, and j refers to the next frame of image; P * is the brightness change rate of the two frames of image, Y i is the brightness of the i-th frame image, Y j is the brightness of the j-th frame image.

[0308] A brightness change rate threshold L can be set, for example, L can be 10%, and the brightness change rate of two adjacent detection frames is compared with L. If the brightness change rate of two adjacent frames is greater than L, it is determined that the device to be detected has backlight flickering, otherwise, there is no backlight flickering.

[0309] In some embodiments, a judgment can also be made based on whether the brightness difference between two adjacent frames of detection images is greater than a brightness threshold. If the brightness difference between two adjacent frames of detection images is greater than the brightness threshold, it is determined that there is backlight flickering on the display screen to be detected. Otherwise, there is no backlight flickering. The brightness threshold is related to at least one of the image acquisition device, shooting parameters, and shooting environment.

[0310] In other embodiments, the brightness of the detection image can be directly compared with a brightness threshold. If the brightness of a detection image frame exceeds the brightness threshold, it is determined that the display screen to be detected has backlight flicker. Otherwise, backlight flicker does not exist. The brightness threshold is related to at least one of the image acquisition device, shooting parameters, and shooting environment.

[0311] In some other embodiments, the computing device determines whether the display screen to be detected has backlight flicker based on the brightness change rate of the detection image and the brightness threshold. The calculation method of the brightness change rate can be found in formula (8) and will not be repeated here. In this case, a threshold Q can be set. If the brightness change rate is greater than Q, it is determined that the device to be detected has backlight flicker. Otherwise, it is determined that there is no backlight flicker. The brightness threshold is related to at least one of the image acquisition device, shooting parameters, and shooting environment.

[0312] In some other embodiments, it is also possible to judge based on whether the difference between the brightness of the detected image and the brightness threshold is greater than a preset value E. If the difference is greater than the preset value E, it is judged that there is backlight flicker on the display screen to be detected; otherwise, there is no backlight flicker. The brightness threshold is related to at least one of the image acquisition device, shooting parameters, and shooting environment.

[0313] S803: The computing device determines whether content flickers on the display screen to be detected based on the captured video.

[0314] In one implementation, Figure 8C As shown, S803 may include some or all of steps S8031-S8033:

[0315] S8031: The computing device obtains a detection image from each frame of the captured video to obtain a detection video.

[0316] The method of obtaining the detection image from the collected image can be found in the relevant description of the above embodiment 3, which will not be repeated here.

[0317] S8032: The computing device obtains a brightness change curve x of the sample video as a benchmark.

[0318] When the sample images in the sample video are in RGB format, the sample images can be converted from RGB format to YUV format according to formula (7) to obtain the brightness of each sample image. Then, the brightness of each frame of the sample image in the sample video is plotted as a curve to obtain the brightness change curve x.

[0319] S8033: The computing device determines a brightness change curve y of the detection video.

[0320] The curve formed by plotting the brightness of each detection image in the detection video is the brightness change curve B. The brightness is calculated according to formula (7), which will not be described here.

[0321] S8034: The computing device calculates the correlation coefficient between the brightness change curve y and the reference x.

[0322] The correlation coefficient may represent the correlation between the brightness change curve of the detection video and the change trend of the benchmark.

[0323] The calculation method of the correlation coefficient is shown in formula (9):

[0324]

[0325] In formula (6), x is the brightness change curve of the sample video; y is the brightness change curve of the detection video; r (x，y) It is the correlation coefficient between the brightness change curve y of the detected video and the reference x. It should be understood that r (x，y) If it is greater than 0, it is positively correlated, indicating that the changing trends of the two sets of data are very close; r (x，y) Equal to 0, indicating that there is no correlation between the changing trends of the two sets of data; r (x，y) If it is less than 0, it is a negative correlation, indicating that the change trends of the two sets of data are opposite. x σ y is the product of the standard deviations of the two sets of data (i.e., x and y), Cov(x, y) is the covariance of x and y, and the covariance is calculated as shown in formula (10);

[0326]

[0327] Among them, x i Indicates the brightness of the example image of the i-th frame in the example video, y i Indicates the brightness of the detection image in the i-th frame of the detection video. Represents the average brightness of all sample images in the sample video; Represents the average brightness of all detection images in the detection video; N represents the number of image frames of the example video or the detection video. It should be understood that the number of image frames contained in the example video is the same as the number of image frames contained in the detection video, both of which are N frames of images.

[0328] S8035: Determine whether content on the display screen to be detected is flickering based on the correlation coefficient.

[0329] Here, we can set the relevant threshold to 0.7, 0.8 or other values ​​less than 1 based on experience. (x，y) If the value is greater than the relevant threshold, it means that the change trends of the two sets of data are similar, and it is judged that there is no content flicker on the device to be detected; otherwise, there is content flicker.

[0330] In an embodiment of the present application, a computing device determines whether content flickering on a display screen to be detected exists by comparing the correlation between the brightness change curve y of a detection video and the change trends of a reference x. This method can mitigate the impact of at least one of external factors, such as the image acquisition device, shooting parameters, and shooting environment, on the brightness data of the image and / or video. Furthermore, it can reduce the interference caused by these external factors on the brightness change curve y of the detection video and the reference x, thereby increasing the accuracy of detecting content flicker on the display screen to be detected.

[0331] The fourth embodiment described above can detect display abnormalities by category, firstly detecting abnormalities during the power-on phase, and then monitoring display flickering after power-on, thereby achieving more accurate and specific detection of backlight flickering and content flickering of the display screen.

[0332] Example 5

[0333] Display anomalies during the dynamic display process after power-on may include non-flickering anomalies such as stripes, noise, ghosting, blue screen, black screen, etc. For display anomalies at this stage, the present application embodiment adopts a machine learning method for detection, which is embodiment five. Figure 9 As shown in the flowchart of Example 5, the method may include but is not limited to the following steps:

[0334] S901: Collect training samples to obtain a training sample set.

[0335] The training samples may include abnormal sample data and normal sample data, wherein the abnormal sample data is the detection image or detection video obtained by shooting an abnormal display screen, which is a negative sample; the normal sample data is the detection image or detection video obtained by shooting a normal display screen, which can also be called a normal display image or normal display video, respectively, and is a positive sample.

[0336] The labels of sample data can be manually marked.

[0337] In some embodiments, the abnormality type of a display screen can be identified, and the sample data obtained by photographing the display screen can be labeled with the abnormality type. The aforementioned abnormality types can include at least one of streaks, noise, ghosting, blue screen, black screen, backlight flicker, content flicker, etc. For example, if a display screen with an abnormality type of noise is photographed, the sample data obtained is labeled as noise. In this case, the labels of the sample data include abnormality types such as streaks, noise, ghosting, blue screen, black screen, backlight flicker, content flicker, etc., as well as normal.

[0338] In other embodiments, it is possible to identify whether a display screen is abnormal, mark sample data obtained by photographing a normal display screen as normal, and mark sample data obtained by photographing an abnormal display screen as abnormal. In this case, the labels of the sample data include normal and abnormal.

[0339] The ratio of positive to negative samples in the training sample set can be 4:1. For example, the number of positive samples can be 36,000, and the number of negative samples can be 9,000. The prepared positive sample data can be generated by the opencv_createsamples tool. The generated positive sample file names can be saved in a file named pos.dat, and the negative sample file names can be saved in a file named neg.dat.

[0340] S902: Train an anomaly detection model using a training sample set.

[0341] Specifically, after obtaining the training sample set, the training device can input the training samples into the anomaly detection model, process the training samples through the anomaly detection model, obtain a prediction result, and adjust the model parameters of the anomaly detection model based on the difference between the prediction result and the label, so that the difference becomes smaller and smaller. When the difference converges or the difference is very small, the trained anomaly detection model is obtained. Among them, the anomaly detection model can be a convolutional neural network, a recurrent neural network or a deep neural network, etc., which is not limited here. The model can be a binary classifier or a multi-classifier. The anomaly detection model can be derived from the classifier in OpenCV. Among them, OpenCV is a cross-platform computer vision and machine learning software library released based on the BSD license (open source), including related models for image processing, model training tools, etc. The training can also be performed by the tools provided by the open source computer vision library OpenCV.

[0342] Exemplarily, the anomaly detection model can be a cascade classifier. The main principle of this cascade classifier is the use of a cascade classification algorithm. The cascade classification algorithm is collectively referred to as the Adaboost algorithm, which is implemented based on boosting and is a feature enhancement algorithm. This process can be understood as implementing explicit anomaly detection through a feature decision tree. The bottom layer of the feature decision tree contains the most basic features. Decisions and classifications are made by comparing the detection results of the feature branches. After classification, samples are further classified using higher-order features. The detection results obtained after traversing the entire feature decision tree are the final output of the anomaly detection model.

[0343] S903: Identify whether the display screen to be detected is abnormal using the trained abnormality detection model.

[0344] When an abnormality detection is required on a display screen of a device to be detected, the device to be detected displays sample data on the display screen. The image acquisition device captures the display screen to obtain captured data, and then obtains a detection image or video from the captured data. The specific implementation is described in the third embodiment above and will not be repeated here. Furthermore, the detection image or video is input into a trained anomaly detection model to obtain a recognition result for the display screen to be detected.

[0345] The display anomaly detection method provided in the embodiments of this application can be used for display anomaly detection during the production process as well as for display anomaly detection during user use. Display anomalies during user use may fall into one of the following three situations: the display anomaly persists, the display anomaly is momentary, or the display anomaly is complex. In the first two situations, the display anomaly detection method provided in the embodiments of this application can be used for detection.

[0346] In the above embodiments, all or part of the functions can be implemented by software, hardware, or a combination of software and hardware. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0347] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular expressions "a," "an," "the," "the," "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0348] It should also be understood that when used in this specification and the appended claims, the terms "include" and "comprising" indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or sets thereof. In addition, the terms "first," "second," "target," etc. are used to distinguish different objects, rather than to describe a particular order. The term "plurality" refers to two or more than two.

[0349] It should be further understood that references to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearance of such phrases in various locations in the specification does not necessarily refer to the same embodiment, nor does it necessarily refer to independent or alternative embodiments that are mutually exclusive of other embodiments. Those skilled in the art understand, both explicitly and implicitly, that the embodiments described in this application may be combined with other embodiments.

[0350] The above description is merely a specific embodiment of the present application, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A display anomaly detection method, characterized in that: The method comprises: The electronic device acquires collected data, wherein the collected data is obtained by photographing a display screen to be inspected of the device to be inspected, and when photographing, the device to be inspected displays sample data through the display screen to be inspected; the sample data includes one or more sample images, and the collected data includes one or more collected images; The electronic device obtains N detection images from the collected data; the detection images are images of example images in the collected images, and N is a positive integer; The electronic device performs abnormality detection on the display screen to be detected based on the N detection images; The one or more example images include N example images, the N example images correspond one-to-one to the N detection images, and the electronic device performs abnormality detection on the display screen to be detected based on the N detection images, including: The electronic device calculates a frame difference value between the detection image and the corresponding example image; the frame difference value includes a difference in pixel values ​​between pixels at the same position on the detection image and the corresponding example image; The electronic device determines, based on a difference frame value between the detection image and the corresponding example image, a difference degree between the detection image and the corresponding example image, wherein the difference degree indicates a degree of difference between the detection image and the corresponding example image; The electronic device performs abnormality detection on the display screen to be detected according to the difference.

2. The method according to claim 1, characterized in that The electronic device obtains N detection images from the collected data, including: The electronic device obtains N images with image changes from the collected data; The electronic device captures an image of the example image from each of the N images to obtain N detection images.

3. The method according to claim 1, characterized in that The electronic device calculates a frame difference value between the detection image and the corresponding example image, including: The electronic device performs frame difference calculation on each of the N detection images and the N example images to obtain N frame difference values; wherein the example image corresponding to the minimum frame difference value among the N frame difference values ​​is the example image corresponding to each detection image.

4. The method according to claim 1 or 3, characterized in that The difference is determined by the variance and mean of the pixel values ​​of all pixels on the difference frame values ​​between the detection image and its corresponding example image.

5. The method according to any one of claims 1 to 4, characterized in that The electronic device performs abnormality detection on the display screen to be detected according to the difference, further comprising: The electronic device determines a target difference based on the differences corresponding to the N detection images; The electronic device performs abnormality detection on the display screen to be detected based on a comparison result between the target difference and a preset difference.

6. The method according to claim 5, characterized in that The target difference is the maximum difference among the differences corresponding to the N detection images; when the target difference is greater than the preset difference, the display screen to be detected has an abnormality; When the target difference is not greater than the preset difference, there is no abnormality in the display screen to be detected.

7. The method according to claim 1, characterized in that The sample data includes a sample image, and the electronic device performs abnormality detection on the display screen to be detected based on the N detection images, further comprising: The electronic device determines the brightness of the N detection images respectively; The electronic device detects backlight flickering of the display screen to be detected according to the brightness of the N detection images.

8. The method according to claim 7, characterized in that If the brightness change rate or brightness difference between two adjacent detection images in the N detection images is greater than the brightness threshold, the display screen to be detected has backlight flickering; or, In the case that there is no brightness change rate between two adjacent images in the N detection images or the brightness difference is greater than the brightness threshold, there is no backlight flicker on the display screen to be detected.

9. The method according to claim 1, characterized in that The sample data includes a plurality of sample images, and the electronic device performs abnormality detection on the display screen to be detected based on the N detection images, including: The electronic device calculates a correlation coefficient between the brightness of the N detection images and a benchmark; the benchmark includes the brightness of each example image in the example data; the correlation coefficient indicates the degree of similarity between the N detection images and the example data; The electronic device detects content flickering on the display screen to be detected according to the correlation coefficient; When the correlation coefficient is greater than a correlation threshold, content flickering exists on the display screen to be detected; or, when the correlation coefficient is not greater than a correlation threshold, content flickering does not exist on the display screen to be detected.

10. The method according to claim 9, characterized in that The correlation coefficient is determined by a covariance between the brightness of the N detection images and the benchmark and a standard deviation between the brightness of the N detection images and the benchmark.

11. The method according to claim 1, wherein Before the electronic device acquires the collected data, the method further includes: The electronic device sends an instruction to the device to be detected, where the instruction carries the sample data and is used to instruct the device to be detected to display the sample data.

12. A display anomaly detection method, characterized in that: The method comprises: The electronic device acquires first collected data, where the first collected data is obtained by the electronic device photographing the display screen of the device to be detected when the device to be detected is powered on, and the device to be detected displays first example data through the display screen during the photographing; the first example data includes one or more example images, and the first collected data includes one or more collected images; The electronic device performs abnormality detection on the display screen to be detected during the power-on phase based on the first collected data; The electronic device acquires second collected data, where the second collected data is obtained by the electronic device photographing the display screen of the device to be detected after the device to be detected is powered on, and the device to be detected displays second example data through the display screen during the photographing; The electronic device detects display flicker on the display screen to be detected based on the second collected data; The one or more example images include N1 example images, the N1 example images correspond one-to-one to N1 detection images, the N1 detection images are obtained from images in which images have changed in the first collected data, N1 is a positive integer, and the electronic device performs abnormality detection on the display screen to be detected during a power-on phase based on the first collected data, including: The electronic device calculates a frame difference value between the detection image and the corresponding example image; the frame difference value includes a difference in pixel values ​​between pixels at the same position on the detection image and the corresponding example image; The electronic device determines, based on a difference frame value between the detection image and the corresponding example image, a difference degree between the detection image and the corresponding example image, wherein the difference degree indicates a degree of difference between the detection image and the corresponding example image; The electronic device performs abnormality detection on the display screen to be detected during a power-on phase according to the difference.

13. The method according to claim 12, characterized in that The electronic device calculates a frame difference value between the detection image and the corresponding example image, including: The electronic device performs frame difference calculation on each of the N1 detection images and the N1 example images to obtain N1 frame difference values; wherein the example image corresponding to the minimum frame difference value among the N1 frame difference values ​​is the example image corresponding to the detection image.

14. The method according to claim 12 or 13, characterized in that The electronic device performs abnormality detection on the display screen to be detected during a power-on phase according to the difference, including: The electronic device determines a target difference based on the N1 differences; The electronic device performs abnormality detection on the display screen to be detected during a power-on phase based on a comparison result between the target difference and a preset difference.

15. The method according to claim 14, characterized in that The target difference is the maximum difference among the differences corresponding to the N1 detection images; when the target difference is greater than the preset difference, the display screen to be detected has an abnormality during the power-on phase; When the target difference is not greater than the preset difference, there is no abnormality in the display screen to be detected during the startup phase.

16. The method according to any one of claims 12 to 15, characterized in that: The second collected data includes first sub-collected data, the second example data includes a target example image, the first sub-collected data is obtained by the electronic device photographing the display screen of the device to be detected after the device to be detected is turned on, and the device to be detected displays the target example image through the display screen to be detected during the photographing, and the electronic device detects display flicker on the display screen to be detected based on the second collected data, including: The electronic device acquires N2 detection images from the first sub-collected data, where N2 is a positive integer; The electronic device determines the brightness of the N2 detection images respectively; The electronic device detects backlight flickering of the display screen to be detected based on the brightness of the N2 detection images.

17. The method according to claim 16, characterized in that If the brightness change rate or brightness difference between two adjacent detection images in the N2 detection images is greater than the brightness threshold, the display screen to be detected has backlight flickering; or, In the case that there is no brightness change rate between two adjacent images in the N2 detection images or the brightness difference is greater than the brightness threshold, there is no backlight flicker on the display screen to be detected.

18. The method according to any one of claims 12 to 17, characterized in that: The second collected data includes second sub-collected data, the second example data includes a target example video, the second sub-collected data is obtained by the electronic device shooting the display screen of the device to be detected after the device to be detected is turned on, and the device to be detected displays the target example video through the display screen to be detected during the shooting, and the electronic device detects display flicker on the display screen to be detected based on the second collected data, including: The electronic device obtains N3 detection images from the second sub-collected data, where N3 is a positive integer; The electronic device calculates a correlation coefficient between the brightness of the N3 detection images and a benchmark; the benchmark includes the brightness of each image in the target example video; the correlation coefficient indicates the degree of similarity between the N3 detection images and the target example video; The electronic device detects content flickering on the display screen to be detected based on the correlation coefficient; when the correlation coefficient is greater than a relevant threshold, content flickering exists on the display screen to be detected; when the correlation coefficient is not greater than the relevant threshold, content flickering does not exist on the display screen to be detected.

19. An electronic device comprising one or more memories and one or more processors, characterized in that: The one or more memories are used to store data and instructions, and the one or more processors are used to call the data and instructions stored in the memories; the electronic device executes the method according to any one of claims 1-18.

20. A computer-readable storage medium, characterized in that include: Computer instructions; when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 18.

Images