Screen detection method, device and storage medium

Abstract

The application discloses a screen detection method and device and a storage medium, and relates to the technical field of device detection.The method comprises the following steps: acquiring a first display image of a target device, wherein the first display image is a photo obtained by shooting a first test image displayed on the screen of the target device; judging whether the screen meets a first determination condition according to a resolution value of the first display image; the first determination condition is that the resolution value is within a preset first threshold range; if the screen meets the first determination condition, acquiring a second display image of the target device, wherein the second display image is a photo obtained by shooting a second test image displayed on the screen; judging whether the screen meets a second determination condition according to an MTF value of the second display image, and obtaining a detection result of the screen; the second determination condition is that the MTF value is within a preset second threshold range.The application solves the problems that the existing method has high requirements for a detection environment and a complicated process, and achieves the effect of improving the screen detection efficiency and precision of the device.

Description

Technical Field

[0001] The present invention relates to the technical field of device detection, and in particular to a screen detection method, device and storage medium. Background Art

[0002] Traditional manual inspection methods have been gradually phased out, replaced by the emergence of screen display inspection methods using machine vision and deep learning algorithms. However, this method has complex inspection environment requirements, such as light shielding and a limited ambient brightness. Deep learning models require extensive training data, resulting in a cumbersome and time-consuming inspection process. Therefore, this method is generally only used in large-scale product production and testing, and is not suitable for small-batch production and testing, as it would increase costs and be unsuitable for practical applications. Summary of the Invention

[0003] The main purpose of the present invention is to provide a screen detection method, device and storage medium, aiming to solve the technical problems in the prior art that the screen detection method has high requirements on the detection environment and the process is cumbersome.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention provides a screen detection method, the method comprising:

[0006] Acquire a first display image of the target device, where the first display image is a photograph obtained by photographing a screen of the target device displaying a preset first test image;

[0007] Determining whether the screen meets a first determination condition based on the resolution value of the first display image; wherein the first determination condition is that the resolution value is within a preset first threshold range;

[0008] If the screen meets the first determination condition, a second display image of the target device is obtained, where the second display image is a photo obtained by shooting the screen displaying a preset second test image;

[0009] According to the MTF value of the second display image, it is determined whether the screen meets the second determination condition, and a detection result of the screen is obtained; wherein the second determination condition is that the MTF value is within a preset second threshold range.

[0010] Optionally, in the above screen detection method, before the step of acquiring the first display image of the target device, the method further includes:

[0011] Adjusting the position of the target device and the position of the camera based on user operations, and fixing the target device and the camera;

[0012] Adjust the camera's operating parameters based on the distance between the target device's screen and the camera to capture the screen.

[0013] Optionally, in the above screen detection method, before the step of determining whether the screen meets the first determination condition based on the resolution value of the first display image, the method further includes:

[0014] Preprocessing and gray-scaling the first display image to obtain a first screen image;

[0015] performing image correction on the first screen image to obtain a corrected first screen image;

[0016] The step of determining whether the screen meets the first determination condition according to the resolution value of the first display image includes:

[0017] According to the resolution value of the corrected first screen image, it is determined whether the screen meets the first determination condition.

[0018] Optionally, in the above screen detection method, before the step of performing image correction on the first screen image to obtain a corrected first screen image, the method further includes:

[0019] Obtaining a calibration image for the camera, where the calibration image is captured when the third test image is located at the screen position, the third test image comprising a plurality of equally spaced concentric circles and at least one straight line passing through the centers of the circles and intersecting each of the concentric circles at two points;

[0020] Comparing the calibration image with the third test image, determining the distances from the centers of the multiple intersection points of the multiple concentric circles and the straight line, and the degree of distortion of the concentric circles where the intersection points are located;

[0021] According to the relationship between distance and distortion degree, the camera distortion compensation function is obtained by fitting;

[0022] The step of performing image correction on the first screen image to obtain a corrected first screen image includes:

[0023] Image correction is performed on the first screen image according to the distortion compensation function to obtain a corrected first screen image.

[0024] Optionally, in the above screen detection method, the step of determining whether the screen meets the first determination condition according to the resolution value of the first display image includes:

[0025] Dividing the first display image into N×N regions to obtain a plurality of first region images;

[0026] For each first region image, a resolution value of the first region image is obtained according to a first calculation formula, wherein the first calculation formula is:

[0027]

[0028] Where R represents the resolution value, x represents the horizontal coordinate of the image, y represents the vertical coordinate of the image, f(x,y) represents the grayscale value of the pixel point (x,y), m represents the number of pixels arranged horizontally in the first region image, and n represents the number of pixels arranged vertically in the first region image;

[0029] Calculating an average resolution value according to the resolution values ​​of the plurality of first region images;

[0030] Determining whether the resolution values ​​of the plurality of first region images meet corresponding preset resolution thresholds, and whether the average resolution value meets a preset average resolution threshold;

[0031] If the resolution values ​​of the plurality of first region images all meet the preset resolution threshold, and the average resolution value meets the average resolution threshold, it is determined that the screen meets the first determination condition.

[0032] Optionally, in the above screen detection method, the second test image is an image with alternating black and white stripes of the same width, there are multiple second test images, and the stripes in the multiple second test images have different widths and / or different directions;

[0033] The step of acquiring the second display image of the target device includes:

[0034] The screen of the target device is controlled to display a plurality of second test images in sequence, and corresponding second display images are acquired to obtain a plurality of second display images.

[0035] Optionally, in the above screen detection method, the step of determining whether the screen meets the second determination condition based on the MTF value of the second display image, and obtaining the screen detection result includes:

[0036] For each second display image, the second display image is equally divided into K×K regions to obtain a plurality of second region images;

[0037] For each second region image, the MTF value of the second region image is obtained according to the second calculation formula, wherein the second calculation formula is:

[0038]

[0039] Where M represents the MTF value, I avg.max It represents the maximum value of the pixel mean of all stripes in the second region image, I avg.min represents the minimum value among the pixel means of all stripes in the second region image;

[0040] Calculating an average MTF value based on the MTF values ​​of the plurality of second region images;

[0041] Determining whether the MTF values ​​of the plurality of second region images meet corresponding preset MTF thresholds, and whether the average MTF value meets a preset average MTF threshold;

[0042] If the MTF values ​​of the plurality of second region images all meet the preset MTF threshold, and the average MTF value meets the average MTF threshold, then it is determined that the second display image meets the MTF requirement;

[0043] When the plurality of second display images all meet the MTF requirement, it is determined that the screen meets the second determination condition, and a qualified detection result of the screen is obtained.

[0044] Optionally, in the above screen detection method, after the step of determining whether the screen meets the first determination condition based on the resolution value of the first display image, the method further includes:

[0045] If the screen does not meet the first judgment condition, the resolution value is analyzed for abnormalities and a data analysis report is obtained;

[0046] Based on the data analysis report, an alarm prompt is issued and / or corresponding expert advice is determined to repair the target equipment.

[0047] In a second aspect, the present invention provides a screen detection device, which includes a processor and a memory. The memory stores a screen detection program. When the screen detection program is executed by the processor, the above-mentioned screen detection method is implemented.

[0048] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by one or more processors, the screen detection method described above is implemented.

[0049] The above one or more technical solutions provided by the present invention may have the following advantages or at least achieve the following technical effects:

[0050] The application provides a screen detection method, device and storage medium. The first test image is displayed on the screen of a target device, a first display image is obtained by shooting, and it is determined whether the screen meets a first determination condition according to the resolution value of the first display image. The first determination condition is that the resolution value is within a preset first threshold range. If the screen meets the first determination condition, the second test image is displayed on the screen, a second display image is obtained by shooting, and it is determined whether the screen meets a second determination condition according to the MTF value of the second display image. The second determination condition is that the MTF value is within a preset second threshold range. Thus, the detection result of the screen is obtained, and the screen quality detection on the production line of the device is realized. Compared with the screen detection method using the deep learning algorithm, the traditional mathematical method combined with the machine vision is used for detection in the application, the process is simplified, the data volume is not required, a large amount of time is not needed, the method can be applied to small-batch product production, and has good applicability. Moreover, the application does not have special requirements on the detection environment, and only the general device production environment is needed, and the application has good flexibility. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description only some embodiments of the application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0052] Figure 1 The flowchart of the first embodiment of the screen detection method of the application is shown.

[0053] Figure 2 The hardware structure diagram of the screen detection device related to the application is shown.

[0054] Figure 3 The third test image of step S131 in the second embodiment of the screen detection method of the application is shown.

[0055] Figure 4 The calibration image of step S131 in the second embodiment of the screen detection method of the application is shown.

[0056] Figure 5 The second test image of step S600 in the second embodiment of the screen detection method of the application is shown.

[0057] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the drawings. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0059] It should be noted that, in the present invention, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. Without further limitation, the phrase "comprise..." does not preclude the presence of other identical elements in the process, method, article, or system comprising such elements. Furthermore, the meaning of "and / or" throughout this document encompasses three parallel solutions. For example, "A and / or B" includes solution A, solution B, or solutions where both A and B are satisfied. In the present invention, any reference to "first," "second," etc., is for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one of such features. Those skilled in the art will understand the specific meanings of these terms in the present invention based on the specific circumstances. In addition, the technical solutions of the various embodiments can be combined with each other, but this is based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0060] An analysis of existing technologies reveals that smart speakers integrate voice recognition and natural language processing technologies with traditional speakers, enabling them to simultaneously play audio resources, interact with intelligent voice, and control smart homes. Due to their rich functionality, smart speakers are considered the control center of smart homes and have become one of the fastest-growing electronic products. Speakers with screens, due to their powerful human-computer interaction capabilities, have become a key development direction for smart speakers. To provide users with a more comfortable display through the screen, screen quality testing is necessary during the production testing phase to prevent defective products from entering the market.

[0061] In related technologies, traditional manual inspection methods have been gradually eliminated, and a method for inspecting screen displays using machine vision and deep learning algorithms has emerged. However, this method has complex inspection environment requirements, such as requiring shading and a low ambient brightness. A detection model is trained using a deep learning algorithm to inspect screen displays, but due to the unexplainable nature of deep learning models, the training process is extremely complex and requires a large amount of training data. Therefore, this method also has the disadvantage of being cumbersome and time-consuming throughout the entire inspection process. When it comes to small-batch product production and testing, this method will incur excessive time and resource costs, which is not in line with actual needs. Therefore, this method is generally only used in large-scale product production and testing processes and is not suitable for small-batch product production and testing, as otherwise it will increase excessive costs and be unsuitable for practical applications.

[0062] In view of the technical problems that the screen detection method in the prior art has high requirements on the detection environment and the process is cumbersome, the present invention provides a screen detection method, the overall idea of ​​which is as follows:

[0063] A first display image of the target device is obtained, where the first display image is a photograph obtained by photographing the target device's screen displaying a preset first test image; based on a resolution value of the first display image, whether the screen meets a first determination condition is determined; wherein the first determination condition is that the resolution value is within a preset first threshold range; if the screen meets the first determination condition, a second display image of the target device is obtained, where the second display image is a photograph obtained by photographing the screen displaying a preset second test image; based on an MTF value of the second display image, whether the screen meets a second determination condition is determined, thereby obtaining a screen detection result; wherein the second determination condition is that the MTF value is within a preset second threshold range.

[0064] The above technical solution enables screen quality inspection on equipment production lines. Compared to screen inspection methods that use deep learning algorithms, this method uses traditional mathematical methods combined with machine vision for inspection, simplifying the process. It eliminates data volume requirements and does not require extensive time. It is suitable for small-batch production and has good applicability. Furthermore, this method has no special requirements for the inspection environment; a general equipment production environment is sufficient, providing greater flexibility.

[0065] The screen detection method, device, and storage medium provided by the present invention are described in detail below with reference to specific embodiments and implementations in conjunction with the accompanying drawings.

[0066] Example 1

[0067] Reference Figure 1 A flow chart of the present invention is provided, which proposes a first embodiment of the screen detection method of the present invention, and the screen detection method is applied to a screen detection device.

[0068] Screen detection equipment refers to terminal equipment or network equipment that can achieve network connection. Screen detection equipment can be terminal equipment such as mobile phones, computers, tablets, portable computers, embedded industrial computers, or network equipment such as servers and cloud platforms.

[0069] like Figure 2 FIG2 is a schematic diagram of the hardware structure of a screen detection device. The screen detection device may include: a processor 1001 , such as a CPU (Central Processing Unit), a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 .

[0070] Specifically, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 is used to connect to the client and communicate data with the client, and the user interface 1003 may include an output unit and an input unit; the network interface 1004 is used to connect to the background server and communicate data with the background server, and the network interface 1004 may include an input / output interface; the memory 1005 is used to store various types of data, which may include, for example, instructions for any application or method in the screen detection device, as well as application-related data, and the memory 1005 may be a built-in memory; optionally, the memory 1005 may also be a storage device independent of the processor 1001, and further reference is made to FIG. Figure 2 The memory 1005 may include an operating system, a network communication module, a user interface module, and a screen detection program; the processor 1001 is configured to call the screen detection program stored in the memory 1005 and perform the following operations:

[0071] Acquire a first display image of the target device, where the first display image is a photograph obtained by photographing a screen of the target device displaying a preset first test image;

[0072] Determining whether the screen meets a first determination condition based on the resolution value of the first display image; wherein the first determination condition is that the resolution value is within a preset first threshold range;

[0073] If the screen meets the first determination condition, a second display image of the target device is obtained, where the second display image is a photo obtained by shooting the screen displaying a preset second test image;

[0074] According to the MTF value of the second display image, it is determined whether the screen meets the second determination condition, and a detection result of the screen is obtained; wherein the second determination condition is that the MTF value is within a preset second threshold range.

[0075] Based on the above screen detection equipment, the following Figure 1The flowchart shown in FIG. 1 is a detailed description of the screen detection method of this embodiment. The method may include the following steps:

[0076] Step S200: Acquire a first display image of a target device, where the first display image is a photograph obtained by photographing a screen of the target device displaying a preset first test image.

[0077] Specifically, the target device can be any electronic device with a display screen, such as a smart speaker with a screen, a mobile phone, a computer, and the like. In actual applications, the screen detection device can be set on the production line of the electronic device, and the electronic devices on the production line can be used as target devices in turn to execute the method of this embodiment, or a few of them can be randomly selected and used as target devices in turn to execute the method of this embodiment. The detection personnel can first fix the target device and the camera, and then control the screen of the target device to display a preset first test image, and the camera can be used to shoot the image, and then the captured image is sent to the screen detection device, and the screen detection device obtains the first display image of the target device. Among them, the first test image can be any image with a pattern or text, and the screen detection device can send the first test image to the target device to display it on the screen of the target device.

[0078] Step S400: judging whether the screen satisfies a first determination condition based on the resolution value of the first display image; wherein the first determination condition is that the resolution value is within a preset first threshold range.

[0079] Specifically, when determining whether the screen meets the first determination condition based on the first display image, the resolution value of the first display image can be directly calculated and then compared with a preset first threshold range to determine whether it is within the first threshold range. Alternatively, the first display image can be divided into multiple regions, and the resolution value of each region is calculated separately. The resolution value determination is performed for each region, and each region can have its own preset resolution value threshold range, namely the first threshold range, to determine whether the first display image meets the first determination condition. Optionally, since the boundary of the first display image captured in step S200 may not be exactly the boundary of the screen, the image may also include background outside the screen. In this case, the first display image can be processed first to extract the image including only the screen area, and then the resolution value of the image can be calculated for determination.

[0080] Step S600: If the screen meets the first determination condition, a second display image of the target device is obtained, where the second display image is a photo obtained by shooting a preset second test image displayed on the screen.

[0081] Specifically, after step S400, if the screen meets the first determination condition, subsequent testing steps can be performed. If the target device and camera have already been secured in step S200, the target device's screen can be directly controlled to display a preset second test image, which can be captured by the camera and then sent to the screen testing device. The screen testing device then obtains the second display image of the target device. The second test image can be a stripe image, which can be sent by the screen testing device to the target device for display on the target device's screen.

[0082] To achieve more accurate detection, the second test image can include multiple different stripe images, allowing for multiple tests. Accordingly, the target device's screen needs to display these multiple stripe images sequentially, allowing the screen detection device to acquire multiple second display images. To prevent the second display image corresponding to the first stripe image from failing the test, resulting in wasted resources and the acquisition of additional second display images, the target device can be controlled to display the next stripe image after a positive test result is obtained for the displayed stripe image. This cycle continues until the final stripe image passes the test, at which point the target device's screen is deemed qualified.

[0083] After step S400 is judged, if the screen does not meet the first judgment condition, there is no need to continue subsequent testing. It can be directly repaired, or the calculated Resolution value can be sent to the expert system to obtain corresponding expert advice and other operations. The expert system can be set together in the screen detection device, and the obtained expert advice can be displayed on the display screen of the screen detection device itself; it can be understood that the expert system can also be stored in an independent device. In this case, the screen detection device can output the calculated Resolution value to the corresponding device.

[0084] Step S800: judging whether the screen meets a second determination condition based on the MTF value of the second display image, and obtaining a detection result of the screen; wherein the second determination condition is that the MTF value is within a preset second threshold range.

[0085] Specifically, after obtaining the second display image in step S600, the screen detection device calculates its MTF (Modulation Transfer Function) value, and then compares the MTF value with a preset second threshold range to determine whether the screen meets the second judgment condition. During the judgment, the MTF value of the second display image can be directly calculated for judgment, or the second display image can be divided into multiple regions, and the MTF value of each region is calculated separately. The MTF value judgment is performed for each region. Each region can have its own preset MTF value threshold range, i.e., the second threshold range, to determine whether the second display image meets the second judgment condition, thereby obtaining a screen detection result.

[0086] When multiple second display images are obtained in step S600, a single process can be used to perform the test on each second display image, or multiple processes can be used to perform the test on each second display image separately. Finally, the test results of each process can be directly summarized to obtain the final test result. In this case, the number of processes is the same as the number of second display images. When performing the test in a single process, one second display image can be first obtained, its MTF value calculated, and then a second image can be obtained after the test is completed and the conditions are met. This process repeats steps S600 and S800. Alternatively, multiple second display images can be first obtained, the MTF value of the first image calculated, and the MTF value of the second image calculated after the test is completed and the conditions are met. This process repeats only step S800.

[0087] Optionally, since the boundary of the second display image captured in step S600 may not be exactly the boundary of the screen, the background outside the screen may also be captured in the image. In this case, the second display image can be processed first to extract the image including only the screen area, and then its MTF value is calculated for judgment.

[0088] Resolution and MTF values ​​can evaluate image clarity and describe the imaging system's ability to recover scene details. The method of this embodiment calculates these two metrics to reflect the display capabilities of the screen, automatically detecting screen display quality. This is a fully automated screen quality detection method that can simultaneously improve detection efficiency and accuracy. This embodiment automates screen detection, which, compared to traditional methods of assessing screen quality through manual visual inspection, does not incur excessive labor costs, offers high production line detection efficiency, is robust, and is less susceptible to subjective human factors. It is less likely to miss or misdetect, thereby improving detection accuracy.

[0089] It should be noted that in actual applications, only steps S200 and S400 may be performed, i.e., obtaining a first display image of the target device and determining whether the screen meets the first determination condition based on the resolution value of the first display image. This is used to determine whether the target device screen has display problems. This can be applied to some scenarios where only the screen display condition needs to be tested. If the screen meets the first determination condition, it means that the target device screen image is displaying well. Alternatively, steps S600 and S800 may be directly performed, i.e., obtaining a second display image of the target device and determining whether the screen meets the second determination condition based on the MTF value of the second display image. This is used to determine whether the target device screen has clarity problems. This can be applied to some scenarios where only the screen display clarity needs to be tested. If the screen meets the second determination condition, it means that the target device screen display has clarity problems. In this embodiment, through steps S200-S800, it is first determined that the screen display is good and then the clarity is determined. Screen quality is tested from two dimensions to ensure that the final product that meets the requirements not only has normal display but also good clarity. It can be directly transferred to the next process or even directly stored in the warehouse. This is more suitable for actual application on factory production lines, especially for the actual production and testing of small batch products.

[0090] The screen detection method provided in this embodiment displays a first test image on the screen of a target device, captures the first display image, and then determines whether the screen meets a first determination condition based on the resolution value of the first display image. The first determination condition is that the resolution value is within a preset first threshold range. If the screen meets the first determination condition, a second test image is displayed on the screen, captures the second display image, and then determines whether the screen meets a second determination condition based on the MTF value of the second display image. The second determination condition is that the MTF value is within a preset second threshold range, thereby obtaining a screen detection result and implementing screen quality inspection on the device production line. Compared to screen detection methods that use deep learning algorithms, the present invention uses traditional mathematical methods combined with machine vision for detection, simplifying the process, eliminating data volume requirements, and not requiring a large amount of time. It is suitable for small-batch product production and has good applicability. Furthermore, the present invention has no special requirements for the detection environment, and only a general device production environment can be used, which provides good flexibility.

[0091] Example 2

[0092] Based on the same invention concept, Figures 3 to 5 , a second embodiment of the screen detection method of the present invention is proposed, which is applied to a screen detection device. The screen detection method of this embodiment is described in detail below. The method may include the following steps:

[0093] Step S100: Fix the target device and the camera, and adjust the working parameters of the camera.

[0094] Specifically, step S100 can include:

[0095] Step S110: adjusting the position of the target device and the position of the camera based on the user operation, and fixing the target device and the camera.

[0096] In specific implementation, a test environment can be built first. This process can be realized by manual operation or by a screen detection device in cooperation with manual operation. Specifically, the screen detection device itself can be associated with an adjustment table, a mechanical arm, and other mechanical devices to adjust the position of the target device and the position of the camera, and to fix the target device and the camera, so as to capture the screen of the target device by the camera, to obtain the display image of the screen by machine vision, and to perform display quality detection. The camera can be an industrial camera. In specific application, in order to better perform detection and have more accurate detection results, the above operation can be performed in a stable dark environment without light source interference.

[0097] Step S120: adjusting the working parameters of the camera according to the distance between the screen of the target device and the camera, so as to capture the screen by the camera.

[0098] In specific implementation, after the target device and the camera are fixed, the camera parameters are adjusted according to the distance between the target device and the camera, so that the working parameters such as focal length and exposure time are appropriate. The actual needs can be adjusted.

[0099] The test environment built in the early preparation stage and the camera parameter adjustment are mainly to ensure a stable detection environment, so that the detection results are not affected by external factors.

[0100] In one implementation, after step S100, the method can further include:

[0101] Step S130: calibrating the camera to obtain a distortion compensation function of the camera.

[0102] In specific implementation, the industrial camera can have barrel distortion due to the convex lens imaging problem of the wide-angle lens. Therefore, after the test environment is built in the early preparation stage, camera calibration can be further performed. This process can be realized by manual operation or by a screen detection device in cooperation with manual operation. Specifically, the industrial camera is adjusted and calibrated to obtain the distortion parameters of the camera, which are applied to the subsequent Resolution value and MTF value calculation process as the basis for distortion compensation.

[0103] Specifically, step S130 can include:

[0104] Step S131: Acquire a calibration image of the camera, where the calibration image is an image captured when the third test image is located at the position of the screen, and the third test image includes a plurality of equally spaced concentric circles and at least one straight line, where the straight line passes through the center of the circle and has two intersections with each concentric circle;

[0105] Step S132: Comparing the calibration image and the third test image, determining the distances from the intersection points of the concentric circles and the straight lines to the centers of the circles, and the degree of distortion of the concentric circles where the intersection points are located;

[0106] Step S133: fitting is performed based on the relationship between the distance and the degree of distortion to obtain a distortion compensation function of the camera.

[0107] When implementing it, be prepared Figure 3 The third test image shown in the figure contains multiple, specifically 7 equally spaced concentric circles and two straight lines, which pass through the centers of the circles and have two intersections with each concentric circle. The third test image is placed on the screen of the target device and photographed by the camera, and a distorted photo can be obtained, as shown in FIG. Figure 4 The calibration image shown in the figure shows that the spacing between the 7 concentric circles in the image has changed. Figure 3 The third test image and Figure 4 The calibration image is used to determine the distances from the multiple intersection points of the 7 concentric circles and the straight line to the center of the circle and the degree of distortion of the concentric circles where the intersection points are located. Then a fitting function is calculated and recorded to obtain the distortion compensation function of the camera for future use.

[0108] Step S200: Acquire a first display image of a target device, where the first display image is a photograph obtained by photographing a screen of the target device displaying a preset first test image.

[0109] The first test image is an image containing text, such as text covering the entire screen. This image is a specialized image used to test resolution. When the target device's screen displays this first test image, a camera captures the image. The captured image is then transmitted via a serial port to the screen detection device, which then captures the target device's first display image.

[0110] Step S300: performing image processing on the first display image.

[0111] Specifically, step S300 may include:

[0112] Step S310: pre-processing and gray-scaling the first display image to obtain a first screen image;

[0113] Step S320: performing image correction on the first screen image to obtain a corrected first screen image.

[0114] In a specific implementation, after the screen detection device obtains the first display image, it can pre-process the first display image, such as performing screen border recognition, image segmentation, and other pre-processing, extracting an image containing only the screen area, removing unnecessary background or borders, and then performing grayscale processing to obtain the first screen image; then the first screen image can be corrected to obtain a corrected first screen image.

[0115] Correspondingly, after obtaining the camera distortion compensation function in step S130, step S320 may include:

[0116] Step S321: performing image correction on the first screen image according to the distortion compensation function to obtain a corrected first screen image.

[0117] In a specific implementation, the distortion compensation function obtained in step S130 is used to correct the first screen image into a standard rectangle corresponding to the image actually displayed on the screen, thereby preventing camera distortion from affecting the detection result.

[0118] Step S400: judging whether the screen satisfies a first determination condition based on the resolution value of the first display image; wherein the first determination condition is that the resolution value is within a preset first threshold range.

[0119] Correspondingly, after obtaining the corrected first screen image in step S320, step S400 may include:

[0120] Step S400a: judging whether the screen satisfies a first determination condition according to the resolution value of the corrected first screen image.

[0121] Calculating the resolution value of the image obtained after image processing, that is, the corrected first screen image, and then making a judgment can improve the detection accuracy and prevent the background image outside the screen in the original image from affecting the detection result.

[0122] Specifically, step S400 may include:

[0123] Step S410: Divide the first display image into N×N regions to obtain a plurality of first region images;

[0124] The first detection of the entire detection process is performed based on the acquired first display image or the first screen image corrected in step S320. First, the first display image or the first screen image is divided into N×N small rectangular areas, such as 9×9 rectangular areas. Multiple first area images can be obtained, and 81 first area images are obtained here.

[0125] Step S420: For each first region image, obtain the resolution value of the first region image according to a first calculation formula, wherein the first calculation formula is:

[0126]

[0127] Wherein, R represents the resolution value, x represents the horizontal coordinate of the image, y represents the vertical coordinate of the image, f(x,y) represents the grayscale value of the pixel point (x,y), m represents the number of pixels arranged horizontally in the first region image, and n represents the number of pixels arranged vertically in the first region image.

[0128] By calculating the resolution value of each first region image, the resolution values ​​of all first region images can be obtained, thereby ensuring that the display resolution of each region on the screen is good.

[0129] Step S430: Calculate an average resolution value according to the resolution values ​​of the plurality of first region images.

[0130] After obtaining the resolution values ​​of all first region images, their mean is calculated to obtain the average resolution value.

[0131] Step S440: determining whether the resolution values ​​of the plurality of first region images meet corresponding preset resolution thresholds, and whether the average resolution value meets a preset average resolution threshold.

[0132] For each first region image, the calculated resolution value of the first region image is compared with a preset resolution threshold corresponding to the first region image, and at the same time, the calculated average resolution value is compared with the preset average resolution threshold.

[0133] Step S450: If the resolution values ​​of the plurality of first region images all meet the preset resolution threshold, and the average resolution value meets the average resolution threshold, it is determined that the screen meets the first determination condition.

[0134] Step S460: If the resolution value of a first region image does not meet its corresponding preset resolution threshold, or the average resolution value does not meet the preset average resolution threshold, it is determined that the screen does not meet the first determination condition.

[0135] When the resolution values ​​of all first-region images meet their corresponding preset resolution thresholds, and the average resolution value also meets the average resolution threshold, it is determined that the target device screen meets the first determination condition, and subsequent MTF value testing can be performed. If the resolution value of any first-region image does not meet its corresponding preset resolution threshold, or the average resolution value does not meet the preset average resolution threshold, it is determined that the target device screen does not meet the first determination condition, and subsequent MTF value testing is not required.

[0136] Step S510: If the screen does not meet the first determination condition, an abnormality analysis is performed on the resolution value to obtain a data analysis report;

[0137] Step S520: Based on the data analysis report, an alarm is issued and / or corresponding expert advice is determined to repair the target device.

[0138] If the screen of the target device does not meet the first judgment condition, the screen detection result is unqualified. For unqualified target devices, the calculated data, here the resolution value, can be transmitted to the expert system for analysis, including abnormality type analysis, unqualified cause analysis, etc., to obtain a data analysis report so that the screen detection device can issue an alarm to remind the inspection personnel that there is an unqualified product and that timely measures must be taken. The expert system can also be used to obtain corresponding expert advice to remind maintenance personnel how to overcome the defects of the equipment and return the target device for repair.

[0139] Step S600: If the screen meets the first determination condition, a second display image of the target device is obtained, where the second display image is a photo obtained by shooting a preset second test image displayed on the screen.

[0140] The second test image is an image with alternating black and white stripes and the width of the black and white stripes is the same. Figure 5 The figure shows a schematic diagram of the second test image, which is a fringe image specifically used to detect the MTF value. Figure 5 (a) shows an image with alternating black and white stripes, in which the widths of the black and white stripes are the same.

[0141] Furthermore, there may be multiple second test images, and the stripes in the multiple second test images may have different widths and / or different directions. Figure 5 There are four second test images shown, Figure 5 (b) The stripe image and Figure 5 The stripe image in (a) has different widths. Figure 5 (c) The stripe image and Figure 5 The fringe image of (a) has different directions. Figure 5 (d) The stripe image and Figure 5 The fringe images in (a) vary in width and direction. In practical applications, a larger number of second test images or more different fringe images can be used. Using different images to test the MTF of a screen in multiple dimensions provides a more comprehensive and accurate test result.

[0142] Specifically, step S600 may include:

[0143] Step S610: controlling the screen of the target device to display a plurality of second test images in sequence, and acquiring corresponding second display images to obtain a plurality of second display images.

[0144] The target device's screen displays a variety of second test images, such as Figure 5 The four second test images are correspondingly photographed by the camera and then sent to the screen detection device, and the screen detection device can obtain the four second display images.

[0145] Step S700: performing image processing on the second display image.

[0146] Specifically, step S700 may include:

[0147] Step S710: pre-processing and gray-scaling the second display image to obtain a second screen image, wherein the size of the second screen image is consistent with the size of the screen area in the second display image;

[0148] Step S720: performing image correction on the second screen image to obtain a corrected second screen image.

[0149] Steps S710-S720 of processing the second display image are similar to steps S310 and S320 of processing the first display image, and are not described in detail here. It should be noted that when processing the second display image, the size of the screen area cannot be changed. For example, if the second display image is 50cm×60cm in size, due to the presence of other background frames, the actual size of the screen area within the image is only 30×40cm. The size of the second screen image obtained after image processing is also 30×40cm. The reason for not changing the size of the second screen image is to prevent the occurrence of moiré patterns, which could affect the calculation of the MTF value and subsequent test results.

[0150] Correspondingly, after obtaining the camera distortion compensation function in step S130, step S720 may include:

[0151] Step S721: performing image correction on the second screen image according to the distortion compensation function to obtain a corrected second screen image.

[0152] In a specific implementation, the distortion compensation function obtained in step S130 is used to correct the second screen image into a standard rectangle corresponding to the image actually displayed on the screen, thereby preventing camera distortion from affecting the detection result.

[0153] Step S800: judging whether the screen meets a second determination condition based on the MTF value of the second display image, and obtaining a detection result of the screen; wherein the second determination condition is that the MTF value is within a preset second threshold range.

[0154] Correspondingly, after obtaining the corrected second screen image in step S720, step S800 may include:

[0155] Step S800a: judging whether the screen meets the second determination condition according to the MTF value of the corrected second screen image, to obtain a detection result of the screen.

[0156] The MTF value is calculated for the image obtained after image processing, i.e., the corrected second screen image, and then the judgment is performed, which can improve the detection accuracy and prevent the background image other than the screen in the original image from affecting the detection result.

[0157] Specifically, step S800 can include:

[0158] Step S810: dividing each second display image into K×K regions to obtain a plurality of second region images.

[0159] Based on the obtained plurality of second display images or the second screen image corrected by step S720, the second detection of the entire detection process is performed. First, each second display image or each second screen image is divided into K×K small rectangular regions, such as 3×3 rectangular regions, to obtain a plurality of second region images, and 9 second region images are obtained here.

[0160] Step S820: obtaining the MTF value of each second region image according to a second calculation formula, wherein the second calculation formula is:

[0161]

[0162] wherein M represents the MTF value, I avg.max represents the maximum value in the pixel mean of all the fringes in the second region image, and I avg.min represents the minimum value in the pixel mean of all the fringes in the second region image.

[0163] The second display image is a fringe image, and the second region image obtained by dividing the second display image is an image with smaller overall size but still containing fringes. In the second region image, the pixel mean of each fringe is calculated first to obtain the pixel mean of all the fringes, and then comparison is performed to select the maximum value as I avg.max , and the minimum value as I avg.min, and then the MTF value of the second region image can be calculated according to the second calculation formula. It can be understood that since the pixel mean of pure black is 0 and the pixel mean of pure white is 255, even if the black and white stripes in the captured second display image are no longer pure black or pure white, the calculated pixel mean values ​​will not differ significantly. In other words, the maximum pixel mean value of all stripes in the second region image is also the maximum pixel mean value of all white stripes in the second region image, and the minimum pixel mean value of all stripes in the second region image is also the minimum pixel mean value of all black stripes in the second region image.

[0164] For each second region image, the respective MTF value is calculated respectively, and the MTF values ​​of all second region images can be obtained to ensure that the clarity of each region of the screen is good.

[0165] Step S830: Calculate an average MTF value based on the MTF values ​​of the plurality of second region images.

[0166] After obtaining the MTF values ​​of all the second region images, their average is calculated to obtain the average MTF value.

[0167] Step S840: determining whether the MTF values ​​of the plurality of second region images meet corresponding preset MTF thresholds, and whether the average MTF value meets a preset average MTF threshold;

[0168] For each second region image, the calculated MTF value of the second region image is compared with a preset MTF threshold corresponding to the second region image, and at the same time, the calculated average MTF value is compared with the preset average MTF threshold.

[0169] Step S850: If the MTF values ​​of the plurality of second region images all meet the preset MTF threshold, and the average MTF value meets the average MTF threshold, it is determined that the second display image meets the MTF requirement.

[0170] Correspondingly, if the MTF value of a second region image does not meet its corresponding preset MTF threshold, or the average MTF value does not meet the preset average MTF threshold, it is determined that the current second display image does not meet the MTF requirement.

[0171] When the MTF values ​​of all second-area images meet their corresponding preset MTF thresholds and the average MTF value also meets the average MTF threshold, it is determined that the current second display image meets the MTF requirement. However, it is not yet clear whether other second display images meet the MTF requirement, and therefore it is not clear whether the screen meets the second determination condition. In order to accurately ensure that the clarity of each area of ​​the screen is good, it is necessary to repeat the above steps S810-S850 to determine all second display images.

[0172] Step S860: When the plurality of second display images all meet the MTF requirement, it is determined that the screen meets the second determination condition, and a qualified detection result of the screen is obtained.

[0173] Step S870: When one of the second display images does not meet the MTF requirement, it is determined that the screen does not meet the second determination condition, and a detection result that the screen is unqualified is obtained.

[0174] In this embodiment, after repeating steps S810-S850 four times for four second display images, if all four second display images meet the MTF requirement, the target device's screen can be determined to meet the second determination condition, resulting in a final test result of the screen being qualified. However, if one second display image does not meet the MTF requirement, the target device's screen can be determined to not meet the second determination condition, resulting in a final test result of the screen being unqualified.

[0175] Step S910: If the screen does not meet the second judgment condition, perform an abnormality analysis on the MTF value to obtain a data analysis report;

[0176] Optionally, the resolution value and MTF value calculated for the target device in the above steps may be input into the expert system to perform anomaly analysis and obtain a corresponding data analysis report;

[0177] Step S920: Based on the data analysis report, an alarm is issued and / or corresponding expert advice is determined to repair the target device.

[0178] In this embodiment, for equipment that passes the inspection, it can flow into the next workstation for subsequent inspection of other components of the equipment or for packaging and storage; for equipment that fails the inspection, its display effect does not meet the standard, and the data obtained during its inspection process can be input into the expert system for analysis to obtain the next processing suggestions. Specifically, when step S460 determines that the screen does not meet the first judgment condition or step S870 determines that the screen does not meet the second judgment condition, it can be determined that the screen of the target device is unqualified, and the target device is an unqualified product. Next, the problematic image and the calculated resolution value and / or MTF value can be input into the expert system for sorting and analyzing the abnormal data, automatically outputting an abnormality report, summarizing the screen areas where problems often occur and forming a report, and can also issue warnings for abnormalities with a high probability of occurrence, reminding the inspection personnel to check whether there is a problem with the design or process, optimize the production process in advance, avoid producing more defective products, and help realize the self-organization, self-regulation, and self-decision-making of the intelligent factory. The expert system can also provide corresponding repair suggestions based on the data analysis report, such as simply classifying the anomalies, reminding to check whether the anomalies are caused by factors such as assembly process, component thermal drift, dust influence, etc., or provide other recommended treatment methods. It can also sort the products into the repair station that handles the corresponding anomalies for repair based on the repair suggestions, and finally re-test the screen of the processed products.

[0179] For more implementation details of the specific implementation of the above method steps, please refer to the description of the specific implementation in Example 1. For the sake of brevity of the description, they will not be repeated here.

[0180] The screen detection method provided in this embodiment realizes automated detection and introduces an expert system to give corresponding repair suggestions, so that defective products can be directly sent to the corresponding processing station to improve production efficiency. This method has high integration, high reliability and high automation, improves detection efficiency and detection accuracy, and can be extended to the production lines of many electronic equipment products containing screens to detect the display quality of the screen during the production test stage. This embodiment adds an expert system that can analyze the detection data and give corresponding repair suggestions for screen defects, so that defective products can be directly sent to the corresponding processing station to improve production efficiency. This method has high integration, high reliability and high automation, can perform accurate screen quality detection of electronic equipment, reduce labor costs, and can be extended to the production of more electronic equipment containing screens, with good applicability.

[0181] Example 3

[0182] Based on the same invention concept, Figure 2The hardware structure diagram of the screen detection device is shown in FIG. 1. The present embodiment provides a screen detection device, which can include a processor and a memory. The memory stores a screen detection program. When the screen detection program is executed by the processor, all or part of the steps of the screen detection method of the present embodiment are implemented.

[0183] Specifically, the screen detection device refers to a terminal device or a network device capable of realizing network connection. The terminal device can be a mobile phone, a computer, a tablet computer, a portable computer, an embedded industrial computer, etc. The network device can be a server, a cloud platform, etc.

[0184] It can be understood that the screen detection device may further include a communication bus, a user interface and a network interface. Among them, the communication bus is used to realize the connection and communication between these components; the user interface is used to connect to the client and communicate data with the client. The user interface may include output units such as display screens, speakers, etc., and input units such as keyboards, microphones, etc.; the network interface is used to connect to the background server and communicate data with the background server. The network interface may include input / output interfaces, such as standard wired interfaces and wireless interfaces such as Wi-Fi interfaces; the memory is used to store various types of data, which may include instructions of any application or method in the screen detection device, as well as application-related data. The memory may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), etc. Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk, etc.; optionally, the memory may also be a storage device independent of the processor; the processor is used to call the screen detection program stored in the memory and execute the screen detection method as described above. The processor may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and is used to execute all or part of the steps of each embodiment of the screen detection method as described above.

[0185] Need to explain, Figure 2The hardware structure shown in the figure does not constitute a limitation on the screen detection device of the present invention, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0186] Example 4

[0187] Based on the same inventive concept, this embodiment provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic memory, a magnetic disk, an optical disk, a server, etc. The storage medium stores a computer program that can be executed by one or more processors. When executed by the processor, the computer program can implement all or part of the steps of each embodiment of the screen detection method of the present invention.

[0188] It should be noted that the above embodiments are only optional embodiments of the present invention and do not limit the patent scope of the present invention. All equivalent structures or equivalent process changes made by using the contents of the present invention description and drawings under the inventive concept of the present invention, or directly or indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A screen detection method, characterized in that: The method comprises: Acquire a first display image of a target device, where the first display image is a photograph obtained by photographing a preset first test image displayed on a screen of the target device; Determining whether the screen meets a first determination condition based on the resolution value of the first display image; wherein the first determination condition is that the resolution value is within a preset first threshold range; If the screen meets the first determination condition, obtaining a second display image of the target device, where the second display image is a photo obtained by photographing the screen displaying a preset second test image; According to the MTF value of the second display image, it is determined whether the screen meets a second determination condition, and a detection result of the screen is obtained; wherein the second determination condition is that the MTF value is within a preset second threshold range.

2. The screen detection method according to claim 1, wherein: Before the step of acquiring the first display image of the target device, the method further includes: adjusting the position of the target device and the position of the camera based on a user operation, and fixing the target device and the camera; According to the distance between the screen of the target device and the camera, the working parameters of the camera are adjusted to shoot the screen through the camera.

3. The screen detection method according to claim 2, wherein: Before the step of determining whether the screen satisfies a first determination condition based on the resolution value of the first display image, the method further includes: Preprocessing and gray-scaling the first display image to obtain a first screen image; performing image correction on the first screen image to obtain a corrected first screen image; The step of judging whether the screen satisfies a first determination condition according to the resolution value of the first display image includes: According to the resolution value of the corrected first screen image, it is determined whether the screen meets a first determination condition.

4. The screen detection method according to claim 3, wherein: Before the step of performing image correction on the first screen image to obtain a corrected first screen image, the method further includes: Obtaining a calibration image of the camera, where the calibration image is an image captured when a third test image is located at the position of the screen, the third test image comprising a plurality of equally spaced concentric circles and at least one straight line, wherein the straight line passes through the center of the circle and has two intersections with each concentric circle; Comparing the calibration image with the third test image, determining distances from a plurality of intersection points of the plurality of concentric circles and the straight line to the center of the circle, and a degree of distortion of the concentric circles where the intersection points are located; Performing fitting based on the relationship between the distance and the degree of distortion to obtain a distortion compensation function of the camera; The step of performing image correction on the first screen image to obtain a corrected first screen image includes: Image correction is performed on the first screen image according to the distortion compensation function to obtain a corrected first screen image.

5. The screen detection method according to claim 1, wherein: The step of judging whether the screen satisfies a first determination condition according to the resolution value of the first display image includes: Dividing the first display image into N×N regions to obtain a plurality of first region images; For each first region image, a resolution value of the first region image is obtained according to a first calculation formula, wherein the first calculation formula is: Wherein, R represents the resolution value, x represents the horizontal coordinate of the image, y represents the vertical coordinate of the image, f(x,y) represents the grayscale value of the pixel point (x,y), m represents the number of pixels arranged horizontally in the first region image, and n represents the number of pixels arranged vertically in the first region image; Calculating an average resolution value according to the resolution values ​​of the plurality of first region images; Determining whether the resolution values ​​of the plurality of first region images meet corresponding preset resolution thresholds, and whether the average resolution value meets a preset average resolution threshold; If the resolution values ​​of the plurality of first region images all meet the preset resolution threshold, and the average resolution value meets the average resolution threshold, it is determined that the screen meets the first determination condition.

6. The screen detection method according to claim 1, wherein: The second test image is an image with alternating black and white stripes of the same width, there are multiple second test images, and the stripes in the multiple second test images have different widths and / or different directions; The step of acquiring the second display image of the target device includes: The screen of the target device is controlled to display the multiple second test images in sequence, and the corresponding second display images are acquired to obtain multiple second display images.

7. The screen detection method according to claim 6, wherein: The step of determining whether the screen meets a second determination condition based on the MTF value of the second display image and obtaining a detection result of the screen includes: For each second display image, the second display image is equally divided into K×K regions to obtain a plurality of second region images; For each second region image, the MTF value of the second region image is obtained according to a second calculation formula, wherein the second calculation formula is: Where M represents the MTF value, I avg.max represents the maximum value among the pixel means of all stripes in the second region image, I avg.min represents the minimum value among the pixel means of all stripes in the second region image; Calculating an average MTF value according to the MTF values ​​of the plurality of second region images; Determining whether the MTF values ​​of the plurality of second region images meet corresponding preset MTF thresholds, and whether the average MTF value meets a preset average MTF threshold; If the MTF values ​​of the plurality of second region images all meet the preset MTF threshold, and the average MTF value meets the average MTF threshold, then determining that the second display image meets the MTF requirement; When the plurality of second display images all meet the MTF requirement, it is determined that the screen meets the second determination condition, and a detection result that the screen is qualified is obtained.

8. The screen detection method according to claim 1, wherein: After the step of determining whether the screen satisfies a first determination condition based on the resolution value of the first display image, the method further includes: If the screen does not meet the first determination condition, performing abnormality analysis on the resolution value to obtain a data analysis report; Based on the data analysis report, an alarm prompt is issued and / or corresponding expert advice is determined to repair the target device.

9. A screen detection device, characterized in that: The device includes a processor and a memory, wherein a screen detection program is stored in the memory, and when the screen detection program is executed by the processor, the screen detection method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by one or more processors, the screen detection method according to any one of claims 1 to 8 is implemented.

Images