Display screen detection method and device

Abstract

The invention relates to a display screen detection method and device, and the method comprises the steps: obtaining an image set displayed by a display screen, and the image set comprises at least one image; obtaining a standard image set corresponding to the image set, wherein the standard image set is an image set in a lossless state; performing gradient feature extraction on each image in the image set and the standard image set to obtain a gradient feature corresponding to each image; performing local binary statistics on each image in the image set and the standard image set to obtain a local binary feature corresponding to each image; fusing the gradient feature and the local binary feature corresponding to each image to obtain a feature descriptor corresponding to each image; and performing similarity matching on the feature descriptor corresponding to each image in the image set and the feature descriptor corresponding to each image in the standard image set to obtain a detection result. The display screen detection efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of display screen detection, and in particular to a display screen detection method and device. Background Art

[0002] With the continuous advancement of social science and technology, more and more smart devices are constantly integrated into home appliances. Data linkage and infrared sensor light technology have long been used in home kitchens, among which LCD display screens are the core technologies and functions of smart home appliances and data linkage. However, in the process of LCD screen software development, various undesirable phenomena will occur due to the influence of the software and hardware operating environment, such as display abnormalities, animation freezes, etc. In order to detect the undesirable phenomena of the display function, it is often necessary to test the display function of the display. According to the operation mode: it can be divided into manual testing and automated program testing. In related technologies, display function testing usually needs to be done manually, and display control problems are extremely random, resulting in manual testing that requires a lot of time and effort, and extremely low efficiency, and even some abnormal interfaces are skipped because they are not checked in time by humans. Summary of the invention

[0003] In order to solve at least one of the above-mentioned technical problems, the present disclosure provides a display screen detection method and device.

[0004] In one aspect, the present invention provides a display screen detection method, the method comprising:

[0005] Acquire an image set presented on a display screen, the image set including at least one image;

[0006] Obtain a standard image set corresponding to the image set, where the standard image set is an image set in a lossless state;

[0007] Gradient features are extracted for each image in the image set and the standard image set to obtain the gradient features corresponding to each image;

[0008] Perform local binary statistics on each image in the image set and the standard image set to obtain local binary features corresponding to each image;

[0009] The gradient features and local binary features corresponding to each image are fused to obtain the feature descriptor corresponding to each image;

[0010] The feature descriptor corresponding to each image in the image set is matched with the feature descriptor corresponding to each image in the standard image set to obtain the detection result.

[0011] In an optional embodiment, the images in the image set have status tags, and the status tags indicate the working status of the display screen when the images are displayed on the display screen. Acquiring a standard image set corresponding to the image set includes:

[0012] Obtain an initial image set, the initial image set includes images in a standard image set, and all images in the initial image set are labeled with a state;

[0013] Performing clustering processing on the initial image set based on the state labels to obtain multiple classified image sets and cluster centers corresponding to the multiple classified image sets;

[0014] Calculating the similarity between the image set and the cluster centers corresponding to the multiple classified image sets to obtain a similarity threshold between the image set and the cluster centers corresponding to the multiple classified image sets;

[0015] The classified image set corresponding to the similarity threshold greater than the preset threshold is determined as the standard image set.

[0016] In an optional embodiment, when the detection result indicates that the display screen functions normally, the method further includes:

[0017] The image set is incorporated into the classified image set to obtain an updated classified image set;

[0018] Perform clustering processing on the updated classified image set to obtain updated cluster centers.

[0019] In an optional embodiment, gradient features are extracted for each image in the image set and the standard image set to obtain gradient features corresponding to each image, including:

[0020] Determine a key point in each image and a plurality of sampling points around the key point;

[0021] Gradient features are extracted from key points and multiple sampling points in each image to obtain the gradient features corresponding to each image.

[0022] In an optional embodiment, local binary statistics are performed on each image in the image set and the standard image set to obtain local binary features corresponding to each image, including:

[0023] Local binary statistics are performed on multiple sampling points in each image to obtain local binary features corresponding to each image.

[0024] In an optional embodiment, similarity matching is performed on a feature descriptor corresponding to each image in the image set and a feature descriptor corresponding to each image in the standard image set to obtain a detection result, including:

[0025] Performing similarity matching on the feature descriptor of the target image and the feature descriptor corresponding to each image in the standard image set, respectively, to obtain multiple matching values ​​corresponding to the target image, where the target image is any image in the image set;

[0026] When a matching value among the multiple matching values ​​is greater than the matching threshold, determining that the display function of the display screen is normal;

[0027] When the multiple matching values ​​are all smaller than the matching threshold, it is determined that the display function of the display screen is abnormal.

[0028] In a second aspect, the present invention further provides a display screen detection device, which is used to implement the above-mentioned display screen detection method, and the device includes a camera module, a networking and data storage module and a bracket;

[0029] The camera module is used to obtain the image set presented by the display screen;

[0030] The networking and data storage module is connected to the server for communication, and the networking and data storage module is used to store the image set and send the image set to the server;

[0031] The bracket is used to support the camera module.

[0032] In an optional embodiment, the bracket further includes: a telescopic mechanism and a base:

[0033] One end of the telescopic mechanism is connected to the camera module, and the other end of the telescopic mechanism is connected to the base;

[0034] The telescopic mechanism is at least used to adjust the height of the camera module.

[0035] In an optional embodiment, the bracket includes: a rotating mechanism;

[0036] The camera module is connected to the telescopic mechanism through a rotating mechanism, and the rotating mechanism is used to adjust the shooting angle of the camera module.

[0037] In an optional embodiment, the support further comprises: a moving mechanism:

[0038] The base is arranged on a moving mechanism, and the moving mechanism is used for adjusting the position of the camera module in the horizontal direction.

[0039] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0040] The implementation of this disclosure has the following beneficial effects:

[0041] An image set presented on a display screen is obtained, the image set including at least one image; a standard image set corresponding to the image set is obtained, the standard image set being an image set in a lossless state; gradient feature extraction is performed on each image in the image set and the standard image set, respectively, to obtain a gradient feature corresponding to each image; local binary statistics are performed on each image in the image set and the standard image set, respectively, to obtain a local binary feature corresponding to each image; the gradient feature and the local binary feature corresponding to each image are fused, to obtain a feature descriptor corresponding to each image; similarity matching is performed between the feature descriptor corresponding to each image in the image set and the feature descriptor corresponding to each image in the standard image set, to obtain a detection result.

[0042] The present disclosure can obtain the gradient features and local binary features corresponding to each image by performing gradient feature extraction and local binary statistics on each image in the image set and the standard image set respectively, and obtain the feature descriptor corresponding to each image by fusing the gradient features and local binary features corresponding to each image, so that the feature descriptor corresponding to each image can be enriched, and the efficiency of similarity matching between the feature descriptor corresponding to each image in the image set and the feature descriptor corresponding to each image in the standard image set is improved, thereby improving the efficiency of display screen detection.

[0043] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0045] Figure 1 is a schematic diagram of an implementation environment according to an exemplary embodiment;

[0046] Figure 2 is a flow chart of a display screen detection method according to an exemplary embodiment;

[0047] Figure 3 is a schematic diagram of a display screen detection device according to an exemplary embodiment;

[0048] Figure 4is a schematic diagram of a rotating mechanism 303 according to an exemplary embodiment;

[0049] Figure 5 is a detailed structural schematic diagram of a moving mechanism 304 according to an exemplary embodiment;

[0050] Figure 6 is a detailed structural schematic diagram of a composite fixing structure 401 according to an exemplary embodiment;

[0051] Figure 7 is a schematic diagram showing a working mode of a display screen detection device according to an exemplary embodiment;

[0052] Figure 8 is a block diagram of a display screen detection device according to an exemplary embodiment;

[0053] Fig. 9 is a block diagram of an electronic device for display screen detection according to an exemplary embodiment;

[0054] The following is a supplementary description of the attached drawings:

[0055] 1- camera module; 2- networking and data storage module; 3- bracket; 301- telescopic mechanism; 302- base; 303- rotating mechanism; 304- moving mechanism; 3041, 3042, 3043, 3044- slide rails; 401- composite fixed structure; 4011- magnetic structure; 4012 vacuum suction cup. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0058] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated. The word "exemplary" is used specifically herein to mean "serving as an example, embodiment, or illustrative". Any embodiment described herein as "exemplary" is not necessarily to be construed as being superior or better than other embodiments.

[0059] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0060] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0061] See also Figure 1 , Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment. Figure 1 As shown, the application environment may include a server 01 and a terminal 02 .

[0062] In an optional embodiment, the server 01 can be used for computing and processing in the display screen detection method. Specifically, the server 01 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms.

[0063] In an optional embodiment, the terminal 02 can perform calculation processing in combination with the display screen detection method of the server 01. Specifically, the terminal 02 may include but is not limited to electronic devices such as smart phones, desktop computers, tablet computers, laptop computers, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, etc. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux system, Windows system, Unix system, etc.

[0064] For example, the image set presented on the display screen and the standard image set corresponding to the image set are obtained through the terminal 02 and transmitted to the server 01. The image set includes at least one image, and the standard image set is an image set in a lossless state. The server 01 performs gradient feature extraction on each image in the image set and the standard image set to obtain the gradient feature corresponding to each image; performs local binary statistics on each image in the image set and the standard image set to obtain the local binary feature corresponding to each image; fuses the gradient feature and local binary feature corresponding to each image to obtain the feature descriptor corresponding to each image; performs similarity matching on the feature descriptor corresponding to each image in the image set and the feature descriptor corresponding to each image in the standard image set to obtain the detection result, and transmits the detection result to the terminal 02.

[0065] In addition, it should be noted that Figure 1 What is shown is only one application environment provided by the present disclosure. In actual application, other application environments may also be included.

[0066] In the embodiments of this specification, the server 01 and the terminal 02 may be directly or indirectly connected via wired or wireless communication, which is not limited in this disclosure.

[0067] Figure 2 is a flow chart of a display screen detection method according to an exemplary embodiment. Figure 2The display screen detection method shown includes the following:

[0068] Step S201: Acquire an image set presented on a display screen, where the image set includes at least one image.

[0069] In the embodiment of the present disclosure, a method for obtaining an image set presented on a display screen may be to set up a camera module in front of the display screen to obtain the image set presented on the display screen, and the images in the image set presented on the display screen are images displayed on the display screen in different working states.

[0070] Step S202: Acquire a standard image set corresponding to the image set, where the standard image set is an image set in a lossless state.

[0071] In an optional embodiment, the images in the image set have status tags, and the status tags indicate the working status of the display screen when the images are displayed on the display screen. Acquiring a standard image set corresponding to the image set includes:

[0072] Step S2021: obtaining an initial image set, where the initial image set includes images in a standard image set, and all images in the initial image set are provided with status labels.

[0073] In the embodiment of the present disclosure, the multiple images in the initial image set are composed of lossless images displayed by different display screens in different working states, and the state labels carried by the images in the initial image set are manually annotated. The annotation formula is as follows (1):

[0074] y i |y i ∈Y=(x i ,σ j )(j=1,2,3,4…f) (1)

[0075] In formula (1), Y represents the initial image set, y i represents the i-th data in the initial image set, x i represents the image in the initial image set, σ j is the state label, indicating image x i The jth of the f state parameters of the display when the display is displayed.

[0076] Step S2022: clustering the initial image set based on the state labels to obtain a plurality of classified image sets and cluster centers corresponding to the plurality of classified image sets.

[0077] In the embodiment of the present disclosure, in order to improve the processing efficiency, the state tag σ is used. j As a classification reference, the initial image set is input into the clustering model for classification, and multiple classification image sets Y are obtained. k|k=0,1,2,...,K, simultaneously calculate and obtain the cluster centers corresponding to multiple classification image sets Among them, K is the number of classifications preset in the clustering model, and the value depends on the number of categories of the working status.

[0078] Step S2023: Calculate the similarity between the image set and the cluster centers corresponding to each of the multiple classified image sets to obtain a similarity threshold between the image set and the cluster centers corresponding to each of the multiple classified image sets.

[0079] In the embodiment of the present disclosure, the similarity calculation between the image set and the cluster centers corresponding to each of the multiple classified image sets can be performed by comparing the similarity between the state labels carried by each image in the image set and the state labels carried by each image in each classified image set, and obtaining multiple similarity thresholds corresponding to different classified image sets of the image set. The calculation formula of the similarity threshold is as follows:

[0080]

[0081] In formula (2), is the similarity threshold of the image set corresponding to a certain classification image set, It indicates the number of state labels carried by an image in the image set that are the same as the state labels carried by an image in the classification image set, and f indicates the total number of state labels carried by the classification image set.

[0082] Step S2024: Determine the classified image set corresponding to the similarity threshold greater than the preset threshold as the standard image set.

[0083] In the embodiment of the present disclosure, the preset threshold can be set to 0.9. When the similarity threshold of an image set corresponding to a certain classified image set is greater than 0.9, the classified image set is determined as the standard image set.

[0084] Based on the above, it can be known that in the embodiment of the present disclosure, by clustering the initial image set based on the state label, the initial image set can be classified according to the different states corresponding to different images through the clustering model, and multiple classified image sets and the cluster centers corresponding to each of the multiple classified image sets can be obtained; by calculating the similarity between the image set and the cluster centers corresponding to each of the multiple classified image sets, the similarity threshold between the image set and the cluster centers corresponding to each of the multiple classified image sets is obtained, and the classified image set corresponding to the similarity threshold greater than the preset threshold is determined as the standard image set, and the standard image set corresponding to the image set can be automatically screened out.

[0085] Step S203: performing gradient feature extraction on each image in the image set and the standard image set to obtain the gradient feature corresponding to each image.

[0086] In an optional embodiment, gradient features are extracted for each image in the image set and the standard image set to obtain gradient features corresponding to each image, including:

[0087] Step S2031: Determine the key points in each image and a plurality of sampling points around the key points.

[0088] In the embodiment of the present disclosure, each image includes a plurality of key points, and determining the key points in each image and a plurality of sampling points around the key points may be to establish four sampling points within a radius r of each key point.

[0089] Step S2032: extracting gradient features from key points and multiple sampling points in each image, and obtaining gradient features corresponding to each image.

[0090] In the disclosed embodiment, the features of the key points and the surrounding areas of the four sampling points corresponding to the key points in 8 directions (up, down, left, right, upper left, lower left, upper right, lower right) are counted to obtain the 5×8=40-dimensional gradient features of the key points. Optionally, the four sampling points corresponding to each key point are distributed on the upper, lower, left, and right sides of the key point. The 40-dimensional gradient features corresponding to the multiple key points of each image are combined to obtain the gradient features corresponding to each image.

[0091] Based on the above, it can be seen that in the embodiment of the present disclosure, by extracting gradient features of the key points in each image and multiple sampling points around them, it is possible to collect gradient features of image areas related to the image key points, improve the efficiency of gradient feature extraction for each image, and improve the accuracy of gradient features.

[0092] Step S204: performing local binary statistics on each image in the image set and the standard image set to obtain local binary features corresponding to each image.

[0093] In an optional embodiment, local binary statistics are performed on each image in the image set and the standard image set to obtain local binary features corresponding to each image, including:

[0094] Local binary statistics are performed on multiple sampling points in each image to obtain local binary features corresponding to each image.

[0095] In the disclosed embodiment, a local binary pattern (LBP) method is used to perform local binary statistics on multiple sampling points in each image. LBP is a feature extraction method for texture classification, which constructs texture features by comparing the grayscale values ​​of pixels and their surrounding neighborhoods. LBP features have the advantages of grayscale invariance and rotation invariance, which enables it to maintain stable feature expression under illumination changes and image rotation. The basic principle of LBP is to select a pixel as the center point in the image, and then compare the neighborhood pixels around the point with the center pixel. If the grayscale value of the neighborhood pixel is greater than or equal to the grayscale value of the center pixel, the binary value of the neighborhood pixel is 1, otherwise it is 0. In this way, each center pixel will obtain a binary pattern, which is usually converted to a decimal representation. These binary patterns can be used to construct feature descriptors for the entire image. LBP statistics are performed on the four sampling points around the key point in each image to obtain the binary values ​​of the key point. 4 = 16-dimensional LBP description data, the 16-dimensional LBP description data corresponding to multiple key points of each image are combined to obtain the local binary features corresponding to each image.

[0096] Based on the above, it can be seen that in the embodiments of the present disclosure, by performing local binary statistics on multiple sampling points in each image, local binary statistics on image areas related to image key points can be achieved, thereby enriching the feature descriptors corresponding to each image.

[0097] Step S205: Fusing the gradient features and local binary features corresponding to each image to obtain a feature descriptor corresponding to each image.

[0098] In the embodiment of the present disclosure, the i-th image y i As an example, the gradient features and local binary features corresponding to the image are combined to obtain the 56-dimensional feature descriptor corresponding to the image.

[0099] Step S206: performing similarity matching on the feature descriptor corresponding to each image in the image set and the feature descriptor corresponding to each image in the standard image set to obtain a detection result.

[0100] In an optional embodiment, similarity matching is performed on a feature descriptor corresponding to each image in the image set and a feature descriptor corresponding to each image in the standard image set to obtain a detection result, including:

[0101] Step S2061: performing similarity matching on the feature descriptor of the target image and the feature descriptor corresponding to each image in the standard image set, respectively, to obtain a plurality of matching values ​​corresponding to the target image, where the target image is any image in the image set.

[0102] In the embodiment of the present disclosure, after the feature descriptors corresponding to each image are obtained, the feature descriptors of the target image and the feature descriptors corresponding to each image in the standard image set are respectively matched for similarity in the following manner:

[0103] Taking the process of similarity matching between the target image and the jth image in the standard image set as an example, the target image corresponds to the jth image in the standard image set, that is, the matching value p between the target image and the jth image in the standard image set is j Calculated by the following formula (3):

[0104]

[0105] In formula (3), n represents the number of feature descriptors obtained in the target image. represents the feature descriptor of the i-th key point of the j-th sequence image in the standard image set, S k and S m Respectively represent the distance R in the target image i The feature descriptors of the closest point and the second closest point, d(R i j ,S k ) represents the feature descriptor R i j and S k The similarity between i j ,S m ) represents the feature descriptor R i j and S m The similarity between d(R i j ,S k ) can be calculated by the Euler formula (4): i j ,S m ) is calculated in the same way.

[0106]

[0107] When the feature descriptor R i j Its closest point feature descriptor S k and the next closest point feature descriptor S m The ratio is less than the threshold value T, that is

[0108]

[0109] It indicates that the similarity with the two descriptors is high, and the number of similar points is increased by 1, where T is the judgment threshold, which can be set to 0.8. Finally, the number of similar points is divided by the total number of key points to obtain the matching value p between the target image and the jth image in the standard image set. j .

[0110] Step S2062: when a matching value among the multiple matching values ​​is greater than the matching threshold, determining that the display function of the display screen is normal.

[0111] In the disclosed embodiment, the matching threshold may be set to 0.98. When a matching value among a plurality of matching values ​​corresponding to the target image is greater than 0.98, it is determined that the display function of the display screen is normal.

[0112] Step S2063: When the plurality of matching values ​​are all smaller than the matching threshold, it is determined that the display function of the display screen is abnormal.

[0113] In the embodiment of the present disclosure, when the multiple matching values ​​corresponding to the target image are all less than 0.98, it is determined that the display function of the display screen is abnormal.

[0114] Based on the above, it can be known that in the embodiment of the present disclosure, by performing similarity matching between the feature descriptor of the target image and the feature descriptor corresponding to each image in the standard image set, multiple matching values ​​corresponding to the target image are obtained, and image matching between the target image and multiple images in the standard image set can be completed to obtain an indication parameter indicating whether the target image is presented normally. When a certain matching value among the multiple matching values ​​is greater than the matching threshold, it is determined that the display function of the display screen is normal; when the multiple matching values ​​are all less than the matching threshold, it is determined that the display function of the display screen is abnormal, and the display function of the display screen can be automatically detected.

[0115] In an optional embodiment, when the detection result indicates that the display screen functions normally, the method further includes:

[0116] Step S301: Incorporate the image set into the classified image set to obtain an updated classified image set.

[0117] In the disclosed embodiment, when the detection result indicates that the display screen functions normally, the images in the image set can be regarded as images displayed losslessly by the display screen. Therefore, the image set can be included in the classified image set to update the classified image set.

[0118] Step S302: performing clustering processing on the updated classified image set to obtain updated cluster centers.

[0119] In the disclosed embodiment, the updated classified image set is re-input into the above clustering model to obtain updated cluster centers.

[0120] Based on the above, it can be seen that in the embodiment of the present disclosure, by incorporating the image set into the classified image set and clustering the updated classified image set, it is possible to update the cluster center, improve the accuracy and efficiency of subsequent clustering processing of the image set, and make the determination of the standard image more accurate.

[0121] Figure 3 It is a schematic diagram of a display screen detection device according to an exemplary embodiment, the device is used to implement the above-mentioned display screen detection method, the device includes a camera module 1, a networking and data storage module 2 and a bracket 3; the camera module 1 is used to obtain an image set presented on the display screen; the networking and data storage module 2 is communicatively connected to a server, and the networking and data storage module 2 is used to store the image set and send the image set to the server; the bracket 3 is used to support the camera module 1.

[0122] In the disclosed embodiment, the camera in the camera module 1 is a high-definition camera, and the networking and data storage module 2 includes a WiFi module and a storage module. The networking and data storage module 2 can be set on the outer shell of the camera module 1, and the camera module 1 is supported by the bracket 3 for shooting.

[0123] Based on the above, it can be known that in the embodiment of the present disclosure, the image set presented on the display screen is acquired through the camera module 1, and the image set is stored through the networking and data storage module 2 and sent to the server, so that the image set presented on the display screen can be collected and transmitted, so that the collected image set presented on the display screen can be detected by the server, thereby realizing an automated display screen detection method.

[0124] In an optional embodiment, if Figure 3 As shown, the bracket 3 also includes: a telescopic mechanism 301 and a base 302: one end of the telescopic mechanism 301 is connected to the camera module 1, and the other end of the telescopic mechanism 301 is connected to the base 302; the telescopic mechanism 301 is at least used to adjust the height of the camera module 1.

[0125] In the disclosed embodiment, the telescopic mechanism 301 includes an inner rod and an outer rod. The outer rod is sleeved on the outside of the inner rod. The outer rod can be slidably adjusted along the inner rod. One end of the inner rod is connected to the camera module 1, and the other end of the inner rod is connected to the base 302.

[0126] Based on the above, it can be known that in the embodiment of the present disclosure, by setting a telescopic mechanism 301 to connect the camera module 1 and the base 302, the height of the camera module 1 can be adjusted by adjusting the height of the telescopic mechanism 301, so that the shooting height of the camera module 1 can adapt to the settings of the display screens of different home appliances.

[0127] In an optional embodiment, if Figure 4As shown, the bracket 3 includes: a rotating mechanism 303 ; the camera module 1 is connected to the telescopic mechanism 301 via the rotating mechanism 303 , and the rotating mechanism 303 is used to adjust the shooting angle of the camera module 1 .

[0128] In the disclosed embodiment, the rotating mechanism 303 is a hinge structure, which can support the module 1 to rotate in multiple directions.

[0129] Based on the above, it can be known that in the embodiment of the present disclosure, by setting the camera module 1 to be connected to the telescopic mechanism 301 through the rotating mechanism 303, the rotation adjustment of the rotating mechanism 303 can drive the adjustment of the shooting angle of the camera module 1, so that the shooting angle of the camera module 1 can adapt to the settings of the display screens of different home appliances.

[0130] In an optional embodiment, if Figure 3 As shown, the bracket 3 further includes: a moving mechanism 304: the base 302 is arranged on the moving mechanism 304, and the moving mechanism 304 is used to adjust the position of the camera module 1 in the horizontal direction.

[0131] In the present disclosure, Figure 5 As shown, the moving mechanism 304 includes two groups of slide rails, the first group of slide rails is composed of slide rails 3041 and slide rails 3042, slide rails 3041 and slide rails 3042 are parallel, the second group of slide rails is composed of slide rails 3043 and slide rails 3044, slide rails 3043 and slide rails 3044 are parallel, and the first group of slide rails is perpendicular to the second group of slide rails. The two sides of the base 302 are respectively connected with the slide rails 3043 and slide rails 3044 of the second group of slide rails, and the base 302 can slide along the slide rails 3043 and slide rails 3044 in the left and right directions, thereby realizing the left and right position adjustment of the base 302. The two ends of the second group of slide rails are respectively connected with the slide rails 3041 and slide rails 3042 of the first group of slide rails, and the first group of slide rails can slide along the slide rails 3041 and slide rails 3042 in the front and rear directions, thereby realizing the front and rear position adjustment of the base 302.

[0132] In the embodiment of the present disclosure, the bottom of both ends of the slide rail 3041 and the slide rail 3042 are provided with a composite fixing structure 401, such as Figure 6 As shown, the composite fixing structure 401 includes two fixing structures, a magnetic suction structure 4011 and a vacuum suction cup 4012, so that the composite fixing structure 401 can be better fixed on the home appliance.

[0133] Based on the above, it can be known that in the embodiment of the present disclosure, by setting the base 302 on the moving mechanism 304, the moving mechanism 304 can drive the base 302 to move forward, backward, left and right in the horizontal plane, so as to further realize the adjustment of the camera module 1 in the front, back, left and right directions in the horizontal plane through the connection relationship between the base 302 and the camera module 1, so that the shooting position of the camera module 1 can adapt to the position of the display screen of different home appliances.

[0134] In a specific embodiment, Figure 7 As shown, the process of the display screen detection device implementing the above-mentioned display screen detection method is as follows: fix the display screen detection device on the home appliance, and adjust the position through the telescopic mechanism 301, the rotating mechanism 303 and the moving mechanism 304, so that the display screen of the home appliance is placed in the field of view of the camera module 1. The display screen detection device and the display screen module are connected to the server through communication means such as WiFi and Bluetooth. After starting the detection program, the display screen detection device transmits the image presented by the display screen in real time to the server, and the home appliance transmits the operating status data of the display screen to the server. Through the initial image set constructed in advance, the image presented by the display screen in real time, and the operating status data of the display screen, the server determines whether it is currently operating normally.

[0135] Figure 8 is a block diagram of a display screen detection device according to an exemplary embodiment. Figure 8 The device includes a first acquisition module 801, a second acquisition module 802, a first feature extraction module 803, a second feature extraction module 804, a fusion module 805 and a matching module 806, wherein:

[0136] A first acquisition module 801 is used to acquire an image set presented on a display screen, where the image set includes at least one image;

[0137] The second acquisition module 802 is used to acquire a standard image set corresponding to the image set, where the standard image set is an image set in a lossless state;

[0138] The first feature extraction module 803 is used to extract gradient features from each image in the image set and the standard image set to obtain gradient features corresponding to each image;

[0139] The second feature extraction module 804 is used to perform local binary statistics on each image in the image set and the standard image set to obtain local binary features corresponding to each image;

[0140] A fusion module 805 is used to fuse the gradient features and local binary features corresponding to each image to obtain a feature descriptor corresponding to each image;

[0141] The matching module 806 is used to perform similarity matching between the feature descriptor corresponding to each image in the image set and the feature descriptor corresponding to each image in the standard image set to obtain a detection result.

[0142] In an optional embodiment, the images in the image set are provided with status labels, and the status labels indicate the working status of the display screen when the images are displayed on the display screen. The second acquisition module 802 includes:

[0143] An initial image set acquisition module is used to acquire an initial image set, wherein the initial image set includes images in a standard image set, and each image in the initial image set has a state label;

[0144] A first clustering module is used to perform clustering processing on the initial image set based on the state label to obtain a plurality of classified image sets and cluster centers corresponding to the plurality of classified image sets;

[0145] A similarity calculation module is used to perform similarity calculation on the image set and the cluster centers corresponding to each of the multiple classified image sets to obtain a similarity threshold between the image set and the cluster centers corresponding to each of the multiple classified image sets;

[0146] The standard image set determination module is used to determine the classified image set corresponding to the similarity threshold greater than the preset threshold as the standard image set.

[0147] In an optional embodiment, when the detection result indicates that the display screen functions normally, the device further includes:

[0148] A classification image set updating module, used for incorporating the image set into the classification image set to obtain an updated classification image set;

[0149] The second clustering module is used to perform clustering processing on the updated classified image set to obtain updated cluster centers.

[0150] In an optional embodiment, the first feature extraction module 803 includes:

[0151] A key point determination module, used to determine the key point in each image and multiple sampling points around the key point;

[0152] The gradient feature extraction module is used to extract gradient features from key points and multiple sampling points in each image, and obtain the gradient features corresponding to each image.

[0153] In an optional embodiment, the second feature extraction module 804 includes:

[0154] The local binary statistics module is used to perform local binary statistics on multiple sampling points in each image to obtain local binary features corresponding to each image.

[0155] In an optional embodiment, the matching module 806 includes:

[0156] A matching submodule, used for performing similarity matching between the feature descriptor of the target image and the feature descriptor corresponding to each image in the standard image set, and obtaining a plurality of matching values ​​corresponding to the target image, where the target image is any image in the image set;

[0157] A first function determination module, configured to determine that the display function of the display screen is normal when a certain matching value among the multiple matching values ​​is greater than a matching threshold;

[0158] The second function judgment module is used to determine that the display function of the display screen is abnormal when multiple matching values ​​are all smaller than the matching threshold.

[0159] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware such as processing circuits or memories, or a combination thereof. Similarly, one processor or multiple processors or memories can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0160] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is used for the instructions to implement the display screen detection method in the embodiment of the present disclosure.

[0161] Fig. 9 is a block diagram of an electronic device for display screen detection according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The electronic device includes a processor, a memory, a network interface, a display panel and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a display screen detection method is implemented. The display panel of the electronic device can be a liquid crystal display panel or an electronic ink display panel, and the input device of the electronic device can be a touch layer covered on the display panel, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0162] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0163] In an exemplary embodiment, a storage medium is further provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the display screen detection method in the embodiment of the present disclosure.

[0164] In an exemplary embodiment, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer is enabled to execute the display screen detection method in the embodiment of the present disclosure.

[0165] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present disclosure may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0166] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0167] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A display screen detection method, characterized in that: The method comprises: Acquire an image set presented on a display screen, the image set comprising at least one image; Acquire a standard image set corresponding to the image set, wherein the standard image set is an image set in a lossless state; Extracting gradient features from each image in the image set and the standard image set to obtain gradient features corresponding to each image; Performing local binary statistics on each image in the image set and the standard image set to obtain local binary features corresponding to each image; Fusing the gradient features and local binary features corresponding to each of the images to obtain a feature descriptor corresponding to each of the images; A similarity match is performed between the feature descriptor corresponding to each image in the image set and the feature descriptor corresponding to each image in the standard image set to obtain a detection result.

2. The method according to claim 1, characterized in that The images in the image set have status labels, and the status labels indicate the working status of the display screen when the images are displayed on the display screen. The step of obtaining the standard image set corresponding to the image set includes: Acquire an initial image set, wherein the initial image set includes images in the standard image set, and the images in the initial image set all have the state label; Performing clustering processing on the initial image set based on the state label to obtain a plurality of classified image sets and cluster centers corresponding to the plurality of classified image sets; Calculating similarity between the image set and the cluster centers corresponding to the multiple classified image sets to obtain similarity thresholds between the image set and the cluster centers corresponding to the multiple classified image sets; The classified image set corresponding to the similarity threshold greater than the preset threshold is determined as the standard image set.

3. The method according to claim 2, characterized in that When the detection result indicates that the display screen functions normally, the method further includes: Incorporating the image set into the classified image set to obtain an updated classified image set; The updated classified image set is clustered to obtain updated cluster centers.

4. The method according to claim 1, characterized in that: The step of extracting gradient features from each image in the image set and the standard image set to obtain gradient features corresponding to each image includes: Determine a key point in each of the images and a plurality of sampling points around the key point; Gradient features are extracted from the key points and the multiple sampling points in each image to obtain gradient features corresponding to each image.

5. The method according to claim 2, characterized in that: The performing local binary statistics on each image in the image set and the standard image set to obtain local binary features corresponding to each image includes: Perform local binary statistics on the multiple sampling points in each image to obtain local binary features corresponding to each image.

6. The method according to claim 1, characterized in that The performing similarity matching on the feature descriptor corresponding to each image in the image set and the feature descriptor corresponding to each image in the standard image set to obtain the detection result includes: Performing similarity matching on a feature descriptor of a target image and a feature descriptor corresponding to each image in the standard image set, respectively, to obtain a plurality of matching values ​​corresponding to the target image, wherein the target image is any image in the image set; When a matching value among the multiple matching values ​​is greater than a matching threshold, determining that the display function of the display screen is normal; When the multiple matching values ​​are all smaller than the matching threshold, it is determined that the display function of the display screen is abnormal.

7. A display screen detection device, characterized in that: The device is used to implement the display screen detection method according to any one of claims 1 to 7, and the device comprises a camera module (1), a networking and data storage module (2) and a bracket (3); The camera module (1) is used to obtain an image set presented on a display screen; The networking and data storage module (2) is connected to the server for communication, and the networking and data storage module (2) is used to store the image set and send the image set to the server; The bracket (3) is used to support the camera module (1).

8. The device according to claim 7, characterized in that The support (3) further comprises: a telescopic mechanism (301) and a base (302): One end of the telescopic mechanism (301) is connected to the camera module (1), and the other end of the telescopic mechanism (301) is connected to the base (302); The telescopic mechanism (301) is at least used to adjust the height of the camera module (1).

9. The device according to claim 8, characterized in that The support (3) comprises: a rotating mechanism (303); The camera module (1) is connected to the telescopic mechanism (301) via the rotating mechanism (303), and the rotating mechanism (303) is used to adjust the shooting angle of the camera module (1).

10. The device according to claim 8, characterized in that The support (3) further comprises: a moving mechanism (304): The base (302) is arranged on the moving mechanism (304), and the moving mechanism (304) is used to adjust the position of the camera module (1) in the horizontal direction.

Images