Display interface testing method and device, storage medium, and electronic device

Abstract

The present disclosure relates to a method and device for testing a display interface, a storage medium, and an electronic device, and relates to the field of automated testing technology. The method includes: obtaining a current screen image of a display terminal and an expected screen image corresponding to the current screen image; calculating a first mean square error between the current screen image and the expected screen image, and determining a first image matching relationship between the current screen image and the expected screen image based on the first mean square error; calculating a first difference map between the current screen image and the expected screen image, and determining a second image matching relationship between the current screen image and the expected screen image based on the first difference map; and testing the display interface of the display terminal based on the first image matching relationship and the second image matching relationship. The present disclosure improves testing efficiency.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of automated testing technology, and in particular, to a display interface testing method, a display interface testing device, a computer-readable storage medium, and an electronic device. Background Art

[0002] In the existing testing method, the display interface of the display terminal needs to be tested manually. However, manual testing will result in low testing efficiency and low accuracy of the test results.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0004] The purpose of the present disclosure is to provide a display interface testing method, a display interface testing device, a computer-readable storage medium and an electronic device, thereby at least to a certain extent overcoming the problems of low testing efficiency and low accuracy of test results caused by the limitations and defects of related technologies.

[0005] According to one aspect of the present disclosure, a method for testing a display interface is provided, comprising:

[0006] Acquire a current screen image of the display terminal and an expected screen image corresponding to the current screen image;

[0007] calculating a first mean square error between the current screen image and the expected screen image, and determining a first image matching relationship between the current screen image and the expected screen image according to the first mean square error;

[0008] calculating a first difference map between the current screen image and the expected screen image, and determining a second image matching relationship between the current screen image and the expected screen image based on the first difference map;

[0009] The display interface of the display terminal is tested according to the first image matching relationship and the second image matching relationship.

[0010] In an exemplary embodiment of the present disclosure, calculating a first mean square error between the current screen image and the expected screen image, and determining a first image matching relationship between the current screen image and the expected screen image based on the first mean square error, includes:

[0011] Acquire a first pixel value of a first pixel point included in the current screen image, and a second pixel value of a second pixel point in the expected screen image;

[0012] Calculating a pixel difference between the first pixel value and a second pixel value of a second pixel point corresponding to the first pixel point, and calculating the first mean square error based on the pixel difference;

[0013] A first difference relationship between the first mean square error and a first preset threshold is calculated, and a first image matching relationship between the current screen image and the expected screen image is obtained according to the first difference relationship.

[0014] In an exemplary embodiment of the present disclosure, calculating a first difference map between the current screen image and the expected screen image, and determining a second image matching relationship between the current screen image and the expected screen image based on the first difference map includes:

[0015] When it is determined that the first image matching relationship exists, generating a first difference map between the current screen image and the expected screen image according to the pixel difference;

[0016] Segmenting the first difference image based on a preset image segmentation algorithm to obtain a foreground image and a background image included in the first difference image, and calculating the number of non-zero pixels included in the foreground image;

[0017] A second image matching relationship between the current screen image and the expected screen image is obtained according to the number of non-zero pixels.

[0018] In an exemplary embodiment of the present disclosure, testing the display interface of the display terminal according to the first image matching relationship and the second image matching relationship includes:

[0019] When it is determined that the first image matching relationship exists and the second image matching relationship exists, determining that a display function test of a display interface of the display terminal passes;

[0020] When it is determined that the first image matching relationship does not exist or the second image matching relationship does not exist, it is determined that a display function test of the display interface of the display terminal has failed.

[0021] In an exemplary embodiment of the present disclosure, the display interface testing method further includes:

[0022] In response to an erasure event caused by a preset event input object acting on a preset area in the display interface, erasing the preset area to obtain a target screen image including the erased area;

[0023] Acquire an erased area in the target screen image and an original area in the display interface corresponding to the preset area, and calculate a second mean square error and a second difference map between the erased area and the original area;

[0024] determining a first area matching relationship and a second area matching relationship between the erased area and the preset area according to the second mean square error and the second difference map;

[0025] The erasing function of the display interface of the display terminal is tested according to the first area matching relationship and the second area matching relationship.

[0026] In an exemplary embodiment of the present disclosure, the display interface testing method further includes:

[0027] Obtaining a control screenshot corresponding to the interactive control included in the interactive interface, and matching a template page image of the interactive control corresponding to the control screenshot in a preset image template library;

[0028] Obtaining the coordinate position of the interactive control in the template page image, and sending the coordinate position to a preset event input object;

[0029] In response to a touch event of the preset event input object based on the coordinate position in the display interface, obtaining a touch area corresponding to the touch event, and calculating a third mean square error and a third difference map between the touch area and the control screenshot;

[0030] Based on the third mean square error and the third difference map, the first control matching relationship and the second control matching relationship between the touch area and the control screenshot are determined, and based on the first control matching relationship and the second control matching relationship, the touch function of the interactive control in the display interface of the display terminal is tested.

[0031] In an exemplary embodiment of the present disclosure, the display interface testing method further includes:

[0032] In response to a first input event of inputting preset content by a preset event input object acting on the display interface, generating original content corresponding to the first input event, and performing standardization processing on the original content to obtain standard content;

[0033] calculating a fourth difference map between a first target interface including the target content and the display interface, and calculating a sub-content image of each sub-content included in the standard content based on the fourth difference map;

[0034] calculating a fourth mean square error between the sub-content image and a sub-template image of a sub-template content corresponding to the sub-content in the preset content, and determining a content matching relationship between the sub-content image and the sub-template image based on the fourth mean square error;

[0035] The input function and / or machine learning function of the display interface of the display terminal is tested according to the content matching relationship.

[0036] In an exemplary embodiment of the present disclosure, calculating the sub-content image of each sub-content included in the standard content according to the fourth difference map includes:

[0037] Performing grayscale conversion on the fourth difference image to obtain a first binary difference image, and performing image segmentation on the first binary difference image using a preset image segmentation algorithm to obtain a first segmented black-and-white image;

[0038] Acquire a first original connected area included in the first segmented black-and-white image, and filter the first original connected area to obtain a first target connected area;

[0039] Cropping the first target connected area from the first segmented black-and-white image to obtain a first initial cropping result, and searching, based on a first preset search direction, for a first pixel point in the first initial cropping result whose third pixel value is greater than a second preset threshold;

[0040] Performing region growing with the first pixel as a seed point to fill an internal region of the graphic outline of the first initial cropping result to obtain a first target cropping result, and adjusting the size of the first target cropping result to obtain a sub-content image of each sub-content included in the standard content.

[0041] In an exemplary embodiment of the present disclosure, the original content includes original graphics and / or original characters;

[0042] The primitive shape includes any one or more of a rectangle, an ellipse, a triangle, a regular polygon and an irregular polygon.

[0043] In an exemplary embodiment of the present disclosure, when the original graphic is a rectangle, after calculating the sub-content image of each sub-content included in the standard content based on the fourth difference map, the display interface testing method further includes:

[0044] Obtaining the length of the long side and the length of the wide side of the rectangle, and calculating a first ratio between the length of the long side and the length of the wide side;

[0045] When it is determined that the first ratio is greater than a third preset threshold or less than a fourth preset threshold, determining that the original graphic is a rectangle;

[0046] When it is determined that the first ratio is between the third preset threshold and the fourth preset threshold, the original graphic is determined to be a square.

[0047] In an exemplary embodiment of the present disclosure, when the original graphic is an ellipse, after calculating the sub-content image of each sub-content included in the standard content based on the fourth difference map, the display interface testing method further includes:

[0048] Obtaining a length value of a first diameter of the ellipse in the ordinate direction and a length value of a second diameter in the abscissa direction, and calculating a second ratio between the length value of the first diameter and the length value of the second diameter;

[0049] When it is determined that the first ratio is greater than a third preset threshold or less than a fourth preset threshold, determining that the original graphic is an ellipse;

[0050] When it is determined that the first ratio is between the third preset threshold and the fourth preset threshold, the original graphic is determined to be a circle.

[0051] In an exemplary embodiment of the present disclosure, the display interface testing method further includes:

[0052] In response to a second input event in which a preset event input object acts on the display interface to input a line shape, generating an original line shape corresponding to the second input event;

[0053] Calculating a fifth difference map between the second target interface including the original line shape and the display interface, and performing a binarization process on the fifth difference map to obtain a second binary difference map;

[0054] Performing image segmentation on the second binary difference image using a preset image segmentation algorithm to obtain a second segmented black-and-white image, and acquiring a second original connected region included in the second segmented black-and-white image;

[0055] The second original connected area is filtered to obtain a second target connected area, and the line input function of the display interface of the display terminal is tested based on the second target connected area.

[0056] In an exemplary embodiment of the present disclosure, the line input function includes at least one of a line color input function, a line smoothness input function, and a line width input function.

[0057] In an exemplary embodiment of the present disclosure, when the line input function is a line color input function, testing the line input function of the display interface of the display terminal based on the second target connected area includes:

[0058] Traversing each pixel point within the area corresponding to the second target interface and the second target connected area, obtaining a fourth pixel value of each pixel point within the area, and calculating a first average value of the fourth pixel values;

[0059] calculating a color matching relationship of the original line shape according to a first average value of the fourth pixel values ​​and a target pixel value of the original line shape;

[0060] The line color input function of the display interface of the display terminal is tested according to the color matching relationship.

[0061] In an exemplary embodiment of the present disclosure, when the line input function is a line width input function, testing the line input function of the display interface of the display terminal based on the second target connected area includes:

[0062] Cropping the second target connected area from the second segmented black-and-white image to obtain a second initial cropping result, and scanning the second initial cropping result for a first pixel whose fifth pixel value is greater than a second preset threshold based on a second preset search direction and a preset pixel width;

[0063] Obtaining sixth pixel values ​​and seventh pixel values ​​of two pixel points adjacent to the first pixel point in a horizontal position, and determining a target drawing direction according to the sixth pixel values ​​and the seventh pixel values;

[0064] Drawing a square that meets a preset condition based on the target drawing direction, and calculating a second average value of side lengths of the square that meets the preset condition;

[0065] A line width matching relationship of the original line shape is calculated based on the second average value and the target line width of the original line shape, and a line width input function of the display interface of the display terminal is tested based on the line width matching relationship.

[0066] In an exemplary embodiment of the present disclosure, when the line input function is a line smoothness input function, testing the line input function of the display interface of the display terminal based on the second target connected area includes:

[0067] Searching for the first pixel point in the second initial cropping result whose sixth pixel value is greater than the second preset threshold based on the first preset search direction, and taking the first pixel point in the second initial cropping result whose sixth pixel value is greater than the second preset threshold as the target pixel point;

[0068] Calculating a first geodesic distance between all pixel points included in the second initial cropping result and the target pixel point, and determining a first endpoint of the original line shape from all pixel points according to the first geodesic distance;

[0069] Calculating a second geodesic distance between all pixel points included in the second initial cropping result and the first endpoint, and determining a second endpoint of the original line shape from all pixel points according to the second geodesic distance;

[0070] Based on the endpoint position of the first endpoint, the endpoint position of the second endpoint, and the endpoint position of the target endpoint of the original line shape, the line smoothness matching relationship of the original line shape is calculated, and the line smoothness input function of the display interface of the display terminal is tested based on the line smoothness matching relationship.

[0071] In an exemplary embodiment of the present disclosure, the preset event input object includes a robotic arm.

[0072] According to one aspect of the present disclosure, there is provided a testing device for a display interface, comprising:

[0073] A first image acquisition module, configured to acquire a current screen image of the display terminal and an expected screen image corresponding to the current screen image;

[0074] a first calculation module, configured to calculate a first mean square error between the current screen image and the expected screen image, and determine a first image matching relationship between the current screen image and the expected screen image according to the first mean square error;

[0075] a second calculation module, configured to calculate a first difference map between the current screen image and the expected screen image, and determine a second image matching relationship between the current screen image and the expected screen image based on the first difference map;

[0076] The display interface testing module is configured to test the display interface of the display terminal according to the first image matching relationship and the second image matching relationship.

[0077] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements any one of the above-mentioned display interface testing methods.

[0078] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0079] processor; and

[0080] a memory for storing executable instructions of the processor;

[0081] The processor is configured to execute any one of the above-mentioned display interface testing methods by executing the executable instructions.

[0082] A method for testing a display interface provided by an embodiment of the present disclosure, on the one hand, because the display function of the display interface of the display terminal can be tested based on the first image matching relationship and the second image matching relationship between the current screen image and the expected screen image, automated testing of the display interface of the display terminal is achieved, solving the problem of low testing efficiency caused by manual testing in the prior art; on the other hand, a first mean square error between the current screen image and the expected screen image is calculated, and a first image matching relationship between the current screen image and the expected screen image is determined based on the first mean square error; then a first difference map between the current screen image and the expected screen image is calculated, and a second image matching relationship between the current screen image and the expected screen image is determined based on the first difference map; finally, the display interface of the display terminal is tested based on the first image matching relationship and the second image matching relationship. Since the first image matching relationship and the second image matching relationship are simultaneously considered in the process of testing the display interface, the problem of low accuracy of the test result caused by considering only one image matching relationship can be avoided, thereby improving the accuracy of the test result.

[0083] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0085] Figure 1 The flowchart schematically shows a method for testing a display interface according to an exemplary embodiment of the present disclosure.

[0086] Figure 2 An exemplary diagram schematically illustrates a display terminal according to an exemplary embodiment of the present disclosure.

[0087] Figure 3 A scene example diagram of a non-dot grid writing background according to an exemplary embodiment of the present disclosure is schematically shown.

[0088] Figure 4 A scene example diagram of a dot-grid writing background according to an exemplary embodiment of the present disclosure is schematically shown.

[0089] Figure 5 A flowchart schematically illustrates a method for testing an erase function included in a display interface of a display terminal according to an exemplary embodiment of the present disclosure.

[0090] Figure 6 A diagram schematically illustrates an example scenario of a target screen image including an erase area according to an exemplary embodiment of the present disclosure.

[0091] Figure 7 A flowchart schematically illustrates a method for testing the touch function of an interactive control included in a display interface of a display terminal according to an exemplary embodiment of the present disclosure.

[0092] Figure 8 A diagram schematically illustrates an example scenario of an interactive control in two different states according to an exemplary embodiment of the present disclosure.

[0093] Figure 9 A flowchart schematically illustrates a method for testing an input function and / or a machine learning function of a display interface of a display terminal according to an exemplary embodiment of the present disclosure.

[0094] Figure 10 An example diagram schematically illustrates an original character and a standardized character according to an exemplary embodiment of the present disclosure.

[0095] Figure 11 A scene example diagram schematically illustrates a first initial cropping result according to an exemplary embodiment of the present disclosure.

[0096] Figure 12 A diagram schematically illustrates an example scene of a first target cropping result according to an exemplary embodiment of the present disclosure.

[0097] Figure 13 A flowchart schematically illustrates a method for testing a line input function of a display interface of a display terminal according to an exemplary embodiment of the present disclosure.

[0098] Figure 14 A diagram schematically illustrates an example scene of a specific shape of an original line according to an exemplary embodiment of the present disclosure.

[0099] Figure 15An exemplary diagram schematically illustrates squares included in an original line according to an exemplary embodiment of the present disclosure.

[0100] Figure 16 A diagram schematically illustrates an example scenario of geodesic distance according to an example embodiment of the present disclosure.

[0101] Figure 17 A block diagram schematically illustrates a testing device for a display interface according to an exemplary embodiment of the present disclosure.

[0102] Figure 18 An electronic device for implementing the above-mentioned display interface testing method according to an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0103] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be more comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or that other methods, components, devices, steps, etc. may be employed. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0104] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0105] The market for all-in-one conference machines has continued to expand in recent years. During the production process, the quality of both the software and hardware must be carefully controlled, for example, ensuring that all levels of the screen display function correctly and that writing is smooth. Traditional quality inspection methods require production line workers to continuously click through various function menus for testing, which consumes a lot of manpower.

[0106] Based on this, this example embodiment first provides a method for testing a display interface. This method can be run on a display terminal, which can be used in various scenarios such as conferences, teaching, and business. The smart interactive tablet has interactive functions such as touch, a large size (for example, generally larger than 50 inches), and is equipped with handwriting, wireless projection, and remote video conferencing software. Of course, those skilled in the art can also run the method disclosed in this disclosure on other platforms (for example, servers, server clusters, or cloud servers, etc.) as needed, and this exemplary embodiment does not specifically limit this. Specifically, refer to Figure 1 As shown, the test method of the display interface may include the following steps:

[0107] Step S110. Acquire a current screen image of the display terminal and an expected screen image corresponding to the current screen image;

[0108] Step S120. Calculating a first mean square error between the current screen image and the expected screen image, and determining a first image matching relationship between the current screen image and the expected screen image based on the first mean square error;

[0109] Step S130. Calculating a first difference map between the current screen image and the expected screen image, and determining a second image matching relationship between the current screen image and the expected screen image based on the first difference map;

[0110] Step S140: Testing the display interface of the display terminal according to the first image matching relationship and the second image matching relationship.

[0111] In the above-mentioned display interface testing method, on the one hand, since the display function of the display interface of the display terminal can be tested based on the first image matching relationship and the second image matching relationship between the current screen image and the expected screen image, automated testing of the display interface of the display terminal is realized, which solves the problem of low testing efficiency caused by manual testing in the existing technology; on the other hand, by calculating the first mean square error between the current screen image and the expected screen image, and determining the first image matching relationship between the current screen image and the expected screen image based on the first mean square error; then calculating the first difference map between the current screen image and the expected screen image, and determining the second image matching relationship between the current screen image and the expected screen image based on the first difference map; finally, based on the first image matching relationship and the second image matching relationship, the display interface of the display terminal is tested. Since the first image matching relationship and the second image matching relationship are considered at the same time in the process of testing the display interface, the problem of low accuracy of the test results caused by considering only one image matching relationship can be avoided, thereby improving the accuracy of the test results.

[0112] Hereinafter, the method for testing the display interface according to the exemplary embodiment of the present disclosure will be explained and illustrated in detail with reference to the accompanying drawings.

[0113] First, the display terminal described in the exemplary embodiment of the present disclosure is explained and described. Figure 2 As shown, the display terminal 200 may include a display (front end) 210, a processor (back end, which is provided with a corresponding algorithm, and therefore may also be referred to as the bottom layer of the algorithm) 220, and an input component, which may include a touch component; it should be noted here that the processor may include a processor provided on the cloud server side, or may include a local processor provided on the smart interactive tablet, and this example does not impose any special restrictions on this.

[0114] It should be noted here that the display interface testing method recorded in the example embodiment of the present disclosure can test the display function of the display interface of the display terminal. Of course, it can also test the erasing function of the display interface, the touch function of the interactive controls included in the display interface, the graphic beautification function of the display interface of the display terminal (that is, the AI ​​algorithm recognition function) and the line input function of the display interface of the display terminal; of course, in actual application, other functions of the display interface of the display terminal can also be tested, and this example does not impose any special restrictions on this.

[0115] Secondly, in a method for testing a display interface provided in an exemplary embodiment of the present disclosure:

[0116] In step S110, a current screen image of the display terminal and an expected screen image corresponding to the current screen image are acquired.

[0117] Specifically, in the actual application process, since the functional pages of each level of the all-in-one machine are pre-configured with specified content by the front end and then displayed. Therefore, when performing automated testing (also known as automated proofreading), the front end can pass the expected content image (the expected screen image corresponding to the current screen image) and the current screen of the all-in-one machine (the current screen image) to the algorithm bottom layer; that is, the algorithm bottom layer (processor) can obtain the current screen image of the display terminal and the expected screen image corresponding to the current screen image from the front end. Among them, the current screen image may include Figure 3 as well as Figure 4 As shown. Further, in Figure 3 The current screen image shown is a non-dot grid writing background, that is, a normal writing background board; Figure 4 In the current screen image shown, the background is a dot grid. Figure 3 as well as Figure 4 In the current screen image shown in , the corresponding interactive control area 301 and the interactive control area 401 are displayed below the current screen image. The interactive controls displayed in the interactive control area may include but are not limited to: color interactive controls, erase interactive controls, delete interactive controls, withdraw interactive controls, line interactive controls, handwriting interactive controls, etc. In the specific application process, the corresponding interactive controls can be selected according to actual needs, and this example does not impose any special restrictions on this.

[0118] In step S120, a first mean square error between the current screen image and the expected screen image is calculated, and a first image matching relationship between the current screen image and the expected screen image is determined based on the first mean square error.

[0119] In this example embodiment, calculating the first mean square error between the current screen image and the expected screen image, and determining the first image matching relationship between the current screen image and the expected screen image based on the first mean square error can be achieved in the following manner: first, obtaining the first pixel value of the first pixel point included in the current screen image and the second pixel value of the second pixel point of the expected screen image; secondly, calculating the pixel difference between the first pixel value and the second pixel value of the second pixel point corresponding to the first pixel point, and calculating the first mean square error based on the pixel difference; then, calculating the first difference relationship between the first mean square error and a first preset threshold, and obtaining the first image matching relationship between the current screen image and the expected screen image based on the first difference relationship.

[0120] In an exemplary embodiment, the mean square error (MSE) can be represented by MSE. The mean square error can be used to characterize the average sum of the squares of the distances of each data from the true value, that is, the average sum of the squares of the errors. The specific calculation formula for the first mean square error can be shown as the following formula (1):

[0121]

[0122] Where n is the number of pixels in the current screen image and / or the expected screen image, X csi,i is the first pixel value of the first pixel point i in the current screen image, X dsi,iis the second pixel value of the i-th second pixel point in the desired screen image (Desired Screen Image). Further, after obtaining the first mean square error, the first image matching relationship can be determined based on the first mean square error; wherein, in a specific application process, when the first mean square error is less than a pre-set first preset threshold (the example embodiment of the present disclosure can set the threshold to an empirical value of 10), it is considered that the two page images (the current screen image and the desired screen image) may have a first image matching relationship. That is, if the absolute value of the first difference relationship between the first mean square error and the first preset threshold is less than or equal to 0.05 or 0.1, it can be determined that the first image matching relationship exists.

[0123] In step S130 , a first difference map between the current screen image and the expected screen image is calculated, and a second image matching relationship between the current screen image and the expected screen image is determined based on the first difference map.

[0124] In this example embodiment, calculating a first difference map between the current screen image and the expected screen image, and determining a second image matching relationship between the current screen image and the expected screen image based on the first difference map can be achieved in the following manner: first, when it is determined that the first image matching relationship exists, generating a first difference map between the current screen image and the expected screen image based on the pixel difference; second, segmenting the first difference map based on a preset image segmentation algorithm to obtain a foreground image and a background image included in the first difference map, and calculating the number of non-zero pixels included in the foreground image; then, obtaining the second image matching relationship between the current screen image and the expected screen image based on the number of non-zero pixels.

[0125] In an exemplary embodiment, when it is determined that there is a first image matching relationship between the current screen image and the expected screen image, the two images can be considered to be two images that may be matched; therefore, in order to further improve the accuracy of the test results, when it is determined that the first image matching relationship exists, a first difference map between the current screen image and the expected screen image can be calculated; wherein, in the process of calculating the first difference map, the first pixel value of the first pixel point included in the current screen image can be subtracted from the second pixel value of the second pixel point included in the expected screen image, and then the first difference map between the current screen image and the expected screen image can be obtained according to the difference between the first pixel value and the second pixel value; after obtaining the first difference map, the grayscale conversion can be performed on the first difference map, and then the grayscale processing of the first difference map can be performed based on a preset image segmentation algorithm. The value map is segmented to obtain the foreground image and the background image included in the first difference map; wherein, the preset image segmentation algorithm recorded here can be the maximum between-class variance method (OTSU algorithm), of course, it can also be other algorithms, such as Gaussian filtering algorithm, etc., and this example does not impose special restrictions on this; further, after obtaining the foreground image, the number of non-zero pixels included in the foreground image can be counted, and finally, based on the number of non-zero pixels, the second image matching relationship between the current screen image and the expected screen image is obtained; wherein, if the number of non-zero pixels is less than a preset threshold value (the example embodiment of the present disclosure sets the preset threshold value to an empirical value of 500), it is considered that the two page images do have a matching relationship (the second image matching relationship exists), and then it can be considered that the current screen image and the expected screen image are successfully matched.

[0126] In a possible example embodiment, the first image matching relationship and the second image matching relationship may also be calculated separately, that is, regardless of whether the first image matching relationship exists, the second image matching relationship is calculated, and finally corresponding tests are performed based on the first image matching relationship and the second image matching relationship.

[0127] In step S140 , the display interface of the display terminal is tested according to the first image matching relationship and the second image matching relationship.

[0128] In this example embodiment, the display interface of the display terminal is tested based on the first image matching relationship and the second image matching relationship. This can be specifically achieved in the following manner: on the one hand, when it is determined that the first image matching relationship exists and the second image matching relationship exists, it is determined that the display function test of the display interface of the display terminal has passed; on the other hand, when it is determined that the first image matching relationship does not exist or the second image matching relationship does not exist, it is determined that the display function test of the display interface of the display terminal has failed. That is, if the first image matching relationship exists and the second image matching relationship exists, it can be considered that the display effect of the current screen image displayed on the display interface of the display terminal is consistent with the desired display effect of the current screen image, and it can be considered that the display function of the display interface of the display terminal is fine and the test has passed; if either image matching relationship does not exist, it can be considered that there is a problem with the display function of the display interface and the test has failed; if the test has failed, the display interface in the display can be corrected.

[0129] Figure 5 The flowchart of a method for testing the erase function included in the display interface of the display terminal according to an exemplary embodiment of the present disclosure is schematically shown. Figure 5 As shown, the following steps may be included:

[0130] Step S510 , in response to an erasure event caused by a preset event input object acting on a preset area in the display interface, the preset area is erased to obtain a target screen image including the erased area.

[0131] Specifically, the preset event input object recorded here can be, for example, a robotic arm, or of course other input objects, such as an external input device, a finger, or a wireless stylus, etc. This example does not impose any special restrictions on this; the corresponding event input is performed through a robotic arm here in order to weaken the human factors in the testing process of the display interface of the display terminal, thereby achieving the goal of improving test efficiency and reducing labor costs on the basis of improving the accuracy of the test results.

[0132] Furthermore, in a specific test process, first, the range of the erasing area can be set and sent to the control unit of the robot arm, and then the robot arm is controlled to erase within the range of the set erasing area in the display interface; wherein, the target screen image including the erasing area is obtained, which can be specifically referred to Figure 6 shown.

[0133] Step S520 , obtaining the erased area in the target screen image and the original area in the display interface corresponding to the preset area, and calculating a second mean square error and a second difference map between the erased area and the original area.

[0134] Specifically, after obtaining the target image including the erased area, the display (front end) can pass the erased page (including the target screen image of the erased area), the original unwritten page (display interface) and the corner point coordinates of the specified erased area to the algorithm bottom layer (processor); then, the processor crops the erased area and the original area, and calculates the second mean square error and the second difference map between the erased area and the original area; wherein, the specific calculation process of the second mean square error and the second difference map is similar to the specific calculation process of the first mean square error and the first difference map, and will not be repeated here.

[0135] Step S530 : determining a first region matching relationship and a second region matching relationship between the erased region and the preset region according to the second mean square error and the second difference map.

[0136] Specifically, the specific process of determining the first region matching relationship and the second region matching relationship is similar to the specific process of determining the first image matching relationship and the second image matching relationship, and will not be further elaborated here.

[0137] Step S540: Testing the erase function of the display interface of the display terminal according to the first area matching relationship and the second area matching relationship.

[0138] Specifically, if both the first area matching relationship and the second area matching relationship exist, it is determined that the erasing function test in the display interface of the display terminal has passed, that is, in the specific application process, the erasing function can be executed normally; if either the first area matching relationship or the second area matching relationship does not exist, it is determined that the erasing function test in the display interface of the display terminal has failed. If it fails, further correction is required.

[0139] Figure 7 The following schematically illustrates a flow chart of a method for testing the touch function of an interactive control included in a display interface of a display terminal according to an exemplary embodiment of the present disclosure. Figure 7 As shown, the following steps may be included:

[0140] Step S710: Obtain a control screenshot corresponding to the interactive control included in the interactive interface, and match a template page image of the interactive control corresponding to the control screenshot in a preset image template library.

[0141] Specifically, when it is necessary to test one or more interactive controls included in the display interface, first, the front end can capture a control screenshot of the one or more interactive controls to be tested included in the display interface, and send the spatial screenshot of the one or more interactive controls to be tested and the display interface including the one or more interactive controls to be tested to the processor, and then the processor will match the template page image corresponding to the control screenshot in the preset image template library. It should be noted here that, since the interactive control can include two states, one is the selected state and the other is the unselected state; therefore, when sending the control screenshot of the interactive control to be tested, the control screenshots of both states can be sent to the processor. Among them, the specific state diagrams of the interactive controls in two different states can be referred to. Figure 8 As shown; wherein, the selected state can refer to 801, and the unselected state can refer to 802.

[0142] Step S720: Obtain the coordinate position of the interactive control in the template page image, and send the coordinate position to a preset event input object.

[0143] Specifically, after obtaining the corresponding template page image, the coordinate position of the interactive control in the template page image can be obtained, and then the coordinate position is sent to the preset event input object; wherein, the preset event input object can be a robotic arm; of course, it can also be other input objects, such as external input devices, fingers or wireless styluses, etc., this example does not impose special restrictions on this; the corresponding event input is performed through the robotic arm here in order to weaken the human factors in the test process of the display interface of the display terminal, thereby achieving the purpose of improving the test efficiency and reducing labor costs on the basis of improving the accuracy of the test results.

[0144] Step S730, in response to a touch event of the preset event input object based on the coordinate position in the display interface, obtaining a touch area corresponding to the touch event, and calculating a third mean square error and a third difference map between the touch area and the control screenshot.

[0145] Specifically, after the robotic arm receives the above-mentioned coordinate position, it can touch the area corresponding to the coordinate position in the display interface based on the coordinate position; then, the front end can obtain the touch area and send the touch area to the processor; after the processor receives the touch area, it can calculate the third mean square error and the third difference map between the touch area and the control screenshot to date; wherein, the specific calculation process of the third mean square error and the third difference map is similar to the calculation process of the first mean square error and the first difference map, and will not be further elaborated here. It should be noted here that since the control screenshot can include a screenshot of the control in a selected state and a screenshot of the control in an unselected state, it is necessary to calculate the corresponding third mean square error and third difference map separately in the specific calculation process.

[0146] Step S740, determine the first control matching relationship and the second control matching relationship between the touch area and the control screenshot based on the third mean square error and the third difference map, and test the touch function of the interactive control in the display interface of the display terminal based on the first control matching relationship and the second control matching relationship.

[0147] Specifically, after obtaining the third mean square error and the third difference map, the first control matching relationship and the second control matching relationship can be calculated; wherein, the specific calculation process of the first control matching relationship and the second control matching relationship is similar to the specific calculation process of the first image matching relationship and the second image matching relationship, and will not be further elaborated here; at the same time, after obtaining the first control matching relationship and the second control matching relationship, the touch function of the interactive control in the display interface of the display terminal can be tested; wherein, if both the first control matching relationship and the second control matching relationship exist, it is determined that the touch function test of the interactive control in the display interface of the display terminal has passed, that is, normal interaction can be carried out; if either the first control matching relationship or the second control matching relationship does not exist, it is considered that the touch function test of the interactive control in the display interface of the display terminal has failed, and further correction of the touch function of the interactive control is required.

[0148] It is necessary to add here that Figure 7The method for automatically verifying the function buttons on the all-in-one machine page shown in supports the rapid identification of the location of the specified button. In the specific application process, the front end passes the screenshot to be identified and the specified button screenshot to the bottom layer of the algorithm. Since the button display is generally divided into two states: selected and unselected, screenshots in both states can be passed; at the same time, the bottom layer of the algorithm uses the button screenshot to perform template matching on the screen page. When the matching score reaches a preset threshold (the specific threshold can be set to an empirical value of 0.99), the button location is found; further, the algorithm layer passes the coordinates of the upper left and lower right corners of the matched button area to the robotic arm controller, controlling the robotic arm to click the button; and then, in conjunction with the page verification function, it can realize automated testing of whether the button click function is normal.

[0149] Figure 9 The following schematically illustrates a flow chart of a method for testing the input function and / or machine learning function of a display interface of a display terminal according to an exemplary embodiment of the present disclosure. Figure 9 As shown, the following steps may be included:

[0150] Step S910 , in response to a first input event of inputting preset content by a preset event input object acting on the display interface, generating original content corresponding to the first input event, and standardizing the original content to obtain standard content.

[0151] Specifically, the display terminal (also known as a conference all-in-one machine, or an all-in-one machine) described in the exemplary embodiment of the present disclosure provides an AI (Artificial Intelligence) graphic or character recognition function, and the AI ​​algorithm can recognize handwritten graphics or characters and replace them with beautified graphic templates or character templates. Therefore, in the process of testing the AI ​​function, first, in response to a first input event in which a preset time input object acts on the display interface to input preset content, the original content corresponding to the first input event can be generated; wherein, the preset event input object is a robotic arm, and the original content may include original graphics, original characters, etc. The original graphics may include rectangles, ellipses, triangles, regular polygons, irregular polygons, etc., and of course may also include other content, and this example does not impose special restrictions on this. Further, after obtaining the original content, the original content can be standardized to obtain standard content; wherein, taking the original content as original characters as an example, the original characters and the standardized characters can refer to Figure 10 shown.

[0152] Step S920 , calculating a fourth difference map between the first target interface including the target content and the display interface, and calculating a sub-content image of each sub-content included in the standard content according to the fourth difference map.

[0153] In this example embodiment, first, a fourth difference map is calculated between the first target interface including the target content and the display interface; wherein the specific calculation process of the fourth difference map is similar to the specific calculation process of the first difference map, and is not repeated here; further, after obtaining the first difference map, the sub-content image of each sub-content included in the standard content can be calculated based on the fourth difference map. Among them, the specific calculation process can be implemented as follows: first, the fourth difference map is grayscale converted to obtain a first binary difference map, and the first binary difference map is segmented using a preset image segmentation algorithm to obtain a first segmented black and white image; second, the first original connected area included in the first segmented black and white image is obtained, and the first original connected area is filtered to obtain a first target connected area; then, the first target connected area is cropped from the first segmented black and white image to obtain a first initial cropping result, and the first pixel point whose third pixel value is greater than the second preset threshold in the first initial cropping result is searched based on a first preset search direction; finally, region growing is performed with the first pixel point as the seed point, and the internal area of ​​the graphic contour of the first initial cropping result is filled to obtain a first target cropping result, and the size of the first target cropping result is adjusted to obtain a sub-content image of each sub-content included in the standard content.

[0154] In some example embodiments, in the specific calculation process of the sub-content image, first, the fourth difference image can be converted into a grayscale image and then binary segmented using the OTSU algorithm to obtain a first segmented black-and-white image; second, the first original connected area is extracted from the first segmented black-and-white image, and connected areas with an area smaller than a specified threshold (in this example embodiment, the threshold can be set to an empirical value of 600) are filtered out to obtain a first target connected area; further, the areas corresponding to all first target connected areas are cropped from the entire black-and-white image to obtain a first initial cropping result; wherein, the obtained first initial cropping result can be specifically referred to. Figure 11 As shown; further, for the first initial cropping result corresponding to each first target connected area, the first pixel point whose third pixel value is not zero in the first initial cropping result is searched from top to bottom (first preset search direction); further, the region growing is started with the first pixel point as the seed point to fill the internal area of ​​the graphic outline to obtain the first target cropping result; wherein, the obtained first target cropping result can be specifically as shown Figure 12As shown; finally, the size of the first target cropping result is adjusted (Resized) to 256*256 to obtain the sub-content image.

[0155] In an example embodiment, when the original graphic is a rectangle, after calculating the sub-content image of each sub-content included in the standard content based on the fourth difference map, the testing method of the display interface further includes: first, obtaining the length value of the long side and the length value of the wide side of the rectangle, and calculating a first ratio between the length value of the long side and the length value of the wide side; second, when it is determined that the first ratio is greater than a third preset threshold or less than a fourth preset threshold, determining that the original graphic is a rectangle; finally, when it is determined that the first ratio is between the third preset threshold and the fourth preset threshold, determining that the original graphic is a square. In another example embodiment, when the original graphic is an ellipse, after calculating the sub-content image of each sub-content included in the standard content based on the fourth difference map, the testing method of the display interface also includes: first, obtaining the length value of the first diameter of the ellipse in the vertical coordinate direction and the length value of the second diameter in the horizontal coordinate direction, and calculating the second ratio between the length value of the first diameter and the length value of the second diameter; secondly, when it is determined that the first ratio is greater than a third preset threshold or less than a fourth preset threshold, determining that the original graphic is an ellipse; finally, when it is determined that the first ratio is between the third preset threshold and the fourth preset threshold, determining that the original graphic is a circle. That is, in a specific application process, since squares and rectangles, circles and ellipses are distinguished only in aspect ratio, the example embodiment of the present disclosure does not distinguish between squares and rectangles, circles and ellipses in the above-mentioned recognition process; instead, after obtaining the recognition result, it distinguishes according to the aspect ratio of the cropped result image, and considers that a square with an aspect ratio greater than a specified threshold (the third preset threshold set in the example embodiment of the present disclosure can be an empirical value of 0.9, and the fourth preset threshold can be an empirical value of 1.1) is a rectangle, and a circle is an ellipse; the distinction based on this method is simple and effective, and in the preliminary recognition process, it only needs to compare the target graphic with the square and circle templates respectively, which saves the amount of calculation.

[0156] Step S930 , calculating a fourth mean square error between the sub-content image and a sub-template image of a sub-template content corresponding to the sub-content in the preset content, and determining a content matching relationship between the sub-content image and the sub-template image based on the fourth mean square error.

[0157] Specifically, after obtaining the sub-content image, the fourth mean square error between the sub-content image and the sub-template image can be calculated, and then the fourth mean square error between the sub-content image and the sub-template image can be determined based on the fourth mean square error. The size of the character template image is consistent with the size of the sub-content image, which can be 256*256. Of course, it can also be of other sizes, and this example does not impose any special restrictions on this. At the same time, the specific calculation process of the fourth mean square error is similar to the specific calculation process of the first mean square error and will not be repeated here. Further, after obtaining the fourth mean square error, the content matching relationship between the sub-content image and the character template image can be determined based on the fourth mean square error. In the process of determining the content matching relationship, if the fourth mean square error is less than a specified threshold, it can be considered that the content matching relationship exists, that is, it can be considered that the image drawn by the robotic arm is the corresponding template image. The specified threshold recorded here can be set to an empirical value of 5000 according to actual needs, and of course it can also be set to other values. This example does not impose any special restrictions on this.

[0158] Step S940: Testing the input function and / or machine learning function of the display interface of the display terminal according to the content matching relationship.

[0159] Specifically, after obtaining the content matching relationship, the input function and / or machine learning function of the display interface of the display terminal can be tested according to the content matching relationship; that is, if the content matching relationship exists, it can be considered that the input function and / or machine learning function of the display interface of the display terminal has passed the test, and in actual application, input can be made and converted into standard graphics or standard characters for display; if the content matching relationship does not exist, it is considered that the input function and / or machine learning function of the display interface of the display terminal has failed the test and needs further correction.

[0160] Figure 13 The flowchart of a method for testing the line input function of the display interface of the display terminal according to an exemplary embodiment of the present disclosure is schematically shown. Figure 13 As shown, the following steps may be included:

[0161] Step S1310 : In response to a second input event in which a preset event input object acts on the display interface to input a line shape, an original line shape corresponding to the second input event is generated.

[0162] Specifically, the example embodiment of the present disclosure also provides a function for verifying handwritten lines on the all-in-one machine, so as to test the line input function in the all-in-one machine, so as to ensure that when the user uses the all-in-one machine, when inputting lines, the line color, line smoothness, line width and other functions can be guaranteed. In the specific testing process, the robotic arm (preset event input object) can be controlled to write in a designated area on the all-in-one machine screen; wherein, in the process of line input, one robotic arm input can be controlled, or multiple robotic arms input can be controlled at the same time, and this example does not impose special restrictions on this; further, when the display receives a line input event (second input event) from the robotic arm, it can generate an original line shape corresponding to the second input event; wherein, the original line shape can be specifically referred to Figure 14 shown.

[0163] Step S1320 , calculating a fifth difference map between the second target interface including the original line shape and the display interface, and performing a binarization process on the fifth difference map to obtain a second binary difference map.

[0164] Specifically, after obtaining the original line shape, the display can send the second target interface including the original line shape and the display interface (the original unwritten screenshot) to the algorithm bottom layer, wherein the processing flow of the algorithm bottom layer is: calculate the fifth difference map of the two images (the second target interface and the display interface); wherein, the specific calculation process of the fifth difference map is similar to the specific calculation process of the first difference map, and will not be further elaborated here; then, the fifth difference map is binarized (that is, grayscale processed) to obtain a second binary difference map.

[0165] Step S1330 : performing image segmentation on the second binary difference image using a preset image segmentation algorithm to obtain a second segmented black-and-white image, and acquiring a second original connected area included in the second segmented black-and-white image.

[0166] Specifically, after obtaining the second binary difference map, the preset image segmentation algorithm (maximum inter-class variance OTSU algorithm) can be used to perform image segmentation on the second binary difference map to obtain a second segmented black and white image; then, the second original connected area included in the second segmented black and white image is extracted.

[0167] Step S1340: Filter the second original connected area to obtain a second target connected area, and test the line input function of the display interface of the display terminal based on the second target connected area; wherein the line input function includes at least one of a line color input function, a line smoothness input function, and a line width input function.

[0168] Specifically, after obtaining the second original connected area, the second original connected area can be filtered to remove the second original connected area whose area is less than a specified threshold (the specified threshold can be set to an empirical value of 600 or other values, and this example does not impose any special restrictions on this), thereby obtaining the second target connected area. Furthermore, after obtaining the second target connected area, the line input function of the display interface of the display terminal can be tested based on the second target connected area. The testing of the line input function can include the following aspects:

[0169] On the one hand, when the line input function is a line color input function, the line input function of the display interface of the display terminal is tested based on the second target connected area, which can be achieved in the following way: first, traverse each pixel point inside the area corresponding to the second target interface and the second target connected area, obtain the fourth pixel value of each pixel point inside the area, and calculate the first average value of the fourth pixel value; secondly, calculate the color matching relationship of the original line shape according to the first average value of the fourth pixel value and the target pixel value of the original line shape; finally, test the line color input function of the display interface of the display terminal according to the color matching relationship. That is, in the process of testing the line color input function, for each second target connected area, the fourth pixel value of each pixel point located inside the area in the writing screenshot image (including the second target interface of the original line shape) can be traversed, and the mean of the three color channels of all pixel points is calculated (that is, the first average value of all fourth pixel values ​​is calculated). The extracted color value is compared with the target pixel value of the original line shape to obtain a color matching relationship, and the line color input function of the display interface of the display terminal is tested based on the color matching relationship.

[0170] It should be noted here that, in the process of comparing the first average value and the target pixel value, the mean square error and / or difference map between the first average value and the target pixel value can be calculated to determine the color matching relationship; at the same time, if the color matching relationship exists, it can be determined that the line color input function test has passed; if the color matching relationship does not exist, it can be determined that the line color input function test has failed; if the test fails, it needs to be calibrated again.

[0171] On the other hand, when the line input function is a line width input function, the line input function of the display interface of the display terminal is tested based on the second target connected area. This can be achieved in the following manner: first, the second target connected area is cropped from the second segmented black-and-white image to obtain a second initial cropping result, and the first pixel point whose fifth pixel value is greater than a second preset threshold in the second initial cropping result is scanned based on a second preset search direction and a preset pixel width; secondly, the sixth pixel value and the seventh pixel value of two pixel points adjacent to the first pixel point in the horizontal position are obtained, and a target drawing direction is determined based on the sixth pixel value and the seventh pixel value; then, a square that meets the preset conditions is drawn based on the target drawing direction, and a second average value of the side length of the square that meets the preset conditions is calculated; finally, a line width matching relationship of the original line shape is calculated based on the second average value and the target line width of the original line shape, and the line width input function of the display interface of the display terminal is tested based on the line width matching relationship. That is, in the process of line width verification, for each second target connected area, the corresponding area can be cropped from the second segmented black back image to obtain a second initial cropping result; then, the second initial cropping result is scanned every 10pix (preset pixel width) from left to right (second preset search direction) to find the first non-zero pixel on each scan line; check the pixel values ​​on both sides of the first pixel in the horizontal direction (the sixth pixel value and the seventh pixel value), and select a non-zero direction (target drawing direction, if the left pixel is not zero, then draw the largest (preset condition) square so that the square falls entirely within the line area; wherein, if Figure 15 The figure shows all the scanned squares. The second average value of the lengths of all positive sides is the line width. Furthermore, the line width matching relationship of the original line shape can be calculated based on the second average value and the target line width, and the line width input function of the display interface of the display terminal can be tested based on the line width matching relationship.

[0172] It should be noted that when calculating the line width matching relationship between the second average value and the target line width, this can be achieved through the mean square error between the second average value and the target line width, or through other methods, which are not specifically limited in this example. In addition, line width verification can not only confirm whether the all-in-one machine's selection of different line widths is effective; in addition, the handwriting beautification algorithm on the all-in-one machine generally requires adjusting the line width based on writing pressure and speed to achieve a simulated effect. This disclosure also provides technical support for automated testing in this scenario.

[0173] On the other hand, when the line input function is a line smoothness input function, the line input function of the display interface of the display terminal is tested based on the second target connected area. This can be achieved by: first, searching for the first pixel point in the second initial cropping result whose sixth pixel value is greater than a second preset threshold based on a first preset search direction, and using the first pixel point in the second initial cropping result whose sixth pixel value is greater than the second preset threshold as the target pixel point; second, calculating a first geodesic distance between all pixels included in the second initial cropping result and the target pixel point, and determining a first endpoint of the original line shape from all pixels based on the first geodesic distance; then, calculating a second geodesic distance between all pixels included in the second initial cropping result and the first endpoint, and determining a second endpoint of the original line shape from all pixels based on the second geodesic distance; finally, calculating a line smoothness matching relationship of the original line shape based on the endpoint position of the first endpoint, the endpoint position of the second endpoint, and the endpoint position of the target endpoint of the original line shape, and testing the line smoothness input function of the display interface of the display terminal based on the line smoothness matching relationship.

[0174] In an example embodiment, testing the line flow input function in the line input function can also be understood as checking the endpoints of the lines. Specifically, in the process of checking the endpoints of the lines, for each second target connected area, the corresponding second target connected area is cut out from the second segmented black and white image to obtain a second initial cut result; then, the first pixel point whose pixel value is not 0 (can also be understood as a pixel point whose pixel value is greater than zero) in the second initial cut result is searched from top to bottom (first preset search direction), and the first pixel point is used as the target pixel point; further, the first geodesic distance between all points of the second initial cut result and the target pixel point is calculated (wherein, the first geodesic distance can be specifically referred to Figure 16 As shown in the figure, the point with the farthest first geodesic distance from the target pixel is an endpoint of the line (i.e., the first endpoint). The second geodesic distances of all pixels in the image to the first endpoint are then calculated, and the point with the farthest second geodesic distance is selected as the second endpoint. Finally, based on the endpoint positions of the first endpoint, the second endpoint, and the target endpoint of the original line shape, the line smoothness matching relationship of the original line shape is calculated, and the line smoothness input function of the display interface of the display terminal is tested based on the line smoothness matching relationship. Furthermore, by comparing the actual written line endpoints with the target endpoint positions, it is possible to verify whether the line is broken, which is particularly important for writing on the edge of an all-in-one screen.

[0175] It should be noted here that the word "geodesic" comes from geodesy, which is a discipline that measures the size and shape of the earth. The original meaning of geodesic is the shortest path between two points on the earth's surface. Therefore, Geodesic Distance originally referred to the shortest distance between two points on the earth's surface, but then this concept was extended to the measurement of mathematical space. For example, in graph theory, Geodesic Distance is the distance of the shortest path between two nodes in the graph, which is different from the Euclidean Distance commonly used in geometric space, that is, the shortest distance between two points. Specifically, in Figure 16 In the figure, the Euclidean distance between the two black points is the length d15 of the dashed line segment, while the Geodesic distance, as the shortest distance of the actual path, should be the minimum sum of the distances of the solid line segments along the path, that is, d12+d23+d34+d45. Furthermore, in a three-dimensional mesh, the Geodesic distance is the shortest distance between two vertices along the mesh surface. In addition, to calculate the Geodesic distance between two points in a three-dimensional point cloud, we first construct a surface structure similar to a grid using all the points in the point cloud. Then, we calculate the Geodesic distance by finding the shortest path between the two points in the graph.

[0176] At this point, the test method for the display interface recorded in the exemplary embodiment of the present disclosure has been completed. Based on the above-mentioned content, it can be known that the test method for the display interface recorded in the exemplary embodiment of the present disclosure has at least the following advantages: on the one hand, the test method for the display interface recorded in the exemplary embodiment of the present disclosure can be a method for automatically proofreading the all-in-one page, which can not only quickly confirm whether the display of the function pages at all levels is accurate, but also support the verification of the erase function of the writing page; on the other hand, the test method for the display interface recorded in the exemplary embodiment of the present disclosure can be an automatic verification method for the function buttons on the all-in-one page, supporting the rapid identification of the location of the specified button, thereby controlling the robotic arm to perform the click operation; on the other hand, the test method for the display interface recorded in the exemplary embodiment of the present disclosure can recognize the beautification graphics on the all-in-one; at the same time, for the AI ​​graphic recognition function on the all-in-one, quickly verify whether the recognized graphics are accurate; further, the test method for the display interface recorded in the exemplary embodiment of the present disclosure can verify the handwritten lines on the all-in-one, realizing the inspection of line color, line smoothness and line width.

[0177] The exemplary embodiment of the present disclosure also provides a test device for displaying an interface. Figure 17 As shown, the display interface testing device may include a first image acquisition module 1710, a first calculation module 1720, a second calculation module 1730, and a display interface testing module 1740.

[0178] The first image acquisition module 1710 may be configured to acquire a current screen image of the display terminal and an expected screen image corresponding to the current screen image;

[0179] A first calculation module 1720 may be configured to calculate a first mean square error between the current screen image and the expected screen image, and determine a first image matching relationship between the current screen image and the expected screen image based on the first mean square error;

[0180] The second calculation module 1730 may be configured to calculate a first difference map between the current screen image and the expected screen image, and determine a second image matching relationship between the current screen image and the expected screen image based on the first difference map;

[0181] The display interface testing module 1740 may be configured to test the display interface of the display terminal according to the first image matching relationship and the second image matching relationship.

[0182] In an exemplary embodiment of the present disclosure, calculating a first mean square error between the current screen image and the expected screen image, and determining a first image matching relationship between the current screen image and the expected screen image based on the first mean square error, includes:

[0183] Acquire a first pixel value of a first pixel point included in the current screen image, and a second pixel value of a second pixel point in the expected screen image;

[0184] Calculating a pixel difference between the first pixel value and a second pixel value of a second pixel point corresponding to the first pixel point, and calculating the first mean square error based on the pixel difference;

[0185] A first difference relationship between the first mean square error and a first preset threshold is calculated, and a first image matching relationship between the current screen image and the expected screen image is obtained according to the first difference relationship.

[0186] In an exemplary embodiment of the present disclosure, calculating a first difference map between the current screen image and the expected screen image, and determining a second image matching relationship between the current screen image and the expected screen image based on the first difference map includes:

[0187] When it is determined that the first image matching relationship exists, generating a first difference map between the current screen image and the expected screen image according to the pixel difference;

[0188] Segmenting the first difference image based on a preset image segmentation algorithm to obtain a foreground image and a background image included in the first difference image, and calculating the number of non-zero pixels included in the foreground image;

[0189] A second image matching relationship between the current screen image and the expected screen image is obtained according to the number of non-zero pixels.

[0190] In an exemplary embodiment of the present disclosure, testing the display interface of the display terminal according to the first image matching relationship and the second image matching relationship includes:

[0191] When it is determined that the first image matching relationship exists and the second image matching relationship exists, determining that a display function test of a display interface of the display terminal passes;

[0192] When it is determined that the first image matching relationship does not exist or the second image matching relationship does not exist, it is determined that a display function test of the display interface of the display terminal has failed.

[0193] In an exemplary embodiment of the present disclosure, the testing device for the display interface further includes:

[0194] A first event response module may be configured to respond to an erasure event caused by a preset event input object acting on a preset area in the display interface, erase the preset area, and obtain a target screen image including the erased area;

[0195] a third calculation module, configured to obtain an erased area in the target screen image and an original area in the display interface corresponding to the preset area, and calculate a second mean square error and a second difference map between the erased area and the original area;

[0196] a fourth calculation module, configured to determine a first region matching relationship and a second region matching relationship between the erased region and the preset region according to the second mean square error and the second difference map;

[0197] The erasing function testing module may be configured to test the erasing function of the display interface of the display terminal according to the first area matching relationship and the second area matching relationship.

[0198] In an exemplary embodiment of the present disclosure, the testing device for the display interface further includes:

[0199] The template page image matching module can be used to obtain a control screenshot corresponding to the interactive control included in the interactive interface, and match the template page image of the interactive control corresponding to the control screenshot in a preset image template library;

[0200] A coordinate position sending module may be used to obtain the coordinate position of the interactive control in the template page image and send the coordinate position to a preset event input object;

[0201] A second event response module may be configured to, in response to a touch event of the preset event input object based on the coordinate position in the display interface, obtain a touch area corresponding to the touch event, and calculate a third mean square error and a third difference map between the touch area and the control screenshot;

[0202] The touch function testing module can be used to determine the first control matching relationship and the second control matching relationship between the touch area and the control screenshot based on the third mean square error and the third difference map, and test the touch function of the interactive control in the display interface of the display terminal based on the first control matching relationship and the second control matching relationship.

[0203] In an exemplary embodiment of the present disclosure, the testing device for the display interface further includes:

[0204] a third event response module, configured to respond to a first input event in which a preset event input object acts on the display interface to input preset content, generate original content corresponding to the first input event, and perform standardization processing on the original content to obtain standard content;

[0205] a fifth calculation module, configured to calculate a fourth difference map between the first target interface including the target content and the display interface, and calculate a sub-content image for each sub-content included in the standard content based on the fourth difference map;

[0206] a sixth calculation module, configured to calculate a fourth mean square error between the sub-content image and a sub-template image of a sub-template content corresponding to the sub-content in the preset content, and determine a content matching relationship between the sub-content image and the sub-template image based on the fourth mean square error;

[0207] The input function testing module can be used to test the input function and / or machine learning function of the display interface of the display terminal based on the content matching relationship.

[0208] In an exemplary embodiment of the present disclosure, calculating the sub-content image of each sub-content included in the standard content according to the fourth difference map includes:

[0209] Performing grayscale conversion on the fourth difference image to obtain a first binary difference image, and performing image segmentation on the first binary difference image using a preset image segmentation algorithm to obtain a first segmented black-and-white image;

[0210] Acquire a first original connected area included in the first segmented black-and-white image, and filter the first original connected area to obtain a first target connected area;

[0211] Cropping the first target connected area from the first segmented black-and-white image to obtain a first initial cropping result, and searching, based on a first preset search direction, for a first pixel point in the first initial cropping result whose third pixel value is greater than a second preset threshold;

[0212] Performing region growing with the first pixel as a seed point to fill an internal region of the graphic outline of the first initial cropping result to obtain a first target cropping result, and adjusting the size of the first target cropping result to obtain a sub-content image of each sub-content included in the standard content.

[0213] In an exemplary embodiment of the present disclosure, the original content includes original graphics and / or original characters;

[0214] The primitive shape includes any one or more of a rectangle, an ellipse, a triangle, a regular polygon and an irregular polygon.

[0215] In an exemplary embodiment of the present disclosure, when the original graphic is a rectangle, the sixth calculation module can also be configured to: obtain the length value of the long side and the length value of the wide side of the rectangle, and calculate the first ratio between the length value of the long side and the length value of the wide side; when it is determined that the first ratio is greater than a third preset threshold, or less than a fourth preset threshold, the original graphic is determined to be a rectangle; when it is determined that the first ratio is between the third preset threshold and the fourth preset threshold, the original graphic is determined to be a square.

[0216] In an exemplary embodiment of the present disclosure, when the original graphic is an ellipse, the sixth calculation module can also be configured to: obtain the length value of the first diameter of the ellipse in the ordinate direction and the length value of the second diameter in the abscissa direction, and calculate the second ratio between the length value of the first diameter and the length value of the second diameter; when it is determined that the first ratio is greater than a third preset threshold, or less than a fourth preset threshold, the original graphic is determined to be an ellipse; when it is determined that the first ratio is between the third preset threshold and the fourth preset threshold, the original graphic is determined to be a circle.

[0217] In an exemplary embodiment of the present disclosure, the testing device for the display interface further includes:

[0218] A fourth event response module may be configured to generate an original line shape corresponding to a second input event in which a preset event input object acts on the display interface to input a line shape;

[0219] a seventh calculation module, which may be used to calculate a fifth difference map between the second target interface including the original line shape and the display interface, and perform a binarization process on the fifth difference map to obtain a second binary difference map;

[0220] an eighth computing module, configured to perform image segmentation on the second binary difference image using a preset image segmentation algorithm to obtain a second segmented black-and-white image, and obtain a second original connected area included in the second segmented black-and-white image;

[0221] The line input function testing module may be configured to filter the second original connected area to obtain a second target connected area, and test the line input function of the display interface of the display terminal based on the second target connected area.

[0222] In an exemplary embodiment of the present disclosure, the line input function includes at least one of a line color input function, a line smoothness input function, and a line width input function.

[0223] In an exemplary embodiment of the present disclosure, when the line input function is a line color input function, testing the line input function of the display interface of the display terminal based on the second target connected area includes:

[0224] Traversing each pixel point within the area corresponding to the second target interface and the second target connected area, obtaining a fourth pixel value of each pixel point within the area, and calculating a first average value of the fourth pixel values;

[0225] calculating a color matching relationship of the original line shape according to a first average value of the fourth pixel values ​​and a target pixel value of the original line shape;

[0226] The line color input function of the display interface of the display terminal is tested according to the color matching relationship.

[0227] In an exemplary embodiment of the present disclosure, when the line input function is a line width input function, testing the line input function of the display interface of the display terminal based on the second target connected area includes:

[0228] Cropping the second target connected area from the second segmented black-and-white image to obtain a second initial cropping result, and scanning the second initial cropping result for a first pixel whose fifth pixel value is greater than a second preset threshold based on a second preset search direction and a preset pixel width;

[0229] Obtaining sixth pixel values ​​and seventh pixel values ​​of two pixel points adjacent to the first pixel point in a horizontal position, and determining a target drawing direction according to the sixth pixel values ​​and the seventh pixel values;

[0230] Drawing a square that meets a preset condition based on the target drawing direction, and calculating a second average value of side lengths of the square that meets the preset condition;

[0231] A line width matching relationship of the original line shape is calculated based on the second average value and the target line width of the original line shape, and a line width input function of the display interface of the display terminal is tested based on the line width matching relationship.

[0232] In an exemplary embodiment of the present disclosure, when the line input function is a line smoothness input function, testing the line input function of the display interface of the display terminal based on the second target connected area includes:

[0233] Searching for the first pixel point in the second initial cropping result whose sixth pixel value is greater than the second preset threshold based on the first preset search direction, and taking the first pixel point in the second initial cropping result whose sixth pixel value is greater than the second preset threshold as the target pixel point;

[0234] Calculating a first geodesic distance between all pixel points included in the second initial cropping result and the target pixel point, and determining a first endpoint of the original line shape from all pixel points according to the first geodesic distance;

[0235] Calculating a second geodesic distance between all pixel points included in the second initial cropping result and the first endpoint, and determining a second endpoint of the original line shape from all pixel points according to the second geodesic distance;

[0236] Based on the endpoint position of the first endpoint, the endpoint position of the second endpoint, and the endpoint position of the target endpoint of the original line shape, the line smoothness matching relationship of the original line shape is calculated, and the line smoothness input function of the display interface of the display terminal is tested based on the line smoothness matching relationship.

[0237] In an exemplary embodiment of the present disclosure, the preset event input object includes a robotic arm.

[0238] The specific details of each module in the above-mentioned display interface testing device have been described in detail in the corresponding display interface testing method, so they will not be repeated here.

[0239] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0240] Furthermore, although the steps of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0241] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0242] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods, or program products. Therefore, various aspects of the present disclosure may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0243] Refer to the following Figure 18 1800 according to this embodiment of the present disclosure will be described. Figure 18 The electronic device 1800 shown is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0244] like Figure 18 As shown, electronic device 1800 is implemented as a general-purpose computing device. Components of electronic device 1800 may include, but are not limited to, the aforementioned at least one processing unit 1810, the aforementioned at least one storage unit 1820, a bus 1830 connecting various system components (including storage unit 1820 and processing unit 1810), and a display unit 1840.

[0245] The storage unit stores program codes, which can be executed by the processing unit 1810, so that the processing unit 1810 performs the steps described in the "Exemplary Method" section of the present disclosure according to various exemplary embodiments. For example, the processing unit 1810 can perform the following steps: Figure 1 Step S110 shown in: obtaining a current screen image of the display terminal and an expected screen image corresponding to the current screen image; step S120: calculating a first mean square error between the current screen image and the expected screen image, and determining a first image matching relationship between the current screen image and the expected screen image based on the first mean square error; step S130: calculating a first difference map between the current screen image and the expected screen image, and determining a second image matching relationship between the current screen image and the expected screen image based on the first difference map; step S140: testing the display interface of the display terminal based on the first image matching relationship and the second image matching relationship.

[0246] The storage unit 1820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 18201 and / or a cache memory unit 18202 , and may further include a read-only memory unit (ROM) 18203 .

[0247] The storage unit 1820 may also include a program / utility 18204 having a set (at least one) of program modules 18205, such program modules 18205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0248] Bus 1830 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0249] Electronic device 1800 can also communicate with one or more external devices 1900 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1800, and / or any device that enables electronic device 1800 to communicate with one or more other computing devices (e.g., a router, modem, etc.). Such communication can occur via input / output (I / O) interface 1850. Furthermore, electronic device 1800 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 1860. As shown, network adapter 1860 communicates with other modules of electronic device 1800 via bus 1830. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with electronic device 1800, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0250] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0251] In exemplary embodiments of the present disclosure, a computer-readable storage medium is also provided, on which is stored a program product capable of implementing the aforementioned methods of this specification. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product comprising program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to execute the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present disclosure.

[0252] According to an embodiment of the present disclosure, a program product for implementing the above-mentioned method can be a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0253] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0254] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0255] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0256] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0257] Furthermore, the above-mentioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the above-mentioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0258] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not invented herein. The specification and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

Claims

1. A method for testing a display interface, characterized in that: include: Acquire a current screen image of the display terminal and an expected screen image corresponding to the current screen image; Acquire a first pixel value of a first pixel point included in the current screen image, and a second pixel value of a second pixel point in the expected screen image; Calculating a pixel difference between the first pixel value and a second pixel value of a second pixel point corresponding to the first pixel point, and calculating a first mean square error based on the pixel difference; Calculating a first difference relationship between the first mean square error and a first preset threshold, and obtaining a first image matching relationship between the current screen image and the expected screen image based on the first difference relationship; When it is determined that the first image matching relationship exists, generating a first difference map between the current screen image and the expected screen image based on the pixel difference; segmenting the first difference map based on a preset image segmentation algorithm to obtain a foreground image and a background image included in the first difference map, and calculating the number of non-zero pixels included in the foreground image; Obtaining a second image matching relationship between the current screen image and the expected screen image according to the number of non-zero pixels; When it is determined that the first image matching relationship exists and the second image matching relationship exists, determining that the display function test of the display interface of the display terminal has passed; when it is determined that the first image matching relationship does not exist or the second image matching relationship does not exist, determining that the display function test of the display interface of the display terminal has failed; Among them, the input function and / or machine learning function of the display interface of the display terminal is tested in the following manner: in response to a first input event in which a preset event input object acts on the display interface to input preset content, original content corresponding to the first input event is generated, and the original content is standardized to obtain standard content; a fourth difference map between the first target interface including the target content and the display interface is calculated, and a sub-content image of each sub-content included in the standard content is calculated based on the fourth difference map; a fourth mean square error is calculated between the sub-content image and the sub-template image of the sub-template content corresponding to the sub-content in the preset content, and a content matching relationship between the sub-content image and the sub-template image is determined based on the fourth mean square error; the input function and / or machine learning function of the display interface of the display terminal is tested based on the content matching relationship.

2. The display interface testing method according to claim 1, characterized in that: The display interface testing method further includes: In response to an erasure event caused by a preset event input object acting on a preset area in the display interface, erasing the preset area to obtain a target screen image including the erased area; Acquire an erased area in the target screen image and an original area in the display interface corresponding to the preset area, and calculate a second mean square error and a second difference map between the erased area and the original area; determining a first area matching relationship and a second area matching relationship between the erased area and the preset area according to the second mean square error and the second difference map; The erasing function of the display interface of the display terminal is tested according to the first area matching relationship and the second area matching relationship.

3. The display interface testing method according to claim 1, wherein: The display interface testing method further includes: Obtaining a control screenshot corresponding to the interactive control included in the display interface, and matching a template page image of the interactive control corresponding to the control screenshot in a preset image template library; Obtaining the coordinate position of the interactive control in the template page image, and sending the coordinate position to a preset event input object; In response to a touch event of the preset event input object based on the coordinate position in the display interface, obtaining a touch area corresponding to the touch event, and calculating a third mean square error and a third difference map between the touch area and the control screenshot; Based on the third mean square error and the third difference map, the first control matching relationship and the second control matching relationship between the touch area and the control screenshot are determined, and based on the first control matching relationship and the second control matching relationship, the touch function of the interactive control in the display interface of the display terminal is tested.

4. The display interface testing method according to claim 1, characterized in that: Calculating a sub-content image of each sub-content included in the standard content according to the fourth difference map includes: Performing grayscale conversion on the fourth difference image to obtain a first binary difference image, and performing image segmentation on the first binary difference image using a preset image segmentation algorithm to obtain a first segmented black-and-white image; Acquire a first original connected area included in the first segmented black-and-white image, and filter the first original connected area to obtain a first target connected area; Cropping the first target connected area from the first segmented black-and-white image to obtain a first initial cropping result, and searching, based on a first preset search direction, for a first pixel point in the first initial cropping result whose third pixel value is greater than a second preset threshold; Performing region growing with the first pixel as a seed point to fill an internal region of the graphic outline of the first initial cropping result to obtain a first target cropping result, and adjusting the size of the first target cropping result to obtain a sub-content image of each sub-content included in the standard content.

5. The display interface testing method according to claim 1, characterized in that: The original content includes original graphics and / or original characters; the original graphics include any one or more of rectangles, ellipses, triangles, regular polygons and irregular polygons.

6. The display interface testing method according to claim 5, characterized in that: When the original graphic is a rectangle, after calculating the sub-content image of each sub-content included in the standard content according to the fourth difference map, the display interface testing method further includes: Obtaining the length of the long side and the length of the wide side of the rectangle, and calculating a first ratio between the length of the long side and the length of the wide side; When it is determined that the first ratio is greater than a third preset threshold or less than a fourth preset threshold, determining that the original graphic is a rectangle; When it is determined that the first ratio is between the third preset threshold and the fourth preset threshold, the original graphic is determined to be a square.

7. The display interface testing method according to claim 5, characterized in that: When the original graphic is an ellipse, after calculating the sub-content image of each sub-content included in the standard content according to the fourth difference map, the display interface testing method further includes: Obtaining a length value of a first diameter of the ellipse in the ordinate direction and a length value of a second diameter in the abscissa direction, and calculating a first ratio between the length value of the first diameter and the length value of the second diameter; When it is determined that the first ratio is greater than a third preset threshold or less than a fourth preset threshold, determining that the original graphic is an ellipse; When it is determined that the first ratio is between the third preset threshold and the fourth preset threshold, the original graphic is determined to be a circle.

8. The display interface testing method according to claim 1, characterized in that: The display interface testing method further includes: In response to a second input event in which a preset event input object acts on the display interface to input a line shape, generating an original line shape corresponding to the second input event; Calculating a fifth difference map between the second target interface including the original line shape and the display interface, and performing a binarization process on the fifth difference map to obtain a second binary difference map; Performing image segmentation on the second binary difference image using a preset image segmentation algorithm to obtain a second segmented black-and-white image, and obtaining a second original connected region included in the second segmented black-and-white image; The second original connected area is filtered to obtain a second target connected area, and the line input function of the display interface of the display terminal is tested based on the second target connected area.

9. The display interface testing method according to claim 8, characterized in that: The line input function includes at least one of a line color input function, a line smoothness input function, and a line width input function.

10. The display interface testing method according to claim 9, characterized in that: When the line input function is a line color input function, testing the line input function of the display interface of the display terminal based on the second target connected area includes: Traversing each pixel point within the area corresponding to the second target interface and the second target connected area, obtaining a fourth pixel value of each pixel point within the area, and calculating a first average value of the fourth pixel values; calculating a color matching relationship of the original line shape according to a first average value of the fourth pixel values ​​and a target pixel value of the original line shape; The line color input function of the display interface of the display terminal is tested according to the color matching relationship.

11. The display interface testing method according to claim 9, characterized in that: When the line input function is a line width input function, testing the line input function of the display interface of the display terminal based on the second target connected area includes: Cropping the second target connected area from the second segmented black-and-white image to obtain a second initial cropping result, and scanning the second initial cropping result for a first pixel whose fifth pixel value is greater than a second preset threshold based on a second preset search direction and a preset pixel width; Obtaining sixth pixel values ​​and seventh pixel values ​​of two pixel points adjacent to the first pixel point in a horizontal position, and determining a target drawing direction according to the sixth pixel values ​​and the seventh pixel values; Drawing a square that meets a preset condition based on the target drawing direction, and calculating a second average value of side lengths of the square that meets the preset condition; A line width matching relationship of the original line shape is calculated based on the second average value and the target line width of the original line shape, and a line width input function of the display interface of the display terminal is tested based on the line width matching relationship.

12. The display interface testing method according to claim 11, characterized in that: When the line input function is a line smoothness input function, testing the line input function of the display interface of the display terminal based on the second target connected area includes: Searching for the first pixel point in the second initial cropping result whose sixth pixel value is greater than the second preset threshold based on the first preset search direction, and taking the first pixel point in the second initial cropping result whose sixth pixel value is greater than the second preset threshold as the target pixel point; Calculating a first geodesic distance between all pixel points included in the second initial cropping result and the target pixel point, and determining a first endpoint of the original line shape from all pixel points according to the first geodesic distance; Calculating a second geodesic distance between all pixel points included in the second initial cropping result and the first endpoint, and determining a second endpoint of the original line shape from all pixel points according to the second geodesic distance; Based on the endpoint position of the first endpoint, the endpoint position of the second endpoint, and the endpoint position of the target endpoint of the original line shape, the line smoothness matching relationship of the original line shape is calculated, and the line smoothness input function of the display interface of the display terminal is tested based on the line smoothness matching relationship.

13. The display interface testing method according to any one of claims 7 to 12, characterized in that: The preset event input object includes a robotic arm.

14. A display interface testing device, characterized in that: include: A first image acquisition module, configured to acquire a current screen image of the display terminal and an expected screen image corresponding to the current screen image; A first calculation module is configured to obtain a first pixel value of a first pixel point included in the current screen image and a second pixel value of a second pixel point in the expected screen image; Calculating a pixel difference between the first pixel value and a second pixel value of a second pixel point corresponding to the first pixel point, and calculating a first mean square error based on the pixel difference; Calculating a first difference relationship between the first mean square error and a first preset threshold, and obtaining a first image matching relationship between the current screen image and the expected screen image based on the first difference relationship; a second calculation module, configured to, when determining that the first image matching relationship exists, generate a first difference map between the current screen image and the expected screen image based on the pixel difference; segment the first difference map based on a preset image segmentation algorithm to obtain a foreground image and a background image included in the first difference map, and calculate the number of non-zero pixels included in the foreground image; Obtaining a second image matching relationship between the current screen image and the expected screen image according to the number of non-zero pixels; a display interface testing module configured to determine that a display function test of the display interface of the display terminal has passed when it is determined that the first image matching relationship exists and the second image matching relationship exists; and to determine that the display function test of the display interface of the display terminal has failed when it is determined that the first image matching relationship does not exist or the second image matching relationship does not exist; Among them, the input function and / or machine learning function of the display interface of the display terminal is tested in the following manner: in response to a first input event in which a preset event input object acts on the display interface to input preset content, original content corresponding to the first input event is generated, and the original content is standardized to obtain standard content; a fourth difference map between the first target interface including the target content and the display interface is calculated, and a sub-content image of each sub-content included in the standard content is calculated based on the fourth difference map; a fourth mean square error is calculated between the sub-content image and the sub-template image of the sub-template content corresponding to the sub-content in the preset content, and a content matching relationship between the sub-content image and the sub-template image is determined based on the fourth mean square error; the input function and / or machine learning function of the display interface of the display terminal is tested based on the content matching relationship.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for testing the display interface according to any one of claims 1 to 13 is implemented.

16. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the display interface testing method according to any one of claims 1 to 13 by executing the executable instructions.

Images