Intelligent test system for touch performance of large-size touch screen

Abstract

The invention relates to the technical field of screen testing, in particular to an intelligent testing system for touch performance of a large-size touch screen, which comprises a test case construction module for determining a plurality of touch points according to a tested touch screen and constructing a first test case according to the touch points and a specified operation set; the pollution environment parameter generation module generates simulation pollution parameters of different gradients for each touch point according to different surface pollutants and preset gradients, and a second test case is constructed according to the simulation pollution parameters. And the test module generates a corresponding UI layout according to the first test case and the second test case and executes test operation. The video acquisition module records the whole test process to obtain a test process video; and the data acquisition module acquires output data of the touch screen in real time through the communication interface and the touch screen driving IC. And the data analysis module analyzes the test video through a template matching technology and combines the output data to obtain a touch performance test result. Through systematic and comprehensive test process design and analysis, the test efficiency is improved.

Description

Technical Field

[0001] The invention relates to the technical field of screen testing, and in particular to a touch performance intelligent testing system for a large-size touch screen. Background Art

[0002] A touch screen is an input device that allows an operator to input and control information by directly touching the screen. It interacts with the device by detecting the touch position on the screen, providing the operator with an intuitive and convenient interactive experience. In recent years, with the advancement of touch screen research and development technology, the size of touch screens produced has also been increasing. At present, large-size touch screens are widely used in various fields such as medical, education, commerce and industry. At the same time, as the size of touch screens continues to grow, touch performance testing becomes more complex, because larger touch screens often need to support more touch points and complex gesture operations, which puts higher requirements on test accuracy and multi-touch detection.

[0003] In existing research, although the multi-touch capability of large-size touch screens has been tested and analyzed, the existing analysis and research is still not in-depth and comprehensive enough. The current intelligent testing technology for touch screen performance does not fully consider the impact of foreign matter that may occur during use, and the test cases cannot fully simulate actual usage, such as ignoring the grease, dust, sweat, dirt on the fingers, and tiny particles in the external environment. Therefore, the existing intelligent testing methods for large-size touch screen performance lack systematicity and comprehensiveness, which affects the overall efficiency of the test.

[0004] Therefore, an intelligent testing system for touch performance of large-size touch screens is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a large-size touch screen touch performance intelligent test system, including: a test case construction module, which determines a number of touch points according to the test touch screen, and constructs a first test case based on the touch points and a specified operation set; a pollution environment parameter generation module generates simulated pollution parameters of different gradients for each touch point according to different surface pollutants and preset gradients, and constructs a second test case. The test module generates a corresponding UI layout according to the first and second test cases and performs corresponding test operations. The video acquisition module is responsible for recording the entire test process through a camera to obtain a video of the test process; the data acquisition module collects the output data of the touch screen in real time through a communication interface and a touch screen driver IC. The data analysis module analyzes the test video through template matching technology, and combines the test output data to obtain the ideal touch performance test value and the touch performance test results under different pollution environments.

[0006] The present invention collects historical maintenance data of the same model as the test touch screen, and the system automatically identifies high-frequency fault areas, selects touch points more densely for testing in these areas prone to problems, and optimizes the distribution of touch points by combining the M×N grid division method to ensure comprehensive testing of key areas, thereby improving the pertinence and reliability of the test. The system automatically generates and optimizes test cases based on a test case construction method based on a genetic algorithm, so that the test case set achieves optimal effects in multiple dimensions such as global coverage, operation diversity, touch pressure change, and touch time change, thereby reducing redundant tests and improving the efficiency and accuracy of the test. By analyzing the test process video and extracting key image features such as shape, position, color, and movement trajectory, the system can automatically detect changes in the UI state, and accurately evaluate the response time, operation accuracy, and touch sensitivity of the touch operation based on pixel-level image difference analysis, thereby significantly improving the accuracy and consistency of the test results. Finally, by combining the dual-modal analysis of signal output data and key frame recognition, the system comprehensively evaluates the performance of the touch screen, which not only verifies the accuracy of the touch operation on physical data, but also accurately evaluates the response and operation fluency of the touch control through video feedback, thereby avoiding the limitations of single data analysis and improving the accuracy of the test results.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A large-size touch screen touch performance intelligent testing system, comprising:

[0009] A test case construction module is used to determine a number of touch points according to the test touch screen to obtain a touch point set; construct a first test case according to the touch point set and a specified operation set; the specified operation set includes click, slide, continuous click and multi-finger simultaneous operation;

[0010] A simulated pollution environment parameter generation module, used for selecting surface pollutants, generating simulated pollution parameters of different gradients for each touch point according to the surface pollutants and a preset gradient, and obtaining N simulated pollution environment parameters;

[0011] A real environment simulation test case construction module, used to construct a second test case according to the N simulated pollution environment parameters, in combination with the touch points and the specified operation set;

[0012] A testing module, configured to generate a corresponding UI layout according to the first test case and the second test case and perform corresponding test operations; the UI layout is configured to present interactive elements based on the touch point set;

[0013] The video acquisition module is used to use a camera to acquire a full-process test video when performing the test operation to obtain a test process video;

[0014] A data acquisition module, used to communicate with the touch screen driver IC through a communication interface, and to collect output data of the test touch screen in real time when performing the test operation to obtain test output data;

[0015] The data analysis module is used to obtain the test result through the template matching technology according to the test process video; analyze the test output data and the test result to obtain the ideal touch performance test value and N surface contamination touch performance test values.

[0016] Furthermore, the system further comprises: a surface contamination detection module, which is used to detect whether there are contaminants on the surface of the test touch screen, comprising the following steps:

[0017] Step 1: collecting a surface image of the test touch screen to obtain a first surface image;

[0018] Step 2: performing image processing on the first surface image to extract characteristic information of the pollutant, including pollutant type, pollutant shape and pollutant size;

[0019] Step 3: judging whether the pollutant exists according to the characteristic information;

[0020] When the presence of the contaminant is detected, a cleaning suggestion is generated; after a cleaning operation is performed according to the cleaning suggestion, steps one to three are repeated until the contaminant is no longer present.

[0021] Further, determining the plurality of touch points according to the test touch screen includes:

[0022] Collecting historical maintenance data of the touch screen of the same model as the tested touch screen; the historical maintenance data includes the fault type and the coordinates of the fault point;

[0023] Analyzing the historical maintenance data to identify high-frequency fault points in the high-frequency fault area;

[0024] The touch points include a first touch point, a second touch point and a third touch point;

[0025] Setting the high-frequency fault point as the first touch point;

[0026] Dividing the test touch screen into a plurality of grids by a grid division method, wherein the size of each grid is determined according to the resolution and physical size of the test touch screen;

[0027] If the overlap between the grid and the high-frequency fault area exceeds a preset threshold, the center point and four edge points of the grid are simultaneously set as the second touch points;

[0028] If the degree of overlap between the grid and the high-frequency fault area does not exceed the preset threshold, the center point of the grid is used as the third touch point.

[0029] Further, constructing the first test case according to the touch point set and the specified operation set includes:

[0030] Setting a touch pressure set including multiple touch pressure levels; setting a touch time set including different touch time lengths; setting the specified operation set including single-finger operation and multi-finger operation; selecting a single touch point when constructing a test case including the single-finger operation; selecting multiple touch points when constructing a test case including the multi-finger operation, wherein the multi-finger operation includes pinching of multiple touch points, rotation of multiple touch points, and multi-finger sliding;

[0031] Generate a set of initial test case sets, the initial test case set includes a plurality of test cases, each of the test cases includes: a touch point to be tested, an operation to be tested, an operation time, a touch pressure and an expected effect; the touch point to be tested is selected from the touch point set; the operation to be tested is selected from the specified operation set; the operation time is selected from the touch time set; the touch pressure is selected from the touch pressure set;

[0032] Fitness evaluation step: defining a fitness function according to global coverage, operation diversity, touch pressure change, touch time change and global operation total time; calculating a fitness value according to the fitness function;

[0033] Selection operation steps: Get the screening test case set through the sorting selection method;

[0034] Cross-operation step: performing a cross-operation on the screening test cases to generate a new test case combination to obtain a cross-case set;

[0035] Mutation operation step: randomly changing the parameters of the cross test case set to obtain a mutation test case set;

[0036] Next generation population generation step: combining the screening test case set, the cross test case set and the mutation test case set to form the next generation population;

[0037] The fitness evaluation step, the selection operation step, the crossover operation step, the mutation operation step and the next generation population generation step are repeated in sequence until a preset stop condition is met, and the test case set with the highest fitness value is output as the first test case.

[0038] Furthermore, the surface pollutants include: synthetic oil and fat pollutants, dust, tiny particles, moisture, artificially synthesized sweat and metal dust.

[0039] Further, performing a corresponding test operation according to the second test case includes:

[0040] Step 1: acquiring the current touch point of the current test case and the simulated pollution environment parameters from the second test case, and arranging the surface pollutants at the current touch point according to the simulated pollution environment parameters;

[0041] Step 2: After the surface contaminants are arranged, executing the current test case;

[0042] Step 3: After completing the current test case, clean the surface of the test touch screen to restore the surface of the test touch screen to an original pollution-free state;

[0043] Step 4: Repeat step 1 to step 3 until all the second test cases are tested.

[0044] Furthermore, the test output data includes: touch point coordinates, touch pressure, touch time and operation type.

[0045] Further, obtaining the test result by the template matching technology according to the test process video includes:

[0046] The test results include response time, operation accuracy and touch sensitivity; the process of obtaining the test results includes the following steps:

[0047] Obtain the expected effect corresponding to the test process video;

[0048] According to the test process video, the instantaneous image before the test and the instantaneous image after the test are identified and extracted by template matching technology;

[0049] Calculating the pixel-level difference between the instantaneous image before the test and the instantaneous image after the test using an image difference algorithm to generate a difference image;

[0050] According to the difference image, through binarization and connectivity analysis, the screen content change area caused by the touch operation is identified, and the image features of the screen content change area are extracted, and the image features include shape, position, color and movement trajectory;

[0051] According to the extracted image features, detecting UI state changes and obtaining actual operation feedback;

[0052] Obtaining the response time according to the timestamps of the instantaneous image before the test and the instantaneous image after the test;

[0053] By comparing the expected effect with the actual operation feedback, the operation accuracy is obtained;

[0054] The touch sensitivity is obtained by combining the response time and the operation accuracy.

[0055] Furthermore, analyzing the test output data and the test result to obtain the ideal touch performance test value and N surface contamination touch performance test values ​​includes:

[0056] Dividing the test output data into first test output data and second test output data according to the first test case and the second test case; dividing the second test output data into N polluted environment test output data according to the N simulated polluted environment parameters;

[0057] Preprocessing the first test output data and the N polluted environment test output data respectively to obtain first standard test output data and N standard polluted environment test output data;

[0058] Calculating a first touch performance test value according to the first standard test output data, the first touch performance test value including response time, operation accuracy and touch sensitivity;

[0059] Obtaining N second touch performance test values ​​according to the N standard pollution environment test output data, wherein the second touch performance test values ​​include response time, operation accuracy and touch sensitivity;

[0060] Dividing the test result into a first test result and a second test result according to the first test case and the second test case; dividing the second test result into N pollution environment test output results according to the N simulated pollution environment parameters;

[0061] averaging the first test result and the first touch performance test value and performing weighted addition to obtain a weighted first test result and a weighted first touch performance test value; adding the weighted first test result and the weighted first touch performance test value to obtain the ideal touch performance test value;

[0062] The polluted environment test output result and the corresponding second touch performance test value are respectively averaged and weighted to obtain a weighted polluted environment test output result and a weighted second touch performance test value; the weighted polluted environment test output result and the weighted second touch performance test value are added to obtain the surface contamination touch performance test value.

[0063] Furthermore, the system further comprises:

[0064] A feedback module is used to generate a feedback report according to the ideal touch performance test value and the surface contamination touch performance test value.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] 1. By collecting historical maintenance data of the same model as the touch screen being tested and analyzing the fault type and fault point, the system can automatically identify high-frequency fault areas and high-frequency fault points, and select touch points more densely for testing in these areas prone to problems. This touch point selection method based on fault history data ensures that the test covers the most critical areas and improves the pertinence and reliability of the test. At the same time, the M×N grid division method combines the resolution and physical size of the touch screen to accurately distribute the touch points to the entire screen surface, avoiding the omission of touch points. The construction of the touch point set also lays the foundation for the efficient execution of subsequent touch tests. By setting different touch point strategies for high-frequency fault areas and ordinary areas, the system can effectively optimize the distribution of touch points, which not only ensures comprehensive testing of key areas, but also improves the efficiency and accuracy of the test and reduces redundant testing.

[0067] 2. The present invention proposes a test case construction method based on genetic algorithm. Through genetic algorithm, the system can automatically generate and optimize test cases, so that the test case set can achieve the best effect in multiple dimensions such as global coverage of the touch screen, operation diversity, touch pressure change and touch time change. The genetic algorithm continuously optimizes the test case set through fitness evaluation, selection, crossover, mutation and other steps, making the testing process more efficient and accurate. Compared with the traditional manual design of test cases, genetic algorithms can intelligently select key parameter combinations and quickly find the most comprehensive and targeted test cases, thereby improving the comprehensiveness and efficiency of the test. In addition, through generational iteration, the system can dynamically adjust and optimize test cases to adapt to complex and changeable operating scenarios, ensuring that the performance of the touch screen under various pressure, time and operation combinations is fully evaluated. The genetic algorithm not only reduces redundant testing, but also improves the accuracy and execution efficiency of the test.

[0068] 3. By analyzing the video of the test process, the system can accurately capture the changes in screen content caused by touch operations, extract key image features such as shape, position, color and movement trajectory, and automatically detect changes in UI status. Based on pixel-level image difference analysis, the system can quickly detect the response time, operation accuracy and touch sensitivity of touch operations, improving the accuracy and consistency of test results. The system greatly improves test efficiency, reduces potential operation deviations, and ensures the stability and repeatability of multiple tests. Through this automated analysis method, the system can quickly and efficiently obtain the performance of the touch screen under a variety of operating conditions, significantly improving the efficiency and accuracy of the test. Combined with the comprehensive analysis of response time and operation accuracy, the system can comprehensively evaluate touch sensitivity to ensure that the response performance of the touch screen is reliably verified.

[0069] 4. The system can more comprehensively evaluate the performance of the touch screen by combining dual-mode analysis of signal output data and key frame recognition, avoiding the limitations of single data analysis. Video data and key frame recognition effectively supplement the shortcomings of signal output by capturing screen changes and visual feedback. It can not only verify the accuracy of touch operation in physical data, but also accurately evaluate the response and operation smoothness of touch through video feedback. Thereby improving the accuracy of touch test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A structural diagram of a large-size touch screen touch performance intelligent testing system provided by an embodiment of the present invention;

[0071] Figure 2 A schematic diagram of a touch screen coordinate system provided by an embodiment of the present invention;

[0072] Figure 3 A flowchart of generating a first test case based on a genetic algorithm provided in an embodiment of the present invention.

[0073] In the figure: 1. Test the touch screen. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] In recent years, with the advancement of touch screen technology, large-size touch screens have been used more and more widely. Due to their large size and many touch points, these touch screens usually need to support complex multi-touch operations and gesture recognition. However, as the size increases, the performance test of the touch screen becomes more complicated, and the selection and distribution of test points also face greater challenges. Large-size touch screens must ensure good touch response across the entire screen, while also needing to cope with a variety of user operation methods, which makes the design of the test plan more complicated.

[0076] In this context, most of the existing touch screen tests rely on test cases to verify specific functions or system behaviors. Test cases are definitions and documents used to verify a specific function or system behavior in a test. It describes input, operation steps, expected results and actual results, and is used to evaluate whether the system works as expected under various circumstances. Most of the existing touch tests are based on the signal output data of the test case. Although the signal output can provide data of physical quantities such as touch coordinates, pressure and time, it cannot fully capture the visual effects of user operations, especially when evaluating the response delay, touch feedback and user experience of the touch screen, video data and key frame recognition can supplement the shortcomings of signal output. At the same time, the design of existing test cases is usually difficult to fully cover all functions of the touch screen, and often cannot fully reflect the actual usage scenarios and diversified operation modes of the user. The present invention is described below in conjunction with two specific embodiments.

[0077] Embodiment 1

[0078] Company A is developing a large-size touch screen and needs to conduct comprehensive tests on its touch performance under different operating conditions and environments. The touch screen has an anti-pollution touch function, which can maintain the accuracy, sensitivity and response speed of touch operation when the screen surface is affected by a certain degree of contaminants such as grease, dust, and water droplets. This technology optimizes the sensing and signal processing of the touch screen, allowing it to work normally in complex environments without affecting the user experience due to surface contaminants. To this end, Company A introduced a large-size touch screen touch performance intelligent testing system to ensure the response speed, touch sensitivity and operation accuracy of its products in different usage environments.

[0079] A large-size touch screen touch performance intelligent test system, such as Figure 1 As shown, including:

[0080] A test case construction module is used to determine a number of touch points according to the test touch screen to obtain a touch point set; construct a first test case according to the touch point set and a specified operation set; the specified operation set includes single-finger operation and multi-finger simultaneous operation; the single-finger operation includes click, slide and continuous click;

[0081] Further, determining the plurality of touch points according to the test touch screen includes:

[0082] Collecting historical maintenance data of a touch screen of the same model as the test touch screen (a large-size touch screen for touch testing); the historical maintenance data includes the fault type and the coordinates of the fault point;

[0083] Analyzing the historical maintenance data to identify high-frequency fault points in the high-frequency fault area;

[0084] The touch points include a first touch point, a second touch point and a third touch point;

[0085] Setting the high-frequency fault point as the first touch point;

[0086] Dividing the test touch screen into a plurality of grids by an M×N grid division method, wherein the size of each grid is determined according to the resolution and physical size of the test touch screen;

[0087] If the overlap between the grid and the high-frequency fault area exceeds a preset threshold, the center point and four edge points of the grid are simultaneously set as the second touch points;

[0088] If the degree of overlap between the grid and the high-frequency fault area does not exceed the preset threshold, the center point of the grid is used as the third touch point.

[0089] In this embodiment, the physical dimensions of the test touch screen are: width 1200 mm, height 800 mm; resolution: 3840×2160 pixels (4K resolution);

[0090] like Figure 2 As shown, a rectangular coordinate system is established with the upper left corner of the test touch screen 1 as the origin, the X axis represents the width direction of the screen, and the Y axis represents the height direction of the screen. The position of each touch point of the test touch screen 1 is represented by (X, Y) coordinates.

[0091] In the rectangular coordinate system established with the upper left corner of the test touch screen 1 as the origin, the coordinate axis directions are:

[0092] X-axis: represents the horizontal direction (the width of the screen), increasing from left to right. That is, the value of the X-axis increases gradually from 0 (the left side of the screen) to the right, and the maximum value is the width of the screen, for example 3840 pixels.

[0093] Y axis: represents the vertical direction (the height direction of the screen), increasing from top to bottom. That is, the value of the Y axis gradually increases from 0 (the top of the screen) downward, and the maximum value is the height of the screen, for example, 2160 pixels.

[0094] The historical maintenance data is a maintenance record database of the same model touch screen maintained internally by Company A, including all maintenance records in the past two years.

[0095] Draw a scatter plot of all fault point coordinates on the screen coordinate system;

[0096] Use density clustering algorithm to cluster fault points and identify fault-intensive areas;

[0097] Determine the high-frequency fault area, as shown in Table 1;

[0098] Table 1. Statistics of high-frequency fault areas

[0099] area Coordinate range (X,Y) Number of fault points (items) Region A X: 0-800, Y: 0-400 300 Region B X: 2000-2800, Y: 1000-1600 250 Region C X: 3200-3840, Y: 1800-2160 150

[0100] In each high-frequency fault area, we further identify locations where fault points are highly concentrated, which are called high-frequency fault points. Table 2 shows the fault points in areas A, B, and C. The fault number threshold is 60 times, and fault points with a fault number greater than 60 times are identified as high-frequency fault points.

[0101] Table 2. High frequency fault point table

[0102]

[0103] According to the physical size and resolution of the touch screen, the test touch screen is divided into a plurality of grids by an M×N grid division method; in this embodiment, M=16, N=9; the total number of grids: 16×9=144 (grids);

[0104] Calculate the size of each grid (grid width × grid height), ΔX is the grid width, and ΔY is the grid height;

[0105] ΔX = W / M = 3840 / 16 = 240;

[0106] ΔY=H / N=2160 / 9=240;

[0107] Wherein, W represents the width of the test touch screen, and H represents the height of the test touch screen;

[0108] Starting from the upper left corner, all grids are numbered in row-first order, from grid 1 to grid 144. The position of each grid is represented as grid (i, j), where i represents the column (1≤i≤16) and j represents the row (1≤j≤9). The coordinate range of the grid is as follows:

[0109] In the X direction, the left boundary is (i-1)×240 pixels; the right boundary is i×240 pixels;

[0110] In the Y direction, the upper boundary is (j-1)×240 pixels; the lower boundary is j×240 pixels;

[0111] The ratio of the overlapping area between each grid and the high-frequency fault area to the total area of ​​the grid is calculated, and the preset threshold is 50%.

[0112] Determine the high frequency fault grid, an example is as follows:

[0113] Grid 1 (top left):

[0114] Located at X: 0-240, Y: 0-240;

[0115] Completely within area A, the overlap is 100% greater than 50%, marked as a high-frequency fault grid.

[0116] The touch point setting process is as follows:

[0117] Set the touch point at the high-frequency fault point:

[0118] High frequency fault point (first touch point): point 1: (400, 200); point 2: (2400, 1300); point 3: (3500, 2000);

[0119] A touch point (second touch point) is set in the high-frequency fault grid, including the center point and four edge points of the high-frequency fault grid, for a total of 5 touch points.

[0120] A touch point (third touch point) is set in the general grid, and only the center point of the general grid is set as the touch point.

[0121] In this embodiment, the number of high-frequency fault grids is as follows: 8 in area A, 6 in area B, and 6 in area C, totaling: 8+6+6=20 (grids);

[0122] If the touch points set according to the high-frequency fault grid and the ordinary grid overlap with the high-frequency fault touch points, the overlapping touch points are merged and only one touch point is retained to avoid repeated setting of touch points and ensure test efficiency and uniqueness of the touch point set. In this embodiment, the high-frequency fault touch points do not overlap with the touch points set according to the high-frequency fault grid and the ordinary grid.

[0123] Number of touch points: 20×5+(144-20)+3=227; Finally, a total of 227 touch points are set, and these touch points constitute a touch point set.

[0124] By analyzing historical maintenance data, high-frequency fault points are identified and selected as touch points, so that the test system can focus on covering and evaluating the vulnerable areas of the touch screen. In addition, the M×N grid division method is used to evenly divide the test touch screen, and the touch points are reasonably distributed in combination with the overlap of high-frequency fault areas to ensure that the test not only focuses on high-fault areas, but also covers the entire touch screen in all directions. This touch point selection method can improve the test coverage rate and ensure the balance and rationality of the test cases, thereby providing strong support for the comprehensive performance evaluation of the touch screen.

[0125] Specifically, make full use of high-frequency fault information in historical data and the resolution and physical size characteristics of the touch screen to ensure that the test points cover areas prone to problems, thereby improving the pertinence and comprehensiveness of the test. By analyzing historical maintenance data, identifying high-frequency fault points, and setting these locations as touch points first, the performance and fault areas of the touch screen can be detected more efficiently. At the same time, combined with the grid division method, ensure that the test covers the entire screen surface to avoid missing other possible fault points. In addition, by setting the edge points of the grid with a high degree of overlap as touch points, the accuracy and coverage of the test are further improved to ensure the full performance evaluation of the touch screen. It can take into account the testing of other areas while ensuring that the key fault points are fully tested, ensuring the reliability and comprehensiveness of the test results.

[0126] The simulated pollution environment parameter generation module is used to select surface pollutants, generate simulated pollution parameters with different gradients for each touch point according to the surface pollutants and preset gradients, and obtain N simulated pollution environment parameters to simulate the touch screen surface under different pollution degrees.

[0127] A real environment simulation test case construction module, used to construct a second test case according to the N simulated pollution environment parameters, in combination with the touch points and the specified operation set;

[0128] A testing module, configured to generate a corresponding UI layout according to the first test case and the second test case and perform corresponding test operations; the UI layout is configured to present interactive elements based on the touch point set;

[0129] In this embodiment, the UI layout is generated based on the test case, which is mainly used to simulate the user's interactive operation environment so that the test can be performed automatically and accurately. Existing test software and frameworks, such as automated testing tools, can map touch point sets and operation instructions to specific UI controls (such as buttons, sliders, etc.) to achieve touch testing. The system will dynamically generate corresponding interface controls based on the touch points and operation types (such as single click, slide, pinch, etc.) in the test case to ensure that the test environment is consistent with the actual application scenario.

[0130] Specifically, according to the first test case, a corresponding UI layout is generated and a corresponding test operation is performed, and the test operation is performed based on an ideal environment (there are no pollutants on the touch screen surface); according to the second test case, a corresponding UI layout is generated and a corresponding test operation is performed (testing according to the test case), and the test operation is performed based on N polluted environments (there are pollutants on the touch screen surface); the second test case contains multiple test cases. Before executing each test case, it is necessary to arrange corresponding surface pollutants on the touch point of the test case according to the corresponding simulated pollution environment parameters to simulate the pollution environment conditions in actual use.

[0131] The video acquisition module is used to use a camera to acquire a full-process test video when performing the test operation to obtain a test process video;

[0132] A data acquisition module, used to communicate with the touch screen driver IC through a communication interface, and to collect output data of the test touch screen in real time when performing the test operation to obtain test output data;

[0133] The data analysis module is used to obtain the test result through the template matching technology according to the test process video; analyze the test output data and the test result to obtain the ideal touch performance test value and N surface contamination touch performance test values.

[0134] Furthermore, the system further comprises: a surface contamination detection module, which is used to detect whether there are contaminants on the surface of the test touch screen, comprising the following steps:

[0135] Step 1: collecting a surface image of the test touch screen to obtain a first surface image;

[0136] Step 2: performing image processing on the first surface image to extract characteristic information of the pollutant, including pollutant type, pollutant shape and pollutant size;

[0137] Step 3: judging whether the pollutant exists according to the characteristic information;

[0138] When the presence of the contaminant is detected, a cleaning suggestion is generated; after a cleaning operation is performed according to the cleaning suggestion, steps one to three are repeated until the contaminant is no longer present.

[0139] In this embodiment, the test module calls the surface contamination detection module when generating the corresponding UI layout according to the first test case and performing the corresponding test operation, which is used to ensure that the surface of the test screen of the first test case is free of contamination and is in an ideal state; similarly, when the test module generates the corresponding UI layout according to the second test case and performs the test operation, it also calls the surface contamination detection module, which is specifically used to: after completing the current test case, clean the surface of the test touch screen and restore the test touch screen surface to its original non-contaminated state. Ensure that after each test case of the second test case is executed, the test touch screen surface is cleaned so that the next test case is performed under the new simulated pollution environment parameters and is not affected by the previous test environment.

[0140] The surface contamination detection module can ensure that the touch screen surface is in an ideal or set state before and after each test, without pollution interference. Through image acquisition and processing, it automatically detects and identifies the type, shape and size of contaminants, cleans the touch screen surface in time, and avoids the residual contaminants from the previous test affecting the subsequent test results. In addition, the module can maintain the clean state of the test touch screen in multiple rounds of testing, ensuring that each test case is executed under ideal or set pollution conditions, thereby improving the accuracy, repeatability and reliability of the test results. This design helps to improve the performance evaluation quality of large-size touch screens in complex operating environments.

[0141] Table 3. Touch screen test parameter set table

[0142] Collection Types Collection Contents Touch pressure collection {1,2,3,4,5} Touch Time Collection {0.5s,1.0s,1.5s,2.0s,2.5s,3.0s} One-finger operation set {Click, slide, click continuously} Multi-finger operation set {Pinch, rotate, multi-finger slide}

[0143] Further, constructing the first test case according to the touch point set and the specified operation set includes:

[0144] Referring to Table 3, a touch pressure set including multiple touch pressure levels is set; a touch time set including different touch time lengths is set; the specified operation set including single-finger operation and multi-finger operation is set; when constructing a test case including the single-finger operation, a single touch point is selected; when constructing a test case including the multi-finger operation, a plurality of touch points are selected, and the multi-finger operation includes pinching of the plurality of touch points, rotation of the plurality of touch points, and multi-finger sliding;

[0145] Referring to Table 3, in the touch pressure set, 1 represents a slight touch; 2 represents a light touch; 3 represents a medium touch; 4 represents a heavier touch; and 5 represents a strong touch.

[0146] like Figure 3 As shown, a set of initial test case sets is generated, the initial test case set includes multiple test cases, each of the test cases includes: a touch point to be tested, an operation to be tested, an operation time, a touch pressure and an expected effect; the touch point to be tested is selected from the touch point set; the operation to be tested is selected from the specified operation set; the operation time is selected from the touch time set; the touch pressure is selected from the touch pressure set;

[0147] Fitness evaluation step: defining a fitness function according to global coverage, operation diversity, touch pressure change, touch time change and global operation total time; calculating a fitness value according to the fitness function;

[0148] Specifically, the global coverage, operation diversity, touch pressure change, touch time change and global operation total time are normalized by minimum-maximum normalization, and the unified fitness function is:

[0149] ””'

[0150] F=ω1·C global +ω2·D operation +ω3·V pressure +ω4·V time -ω5·T total ;

[0151] Where F represents the fitness function; C' global is the normalized global coverage, D' operation is the normalized operation diversity, V' pressure is the normalized touch pressure change, V' time is the normalized touch time change, T' total is the normalized total global operation time; ω1, ω2, ω3, ω4 and ω5 are the first weight coefficient, the second weight coefficient, the third weight coefficient, the fourth weight coefficient and the fifth weight coefficient respectively;

[0152] Selection operation steps: Get the screening test case set through the sorting selection method;

[0153] Cross-operation step: performing a cross-operation on the screening test cases to generate a new test case combination to obtain a cross-case set;

[0154] Mutation operation step: randomly changing the parameters of the cross test case set to obtain a mutation test case set;

[0155] Next generation population generation step: combining the screening test case set, the cross test case set and the mutation test case set to form the next generation population;

[0156] The fitness evaluation step, the selection operation step, the crossover operation step, the mutation operation step and the next generation population generation step are repeated in sequence until the preset stop condition is met (the number of iterations exceeds 150 times), and the test case set with the highest fitness value is output as the first test case, as shown in Table 4.

[0157] Table 4. Example table of the first test case

[0158]

[0159] The large-size touch screen touch performance intelligent test system can automatically generate and optimize test cases through the setting and combination of multiple test parameters. The system ensures the comprehensiveness of test coverage through the comprehensive design of variables such as touch points, pressure and operation time, and can effectively avoid the omission of touch points that may occur in traditional tests. Secondly, the system introduces a mechanism for automated test case generation and optimization. Through fitness function evaluation, crossover and mutation operations, it can continuously evolve the optimal test scheme in multiple generations of populations, reducing the shortcomings of test case design based on experience. In addition, the system also takes into account the changes in different operating conditions, such as the adjustment of pressure and operation time, and can more accurately simulate and evaluate the actual operation of users, thereby better reflecting the user experience of the touch screen. Finally, through rapid adaptive optimization, the system greatly improves the test efficiency, and can generate a comprehensive and highly optimized test case set in a short time, ensuring the efficient performance of the touch screen under various operating conditions. This design greatly reduces the time and complexity of manual testing, making the performance verification of the touch screen more intelligent and precise.

[0160] Furthermore, the surface pollutants include: synthetic oil pollutants, dust, micro-particles, moisture, artificially synthesized sweat and metal dust. These pollutants can accurately control their type, concentration, coverage area and distribution uniformity in a standardized manner, thereby ensuring the repeatability of the test environment. When testing at different times and places, these pollutants can ensure the consistency and reproducibility of experimental conditions. In addition, the design of selecting these pollutants can effectively simulate various pollution conditions of the touch screen in real-life usage scenarios, including industrial dust, grease pollution, sweat and humidity, etc., and comprehensively cover different actual environments. It can accurately evaluate the response and performance of the touch screen in various environments, ensuring that the test results are not only scientific and rigorous, but also highly comparable and practical. The test system designed in this way can reproduce the test multiple times under controllable parameters, thereby improving the reliability of the experiment and the credibility of the test data.

[0161] Further, performing a corresponding test operation according to the second test case includes:

[0162] Step 1: Obtain the current touch point and the simulated pollution environment parameters of the current test case from the second test case, and arrange the surface pollutants at the current touch point according to the simulated pollution environment parameters; in this embodiment, the touch point is a general term for 227 points, and the current touch point is a specific test point corresponding to the current test case (a certain test case). If the specified operation of this test case is a single-finger operation, the current touch point is one point; if the specified operation of this test case is a multi-finger operation, the current touch point is multiple points (for example, two points).

[0163] Step 2: After the surface contaminants are arranged, executing the current test case;

[0164] Step 3: After completing the current test case, clean the surface of the test touch screen to restore the surface of the test touch screen to an original pollution-free state;

[0165] Step 4: Repeat step 1 to step 3 until all the second test cases are tested.

[0166] The first test case and the second test case share the same operating steps, including the selection of the touch point to be tested, the specification of the operation type, the setting of the operation time and touch pressure, etc. The core operating logic of the two is consistent, and the interactive elements, operation sequence and touch behavior used in the test process have not changed. However, the difference is that the second test case introduces N simulated pollution environment parameters, that is, when performing the test operation, the interference of surface contaminants is added. Compared with the "ideal state" of the first test case, the second test case simulates the touch performance test in a real environment by arranging different surface contaminants on each touch point. The expected effect is therefore different. The expected results of the second test case need to take into account the impact of contaminants on touch performance, such as delays in response time and changes in sensitivity.

[0167] Table 5. Second test case example table

[0168]

[0169] Refer to Table 5, which is an example display of a specific pollution environment (simulated pollution environment parameters) in the second test case.

[0170] The present invention ensures that the touch screen performance test is performed under different simulated environmental conditions by introducing simulated pollution environment parameters into the test process. Steps one to four not only ensure comprehensive coverage of various pollutants, but also ensure the independence of each test case by cleaning the screen after each test, thereby avoiding the influence of residual pollutants from the previous test on subsequent test results. This design enables the second test case to accurately evaluate the influence of different pollutants and environmental conditions on touch performance, thereby improving the reliability and reproducibility of the test results. In addition, by cleaning the touch screen surface each time, the consistency of the test environment between different test cases can be ensured, eliminating the interference of the external environment on the results, and making the test process more rigorous and scientific.

[0171] Furthermore, the test output data includes: touch point coordinates, touch pressure, touch time and operation type.

[0172] When the data acquisition module performs the test operation, the touch point coordinates, touch pressure, touch time and operation type are collected in real time, which helps to comprehensively evaluate the touch performance of the touch screen. These data are precise parameters that cannot be captured by video data, and lay a data foundation for subsequent comprehensive analysis combined with video data, so as to understand the working condition and user experience of the touch screen in a more comprehensive and detailed manner. Collecting these data not only provides a quantitative basis for the test results, but also ensures the comprehensiveness and scientificity of the touch screen performance evaluation.

[0173] Further, obtaining the test result by the template matching technology according to the test process video includes:

[0174] The test results include response time, operation accuracy and touch sensitivity; the process of obtaining the test results includes the following steps:

[0175] Obtain the expected effect corresponding to the test process video;

[0176] According to the test process video, the instantaneous image before the test and the instantaneous image after the test are identified and extracted by template matching technology;

[0177] Calculating the pixel-level difference between the instantaneous image before the test and the instantaneous image after the test using an image difference algorithm to generate a difference image;

[0178] According to the difference image, through binarization and connectivity analysis, the screen content change area caused by the touch operation is identified, and the image features of the screen content change area are extracted, and the image features include shape, position, color and movement trajectory;

[0179] According to the extracted image features, detecting UI state changes and obtaining actual operation feedback;

[0180] Obtaining the response time according to the timestamps of the instantaneous image before the test and the instantaneous image after the test;

[0181] By comparing the expected effect with the actual operation feedback, the operation accuracy is obtained;

[0182] The touch sensitivity is obtained by combining the response time and the operation accuracy.

[0183] By combining the test process video with the use of template matching technology, the present invention can accurately identify and extract instantaneous images before and after the test, thereby calculating the screen content change area caused by the touch operation through the image difference algorithm, and then analyzing the response time, operation accuracy and touch sensitivity. By combining video data and image processing technology, the actual feedback of the touch operation can be monitored in real time, providing a more intuitive touch performance evaluation method. This method can not only accurately measure the response time of the touch screen, but also ensure the evaluation of the operation accuracy by comparing the expected effect with the actual feedback, and at the same time combine the two to derive the sensitivity of the touch screen. Compared with the traditional single data analysis method, this method significantly improves the test accuracy and comprehensiveness, and provides a more scientific and reliable basis for the performance evaluation of large-size touch screens.

[0184] Furthermore, analyzing the test output data and the test result to obtain the ideal touch performance test value and N surface contamination touch performance test values ​​includes:

[0185] The test output data includes Q first test output data (ideal environment) and N×Q second test output data (N polluted environments); the test results include Q first test results (ideal environment) and N×Q second test results (N polluted environments);

[0186] Dividing the test output data into first test output data and second test output data according to the first test case and the second test case; dividing the second test output data into N polluted environment test output data according to the N simulated polluted environment parameters;

[0187] Preprocessing the first test output data and the N polluted environment test output data respectively to obtain first standard test output data and N standard polluted environment test output data;

[0188] Calculating a first touch performance test value according to the first standard test output data, the first touch performance test value including response time, operation accuracy and touch sensitivity;

[0189] Obtaining N second touch performance test values ​​according to the N standard pollution environment test output data, wherein the second touch performance test values ​​include response time, operation accuracy and touch sensitivity;

[0190] Dividing the test result into a first test result and a second test result according to the first test case and the second test case; dividing the second test result into N pollution environment test output results according to the N simulated pollution environment parameters;

[0191] averaging the first test result and the first touch performance test value and performing weighted addition to obtain a weighted first test result and a weighted first touch performance test value; adding the weighted first test result and the weighted first touch performance test value to obtain the ideal touch performance test value;

[0192] Table 6. Calculation table of ideal touch performance test values

[0193]

[0194] Refer to Table 6, wherein t, A and S represent response time, operation accuracy and touch sensitivity respectively; the weighted first test result and the weighted first touch performance test value are added to obtain the ideal touch performance test value, specifically: the ideal touch performance test value includes an ideal response time, an ideal operation accuracy and an ideal touch sensitivity;

[0195] Ideal response time T * The calculation formula is:

[0196]

[0197] Ideal operation accuracy A * The calculation formula is:

[0198]

[0199] Ideal touch sensitivity S * The calculation formula is:

[0200]

[0201] Wherein, α is a weight coefficient for weighting the first test result, and β is a weight coefficient for weighting the first touch performance test value.

[0202] The polluted environment test output result and the corresponding second touch performance test value (under the same simulated polluted environment parameters) are averaged and weighted to obtain a weighted polluted environment test output result and a weighted second touch performance test value; the weighted polluted environment test output result and the weighted second touch performance test value are added to obtain the surface contamination touch performance test value. The calculation method of the surface contamination touch performance test value is the same as that of Table 6, and finally N surface contamination touch performance test values ​​are calculated.

[0203] By comprehensively analyzing the test output data and the key frame recognition results, the touch performance of the touch screen in an ideal state and a simulated pollution environment can be more accurately evaluated. By analyzing the touch screen performance in an ideal environment and the touch screen performance in a simulated pollution environment respectively, the performance of the touch screen under different usage conditions can be comprehensively and accurately evaluated. By averaging and weighting the test results and touch performance test values, the results of all test cases can be comprehensively analyzed, the contingency and deviation of a single test result can be reduced, the stability and representativeness of the test results can be ensured, and the overall performance of the touch screen can be more accurately evaluated. At the same time, considering the three key indicators of response time, operation accuracy and touch sensitivity, a multi-dimensional evaluation of the touch screen performance is ensured. The final weighted result can better reflect the comprehensive performance of the touch screen under ideal conditions and pollution conditions.

[0204] Furthermore, the system further comprises:

[0205] A feedback module is used to generate a feedback report according to the ideal touch performance test value and the surface contamination touch performance test value.

[0206] Specifically, the product touch test standard of the test touch screen is obtained, and the product touch test standard includes standard response time, standard operation accuracy and standard touch sensitivity under ideal conditions (no surface contamination) and N surface contamination levels; the ideal touch performance test value and N surface contamination touch performance test values ​​are compared and analyzed with the product touch test standard, the performance of the touch screen under ideal conditions and various contamination levels is evaluated, and a feedback report including response time deviation, operation accuracy deviation and touch sensitivity deviation is generated. The feedback report can provide performance gap analysis, optimization suggestions and conclusions on whether it meets product standards, providing a reference for the optimization and improvement of the touch screen.

[0207] The feedback module can effectively summarize and present the test results, helping users to more intuitively understand the performance of the touch screen under different test conditions.

[0208] Through the synergy of the test case construction module and the simulated pollution environment parameter generation module, the system can flexibly simulate the touch performance under various pollution environments according to different touch points and operation types; through the real environment simulation test case construction module, it is further ensured that the test covers a variety of common interaction methods such as click, slide, continuous click and multi-finger operation, making the test process more comprehensive. The video acquisition module is combined with the data acquisition module, which can not only capture the video data during the test in real time, but also obtain the real-time response data of the touch screen through the communication interface to ensure the accuracy of the test results. Finally, the data analysis module uses template matching technology and combines multi-dimensional data to obtain the ideal touch performance and the touch performance test value under multi-gradient pollution environment, so as to effectively evaluate the reliability and stability of the touch screen in a complex environment. The system significantly improves the accuracy of the touch screen performance test. At the same time, the system can automatically determine the touch points and generate test cases through the test case construction module, avoiding the tedious process of manually setting the test operation, and greatly improving the automation of the test. The automatic generation of the first test case and the second test case ensures the full coverage of the operation and pollution conditions and improves the efficiency of the test.

[0209] Embodiment 2

[0210] Company B is a high-tech enterprise specializing in the research and development, production and sales of touch screens. Its main business covers touch screens of various sizes and types. Its products are widely used in smart phones, tablets, self-service terminals, smart homes, educational electronic whiteboards and industrial control. With the growing market demand for large-size touch screens, Company B is committed to improving product quality and production efficiency to meet customer demand for high-performance touch screens. In the production process of touch screens, factory inspection is a key link to ensure product quality. Traditional inspection methods mainly rely on random sampling and simple automated testing, which cannot fully cover all functional areas of the touch screen and have low efficiency. In order to improve the efficiency and accuracy of factory inspection of large-size touch screens, Company B introduced a large-size touch screen touch performance intelligent test system to achieve comprehensive, rapid and intelligent detection of touch performance of touch screens.

[0211] A large-size touch screen touch performance intelligent testing system, comprising:

[0212] A test case construction module is used to determine a number of touch points according to the test touch screen to obtain a touch point set; construct a first test case according to the touch point set and a specified operation set; the specified operation set includes click, slide, continuous click and multi-finger simultaneous operation;

[0213] Further, determining the plurality of touch points according to the test touch screen includes:

[0214] Collecting historical maintenance data of the touch screen of the same model as the tested touch screen; the historical maintenance data includes the fault type and the coordinates of the fault point;

[0215] Analyzing the historical maintenance data to identify high-frequency fault points in the high-frequency fault area;

[0216] Setting the high-frequency fault point as the touch point;

[0217] By using an M×N grid division method, the test touch screen is divided into a plurality of grids, the size of each grid being determined according to the resolution and physical size of the test touch screen; and the center position of each grid is used as the touch point;

[0218] If the overlap between the grid and the high-frequency fault area exceeds a preset threshold, the center point and four edge points of the grid are simultaneously set as the touch points;

[0219] If the degree of overlap between the grid and the high-frequency fault area does not exceed the preset threshold, the center point of the grid is used as the touch point.

[0220] Further, constructing the first test case according to the touch point set and the specified operation set includes:

[0221] Setting a touch pressure set including multiple touch pressure levels; setting a touch time set including different touch time lengths; setting the specified operation set including single-finger operation and multi-finger operation; selecting a single touch point when constructing a test case including the single-finger operation; selecting multiple touch points when constructing a test case including the multi-finger operation, wherein the multi-finger operation includes pinching of multiple touch points, rotation of multiple touch points, and multi-finger sliding;

[0222] Generate a set of initial test case sets, the initial test case set includes a plurality of test cases, each of the test cases includes: a touch point to be tested, an operation to be tested, an operation time, a touch pressure and an expected effect; the touch point to be tested is selected from the touch point set; the operation to be tested is selected from the specified operation set; the operation time is selected from the touch time set; the touch pressure is selected from the touch pressure set;

[0223] Fitness evaluation step: defining a fitness function according to global coverage, operation diversity, touch pressure change, touch time change and global operation total time; calculating a fitness value according to the fitness function;

[0224] Selection operation steps: Get the screening test case set through the sorting selection method;

[0225] Cross-operation step: performing a cross-operation on the screening test cases to generate a new test case combination to obtain a cross-case set;

[0226] Mutation operation step: randomly changing the parameters of the cross test case set to obtain a mutation test case set;

[0227] Next generation population generation step: combining the screening test case set, the cross test case set and the mutation test case set to form the next generation population;

[0228] The fitness evaluation step, the selection operation step, the crossover operation step, the mutation operation step and the next generation population generation step are repeated in sequence until a preset stop condition is met, and the test case set with the highest fitness value is output as the first test case.

[0229] A simulated pollution environment parameter generation module, used for selecting surface pollutants, generating simulated pollution parameters of different gradients for each touch point according to the surface pollutants and a preset gradient, and obtaining N simulated pollution environment parameters;

[0230] Furthermore, the surface pollutants include: synthetic oil and fat pollutants, dust, tiny particles, moisture, artificially synthesized sweat and metal dust.

[0231] A real environment simulation test case construction module, used to construct a second test case according to the N simulated pollution environment parameters, in combination with the touch points and the specified operation set; the specified operation set includes click, slide, continuous click and multi-finger simultaneous operation;

[0232] Furthermore, the system further comprises: a surface contamination detection module, which is used to detect whether there are contaminants on the surface of the test touch screen, comprising the following steps:

[0233] Step 1: collecting a surface image of the test touch screen to obtain a first surface image;

[0234] Step 2: performing image processing on the first surface image to extract characteristic information of the pollutant, including pollutant type, pollutant shape and pollutant size;

[0235] Step 3: judging whether the pollutant exists according to the characteristic information;

[0236] When the presence of the contaminant is detected, a cleaning suggestion is generated; after a cleaning operation is performed according to the cleaning suggestion, steps one to three are repeated until the contaminant is no longer present.

[0237] In this embodiment, when the test module generates the corresponding UI layout according to the first test case and performs the corresponding test operation, the surface contamination detection module should be called to ensure that the surface of the test touch screen is free of contamination when the first test case is executed. In this way, the ideal touch performance test value obtained subsequently is the ideal test value obtained when the touch test is performed without surface contamination.

[0238] Further, performing a corresponding test operation according to the second test case includes:

[0239] Step 1: acquiring the current touch point of the current test case and the simulated pollution environment parameters from the second test case, and arranging the surface pollutants at the current touch point according to the simulated pollution environment parameters;

[0240] Step 2: After the surface contaminants are arranged, executing the current test case;

[0241] Step three: After completing the current test case, clean the surface of the test touch screen and restore the surface of the test touch screen to its original uncontaminated state; in this embodiment, in order to restore the surface of the test touch screen to its original uncontaminated state, a surface contamination detection module should be called in step three.

[0242] Step 4: Repeat step 1 to step 3 until all the second test cases are tested.

[0243] A testing module, configured to generate a corresponding UI layout according to the first test case and the second test case and perform corresponding test operations; the UI layout is configured to present interactive elements based on the touch point set;

[0244] The video acquisition module is used to use a camera to acquire a full-process test video when performing the test operation to obtain a test process video;

[0245] A data acquisition module, used to communicate with the touch screen driver IC through a communication interface, and to collect output data of the test touch screen in real time when performing the test operation to obtain test output data;

[0246] Furthermore, the test output data includes: touch point coordinates, touch pressure, touch time and operation type.

[0247] The data analysis module is used to obtain the test result through the template matching technology according to the test process video; analyze the test output data and the test result to obtain the ideal touch performance test value and N surface contamination touch performance test values.

[0248] Further, obtaining the test result by the template matching technology according to the test process video includes:

[0249] The test results include response time, operation accuracy and touch sensitivity; the process of obtaining the test results includes the following steps:

[0250] Obtain the expected effect corresponding to the test process video;

[0251] According to the test process video, the instantaneous image before the test and the instantaneous image after the test are identified and extracted by template matching technology;

[0252] Calculating the pixel-level difference between the instantaneous image before the test and the instantaneous image after the test using an image difference algorithm to generate a difference image;

[0253] According to the difference image, through binarization and connectivity analysis, the screen content change area caused by the touch operation is identified, and the image features of the screen content change area are extracted, and the image features include shape, position, color and movement trajectory;

[0254] According to the extracted image features, detecting UI state changes and obtaining actual operation feedback;

[0255] Obtaining the response time according to the timestamps of the instantaneous image before the test and the instantaneous image after the test;

[0256] By comparing the expected effect with the actual operation feedback, the operation accuracy is obtained;

[0257] The touch sensitivity is obtained by combining the response time and the operation accuracy.

[0258] Furthermore, analyzing the test output data and the test result to obtain the ideal touch performance test value and N surface contamination touch performance test values ​​includes:

[0259] Dividing the test output data into first test output data (ideal environment) and second test output data (surface contaminated environment) according to the first test case and the second test case; dividing the second test output data into N contaminated environment test output data according to the N simulated contaminated environment parameters;

[0260] Preprocessing the first test output data and the N polluted environment test output data respectively to obtain first standard test output data and N standard polluted environment test output data;

[0261] Calculating a first touch performance test value according to the first standard test output data, the first touch performance test value including response time, operation accuracy and touch sensitivity;

[0262] Obtaining N second touch performance test values ​​according to the N standard pollution environment test output data, wherein the second touch performance test values ​​include response time, operation accuracy and touch sensitivity;

[0263] Dividing the test result into a first test result and a second test result according to the first test case and the second test case; dividing the second test result into N pollution environment test output results according to the N simulated pollution environment parameters;

[0264] averaging the first test result and the first touch performance test value and performing weighted addition to obtain a weighted first test result and a weighted first touch performance test value; adding the weighted first test result and the weighted first touch performance test value to obtain the ideal touch performance test value;

[0265] The polluted environment test output result and the corresponding second touch performance test value are respectively averaged and weighted to obtain a weighted polluted environment test output result and a weighted second touch performance test value; the weighted polluted environment test output result and the weighted second touch performance test value are added to obtain the surface contamination touch performance test value.

[0266] Furthermore, the system further comprises:

[0267] A feedback module is used to generate a feedback report according to the ideal touch performance test value and the N surface contamination touch performance test values.

[0268] Before the introduction of the large-size touch screen touch performance intelligent test system, Company B mainly used a simple automated test method to conduct factory inspection of touch performance; specifically, it used simple test equipment to perform automated testing on some functions of the touch screen.

[0269] Table 7. Comparison of indicators before and after system application

[0270]

[0271] The comparison data of the indicators before and after the application of the system are shown in Table 7. Through the comparison of the data in Table 7, it can be concluded that after the introduction of the intelligent test system of the present invention, the average detection time is shortened from 42 minutes to 20 minutes, the functional area detection coverage is increased from 70% to 95%, and the detection accuracy is increased from 85% to 97%. This shows that the intelligent test system significantly improves the efficiency, comprehensiveness and accuracy of detection.

[0272] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A large-size touch screen touch performance intelligent testing system, characterized in that: include: A test case building module is used to determine a number of touch points according to the test touch screen to obtain a touch point set; Constructing a first test case according to the touch point set and the specified operation set; The specified operation set includes click, slide, continuous click and multi-finger simultaneous operation; A simulated pollution environment parameter generation module, used for selecting surface pollutants, generating simulated pollution parameters of different gradients for each touch point according to the surface pollutants and a preset gradient, and obtaining N simulated pollution environment parameters; A real environment simulation test case construction module, used to construct a second test case according to the N simulated pollution environment parameters, in combination with the touch points and the specified operation set; A testing module, used to generate a corresponding UI layout according to the first test case and the second test case and perform corresponding testing operations; The UI layout is used to present interactive elements based on the touch point set; The video acquisition module is used to use a camera to acquire a full-process test video when performing the test operation to obtain a test process video; A data acquisition module, used to communicate with the touch screen driver IC through a communication interface, and to collect output data of the test touch screen in real time when performing the test operation to obtain test output data; A data analysis module, used to obtain test results based on the test process video by template matching technology; The test output data and the test results are analyzed to obtain an ideal touch performance test value and N surface contamination touch performance test values.

2. According to claim 1, a large-size touch screen touch performance intelligent testing system is characterized in that: The system further comprises: a surface contamination detection module, which is used to detect whether there are contaminants on the surface of the test touch screen, comprising the following steps: Step 1: collecting a surface image of the test touch screen to obtain a first surface image; Step 2: performing image processing on the first surface image to extract characteristic information of the pollutant, including pollutant type, pollutant shape and pollutant size; Step 3: judging whether the pollutant exists according to the characteristic information; When the presence of the contaminant is detected, a cleaning suggestion is generated; after a cleaning operation is performed according to the cleaning suggestion, steps one to three are repeated until the contaminant is no longer present.

3. The large-size touch screen touch performance intelligent testing system according to claim 1 is characterized in that: Determining the plurality of touch points according to the test touch screen comprises: Collecting historical maintenance data of the touch screen of the same model as the tested touch screen; the historical maintenance data includes the fault type and the coordinates of the fault point; Analyzing the historical maintenance data to identify high-frequency fault points in the high-frequency fault area; The touch points include a first touch point, a second touch point and a third touch point; Setting the high-frequency fault point as the first touch point; Dividing the test touch screen into a plurality of grids by a grid division method, wherein the size of each grid is determined according to the resolution and physical size of the test touch screen; If the overlap between the grid and the high-frequency fault area exceeds a preset threshold, the center point and four edge points of the grid are simultaneously set as the second touch points; If the degree of overlap between the grid and the high-frequency fault area does not exceed the preset threshold, the center point of the grid is used as the third touch point.

4. The large-size touch screen touch performance intelligent testing system according to claim 1 is characterized in that: Constructing the first test case according to the touch point set and the specified operation set includes: Setting a touch pressure set including multiple touch pressure levels; setting a touch time set including different touch time lengths; setting the specified operation set including single-finger operation and multi-finger operation; selecting a single touch point when constructing a test case including the single-finger operation; selecting multiple touch points when constructing a test case including the multi-finger operation, wherein the multi-finger operation includes pinching of multiple touch points, rotation of multiple touch points, and multi-finger sliding; Generate a set of initial test case sets, the initial test case set includes a plurality of test cases, each of the test cases includes: a touch point to be tested, an operation to be tested, an operation time, a touch pressure and an expected effect; the touch point to be tested is selected from the touch point set; the operation to be tested is selected from the specified operation set; the operation time is selected from the touch time set; the touch pressure is selected from the touch pressure set; Fitness evaluation step: defining a fitness function according to global coverage, operation diversity, touch pressure change, touch time change and global operation total time; calculating a fitness value according to the fitness function; Selection operation steps: Get the screening test case set through the sorting selection method; Cross-operation step: performing a cross-operation on the screening test cases to generate a new test case combination to obtain a cross-case set; Mutation operation step: randomly changing the parameters of the cross test case set to obtain a mutation test case set; Next generation population generation step: combining the screening test case set, the cross test case set and the mutation test case set to form the next generation population; The fitness evaluation step, the selection operation step, the crossover operation step, the mutation operation step and the next generation population generation step are repeated in sequence until a preset stop condition is met, and the test case set with the highest fitness value is output as the first test case.

5. The large-size touch screen touch performance intelligent testing system according to claim 1, characterized in that: The surface pollutants include: synthetic grease pollutants, dust, tiny particles, moisture, artificial sweat and metal dust.

6. The large-size touch screen touch performance intelligent testing system according to claim 1 is characterized in that: Executing corresponding test operations according to the second test case includes: Step 1: acquiring the current touch point of the current test case and the simulated pollution environment parameters from the second test case, and arranging the surface pollutants at the current touch point according to the simulated pollution environment parameters; Step 2: After the surface contaminants are arranged, executing the current test case; Step 3: After completing the current test case, clean the surface of the test touch screen to restore the surface of the test touch screen to an original pollution-free state; Step 4: Repeat step 1 to step 3 until all the second test cases are tested.

7. The large-size touch screen touch performance intelligent testing system according to claim 1 is characterized in that: The test output data includes: touch point coordinates, touch pressure, touch time and operation type.

8. The large-size touch screen touch performance intelligent testing system according to claim 1, characterized in that: Obtaining the test result by the template matching technology according to the test process video includes: The test results include response time, operation accuracy and touch sensitivity; the process of obtaining the test results includes the following steps: Obtain the expected effect corresponding to the test process video; According to the test process video, the instantaneous image before the test and the instantaneous image after the test are identified and extracted by template matching technology; Calculating the pixel-level difference between the instantaneous image before the test and the instantaneous image after the test using an image difference algorithm to generate a difference image; According to the difference image, through binarization and connectivity analysis, the screen content change area caused by the touch operation is identified, and the image features of the screen content change area are extracted, and the image features include shape, position, color and movement trajectory; According to the extracted image features, detecting UI state changes and obtaining actual operation feedback; Obtaining the response time according to the timestamps of the instantaneous image before the test and the instantaneous image after the test; By comparing the expected effect with the actual operation feedback, the operation accuracy is obtained; The touch sensitivity is obtained by combining the response time and the operation accuracy.

9. The large-size touch screen touch performance intelligent testing system according to claim 1, characterized in that: Analyzing the test output data and the test result to obtain the ideal touch performance test value and N surface contamination touch performance test values ​​includes: Dividing the test output data into first test output data and second test output data according to the first test case and the second test case; dividing the second test output data into N polluted environment test output data according to the N simulated polluted environment parameters; Preprocessing the first test output data and the N polluted environment test output data respectively to obtain first standard test output data and N standard polluted environment test output data; Calculating a first touch performance test value according to the first standard test output data, the first touch performance test value including response time, operation accuracy and touch sensitivity; Obtaining N second touch performance test values ​​according to the N standard pollution environment test output data, wherein the second touch performance test values ​​include response time, operation accuracy and touch sensitivity; Dividing the test result into a first test result and a second test result according to the first test case and the second test case; dividing the second test result into N pollution environment test output results according to the N simulated pollution environment parameters; averaging the first test result and the first touch performance test value and performing weighted addition to obtain a weighted first test result and a weighted first touch performance test value; adding the weighted first test result and the weighted first touch performance test value to obtain the ideal touch performance test value; The polluted environment test output result and the corresponding second touch performance test value are respectively averaged and weighted to obtain a weighted polluted environment test output result and a weighted second touch performance test value; the weighted polluted environment test output result and the weighted second touch performance test value are added to obtain the surface contamination touch performance test value.

10. The large-size touch screen touch performance intelligent testing system according to claim 1, characterized in that: The system further comprises: A feedback module is used to generate a feedback report according to the ideal touch performance test value and the surface contamination touch performance test value.

Images