Resolution rectification method and device, and multi-terminal cooperative testing method and device

Abstract

The application discloses a resolution rectification method and device, a multi-terminal cooperative testing method and device and a computing device. The resolution rectification method comprises the following steps: in response to finding or operating a certain element and prompting an exception after a sliding operation on a screen of a master terminal, performing a screenshot operation on the master terminal; when there is an intersection matrix between a screenshot image matrix of the master terminal and screenshot image matrices of each measured terminal, calculating resolution offset of each measured terminal according to positions of the intersection matrix in the screenshot image matrix of the master terminal and the screenshot image matrices of the measured terminals and a distance of the sliding operation, and completing resolution rectification. The multi-terminal cooperative testing method comprises the following steps: in response to a sliding operation on a screen of a master terminal, controlling content displayed on screens of each measured terminal to move in a sliding direction, and the moving distance is a sum of a sliding operation distance and a resolution offset; and finding / operating a certain element on the screen of the master terminal, and taking a finding / operation result as a testing result.

Description

Technical Field

[0001] This invention relates to the field of mobile terminal testing technology, and in particular to a resolution correction method and apparatus, a multi-terminal collaborative testing method and apparatus, and a computing device. Background Technology

[0002] After application development is complete, it needs to be run on various brands and models of mobile terminals to test the performance of different terminals. The testing process can be accomplished through a testing platform, such as a mobile multi-device collaborative testing platform that combines OpenSTF (a cloud-based real device solution) and tidevice (a cross-platform multi-device connection solution). This testing platform, based on cloud-based real devices, enables all mobile terminal devices to run according to the same automated instructions, achieving the goal of testing multiple devices at once.

[0003] During testing, when performing vertical scrolling on the terminal screen, if the devices have different resolution ratios, even if a relative ratio operation is used, differences in page lazy loading will occur, causing the next operation to fail to find the element. This requires manual investigation to find the cause, which affects testing efficiency.

[0004] In summary, the existing mobile multi-device collaborative testing technologies mentioned above are prone to issues such as differences in lazy loading of pages, which can prevent the next operation from being completed. Summary of the Invention

[0005] To address this, the present invention provides a resolution correction method and apparatus, a multi-terminal collaborative testing method and apparatus, and a computing device, in an attempt to solve or at least alleviate at least one of the problems mentioned above.

[0006] According to one aspect of the present invention, a resolution correction method is provided, comprising: responding to a swipe operation on the screen of a main control terminal followed by searching for or operating on an element and prompting an error, wherein the swipe operation is an upward swipe operation or a downward swipe operation; performing a screenshot operation on the main control terminal to obtain a screenshot image matrix of the main control terminal and original screenshot image matrices of each tested terminal; performing matrix processing and vector scaling on the original screenshot image matrices of each tested terminal to obtain a screenshot image matrix of each tested terminal; determining whether there is an intersection matrix between the screenshot image matrix of the main control terminal and the screenshot image matrices of each tested terminal; and if there is an intersection matrix, calculating the resolution offset of each tested terminal based on the position of the intersection matrix in the screenshot image matrix of the main control terminal and the screenshot image matrix of each tested terminal and the distance of the swipe operation, so as to complete the resolution correction of the tested terminal.

[0007] Optionally, in the resolution correction method according to the present invention, the step of determining whether there is an intersection matrix between the screenshot image matrix of the master control terminal and the screenshot image matrices of each tested terminal includes: performing binarization processing on the screenshot image matrix of the master control terminal and the screenshot image matrices of each tested terminal; and determining whether there is an intersection matrix between the binarized screenshot image matrix of the master control terminal and the screenshot image matrices of each tested terminal.

[0008] Optionally, in the resolution correction method according to the present invention, the step of determining whether there is an intersection matrix between the screenshot image matrix of the main control terminal and the screenshot image matrices of each tested terminal further includes: extracting the edge of the screenshot image matrix of the main control terminal as the intersection matrix, wherein the edge is the edge in the opposite direction to the sliding operation.

[0009] Optionally, in the resolution correction method according to the present invention, when the sliding operation is an upward sliding operation, the distance between the bottom of the intersection matrix and the bottom of the screenshot image matrix of the terminal under test is taken as the position of the intersection matrix in the screenshot image matrix of the terminal under test; when the sliding operation is a downward sliding operation, the distance between the top of the intersection matrix and the top of the screenshot image matrix of the terminal under test is taken as the position of the intersection matrix in the screenshot image matrix of the terminal under test.

[0010] Optionally, in the resolution correction method according to the present invention, the method further includes: if there is no intersection matrix, continuing to slide the main control terminal screen along the original sliding direction, performing a screenshot operation on the main control terminal, using the main control terminal screenshot image matrix obtained from the previous screenshot operation as the new main control terminal screenshot image matrix, and using the original screenshot image matrix of the tested terminal obtained from the current screenshot operation as the new original screenshot image matrix of the tested terminal; then returning to the step of performing matrix processing and vector scaling on the original screenshot image matrices of each tested terminal to obtain the tested terminal screenshot image matrix of each tested terminal.

[0011] Optionally, in the resolution correction method according to the present invention, the method further includes: controlling the content displayed on the main control terminal screen and the screen of the tested terminal to return to the initial position; responding to a sliding operation on the main control terminal screen, controlling the content displayed on each screen of the tested terminal to move in the direction of the sliding operation, the moving distance being the sum of the distance of the sliding operation and the resolution offset of the corresponding tested terminal; and searching for or operating an element on the main control terminal screen and checking whether there is an abnormal prompt.

[0012] According to another aspect of the present invention, a multi-terminal collaborative testing method based on the above-described resolution correction method is also provided, comprising: responding to a sliding operation on the screen of a master control terminal, controlling the content displayed on the screen of each tested terminal to move in the direction of the sliding operation, wherein the moving distance is the sum of the distance of the sliding operation and the resolution offset of the corresponding tested terminal; and searching / operating on an element on the screen of the master control terminal, and taking the search / operation result of each tested terminal as the test result of the corresponding tested terminal.

[0013] According to another aspect of the present invention, a resolution correction device is also provided, comprising: a screenshot unit adapted to, in response to a swipe operation on the screen of a main control terminal, search for or operate on an element and prompt an error, wherein the swipe operation is an upward swipe operation or a downward swipe operation, perform a screenshot operation on the main control terminal to obtain a screenshot image matrix of the main control terminal and original screenshot image matrices of each tested terminal; a preprocessing unit adapted to perform matrix processing and vector scaling on the original screenshot image matrices of each tested terminal to obtain a screenshot image matrix of each tested terminal; a judgment unit adapted to determine whether there is an intersection matrix between the screenshot image matrix of the main control terminal and the screenshot image matrices of each tested terminal; and a correction unit adapted to, if there is an intersection matrix, calculate the resolution offset of each tested terminal based on the position of the intersection matrix in the screenshot image matrix of the main control terminal and the screenshot image matrix of each tested terminal and the distance of the swipe operation, so as to complete the resolution correction of the tested terminal.

[0014] According to another aspect of the present invention, a multi-terminal collaborative testing device is also provided, comprising: a control unit adapted to control the content displayed on the screen of each tested terminal to move in the direction of the sliding operation in response to a sliding operation on the screen of a master control terminal, wherein the moving distance is the sum of the distance of the sliding operation and the resolution offset of the corresponding tested terminal; and a testing unit adapted to search for / operate on an element on the screen of the master control terminal, and to take the search / operation result of each tested terminal as the test result of the corresponding tested terminal.

[0015] According to another aspect of the present invention, a computing device is also provided, comprising: at least one processor and a memory storing program instructions; when the program instructions are read and executed by the processor, the computing device causes the computing device to perform the resolution correction method or the multi-terminal collaborative testing method described above.

[0016] According to the resolution correction method and apparatus, multi-terminal collaborative testing method and apparatus and computing device of the present invention, at least one of the following beneficial effects can be achieved: through image processing, the resolution offset of the terminal under test relative to the master control terminal is automatically obtained and the resolution offset is calibrated, and the vertical scrolling of the screen of the terminal under test can be adjusted according to the resolution offset, which can effectively avoid errors in multi-machine collaborative testing caused by different pixel ratios of mobile phones. The whole process is completed automatically without manual intervention. Attached Figure Description

[0017] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of this disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout this disclosure, the same reference numerals generally refer to the same parts or elements.

[0018] Figure 1 A schematic diagram of a multi-terminal collaborative testing system 100 according to an embodiment of the present invention is shown;

[0019] Figure 2 A schematic diagram of the structure of a computing device 200 according to an embodiment of the present invention is shown;

[0020] Figure 3 A flowchart of a resolution correction method 300 according to an embodiment of the present invention is shown;

[0021] Figure 4 A flowchart of another resolution correction method according to an embodiment of the present invention is shown;

[0022] Figure 5 A flowchart of a multi-terminal collaborative testing method 500 according to an embodiment of the present invention is shown;

[0023] Figure 6 A schematic diagram of the structure of a resolution correction device 600 according to an embodiment of the present invention is shown;

[0024] Figure 7 A schematic diagram of the structure of a multi-terminal collaborative testing device 700 according to an embodiment of the present invention is shown. Detailed Implementation

[0025] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0026] To address the issue that existing mobile multi-device collaborative testing technologies are prone to page lazy loading discrepancies, which can prevent subsequent operations from being completed, this invention provides an automatic verification method for CD burning. This method can automatically verify the CD inserted by the user, making subsequent burning less prone to problems such as garbled characters or damage.

[0027] Figure 1 A schematic diagram of a multi-terminal collaborative testing system 100 according to an embodiment of the present invention is shown.

[0028] like Figure 1 As shown, the multi-terminal collaborative testing system 100 includes a data processing device 110 and an interface device 120, with the data processing device 110 and interface device 120 being communicatively connected. The interface device 120 provides several interfaces, and the master terminal and the terminal under test can be connected to the interfaces via data cables. The master terminal and the terminal under test typically refer to mobile terminal devices with touchscreens, such as mobile phones or tablets.

[0029] During terminal testing, if screen swiping (up or down) is involved, the user can manually swipe on the main control terminal screen or control the swipe using the data processing device 110. When the user manually swipes on the main control terminal screen, for example, swiping upwards for a distance of 1080 pixels, the data processing device 110 sends a control signal to each tested terminal, causing each tested terminal's screen to produce an effect equivalent to a 1080-pixel upward swipe. If the swipe is controlled via the data processing device 110, the user can set the swipe direction to upwards and the swipe distance to 1080 pixels. The data processing device 110 will then control the main control terminal screen and each tested terminal's screen to produce an effect equivalent to a 1080-pixel upward swipe.

[0030] It should be noted that the present invention does not limit the specific type of data processing device 110. For example, data processing device 110 can be implemented as a desktop computer, laptop computer, processor chip, mobile phone, tablet computer, or other computing device, but is not limited thereto; it can also be an application program residing on the computing device.

[0031] In embodiments of the present invention, the data processing device 110 is adapted to execute a multi-terminal collaborative testing method. The multi-terminal collaborative testing method 500 of the present invention will be described in detail below.

[0032] In one embodiment, the data processing device 110 of the present invention can be implemented as a computing device, enabling the multi-terminal collaborative testing method of the present invention to be executed in the computing device. The computing device can be any device with storage and computing capabilities, such as a server, workstation, or a personal computer such as a desktop computer or laptop computer, or a terminal device such as a mobile phone, tablet computer, smart wearable device, or Internet of Things device, but is not limited thereto.

[0033] Figure 2 A structural diagram of a computing device 200 according to an embodiment of the present invention is shown. Figure 2 As shown, in the basic configuration 202, the computing device 200 typically includes a system memory 206 and one or more processors 204. A memory bus 208 can be used for communication between the processors 204 and the system memory 206.

[0034] Depending on the desired configuration, processor 204 can be any type of processor, including but not limited to: microprocessor (UP), microcontroller (UC), digital information processor (DSP), or any combination thereof. Processor 204 may include one or more levels of cache such as L1 cache 210 and L2 cache 212, processor core 214, and registers 216. Example processor core 214 may include an arithmetic logic unit (ALU), floating-point unit (FPU), digital signal processing core (DSP core), or any combination thereof. Example memory controller 218 may be used with processor 204, or in some implementations, memory controller 218 may be an internal part of processor 204.

[0035] Depending on the desired configuration, system memory 206 can be any type of memory, including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 106 may include operating system 220, one or more applications 222, and program data 224. Application 222 is actually a set of program instructions that instruct processor 204 to perform corresponding operations. In some embodiments, application 222 may be arranged to cause processor 204 to operate using program data 224 on the operating system.

[0036] The computing device 200 may also include a storage interface bus 234. The storage interface bus 234 enables communication from storage devices 232 (e.g., removable storage 236 and non-removable storage 238) to the basic configuration 202 via the bus / interface controller 230. At least a portion of the operating system 220, applications 222, and data 224 may be stored on the removable storage 236 and / or the non-removable storage 238, and loaded into system memory 206 via the storage interface bus 234 when the computing device 200 is powered on or when the application 222 is to be executed, and executed by one or more processors 204.

[0037] The computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to the basic configuration 202 via a bus / interface controller 230. Example output devices 242 include a graphics processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices such as displays or speakers via one or more A / V ports 252. Example peripheral interfaces 244 may include a serial interface controller 254 and a parallel interface controller 256, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices) or other peripherals (e.g., printers, scanners, etc.) via one or more I / O ports 258. Example communication devices 246 may include a network controller 260, which may be arranged to facilitate communication with one or more other computing devices 262 via a network communication link through one or more communication ports 264.

[0038] A network communication link can be an example of a communication medium. A communication medium can typically be embodied in a modulated data signal, such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A “modulated data signal” can be a signal in which one or more of its data sets, or changes thereof, can be encoded as information within the signal. As a non-limiting example, a communication medium can include wired media such as wired networks or leased lines, and various wireless media such as voice, radio frequency (RF), microwave, infrared (IR), or other wireless media. The term “computer-readable medium” as used herein can include both storage media and communication media.

[0039] In the computing device 200 according to the present invention, application 222 includes multiple program instructions for executing a resolution correction method 300 or a multi-terminal collaborative testing method 500. These program instructions can instruct the processor 204 to execute the resolution correction method 300 or the multi-terminal collaborative testing method 500 of the present invention, so that the computing device 200 executes the resolution correction method 300 or the multi-terminal collaborative testing method 500 of the present invention.

[0040] A resolution correction method 300 according to an embodiment of the present invention includes: responding to a swipe operation on the screen of a main control terminal followed by searching for or operating on an element and prompting an error, wherein the swipe operation is an upward swipe or a downward swipe operation; taking a screenshot of the main control terminal to obtain a screenshot image matrix of the main control terminal and original screenshot image matrices of each tested terminal; performing matrix processing and vector scaling on the original screenshot image matrices of each tested terminal to obtain a screenshot image matrix of each tested terminal; determining whether there is an intersection matrix between the screenshot image matrix of the main control terminal and the screenshot image matrices of each tested terminal; and if there is an intersection matrix, calculating the resolution offset of each tested terminal based on the position of the intersection matrix in the screenshot image matrix of the main control terminal and the screenshot image matrix of each tested terminal and the distance of the swipe operation, so as to complete the resolution correction of the tested terminal.

[0041] Figure 3 A flowchart of a resolution correction method 300 according to an embodiment of the present invention is shown. Method 300 can be executed in the data processing device 110 of system 100 (e.g., the aforementioned computing device 200), and is used to correct the resolution offset of each terminal under test relative to the main control terminal when controlling several terminals under test via a main control terminal, so that when up and down swiping operations are performed on the main control terminal, the content displayed on the screen of each terminal under test is the same as the content displayed on the screen of the main control terminal.

[0042] like Figure 3 As shown, method 300 begins with step S310.

[0043] In step S310, in response to searching for or operating on an element after a swipe operation on the main control terminal screen and prompting an error, the swipe operation is either an upward swipe or a downward swipe operation. A screenshot operation is performed on the main control terminal to obtain the main control terminal screenshot image matrix and the original screenshot image matrix of each tested terminal.

[0044] When testing the terminals under test, after swiping a certain distance in a certain direction on the main control terminal screen, an element appears on the page. The next step is to interact with this element. All the terminals under test then swipe the same distance in the same direction. If the element is not found on any of the terminals under test, the testing platform will indicate an anomaly. For example, if a user swipes up 1080 pixels on the main control terminal screen, this operation is equivalent to swiping up 1080 pixels on all the terminals under test. After the swipe, a survey image appears at the bottom of the main control terminal screen. Clicking this image leads to the survey interface. Next, the user clicks an image on the main control terminal screen, which is equivalent to clicking the survey image at the bottom of each terminal under test. However, due to the different screen resolutions of the terminals under test, the survey image may not appear at the bottom of some terminals. Assuming the terminal's hardware and software are functioning correctly, there are two possible reasons for this: first, the survey image is already on the terminal under test, but not at the bottom; second, the survey image is not currently on the terminal under test. If the user continues to swipe up, the survey image will appear. Regardless of the reason why the questionnaire image does not appear at the bottom of the tested terminal screen, the operation of clicking the questionnaire image at the bottom of the tested terminal screen cannot be completed. At this time, the testing platform will issue an error message, indicating that the tested terminal is malfunctioning.

[0045] After the test platform issues an error message, it takes screenshots of both the main control terminal and the terminal under test that issued the error message. Within the test platform, any operation on the main control terminal will trigger the same operation on all terminals under test. Therefore, screenshots can also be taken from the main control terminal. Subsequent image processing simply involves selecting the screenshots from both the main control terminal and the terminal under test that issued the error message. The resulting screenshot is an RGB image, and the image matrix reflects the grayscale values ​​of each pixel. In this embodiment, the image generated by the screenshot operation on the terminal under test is called the original screenshot image of the terminal under test, and the grayscale value matrix of the original screenshot image is called the original screenshot image matrix of the terminal under test.

[0046] A retry count can be set. After the test platform issues a "NotFind exception" message, it will retry the swipe operation, which is equivalent to retrying once. If a "NotFind exception" message is issued again, it will retry once more. If the number of retries reaches the limit and the element to be operated still does not appear on the screen of the tested terminal, a screenshot operation will be triggered.

[0047] Next, in step S320, the original screenshot image matrices of each tested terminal that indicated an anomaly are subjected to matrix processing and vector scaling to obtain the screenshot image matrix of each tested terminal. This step uses existing matrix processing and vector scaling techniques to scale the original screenshot image matrix of the tested terminal proportionally in both length and width directions, so that the scaled original screenshot image matrix of the tested terminal has the same width as the screenshot image matrix of the main control terminal.

[0048] Next, in step S330, it is determined whether there is an intersection matrix between the screenshot image matrix of the main control terminal and the screenshot image matrix of each tested terminal.

[0049] It should be noted that step S330 requires comparing the screenshot image matrix of the main control terminal with the screenshot image matrix of each tested terminal one by one to determine whether there is an intersection matrix.

[0050] This step essentially involves finding identical graphic elements in the screenshots from the main control terminal and the tested terminal. To reduce the complexity of image processing, the main control terminal screenshot matrix and the screenshot matrices from each tested terminal can be binarized first. Then, the binarized main control terminal screenshot matrix is ​​compared with the screenshot matrices from each tested terminal to determine if there is an intersection matrix. If there is an intersection matrix between the main control terminal screenshot matrix and a certain tested terminal screenshot matrix, it indicates that the main control terminal screenshot and that tested terminal screenshot have identical graphic elements (e.g., the questionnaire image in step S310).

[0051] The tested terminal and the main control terminal are the same type of mobile terminal, such as both being mobile phones. During the test, the distance of each swipe will not be too large. Taking a mobile phone with a screen resolution of 2880×1440 pixels as an example, the distance of each upward or downward swipe during the test will typically not exceed 2880 pixels. Taking an upward swipe as an example, suppose we are testing the operation of a certain app on different mobile phones. Since the aspect ratios of the screens of different phones are different, after opening the app on each phone, the content displayed at the top of the screen will be the same, but the content at the bottom may differ. Swiping upward once on the main control phone screen may result in differences in the content displayed on the screens of the tested phones, with these differences being more noticeable at or near the bottom of the screen. If swiping in the opposite direction, the differences in the content displayed on the screens will be more noticeable at or near the top of the screen.

[0052] As long as the aspect ratio of the screen of the terminal under test is not significantly different from that of the screen of the main control terminal, the screenshot images of the main control terminal and the screenshot images of the terminal under test will have the same elements, and the binarized screenshot image matrix of the main control terminal and the screenshot image matrix of the terminal under test will have an intersection matrix.

[0053] It's important to note that the "identical element" here must be a specific graphic, not just the background. For example, if the background of a screenshot from the main control terminal is blue, and the center of the image contains a questionnaire graphic, then the background elements in the binarized main control terminal screenshot matrix will be 1s, and the questionnaire image portion will consist of 1s and 0s. Similarly, if the background of a screenshot from the tested terminal is also blue, and the lower center of the image contains a questionnaire graphic, then the background elements in the binarized tested terminal screenshot matrix will be 1s, and the questionnaire image portion will consist of 1s and 0s. When determining whether the main control terminal screenshot and the tested terminal screenshot contain identical graphics, although the background portions of both include matrices composed of several 1s, they cannot be considered as the intersection matrix.

[0054] For the screenshot image matrix of the tested terminal that has an intersection matrix with the screenshot image matrix of the main control terminal, proceed to step S340.

[0055] In step S340, the resolution offset of each tested terminal is calculated based on the position of the intersection matrix in the screenshot image matrix of the main control terminal and the screenshot image matrix of each tested terminal, and the distance of the sliding operation, so as to complete the resolution correction of the tested terminal.

[0056] It should be noted that the resolution offset of the tested terminal here refers to the resolution offset of the tested terminal relative to the main control terminal.

[0057] After sliding several times on the main control terminal screen, the position of a certain graphic element on the main control terminal screen differs from its position on the tested terminal screen. This difference can be calculated based on the position of the intersection matrix in the screenshot matrix of the main control terminal and the screenshot matrix of the tested terminal, respectively. Combining the total sliding distance, the resolution offset of the tested terminal screen relative to the main control terminal screen is calculated for each slide (or each slide a certain distance).

[0058] Using the questionnaire graphic as an example, swiping upwards once on the main control terminal screen (a distance of 2400 pixels) will cause the questionnaire graphic to appear near the top of both the main control terminal and the top of the tested terminal's screen. This can be interpreted as the tested terminal displaying content ahead of the main control terminal. Next, two reference points need to be selected. The first reference point is a point on the questionnaire graphic, which can be, for example, the center point, the top edge, or the bottom edge of the graphic. The second reference point is a point on a screenshot image matrix, which can be, for example, the center row, the first row, or the last row of the screenshot image matrix. The distances from the first to the second reference points in the main control terminal's screenshot image matrix, and the distances from the first to the second reference points in the tested terminal's screenshot image matrix, are then calculated. For example, using the center point of the questionnaire image as the first reference and the center row of the screenshot image matrix as the second reference, assuming the calculated distance between the first and second references in the screenshot image matrix of the main control terminal is 1000 pixels, and the distance between the first and second references in the screenshot image matrix of the tested terminal is 1300 pixels, that is, after sliding 2400 pixels, the total resolution offset of the tested terminal is 300 pixels. In subsequent tests, for every 2400 pixels the main control terminal slides upwards, the test platform should control the tested terminal to move upwards by (2400-300) pixels.

[0059] The different locations selected for the first and second reference points will result in different resolution offsets of the terminal under test, but these are all within an acceptable range and will not interfere with the testing process.

[0060] According to one implementation of the present invention, when determining whether there is an intersection matrix between the screenshot image matrix of the main control terminal and the screenshot image matrices of each tested terminal, the edge of the screenshot image matrix of the main control terminal is taken as the intersection matrix, and the edge is the edge in the opposite direction to the sliding operation.

[0061] It should be noted that in the above implementation, the "edge" is a matrix, located at the top or bottom of the screenshot image matrix. The edges of the main terminal's screenshot image matrix are captured as the intersection matrix, and the presence of this intersection matrix in the tested terminal's screenshot image matrix is ​​checked. The intersection matrix is ​​preferably the object to be operated on in the next step, such as the questionnaire image in step S310. The main terminal screen is swiped upwards; when the questionnaire image appears at the bottom of the main terminal screen, the screenshot operation begins. Then, the matrix corresponding to the questionnaire image is captured from the main terminal's screenshot image matrix as the intersection matrix, and the presence of this intersection matrix is ​​checked in the tested terminal's screenshot image matrix. For the upward swiping operation, the purpose of resolution correction is to level the bottom of the tested terminal's screen, that is, to make the content displayed at the bottom of the tested terminal's screen the same as the content displayed at the bottom of the main terminal's screen. Therefore, the above implementation can obtain the resolution offset more accurately.

[0062] Furthermore, based on the above implementation, to obtain a more accurate resolution offset, when the sliding operation is an upward sliding operation, the distance between the bottom of the intersection matrix and the bottom of the screenshot image matrix of the tested terminal is used as the position of the intersection matrix in the screenshot image matrix of the tested terminal; conversely, when the sliding operation is a downward sliding operation, the distance between the top of the intersection matrix and the top of the screenshot image matrix of the tested terminal is used as the position of the intersection matrix in the screenshot image matrix of the tested terminal. For example... Figure 4 As shown, taking an upward swipe as an example, during the test, the main control terminal screen is swiped upwards to search for or manipulate a certain element. If the element is not found on the screen of a terminal under test, it is retried once (i.e., the screen returns to its initial position and the swipe is repeated). If the maximum number of retries is reached and the element is still not found, a screenshot operation is initiated. Then, the image recognition process begins: the bottom edge of the screenshot image from the main control terminal is captured, and the bottom edge image of the screenshot image from the terminal under test is checked to see if it exists. If it does not exist, an exception is triggered, and the upward swipe operation can continue, and the aforementioned process is repeated. If it exists, the resolution offset is calculated, and the distance that the upward swipe operation should cover is further calculated. Then, the upward swipe operation is performed again, and it is checked again whether the bottom edge image of the screenshot image from the main control terminal can be found on the screenshot image from the terminal under test.

[0063] In the above implementation, since only a portion of the screenshot image matrix of the main control terminal is captured for comparison with the screenshot image matrix of the tested terminal, the comparison result may or may not have an intersection matrix. Taking mobile phones as an example, the aspect ratios of screens of various popular brands and models on the market are generally between 1.5 and 2. If the aspect ratio of the tested terminal's screen is smaller than that of the main control terminal's screen, the image at the bottom of the main control terminal's screenshot image matrix will appear on the tested terminal's screen when scrolling upwards. If the aspect ratio of the tested terminal's screen is larger than that of the main control terminal's screen, the image at the bottom of the main control terminal's screenshot image matrix may not yet appear on the tested terminal's screen when scrolling upwards. In this case, it is necessary to continue scrolling upwards on the main control terminal's screen, take a new screenshot, and find the intersection matrix again.

[0064] For the screenshot image matrix of the terminal under test that does not intersect with the screenshot image matrix of the master terminal, it is necessary to continue sliding the screen of the master terminal along the original sliding direction to perform a screenshot operation on the master terminal. The screenshot image matrix of the master terminal obtained in the previous screenshot operation is used as the new screenshot image matrix of the master terminal, and the original screenshot image matrix of the terminal under test obtained in this screenshot operation is used as the new original screenshot image matrix of the terminal under test; then return to step S320.

[0065] Re-initiating the screenshot operation results in obtaining a new screenshot image matrix from the master terminal and several original screenshot image matrices from the tested terminals. It's important to note that in subsequent steps, the master terminal screenshot image matrix obtained from the previous screenshot operation should be used as the master terminal screenshot image matrix, and the original screenshot image matrix from the tested terminals obtained in this screenshot operation should be used as the original screenshot image matrix for the tested terminals. Then, check if there is an intersection matrix. When calculating the resolution offset, the distance of the sliding operation should be the sum of the distances of the two sliding operations.

[0066] After the above steps, the resolution offset of each tested terminal relative to the main control terminal can be obtained. Next, the obtained resolution offsets of each tested terminal relative to the main control terminal can be verified. The verification method is as follows: control the content displayed on the main control terminal screen and the tested terminal screen to return to their initial positions. The initial position of the displayed content refers to the position of this content on the main control terminal screen before the sliding operation in step S310. For example, if "Autohome" was displayed at the top of the main control terminal screen before the sliding operation in step S310, then "Autohome" also needs to be returned to the top of the main control terminal screen during verification. Next, in response to the sliding operation on the main control terminal screen, control the content displayed on each tested terminal screen to move in the direction of the sliding operation. The moving distance is the sum of the sliding operation distance and the corresponding resolution offset of the tested terminal. Also, search for or manipulate a certain element on the main control terminal screen and check for any abnormal prompts. If no abnormal prompts are found, it indicates that the resolution correction effect meets the requirements.

[0067] Method 300 can be applied to multi-terminal collaborative testing platforms or other multi-terminal collaborative control systems.

[0068] Embodiments of the present invention also provide a multi-terminal collaborative testing method 500 based on the above-described resolution correction method.

[0069] The multi-terminal collaborative testing method 500 according to an embodiment of the present invention includes: responding to a sliding operation on the screen of a master control terminal, controlling the content displayed on the screen of each tested terminal to move in the direction of the sliding operation, the moving distance being the sum of the distance of the sliding operation and the resolution offset of the corresponding tested terminal; and searching for / operating on a certain element on the screen of the master control terminal, and taking the search / operation result of each tested terminal as the test result of the corresponding tested terminal.

[0070] Method 500 is implemented using a testing platform. Users can configure operations on the terminals through the testing platform's interactive interface, or they can manually operate the master terminal. After configuring the terminal operations through the testing platform's interactive interface, the testing platform will perform the same operations on the master terminal and all terminals under test.

[0071] Figure 5 A flowchart of a terminal collaboration testing method 500 according to an embodiment of the present invention is shown. Figure 5 As shown, method 500 begins with step S510.

[0072] In step S510, in response to the sliding operation on the main control terminal screen, the content displayed on the screen of each tested terminal is controlled to move in the direction of the sliding operation, and the moving distance is the sum of the distance of the sliding operation and the resolution offset of the corresponding tested terminal.

[0073] During testing, after a swipe operation occurs on the main control terminal screen, it is necessary to locate a specific element on the screen and perform the next operation on that element. If the element does not appear on the screen of a tested terminal (or the test platform issues a "NotFind exception"), then method 300 is executed. After method 300 completes, the resolution offset of each tested terminal is obtained. The next time a swipe operation is performed, the test platform will take the resolution offset into account when controlling the swipe distance on the tested terminal screen, ensuring that the element to be operated on is displayed on the screen of each tested terminal.

[0074] Next, in step S520, the elements that need to be operated displayed on the screen of each tested terminal in step S510 are operated accordingly, such as clicking or sliding an element, to determine whether the application can run normally on each tested terminal.

[0075] Embodiments of the present invention also provide a resolution correction device 600, which is capable of performing the steps of the resolution correction method 300 described above. Below, in conjunction with... Figure 6 The resolution correction device 600 described above is used to describe this device.

[0076] like Figure 6 As shown, the resolution correction device 600 includes a screenshot unit 610, a preprocessing unit 620, a judgment unit 630, and a correction unit 640.

[0077] The screenshot unit 610 is suitable for responding to a swipe operation on the main control terminal screen to search for or operate on a certain element and prompting an error. The swipe operation is either an upward swipe or a downward swipe operation. The unit performs a screenshot operation on the main control terminal to obtain the screenshot image matrix of the main control terminal and the original screenshot image matrix of each tested terminal.

[0078] The preprocessing unit 620 is adapted to perform matrix processing and vector scaling on the original screenshot image matrix of each tested terminal to obtain the screenshot image matrix of each tested terminal.

[0079] The judgment unit 630 is suitable for judging whether there is an intersection matrix between the screenshot image matrix of the main control terminal and the screenshot image matrix of each tested terminal.

[0080] The correction unit 640 is adapted to calculate the resolution offset of each tested terminal based on the position of the intersection matrix in the screenshot image matrix of the main control terminal and the screenshot image matrix of each tested terminal, and the distance of the sliding operation, if an intersection matrix exists, so as to complete the resolution correction of the tested terminal.

[0081] According to one implementation of the present invention, the judgment unit 630 includes a binarization processing unit and an intersection matrix lookup unit.

[0082] The binarization processing unit is suitable for binarizing the screenshot image matrix of the main control terminal and the screenshot image matrix of each tested terminal.

[0083] The intersection matrix lookup unit is suitable for determining whether there is an intersection matrix between the binarized screenshot image matrix of the master terminal and the screenshot image matrices of each tested terminal.

[0084] According to one implementation of the present invention, the determination unit 630 further includes an image cropping unit.

[0085] The image cropping unit is suitable for cropping the edges of the screenshot image matrix of the main control terminal as the intersection matrix, and the edges are the edges in the opposite direction to the sliding operation.

[0086] According to one implementation of the present invention, when the sliding operation is an upward sliding operation, the distance between the bottom of the intersection matrix and the bottom of the screenshot image matrix of the tested terminal is taken as the position of the intersection matrix in the screenshot image matrix of the tested terminal; when the sliding operation is a downward sliding operation, the distance between the top of the intersection matrix and the top of the screenshot image matrix of the tested terminal is taken as the position of the intersection matrix in the screenshot image matrix of the tested terminal.

[0087] According to one implementation of the present invention, the device 600 further includes a re-sliding unit, which is adapted to continue sliding the main control terminal screen along the original sliding direction if there is no intersection matrix, perform a screenshot operation on the main control terminal, use the main control terminal screenshot image matrix obtained in the previous screenshot operation as the new main control terminal screenshot image matrix, and use the original screenshot image matrix of the tested terminal obtained in this screenshot operation as the new original screenshot image matrix of the tested terminal.

[0088] According to one implementation of the present invention, the device 600 further includes a first verification unit, a second verification unit, and a third verification unit.

[0089] The first verification unit is adapted to control the content displayed on the main control terminal screen and the tested terminal screen to return to the initial position.

[0090] The second verification unit responds to the sliding operation on the main control terminal screen by controlling the content displayed on the screen of each tested terminal to move in the direction of the sliding operation. The moving distance is the sum of the sliding operation distance and the resolution offset of the corresponding tested terminal.

[0091] The third verification unit is suitable for searching or operating a certain element on the main control terminal screen and checking whether there are any abnormal prompts.

[0092] Embodiments of the present invention also provide a multi-terminal collaborative testing device 700, which is capable of performing the steps of the multi-terminal collaborative testing method 500 described above. Below, in conjunction with... Figure 7 The above-mentioned multi-terminal collaborative testing device 700 is described below.

[0093] like Figure 7 As shown, the multi-terminal collaborative testing device 700 includes a control unit 710 and a testing unit 720.

[0094] The control unit 710 is adapted to respond to a sliding operation on the main control terminal screen and control the content displayed on the screen of each tested terminal to move in the direction of the sliding operation. The moving distance is the sum of the sliding operation distance and the resolution offset of the corresponding tested terminal.

[0095] The test unit 720 is suitable for searching / operating a certain element on the main control terminal screen, and taking the search / operation results of each terminal under test as the test results of the corresponding terminal under test.

[0096] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0097] When the program code is executed on a programmable computer, the computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the resolution correction method or multi-terminal collaborative testing method of the present invention according to instructions in the program code stored in the memory.

[0098] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0099] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of the invention.

[0100] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0101] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more aspects of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Those skilled in the art will understand that modules, units, or components of the devices in the examples disclosed herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or further divided into multiple sub-modules.

[0102] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0103] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention and form different embodiments. Additionally, some of the embodiments described herein are described as methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the method or method elements forms means for implementing the method or method elements. Furthermore, the elements described herein in the device embodiments are examples of means for implementing the functions performed by elements for the purposes of carrying out the invention.

[0104] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0105] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for the purpose of explaining or limiting the subject matter of the invention.

Claims

1. A resolution rectification method, comprising: in response to a search or operation of an element after a swipe operation on a master terminal screen, and an exception prompt, the swipe operation being an upward swipe operation or a downward swipe operation, performing a screen capture operation on the master terminal to obtain a master terminal screen capture image matrix and a raw screen capture image matrix of each measured terminal; performing matrix processing vector scaling on the raw screen capture image matrix of each measured terminal to obtain a measured terminal screen capture image matrix of each measured terminal; respectively judging whether there is an intersection matrix between the master terminal screen capture image matrix and the measured terminal screen capture image matrix; if there is an intersection matrix, calculating a resolution offset of each measured terminal according to a position of the intersection matrix in the master terminal screen capture image matrix and the measured terminal screen capture image matrix and a distance of the swipe operation to complete resolution rectification of the measured terminal. The step of judging whether there is an intersection matrix between the master terminal screen capture image matrix and the measured terminal screen capture image matrix comprises: performing binary processing on the master terminal screen capture image matrix and the measured terminal screen capture image matrix; judging whether there is an intersection matrix between the binary processed master terminal screen capture image matrix and the measured terminal screen capture image matrix; cutting an edge of the master terminal screen capture image matrix as the intersection matrix, the edge being an edge in a direction opposite to the swipe operation. The step of calculating the resolution offset of each measured terminal comprises: determining a first position of the intersection matrix in the master terminal screen capture image matrix and a second position of the intersection matrix in the measured terminal screen capture image matrix; calculating the resolution offset of the measured terminal relative to the master terminal according to a difference between the first position and the second position and the distance of the swipe operation. 2.The resolution rectification method of claim 1, wherein when the swipe operation is the upward swipe operation, a distance between a bottom of the intersection matrix and a bottom of the measured terminal screen capture image matrix is taken as the position of the intersection matrix in the measured terminal screen capture image matrix; when the swipe operation is the downward swipe operation, a distance between a top of the intersection matrix and a top of the measured terminal screen capture image matrix is taken as the position of the intersection matrix in the measured terminal screen capture image matrix. 3.The resolution rectification method of claim 2, further comprising: if there is no intersection matrix, continuing to swipe the master terminal screen in the original swipe direction, performing a screen capture operation on the master terminal, taking the master terminal screen capture image matrix obtained in the last screen capture operation as a new master terminal screen capture image matrix, and taking the raw screen capture image matrix of the measured terminal obtained in the current screen capture operation as a new raw screen capture image matrix of the measured terminal; then returning to the step of performing matrix processing vector scaling on the raw screen capture image matrix of each measured terminal to obtain a measured terminal screen capture image matrix of each measured terminal.

4. The resolution skew correction method of any one of claims 1 to 3, wherein, The method further comprises: controlling the content displayed on the master terminal screen and the measured terminal screen to return to an initial position. in response to a sliding operation on the master terminal screen, controlling the content displayed on each of the terminal screens to move in the direction of the sliding operation, the moving distance being the sum of the distance of the sliding operation and the resolution offset of the corresponding terminal; finding or operating an element on the master terminal screen and checking whether there is an abnormal prompt.

5. A multi-terminal cooperative testing method based on the resolution offset correction method according to any one of claims 1 to 4, comprising: in response to a sliding operation on the master terminal screen, controlling the content displayed on each of the terminal screens to move in the direction of the sliding operation, the moving distance being the sum of the distance of the sliding operation and the resolution offset of the corresponding terminal; finding / operating an element on the master terminal screen and taking the finding / operating result of each of the terminal as the testing result of the corresponding terminal.

6. A resolution offset correction device, comprising: a screenshot unit adapted to, in response to finding or operating an element after a sliding operation on the master terminal screen, the sliding operation being an upward sliding operation or a downward sliding operation, perform a screenshot operation on the master terminal to obtain a master terminal screenshot image matrix and a terminal original screenshot image matrix of each of the terminals; a preprocessing unit adapted to perform matrix processing vector scaling on the terminal original screenshot image matrix of each of the terminals to obtain a terminal screenshot image matrix of each of the terminals; a judging unit adapted to judge whether there is an intersection matrix between the master terminal screenshot image matrix and the terminal screenshot image matrix of each of the terminals; a correction unit adapted to, if there is an intersection matrix, calculate the resolution offset of each of the terminals according to the position of the intersection matrix in the master terminal screenshot image matrix and the terminal screenshot image matrix and the distance of the sliding operation to complete resolution offset correction of the terminal. wherein the judging whether there is an intersection matrix between the master terminal screenshot image matrix and the terminal screenshot image matrix of each of the terminals comprises: performing binary processing on the master terminal screenshot image matrix and the terminal screenshot image matrix of each of the terminals; judging whether there is an intersection matrix between the binary processed master terminal screenshot image matrix and the terminal screenshot image matrix of each of the terminals; cutting the edge of the master terminal screenshot image matrix as the intersection matrix, the edge being the edge in the direction opposite to the sliding operation; wherein the calculating the resolution offset of each of the terminals comprises: determining the first position of the intersection matrix in the master terminal screenshot image matrix and the second position of the intersection matrix in the terminal screenshot image matrix of each of the terminals; calculating the resolution offset of the terminal relative to the master terminal according to the difference between the first position and the second position and the distance of the sliding operation.

7. A multi-terminal cooperative testing device, comprising: a control unit adapted to control the content displayed on each of the terminal screens to move in the direction of a sliding operation on the master terminal screen according to the resolution offset of the corresponding terminal screen, the moving distance being the sum of the distance of the sliding operation and the resolution offset of the corresponding terminal screen, wherein the resolution offset is obtained according to the resolution offsetting method of any one of claims 1-4; a test unit adapted to find / operate a certain element on the master terminal screen and take the finding / operating result of each of the terminal screens as the test result of the corresponding terminal screen.

8. A computing device comprising: at least one processor and a memory having stored thereon program instructions; when the program instructions are read and executed by the processor, cause the computing device to perform the resolution offsetting method of any one of claims 1-4 or the multi-terminal cooperative testing method of claim 5.

Images