Screen testing method, device, equipment, medium, program product and system

Abstract

The invention relates to the technical field of screen testing, and discloses a screen testing method, device and equipment, a medium and a program product and system.The method comprises the steps that the current screen content of a current tested screen is obtained, and the current screen content is displayed on a screen projection interface; based on the test script, determining a preset operation object in the current display content of the screen projection interface, executing a corresponding preset operation action, and sending a generated operation command to the current tested screen; and acquiring the next display content of the screen projection interface, comparing the next display content with the preset screen content, and testing the function of the current tested screen. According to the invention, independent screen projection display can be carried out on the tested screen by combining the screen projection tool, different tested screens are distinguished by the screen projection window, independent operation of a user on each screen is simulated on a software layer and testing is carried out, the problems of inconvenience in element positioning and complex operation in a vehicle-mounted co-driver screen testing process are solved, and the testing efficiency is improved. And automatic testing of the vehicle-mounted multi-screen equipment is completed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of screen testing, and in particular to a screen testing method, device, equipment, medium, program product and system. BACKGROUND

[0002] With the development of automobile technology, the car machine (in-vehicle entertainment information system) on the car is gradually updated from the traditional QNX, WINCE, Linux system to the open source Android system with more abundant ecology. Moreover, more and more cars are equipped with corresponding co-driver screens to provide more abundant entertainment functions. Correspondingly, the demand for multi-screen testing is gradually increasing, and the testing of the co-driver screen is a necessary means to ensure the implementation of the function of the co-driver screen.

[0003] The prior art usually obtains UI element (User Interface Elements) information in the screen according to the device number of the screen, and tests the screen based on the UI element. However, for the car machine, the main driver screen and the co-driver screen share the same device number, the co-driver screen can be regarded as a sub-window or an extended view of the main driver screen, and the UI element of the co-driver screen is not assigned an independent device-level identifier. If the testing is performed according to the device number, the UI element in the co-driver screen cannot be positioned, and therefore the testing of the co-driver screen cannot be implemented at the software layer. SUMMARY

[0004] Therefore, the present application provides a screen testing method, device, equipment, medium, program product and system to solve the problem that the co-driver screen cannot be tested at the software layer.

[0005] In a first aspect, the present application provides a screen testing method, which comprises: selecting at least one of a plurality of to-be-tested screens as a current to-be-tested screen, obtaining current screen content of the current to-be-tested screen, and displaying the current screen content on a projection interface; determining a preset operation object in the current display content of the projection interface based on a test script, executing a corresponding preset operation action based on the preset operation object, and sending a generated operation command to the current to-be-tested screen; obtaining next display content of the projection interface, comparing the next display content with preset screen content, and testing the function of the current to-be-tested screen according to a comparison result.

[0006] The screen test method provided by the application can realize independent projection display of the test screen by combining the projection tool, distinguish different test screens through the projection window, and then execute preset operation actions corresponding to preset operations on the projection interface based on the test script, and send the generated operation commands to the current test screen, obtain the next projection interface of the projection interface, compare the next projection interface with the preset screen content, test the function of the current test screen, realize the simulation of the single operation of the user on each screen at the software layer and the test, solve the problems of inconvenient positioning of elements and complex operation in the test process of the vehicle-mounted copilot screen, complete the automatic test of the vehicle-mounted multi-screen device, improve the test efficiency, save the labor cost, and then guarantee the normal use of the copilot screen function, improve the experience and satisfaction of the user.

[0007] In an optional implementation, the at least one of the multiple test screens is selected as the current test screen, the current screen content of the current test screen is obtained based on the projection tool, and the current screen content is displayed on the projection interface, including: obtaining screen numbers corresponding to the multiple test screens with the same device number, and determining the screen number of the current test screen; generating a projection instruction based on the screen number of the current test screen, and sending the projection instruction to the projection server end of the projection tool deployed on the side where the current test screen is located by the projection client of the projection tool deployed in advance; and when the projection server end captures the current screen content of the current test screen, the current screen content received by the projection client is displayed on the projection interface.

[0008] The application realizes the real-time mapping of the display content of the main driver screen or the copilot screen to the independent window of the test terminal through the projection tool, each projection window corresponds to a unique logical screen ID, and thus the independent control of the multiple screens is realized.

[0009] In an optional implementation, based on the test script, the preset operation object is determined in the current display content of the projection interface, and the corresponding preset operation action is executed based on the preset operation object, including: obtaining a test script designed in advance; determining the preset operation object and the preset operation action according to the test script, and obtaining the area image corresponding to the preset operation object and the target position information corresponding to the preset operation action; performing image recognition in the current display content of the projection interface based on the area image, and determining the position area of the preset operation object; obtaining the initial position information from the position area, and executing the preset operation action based on the initial position information and the target position information.

[0010] The application can realize dynamic operation actions based on the coordinates of the position area of the preset operation object and the coordinates of the preset operation action, simulate diversified operations such as clicking and sliding of the user on the screen, and achieve flexible dynamic testing.

[0011] In an optional embodiment, the design process of the test script comprises: obtaining first initial display content of the tested screen in a current test step when a preset operation object in any tested screen is controlled according to a preset test step; obtaining a region image of the preset operation object after taking a screenshot of the first initial display content according to test requirements; obtaining target position information corresponding to the preset operation action after the preset operation action is performed on the preset operation object, and obtaining second initial display content of the tested screen in a next test step; obtaining an expected response image corresponding to the preset operation action after taking a screenshot of the second initial display content according to test requirements, and taking the expected response image as preset screen content; and generating the test script according to the preset operation object, the region image, the preset operation action, the target position information, and the preset screen content.

[0012] The application can accurately position the operation area and establish the association between the interface states before and after the operation, realize efficient generation of the automatic test script, take the display interface after the operation as the expected response result, provide a comparison reference for the automatic test, ensure the automatic operation of the entire test script, improve the test efficiency, and reduce the labor cost.

[0013] In an optional embodiment, the next display content of the screen projection interface is obtained, and the next display content is compared with the preset screen content, and the function of the current tested screen is tested according to the comparison result, which comprises: taking a screenshot of the next display content to obtain an actual response image; comparing the actual response image with the expected response image to calculate the similarity between the actual response image and the expected response image; and if the similarity is higher than a preset threshold, it is determined that the page display function of the current tested screen is normal.

[0014] The application can compare the actual response image with the expected response image, determine whether the expected effect is obtained after the preset operation action is performed on the preset operation object, determine whether the function of the current tested screen is normal or whether the current tested screen is affected by other screens, and thus perform targeted debugging on the screen according to the test result to ensure the experience of the user in the use process.

[0015] In a second aspect, the present application provides a screen testing device, the device comprising: a screen projection display module configured to select at least one of a plurality of screens under test as a current screen under test, acquire current screen content of the current screen under test based on a screen projection tool, and display the current screen content on a screen projection interface; an operation control module configured to determine a preset operation object in the current display content of the screen projection interface based on a test script, perform a preset operation action corresponding to the preset operation object, and send an operation command generated to the current screen under test; and a function testing module configured to acquire next display content of the screen projection interface, compare the next display content with preset screen content, and test a function of the current screen under test according to a comparison result.

[0016] In a third aspect, the present application provides a computer device, comprising: a memory and a processor, which are communicatively connected to each other, and the memory stores computer instructions; the processor executes the computer instructions to perform the screen testing method of the first aspect or any of the corresponding embodiments thereof.

[0017] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the screen testing method of the first aspect or any of the corresponding embodiments thereof.

[0018] In a fifth aspect, the present application provides a computer program product, which comprises computer instructions, and the computer instructions are used to make a computer execute the screen testing method of the first aspect or any of the corresponding embodiments thereof.

[0019] In a sixth aspect, the present application provides a screen testing system, comprising: a vehicle terminal, which is deployed with a plurality of screens under test; and a control terminal, which is connected to the vehicle terminal through an Android debugging bridge, is deployed with a screen projection tool and a corresponding screen projection interface, and is used to execute the screen testing method of the first aspect or any of the corresponding embodiments thereof.

[0020] Because the screen testing system performs function testing on the screens of the vehicle terminal based on the screen testing method, the screen testing system has the same effects as the screen testing method, and thus will not be described herein.

[0021] The present application has the following beneficial effects:

[0022] (1) The present application can independently project and display the screens under test by combining the screen projection tool, distinguish different screens under test through the screen projection window, simulate individual operations of the user on each screen in the software layer, and test the individual operations, thereby solving the problems of inconvenient positioning of elements and complex operations in the process of testing the vehicle co-driver screen.

[0023] (2) The application can improve test efficiency, save labor cost, ensure normal use of the main driver screen and the co-driver screen function, and improve user experience and satisfaction by automatically testing the vehicle-mounted multi-screen device.

[0024] (3) The application can realize static or dynamic operation actions based on the screen projection interface, simulate diversified operations such as clicking and sliding of the user on the screen, achieve flexible dynamic testing, and have more comprehensive test range. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0026] Figure 1 is a flowchart of a screen test method according to an embodiment of the present application;

[0027] Figure 2 is a structural framework of a screen test system according to an embodiment of the present application;

[0028] Figure 3 is a flowchart of another screen test method according to an embodiment of the present application;

[0029] Figure 4 is a flowchart of still another screen test method according to an embodiment of the present application;

[0030] Figure 5 is a cycle test flowchart of still another screen test method according to an embodiment of the present application;

[0031] Figure 6 is a structural block diagram of a screen test device according to an embodiment of the present application;

[0032] Figure 7 is a hardware structure schematic diagram of a computer device of an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0034] Embodiments of the present application are suitable for the scene of testing the functions of multiple screens attached to the car machine. In a traditional single-screen car machine, when other users want to adjust the entertainment system, navigation or air conditioner through the screen, the main driver needs to frequently operate the screen or be distracted by the guide, which has a safety hazard. The copilot screen can allow other users to independently control the relevant functions, and the main driver does not need to interrupt the driving operation, and the line of sight does not need to leave the road, thereby reducing the risk of accidents. In addition, the main driver screen can focus on displaying key driving information such as navigation, vehicle speed, vehicle state, etc., and the copilot screen is responsible for entertainment, vehicle settings and other non-driving related content, avoiding operation errors caused by mixed multi-task interfaces. In this scenario, it is necessary to ensure that the driver and passengers can use different screens at the same time, for example, the copilot adjusts the entertainment system without affecting the main driver's navigation interface, thereby improving the interaction efficiency.

[0035] However, although multiple screens are deployed on the car, the main driver screen and the copilot screen are usually driven by the same mainboard or car entertainment host, and share the display controller, so the main driver screen and the copilot share the same device number, and the UI elements of the copilot screen are not assigned an independent device-level identifier, so for the car machine, multiple screens are generally regarded as the same display device node. When the related technology tests the screen, the screen is connected based on the device number, and then the UI elements of the screen are obtained, wherein each UI element corresponds to a control (button, text box, etc.) with a unique coordinate, ID or attribute identifier. The test script specifies the target screen through the device number, and then locates the specific control based on the element ID, text content or coordinate parameters to realize clicking, inputting and other operations. If there is another copilot screen at this time, the UI elements of the copilot screen are not assigned an independent device-level identifier, and the physical coordinate space of the main driver screen and the copilot screen is independent, but the test system may combine them into a global coordinate system. For example: the coordinate range of the main driver screen is (0, 0)-(1920, 1080), and the copilot screen may be mapped to (1920, 0)-(3840, 1080), but the test tool cannot distinguish the two areas through the device number, resulting in the main driver screen control being touched by mistake when the element is located, so independent testing of the copilot screen cannot be realized. If the copilot screen is tested automatically using the element positioning method in the related technology, the connection line needs to be physically modified, which is relatively cumbersome to operate. Therefore, the embodiment of the present application provides a screen testing method, which combines the screen projection tool to directly perform various operations on the copilot screen, and realizes the testing effect at the software layer.

[0036] According to the embodiments of the present application, a screen testing method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0037] In this embodiment, a screen testing method is provided, which can be used for a computer device serving as a control terminal. Figure 1 : is a flow chart of a screen testing method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0038] Step S101: select at least one of the multiple tested screens as the current tested screen, obtain the current screen content of the current tested screen based on the screen projection tool, and display the current screen content on the screen projection interface.

[0039] Specifically, in the embodiment of the present invention, a screen test system is pre-built, such as Figure 2 As shown, it includes a car terminal and a control terminal. The control terminal is a computer device. The computer device is pre-installed with screen projection software as a screen projection tool. The computer device and the car terminal establish an ADB (Android Debug Bridge) connection through USB (Universal Serial Bus) or a local area network. ADB connection refers to establishing a communication connection between a computer and an Android device (such as a mobile phone or tablet) through the Android debug bridge tool, so as to debug, manage and control the device on the computer. In the embodiment of the present invention, the car terminal is the corresponding Android device. In addition, in order to realize automated testing, the computer device is preferably installed with a third-party Python library, such as PyAutoGUI, which can be used to automatically control the mouse and keyboard, simulate mouse movement, keyboard input, screenshots and other operations, and realize automated control of applications, which is suitable for developing automated testing, etc.

[0040] In some optional implementations, Python's os module is called, and the screen projection tool is opened using the cast command to select the screen to be projected. The various screens under test can be distinguished based on the parameter representing the screen number in the cast command. 0 defaults to the primary screen, and 1 defaults to the secondary screen. The current screen under test is then determined, and the current screen content of the current screen under test is obtained, and the current screen content is displayed on the projection interface of the computer device. During the test process, you can choose to test the primary screen or the secondary screen independently to verify the independent functions of each screen, or you can test the primary and secondary screens together to verify the interactive functions of the primary and secondary screens.

[0041] Step S102: Based on the test script, determine the preset operation object in the current display content of the projection interface, execute the corresponding preset operation action based on the preset operation object, and send the generated operation command to the current screen under test.

[0042] Specifically, in the embodiment of the present application, the test script is designed in advance according to the test requirements, including the function test of the control corresponding to each UI element in the main screen and the function test of the control corresponding to each UI element in the secondary screen. In the process of designing the test script, the control corresponding to the UI element is the preset operation object, including buttons, charts, sliders, etc., and the control action is the preset operation action, including clicking, sliding, etc. In the test process, after the screen of the current tested screen is projected, the preset operation object and the corresponding preset operation action are determined based on the test script, the preset operation object is identified in the current display content of the projection interface, and the preset operation action is performed. After the execution is completed, the computer device can generate the corresponding operation command according to the execution action, and send the operation command to the current tested screen. The current tested screen completes the corresponding function according to the operation command. For example, the QQ music function in the secondary screen is tested, the icon of QQ music is taken as the preset operation object, and the click control is taken as the preset operation action. When the icon of QQ music is identified in the current display content, the click action is performed, at this time, the open command is generated, and the open command is sent to the secondary screen. The secondary screen opens QQ music, which is only an example, not limited thereto.

[0043] In step S103, the next display content of the projection interface is obtained, and the next display content is compared with the preset screen content. The function of the current tested screen is tested according to the comparison result.

[0044] Specifically, in the embodiment of the present application, if the current tested screen runs normally, after the operation command is executed, it will be switched to the corresponding interface. At this time, since the computer device is connected with the car machine end and the current tested screen is projected, the projection interface of the computer device will be switched to the corresponding display content according to the interface of the current tested screen. If the current tested screen runs abnormally, it will not be switched to the corresponding interface, and it may stay in the current screen content or be switched to other error interface. Correspondingly, the computer device synchronously displays the screen content of the current tested screen, so there is also an error.

[0045] In some optional embodiments, the embodiment of the present application stores the response interface after the preset operation object is executed when designing the test script of the interface, and takes it as the preset response content. Therefore, the next display content of the projection interface after the preset operation action is executed is compared with the expected display content, to determine whether the projection interface is consistent with the expected result. If it is consistent, it proves that the screen content of the current tested screen is normal, and the function of the preset operation object in the current tested screen is normal.

[0046] The screen test method provided by the application can realize the simulation of the individual operation of each screen by a user at a software layer and the test of the individual operation, solve the problems of inconvenient positioning of elements and complex operation in the test process of the vehicle-mounted co-driver screen, complete the automatic test of the vehicle-mounted multi-screen device, improve the test efficiency, save the labor cost, and then guarantee the normal use of the co-driver screen function, improve the experience and satisfaction of users.

[0047] In the embodiment, a screen test method is provided, which can be used for the computer device described above, Figure 3 The flowchart of the screen test method according to the embodiment of the application is shown in FIG. 1, which includes the following steps: Figure 3

[0048] In step S301, at least one of the multiple test screens is selected as a current test screen, the current screen content of the current test screen is obtained based on a screen projection tool, and the current screen content is displayed on a screen projection interface.

[0049] Specifically, step S301 includes the following steps.

[0050] In step S3011, the screen numbers corresponding to the multiple test screens with the same device number are obtained, and the screen number of the current test screen is determined.

[0051] Specifically, in the embodiment of the application, the same graphics card of the vehicle machine end can be connected to different types of screens, including a main driver screen and a co-driver screen, and the co-driver screen also includes a rear screen. Each screen is assigned a unique identifier as a physical number at the hardware layer, which is assigned by a display controller, and each screen is assigned an abstract ID as a logical number in an operating system. Therefore, different screens have different screen numbers. After the computer device is connected to the vehicle machine end, the test script of the computer device can directly determine the screen number of the current test screen according to the screen numbers corresponding to the multiple test screens, for example, the number 1 corresponds to the co-driver screen.

[0052] In step S3012, a screen projection instruction is generated based on the screen number of the current test screen, and the screen projection instruction is sent to the screen projection server end of the screen projection tool deployed on the side where the current test screen is located by the screen projection client of the screen projection tool deployed in advance.

[0053] ​Specifically, in the embodiment of the present application, the test script generates the screen mirroring instruction according to the screen label, and the screen mirroring client (client end) of the screen mirroring tool deployed on the computer device sends the screen mirroring instruction to the screen mirroring server end (server end) of the screen mirroring tool deployed on the vehicle terminal. In the screen mirroring technology, the client end and the server end are two different roles, the client end is deployed on the computer device, is the initiating end of the user operation, is responsible for sending the screen mirroring instruction, receiving and decoding the picture data, and finally displaying on the local interface. The server end is deployed on the vehicle terminal, is the target end of the screen mirroring, is responsible for acquiring the screen content, encoding the data and transmitting to the client end. The client end of the computer device sends a specific installation instruction to the vehicle terminal, the instruction contains the installation package of the server end program. The vehicle terminal installs the server end according to the installation instruction, after the installation is completed, the server end in the vehicle terminal performs the following operations: acquiring the screen pixel data through the API provided by the vehicle terminal, thereby capturing the screen content; encoding the screen content to reduce the data transmission amount; sending the encoded video stream to the client end of the computer device through the TCP protocol / UDP protocol.

[0054] Step S3013, after the screen mirroring server end captures the current screen content of the current screen to be tested, the current screen content received by the screen mirroring client is displayed on the screen mirroring interface.

[0055] Specifically, in the embodiment of the present application, as Figure 2As shown, the server continuously acquires the screen content and transmits it to the client, the client decodes the video frame and displays it on the screen interface, thereby realizing screen projection based on the screen projection tool. The screen projection tool is a kind of technical tool realized by software or hardware, and its core function is to transmit and display the screen content of the source device (such as mobile phone, computer, car machine) to the target device (such as TV, projector, another terminal) in real time, which can break the physical limit between devices and realize cross-device picture sharing. The technical implementation of the screen projection tool can be divided into three links of signal acquisition, encoding transmission and decoding display. The signal acquisition and encoding transmission are completed by the server, and the decoding display layer is completed by the client. For signal acquisition, the screen frame data is directly acquired through the operating system API (Application Programming Interface, such as MediaProjection of Android), supporting multi-screen recognition (such as distinguishing between main and auxiliary screens). For encoding transmission, H.264, H.265 (HEVC) or AV1 encoding standard is adopted to compress the original frame data to 1 / 10-1 / 50 of the volume, balance the picture quality and bandwidth demand, and automatically switch the code rate according to the network condition (such as from 1080p30fps to 720p 15fps), avoid lag through dynamic parameter adjustment; select the appropriate transmission protocol, such as TCP protocol or UDP protocol, etc. For decoding display, hardware decoding (such as GPU acceleration) is preferred, which reduces the CPU load, supports 4K / 8K ultra-high definition decoding, software decoding (such as FFmpeg) is used as backup, which has stronger compatibility but higher performance consumption, at the same time, the computer device automatically matches the source picture (such as black edge filling when 16:9 source picture is adapted to 4:3 screen), avoids picture tearing through vertical synchronization (VSync), and improves the smoothness of interaction.

[0056] In step S302, based on the test script, a preset operation object is determined in the current display content of the screen projection interface, a corresponding preset operation action is executed based on the preset operation object, and the generated operation command is sent to the current screen under test. For details, please refer to Figure 1 The step S102 of the embodiment shown will not be repeated here.

[0057] In step S303, the next display content of the screen projection interface is acquired, and the next display content is compared with the preset screen content, and the function of the current screen under test is tested according to the comparison result. For details, please refer to Figure 1 The step S103 of the embodiment shown will not be repeated here.

[0058] The screen test method provided by the application can realize the simulation of the individual operation of each screen by a user in a software layer and the test of the individual operation, solve the problems of inconvenient positioning of elements and complex operation in the test process of the vehicle-mounted co-driver screen, complete the automatic test of the vehicle-mounted multi-screen device, improve the test efficiency, save the labor cost, and then guarantee the normal use of the co-driver screen function, improve the experience and satisfaction of the user.

[0059] In the embodiment, a screen test method is provided, which can be used for the computer device described above, Figure 4 The flowchart of the screen test method according to the embodiment of the application is shown in FIG. 3, which includes the following steps: Figure 4

[0060] In step S401, at least one of the multiple test screens is selected as a current test screen, the current screen content of the current test screen is acquired based on a screen projection tool, and the current screen content is displayed on a screen projection interface. For details, refer to step S301 of the embodiment shown in FIG. 3, which will not be described here again. Figure 3

[0061] In step S402, a preset operation object is determined in the current display content of the screen projection interface based on a test script, a corresponding preset operation action is performed based on the preset operation object, and the generated operation command is sent to the current test screen.

[0062] Specifically, step S402 includes the following steps.

[0063] In step S4021, a test script designed in advance is acquired.

[0064] Specifically, in the embodiment of the application, after the test vehicle machine end and the computer device are connected through ADB by USB or a local area network, a test script needs to be designed in advance for the later automatic test, and the process includes the following steps.

[0065] In step a1, when a preset operation object in any test screen is controlled according to a preset test step, the first initial display content of the test screen in the current test step is acquired.

[0066] In step a2, after the first initial display content is captured according to the test requirement, the area image of the preset operation object is obtained.​​

[0067] Step a3, after performing the preset operation action on the preset operation object, obtaining the target position information corresponding to the preset operation action, and obtaining the second initial display content of the screen under test in the next test step.

[0068] Step a4, obtaining the expected response image corresponding to the preset operation action after taking a screenshot of the second initial display content according to the test requirements, and taking the expected response image as the preset screen content.

[0069] Step a5, generating a test script according to the preset operation object, the region image, the preset operation action, the target position information, and the preset screen content.

[0070] Specifically, in the embodiment of the present application, the screen (one or more) of the screen under test on the car machine side is projected in real time to the projection interface of the computer device as the first initial display content of the screen under test, and a screenshot tool is used to take a screenshot of the preset operation object in the projection interface, such as a button, a chart, or a slider, etc., to obtain a region image of the preset operation object. A corresponding preset operation action is performed on the preset operation object in the projection interface, such as a mouse operation action of clicking, sliding, dragging, etc., to realize operation control of the preset operation object, and the target position information corresponding to the preset operation action is obtained in the process. If the preset operation action is clicking, the position information of the mouse when clicking is obtained; if the preset operation action is sliding, the position information of the mouse at least at the beginning and end of sliding is obtained; if the preset operation action is dragging, the position information of the mouse at least at the beginning and end of dragging is obtained, which is only an example and is not limited thereto. Correspondingly, an operation command is generated after performing the preset operation action, and is sent to the corresponding screen under test on the car machine side. The screen under test responds according to the operation command and switches to the display interface (second initial display content) corresponding to the preset operation object, while the projection interface of the computer device is synchronously switched to the second initial display content. A screenshot tool is used to take a screenshot of the projection interface, and the screenshot is taken as the expected response image corresponding to the preset operation action, i.e., the preset screen content. By independently or jointly operating and controlling each screen, the region image of each preset operation object, the preset operation action corresponding to the preset operation object, the target position information of the preset operation action, and the expected response image are obtained, and the region image and the expected response image are named and saved. A Python test script (combined with PyAutoGUI command) is written to make a test script according to the above test case.

[0071] In some optional embodiments, the UI elements in different screens of the test stage change frequently, and the frequency of change of the UI element information is higher, and the embodiment of the application controls the operation by using the picture recognition mode, and the change of the information such as the name, ID, xpath or coordinates of the UI element does not affect the normal test case execution, and the maintenance cost is greatly reduced in the later period. Even if the page changes greatly, there is no need to modify the code, and only the corresponding screenshots need to be updated.

[0072] In step S4022, the preset operation object and the preset operation action are determined according to the test script, and the region image corresponding to the preset operation object and the target position information corresponding to the preset operation action are obtained.

[0073] Specifically, in the embodiment of the application, the test steps are pre-set in the test script, the preset operation object, the preset operation action and the target position information corresponding to the preset operation action corresponding to the current test step are obtained, and the mouse operation is simulated on the basis of this.

[0074] In step S4023, the region image is used for image recognition in the current display content of the projection interface, and the position region of the preset operation object is determined.

[0075] Specifically, in the embodiment of the application, as shown in Figure 5 The screen capture and image recognition capability of PyAutoGUI is used to find a specific image or color in the current display content of the projection interface according to the pre-stored region image of the preset operation object, and the position region of the preset operation object is determined. For example, the preset operation object corresponding to the test step is the QQ music icon, and the icon is pre-screenshot, so when the preset operation object of the current test step is determined to be QQ music, the QQ music icon is found and recognized in the current display content, and the position region of the QQ music icon is determined.

[0076] In step S4024, the initial position information is obtained by obtaining the position region, and the preset operation action is executed based on the initial position information and the target position information.

[0077] Specifically, in the embodiment of the present application, the center position of the position area is taken as the initial position information, and the preset operation object is executed according to the target position information of the preset operation object. For example, if the target position information is the same as the initial position information (there can be a certain error), it represents that the preset operation action is a static operation action such as clicking, and the clicking operation is executed at the initial position; if the target position information is different from the initial position information, it represents that the preset operation action is a dynamic operation action such as sliding or dragging, and the preset operation object is dragged from the initial position to the target position. At this time, the client end of the screen projection tool in the computer device captures the input event (mainly refers to the mouse operation) and sends it to the server end, and the server end injects the above event into the current measured screen of the car machine end to control the car machine end to execute the corresponding operation.

[0078] In step S403, the next display content of the screen projection interface is acquired, and the next display content is compared with the preset screen content, and the function of the current measured screen is tested according to the comparison result.

[0079] Specifically, the above step S403 includes:

[0080] In step S4031, the next display content is captured to obtain an actual response image.

[0081] Specifically, in the embodiment of the present application, after the preset operation action is executed on the preset operation object, the corresponding operation is executed by the car machine end, the display content of the current measured screen is switched, and the display content of the screen projection interface of the computer device is switched along with the display content of the current measured screen of the car machine end. If the function of the preset operation object in the current measured screen is normal, the corresponding display content is switched, for example, after the QQ music icon of the co-pilot screen is clicked, the co-pilot screen is normally switched to the home page of QQ music, and the computer device screen projection interface also displays the home page of QQ music. If the function of the preset operation object is abnormal, the current measured screen may be stuck and not switch the display content, or display abnormal display content, and the corresponding computer device screen projection interface also does not switch the display content or switches to abnormal content.

[0082] Therefore, after the preset operation action is executed, the next display content of the computer device screen projection interface is captured, and this next display content does not necessarily represent content switching, but represents the interface content after the preset operation action is executed. In order to prevent screen projection delay, a specific time interval can be set, for example, the screen projection interface is captured 3s after the preset operation action is executed to obtain the actual response image of the current measured screen.

[0083] In step S4032, the actual response image is compared with the expected response image, and the similarity of the actual response image and the expected response image is calculated.

[0084] Step S4033, if the similarity is higher than the preset threshold, it is determined that the page display function of the current screen under test is normal.

[0085] Specifically, in the embodiment of the present application, when designing the test script, the correct expected response content is ensured in advance according to the normal function of the preset operation object. Therefore, the actual response content in the test process is compared with the expected response content, the similarity of the two is calculated, and if the similarity exceeds the preset threshold, it is determined that the page display function of the current screen under test is normal. The encapsulated PyAutoGUI command is used in the embodiment of the present application to perform picture recognition assertion, or the assertion mode provided by other test frameworks of Python is combined to flexibly perform assertion. The picture recognition assertion is a concept in the field of computer vision and automated testing, which refers to a process of verifying whether the recognition result meets the expected condition after analyzing and recognizing the content of a picture through an algorithm or a tool.

[0086] In some optional embodiments, automatic cyclic testing is performed based on the test script, and after the various functions of each preset operation object in the current screen under test are tested, the next screen under test is switched to, and the test script is completed. The developer can debug the screen under test according to the test result to ensure that each screen function is normal and to avoid abnormality in user use.

[0087] The screen testing method provided by the present application can distinguish different screens under test by displaying the screens under test independently through the projection tool, and then perform the preset operation action corresponding to the preset operation on the projection interface based on the test script, and send the generated operation command to the current screen under test, obtain the next projection interface of the projection interface, compare the next projection interface with the preset screen content, test the function of the current screen under test, simulate the single operation of each screen by the user in the software layer and perform the test, solve the problems of inconvenient positioning of elements and complex operation in the test process of the vehicle-mounted co-driver screen, complete the automatic test of the vehicle-mounted multi-screen device, improve the test efficiency, save the labor cost, and then ensure the normal use of the co-driver screen function, improve the experience and satisfaction of the user.

[0088] In the embodiment, a screen testing device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and details are not repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and contemplated.

[0089] The embodiment provides a screen testing device, such as Figure 6 as shown, comprising:

[0090] The screen display module 601 is configured to select at least one of the multiple screens under test as a current screen under test, acquire current screen content of the current screen under test based on a screen projection tool, and display the current screen content on a screen projection interface.

[0091] The operation control module 602 is configured to determine a preset operation object in the current display content of the screen projection interface based on a test script, perform a preset operation action based on the preset operation object, and send a generated operation command to the current screen under test.

[0092] The function test module 603 is configured to acquire next display content of the screen projection interface, compare the next display content with preset screen content, and test a function of the current screen under test according to a comparison result.

[0093] In some optional embodiments, the screen display module 601 comprises:

[0094] A screen selection unit is configured to acquire screen numbers corresponding to the multiple screens under test with the same device number, and determine a screen number of the current screen under test.

[0095] An information acquisition unit is configured to generate a screen projection instruction based on the screen number of the current screen under test, and send the screen projection instruction to a screen projection server end of the screen projection tool deployed on a side where the current screen under test is located by a screen projection client of the screen projection tool.

[0096] A content display unit is configured to display current screen content received by the screen projection client on the screen projection interface after the screen projection server end captures the current screen content of the current screen under test.

[0097] In some optional embodiments, the operation control module 602 comprises:

[0098] A script acquisition unit is configured to acquire a test script designed in advance.

[0099] An object determination unit is configured to determine a preset operation object and a preset operation action according to the test script, and acquire a region image corresponding to the preset operation object and target position information corresponding to the preset operation action.

[0100] A region determination unit is configured to perform image recognition in the current display content of the screen projection interface based on the region image, and determine a position region of the preset operation object.

[0101] An action execution unit is configured to acquire initial position information from the position region, and perform the preset operation action based on the initial position information and the target position information.

[0102] In some optional embodiments, the apparatus further comprises a script design module, the script design module comprising:

[0103] a first content acquisition unit, configured to acquire first initial display content of the screen under test in a current test step when performing operation control on a preset operation object in any screen under test according to a preset test step.

[0104] a first image capturing unit, configured to acquire a region image of the preset operation object after capturing the first initial display content according to test requirements;

[0105] a second content acquisition unit, configured to acquire target position information corresponding to the preset operation action and second initial display content of the screen under test in a next test step after performing the preset operation action on the preset operation object.

[0106] a second image capturing unit, configured to acquire an expected response image corresponding to the preset operation action after capturing the second initial display content according to test requirements, and take the expected response image as preset screen content.

[0107] a script generation unit, configured to generate a test script according to the preset operation object, the region image, the preset operation action, the target position information and the preset screen content.

[0108] In some optional embodiments, the function test module 603 comprises:

[0109] a third image capturing unit, configured to capture the next display content to obtain an actual response image.

[0110] an object comparison unit, configured to compare the actual response image with the expected response image, and calculate similarity between the actual response image and the expected response image.

[0111] a result determination unit, configured to determine that the page display function of the current screen under test is normal if the similarity is higher than a preset threshold.

[0112] Further function descriptions of the above-mentioned modules and units are the same as those of the corresponding embodiments, and will not be described here.

[0113] The screen test apparatus in the present embodiment is presented in the form of function units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories executing one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0114] The embodiment of the present application provides a screen test system, as shown in Figure 2 The vehicle terminal is deployed with a plurality of to-be-tested screens. Figure 1 、 Figure 3 or Figure 4 The screen test method shown in

[0115] Specifically, in the embodiment of the present application, as shown in Figure 2 The computer device is used as the control terminal, and the computer device is deployed with the client end of the screen projection tool, and the vehicle terminal is deployed with the serve end of the screen projection tool. In addition, the screen test system of the embodiment of the present application can also be used for screen test of other installation devices, which is not limited herein. The PyAutoGUI deployed on the computer device is a third-party library of Python, and a user can encapsulate the method according to test requirements, and the method can be used in combination with other test frameworks, and the execution mode and the assertion mode can be flexibly selected.

[0116] Because the screen test system performs function test on the screen of the vehicle terminal based on the screen test method, the screen test system has the same effect as the screen test method, which is not described herein again.

[0117] The embodiment of the present application further provides a computer device with the screen test device shown in Figure 6 .

[0118] Please refer to Figure 7 , Figure 7 is a structural schematic diagram of a computer device provided by an optional embodiment of the present application, as shown in Figure 7 The computer device comprises one or more processors 10, a memory 20, and an interface for connecting components, including a high-speed interface and a low-speed interface. The components are communicatively connected with each other by different buses, and can be installed on a common mainboard or in other manners according to requirements. The processor can process instructions executed in the computer device, including instructions stored in the memory or the memory to display GUI on an external input / output device (such as a display device coupled to the interface) to display graphical information. In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Likewise, multiple computer devices can be connected, and each device provides part of necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 In the figure, the processor 10 is taken as an example.

[0119] The processor 10 can be a central processing unit, a network processing unit, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic device, a generic array logic, or any combination thereof.

[0120] The memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods illustrated in the above embodiments.

[0121] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system and application programs required by at least one function. The data storage area can store data created according to the use of the computer device, and the like. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0122] The memory 20 can include a volatile memory, such as a random access memory, and can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid state disk. The memory 20 can further include a combination of the above-mentioned types of memories.

[0123] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 can be connected by a bus or other means, Figure 7 For example, the connection by the bus is taken as an example.

[0124] The input device 30 can receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, and the like. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), a tactile feedback device (e.g., a vibration motor), and the like. The display device includes, but is not limited to, a liquid crystal display, a light emitting diode, a display, and a plasma display. In some alternative embodiments, the display device can be a touch screen.

[0125] The embodiments of the present application further provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium through network, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0126] Part of the present application can be applied as a computer program product, for example, computer program instructions, when executed by a computer, the operation of the computer can invoke or provide the method and / or technical solutions according to the present application. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source file, executable file, installation package file, etc. Correspondingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0127] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A screen testing method, characterized in that: The method comprises: Selecting at least one of the multiple tested screens as the current tested screen, obtaining the current screen content of the current tested screen based on the screen projection tool, and displaying the current screen content on the screen projection interface; Based on the test script, determine the preset operation object in the current display content of the projection interface, perform the corresponding preset operation action based on the preset operation object, and send the generated operation command to the current tested screen; Obtain the next display content of the projection interface, compare the next display content with the preset screen content, and test the function of the current tested screen according to the comparison result.

2. The method according to claim 1, characterized in that The selecting at least one of the multiple tested screens as the current tested screen, obtaining the current screen content of the current tested screen based on the screen projection tool, and displaying the current screen content on the screen projection interface includes: Obtaining screen numbers corresponding to multiple tested screens having the same device number, and determining the screen number of the current tested screen; Generate a screen projection instruction based on the screen number of the current screen under test, and send the screen projection instruction to the screen projection server of the screen projection tool deployed on the side where the current screen under test is located by the pre-deployed screen projection client; After the screen projection server captures the current screen content of the current screen under test, the current screen content received by the screen projection client is displayed on the screen projection interface.

3. The method according to claim 1, characterized in that The method of determining a preset operation object in the current display content of the projection interface based on the test script, and performing a corresponding preset operation action based on the preset operation object, includes: Get pre-designed test scripts; Determine a preset operation object and a preset operation action according to the test script, and obtain an area image corresponding to the preset operation object and target position information corresponding to the preset operation action; Performing image recognition on the current display content of the projection interface based on the regional image to determine the location area of ​​the preset operation object; The location area is acquired to obtain initial location information, and the preset operation action is performed based on the initial location information and the target location information.

4. The method according to claim 3, characterized in that The test script design process includes: When operating and controlling a preset operation object in any screen under test according to a preset test step, obtaining a first initial display content of the screen under test in the current test step; Acquire a regional image of the preset operation object obtained by taking a screenshot of the first initial display content according to test requirements; After performing a preset operation on the preset operation object, obtaining target position information corresponding to the preset operation, and obtaining a second initial display content of the screen under test in the next test step; Obtaining an expected response image corresponding to the preset operation action after taking a screenshot of the second initial display content according to the test requirements, and using the expected response image as the preset screen content; The test script is generated according to the preset operation object, the area image, the preset operation action, the target position information and the preset screen content.

5. The method according to claim 4, characterized in that The obtaining of the next display content of the projection interface, comparing the next display content with the preset screen content, and testing the function of the current screen under test according to the comparison result includes: Taking a screenshot of the next display content to obtain an actual response image; Comparing the actual response image with the expected response image, and calculating the similarity between the actual response image and the expected response image; If the similarity is higher than a preset threshold, it is determined that the page display function of the current tested screen is normal.

6. A screen testing device, characterized in that: The device comprises: A screen projection display module, configured to select at least one of the multiple tested screens as the current tested screen, obtain the current screen content of the current tested screen based on the screen projection tool, and display the current screen content on the screen projection interface; An operation control module is used to determine a preset operation object in the current display content of the projection interface based on a test script, perform a corresponding preset operation action based on the preset operation object, and send the generated operation command to the current tested screen; The functional testing module is used to obtain the next display content of the projection interface, compare the next display content with the preset screen content, and test the function of the current tested screen according to the comparison result.

7. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the screen testing method according to any one of claims 1 to 5 by executing the computer instructions.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the screen testing method according to any one of claims 1 to 5.

9. A computer program product, characterized in that The invention comprises computer instructions, wherein the computer instructions are used to make a computer execute the screen testing method according to any one of claims 1 to 5.

10. A screen testing system, characterized in that: include: A vehicle computer side, wherein the vehicle computer side is equipped with multiple screens to be tested; The control terminal is connected to the vehicle terminal through an Android debugging bridge, and is deployed with a screen projection tool and a corresponding screen projection interface, and is used to execute the screen testing method described in any one of claims 1 to 5.

Images