Automatic testing method, device and system for page switching of in-vehicle infotainment system

Abstract

The invention discloses an automatic testing method, device and system for locomotive page switching, and the method comprises the steps: pre-training a convolutional neural network model based on image data sets of UI elements of various locomotive systems; identifying a locomotive system, and selecting a corresponding test mode and a test path; testing the locomotive system according to the test mode and the test path, and obtaining image information of a locomotive screen user input interface, states before and after page switching and page switching response time; identifying and analyzing the image content through a convolutional neural network model, judging whether a page is correctly switched, generating test data, and feeding back and optimizing the test data; according to the invention, automatic monitoring and testing of the UI interface of the vehicle-mounted terminal are realized, and accuracy of page switching logic is ensured; manual intervention is remarkably reduced through automatic testing, and the testing speed and the coverage range are improved; the image recognition technology ensures that each page switching can be accurately recognized, and misjudgment is reduced; the labor cost of quality guarantee is greatly reduced; the system is flexible in design, is easy to adapt to different in-vehicle infotainment systems, and supports the test requirements of new functions in the future.

Description

Technical Field

[0001] The present invention relates to the field of vehicle testing technology, and in particular to a method, device and system for automatic testing of vehicle computer page switching. Background Art

[0002] With the development of automobile intelligence, the complexity of the car system is increasing, especially the interaction and switching logic between pages has become more diverse. The traditional manual testing method is difficult to meet the needs of efficient and comprehensive testing, and there are problems such as long testing time and easy to miss errors. Therefore, it is particularly important to develop a system that can automatically identify changes in the car interface, simulate user operations, and verify the correctness of page switching under instructions. Summary of the invention

[0003] The purpose of the present invention is to provide a method, device and system for automatic testing of vehicle computer page switching. The automatic testing system for vehicle computer page switching based on image recognition of the present invention is an innovative testing solution for verifying the responsiveness, accuracy and functionality of the user interface (UI) of an in-vehicle infotainment system (vehicle computer) to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a method for automatically testing a vehicle computer page switch, comprising:

[0005] Pre-training convolutional neural network models based on image datasets of various locomotive system UI elements;

[0006] Identify locomotive systems and select corresponding test methods and test paths;

[0007] Test the locomotive system according to the test method and test path, obtain the image information of the locomotive screen user input interface, the state before and after the page switch, and the page switch response time;

[0008] The convolutional neural network model is used to identify and parse image content, determine whether the page is switched correctly, generate test data, and provide feedback and optimization on the test data.

[0009] Furthermore, the convolutional neural network model is pre-trained based on the image dataset of various locomotive system UI elements, and further includes:

[0010] Use a pre-trained convolutional neural network as the base model;

[0011] A dataset of images of various car system UI elements is used for training, where UI components include buttons, sliders, list items, icons, and text.

[0012] Furthermore, identifying the locomotive system and selecting the corresponding test method and test path also includes:

[0013] Build an automated testing framework, which includes test methods and test paths;

[0014] Construct test scenarios and establish mappings between test scenarios and test methods and test paths;

[0015] Set the expected results according to the test scenario.

[0016] Further, the locomotive system is tested according to the test mode and the test path, and the image information of the locomotive screen user input interface, the state before and after the page switching, and the page switching response time are obtained, which also includes:

[0017] Establish a test script, set the expected page switching action based on the test script, and set the expected page switching action and the expected result accordingly;

[0018] Execute the test script to perform the expected page switching actions and collect the test results.

[0019] Furthermore, the convolutional neural network model is used to identify and parse the image content, determine whether the page is switched correctly, and generate test data, which also includes:

[0020] Compare the collected test results with the expected results to determine the image information of the user input interface, the status before and after page switching, the accuracy of page switching response time, and UI anomalies, and generate test data;

[0021] Generate a test report based on the test data.

[0022] Furthermore, the test data is fed back and optimized, and also includes:

[0023] At the end of each test cycle, collect test results and screenshots for analyzing test data;

[0024] Use convolutional neural network models to analyze test data and identify UI design flaws or performance bottlenecks;

[0025] Optimize and update the convolutional neural network model based on UI design defects or performance bottlenecks.

[0026] On the other hand, a vehicle-computer page switching automatic test device is provided, which applies any of the vehicle-computer page switching automatic test methods described above, and is characterized by comprising:

[0027] A building block for building a convolutional neural network model pre-trained on various locomotive system UI element image datasets;

[0028] Identification module, used to identify the locomotive system and select the corresponding test method and test path;

[0029] An acquisition module is used to test the locomotive system according to the test mode and test path, and obtain the image information of the user input interface of the locomotive screen, the state before and after the page switching, and the page switching response time;

[0030] The analysis module is used to identify and analyze image content through a convolutional neural network model, determine whether the page is switched correctly, and generate test data;

[0031] The processing module is used to provide feedback and optimize the test data.

[0032] On the other hand, a vehicle-computer page switching automatic test system is provided, which is applied to the vehicle-computer page switching automatic test device described above, and is characterized by comprising:

[0033] Server, the server is used to carry the vehicle page switching automatic test device;

[0034] Test bench: The test bench is used to fix the vehicle computer, and the vehicle computer establishes communication with the server via wired or wireless;

[0035] Image acquisition device, the image acquisition device is used to collect image information from the vehicle screen, and the image acquisition device is connected to the test bench.

[0036] In another aspect, an electronic device is provided, the electronic device comprising:

[0037] one or more processors;

[0038] a storage device for storing one or more programs,

[0039] When the one or more programs are executed by the one or more processors, the one or more processors implement an automatic testing method for vehicle computer page switching as described in any one of the above.

[0040] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, any one of the above-mentioned methods for automatically testing vehicle computer page switching is implemented.

[0041] Compared with the prior art, the beneficial effects of the present invention are: the present invention realizes automatic monitoring and testing of the vehicle UI interface to ensure the accuracy of the page switching logic; automated testing significantly reduces manual intervention and improves test speed and coverage; image recognition technology ensures that each page switch can be accurately identified to reduce misjudgment; the labor cost of quality assurance is greatly reduced; the system design is flexible and easy to adapt to different vehicle systems, supporting the testing needs of future new functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of an automatic testing method for vehicle computer page switching in an embodiment of the present invention;

[0043] Figure 2 This is a connection block diagram of an automatic test device for switching vehicle pages according to an embodiment of the present invention;

[0044] Figure 3 This is a connection block diagram of an automatic test system for switching vehicle pages according to an embodiment of the present invention;

[0045] Figure 4 A schematic diagram of the connection structure of an electronic device and a computer-readable storage medium in an embodiment of the present invention;

[0046] In the figure: 10, memory; 11, processor; 12, communication bus; 13, communication interface; 14, computer program; 15, storage medium; 16, construction module; 17, identification module; 18, acquisition module; 19, analysis module; 20, processing module; 100, server; 200, test bench; 300, image acquisition device. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the description of the present invention, unless otherwise specified, "plurality" means two or more than two; the orientations or positional relationships indicated by the terms "upper", "lower", "left", "right", "inner", "outer", "front end", "rear end", "head", "tail", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0049] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] Please refer to the attached drawings of the specification, the present invention provides a technical solution: a method for automatically testing the switching of vehicle-mounted computer pages, comprising the following steps:

[0051] S102, pre-training a convolutional neural network model based on image datasets of various locomotive system UI elements;

[0052] Specifically, a pre-trained convolutional neural network is used as the base model;

[0053] A dataset of images of various car system UI elements is used for training, where UI components include buttons, sliders, list items, icons, and text.

[0054] Among them, this embodiment uses image processing algorithms (such as OpenCV) and machine learning models to identify and parse image content, including buttons, icons, texts and other UI elements, compares changes between different pages, and confirms whether the pages are switched correctly; uses pre-trained convolutional neural networks (CNN), such as ResNet, VGG or Inception series, as the basic model; fine-tunes the model and uses a large number of image data sets containing various car UI elements for training to ensure that the model can accurately identify different types of UI components, such as buttons, sliders, list items, etc.; in the embodiment, the model is deployed on a high-performance GPU server to achieve real-time image processing capabilities.

[0055] S104, identifying the locomotive system, and selecting a corresponding test method and test path;

[0056] Specifically, an automated testing framework is constructed, wherein the automated testing framework includes a testing method and a testing path;

[0057] Construct test scenarios and establish mappings between test scenarios and test methods and test paths;

[0058] Set the expected results according to the test scenario.

[0059] Among them, an automatic identification device is used, and the identification is actively turned on after the device is successfully connected. It can actively identify different vehicle systems and automatically select the test routes and methods preset in advance; trigger page switching events by simulating user input (such as touch, button or voice command); preferably, an automated testing framework can be used, such as Sikuli or other tools specially designed for GUI testing, to develop an automated testing framework, write test scripts in languages ​​such as Python or Java, and define a series of test scenarios and expected results. The test scenarios cover common user operations, such as navigating to different menus, adjusting the volume, switching applications, etc.

[0060] S106, testing the locomotive system according to the test mode and test path, obtaining image information of the locomotive screen user input interface, the state before and after the page switching, and the page switching response time;

[0061] Specifically, a test script is established, an expected page switching action is set based on the test script, and the expected page switching action is set corresponding to the expected result;

[0062] Execute the test script to perform the expected page switching actions and collect the test results.

[0063] Among them, a test script is written to describe a series of expected page switching actions and expected results, and the test script is executed and the test results are automatically collected.

[0064] S108. Recognize and analyze the image content through the convolutional neural network model, determine whether the page is switched correctly, generate test data, and provide feedback and optimize the test data.

[0065] Specifically, the collected test results are compared with the expected results to determine the image information of the user input interface, the state before and after the page switch, the accuracy of the page switch response time, and UI anomalies, and generate test data;

[0066] Generate a test report based on the test data.

[0067] The test results are analyzed to check the response time and accuracy of page switching and any possible UI anomalies. All test results are recorded and a detailed test report is generated, including success and failure cases, error screenshots and performance indicators.

[0068] Specifically, at the end of each test cycle, test results and screenshots are collected for analyzing test data;

[0069] Use convolutional neural network models to analyze test data and identify UI design flaws or performance bottlenecks;

[0070] Optimize and update the convolutional neural network model based on UI design defects or performance bottlenecks.

[0071] At the end of each test cycle, the system collects test results and screenshots for subsequent analysis, uses machine learning algorithms to analyze test data, identifies potential UI design issues or performance bottlenecks, and continuously updates the CNN model based on the collected data to improve image recognition accuracy and the overall performance of the test system.

[0072] On the other hand, a vehicle-computer page switching automatic test device is provided, which applies any of the above vehicle-computer page switching automatic test methods, including:

[0073] A construction module 16 is used to construct a convolutional neural network model pre-trained based on various locomotive system UI element image data sets;

[0074] Among them, a pre-trained convolutional neural network is used as the basic model; a data set of images containing various car system UI elements is used for training, among which UI components include buttons, sliders, list items, icons, and texts.

[0075] Identification module 17, used to identify the locomotive system and select the corresponding test mode and test path;

[0076] Among them, an automated testing framework is constructed, wherein the automated testing framework includes a testing method and a testing path; a test scenario is constructed, and a mapping between the test scenario and the testing method and the testing path is established; and corresponding expected results are set according to the test scenario.

[0077] The acquisition module 18 is used to test the locomotive system according to the test mode and the test path, and obtain the image information of the locomotive screen user input interface, the state before and after the page switching, and the page switching response time;

[0078] Among them, a test script is established, an expected page switching action is set based on the test script, and the expected page switching action is set accordingly to the expected result; the test script is executed to execute the expected page switching action, and the test result is collected.

[0079] The analysis module 19 is used to identify and analyze the image content through a convolutional neural network model, determine whether the page is switched correctly, and generate test data;

[0080] Among them, the collected test results are compared with the expected results to judge the image information of the user input interface, the state before and after the page switch, the accuracy of the page switch response time, and UI anomalies, and generate test data; based on the test data, a test report is generated.

[0081] The processing module 20 is used to provide feedback and optimize the test data.

[0082] At the end of each test cycle, test results and screenshots are collected for analyzing test data. A convolutional neural network model is used to analyze test data to identify UI design defects or performance bottlenecks. The convolutional neural network model is optimized and updated based on UI design defects or performance bottlenecks.

[0083] On the other hand, a vehicle-computer page switching automatic test system is provided, which is applied to the vehicle-computer page switching automatic test device described above, and includes:

[0084] The server 100 is used to carry the vehicle machine page switching automatic test device;

[0085] A test bench 200, which is used to fix the vehicle computer, and the vehicle computer establishes communication with the server via wired or wireless communication;

[0086] The image acquisition device 300 is used to acquire image information from the vehicle screen, and the image acquisition device is connected to the test bench.

[0087] In the above embodiment, the entire system is composed of a central server, a test bench, a vehicle computer, an image acquisition device (such as a camera), and an audio input device (such as a microphone). The central server runs image recognition and automated testing software, such as the vehicle computer page switching automated testing device in the embodiment of the present invention. The test bench is used to fix the vehicle computer and install other testing equipment. The vehicle computer communicates with the test system through a wired or wireless connection, receives test instructions and feeds back test results.

[0088] Traditional manual testing methods can no longer meet the needs of efficient and comprehensive testing. Therefore, it is particularly important to develop a system that can automatically identify the vehicle UI and perform automated testing. The present invention uses deep learning technology to perform image recognition and function verification on the vehicle user interface. The system can automatically perform page switching operations, detect changes in UI elements, and evaluate system response speed and accuracy, thereby significantly improving test efficiency and reliability, reducing human errors, and is suitable for large-scale testing scenarios.

[0089] On the other hand, Figure 4 As shown, an electronic device is provided, the electronic device comprising:

[0090] One or more processors 11;

[0091] a storage device for storing one or more programs,

[0092] When the one or more programs are executed by the one or more processors 11, the one or more processors 11 implement any of the above-mentioned methods for automatically testing vehicle computer page switching.

[0093] Specifically, the storage device is a memory 10, a processor (CPU) 11 and a computer program 14, the computer program 14 is stored in the memory 10, and the processor 11 executes the computer program 14 to implement any one of the above-mentioned vehicle computer page switching automatic test methods. The control system also includes: a communication interface 13 and a communication bus 12, the processor 11, the communication interface 13, and the memory 10 communicate through the communication bus 12, wherein the communication bus 12 is an Industry Standard Architecture (Industry Standard Architecture, referred to as ISA) bus, a Peripheral Component (Peripheral Component, referred to as PCI) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, referred to as EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc., and if the memory 10, the processor 11 and the communication interface 13 are integrated on a chip, the memory 10, the processor 11 and the communication interface 13 can communicate with each other through an internal interface.

[0094] The processor 11 may also be a specific integrated circuit or multiple integrated circuits.

[0095] On the other hand, Figure 4 As shown, a computer-readable storage medium 15 is provided, on which a computer program 14 is stored. When the program is executed by a processor 11, an automatic test method for switching vehicle-based pages as described above is implemented.

[0096] Specifically, the computer program 14 may include a program code, which includes a computer operation instruction; a memory 10, which is used to store the computer program 14. The memory 10 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory; a communication interface 13, which is used to communicate with other devices such as a client or other servers.

[0097] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for automatically testing vehicle page switching, characterized in that: include: Pre-training convolutional neural network models based on image datasets of various locomotive system UI elements; Identify locomotive systems and select corresponding test methods and test paths; Test the locomotive system according to the test method and test path, obtain the image information of the locomotive screen user input interface, the state before and after the page switch, and the page switch response time; The convolutional neural network model is used to identify and parse image content, determine whether the page is switched correctly, generate test data, and provide feedback and optimization on the test data.

2. The method for automatic testing of vehicle computer page switching according to claim 1, characterized in that: Pre-trained convolutional neural network models based on image datasets of various locomotive system UI elements, including: Use a pre-trained convolutional neural network as the base model; A dataset of images of various car system UI elements is used for training, where UI components include buttons, sliders, list items, icons, and text.

3. The method for automatic testing of vehicle computer page switching according to claim 1, characterized in that: Identify the locomotive system, select the appropriate test method and test path, and also include: Build an automated testing framework, which includes test methods and test paths; Construct test scenarios and establish mappings between test scenarios and test methods and test paths; Set the expected results according to the test scenario.

4. The method for automatic testing of vehicle-mounted computer page switching according to claim 3 is characterized in that: Test the locomotive system according to the test method and test path, obtain the image information of the locomotive screen user input interface, the state before and after the page switch, and the page switch response time, and also include: Establish a test script, set the expected page switching action based on the test script, and set the expected page switching action and the expected result accordingly; Execute the test script to perform the expected page switching actions and collect the test results.

5. The method for automatic testing of vehicle-mounted computer page switching according to claim 1, characterized in that: The convolutional neural network model is used to identify and parse image content, determine whether the page is switched correctly, and generate test data, which also includes: Compare the collected test results with the expected results to determine the image information of the user input interface, the status before and after page switching, the accuracy of page switching response time, and UI anomalies, and generate test data; Generate a test report based on the test data.

6. The method for automatic testing of vehicle computer page switching according to claim 5, characterized in that: Feedback and optimization of test data also include: At the end of each test cycle, collect test results and screenshots for analyzing test data; Use convolutional neural network models to analyze test data and identify UI design flaws or performance bottlenecks; Optimize and update the convolutional neural network model based on UI design defects or performance bottlenecks.

7. An automatic test device for switching vehicle pages, using an automatic test method for switching vehicle pages according to any one of claims 1 to 6, characterized in that: include: A building block for building a convolutional neural network model pre-trained on various locomotive system UI element image datasets; Identification module, used to identify the locomotive system and select the corresponding test method and test path; An acquisition module is used to test the locomotive system according to the test mode and test path, and obtain the image information of the user input interface of the locomotive screen, the state before and after the page switching, and the page switching response time; The analysis module is used to identify and analyze image content through a convolutional neural network model, determine whether the page is switched correctly, and generate test data; The processing module is used to provide feedback and optimize the test data.

8. An automatic test system for switching vehicle pages, applied to the automatic test device for switching vehicle pages as claimed in claim 7, characterized in that: include: Server, the server is used to carry the vehicle page switching automatic test device; Test bench: The test bench is used to fix the vehicle computer, and the vehicle computer establishes communication with the server through wired or wireless; Image acquisition device, the image acquisition device is used to collect image information from the vehicle screen, and the image acquisition device is connected to the test bench.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement an automatic testing method for vehicle computer page switching as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, an automatic testing method for vehicle computer page switching as described in any one of claims 1 to 6 is implemented.

Images