A vehicle central control test method, device, equipment, medium and product

Abstract

The present invention relates to the field of vehicle software testing technology, and discloses a vehicle central control testing method, device, equipment, medium, and product. The method includes: obtaining a vehicle central control screen projection screen; obtaining and executing a test case to output a test instruction to the vehicle, so that the vehicle executes the test operation included in the test case in the vehicle central control screen according to the test instruction; receiving a test loss message fed back by the vehicle; in response to the test loss message, calling a large model to extract the element meaning of a first target element; locating a second target element corresponding to the element meaning in the vehicle central control screen projection screen using the large model, and correcting the test case based on the second target element; and continuing testing using the corrected test case. The present invention improves test accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle software testing, and in particular to a vehicle central control testing method, device, equipment, medium and product. Background Art

[0002] With the advancement of intelligent cockpits, the iterative development of vehicle-mounted system software is accelerating, project cycles are shortening, and the testing of vehicle central control systems is becoming increasingly demanding. Currently, each project typically contains over 20,000 functional test cases. Due to the extensive interaction with the vehicle's central control screen (such as clicking, typing, dragging, and so on) and the recording of test results, a full functional test conservatively requires over 200 people. This significantly increases testing costs, especially when performing repeated verification tests (for example, verifying that a software function remains error-free after repeated clicks). Manual testing is also susceptible to human error, such as misjudgments and omissions, which can affect the accuracy of test results. This makes manual testing inefficient for projects with frequently changing software requirements, continuously iterating user interfaces, or complex logic. In recent years, intelligent cockpit technology has advanced rapidly, and automated testing methods have also evolved. Some companies have significantly improved testing efficiency by creating code that simulates various manual operations and then automating testing through these functions. However, during the testing process, the screen of the central control screen may change due to iterations, and the test cases are not updated in time, resulting in the inability of the test cases to accurately perform interactive operations, making the test results inaccurate and delaying the test progress. Summary of the Invention

[0003] In view of this, the present invention provides a vehicle central control testing method, device, equipment, medium and product to solve the problem of inaccurate testing.

[0004] In a first aspect, the present invention provides a vehicle central control testing method, which is applied to a host computer, and the host computer establishes a communication connection with the vehicle, and the method includes: obtaining a vehicle central control projection screen, the vehicle central control projection screen is a screen corresponding to the vehicle central control screen and projected in the host computer, and the vehicle central control screen is a screen displayed on the vehicle central control; obtaining and executing a test case to output a test instruction to the vehicle, so that the vehicle performs the test operation included in the test case in the vehicle central control screen according to the test instruction; receiving a test loss message fed back by the vehicle, the test loss message is a message generated when the vehicle cannot find a first target element corresponding to the operation in the vehicle central control screen when performing the test operation; in response to the test loss message, calling a large model to extract the element meaning of the first target element; locating a second target element corresponding to the element meaning in the vehicle central control screen through the large model, and correcting the test case based on the second target element, the second target element being a screen element obtained by adjusting the first target element in the vehicle central control screen; and continuing testing with the corrected test case.

[0005] According to the above technical means, when executing the vehicle central control test through the test case, if it is found that the first target element of the corresponding operation cannot be found in the vehicle central control screen, for example, the corresponding control or icon cannot be found and thus click, drag, screenshot and other operations cannot be performed, the AI ​​big model is called, and through the image understanding ability and semantic understanding ability of the AI ​​big model, the element meaning of the first target element to be manipulated by the test operation is first understood, and then the current vehicle central control screen is analyzed and understood, and the second target element with the same element meaning as the first target element is matched, and then the second target element is used as the first target element to continue the test, and the test case is modified accordingly according to the second target element. Not only does it ensure that the test can continue to be executed accurately and reliably, solve the problem of test interruption, and improve the test accuracy, but it also automatically modifies the test case according to the backup function of the AI ​​big model, so that the test case can also automatically change after the vehicle central control screen is modified, reducing the steps of users manually modifying the test case frequently and reducing labor costs.

[0006] In some optional embodiments, the steps of creating the test case include: detecting the user's current operation on the vehicle's central control screen projection screen; recording the element attribute information of the area corresponding to the current operation, and capturing the area cut-out of the area corresponding to the current operation; extracting the text information in the area cut-out; generating the current test instruction based on the current operation and the element attribute information; saving the current test instruction, the area cut-out and the text information as the recording information of the current operation; and arranging the recording information of each step of the user's operation in sequence to obtain the test case.

[0007] Based on the above technical means, the present invention provides a test case recording method. A communication connection is established between a host computer and a vehicle, and the vehicle's central control screen is then projected onto the host computer screen. The user manually performs test operations on the host computer screen while the recording function is activated. Similar to a video recording function, each user step is recorded and the recorded operations are converted into test cases. Compared to code-based test case creation solutions, the test case recording method provided by the present invention significantly speeds up test case generation, reduces the difficulty of writing test cases, and lowers the threshold for tester recruitment. To convert recorded data into test cases in the form of instructions, it is necessary to record each user's screen operation and the specific location of each operation, thereby extracting element attribute information, screenshots, and text at the corresponding locations. Element attribute information and operation actions can be used to form specific test instructions to indicate which operation was performed on which element object. The screenshots and text determine the specific meaning of the corresponding operation object. Even if the graphic designer subsequently fine-tunes the central control screen, the AI ​​model can automatically correct the original test instructions based on the screenshots and text, improving test reliability.

[0008] In some optional embodiments, the host computer and the vehicle are connected through an Android test bridge, and the recording of element attribute information of the area corresponding to the current operation includes: creating an element structure tree; sending a get user interface layout command to the vehicle at a preset period; sending a get interface file command to the vehicle at a preset period; receiving an interface file fed back by the vehicle, the interface file is a file sent to the host computer by the vehicle each time in response to the get interface file command, the interface file is generated by the vehicle based on the get user interface layout command, the interface file saves interface element layout information, the interface element layout information is extracted from the vehicle's central control screen and is used to represent the screen elements in the vehicle's central control screen; parsing the interface file to obtain the interface element layout information; filling the interface element layout information into the element structure tree for display; searching for the element attribute information from the element structure tree according to the position of the area corresponding to the current operation.

[0009] According to the above technical means, the host computer remotely obtains the interface element layout information of the vehicle's central control screen from the vehicle based on the Android test bridge, and updates the interface element layout information to the element structure tree in real time. Therefore, when the screen changes, the layout and attributes of each element in the screen are updated to the element structure tree as soon as possible. The various attribute information in the interface is displayed to the user through the element structure tree, so that the user and the host computer can clearly understand what the operation object corresponds to each test operation of the user, thereby ensuring the accuracy of test case creation.

[0010] In some optional embodiments, in response to the test loss message, calling the big model to extract the element meaning of the first target element includes: analyzing the icon in the area cut image through the big model to obtain the first element meaning; analyzing the text information through the big model to obtain the second element meaning.

[0011] According to the above technical means, based on the process created by the aforementioned test cases, each first target element being operated includes two parts of backup information: area cut-out image and text information. Therefore, when the first target element is lost, the AI ​​large model understands the meaning of the first target element from both image and text perspectives, and then finds the second target element with the corresponding meaning from the current vehicle central control screen, further improving the accuracy of matching the second target element.

[0012] In some optional embodiments, the method of locating the second target element corresponding to the meaning of the element in the vehicle's central control projection screen through the large model and correcting the test case based on the second target element includes: searching for a target icon that matches the meaning of the first element in the vehicle's central control projection screen through the large model; when the target icon is searched through the first element meaning, recording the target element attribute information corresponding to the target icon, and the target element attribute information and the target icon serve as the second target element; modifying the test instructions in the test case through the target element attribute information; replacing the corresponding area cut image with the target icon; when the target icon cannot be searched through the first element meaning, searching for a target text that matches the meaning of the second element in the vehicle's central control projection screen through the large model; determining the target icon based on the area where the target text is located, and returning to the step of recording the target element attribute information corresponding to the target icon.

[0013] Using this technical approach, the second target element is first matched using the image meaning of the region cut. If the image doesn't match, the textual meaning is then used for matching. When a match is found, a new screenshot of the second target element is taken, replacing the old one with the new one. The test instructions in the test case are adjusted using the attribute parameters of the new screenshot area, thus achieving an automated test case correction process. This dual meaning matching ensures the accuracy of the automated test case correction.

[0014] In some optional embodiments, obtaining the vehicle's central control screen projection screen includes: sending a screen acquisition message to the vehicle; receiving the vehicle's central control screen projection screen feedback from the vehicle, wherein the vehicle's central control screen projection screen is an image that the vehicle takes a screenshot of the central control screen at a preset frame rate after receiving the screen acquisition message, compresses the screenshot, and then deduplicates the screenshot based on the similarity between adjacent frame screenshots and sends the image.

[0015] Using these technologies, the vehicle screen image is compressed as it is captured, reducing image resolution, maintaining image quality while effectively increasing transmission speeds. By using an image comparison algorithm to remove duplicate images, the frame rate is controlled at around 13 FPS, enabling real-time display of the vehicle's image without delay while also meeting basic viewing requirements for projection.

[0016] In some optional embodiments, the compressed screenshot includes a heavily compressed area and a lightly compressed area, wherein the heavily compressed area is an area where the vehicle compresses an unimportant area in the screenshot using a first compression ratio, and the lightly compressed area is an area where the vehicle compresses an important area in the screenshot using a second compression ratio, and the first compression ratio is less than the second compression ratio.

[0017] According to the above technical means, the vehicle's central control screen is divided into important areas and unimportant areas, and the compression rate is increased for important areas and reduced for unimportant areas. This compression method not only reduces the size of the screen but also takes into account the picture quality.

[0018] In some optional embodiments, the method also includes: sending an audio acquisition message to the vehicle; receiving vehicle audio feedback from the vehicle through the Android test bridge, the vehicle audio is an audio recording object created by the vehicle in response to the audio message, the vehicle background audio data is read through the audio recording object, and then the read audio data is compressed into a playable format file, and sent to the host computer through the Android test bridge; the vehicle audio is transmitted to the audio playback application through local inter-process communication for playback.

[0019] The aforementioned technology, based on the Android Debug Bridge, also transmits vehicle audio from the backend to a host computer, which then plays the audio from the vehicle's central control system. This allows further verification of the accuracy of certain tests using vehicle audio. Furthermore, direct transmission of vehicle audio streams from the backend offers higher audio quality and lower barriers to entry compared to audio recording methods such as external microphones.

[0020] In a second aspect, the present invention provides a vehicle central control test device, which is applied to a host computer, and the device includes: a screen projection module, which is used to obtain a vehicle central control screen projection screen, wherein the vehicle central control screen projection screen is a screen corresponding to the vehicle central control screen and projected into the host computer, and the vehicle central control screen is a screen displayed on the vehicle central control; a use case execution module, which is used to obtain and execute a test case to output a test instruction to the vehicle, so that the vehicle performs the test operation included in the test case in the vehicle central control screen according to the test instruction; a target loss module, which is used to receive a test loss message fed back by the vehicle, wherein the test loss message is a message generated when the vehicle cannot find a first target element corresponding to the operation in the vehicle central control screen when performing the test operation; a lost target analysis module, which is used to call a large model to extract the element meaning of the first target element in response to the test loss message; a lost target detection module, which is used to locate a second target element corresponding to the element meaning in the vehicle central control screen through the large model, and correct the test case based on the second target element, wherein the second target element is a screen element obtained by adjusting the first target element in the vehicle central control screen; and a fallback test module, which is used to continue testing with the corrected test case.

[0021] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0023] In a fifth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0024] The technical solution provided by the present invention has the following advantages:

[0025] (1) According to the above technical means, when executing the vehicle central control test through the test case, if it is found that the first target element of the corresponding operation cannot be found in the vehicle central control screen, for example, the corresponding control or icon cannot be found and thus the click, drag, screenshot and other operations cannot be performed, the AI ​​big model is called, and through the image understanding ability and semantic understanding ability of the AI ​​big model, the element meaning of the first target element to be manipulated by the test operation is first understood, and then the current vehicle central control screen is analyzed and understood, and a second target element with the same element meaning as the first target element is matched, and then the second target element is used as the first target element to continue the test, and the test case is modified accordingly according to the second target element. Not only does it ensure that the test can continue to be executed accurately and reliably, solve the problem of test interruption, and improve the test accuracy, but also the test case is automatically modified according to the backup function of the AI ​​big model, so that the test case can also be adaptively changed automatically after the vehicle central control screen is modified, reducing the steps of users manually modifying the test case frequently and reducing labor costs.

[0026] (2) Based on the above technical means, the present invention provides a test case recording method, which establishes a communication connection between the host computer and the vehicle, and then projects the screen image of the vehicle's central control onto the host computer screen. The user manually performs test operations on the host computer screen and turns on the recording function at the same time, which is similar to the video recording function. Each step of the user's operation is recorded and the recorded operation is converted into a test case. Compared with the test case creation scheme of code programming, the test case recording method provided by the present invention significantly improves the speed of test case generation, reduces the difficulty of writing test cases, and lowers the threshold for appointment of test personnel. Among them, in order to realize the conversion of recorded data into test cases in the form of instructions, on the one hand, it is necessary to record each step of the user's operation on the screen, and on the other hand, it is necessary to record the specific location of each step of the operation, so as to extract element attribute information, screenshots and text at the corresponding location. Specific test instructions can be formed by element attribute information and operation actions to indicate which operation is performed on which element object; the specific meaning of the corresponding operation object is determined by screenshots and text. Even if the subsequent image designer fine-tunes the image of the central control screen, the AI ​​large model can automatically correct the original test instructions based on the screenshots and text, thereby improving the reliability of the test.

[0027] (3) According to the above technical means, the host computer remotely obtains the interface element layout information of the vehicle's central control screen from the vehicle based on the Android test bridge, and updates the interface element layout information to the element structure tree in real time. When the screen changes, the layout and attribute information of each element in the screen are updated to the element structure tree in the first time. The various attribute information in the interface is displayed to the user through the element structure tree, so that the user and the host computer can clearly understand what the operation object corresponds to each test operation of the user, thereby ensuring the accuracy of the test case creation.

[0028] (4) According to the above technical means, based on the process created by the aforementioned test case, each first target element being operated includes two parts of backup information: regional cut-out image and text information. Therefore, when the first target element is lost, the AI ​​large model understands the meaning of the first target element from both image and text perspectives, and then finds the second target element with the corresponding meaning from the current vehicle central control screen, further improving the accuracy of matching the second target element.

[0029] (5) Based on the above technical means, the second target element is first matched by the image meaning of the region cut image. If the image does not match, the text meaning is then used for matching. When the corresponding second target element is matched, a new screenshot of the second target element is taken, and then the new screenshot is used to replace the old screenshot. The attribute parameters of the new screenshot area are used to adjust the test instructions in the test case, thus realizing an automated test case correction process. The dual meaning matching ensures the accuracy of the automated test case correction.

[0030] (6) Based on the above technical means, the vehicle screen image is compressed while being acquired, reducing the image resolution, maintaining image quality while effectively improving the transmission speed. By using an image comparison algorithm to remove duplicate images, the frame rate is controlled at around 13FPS, which can not only display the vehicle screen in real time without delay, but also meet the basic viewing and projection effect.

[0031] (7) Based on the above technical means, the vehicle central control screen is divided into important areas and unimportant areas, and the compression rate is increased for the important areas and reduced for the unimportant areas, which not only reduces the size of the screen but also takes into account the quality of the screen.

[0032] (8) Based on the above technical means, the technical means based on the Android debug bridge also transmit the vehicle audio from the background to the host computer, and the host computer plays the audio of the vehicle's central control, so that the accuracy of certain tests can be further verified through the vehicle audio. In addition, the vehicle audio stream is directly transmitted from the background, which has higher audio quality and lower usage barriers compared to audio recording methods such as external microphones. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 1 is a flow chart of a method for testing a vehicle central control system according to an embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of a process for creating a test case according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram showing the effect of displaying an element structure tree according to an embodiment of the present invention;

[0037] Figure 4 is a schematic diagram of a process for obtaining interface element layout information according to an embodiment of the present invention;

[0038] Figure 5 is another flowchart of a vehicle central control testing method according to an embodiment of the present invention;

[0039] Figure 6 is another flowchart of a vehicle central control testing method according to an embodiment of the present invention;

[0040] Figure 7 2 is a schematic structural diagram of a vehicle central control test device according to an embodiment of the present invention;

[0041] Figure 8 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0043] With the advancement of intelligent cockpits, the iterative development of vehicle-mounted system software is accelerating, project cycles are shortening, and the testing of vehicle central control systems is becoming increasingly demanding. Currently, each project typically contains over 20,000 functional test cases. Due to the extensive interaction with the vehicle's central control screen (such as clicking, typing, dragging, and so on) and the recording of test results, a full functional test conservatively requires over 200 people. This significantly increases testing costs, especially when performing repeated verification tests (for example, verifying that a software function remains error-free after repeated clicks). Manual testing is also susceptible to human error, such as misjudgments and omissions, which can affect the accuracy of test results. This makes manual testing inefficient for projects with frequently changing software requirements, continuously iterating user interfaces, or complex logic. In recent years, intelligent cockpit technology has advanced rapidly, and automated testing methods have also evolved. Some companies have significantly improved testing efficiency by creating code that simulates various manual operations and then automating testing through these functions.

[0044] However, during the testing process, the current technology has three major problems:

[0045] 1. The screen of the central control screen may change due to iteration (for example, the UI designer temporarily modified the shape or color of certain element icons), but the test cases are not updated in time. As a result, when the vehicle computer executes the automatic test cases, it does not know where the element objects to be operated by the test cases are, and cannot accurately perform interactive operations, resulting in inaccurate test results and delays in test progress.

[0046] 2. Test cases are written in pure code, which places high demands on personnel, increases the recruitment and learning threshold for testers, and is not conducive to the flexibility of testing.

[0047] 3. Creating test cases by simulating various manual operations requires setting up various software environments on the vehicle in advance to meet the working requirements of its automated testing software framework, which makes the deployment of the test software environment too complicated.

[0048] To solve the above problems, according to an embodiment of the present invention, a method for testing a vehicle central control is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0049] In this embodiment, a vehicle central control test method is provided, which is applied to a host computer, and a communication connection is established between the host computer and the vehicle. Figure 1 1 is a flow chart of a method for testing a vehicle central control system according to an embodiment of the present invention, the flow chart comprising the following steps:

[0050] Step S101, obtaining the vehicle central control screen projection screen, the vehicle central control screen projection screen is the screen corresponding to the vehicle central control screen and projected on the host computer, and the vehicle central control screen is the screen displayed on the vehicle central control;

[0051] Specifically, the vehicle central control testing method provided in embodiments of the present invention is applied to a host computer, which refers to a device such as a personal computer, server, or tablet computer that can establish a communication connection with the vehicle and contains a processor. The testing software environment is primarily deployed on the host computer, and test commands are sent from the host computer to the vehicle to complete remote vehicle testing.

[0052] The preparatory work before the test is to project the vehicle's central control screen onto the host computer's screen to obtain the vehicle's central control screen projection screen. In this embodiment, the vehicle's central control screen refers to the screen displayed on the vehicle's central control screen, and the vehicle's central control screen projection screen refers to the screen displayed on the host computer, projected by the vehicle, and identical to the vehicle's central control screen.

[0053] Step S102 , obtaining and executing a test case to output a test instruction to the vehicle, so that the vehicle executes the test operations included in the test case in the vehicle central control screen according to the test instruction.

[0054] In an embodiment of the present invention, a test case refers to a set of information, including test input, execution conditions, and expected results, designed to verify the software functions or requirements of the vehicle's central control system. It is used to determine whether the system meets specific requirements. Test cases are the core tool of software testing, converting test behavior into a quantifiable and manageable model. In an embodiment of the present invention, test cases can be generated by engineers using traditional code programming methods, or they can be created using the test case recording method and graphical programming methods provided in subsequent embodiments of the present invention. The specific creation method will be described in detail in subsequent embodiments.

[0055] In this step, the host computer retrieves the created test case and, by executing the test case, outputs test instructions to the vehicle. The vehicle responds to the test instructions by simulating a user performing corresponding test operations on the vehicle's central control screen. For example: clicking a group of application icons in sequence to record whether each application launches correctly; repeatedly opening and closing a page 100 times to determine whether a crash occurs. Such test operations, as long as they can be used to test the reliability of the vehicle's central control, can be included in the test case and will not be further described in detail in the present embodiment.

[0056] Step S103: receiving a test loss message fed back by the vehicle. The test loss message is a message generated when the vehicle cannot find the first target element corresponding to the operation in the vehicle central control screen when performing a test operation.

[0057] Specifically, in actual applications, in addition to the upgrade and iteration of vehicle computer performance, there are often design iterations of the user interface, so that the new interface gives users a fresh feeling and brightens their mood. However, after the vehicle's central control screen is modified, the interface designer and the test engineer often do not belong to the same department and rarely perform alignment work on the engineering project, resulting in the test case not being updated in real time, which in turn leads to test failure or test interruption. For example, the color, ID, name or shape of a certain screen icon A has been modified, and the test case records the click on icon A, and also records the original attributes of icon A, including the original color, original shape, original name and original ID. Because the elements have been modified, it is impossible to match the target 100% through operations such as image matching, ID matching, and name matching. As a result, the target is lost when the test case executes the step of clicking icon A, and the first target element of the corresponding operation cannot be found.

[0058] Based on this, when the vehicle executes a test command and cannot find the corresponding first target element in the vehicle's central control screen, a test loss message is generated and returned to the host computer. The test loss message includes the missing first target element and its related attribute information.

[0059] Step S104: In response to the test loss message, the large model is called to extract the element meaning of the first target element.

[0060] Step S105: Using the large model, locate a second target element corresponding to the meaning of the element in the vehicle's central control screen, and modify the test case based on the second target element, where the second target element is a screen element obtained by adjusting the first target element in the vehicle's central control screen;

[0061] Step S106: Continue testing using the corrected test case.

[0062] Specifically, after receiving the test loss message, the host computer learns that the first target element in the test case cannot be found. This may be because the vehicle's central control screen has been adjusted, and the test case has not been updated synchronously. In this case, the host computer uses the large model backup strategy to correct the test case.

[0063] In this embodiment, the large model refers to an AI large model, which refers to a type of artificial intelligence model with a large number of parameters built by an artificial neural network. It is usually pre-trained on massive data through self-supervised learning or semi-supervised learning, and then its performance and capabilities are further optimized through methods such as instruction fine-tuning and human alignment. The large model has the characteristics of a large number of parameters, large training data, and large computing resources. It has the ability to solve general tasks, follow human instructions, and perform complex reasoning. In this embodiment of the present invention, the large model includes but is not limited to a large language model, a large visual model, a large multimodal model, and a large basic science model.

[0064] The embodiment of the present invention determines the element meaning of the first target element in the test case through a large model, for example, analyzing its function based on the icon of the first target element, for example, the meaning of the first target element represents window closing, music playing, etc.

[0065] Afterwards, because the vehicle's central control screen has been projected to the host computer, the host computer can also use the large model to analyze the vehicle's central control screen projection, determine the specific meaning of different elements in the current screen, and determine whether there is a modified element that matches the meaning of the first target element.

[0066] If a second target element with the same meaning as the element is found, the second target element is considered to be a modified representation of the first target element. For example, a modified icon represents a window closing, which is the same as the first target element. The host computer can then use the second target element to modify the original test case, obtain a new test case, and then continue testing based on the new test case.

[0067] According to the above technical approach, when executing a vehicle central control test using a test case, if the first target element corresponding to the operation cannot be found on the vehicle central control screen, the AI ​​large model is invoked. Using its image and semantic understanding capabilities, the AI ​​large model first understands the elemental meaning of the first target element to be manipulated by the test operation. It then analyzes and understands the current vehicle central control screen, matches a second target element with the same elemental meaning as the first target element, and then continues the test using the second target element as the first target element. The test case is then modified accordingly based on the second target element. This not only ensures that the test can continue accurately and reliably, resolving the problem of test interruptions and improving test accuracy, but also automatically modifies the test case based on the AI ​​large model's fallback function, allowing the test case to adapt to changes in the vehicle central control screen after modification. This reduces the need for users to frequently manually modify test cases and reduces labor costs. Furthermore, the test tool is installed on the host computer, and test operations are executed on the host computer. The vehicle computer only needs to respond to the host computer's instructions to execute actions, eliminating the need to configure a complex interactive software framework, reducing the difficulty of deploying the vehicle computer's test software environment.

[0068] In some optional implementations, the step of creating a test case includes:

[0069] Step a1: Detect the user's current operation on the vehicle's central control screen projection screen;

[0070] Step a2: record the element attribute information of the area corresponding to the current operation, and capture the area cut of the area corresponding to the current operation;

[0071] Step a3, extracting text information from the region cut image;

[0072] Step a4, generating a current test instruction based on the current operation and element attribute information;

[0073] Step a5: save the current test instruction, area cut image and text information as the recording information of the current operation;

[0074] Step a6: Arrange the recorded information of each user operation in sequence to obtain a test case.

[0075] Specifically, an embodiment of the present invention provides a recorded test case creation method that combines screen recording and test case generation means.

[0076] like Figure 2 As shown, first of all, since the vehicle's central control screen has been projected on the host computer, the user can directly perform relevant operations on the vehicle's central control screen in the host computer, such as clicking an icon, dragging a window, and so on. The process of recording user operations can be achieved by capturing mouse events through the Windows API, which is a prior art and will not be described in detail in this embodiment. However, since the vehicle's central control screen is a projected screen, the user's operations will not be responded to by the host computer, so it is necessary to let the host computer know which elements and objects the user's interactive operations correspond to in the vehicle's central control screen. Therefore, the embodiment of the present invention remotely extracts the interface element information of the vehicle's central control screen to achieve the effect of determining the user's operation object. In this embodiment, the interface element information refers to the attribute information values ​​of different elements in the vehicle's central control screen. For example, each icon in the interface that can be clicked and triggered records its corresponding specific location, ID, name, path, value, etc.

[0077] Based on the user's operation on the vehicle's central control screen projection screen of the host computer (for example, the mouse clicked on a certain area), the user's operation position is located, and then the interface element information is analyzed to see whether the user's clicked position includes an operable element (for example, it is found that there is a triggerable icon in the area currently clicked by the mouse). The element attribute information of the area corresponding to the current operation is recorded through the interface element information (for example, the ID, name, path, value, etc. of a specific icon in the area clicked by the mouse). By detecting each step of the user's operation and combining it with the element attribute information of the corresponding area, a test instruction (or test script) can be automatically generated. For example, the instruction is: click the close window element with ID XXX. This instruction can be used to perform a test to see if the window can be closed normally.

[0078] During the user's operation, a screenshot is taken of the tested clicked area and saved with each recorded operation. Text recognition technology is also used to convert the text in the image into text and also record the information. Finally, the recorded information corresponding to all user operations is organized one by one in the order of operation to form a complete test case.

[0079] Through the technical solution provided by the embodiment of the present invention, each operation action of the user on the screen is recorded, and the specific location of each operation is also recorded, so that element attribute information, screenshots and text are extracted at the corresponding position. Specific test instructions can be formed through element attribute information and operation actions to indicate what operation is performed on which element object, thereby realizing the conversion of recorded data into test cases in the form of instructions. Compared with the test case creation solution of code programming, the test case recording method provided by the present invention significantly improves the speed of test case generation, reduces the difficulty of writing test cases, and lowers the threshold for appointment of testers. In addition, screenshots are taken at each step of the operation, and the text in the screenshots is saved at the same time. The saved screenshots and text are used to determine the specific meaning of the corresponding operation object in the area cut-out. Even if the subsequent graphic designer fine-tunes the picture of the central control screen, the AI ​​large model can recognize the meaning of the element to be operated based on the screenshots and text, and then automatically correct the original test instructions according to the meaning of the element, thereby improving the reliability of the test.

[0080] Based on the above test case creation steps, step S104 includes:

[0081] Step b1: Analyze the icons in the regional slice using the large model to obtain the meaning of the first element;

[0082] Step b2: Analyze the text information through the large model to obtain the second element meaning.

[0083] Furthermore, step S105 includes:

[0084] Step c1: Searching for a target icon that matches the meaning of the first element in the vehicle's central control screen using the large model;

[0085] Step c2: when a target icon is found through the search of the first element meaning, the target element attribute information corresponding to the target icon is recorded, and the target element attribute information and the target icon are used as the second target element;

[0086] Step c3, modifying the test instructions in the test case according to the target element attribute information;

[0087] Step c4, using the target icon to replace the corresponding area cut image;

[0088] Step c5: When the target icon cannot be found through the first element meaning, the target text that matches the second element meaning is searched for in the vehicle's central control screen using the large model;

[0089] Step c6: determining a target icon based on the area where the target text is located, and returning to the step of recording target element attribute information corresponding to the target icon.

[0090] According to the above technical means, based on the process created by the aforementioned test cases, each first target element being operated includes two parts of backup information: area cut-out image and text information. Therefore, when the first target element is lost, the AI ​​large model understands the meaning of the first target element from both image and text perspectives, and then finds the second target element with the corresponding meaning from the current vehicle central control screen, further improving the accuracy of matching the second target element.

[0091] In the specific matching process of the above steps c1 to c5, the second target element is first matched through the image meaning of the regional cut image. If the image cannot be matched, the text meaning is used for matching. If the second target element cannot be matched, manual processing is required. As long as the corresponding second target element is matched through any one of the image meaning of the regional cut image (the meaning of the first element) and the text meaning of the text information (the meaning of the second element), the embodiment of the present invention locates the target icon corresponding to the second target element, and then re-screens the target icon, replacing the old screenshot with the new screenshot. At the same time, the new screenshot area is used to search for the target element attribute information in the element structure tree, and the new attribute parameters are used to adjust the test instructions in the test case, thereby realizing an automated test case correction process. Dual meaning matching ensures the accuracy of automated test case correction.

[0092] In some optional implementations, the host computer and the vehicle are connected via an Android test bridge, and step a2 includes:

[0093] Step d1, creating an element structure tree;

[0094] Step d2, sending a command to obtain the user interface layout to the vehicle at a preset period;

[0095] Step d3, sending a command to obtain the interface file to the vehicle at a preset period;

[0096] Step d4: Receive an interface file fed back by the vehicle. The interface file is a file sent to the host computer by the vehicle each time in response to the Get Interface File command. The interface file is generated by the vehicle based on the Get User Interface Layout command. The interface file stores interface element layout information, which is information extracted from the vehicle's central control screen and used to represent screen elements in the vehicle's central control screen.

[0097] Step d5, parsing the interface file to obtain interface element layout information;

[0098] Step d6: Fill the interface element layout information into the element structure tree for display;

[0099] Step d7: searching the element attribute information from the element structure tree according to the position of the area corresponding to the current operation.

[0100] Specifically, an embodiment of the present invention provides a specific implementation method for obtaining element attribute information, which is used to locate the specific element object operated by the user in each step of the operation in the projection screen.

[0101] In this embodiment of the present invention, an ElementFrame window (a framework element used to organize and display GUI components) is first created using a GUI (Graphical User Interface) development tool to display the element structure tree interface. A class named JTreeElement is created that inherits the JTree class to display the entire contents of the element structure tree. JTree is a component in the Java Swing library used to display tree-structured data in a graphical user interface. A JListElement component is then created that inherits JList, a component in the Swing framework used to display a list and thus display detailed information about the element selected by the user. This completes the element structure tree.

[0102] In other words, the interface element layout information of the vehicle Android system is taken from the vehicle equipment to the host computer. The interface element layout information records the specific location of each operable object in the vehicle's central control screen as well as its ID, name, path, value and other attributes. For example Figure 3As shown, the interface element layout information will be filled in the pre-created element structure tree. When the user clicks an icon with the mouse, the element structure tree will be searched according to the position where the user clicked the mouse, and the specific attributes corresponding to the icon will be searched. The specific attributes corresponding to the icon can then be displayed through the JListElement component.

[0103] like Figure 4 As shown, interface element layout information is retrieved from the vehicle to the host computer and then populated into the element structure tree. The specific implementation process involves programming a loop on the host computer to retrieve the list of connected vehicle devices every second. After the vehicle has installed the Android system and switched to Android Test Bridge mode, the vehicle's information is automatically reported to the vehicle's device list. The host computer can create a child process at a preset interval (for example, every second) and send the command "adb shell uiautomator dump / sdcard / ui.xml" to the vehicle. This process then executes "adb shell uiautomator dump / sdcard / ui.xml." The vehicle then retrieves the hierarchical information of all controls currently applied on the screen and saves it in XML format to the interface file ui.xml on the SD card. Similarly, the host computer can periodically send the command "adb pull / sdcard / ui.xml" to the vehicle, causing it to transfer the interface file ui.xml from the vehicle's SD card to the host computer. Finally, the host computer parses the ui.xml file to obtain the desired interface element layout information. Within the parsed XML structure, specific UI elements can be found based on their attributes (such as resource-id, text, and class). In this embodiment of the present invention, the layout information of interface elements acquired in the next cycle is compared with that acquired in the previous cycle to determine whether the current screen has changed. When a change is detected on the vehicle's display screen, the element structure tree interface is immediately updated with the latest acquired layout information, completing a real-time synchronous refresh.

[0104] According to the above technical means, the host computer remotely obtains the interface element layout information of the vehicle's central control screen from the vehicle based on the Android test bridge, and updates the interface element layout information to the element structure tree in real time. Therefore, when the screen changes, the layout and attributes of each element in the screen are updated to the element structure tree as soon as possible. The various attribute information in the interface is displayed to the user through the element structure tree, so that the user and the host computer can clearly understand what the operation object corresponds to each test operation of the user, thereby ensuring the accuracy of test case creation.

[0105] In some optional implementations, the step of creating a test case further includes:

[0106] Step g1: Obtain the vehicle's central control screen projection image and the element structure tree filled with interface element layout information;

[0107] Step g2: obtaining a test method block, wherein different test method blocks are used to map different test functions, and the test functions are used to pre-write instructions for the test method;

[0108] Step g3, receiving a tile operation command from the user, and arranging the test method tiles in the use case display area in the order of the test process in response to the tile operation command;

[0109] Step g4, receiving a parameter setting command from the user, and filling in test parameters in the arranged test method blocks in response to the parameter setting command, wherein the parameter setting command is initiated by the user based on the interface element layout information in the element structure tree;

[0110] Step g5: convert the arranged test method blocks and test parameters into test cases.

[0111] Specifically, based on the vehicle central control screen projection image and element structure tree obtained in the above steps, an embodiment of the present invention also provides a specific solution for creating test cases by graphical programming.

[0112] The test method block mechanism provides users with test methods they can use. For example, users can directly call the test method block to perform specific operations such as clicking, dragging, and taking screenshots on certain icons on the vehicle's central control screen.

[0113] In an embodiment of the present invention, commonly used test operations are encapsulated through test method blocks, different test method blocks are used to map different test functions, and different test functions are used to pre-write instructions for different test methods. From the user's perspective, each test method block is just a graphic or text name. The user only needs to click on different test method blocks in a certain order to sort the test method blocks, thereby generating corresponding test step scripts. For example, the user first double-clicks the test method block representing "drag" and then double-clicks the test method block representing "click", then a test script is automatically generated that first drags an icon and then clicks a certain icon, thereby automatically completing the sorting of the test methods. At this time, it is necessary to further fill in the corresponding test parameters to determine which interface elements to operate, thereby completing the creation of the test case.

[0114] In a specific application embodiment, the test method block can be generated through the following process:

[0115] 1. Encapsulate different test methods into callable functions;

[0116] 2. Create a database;

[0117] 3. Create a test method table in the database, and fill in the test function names corresponding to the callable functions into the test method table, and establish a mapping relationship between the test method table and different test methods;

[0118] 4. Render the test method table in the tile panel.

[0119] Specifically, create a MySQL database, for example, called Case in this example. Create a test method table within the Case database to record the names of different test methods. The test method table may also include the test method ID, creation time, and order. Implement the corresponding method functionality through programming code to obtain a callable function. Define the method with the same name in the test method table, and establish a mapping between the test method table and the callable functions corresponding to different test methods.

[0120] Next, create a ListFrame window as the tile panel to display the different test method tiles. Alternatively, the ListFrame window can create a JList interface that inherits the functionality of the JList component (a Java component used to display lists of elements) to display the contents of different callable functions and describe the test method corresponding to each callable function. By following these steps, the test method table is rendered in the tile panel, completing the creation of the test method tile.

[0121] Based on the above technical means, the present invention provides a technical solution for automated operation test methods, implementing the relevant functions of each test method block in the back-end of the tool. The front-end interface presents the pre-implemented test method blocks to the user in the form of a list. This approach significantly reduces the development cost for automation script writers, improves the quality of automation scripts, and makes automated testing more stable and efficient.

[0122] Based on the test method blocks provided in the previous steps, users can give block operation commands such as clicking and dragging on the host computer screen, and add the test method blocks to the use case display area in the order of operations, thereby realizing graphical test case programming.

[0123] For example, in a specific embodiment, based on the aforementioned ListFrame window, the mouseClicked function can be rewritten in the JList interface class. When the number of mouse clicks is determined to be 2, the name of the callable function is passed to the use case display area, displaying the test method name and parameters. Furthermore, a custom JTextFieldSearch component can be created that inherits the JTextField class to support user text input to search for related callable functions. Simultaneously, a JComboBoxCase component can be created that inherits the JComboBox class to support user selection of related callable functions using a drop-down list.

[0124] After starting the test application, the ListFrame window reads the quick callable functions and all callable functions from the Case database, and renders them on the tile panel. In an embodiment of the present invention, quick callable functions and all callable functions can be displayed on two different pages, where quick callable functions represent the user's most commonly used test methods, such as clicks, assertions, sliding, text input, etc.; all callable functions are used to display all automated test methods that users can use. In addition, the embodiment of the present invention also classifies all callable functions. The specific classifications include image operations, assertions, systems, CAN, device operations, control clicks, etc., to facilitate users to quickly find the callable functions they need.

[0125] In a specific application embodiment, a window can be formed by creating an EditFrame class that inherits the JFrame class to serve as the use case display area. A JTextFieldCase component can be created that inherits the JTextField class to display the script text edited by the user. The use case display area can be further designed into seven areas: the view area, the code area, the properties area, the use case information area, the pre-stage area, the use case steps area, and the post-stage area.

[0126] The view area is the graphical programming area, where users manually select callable functions and add them to the view area, such as click, pause, assert, zoom, drag, slide, and screenshot operations. It also supports manually entering target values ​​and annotating use case information (personnel name, use case level, use case description, vehicle information, and business module). Even those without programming experience can write automated use case scripts, lowering the technical threshold for automated testers and improving automation writing efficiency.

[0127] The code area supports writing automated scripts, a traditional way of writing code. In this example, a script prompt function is added, which automatically associates the callable functions that may be needed during writing. Users can easily use the target method by simply pressing Enter or double-clicking to select it. Custom script execution logic is also supported, allowing users to write some common methods, reducing duplication of code and making it more convenient and quick to use.

[0128] The attribute area and use case area can display some attribute information and description information about the use case.

[0129] The pre-stage area, the use case step area, and the post-stage area are the three main areas that come into play when the use case is executed. The pre-stage area is used to define the pre-steps required before the test case is executed, such as powering on the machine, checking component status, and other routine operations. The use case step area is used to display the created test case, which is the completed version of the use case and can be executed directly. The post-stage area is mainly used to display the fallback code edited by the user. That is, no matter what problems arise during the test process, you can jump to the post-stage area and execute the fallback code to complete the test case execution process normally.

[0130] In summary, another test case creation technical solution provided by an embodiment of the present invention encapsulates the callable function corresponding to the automated test method, includes all the operations that can be used in the automated script, and is displayed in the panel as a block for easy use at any time. When in use, the user expands the block panel to view the callable function he needs, or he can enter some related words in the search box to associate the corresponding callable function. By double-clicking the specified target callable function, the target callable function will be automatically displayed in the use case display area. You can also manually enter the specified callable function in the use case display area to write an automated script use case. At the same time, the embodiment of the present invention also provides detailed remarks and usage instructions for each callable function, and displays them through the JList interface. The technical solution provided by the embodiment of the present invention significantly reduces the development cost of automated script writers, improves the quality of automated scripts, and makes automated testing more stable and efficient.

[0131] Based on the test operation they want to perform, users can click or drag the corresponding test method block into the use case display area to convert it into a callable function. The callable function in the use case display area is written into a test script. However, the callable function only corresponds to the operation, but it is not yet known which element the operation is performed on. At this time, combined with the prompt information of the aforementioned element structure tree, the user uses the data displayed in the element structure tree as test parameters to give the corresponding test method block, that is, fill in the test parameters to the corresponding callable function to inform the test method which interface element to operate. Finally, the arranged test method blocks and test parameters are integrated and converted into a test case. The technical solution provided by the present invention, on the one hand, realizes the graphical programming function, even without programming knowledge, it can write automated use case scripts, lowering the technical threshold requirements for automated testers and improving the efficiency of automated writing. On the other hand, based on the car computer screen projection method, the test screen is projected to the host computer and then the use case is written. Each element parameter of the car computer is also accurately extracted, avoiding the installation of too many third-party test software frameworks on the car computer and reducing the deployment complexity of the use case development environment.

[0132] In some optional implementations, the above step S101 includes:

[0133] Step e1, sending a message to obtain the screen to the vehicle;

[0134] Step e2, receiving the vehicle central control screen projection screen feedback from the vehicle. The vehicle central control screen projection screen is an image sent after the vehicle receives the screen acquisition message, takes a screenshot of the central control screen at a preset frame rate, compresses the screenshot, and then deduplicates the screenshot based on the similarity between adjacent frame screenshots.

[0135] Specifically, the traditional device screen projection solution takes screenshots of the original device at high frequency, and then continuously provides them to the user. This will cause high screen projection delay, slow response, and interface display jamming due to the high quality and large number of pictures. The solution provided by the embodiment of the present invention solves the problem of interface display jamming by capturing screen images, compressing them while acquiring them, and effectively improving the transmission speed. For example, the embodiment of the present invention specifically converts the image into a format with a lower compression ratio (such as compressing it into JPEG format, rather than PNG or BMP format with a higher compression ratio). When saving JPEG pictures, the color depth of the picture can be further reduced from 24-bit color to 8-bit, but it cannot be too low otherwise it will cause picture distortion. Preprocessing the picture before compression, such as noise reduction, sharpening, etc., can also improve the visual effect after compression.

[0136] For screenshots of the vehicle's central control screen, the present invention sends a "Get Screen" message from the host computer to the vehicle. In response, the vehicle calls the Android system's JNI interface to instantiate a SurfaceControl object, then calls the screenshot method therein to implement functions such as background screenshots, full-screen screenshots, and screenshots of designated areas. This screenshot solution does not require user permission and does not display any pop-up windows. It is powerful and simple to call, and the screenshot speed is very fast, taking an average of 20ms per screenshot, which is more than three times faster than the traditional UiDevice screenshot method.

[0137] In addition, by using the OpenCV image comparison algorithm to remove duplicate images and control the frame rate to around 13, the number of redundant images is further reduced, enabling real-time display of the vehicle's image without delay while also meeting basic viewing requirements for projection. The host computer converts the received images into binary blob format files and transmits them to the PC desktop playback application via the Socket network interface for real-time playback, reproducing the actual display effect of the vehicle.

[0138] For example, in a specific embodiment, the embodiment of the present invention uses a structural similarity algorithm to first calculate the brightness mean of two images in the adjacent frames, then calculate the contrast (i.e., standard deviation) of the two images, and then calculate the covariance of the two images (reflecting structural information) based on the image structure. Finally, the similarity between the two images is determined by combining the comparison results of the three indicators of brightness, contrast, and covariance. When the similarity of the three indicators of the two images reaches 100%, the duplicate and redundant images will be removed.

[0139] In addition, in an optional embodiment, the compressed screenshot includes a heavily compressed area and a lightly compressed area. The heavily compressed area is an area where the vehicle compresses unimportant areas in the screenshot (e.g., areas excluding icons) using a first compression ratio, and the lightly compressed area (e.g., areas including icons) is an area where the vehicle compresses important areas in the screenshot using a second compression ratio. The first compression ratio is less than the second compression ratio, for example, the first compression ratio is 50% and the second compression ratio is 90%, indicating that the data volume of the area compressed using the first compression ratio is smaller. The partition compression scheme provided by the embodiment of the present invention improves transmission speed while effectively maintaining image quality.

[0140] In some optional embodiments, the vehicle central control testing method provided by the present invention further includes:

[0141] Step f1, sending an audio acquisition message to the vehicle;

[0142] Step f2: Receive vehicle audio feedback from the vehicle through the Android Test Bridge. The vehicle creates an audio recording object after responding to the audio message, reads the vehicle's background audio data through the audio recording object, and then compresses the read audio data into a playable format file and sends it to the host computer through the Android Test Bridge.

[0143] In step f3, the vehicle audio is transmitted to the audio playback application through local inter-process communication for playback.

[0144] Specifically, using the aforementioned technical means, the host computer also sends audio messages to the vehicle, prompting the vehicle to respond. Using the Android Debug Bridge, the vehicle audio is transmitted from the backend to the host computer, which then plays the vehicle's central control audio. This allows the accuracy of certain tests to be further verified using the vehicle audio. Furthermore, the vehicle audio stream is transmitted directly from the backend, resulting in higher audio quality and lower barriers to entry compared to audio recording methods such as external microphones.

[0145] In a specific embodiment, Figure 5 As shown in the figure, when the vehicle receives an audio message, the specific implementation process is as follows: First, an AudioRecord object (audio recording object) is created on the vehicle side to record the audio. AudioRecord is one of the recording APIs provided by Android, which is used to read device audio. Next, a buffer is created on the vehicle side to store subsequent audio data. A background Service is created to loop the read method of AudioRecord to read audio data from the AudioRecord object and load the read audio stream into the buffer until the audio disappears. The Service then calls the stop method of AudioRecord to stop recording, and finally calls the release method of the AudioRecord object to release other related resources.

[0146] Next, the Service uses the MediaCodec class (a component for audio encoding) provided by the Android system to compress the audio stream into AAC (Advanced Audio Codec) format in real time. Then, using commands from the Android Test Bridge, the vehicle system port is redirected to the host computer, sending the vehicle audio to the host computer. After receiving the vehicle audio, the host computer transmits the audio stream in real time to the PC client front-end application via local socket communication (a local inter-process communication mechanism). The front-end application uses the play method of the MediaPlayer class to parse and play the audio, enabling real-time playback of the vehicle system audio on the host computer.

[0147] In some optional implementations, the above step S102 includes:

[0148] Step h1, execute the initialization phase of the test case, which is the information management phase before the test case is executed;

[0149] Step h2: If no exception occurs in the initialization phase, the pre-phase is executed, which includes the pre-steps required before the test case is executed;

[0150] Step h3: If no exception occurs in the pre-stage, the main stage is executed, which includes the test method of the test case;

[0151] Step h4: When an exception occurs during the execution of the initialization phase, the pre-phase, or the main phase, the execution jumps to the post-phase, which includes instructions for causing the test case execution process to terminate normally.

[0152] Specifically, the initialization phase manages information before test case execution. For example, pre-testing requires collecting information about test case attributes, the test environment, and test hardware, so that the test report can be reviewed later to understand the overall test process and test scenarios. The pre-test phase defines the necessary pre-test steps, such as powering on the machine and checking component status. The main phase displays the created test case, a finished version that can be executed directly. The post-test phase primarily displays user-edited fallback code. This means that regardless of any issues during the test, the post-test phase automatically jumps to the post-test phase, where the fallback code executes to terminate the test case execution normally. This can be used as a fallback for any unexpected situations. Therefore, any issues encountered during the test automatically jump to the post-test phase, where the fallback code executes to terminate the test case execution normally (for example, the fallback code could terminate all processes and shut down the system). This significantly improves test reliability and resolves the issue of test failure.

[0153] In order to facilitate the understanding of the overall solution, Figure 6 As shown in the figure, from creating a use case to executing a use case, a complete test process is:

[0154] 1. Preparation stage:

[0155] 1. Set the car computer to Android debug bridge mode.

[0156] 2. Connect the car cable to the host computer.

[0157] 3. The host computer refreshes the device list to display the currently connected vehicle computer.

[0158] 4. Click the projection button on the host computer to display the current interface content of the vehicle computer in real time.

[0159] 5. The user operates the car computer on the host computer and clicks on audio play. At this time, the computer will play the sound of the car computer.

[0160] 6. When the user views the element structure tree panel on the host computer, the element content of the current vehicle interface will be displayed in real time.

[0161] 7. When the user clicks on the car interface, the element structure tree will automatically refresh to the current latest element content.

[0162] 8. When the user clicks a certain position on the vehicle's central control screen, the element structure tree area will display the element attribute values ​​of the current area in real time.

[0163] 2. Use case creation phase:

[0164] 9. Create a new test case project.

[0165] 10. Record test cases based on user operations.

[0166] 3. Use case execution phase:

[0167] 11. After recording the test case, execute the test case.

[0168] 12. When operating an element, first match it through the element attributes, and then perform the operation if the match is successful.

[0169] 13. If the element attribute matching fails and the target is lost, the large model is called to analyze the meaning of the element icon and the icon meaning is matched. If the match is successful, the operation is executed.

[0170] 14. If the icon meaning matching fails, continue to call the large model to analyze the element text meaning, use the text meaning matching, and execute the operation if the match is successful.

[0171] 15. If all matches fail, it means that the use case matching failed and the test case needs to be regenerated.

[0172] This embodiment also provides a vehicle central control test device, which is used to implement the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0173] This embodiment provides a vehicle central control test device, such as Figure 7 As shown, including:

[0174] The screen projection module 601 is used to obtain the vehicle central control screen projection screen. The vehicle central control screen projection screen is the screen corresponding to the vehicle central control screen and projected on the host computer. The vehicle central control screen is the screen displayed on the vehicle central control;

[0175] The test case execution module 602 is used to obtain and execute the test case to output the test instruction to the vehicle so that the vehicle executes the test operation included in the test case in the vehicle central control screen according to the test instruction;

[0176] The target loss module 603 is used to receive a test loss message fed back by the vehicle. The test loss message is a message generated when the vehicle cannot find the first target element corresponding to the operation in the vehicle central control screen when performing a test operation;

[0177] a lost target analysis module 604 for, in response to the test lost message, calling the large model to extract the element meaning of the first target element;

[0178] A lost object detection module 605 is configured to locate a second target element corresponding to the element's meaning in the vehicle's central control screen using the large model, and to modify the test case based on the second target element, where the second target element is an image element obtained by adjusting the first target element in the vehicle's central control screen;

[0179] The fallback test module 606 is used to continue testing through the corrected test case.

[0180] In an optional embodiment, the test case is created through the following steps: detecting the user's current operation on the vehicle's central control screen; recording the element attribute information of the area corresponding to the current operation, and capturing the area cut-out of the area corresponding to the current operation; extracting the text information in the area cut-out; generating the current test instruction based on the current operation and element attribute information; saving the current test instruction, area cut-out and text information as the recording information of the current operation; arranging the recording information of each step of the user's operation in sequence to obtain a test case.

[0181] In an optional embodiment, the host computer and the vehicle are connected through an Android test bridge, and the element attribute information is obtained through the following steps: creating an element structure tree; sending a get user interface layout command to the vehicle at a preset period; sending a get interface file command to the vehicle at a preset period; receiving an interface file fed back by the vehicle, the interface file is the interface element layout information extracted from the vehicle central control screen each time the vehicle responds to the get user interface layout command, and then saving the interface element layout information as an interface file, and then sending the interface file to the host computer in response to the get interface file command, the vehicle central control screen is the screen corresponding to the vehicle central control projection screen and displayed on the vehicle central control; parsing the interface file to obtain the interface element layout information; filling the interface element layout information into the element structure tree for display; searching for element attribute information from the element structure tree according to the position of the area corresponding to the current operation.

[0182] In an optional embodiment, the lost target analysis module 604 includes:

[0183] A graphic meaning analysis unit is used to analyze the icons in the regional cut map using a large model to obtain the first element meaning;

[0184] The text meaning analysis unit is used to analyze the text information through the large model to obtain the second element meaning.

[0185] In an optional embodiment, the lost target detection module 605 includes:

[0186] A similar icon search unit is used to search for a target icon that matches the meaning of the first element in the vehicle's central control screen through a large model;

[0187] A second element locating unit is configured to, when a target icon is found through the search for the first element meaning, record the target element attribute information corresponding to the target icon, and use the target element attribute information and the target icon as a second target element;

[0188] A test instruction modification unit, used for modifying the test instruction in the test case through the target element attribute information;

[0189] An icon replacement unit, used to replace the corresponding area cut image with the target icon;

[0190] A similar text search unit is used to search for a target text that matches the meaning of the second element in the vehicle's central control screen using a large model when the target icon cannot be found based on the meaning of the first element;

[0191] Returning to the second element positioning unit, it is used to determine the target icon based on the area where the target text is located, and return to the step of recording the target element attribute information corresponding to the target icon.

[0192] In some optional implementations, the screen projection module 601 includes:

[0193] A picture acquisition instruction unit is used to send a picture acquisition message to the vehicle;

[0194] The compression and deduplication screen projection unit is used to receive the vehicle's central control screen projection image feedback from the vehicle. The vehicle's central control screen projection image is an image that the vehicle takes a screenshot of the central control screen at a preset frame rate after receiving the screen acquisition message, compresses the screenshot, and then deduplicates the screenshot based on the similarity between adjacent frame screenshots and sends it.

[0195] In some optional embodiments, the compressed screenshot includes a heavily compressed area and a lightly compressed area. The heavily compressed area is an area where the vehicle compresses an unimportant area in the screenshot using a first compression rate. The lightly compressed area is an area where the vehicle compresses an important area in the screenshot using a second compression rate. The first compression rate is less than the second compression rate.

[0196] In some optional embodiments, the device further comprises:

[0197] An audio acquisition message unit, used for sending an audio acquisition message to the vehicle;

[0198] The audio receiving unit is used to receive vehicle audio feedback from the vehicle through the Android test bridge. The vehicle audio is created by the audio recording object after responding to the audio message, and the vehicle background audio data is read through the audio recording object. The read audio data is then compressed into a playable format file and sent to the host computer through the Android test bridge;

[0199] The playback unit is used to transmit the vehicle audio to the audio playback application through local inter-process communication for playback.

[0200] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0201] See also Figure 8 , Figure 8 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 8As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0202] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0203] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0204] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0205] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0206] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0207] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0208] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0209] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A vehicle central control testing method, characterized in that: Applied to a host computer, the host computer establishes a communication connection with the vehicle, and the method includes: Obtain the vehicle's central control screen projection screen, where the vehicle's central control screen projection screen corresponds to the vehicle's central control screen and is projected onto the host computer. The vehicle's central control screen is the screen displayed on the vehicle's central control; Acquire and execute test cases to output test instructions to the vehicle, so that the vehicle executes the test operations included in the test case in the vehicle's central control screen according to the test instructions; the test case is created by the following steps: detecting the user's current operation on the vehicle's central control screen; recording the element attribute information of the area corresponding to the current operation, and intercepting the area cut image of the area corresponding to the current operation; extracting text information in the area cut image; generating a current test instruction based on the current operation and the element attribute information; saving the current test instruction, the area cut image and the text information as the recording information of the current operation; arranging the recording information of each step of the user's operation in sequence to obtain the test case; receiving a test loss message fed back by the vehicle, wherein the test loss message is a message generated when the vehicle cannot find a first target element corresponding to the operation in the vehicle central control screen when performing the test operation; In response to the test loss message, calling the big model to extract the element meaning of the first target element; in response to the test loss message, calling the big model to extract the element meaning of the first target element includes: analyzing the icon in the area cut image by the big model to obtain the first element meaning; analyzing the text information by the big model to obtain the second element meaning; the first element meaning and the second element meaning both include the function of the first target element; Search the vehicle central control screen projection screen for a target icon that matches the meaning of the first element through the large model; when the target icon is found through the search of the first element meaning, record the target element attribute information corresponding to the target icon, and use the target element attribute information and the target icon as the second target element, which is a screen element obtained by adjusting the first target element in the vehicle central control screen; modify the test instructions in the test case through the target element attribute information; use the target icon to replace the corresponding area cut-out; when the target icon cannot be found through the meaning of the first element, search the vehicle central control screen projection screen for a target text that matches the meaning of the second element through the large model; determine the target icon based on the area where the target text is located, and return to the step of recording the target element attribute information corresponding to the target icon; Continue testing with the corrected test case.

2. The method according to claim 1, characterized in that The host computer and the vehicle are connected via an Android test bridge, and the recording of element attribute information of the area corresponding to the current operation includes: Create an element structure tree; Sending a command to obtain a user interface layout to the vehicle at a preset period; Sending an interface file acquisition command to the vehicle at a preset period; receiving an interface file fed back by the vehicle, the interface file being a file sent to a host computer by the vehicle each time in response to the obtain interface file command, the interface file being generated by the vehicle based on the obtain user interface layout command, the interface file storing interface element layout information, the interface element layout information being extracted from a vehicle central control screen and used to represent information about screen elements in the vehicle central control screen; Parsing the interface file to obtain the interface element layout information; Fill the interface element layout information into the element structure tree for display; The element attribute information is searched from the element structure tree according to the position of the area corresponding to the current operation.

3. The method according to claim 1, characterized in that The method of obtaining the vehicle's central control screen projection image includes: Send a get screen message to the vehicle; The vehicle central control screen projection screen fed back by the vehicle is received. The vehicle central control screen projection screen is an image that is sent after the vehicle receives the screen acquisition message, takes a screenshot of the central control screen at a preset frame rate, compresses the screenshot, and then deduplicates the screenshot based on the similarity between adjacent frame screenshots.

4. The method according to claim 3, characterized in that The compressed screenshot includes a heavily compressed area and a lightly compressed area. The heavily compressed area is an area where the vehicle compresses an unimportant area in the screenshot using a first compression ratio. The lightly compressed area is an area where the vehicle compresses an important area in the screenshot using a second compression ratio. The first compression ratio is less than the second compression ratio.

5. The method according to claim 1, wherein The method further comprises: Send get audio message to vehicle; Receive vehicle audio feedback from the vehicle through the Android test bridge. The vehicle audio is generated by creating an audio recording object after the vehicle responds to the audio message, reading the vehicle background audio data through the audio recording object, and then compressing the read audio data into a playable format file and sending it to the host computer through the Android test bridge; The vehicle audio is transmitted to the audio playback application through local inter-process communication for playback.

6. A vehicle central control testing device, characterized in that: Applied to a host computer, the device includes: The screen projection module is used to obtain the vehicle central control screen projection screen, the vehicle central control screen projection screen is the screen corresponding to the vehicle central control screen, and is projected on the host computer. The vehicle central control screen is the screen displayed on the vehicle central control; A use case execution module is used to obtain and execute test cases to output test instructions to the vehicle, so that the vehicle executes the test operations included in the test case in the vehicle's central control screen according to the test instructions; the test case is created by the following steps: detecting the user's current operation on the vehicle's central control screen; recording the element attribute information of the area corresponding to the current operation, and intercepting the area cut image of the area corresponding to the current operation; extracting the text information in the area cut image; generating the current test instruction based on the current operation and the element attribute information; saving the current test instruction, the area cut image and the text information as the recording information of the current operation; arranging the recording information of each step of the user's operation in sequence to obtain the test case; a target loss module, configured to receive a test loss message fed back by a vehicle, wherein the test loss message is a message generated when the vehicle cannot find a first target element corresponding to the operation in the vehicle central control screen when performing the test operation; A lost target analysis module is configured to, in response to the test loss message, call the large model to extract the element meaning of the first target element; the calling of the large model to extract the element meaning of the first target element in response to the test loss message includes: analyzing the icon in the area cut image using the large model to obtain the first element meaning; analyzing the text information using the large model to obtain the second element meaning; the first element meaning and the second element meaning both include the function of the first target element; A lost target detection module is used to search for a target icon that matches the meaning of the first element in the vehicle's central control screen through the large model; when the target icon is searched through the first element meaning, record the target element attribute information corresponding to the target icon, and use the target element attribute information and the target icon as the second target element, which is a screen element obtained by adjusting the first target element in the vehicle's central control screen; modify the test instructions in the test case through the target element attribute information; use the target icon to replace the corresponding area cut image; when the target icon cannot be searched through the first element meaning, search for a target text that matches the meaning of the second element in the vehicle's central control screen through the large model; determine the target icon based on the area where the target text is located, and return to the step of recording the target element attribute information corresponding to the target icon; The fallback test module is used to continue testing through the corrected test cases.

7. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 5 by executing the computer instructions.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 5.

9. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method according to any one of claims 1 to 5.

Images