Automated testing method and system

Abstract

The present invention provides an automated testing method, which is suitable for automated testing of a vehicle's device interface under the OMA-DM protocol; step S1, a session is established between a server and a device end; step S2, the device end obtains a corresponding test case from the server, the test case including a corresponding original file name to be read; step S3, the device end reads the corresponding original file locally according to the original file name; step S4, the device end replaces local data with test data and forms request data; step S5, the device end uploads the request data to the server and obtains feedback results until the session ends; step S6, the device end compares the feedback results with the expected results to form a comparison result. Beneficial effects: It can support the application of the OMA-DM protocol in the interface communication protocol test in the OTA field.

Description

Technical Field

[0001] The present invention relates to the field of automated testing, and in particular to an automated testing method and system. Background Art

[0002] In the Internet industry, there are already tools that support interface testing protocols. They support the testing of one or more interface protocols to varying degrees and support online maintenance of test case scripts and the generation of online test reports.

[0003] Based on the existing interface testing tools, in order to be better applied to the interface communication protocol testing in the OTA field, the existing tools or platforms still have the following shortcomings: 1. Incompatible with the test of the OMA-DM protocol 2. No support for online log analysis of the test report results and location of problem codes. The existence of these reasons will directly lead to the inability to effectively test the vehicle-cloud communication interface protocol layer in the OTA field, and insufficient test verification. In the integration test phase, the quality of the product cannot be adequately guaranteed. It will directly lead to more underlying problems being exposed in the project system testing phase, and the worst result will lead to project delays and corporate losses. Summary of the invention

[0004] In order to solve the above problems, the technical solution of the present invention provides an automated testing method and system, the specific solution is as follows:

[0005] The present invention includes an automated testing method, which is applicable to automated testing of a vehicle device interface under the OMA-DM protocol; a plurality of device terminals inside the vehicle are respectively connected to an external testing server through the device interface; in the automated testing method, the process of automated testing of the device interface during a single session includes:

[0006] Step S1, establishing a session based on the OMA-DM protocol between the server and the device end that needs to be tested;

[0007] Step S2, the device end obtains the corresponding test case, the test data included in the test case and the name of the original file corresponding to the test data and needs to be read from the server according to the device interface to be tested as needed;

[0008] Step S3, the device searches and reads the corresponding original file locally according to the original file name;

[0009] Step S4, the device end uses the test data to replace the corresponding local data in the read original file, and forms interface request data of the device interface to be tested according to the replaced original file;

[0010] Step S5, the device uploads the device interface request data to the server, and obtains the corresponding feedback result issued by the server until the session ends;

[0011] Step S6, the device compares the feedback result with the expected result of the test case to form a corresponding comparison result;

[0012] The device side then executes the steps S1-S6 for each session respectively to obtain the corresponding comparison results respectively and integrate them to form a test report corresponding to the device side.

[0013] Preferably, during the automated testing process of the device, a plurality of different sessions have a preset execution order;

[0014] Then in the step S1, when each session is established, the current OMA-DM protocol message is checked to determine whether the current session is executed according to the execution order:

[0015] If yes, continue to execute step S1 to establish a session;

[0016] If not, terminate the session and exit.

[0017] Preferably, in step S1, before establishing each session, the server first authenticates the identity of the device, and then establishes the session after the authentication is passed.

[0018] Preferably, in step S3, the original file read by the device is a file in XML format.

[0019] Preferably, the test data includes a data key value and corresponding data content;

[0020] Then the step S4 specifically includes:

[0021] Step S41, the device locates and obtains the corresponding data node in the original file according to the data key value;

[0022] Step S42, the device uses the data content to replace the local data in the located data node;

[0023] Step S43: the device terminal generates interface request data of the device interface to be tested according to the replaced original file.

[0024] Preferably, in step S5, while the session is maintained, the device uploads the interface request data to the server multiple times to obtain the corresponding feedback result until the server issues a session end instruction to end the session.

[0025] Preferably, after the test report is generated, the device side provides the user with an online query of the log file of the server according to the test report to locate the defect code position associated with the test error in the test report.

[0026] The technical solution of the present invention also includes an automated testing system, which is suitable for automated testing of a vehicle's device interface under the OMA-DM protocol; characterized in that it includes a server and a plurality of device terminals arranged inside the vehicle, and the device terminals respectively establish a remote connection with the server through the device interface;

[0027] Then each of the device ends includes:

[0028] A connection unit, used for establishing a session based on the OMA-DM protocol between the device and the server when the device needs to perform an automated test;

[0029] an acquisition unit, connected to the connection unit, for acquiring, after establishing a session, from the server a corresponding test case, test data included in the test case, and a name of an original file corresponding to the test data and needing to be read, according to a device interface to be tested as needed;

[0030] A search unit, connected to the acquisition unit, for searching and reading the corresponding original file locally according to the original file name;

[0031] A replacement unit, connected to the search unit, for replacing the corresponding local data in the read original file with the test data, and forming interface request data of the device interface to be tested according to the replaced original file;

[0032] A feedback unit, connected to the replacement unit, configured to upload the device interface request data to the server and obtain a corresponding feedback result issued by the server until the session ends;

[0033] A test unit, connected to the feedback unit, for comparing the feedback result with the expected result of the test case to form a corresponding comparison result;

[0034] A report forming unit is connected to the test unit and is used to form a test report corresponding to the device end according to all the comparison results corresponding to all the sessions.

[0035] Preferably, during the automated testing process of the device, a plurality of different sessions have a preset execution order;

[0036] Then the connection unit includes:

[0037] A first verification module, configured to verify the current OMA-DM protocol message when each session is established to determine whether the current session is executed according to the execution order, and output a first determination result;

[0038] then the connection unit continues to establish the session according to the first judgment result when the current session is executed in the execution order; and

[0039] When the current session is not executed according to the execution order, the session is terminated.

[0040] Preferably, the test data includes a data key value and corresponding data content;

[0041] Then the replacement unit comprises:

[0042] A search module, used to locate and obtain a corresponding data node in the original file according to the data key value;

[0043] A replacement module, connected to the search module, configured to replace the local data in the located data node with the data content;

[0044] A forming module is connected to the replacing module and is used to form interface request data of the device interface to be tested according to the replaced original file.

[0045] The beneficial effect of the technical solution of the present invention is that it can support the application of the OMA-DM protocol in the interface communication protocol test in the OTA field. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The figure is a schematic diagram of the overall flow of the testing method in a preferred embodiment of the present invention;

[0047] Figure 2 A flow chart of generating interface request data in a preferred embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the main structure of the test system in a preferred embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of the structure of the connecting unit in a preferred embodiment of the present invention;

[0050] Figure 5This is a schematic diagram of the replacement unit structure in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0053] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0054] The current technical solution is not compatible with the OMA-DM protocol test, nor does it support online analysis of test report results and location of problem codes.

[0055] In view of the above defects existing in the prior art, the present invention provides an automated testing method, which is suitable for automated testing of vehicle device interfaces under the OMA-DM protocol; characterized in that multiple device terminals inside the vehicle are respectively connected to an external test server through device interfaces;

[0056] In automated testing methods, during a single session, Figure 1 As shown in the figure, the process of automated testing of device interfaces includes:

[0057] Step S1, establishing a session based on the OMA-DM protocol between the server and the device to be tested;

[0058] Step S2, the device end obtains the corresponding test case, the test data included in the test case, and the name of the original file corresponding to the test data that needs to be read from the server according to the device interface to be tested as needed;

[0059] Step S3, the device searches and reads the corresponding original file locally according to the original file name;

[0060] Step S4, the device side replaces the corresponding local data in the read original file with the test data, and forms the interface request data of the device interface to be tested according to the replaced original file;

[0061] Step S5, the device uploads the device interface request data to the server, and obtains the corresponding feedback result issued by the server until the session ends;

[0062] Step S6, the device compares the feedback result with the expected result of the test case to form a corresponding comparison result;

[0063] The device side then executes steps S1-S6 for each session respectively to obtain the corresponding comparison results and integrate them to form a test report corresponding to the device side.

[0064] Specifically, in this embodiment, the OMA-DM protocol is a protocol for managing terminals in the network through a remote server. The server can update the firmware of the terminal device through the OMA-DM protocol. In the above protocol, the server is the controller and the device is the controlled party. The server and the device adopt a C / S structure and interact in units of messages.

[0065] In a preferred embodiment of the present invention, during the automated testing process at the device end, a plurality of different sessions have a preset execution order;

[0066] Then in step S1, when each session is established, the current OMA-DM protocol message is checked to determine whether the current session is executed in the execution order:

[0067] If yes, continue to execute step S1 to establish a session;

[0068] If not, terminate the session and exit.

[0069] Specifically, in this embodiment, the server and the device side confirm identity information. After confirming the information, the device side and the server exchange information through the protocol message. The protocol message includes multiple node information, and the node information is exchanged with the server.

[0070] In a preferred embodiment of the present invention, in step S1, before establishing each session, the server first authenticates the device end, and then establishes the session after the authentication is passed.

[0071] Specifically, in this embodiment, if the authentication fails during the execution of the identity authentication, the identity authentication is performed again, and the test can be performed after passing.

[0072] In a preferred embodiment of the present invention, in step S3, the original file read by the device is a file in XML format.

[0073] Specifically, in this embodiment, it is possible to easily load XML data into the device and read it, and output the result in XML format to achieve the test effect.

[0074] In a preferred embodiment of the present invention, the test data includes a data key value and corresponding data content;

[0075] Then step S4 specifically includes, Figure 2 As shown:

[0076] Step S41, the device locates the corresponding data node in the original file according to the data key value;

[0077] Step S42, the device uses the data content to replace the local data in the located data node;

[0078] Step S43: the device side generates interface request data of the device interface to be tested according to the replaced original file.

[0079] Specifically, in this embodiment, by marking the data key value, it can be ensured that no marking error occurs when the above data nodes are changed. This is determined by the characteristics of the key value, and the key value contains several data types to adapt to the usage requirements of different environments.

[0080] In a preferred embodiment of the present invention, in step S5, while the session is being maintained, the device uploads interface request data to the server multiple times to obtain corresponding feedback results until the server issues a session end instruction to end the session.

[0081] Specifically, in this embodiment, due to the limitation of transmission rate, the above-mentioned device end needs to request data from the above-mentioned server upload interface multiple times, and the server also sends data multiple times. In order to avoid the content of different sessions being sent incorrectly, during the same session, the server will send an end instruction. The above-mentioned end instruction marks the end of this session and also marks the beginning of the next session.

[0082] In a preferred embodiment of the present invention, after a test report is generated, the device provides the user with a method for online querying of a server's log file based on the test report to locate the defect code position associated with the test error in the test report.

[0083] Specifically, in this embodiment, the device side connects to the address of the above-mentioned server and the corresponding interface, and can specifically obtain the server's log file online, and obtain the corresponding real-time message information through the GREP pipeline, for example, by grabbing the wrong keywords, and storing the error information after extracting it to the background, and finally displaying the location of the above-mentioned defect code in the form of the above-mentioned test report.

[0084] The technical solution of the present invention also provides an automated testing system, which is suitable for automated testing of vehicle device interfaces under the OMA-DM protocol; it is characterized in that it includes a server 1 and a plurality of device terminals 3 arranged inside the vehicle, and the device terminals 3 respectively establish remote connections with the server 1 through the device interfaces 2;

[0085] Each device end includes, for example Figure 3 As shown:

[0086] A connection unit 31 is used to establish a session based on the OMA-DM protocol between the device and the server when the device needs to perform an automated test;

[0087] The acquisition unit 32 is connected to the connection unit 31 and is used to acquire the corresponding test case, the test data included in the test case and the name of the original file corresponding to the test data and needing to be read from the server according to the device interface to be tested as needed after the session is established;

[0088] The search unit 33 is connected to the acquisition unit 32 and is used to search and read the corresponding original file locally according to the original file name;

[0089] The replacement unit 34 is connected to the search unit 33 and is used to replace the corresponding local data in the read original file with the test data, and form the interface request data of the device interface to be tested according to the replaced original file;

[0090] The feedback unit 35 is connected to the replacement unit 34 and is used to upload the device interface request data to the server and obtain the corresponding feedback result issued by the server until the session ends;

[0091] The testing unit 36 ​​is connected to the feedback unit 35 and is used to compare the feedback result with the expected result of the test case to form a corresponding comparison result;

[0092] The report forming unit 37 is connected to the testing unit 36 ​​and is used to form a test report corresponding to the device end according to all the comparison results corresponding to all the sessions.

[0093] Specifically, in this embodiment, the report forming unit can use the association of the built-in error reporting Log script code method to clearly locate the specific error type, such as a test script error, or a use case execution result return error;

[0094] Furthermore, if an error is returned, the real-time Log of the corresponding interface server is obtained according to the currently tested interface, the error result in the Log is obtained, and the specific cause of the defect is located.

[0095] In a preferred embodiment of the present invention, during the automated testing process at the device end, a plurality of different sessions have a preset execution order, such as Figure 4 As shown;

[0096] Then the connection unit 31 includes:

[0097] The first verification module 311 is used to verify the current OMA-DM protocol message when each session is established to determine whether the current session is executed according to the execution order, and output a first judgment result;

[0098] Then the connection unit 31 continues to establish the session when the current session is executed according to the execution order according to the first judgment result; and

[0099] Terminates the session when the current session is not executed in the execution order.

[0100] Specifically, the present embodiment includes multiple session processes, each of which has a certain order association, and the order between sessions cannot be changed. If an error in the order occurs, such as inversion or misalignment, it is necessary to verify the correctness of the message and terminate the session in time.

[0101] In a preferred embodiment of the present invention, the test data includes a data key value and corresponding data content, such as Figure 5 As shown;

[0102] Then the replacement unit 34 comprises:

[0103] A search module 341 is used to locate the corresponding data node in the original file according to the data key value;

[0104] The replacement module 342 is connected to the search module 341 and is used to replace the local data in the located data node using the data content;

[0105] The forming module 343 is connected to the replacing module 342 and is used to form the interface request data of the device interface to be tested according to the replaced original file.

[0106] Specifically, in this embodiment, the replacement module exchanges information of the data node with the server. For example, the device end in the vehicle sends ECU node information, VIN information and detection instructions, the server matches the task information corresponding to the VIN, and sends the download address information of the firmware update package. The device end downloads and verifies the upgrade package, and after the download is completed, the download result is reported to the server. After the server receives the information, the server sends an update instruction, and the client performs a firmware update. After the update is completed, the upgrade result is reported to the server.

[0107] The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. An automated testing method, suitable for automated testing of vehicle device interfaces under the OMA-DM protocol; characterized in that: The multiple device terminals inside the vehicle are respectively connected to the external test server through the device interface; In the automated testing method, the process of performing automated testing on the device interface during a single session includes: Step S1, establishing a session based on the OMA-DM protocol between the test server and the device end that needs to be tested; Step S2, the device end obtains the corresponding test case, the test data included in the test case and the name of the original file corresponding to the test data and needs to be read from the test server according to the device interface to be tested as needed; Step S3, the device searches and reads the corresponding original file locally according to the original file name; Step S4, the device end uses the test data to replace the corresponding local data in the read original file, and forms interface request data of the device interface to be tested according to the replaced original file; Step S5, the device end uploads the device interface request data to the test server, and obtains the corresponding feedback result issued by the test server until the session ends; Step S6, the device compares the feedback result with the expected result of the test case to form a corresponding comparison result; Then the device side executes the steps S1-S6 for each session respectively, so as to obtain the corresponding comparison results respectively and integrate them to form a test report corresponding to the device side; The test data includes a data key value and corresponding data content; Then the step S4 specifically includes: Step S41, the device locates and obtains the corresponding data node in the original file according to the data key value; Step S42, the device uses the data content to replace the local data in the located data node; Step S43: the device terminal generates interface request data of the device interface to be tested according to the replaced original file.

2. The automated testing method according to claim 1, wherein: During the automated testing process on the device side, a plurality of different sessions have a preset execution order; Then in the step S1, when each session is established, the current OMA-DM protocol message is checked to determine whether the current session is executed according to the execution order: If yes, continue to execute step S1 to establish a session; If not, terminate the session and exit.

3. The automated testing method according to claim 1, wherein: In the step S1, before establishing each session, the test server first authenticates the device end, and then establishes the session after the authentication is passed.

4. The automated testing method according to claim 1, wherein: In step S3, the original file read by the device is a file in XML format.

5. The automated testing method according to claim 1, wherein: In the step S5, while the session is maintained, the device uploads the interface request data to the test server multiple times to obtain the corresponding feedback results, until the test server issues a session end instruction to end the session.

6. The automated testing method according to claim 1, wherein: After the test report is generated, the device side provides the user with an online query of the log file of the test server according to the test report to locate the defect code position associated with the test error in the test report.

7. An automated testing system, suitable for automated testing of vehicle device interfaces under the OMA-DM protocol; characterized in that: It includes a test server and a plurality of device terminals arranged inside the vehicle, wherein the device terminals respectively establish a remote connection with the test server through a device interface; Then each of the device ends includes: A connection unit, used for establishing a session based on the OMA-DM protocol between the device end and the test server when the device end needs to perform an automated test; an acquisition unit, connected to the connection unit, for acquiring, after establishing a session, from the test server the corresponding test case, the test data included in the test case, and the name of the original file corresponding to the test data and needing to be read, according to the device interface to be tested as needed; A search unit, connected to the acquisition unit, for searching and reading the corresponding original file locally according to the original file name; A replacement unit, connected to the search unit, for replacing the corresponding local data in the read original file with the test data, and forming interface request data of the device interface to be tested according to the replaced original file; A feedback unit, connected to the replacement unit, configured to upload the device interface request data to the test server and obtain a corresponding feedback result issued by the test server until the session ends; A test unit, connected to the feedback unit, for comparing the feedback result with the expected result of the test case to form a corresponding comparison result; A report forming unit, connected to the test unit, for forming a test report corresponding to the device end according to all the comparison results corresponding to all the sessions; The test data includes a data key value and corresponding data content; Then the replacement unit comprises: A search module, used to locate and obtain a corresponding data node in the original file according to the data key value; A replacement module, connected to the search module, configured to replace the local data in the located data node with the data content; A forming module is connected to the replacing module and is used to form interface request data of the device interface to be tested according to the replaced original file.

8. The automated testing system according to claim 7, wherein: During the automated testing process on the device side, a plurality of different sessions have a preset execution order; Then the connection unit includes: A first verification module, configured to verify the current OMA-DM protocol message when each session is established to determine whether the current session is executed according to the execution order, and output a first determination result; then the connection unit continues to establish the session according to the first judgment result when the current session is executed in the execution order; and When the current session is not executed according to the execution order, the session is terminated.

Images