Vehicle-mounted system host testing method, device, equipment and readable storage medium

Abstract

The application provides a vehicle-mounted system host test method, device and equipment and a readable storage medium. After receiving a test instruction for testing a target host, a test task corresponding to the test instruction is determined. After determining the test task, target simulation data corresponding to the test task can be selected from a preset target vehicle-mounted data set. After determining the target simulation data, the target host can be tested according to the target simulation data, and a test result of the target host can be obtained. By using the data collected in real road testing as the target simulation data for implementing the test task and testing the real host state by automatically driving the simulation result of the target simulation data, the problem of detecting or evaluating the host state in automatic driving can be solved, the detection speed of the host state can be improved, the loss of hardware can be reduced, the cost can be saved, and the efficiency of maintaining the automatic driving system can be improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus, device and readable storage medium for testing a vehicle-mounted system host. Background Technology

[0002] With the development of science and technology, autonomous driving technology is also improving rapidly. In autonomous driving, a master vehicle typically operates one or more main units. The lifecycle of a main unit generally follows the bathtub curve of electronic products, meaning that the failure rate is relatively high at the beginning and end of the lifecycle, and relatively low in the middle. However, on the one hand, due to the high complexity of computer systems and the demanding usage scenarios of autonomous driving, the performance of the main unit is more prone to failure even in the middle of the bathtub curve. On the other hand, traditional testing techniques mainly break down the main unit into small modules and test each component to ensure the health of the entire main unit, such as testing the memory and disk separately to infer the overall performance of the main unit. However, this testing approach is not only time-consuming and causes significant wear and tear on the hardware, but also expensive and cannot be used frequently. Summary of the Invention

[0003] This application aims to at least solve one of the aforementioned technical defects. In view of this, this application provides a method, apparatus, device and readable storage medium for testing the host system of an in-vehicle system, in order to solve the technical defect of difficulty in testing the state of an autonomous driving host in the prior art.

[0004] A method for testing the host computer of an in-vehicle system includes:

[0005] Receive a test instruction to test the target host, and determine the test task corresponding to the test instruction;

[0006] Select target simulation data corresponding to the test task from the preset target vehicle data set;

[0007] Based on the target simulation data, the target host is tested to obtain the test results of the target host.

[0008] Preferably, target simulation data corresponding to the test task is selected from a preset target vehicle data set, including:

[0009] Determine the first timestamp, which is the timestamp when the target vehicle performs the predetermined operation corresponding to the test task;

[0010] In the target vehicle data set, select each first target data whose occurrence time is located in a first time interval, where the first time interval is from the moment before the first timestamp to the moment after the first timestamp;

[0011] Each of the first target data is preprocessed to obtain target simulation data corresponding to the test task.

[0012] Preferably, the preprocessing of each of the first target data to obtain target simulation data corresponding to the test task includes:

[0013] For each of the first target data, determine whether the first target data meets the preset reproduction conditions in the preset simulation environment;

[0014] If the first target data meets the reproduction condition, add a data tag to the first target data corresponding to the predetermined operation that occurred on the target vehicle;

[0015] Each first target data with added data labels is used as the target simulation data corresponding to the test task.

[0016] Preferably, if the test task is a performance test, the step of testing the target host based on the target simulation data to obtain the test results of the target host includes:

[0017] Based on the target simulation data, multiple different types of comparative tests are performed on the target host to obtain the comparative test results for each test.

[0018] The confidence levels of each of the comparative test results are combined to obtain the test results of the target host.

[0019] Preferably, if the test task is to test the performance of the target host, the method further includes:

[0020] Based on the test results of the target host, determine whether the overall performance of the target host is abnormal.

[0021] If it is determined that the overall performance of the target host is abnormal, then an abnormality test is performed on the target host;

[0022] If it is determined that the overall performance of the target host is not abnormal, then the target host is determined to be in good condition.

[0023] Preferably, determining whether the overall performance of the target host is abnormal includes:

[0024] The first standard deviation is obtained by comparing the test results of the target host with the performance data of the target host in the target simulation data over the same time period;

[0025] If the first standard deviation is greater than a preset first threshold, it is determined that the overall performance of the target host is abnormal.

[0026] Preferably, determining whether the overall performance of the target host is abnormal includes:

[0027] The test results of the target host are compared with the obtained reference performance data to obtain the second standard deviation;

[0028] If the second standard deviation is greater than the preset second threshold, it is determined that the overall performance of the target host is abnormal.

[0029] Preferably, determining whether each first target data satisfies preset reproduction conditions in a preset simulation environment includes:

[0030] Each of the first target data is run in a preset simulation environment. If the first target data generates a first parameter in the preset simulation environment, then the first target data is determined to meet the preset reproduction conditions in the preset simulation environment. The first parameter is the same as the relevant parameter generated when the target vehicle performs a driving task.

[0031] If the first target data cannot generate the first parameter under the preset simulation environment, then it is determined that the first target data does not meet the preset reproduction conditions.

[0032] A vehicle-mounted system host testing device, comprising:

[0033] The instruction receiving unit is used to receive test instructions for testing the target host and determine the test task corresponding to the test instructions.

[0034] The data acquisition unit is used to select target simulation data corresponding to the test task from a preset target vehicle data set;

[0035] The testing unit is used to test the target host based on the target simulation data and obtain the test results of the target host.

[0036] A vehicle-mounted system host testing device includes: one or more processors, and a memory;

[0037] The memory stores computer-readable instructions, characterized in that, when the computer-readable instructions are executed by the one or more processors, they implement the steps of the vehicle system host testing method described above.

[0038] A readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of any of the vehicle system host testing methods described above.

[0039] As can be seen from the above technical solutions, the embodiments of this application can receive a test instruction to test a target host, determine the test task corresponding to the test instruction, select target simulation data corresponding to the test task from a preset target vehicle data set, and test the target host based on the target simulation data to obtain the test result of the target host.

[0040] The method provided in this application embodiment can use data collected in real road tests as target simulation data to achieve test tasks, and test the real host status by running the target simulation data in autonomous driving simulation. This solves the problem of detecting or evaluating the host status in autonomous driving, greatly improves the detection speed of host status, reduces hardware wear and tear, saves costs, and improves the efficiency of maintaining the autonomous driving system. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating a method for testing a vehicle-mounted system host, provided as an embodiment of this application;

[0043] Figure 2 This is a schematic diagram illustrating the structure of a vehicle-mounted system host testing device, as exemplified by an embodiment of this application.

[0044] Figure 3 This is a hardware structure block diagram of a vehicle-mounted system host testing device disclosed in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] Traditional autonomous driving host system status testing techniques primarily involve breaking down the host system under test into small modules and testing them individually. This ensures the health of each component, thereby guaranteeing the overall health of the host system. For example, testing the memory and disk performance separately helps determine the overall performance of the host system. However, this method is extremely time-consuming: individual performance and integrity testing of each component often takes several hours, and testing all the critical components of the host system can take more than ten hours. Furthermore, this method causes significant wear and tear on the hardware and is costly. Using high-grade hardware, for example, with sufficient hardware redundancy to compensate for potential software problems, would be a better approach.

[0047] Given that most current vehicle system host testing solutions involve long testing times, significant hardware wear and tear, and high testing costs, the applicant has researched a vehicle system host testing solution. This solution uses data collected in real road tests as target simulation data to achieve the testing task, and tests the actual host status by running the target simulation data in autonomous driving simulation. This solves the problem of detecting or evaluating the host status in autonomous driving, greatly improves the detection speed of host status, reduces hardware wear and tear, saves costs, and improves the efficiency of maintaining autonomous driving systems.

[0048] The methods provided in this application can be used in a variety of general-purpose or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.

[0049] This application provides a method for detecting faults in a vehicle system. This method can be applied to the vehicle systems of various vehicles or the airborne systems of flight equipment, and can also be applied to various computer terminals or smart terminals. The executing entity can be the processor or server of the computer terminal or smart terminal.

[0050] The following is combined Figure 1 This application describes the process of testing a vehicle-mounted system host according to an embodiment of the present application. The process may include the following steps:

[0051] Step S101: Receive a test instruction to test the target host and determine the test task corresponding to the test instruction.

[0052] Specifically, generally speaking, in autonomous driving, the main vehicle typically has one or more host systems. As mentioned above, the lifecycle of a host system generally follows the bathtub curve of electronic products, meaning that the failure rate is relatively high at the beginning and end of the lifecycle, and relatively low in the middle. However, due to the high complexity of computer systems and the demanding usage scenarios of autonomous driving, host system performance is more prone to failure even in the middle of the bathtub curve.

[0053] During the autonomous driving process, the master vehicle collects the operating data of each host. The on-board data of the master vehicle reflects the operating status of each host. The method provided in this application embodiment can determine the test task corresponding to the test instruction after receiving the test instruction to test the target host.

[0054] The test instructions may include test tasks or test requirements for the target host.

[0055] Step S102: Select target simulation data corresponding to the test task from the preset target vehicle data set.

[0056] Specifically, as described above, the vehicle's onboard data can provide feedback on the operation of each host. However, the vehicle's onboard data is often very large and quite messy. It may contain data related to the host's performance, as well as data related to the autonomous driving vehicle's journey.

[0057] As can be seen from the above steps, the method provided in this application embodiment can determine the test task corresponding to the test instruction after receiving the test instruction to test the target host.

[0058] Therefore, the method provided in this application embodiment can select target simulation data corresponding to the test task from a preset target vehicle data set, based on the test task. The target simulation data can be data used to test the target host to be tested.

[0059] For example, if the test task is to test the performance status of the target host, data related to the performance of the target host can be selected from a preset target vehicle data set as target simulation data for testing the target host.

[0060] Step S103: Based on the target simulation data, test the target host to obtain the test results of the target host.

[0061] Specifically, as can be seen from the above steps, the method provided in this application embodiment can select target simulation data that can be used to test the target host from a preset target vehicle data set.

[0062] Furthermore, based on the acquired target simulation data and the test task, the state of the target host can be tested. This yields test results regarding the state of the target host.

[0063] For example, after obtaining the test results of the target host, the health status of the target host can be further evaluated based on the test results.

[0064] As can be seen from the above technical solution, the method provided in this application embodiment can use the data collected in real road tests as the target simulation data to achieve the test task, and test the real host status by running the target simulation data in autonomous driving simulation. This solves the problem of detecting or evaluating the host status in autonomous driving, improves the detection speed of the host status, reduces hardware wear and tear, saves costs, and improves the efficiency of maintaining the vehicle system of autonomous driving.

[0065] As described above, the method provided in this application can select target simulated data corresponding to the test task from a preset target vehicle data set. For example, in practical applications, it may be necessary to test the performance status of the target host in order to understand whether the target host is healthy. Therefore, if the test task is to test the performance of the target host, it is necessary to obtain simulated data related to the performance of the target host. The process of selecting target simulated data corresponding to the test task from the preset target vehicle data set can include the following:

[0066] Step S201: Determine the first timestamp, which is the timestamp when the target vehicle performs the predetermined operation corresponding to the test task.

[0067] Specifically, in practical applications, when a vehicle is performing an autonomous driving task, the master vehicle collects operational data from various host units during its journey. The master vehicle's onboard data reflects the operational status of each host unit. When an onboard host unit malfunctions, the onboard system records the relevant operational data when a human takes over and continues driving. For example, the onboard system of a target vehicle performing autonomous driving will save the specific time when the host unit took over.

[0068] Therefore, in order to better determine the target simulation data corresponding to the test task in the preset target vehicle data set, a first timestamp can be determined, wherein the first timestamp can be the timestamp when the target vehicle performs a predetermined operation corresponding to the test task, so that the target simulation data can be determined based on the first timestamp.

[0069] For example, if the test task is to test the performance of the target host, the first timestamp of the target host taking over can be determined. This allows relevant data for analyzing the state of the target host to be extracted from the target vehicle data set based on the first timestamp.

[0070] Step S202: Select each first target data whose occurrence time is located in the first time interval from the target vehicle data set. The first time interval is from the moment before the first timestamp to the moment after the first timestamp.

[0071] Specifically, as described above, the method provided in this application embodiment can determine a first timestamp when the target vehicle performs a predetermined operation corresponding to the test task. Based on the first timestamp, the target simulation data can be selected from the target vehicle data set.

[0072] Therefore, after determining the first timestamp, each first target data whose occurrence time is located in the first time interval can be selected from the target vehicle data set. The first time interval is from the moment before the first timestamp to the moment after the first timestamp.

[0073] For example, once the car returns to the garage, data from the moment before the first timestamp to the moment after the first timestamp can be extracted from the target vehicle data set based on the first timestamp and uploaded to the simulation data processing center.

[0074] Step S203: Preprocess each of the first target data to obtain target simulation data corresponding to the test task.

[0075] Specifically, the target vehicle data set may include all data related to the operation of the target vehicle. If the target simulation data is selected directly from the target vehicle data set, the workload may be relatively large.

[0076] Therefore, each of the first target data is preprocessed to obtain target simulation data corresponding to the test task.

[0077] Therefore, after the data extracted from the target vehicle data set is uploaded to the simulation data processing center, the data related to the takeover of the target host can be categorized and stored in the corresponding archive database of the simulation data processing center. This allows for better management of the data extracted from the target vehicle data set.

[0078] For example, the data related to the takeover of the target host can be categorized, and a storage directory related to the data related to the takeover of the target host can be constructed. The data related to the takeover of the target host can then be stored in the corresponding archive database of the simulation data processing center according to the categorized storage directories.

[0079] As can be seen from the above-described technical solutions, the method provided in this application embodiment can select target simulation data corresponding to the test task from a preset target vehicle data set. This allows the data collected in real road tests to be used as the target simulation data for achieving the test task. The actual host status can be tested by running the target simulation data in autonomous driving simulation. This solves the problem of detecting or evaluating the host status in autonomous driving, improves the detection speed of the host status, reduces hardware wear and tear, saves costs, and improves the efficiency of maintaining the vehicle system for autonomous driving.

[0080] As described above, the method provided in this application embodiment can preprocess each of the first target data to obtain target simulation data corresponding to the test task. The process will now be described, and it may include the following steps:

[0081] Step S301: For each of the first target data, determine whether the first target data meets the preset reproduction conditions in the preset simulation environment.

[0082] Specifically, after obtaining each of the first target data, it can be determined whether each of the first target data is data that meets the requirements. Therefore, it can be further determined whether the first target data meets the preset reproduction conditions in the preset simulation environment. If the first target data meets the reproduction conditions, it indicates that the first target data is data that meets the requirements, and thus step S302 can be executed.

[0083] Step S302: Add a data tag to the first target data corresponding to the predetermined operation that occurred on the target vehicle.

[0084] Specifically, as described in the above steps, if the data generated by the takeover of the vehicle where the target host is located can be reproduced in a simulation, although the first target data can be directly used to test the target host, in order to ensure that there is no missing or erroneous data in the intercepted data, the relevant data marked as reproducible in the simulation can be further corrected and confirmed in the data of the target host takeover, and then the first target data can be marked with different data labels according to the different data categories of the first target data.

[0085] The different data tags can be performance-related tags, CPU-related tags, memory-related tags, GPU-related tags, video memory-related tags, and I / O-related tags.

[0086] Step S303: Each first target data with added data tags is used as the target simulation data corresponding to the test task.

[0087] Specifically, as described above, the takeover-related data of the target host can be labeled with different tags according to the different data categories of the data on the takeover of the vehicle where the target host is located. Furthermore, each first target data with added data tags can be used as the target simulation data corresponding to the test task.

[0088] As can be seen from the technical solutions described above, the method provided in this application embodiment can preprocess each of the first target data to obtain target simulation data corresponding to the test task. This allows the data collected in real road tests to be used as target simulation data to achieve the test task, and the real host status can be tested by running the target simulation data in autonomous driving simulation. This solves the problem of detecting or evaluating the host status in autonomous driving, improves the detection speed of the host status, reduces hardware wear and tear, saves costs, and improves the efficiency of maintaining the vehicle system for autonomous driving.

[0089] As described above, the method provided in this application embodiment can test the target host based on the target simulation data to obtain the test results of the target host. In practical applications, if the test task type is determined to be performance testing according to the test instruction, the implementation method of testing the target host based on the target simulation data to obtain the test results of the target host will be different. The following describes the implementation method of testing the target host based on the target simulation data to obtain the test results of the target host when the test task type is performance testing, which may include the following steps:

[0090] Step S401: Based on the target simulation data, perform multiple different types of comparative tests on the target host to obtain the comparative test results for each comparative test.

[0091] Specifically, after obtaining the target simulation data, the target simulation data can be further re-run on the target host based on the target simulation data and the test task, thereby testing the status of the target host and obtaining the test results of the target host.

[0092] It is important to note that the same simulated dataset may be running simultaneously on other hosts undergoing performance testing and on certain standard hosts. When running these simulated datasets on the target host, it is also necessary to collect performance-related data for that host while running the simulated datasets.

[0093] Therefore, after obtaining the target simulation data, multiple different types of comparative tests can be performed on the target host based on the target simulation data to obtain the comparative test results under each comparative test.

[0094] For example, the comparative testing method may include the following, and any one or more of the following testing methods can be selected to conduct comparative testing on the target host.

[0095] First testing method:

[0096] The test results of the target host are compared with the performance data of the corresponding time period of the simulated data to obtain the first standard deviation.

[0097] Specifically, after obtaining the test results of the target host, the test results of the target host can be compared with the performance data of the simulation data for the corresponding time period to obtain the first standard deviation, which can be used to evaluate the status of the target host.

[0098] The second testing method:

[0099] The second standard deviation is obtained by comparing the test results of the target host with the performance data of the target host when running the same simulated dataset to perform autonomous driving tasks.

[0100] Specifically, after obtaining the test results of the target host, the test results of the target host can be compared with the performance data of the target host when running the same simulated dataset to perform autonomous driving tasks, and a second standard deviation can be obtained so as to evaluate the state of the target host.

[0101] The third testing method:

[0102] The third standard deviation is obtained by comparing the test results of the target host with the performance data of the standard host running the same simulated dataset.

[0103] Specifically, after obtaining the test results of the target host, the test results of the target host can be compared with the performance data of a standard host running the same simulated dataset to obtain the third standard deviation, which can be used to evaluate the status of the target host.

[0104] The fourth testing method:

[0105] The fourth standard deviation is obtained by comparing the test results of the target host with the performance data of other hosts running the same simulated dataset.

[0106] Specifically, after obtaining the test results of the target host, the test results of the target host can be compared with the performance data of other hosts running the same simulated dataset to obtain the fourth standard deviation, which can be used to evaluate the status of the target host.

[0107] Step S402: Combine the confidence levels of each of the comparative test results to obtain the test results of the target host.

[0108] Specifically, as described above, the method provided in this application embodiment can perform multiple different types of comparative tests on the target host based on the target simulation data, obtain the comparative test results under each comparative test, and after obtaining the comparative test results under each comparative test, combine the confidence levels of each comparative test result to obtain the test result of the target host. This allows it to determine whether the overall performance of the target host is abnormal. If the overall performance of the target host is abnormal, further testing of the target host is required.

[0109] For example, after obtaining the first standard deviation, the second standard deviation, the third standard deviation, and the fourth standard deviation, the first standard deviation, the second standard deviation, the third standard deviation, and the fourth standard deviation can be weighted to generate a confidence result.

[0110] As can be seen from the technical solutions described above, the method provided in this application embodiment can test the target host based on the target simulation data, obtain the test results of the target host, and comprehensively compare the performance data of the target host under test with various other data through multiple comparative tests, so as to evaluate the status of the target host. This can effectively improve the detection speed of the host status, reduce hardware wear and tear, save costs, and improve the efficiency of maintaining the in-vehicle system for autonomous driving.

[0111] In practical applications, when the test task is a performance test, the method provided in this embodiment can also determine whether the overall performance of the target host is abnormal based on the test results. After obtaining the confidence score, it can further determine whether the overall performance of the target host is abnormal based on the test results. If the overall performance of the target host is abnormal, further testing of the target host is required. The process is described below, and it may include the following steps:

[0112] Step 501: Based on the test results of the target host, determine whether the overall performance of the target host is abnormal.

[0113] Specifically, after obtaining the test results of the target host, it can be determined whether the overall performance of the target host is abnormal based on the test results. If it is determined that the overall performance of the target host is abnormal, it means that the target host needs to be tested for abnormality, and step S502 can be executed. If it is determined that the overall performance of the target host is not abnormal, it means that the target host is in good condition, and step S503 can be executed.

[0114] Step S502: Perform anomaly testing on the target host.

[0115] Specifically, as described above, if the overall performance of the target host is determined to be abnormal, then anomaly testing of the target host is required. The current testing method cannot determine the cause of the target host's anomaly. Therefore, to further pinpoint the anomaly, the target host can be disassembled and tested piece by piece. For example, the CPU, GPU, I / O, video memory, or main memory of the target host can be tested separately to check for anomalies. This will help determine the cause of the target host's anomaly.

[0116] Step S503: Determine that the target host is in good condition.

[0117] Specifically, if it is determined that the overall performance of the target host is not abnormal, it indicates that the target host is in good condition.

[0118] As can be seen from the technical solutions described above, the method provided in this application embodiment can, after obtaining the confidence level result, further determine whether the overall performance of the target host is abnormal based on the test result. This allows for timely anomaly testing of the target host to eliminate faults, effectively improving the detection speed of the host status, reducing hardware wear and tear, saving costs, and improving the efficiency of autonomous driving system maintenance.

[0119] In practical applications, there are different methods to determine whether the overall performance of the target host is abnormal. The following describes a method for determining whether the overall performance of the target host is abnormal, which can include the following three implementation methods:

[0120] The first type,

[0121] The test results of the target host are compared with the performance data of the target host in the target simulation data for the same time period to obtain the first standard deviation. If the first standard deviation is greater than the first preset threshold, it is determined that the overall performance of the target host is abnormal.

[0122] The second type,

[0123] The test results of the target host are compared with the obtained reference performance data to obtain the second standard deviation;

[0124] If the second standard deviation is greater than the preset second threshold, it is determined that the overall performance of the target host is abnormal.

[0125] The third type,

[0126] The test results of the target host are compared with the performance data of the target host in the target simulation data for the same time period to obtain a first standard deviation, and compared with the obtained reference performance data to obtain a second standard deviation; if the first standard deviation is greater than the preset first threshold and the fourth standard deviation is greater than the preset second threshold, then it is determined that the overall performance of the target host is abnormal.

[0127] For example, taking a 60-second simulated dataset as an example, after running this simulated dataset on the host A to be tested, the autonomous driving system will collect relevant data on the CPU, memory, GPU, video memory, and I / O of the P50, P90, P99, and Max systems of the autonomous driving system during this time period.

[0128] If, during this time period, the CPU, memory, GPU, video memory, and I / O performance data of the P50, P90, P99, and Max systems of the autonomous driving system are 15% higher than those obtained from other hosts running the same simulation dataset during the same time period, then it can be concluded that the overall performance of the host A under test may have a problem.

[0129] Alternatively, if the CPU, memory, GPU, video memory, and I / O data of the autonomous driving system's P50, P90, P99, and Max systems are more than 6% higher than the data obtained by the host A under test running the same simulation dataset during this period, it can be determined that the overall performance of the host A under test may have a problem, and further investigation is required.

[0130] Therefore, if the CPU, memory, GPU, video memory, and I / O data of the P50, P90, P99, and Max systems of the autonomous driving system are 15% higher than the CPU, memory, GPU, video memory, and I / O data of the P50, P90, P99, and Max systems of the autonomous driving system obtained by running the same simulation dataset on other hosts, then it can be considered that the overall performance of the host A under test may have problems, and further investigation of host A is required.

[0131] If the CPU, memory, GPU, video memory, and I / O data of the P50, P90, P99, and Max systems of the autonomous driving system during this time period are less than 6% lower than the CPU, memory, GPU, video memory, and I / O data of the P50, P90, P99, and Max systems of the autonomous driving system during this time period obtained from running the same simulation dataset on other hosts, then the overall performance of the host A under test can be considered good, and no further investigation of the host A under test is required.

[0132] As can be seen from the above technical solution, the embodiments of this application can compare the test results of the target host with the performance data of the target host in the target simulation data in the same time period to obtain a first standard deviation, compare it with the obtained reference performance data to obtain a second standard deviation, and compare it with the obtained reference performance data to obtain a third standard deviation. If the first standard deviation is greater than the preset first threshold and the fourth standard deviation is greater than the preset second threshold, the relationship between the preset first standard deviation and the preset first threshold, and the relationship between the fourth standard deviation and the preset second threshold are compared respectively to determine whether the overall performance of the target host is abnormal. This effectively improves the detection speed of the host status, reduces the wear and tear on the hardware, saves costs, and improves the efficiency of maintenance of the autonomous driving system.

[0133] As described above, the method provided in this application embodiment can determine whether each first target data satisfies preset reproduction conditions in a preset simulation environment. The process will be described below, and it may include the following steps:

[0134] Step S701: Run each of the first target data in a preset simulation environment. If the first target data generates a first parameter in the preset simulation environment, then determine that the first target data meets the preset reproduction conditions in the preset simulation environment.

[0135] Specifically, after obtaining each first target data, each first target data can be run in a preset simulation environment. If the first target data generates a first parameter in the preset simulation environment, it indicates that the first target data is data generated by the normal operation of the target. Thus, it can be determined that the first target data meets the preset reproduction conditions in the preset simulation environment. The first parameter is the same as the relevant parameter generated when the target vehicle performs a driving task.

[0136] For example, after extracting data related to the takeover of the target host from the target vehicle data set, it can be further determined whether the data generated by the takeover of the target host can be reproduced in a simulated environment.

[0137] If the data generated by the takeover of the target host produces CPU, memory, and disk I / O curves in a simulated environment similar to those generated when the target host performs an autonomous driving task, then the data generated by the takeover of the target host is considered reproducible in the simulated state. If the data generated by the takeover of the target host can be reproduced in the simulated state, it indicates that the captured data is test data that can be used to test the target host.

[0138] For example, a simulated data processing workflow can be used to automatically verify whether the data generated during the takeover of the target host can be reproduced in a simulated environment using a simulation program.

[0139] Step S702: Determine that the first target data does not meet the preset reproduction conditions.

[0140] Specifically, if the first target data cannot generate the first parameter in the preset simulation environment, it indicates that the first target data includes data when the target host is faulty. Thus, it can be determined that the first target data cannot meet the preset reproduction conditions in the preset simulation environment.

[0141] As can be seen from the technical solutions described above, the method provided in this application embodiment can determine whether each first target data satisfies preset reproduction conditions in a preset simulation environment. This can effectively improve the detection speed of the host status, reduce hardware wear and tear, save costs, and improve the efficiency of maintaining the vehicle-mounted system for autonomous driving.

[0142] The vehicle system host testing apparatus provided in the embodiments of this application is described below. The vehicle system host testing apparatus described below and the vehicle system host testing method described above can be referred to in correspondence.

[0143] See Figure 2 , Figure 2 This is a schematic diagram of the structure of a vehicle-mounted system host testing device disclosed in an embodiment of this application.

[0144] like Figure 2 As shown, the vehicle-mounted system host testing device may include:

[0145] The instruction receiving unit 101 is used to receive a test instruction for testing the target host and determine the test task corresponding to the test instruction;

[0146] The data acquisition unit 102 is used to select target simulation data corresponding to the test task from a preset target vehicle data set;

[0147] The test unit 103 is used to test the target host based on the target simulation data and obtain the test results of the target host.

[0148] As can be seen from the above technical solution, the embodiments of this application can utilize the instruction receiving unit 101 to receive test instructions for testing the target host and determine the test task corresponding to the test instructions; after determining the test task, the data acquisition unit 102 can utilize the data acquisition unit 102 to select target simulated data corresponding to the test task from a preset target vehicle data set; after determining the target simulated data, the test unit 103 can utilize the target simulated data to test the target host and obtain the test result of the target host. The device provided by the embodiments of this application can use data collected in real road tests as target simulated data to achieve the test task, and test the real host state by running the target simulated data in autonomous driving simulation, so as to solve the problem of host state detection or evaluation in autonomous driving, improve the detection speed of host state, reduce hardware wear and tear, save costs, and improve the efficiency of maintaining the vehicle system of autonomous driving.

[0149] Further optionally, the data acquisition unit 102 may include:

[0150] The first data acquisition subunit is used to determine the first timestamp, which is the timestamp when the target vehicle performs a predetermined operation corresponding to the test task.

[0151] The second data acquisition subunit is used to select each first target data whose occurrence time is located in the first time interval from the target vehicle data set. The first time interval is from the moment before the first timestamp to the moment after the first timestamp.

[0152] The third data acquisition subunit is used to preprocess each of the first target data to obtain target simulation data corresponding to the test task.

[0153] Further optionally, the third data acquisition subunit may include:

[0154] The first judgment unit is used to determine, for each first target data, whether the first target data meets the preset reproduction conditions in a preset simulation environment;

[0155] A tag adding unit is used to add a data tag corresponding to a predetermined operation that occurred on the target vehicle to the first target data when the execution result of the first judgment unit determines that the first target data meets the reproduction condition;

[0156] The data determination unit is used to use each first target data with added data labels as target simulation data corresponding to the test task.

[0157] Further optionally, if the test task is a performance test, the test unit 103 may include:

[0158] The comparison test unit is used to perform multiple different types of comparison tests on the target host based on the target simulation data, and obtain the comparison test results under each comparison test;

[0159] The test result acquisition unit is used to combine the confidence levels of each of the comparative test results to obtain the test results of the target host.

[0160] Further optionally, if the test task is to test the performance of the target host, the apparatus may further include:

[0161] The second judgment unit is used to determine whether the overall performance of the target host is abnormal based on the test results of the target host.

[0162] An anomaly testing unit is used to perform anomaly testing on the target host when the execution result of the second judgment unit determines that the overall performance of the target host is abnormal.

[0163] The status judgment unit is used to determine that the target host is in good condition when the execution result of the second judgment unit is that the overall performance of the target host is not abnormal.

[0164] Optionally, the second determining unit may include:

[0165] The first standard deviation acquisition unit is used to obtain the first standard deviation by comparing the test results of the target host with the performance data of the target host in the target simulation data over the same time period.

[0166] The third judgment unit is used to determine that the overall performance of the target host is abnormal when the first standard deviation is greater than a preset first threshold.

[0167] Further optionally, the second determining unit may also include:

[0168] The second standard deviation acquisition unit is used to compare the test results of the target host with the obtained reference performance data to obtain the second standard deviation;

[0169] The fourth judgment unit is used to determine that the overall performance of the target host is abnormal when the second standard deviation is greater than a preset second threshold.

[0170] Further optionally, the first determining unit may include:

[0171] The first parameter determination unit is used to run each first target data in a preset simulation environment. If the first target data generates a first parameter in the preset simulation environment, it is determined that the first target data meets the preset reproduction conditions in the preset simulation environment. The first parameter is the same as the relevant parameter generated when the target vehicle performs a driving task.

[0172] The reproduction determination unit is used to determine that the first target data does not meet the preset reproduction conditions when the execution result of the first parameter judgment unit is that the first target data cannot generate the first parameter in the preset simulation environment.

[0173] The specific processing flow of each unit included in the above-mentioned vehicle system host test device can be found in the relevant introduction of the vehicle system host test method section above, and will not be repeated here.

[0174] The vehicle-mounted system host testing device provided in this application embodiment can be applied to vehicle-mounted system host testing equipment, such as terminals: mobile phones, computers, etc. Optionally, Figure 3 The hardware structure block diagram of the vehicle-mounted system host test equipment is shown. (Refer to...) Figure 3 The hardware structure of the vehicle-mounted system host test equipment may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4.

[0175] In this embodiment, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4.

[0176] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0177] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0178] The memory stores a program, and the processor can call the program stored in the memory. The program is used to implement the various processing flows in the aforementioned terminal vehicle system host test scheme.

[0179] This application embodiment also provides a readable storage medium that can store a program suitable for processor execution, the program being used to implement the various processing flows of the aforementioned terminal in the vehicle system host test scheme.

[0180] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0181] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0182] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Various embodiments can be combined with each other. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of testing a vehicle system host, the method comprising: The method comprises: receiving a test instruction for testing a target host, and determining a test task corresponding to the test instruction; selecting target simulation data corresponding to the test task from a preset target vehicle-mounted data set; the target simulation data is obtained based on preprocessing each first target data selected from the target vehicle-mounted data set; the occurrence time of the first target data is located in a first time interval, the first time interval is from a previous moment of a first time stamp to a next moment of the first time stamp, and the first time stamp is a time stamp when the target vehicle performs a predetermined operation corresponding to the test task; testing the target host according to the target simulation data to obtain a test result of the target host; the test result of the target host is obtained based on confidence combination of each comparative test result, each comparative test result is obtained by performing multiple different types of comparative tests on the target host according to the target simulation data; wherein the comparative test results include a first standard deviation, a second standard deviation, a third standard deviation and a fourth standard deviation; the first standard deviation is obtained by comparing the test result of the target host with performance data of the target host in the same time period in the target simulation data; the second standard deviation is obtained by comparing the test result of the target host with performance data when the target host runs the same simulation data set during automatic driving task; the third standard deviation is obtained by comparing the test result of the target host with performance data when a standard host runs the same simulation data set; and the fourth standard deviation is obtained by comparing the test result of the target host with performance data when other hosts run the same simulation data set.

2. The method of claim 1, wherein, In the preset target vehicle-mounted data set, the target simulation data corresponding to the test task is selected, comprising: determining a first time stamp; selecting each first target data with an occurrence time located in a first time interval in the target vehicle-mounted data set; preprocessing each first target data to obtain target simulation data corresponding to the test task.

3. The method of claim 2, wherein, The preprocessing of each first target data to obtain target simulation data corresponding to the test task comprises: for each first target data, determining whether the first target data meets a preset reproduction condition in a preset simulation environment; if the first target data meets the reproduction condition, adding a data label corresponding to the predetermined operation of the target vehicle to the first target data; each first target data with a data label is used as target simulation data corresponding to the test task.

4. The method of claim 1, wherein, If the test task is to test the performance of the target host, the method further comprises: based on the test result of the target host, determining whether the overall performance of the target host is abnormal; if it is determined that the overall performance of the target host is abnormal, performing an abnormal test on the target host; if it is determined that the overall performance of the target host is not abnormal, determining that the target host is in good condition.

5. The method of claim 4, wherein, The determining whether the overall performance of the target host is abnormal includes: comparing the test result of the target host with performance data of the target host in the target simulation data in the same time period to obtain a first standard deviation; if the first standard deviation is greater than a preset first threshold, determining that the overall performance of the target host is abnormal.

6. The method of claim 4, wherein, The determining whether the overall performance of the target host is abnormal includes: comparing the test result of the target host with the obtained reference performance data to obtain a second standard deviation; if the second standard deviation is greater than a preset second threshold, determining that the overall performance of the target host is abnormal.

7. The method of claim 3, wherein, The determining whether each first target data meets a preset reproduction condition in a preset simulation environment includes: running each first target data in the preset simulation environment, and if the first target data generates a first parameter in the preset simulation environment, determining that the first target data meets the preset reproduction condition in the preset simulation environment, wherein the first parameter is the same as a related parameter generated when the target vehicle performs a driving task; if the first target data cannot generate the first parameter in the preset simulation environment, determining that the first target data does not meet the preset reproduction condition.

8. An in-vehicle system host testing apparatus characterized by comprising: The method includes: an instruction receiving unit configured to receive a test instruction for testing a target host and determine a test task corresponding to the test instruction; a data obtaining unit configured to select target simulation data corresponding to the test task from a preset target vehicle data set; the target simulation data is obtained based on preprocessing each first target data selected from the target vehicle data set; the occurrence time of the first target data is located in a first time interval, the first time interval is a time before a first time stamp to a time after the first time stamp, and the first time stamp is a time stamp when a target vehicle performs a predetermined operation corresponding to the test task; a testing unit configured to test the target host according to the target simulation data and obtain a test result of the target host. The test result of the target host is obtained based on confidence combination of each comparative test result, each comparative test result is obtained by performing a plurality of different types of comparative tests on the target host according to the target simulation data; wherein the each comparative test result includes a first standard deviation, a second standard deviation, a third standard deviation and a fourth standard deviation; the first standard deviation is obtained by comparing the test result of the target host with performance data of the target host in the same time period in the target simulation data; the second standard deviation is obtained by comparing the test result of the target host with performance data when the target host runs the same simulation data set while performing an automatic driving task; the third standard deviation is obtained by comparing the test result of the target host with performance data when a standard host runs the same simulation data set; and the fourth standard deviation is obtained by comparing the test result of the target host with performance data when other hosts run the same simulation data set.

9. An in-vehicle system host testing apparatus characterized by comprising: Comprise: One or more processors, and a memory; The memory has computer readable instructions stored therein, wherein the computer readable instructions, when executed by the one or more processors, implement the steps of the vehicle system host testing method according to any one of claims 1 to 7.

10. A readable storage medium characterized by: The readable storage medium has computer readable instructions stored therein, wherein the computer readable instructions, when executed by one or more processors, cause one or more processors to implement the steps of the vehicle system host testing method according to any one of claims 1 to 7.

Images