A method and device for testing and evaluating vehicle-side middleware of an unmanned vehicle and an unmanned vehicle

Abstract

The present invention discloses a method and equipment for testing and evaluating vehicle-side middleware of an unmanned vehicle, the method comprising: obtaining the vehicle-side middleware to be tested and evaluated; collecting vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and performance parameter measurement values ​​of the vehicle-side middleware, and generating a test standard value corresponding to the vehicle-side middleware; generating a test environment for simulating the actual production environment of the unmanned vehicle based on the vehicle-side environmental parameter measurement values; generating a test scenario based on a communication link table, the test scenario including a real vehicle test scenario and a simulation test scenario; based on the test environment and the test scenario, automatically testing the test data of the vehicle-side middleware, the test data corresponding to the test scenario; generating a test report to determine whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle. The scheme of the present invention forms a standardized method for testing and evaluating vehicle-side middleware, which can effectively verify the availability and correctness of iterative development of vehicle-side middleware.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned vehicles, and in particular to a method and device for testing and evaluating vehicle-side middleware of an unmanned vehicle, and an unmanned vehicle. Background Art

[0002] The vehicle-side middleware of unmanned vehicles is often developed by developers after iterative development. Developers write test cases and conduct unit tests on the new functions of the vehicle-side middleware in the development environment. This requires a lot of time and effort to develop test cases, test scenarios, test data, and verification code. On the one hand, due to the lack of standard process constraints for testing, the test code written for vehicle-side middleware testing cannot guarantee correctness, and the test quality cannot be guaranteed. On the other hand, due to the constraints of development equipment conditions, the development environment is very different from the vehicle-side production environment, and there are large environmental differences in test verification, and the reliability of test results is reduced; in particular, new functions may have an impact on existing functions. In addition, due to the development progress, it may not be possible to guarantee sufficient time to write test code, and incomplete testing brings risks of running in the production environment. Therefore, how to overcome the current situation that the vehicle-side middleware development of unmanned vehicles lacks standard and comprehensive testing in a similar production environment, and propose a set of solutions that are close to the production environment and can fully test and evaluate the iterative development of the vehicle-side middleware, is a technical problem to be solved by the present invention. Summary of the invention

[0003] The purpose of the embodiments of this specification is to provide a method, device and unmanned vehicle for testing and evaluating vehicle-side middleware of an unmanned vehicle in order to address the above-mentioned problems.

[0004] To solve the above technical problems, the embodiments of this specification are implemented as follows:

[0005] First, a test and evaluation method for unmanned vehicle middleware is proposed, including:

[0006] Obtain the vehicle-side middleware to be tested and evaluated;

[0007] Collecting vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and performance parameter measurement values ​​of the vehicle-side middleware, and generating a test evaluation standard value, wherein the test evaluation standard value can be dynamically adjusted according to the vehicle-side environmental parameter measurement values ​​and the performance parameter measurement values;

[0008] Based on the vehicle-side environmental parameter measurement values, generating a test environment for simulating an actual production environment of an unmanned vehicle;

[0009] Generate a test scenario based on the communication link table, wherein the test scenario includes a real vehicle test scenario and a simulation test scenario;

[0010] Based on the test environment and the test scenario, perform automated testing on the test data of the vehicle-side middleware, the test data corresponding to the test scenario;

[0011] Generate a test report to determine whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle.

[0012] Furthermore, a test scenario is generated based on a communication link table, wherein the communication link table includes a first communication link table corresponding to a real vehicle test scenario and a second communication link table corresponding to a simulation test scenario.

[0013] Furthermore, the test data corresponds to the test scenario and includes:

[0014] When corresponding to a real vehicle test scenario, collecting data of the unmanned vehicle as the test data, and the test data corresponds to the first communication link table;

[0015] When corresponding to a simulation test scenario, a data simulation tool is called to generate the test data, and the test data corresponds to the second communication link table.

[0016] Furthermore, based on the test environment and the test scenario, the test data of the vehicle-side middleware is automatically tested, including:

[0017] Calling an environmental load simulation tool to load the test environment for simulating the actual environment of the unmanned vehicle;

[0018] Calling a test scenario automatic generation tool to load the test scenario;

[0019] Call the automated testing tool to load the test data for testing and output the test results.

[0020] Furthermore, the system environment collection tool is called to collect the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and perform data persistence.

[0021] Furthermore, the test report includes a performance report and a function report, the performance report includes the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and the performance parameter measurement values ​​of the vehicle-side middleware, and the function report includes the test results of the test data.

[0022] Furthermore, based on the comparison result between the test report and the test evaluation standard value, it is determined whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle.

[0023] Furthermore, the vehicle-side middleware includes an observation module, which is used to collect the performance parameter measurement values ​​of the vehicle-side middleware and send them to the performance statistics module so that the performance statistics module generates the performance report; and / or, when the vehicle-side middleware is deployed to the unmanned vehicle, the vehicle-side environmental parameter measurement values ​​are collected and sent to the performance statistics module.

[0024] In the second aspect, a vehicle-side middleware testing and evaluation device for an unmanned vehicle is proposed, comprising:

[0025] The first unit is configured to obtain the vehicle-side middleware to be tested and evaluated;

[0026] The second unit is configured to collect vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and performance parameter measurement values ​​of the vehicle-side middleware, and generate a test evaluation standard value, wherein the test evaluation standard value can be dynamically adjusted according to the vehicle-side environmental parameter measurement values ​​and the performance parameter measurement values;

[0027] A third unit is configured to generate a test environment for simulating an actual production environment of an unmanned vehicle based on the vehicle-side environmental parameter measurement value;

[0028] A fourth unit is configured to generate a test scenario based on the communication link table, wherein the test scenario includes a real vehicle test scenario and a simulation test scenario;

[0029] A fifth unit, configured to perform automated testing on test data of the vehicle-side middleware based on the test environment and the test scenario, wherein the test data corresponds to the test scenario;

[0030] The sixth unit is configured to generate a test report to determine whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle.

[0031] In a third aspect, an unmanned vehicle is proposed, which has a vehicle-side middleware; before deploying the vehicle-side middleware to the unmanned vehicle, the unmanned vehicle vehicle-side middleware testing and evaluation method described in the first aspect is executed.

[0032] In a fourth aspect, an electronic device is proposed, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the unmanned vehicle-side middleware testing and evaluation method described in the first aspect.

[0033] In the fifth aspect, a computer-readable storage medium is proposed, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the unmanned vehicle-side middleware testing and evaluation method described in the first aspect.

[0034] The present invention can achieve at least part of the following technical effects:

[0035] The solution of the present invention forms a standardized method that can fully test and evaluate the iterative development of vehicle-side middleware by generating test standard values, constructing a test environment similar to the production environment, designing typical test scenarios and obtaining corresponding test data, and can effectively verify the availability and correctness of the iterative development of vehicle-side middleware. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 This is one of the schematic diagrams of the unmanned vehicle middleware testing and evaluation method provided in the embodiments of this specification.

[0038] Figure 2 The second schematic diagram of the unmanned vehicle middleware testing and evaluation method provided in the embodiment of this specification.

[0039] Figure 3 The third schematic diagram of the unmanned vehicle middleware testing and evaluation method provided in the embodiment of this specification.

[0040] Figure 4 The fourth schematic diagram of the unmanned vehicle middleware testing and evaluation method provided in the embodiment of this specification.

[0041] Figure 5 Schematic diagram 5 of the unmanned vehicle middleware testing and evaluation method provided in the embodiments of this specification.

[0042] Figure 6 Schematic diagram of the unmanned vehicle middleware testing and evaluation device provided in the embodiments of this specification.

[0043] Figure 7 It is a schematic diagram of the structure of an electronic device according to an embodiment of this specification. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be described clearly and completely below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0045] The purpose of the present invention is to overcome the current situation that the development of vehicle-side middleware for unmanned vehicles lacks standard and comprehensive testing in a similar production environment, and proposes a set of solutions that are close to the production environment and can fully test and evaluate the iterative development of vehicle-side middleware. To this end, the technical idea of ​​the solution of the present invention is to form an iterative development and measurement evaluation closed-loop implementation scheme, combined with self-developed tools for test evaluation, such as environmental load simulation tools, system environment collection tools, test scenario automation generation tools, simulation test data generation tools, etc., to form a complete step including data generation, environmental simulation, automated test verification, index collection generation and evaluation index adjustment. At the same time, an observation module is added to the vehicle-side middleware, and the test evaluation of the vehicle-side middleware performance indicators is added in interaction with the external performance statistics module. Specifically, the environmental parameters of the vehicle-side real vehicle are collected through the system environment collection tool, and the indicator values ​​generated by the vehicle-side middleware during operation are collected through the middleware performance statistics module, and the initial version of the evaluation standard value is formulated based on this. Read the communication link table used by the real vehicle, generate the test scenario through the test scenario automation generation tool, and adjust the communication link table based on different requirements for the scenario, and automatically generate a variety of test scenarios. Use the load simulation tool to simulate the vehicle-side production environment, and inject vehicle-side collected data and simulated test data to implement environmental simulation testing. During the test, the system environment collection tool is used to collect the environmental parameters of the test run, and the middleware performance statistics module is used to count the middleware performance. After the test is completed, the functional test results and performance test results are output to determine whether the test results meet the standard value requirements, thereby determining whether the vehicle-side middleware can be used in the production environment. When the new version of the vehicle-side middleware is applied, the system environment collection tool is used again to collect the environmental parameters of the vehicle-side actual vehicle operation, and the middleware performance statistics module is used to collect the indicator values ​​generated by the vehicle-side middleware during operation. Based on this, the evaluation standard value is updated to form a closed loop.

[0046] The following is a detailed description of a test and evaluation scheme for the unmanned vehicle-side middleware involved in this specification through a specific example.

[0047] Embodiment 1

[0048] like Figure 1 FIG. 1 is a schematic diagram of a method for testing and evaluating vehicle-side middleware of an unmanned vehicle according to an embodiment of the present invention. The method comprises:

[0049] S1: Obtain the vehicle-side middleware to be tested and evaluated. Specifically, after iterative development of the functions of the vehicle-side middleware, obtain the target vehicle-side middleware with new functional characteristics as the object of test and evaluation.

[0050] S2: Collect the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and the performance parameter measurement values ​​of the vehicle-side middleware, and generate a test evaluation standard value, which can be dynamically adjusted according to the vehicle-side environmental parameter measurement values ​​and the performance parameter measurement values

[0051] Optionally, a system environment acquisition tool is called to acquire the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and to persist the data. In one embodiment, the system environment acquisition tool may use a Linux scripting language to realize the acquisition function of the system environment, and persist the acquired environmental data into a file for storage.

[0052] Optionally, after the vehicle-side middleware is deployed to the unmanned vehicle, the vehicle-side environmental parameter measurement values ​​are collected and sent to a performance statistics module running outside the vehicle-side middleware test environment, so that the performance statistics module generates the performance report. That is, the vehicle-side environmental parameter measurement values ​​are one of the components of the performance report.

[0053] It should be noted here that, based on the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle collected by the system environment collection tool and the performance parameter measurement values ​​of the vehicle-side middleware collected, the initial version of the standard value for test evaluation can be generated; at the same time, for another implementation method, during the test evaluation process, the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and the performance parameter measurement values ​​of the vehicle-side middleware can still be collected by the system environment real-time collection tool, and the initial version of the test standard value can be revised and updated to make the test process and its evaluation standard value closer to the actual vehicle production environment, thereby making the test evaluation results more realistic and reliable.

[0054] S3: Based on the vehicle-side environmental parameter measurement values, generate a test environment for simulating the actual production environment of the unmanned vehicle.

[0055] Optionally, the environment load simulation tool is called to load the test environment for simulating the actual environment of the unmanned vehicle. In one embodiment, the environment load simulation tool writes a Linux script to set the system parameters of the test environment, and should also have the function of importing the data persistence file generated by the system environment acquisition tool.

[0056] S4: Based on the communication link table, a test scenario is generated, where the test scenario includes a real vehicle test scenario and a simulation test scenario.

[0057] Optionally, a test scenario is generated based on a communication link table, where the communication link table includes a first communication link table corresponding to a real vehicle test scenario and a second communication link table corresponding to a simulation test scenario.

[0058] Optionally, the test data corresponds to the test scenario, such as Figure 2 As shown, including:

[0059] S411: When corresponding to a real vehicle test scenario, collecting data of the unmanned vehicle as the test data, and the test data corresponds to the first communication link table;

[0060] S412: When corresponding to a simulation test scenario, calling a data simulation tool to generate the test data, and the test data corresponds to the second communication link table.

[0061] Specifically, the data simulation tool may use Python scripting language to read a file table or a database for carrying simulation test data corresponding to the second communication link table, and generate a simulation test data set that meets the link requirements.

[0062] Specifically, a standard test scenario is generated through the communication link table used in the actual vehicle. When the user needs to test the vehicle-side middleware in other test scenarios, different communication link tables can be designed to complete the design of different scenarios. For example, the number of test communication links is 1.2 times the number of links in the standard test scenario.

[0063] S5: Based on the test environment and the test scenario, perform automated testing on the test data of the vehicle-side middleware, wherein the test data corresponds to the test scenario.

[0064] Optionally, based on the test environment and the test scenario, the test data of the vehicle-side middleware is automatically tested, such as Figure 3 As shown, including:

[0065] S511: Calling an environmental load simulation tool to load the test environment for simulating the actual environment of the unmanned vehicle.

[0066] S512: Calling a test scenario automatic generation tool to load the test scenario.

[0067] In one embodiment, the test scenario automatic generation tool can use the python scripting language to implement the function of reading the communication link table corresponding to the test scenario. The communication link table can use a data table or a database storage carrier including an Excel table. Its function is to convert the link relationship in the communication link table into a test code. The writing language of the test code includes a high-level programming language including C++.

[0068] S513: Calling an automated testing tool to load the test data for testing and outputting the test results.

[0069] S6: Generate a test report to determine whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle.

[0070] Optionally, the test report includes a performance report and a function report, the performance report includes the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and the performance parameter measurement values ​​of the vehicle-side middleware, and the function report includes the test results of the test data.

[0071] Optionally, based on the comparison result between the test report and the test standard value, it is determined whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle.

[0072] Optionally, the vehicle-side middleware as shown in the figure includes an observation module, which is used to collect the performance parameter measurement values ​​of the vehicle-side middleware and send them to a performance statistics module running outside the vehicle-side middleware test environment, so that the performance statistics module generates the performance report; and / or, when the vehicle-side middleware is deployed to the unmanned vehicle, the vehicle-side environmental parameter measurement values ​​are collected and sent to the performance statistics module.

[0073] In one embodiment, the architecture of the middleware observation module and the performance statistics module is as follows: Figure 4 As shown. It can be seen that the vehicle-side middleware running on physical device A includes a functional module and an observation module, wherein the functional module completes the preset function of the vehicle-side middleware, is an existing functional module of the vehicle-side middleware, and can collect performance parameter data of the vehicle-side middleware and send it to the observation module; the observation module is a new module implanted into the vehicle-side middleware in order to execute the test evaluation method of the embodiment of the present invention, which is responsible for collecting the performance parameter data collected by the functional module and performing preliminary statistics, and sending the results to the performance statistics module on the physical device B using a preset transmission protocol, which is then collected and analyzed centrally by the performance statistics module to generate a performance report. At the same time, since the system environment collection tool is started in the operating environment, the real-time collected vehicle-side environmental parameter measurement values ​​will also be sent to the performance statistics module on the physical device B as part of the performance report. It should be noted that in this embodiment, physical device A can be understood as a dedicated test device, and physical device B can be any terminal device that runs the performance statistics module.

[0074] The following is Figure 5Taking the example of the vehicle-side middleware test and evaluation method of the present invention as an example, the implementation process of the embodiment of the vehicle-side middleware test and evaluation method of the present invention is described in detail. Physical device A includes the vehicle-side middleware to be tested and evaluated, and the vehicle-side middleware includes a functional module and an observation module; at the same time, it also runs an automated testing tool, a system environment acquisition tool, a data simulation tool, an environmental load simulation tool, and auxiliary data format conversion tools, such as the custom BeeBridge tool required for format conversion. Physical device B includes a performance statistics module. Physical device A, which is a dedicated test device, is connected to the unmanned vehicle through the pcap_convert format conversion tool. The specific working process is described as follows: a pcap data packet including a communication link table and real vehicle test data is obtained from the unmanned vehicle, and a Rosbag data packet is formed after the data format is converted by the pcap_convert data format conversion tool, and is sent to the automated testing tool as input. Here, the communication link table in the pcap data packet is also the first communication link table mentioned in the embodiment of the present invention, that is, the communication link table from the unmanned vehicle. During the test, on the one hand, the Rosbag data packet is sent to the functional module of the vehicle-side middleware through the custom BeeBridge tool for corresponding functional testing to form a Beeos data packet. The Beeos data packet test result is then restored to the Rosbag data packet through the reverse tool of the custom BeeBridge tool; on the other hand, the restored Rosbag data packet is compared with the Rosbag data packet used as the test input to verify the functional test results of the middleware and generate a corresponding functional report. At the same time, the observation module of the vehicle-side middleware obtains the vehicle-side middleware performance parameter measurement values ​​fed back by the functional module during the automatic test in real time, and combines them with the vehicle-side environmental parameter measurement values ​​from the system environment collection tool, and sends them to the performance statistics module in the physical device B in the form of UDP communication protocol, and the performance statistics module generates a performance report. In the evaluation stage, the test report including the functional report and the performance report will be compared with the pre-set test standard value. When the corresponding evaluation index meets the threshold setting, the evaluation result of whether it can be deployed to the unmanned vehicle is output.

[0075] The solution of the present invention forms a standardized method that can fully test and evaluate the iterative development of vehicle-side middleware by generating test standard values, constructing a test environment similar to the production environment, designing typical test scenarios and obtaining corresponding test data, and can effectively verify the availability and correctness of the iterative development of vehicle-side middleware.

[0076] Embodiment 2

[0077] Figure 6 This is a schematic diagram of the structure of the unmanned vehicle middleware testing and evaluation device 600 provided in one embodiment of this specification. Figure 6In one implementation, the unmanned vehicle vehicle-side middleware testing and evaluation device 600 includes:

[0078] The first unit 601 is configured to obtain the vehicle-side middleware to be tested and evaluated.

[0079] The second unit 602 is configured to collect the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and the performance parameter measurement values ​​of the vehicle-side middleware, and generate a test evaluation standard value, which can be dynamically adjusted according to the vehicle-side environmental parameter measurement values ​​and the performance parameter measurement values.

[0080] The third unit 603 is configured to generate a test environment for simulating the actual production environment of the unmanned vehicle based on the vehicle-side environmental parameter measurement value.

[0081] The fourth unit 604 is configured to generate a test scenario based on the communication link table, wherein the test scenario includes a real vehicle test scenario and a simulation test scenario.

[0082] The fifth unit 605 is configured to perform automated testing on the test data of the vehicle-side middleware based on the test environment and the test scenario, wherein the test data corresponds to the test scenario.

[0083] The sixth unit 606 is configured to generate a test report to determine whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle.

[0084] It should be understood that the unmanned vehicle middleware testing and evaluation device of the embodiment of this specification can also execute Figures 1 to 5 The method for executing the vehicle-side middleware test and evaluation device (or equipment) of the unmanned vehicle is implemented in Figures 1 to 5 The functions of the example shown will not be described in detail here.

[0085] Embodiment 3

[0086] The present embodiment provides an unmanned vehicle. The unmanned vehicle has a vehicle-side middleware; before deploying the vehicle-side middleware to the unmanned vehicle, the unmanned vehicle vehicle-side middleware testing and evaluation method of embodiment one is executed. The unmanned vehicle executes the technical features of the method described in embodiment one and has corresponding beneficial effects, and the specific technical features are not repeated. In the embodiment of the present invention, the unmanned vehicle is deployed and operated with a vehicle-side middleware. When the vehicle-side middleware is iteratively developed, by generating test standard values, constructing a test environment similar to the production environment, and designing typical test scenarios and obtaining corresponding test data, a standardized method that can fully test and evaluate the iterative development of the vehicle-side middleware is formed, which can effectively verify the availability and correctness of the iterative development of the vehicle-side middleware, and improve the test quality of the iteratively developed vehicle-side middleware before it is deployed to the unmanned vehicle.

[0087] Embodiment 4

[0088] like Figure 7 The figure shows a schematic diagram of the electronic device structure of this embodiment. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device may also include hardware required for other services.

[0089] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0090] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0091] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a shared resource access control device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0092] Obtain the vehicle-side middleware to be tested and evaluated;

[0093] Collecting vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and performance parameter measurement values ​​of the vehicle-side middleware, and generating a test evaluation standard value, wherein the test evaluation standard value can be dynamically adjusted according to the vehicle-side environmental parameter measurement values ​​and the performance parameter measurement values;

[0094] Based on the vehicle-side environmental parameter measurement values, generating a test environment for simulating an actual production environment of an unmanned vehicle;

[0095] Generate a test scenario based on the communication link table, wherein the test scenario includes a real vehicle test scenario and a simulation test scenario;

[0096] Based on the test environment and the test scenario, perform automated testing on the test data of the vehicle-side middleware, the test data corresponding to the test scenario;

[0097] Generate a test report to determine whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle.

[0098] The above is as in this manual Figures 1 to 5 The unmanned vehicle vehicle-side middleware test and evaluation method disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of this specification can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of this specification can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0099] Of course, in addition to software implementation, the electronic device of the embodiments of this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0100] Embodiment 6

[0101] The embodiment of the present specification also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to execute Figures 1 to 5 The unmanned vehicle vehicle-side middleware testing and evaluation method of the illustrated embodiment is specifically used to perform the following method:

[0102] Obtain the vehicle-side middleware to be tested and evaluated;

[0103] Collecting vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and performance parameter measurement values ​​of the vehicle-side middleware, and generating a test evaluation standard value, wherein the test evaluation standard value can be dynamically adjusted according to the vehicle-side environmental parameter measurement values ​​and the performance parameter measurement values;

[0104] Based on the vehicle-side environmental parameter measurement values, generating a test environment for simulating an actual production environment of an unmanned vehicle;

[0105] Generate a test scenario based on the communication link table, wherein the test scenario includes a real vehicle test scenario and a simulation test scenario;

[0106] Based on the test environment and the test scenario, perform automated testing on the test data of the vehicle-side middleware, the test data corresponding to the test scenario;

[0107] Generate a test report to determine whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle.

[0108] In short, the above description is only a preferred embodiment of this specification and is not intended to limit the protection scope of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the protection scope of this specification.

[0109] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0110] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0111] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0112] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A method for testing and evaluating vehicle-side middleware of an unmanned vehicle, characterized in that: Applied to test equipment, including: Obtain the vehicle-side middleware to be tested and evaluated; Collect vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and performance parameter measurement values ​​of the vehicle-side middleware, and generate test evaluation standard values, which are dynamically adjusted according to the vehicle-side environmental parameter measurement values ​​of the actual vehicle operation and the performance parameter measurement values ​​of the vehicle-side middleware collected in real time; Based on the vehicle-side environmental parameter measurement values, generating a test environment for simulating an actual production environment of an unmanned vehicle; Generate a test scenario based on the communication link table, wherein the test scenario includes a real vehicle test scenario and a simulation test scenario; Based on the test environment and the test scenario, perform automated testing on the test data of the vehicle-side middleware, the test data corresponding to the test scenario; Generate a test report to determine whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle based on the test report and the test evaluation standard value, wherein the test report includes a performance report and a function report, and the function report includes a test result of the test data; The vehicle-side middleware includes an observation module, and the test equipment includes a system environment collection tool. The observation module is used to collect the performance parameter measurement values ​​of the vehicle-side middleware and send them to a performance statistics module of a terminal device outside the test equipment. The system environment collection tool is used to collect the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and send them to the performance statistics module, so that the performance statistics module generates the performance report based on the vehicle-side environmental parameter measurement values ​​and the performance parameter measurement values ​​of the vehicle-side middleware.

2. The unmanned vehicle vehicle-side middleware testing and evaluation method according to claim 1 is characterized in that: A test scenario is generated based on a communication link table, wherein the communication link table includes a first communication link table corresponding to a real vehicle test scenario and a second communication link table corresponding to a simulation test scenario.

3. The unmanned vehicle vehicle-side middleware testing and evaluation method according to claim 2 is characterized in that: The test data corresponds to the test scenario and includes: When corresponding to a real vehicle test scenario, collecting data of the unmanned vehicle as the test data, and the test data corresponds to the first communication link table; When corresponding to a simulation test scenario, a data simulation tool is called to generate the test data, and the test data corresponds to the second communication link table.

4. The unmanned vehicle vehicle-side middleware testing and evaluation method according to any one of claims 1 or 3 is characterized in that: Based on the test environment and the test scenario, the test data of the vehicle-side middleware is automatically tested, including: Calling an environmental load simulation tool to load the test environment for simulating the actual environment of the unmanned vehicle; Calling a test scenario automatic generation tool to load the test scenario; Call the automated testing tool to load the test data for testing and output the test results.

5. The unmanned vehicle vehicle-side middleware testing and evaluation method according to any one of claims 1 or 4, characterized in that: The system environment collection tool is called to collect the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and perform data persistence.

6. The unmanned vehicle vehicle-side middleware testing and evaluation method according to claim 1 is characterized in that: The performance report includes the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and the performance parameter measurement values ​​of the vehicle-side middleware.

7. The unmanned vehicle vehicle-side middleware testing and evaluation method according to claim 6 is characterized in that: Based on the comparison result between the test report and the test evaluation standard value, it is determined whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle.

8. The unmanned vehicle vehicle-side middleware testing and evaluation method according to claim 6 is characterized in that: The observation module is used to collect the vehicle-side environmental parameter measurement values ​​and send them to the performance statistics module after the vehicle-side middleware is deployed to the unmanned vehicle.

9. An unmanned vehicle vehicle-side middleware testing and evaluation device, characterized in that: Applied to test equipment, including: The first unit is configured to obtain the vehicle-side middleware to be tested and evaluated; The second unit is configured to collect vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and performance parameter measurement values ​​of the vehicle-side middleware, and generate a test evaluation standard value, wherein the test evaluation standard value is dynamically adjusted according to the vehicle-side environmental parameter measurement values ​​of the actual vehicle operation and the performance parameter measurement values ​​of the vehicle-side middleware collected in real time; A third unit is configured to generate a test environment for simulating an actual production environment of an unmanned vehicle based on the vehicle-side environmental parameter measurement value; A fourth unit is configured to generate a test scenario based on the communication link table, wherein the test scenario includes a real vehicle test scenario and a simulation test scenario; A fifth unit, configured to perform automated testing on test data of the vehicle-side middleware based on the test environment and the test scenario, wherein the test data corresponds to the test scenario; A sixth unit is configured to generate a test report to determine whether the vehicle-side middleware can be updated and deployed to the unmanned vehicle based on the test report and the test evaluation standard value, wherein the test report includes a performance report and a function report, and the function report includes a test result of the test data; The vehicle-side middleware includes an observation module, and the test equipment includes a system environment collection tool. The observation module is used to collect the performance parameter measurement values ​​of the vehicle-side middleware and send them to a performance statistics module of a terminal device outside the test equipment. The system environment collection tool is used to collect the vehicle-side environmental parameter measurement values ​​of the unmanned vehicle and send them to the performance statistics module, so that the performance statistics module generates the performance report based on the vehicle-side environmental parameter measurement values ​​and the performance parameter measurement values ​​of the vehicle-side middleware.

10. An unmanned vehicle, characterized in that: The unmanned vehicle has a vehicle-side middleware; before deploying the vehicle-side middleware to the unmanned vehicle, the unmanned vehicle vehicle-side middleware testing and evaluation method described in any one of claims 1 to 8 is executed.

11. An electronic device, characterized in that: include: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to execute the unmanned vehicle vehicle-side middleware testing and evaluation method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute the unmanned vehicle-side middleware testing and evaluation method described in any one of claims 1 to 8.

Images