Performance test method and device of FOTA cloud platform, server and medium

Abstract

The invention relates to the technical field of testing, in particular to a performance testing method and device for an FOTA cloud platform, a server and a medium, and the method comprises the steps that a target testing type and corresponding testing parameters are acquired, and the testing type comprises concurrent testing and pressure load testing; mapping a target scene library based on the test parameters to generate a target test scene, and creating a plurality of simulation clients based on the test parameters, the simulation clients being used for simulating vehicles; and generating a target task based on the target test scene, issuing the target task to a plurality of simulation clients, sending a task request to the FOTA cloud platform by the simulation clients, and executing a corresponding task based on a task instruction issued by the FOTA cloud platform to complete the performance test of the FOTA cloud platform. Therefore, the problems that in the prior art, only single testing can be achieved, and a real peak scene cannot be simulated are solved.

Description

Technical Field

[0001] The present application relates to the field of testing technology, and in particular to a performance testing method, device, server and medium for a FOTA cloud platform. Background Art

[0002] With the development of intelligent automobiles, OTA (Over the Air, remote wireless upgrade) cloud upgrade technology occupies a very important position in the Internet of Vehicles business. As different models of car manufacturers access OTA, different models have different requirements for OTA, which makes the OTA cloud platform testing work complicated. Especially with the increase in connected vehicles, it is necessary to evaluate the performance and reliability requirements of the cloud platform at any time.

[0003] Traditional OTA stress testing patented technologies are mostly concentrated on the vehicle side. There are few stress testing technologies for the cloud side, and the test scenarios are relatively simple. There is no comprehensive and systematic performance testing method for FOTA (Firmware Over-The-Air, remote firmware upgrade) cloud platform. Existing performance testing technologies for cloud platforms generally use jmeter or mqx third-party tools to construct a single data message structure and perform large-scale data testing on a certain business topic or interface or port. For example, after generating an upgrade progress message, by maintaining the data configuration file, import the configuration file into jmeter, set the number of threads, and then request and observe the performance indicators of the server under the monitoring platform.

[0004] However, existing technologies can only provide single coverage of stress testing services or single-interface stress testing, and cannot fully simulate real peak scenarios. They cannot directly create stress testing scenarios and automatically construct clients for multiple network situations. Summary of the invention

[0005] The present application provides a performance testing method, device, server and medium for a FOTA cloud platform to solve the problems in the prior art that only a single test can be implemented and the real peak scenario cannot be simulated.

[0006] The first aspect of the present application provides a performance method for a FOTA cloud platform, comprising the following steps: obtaining a target test type and corresponding test parameters, the test type including a concurrent test and a stress load test; generating a target test scenario based on a target scenario library mapped to the test parameters, and creating multiple simulation clients based on the test parameters, wherein the simulation client is used to simulate a vehicle; generating a target task based on the target test scenario, and issuing the target task to multiple simulation clients, wherein the simulation client sends a task request to the FOTA cloud platform, and executes a corresponding task based on the task instruction issued by the FOTA cloud platform to complete the performance test of the FOTA cloud platform.

[0007] Optionally, in one embodiment of the present application, after sending the target task to multiple simulated clients, it also includes: obtaining task execution results of the multiple simulated clients; and generating performance test results of the FOTA cloud platform based on the task execution results.

[0008] Optionally, in one embodiment of the present application, the test parameters of the concurrent test include at least one of vehicle information, ECU (Electronic Control Unit) type, task type, number of concurrent vehicles, and region, and the test parameters of the stress load test include at least one of vehicle information, number of vehicles increased in a cycle, cycle time, continuous increase time, and initial number of vehicles.

[0009] Optionally, in one embodiment of the present application, creating multiple simulated clients based on test parameters includes: creating multiple simulated clients using at least one of multi-threading, multi-process or locust.

[0010] Optionally, in one embodiment of the present application, before generating a target test scenario based on the test parameter mapping target scenario library, it includes: writing a target scenario library code, wherein the scenario library code includes a mapping relationship between vehicle information and ECU, ECU and task type, task type and task process; generating a target scenario library based on the target scenario library code.

[0011] Optionally, in one embodiment of the present application, a target test scenario is generated based on a test parameter mapping target scenario library, including: selecting an initial test scenario matching the test parameters from the target scenario library based on the test parameters; and adjusting the initial test scenario based on the test parameters to generate a target test scenario.

[0012] The second aspect of the present application provides a performance testing device for a FOTA cloud platform, including: an acquisition module for acquiring a target test type and corresponding test parameters, the test types including concurrent testing and stress load testing; a creation module for generating a target test scenario based on a target scenario library mapped by test parameters, and creating multiple simulation clients based on the test parameters, wherein the simulation client is used to simulate a vehicle; a sending module for generating a target task based on the target test scenario, and sending the target task to multiple simulation clients, wherein the simulation client sends a task request to the FOTA cloud platform, and executes a corresponding task based on the task instruction sent by the FOTA cloud platform to complete the performance test of the FOTA cloud platform.

[0013] Optionally, in one embodiment of the present application, it also includes: a generation module, which is used to obtain task execution results of multiple simulated clients after sending the target task to multiple simulated clients; and generate performance test results of the FOTA cloud platform based on the task execution results.

[0014] Optionally, in one embodiment of the present application, the test parameters of the concurrent test include at least one of vehicle information, ECU type, task type, number of concurrent vehicles, and region, and the test parameters of the stress load test include at least one of vehicle information, number of vehicles increased in a cycle, cycle time, continuous increase time, and initial number of vehicles.

[0015] Optionally, in one embodiment of the present application, the creation module is further used to: create multiple simulated clients using at least one of multi-threading, multi-process or locust.

[0016] Optionally, in one embodiment of the present application, it also includes: a writing module, which is used to write a target scenario library code before generating a target test scenario based on the test parameter mapping target scenario library, wherein the scenario library code includes a mapping relationship between vehicle information and ECU, ECU and task type, task type and task process; and generate a target scenario library based on the target scenario library code.

[0017] Optionally, in one embodiment of the present application, the creation module is further used to: select an initial test scenario matching the test parameters from a target scenario library based on the test parameters; and adjust the initial test scenario based on the test parameters to generate a target test scenario.

[0018] The third aspect of the present application provides a server, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to execute a performance testing method for a FOTA cloud platform as described in the above embodiment.

[0019] The fourth aspect of the present application provides a computer-readable storage medium on which a computer program or instruction is stored. The computer program or instruction is executed by a processor to perform a performance testing method of a FOTA cloud platform as described in the above embodiment.

[0020] The fifth aspect of the present application provides a computer program product, including a computer program or instructions, which, when executed, implements the performance testing method of the FOTA cloud platform as in the above embodiment.

[0021] Therefore, this application has at least the following beneficial effects:

[0022] The embodiment of the present application can generate a target test scenario based on the test parameter mapping target scenario library corresponding to the target test type, and create multiple simulation clients based on the test parameters, and generate tasks based on the target test scenario, and send them to multiple simulation clients, each simulation client sends a task request to the FOTA cloud platform, and executes the corresponding task based on the task instruction issued by the FOTA cloud platform, thereby realizing the performance test of the FOTA cloud platform, and realizing multi-client concurrent testing, supporting concurrent calls of multiple interfaces and multi-scenario testing, being able to simulate the vehicle-cloud interaction scenario under the real peak value, and realizing multiple performance tests for concurrency and pressure load, so as to help improve the stability and reliability of the FOTA cloud platform, and no manual intervention is required during the test process, and the test efficiency is high. Thus, the technical problems that only a single test can be realized in the prior art and the real peak scenario cannot be simulated are solved.

[0023] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 A flowchart of a performance testing method for a FOTA cloud platform provided according to an embodiment of the present application;

[0026] Figure 2 A flowchart of a concurrent test provided according to an embodiment of the present application;

[0027] Figure 3 A flow chart of a pressure load test provided according to an embodiment of the present application;

[0028] Figure 4 This is an example diagram of a performance test device for a FOTA cloud platform provided according to an embodiment of the present application;

[0029] Figure 5 A schematic diagram of the structure of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0031] The following describes the performance test method, device, server and medium of the FOTA cloud platform of the embodiment of the present application with reference to the accompanying drawings. In view of the fact that the prior art mentioned in the above background technology can only do a single coverage of the stress test business, or a single interface stress test, it is impossible to fully simulate the real peak scenario, and it is impossible to directly create a stress test scenario, and automatically construct the client of multiple network situations, the present application provides a performance test method of a FOTA cloud platform, in which a target test scenario library can be generated based on the test parameter mapping corresponding to the target test type, and multiple simulated clients can be created based on the test parameters, and tasks are generated based on the target test scenario, and sent to multiple simulated clients, each simulated client sends a task request to the FOTA cloud platform, and executes the corresponding task based on the task instruction issued by the FOTA cloud platform, thereby realizing the performance test of the FOTA cloud platform. Thus, the problems that only a single test can be realized in the prior art and the real peak scenario cannot be simulated are solved.

[0032] Specifically, Figure 1 A flowchart of a performance testing method for a FOTA cloud platform provided in an embodiment of the present application.

[0033] like Figure 1 As shown, the performance testing method of the FOTA cloud platform includes the following steps:

[0034] In step S101, a target test type and corresponding test parameters are obtained.

[0035] Among them, the test types include concurrent testing and stress load testing. The test parameters of the concurrent test include at least one of vehicle information, ECU type, task type, number of concurrent vehicles, and region; the test parameters of the stress load test include at least one of vehicle information, number of vehicles increased in a cycle, cycle time, continuous increase time, and initial number of vehicles. The vehicle information includes at least one of vehicle brand, car series, and vehicle model data. Both concurrent testing and stress load testing support multiple groups of different types of vehicle model data.

[0036] Among them, the concurrency test is mainly used to test the ability of the FOTA cloud platform (hereinafter referred to as the cloud platform) to handle concurrent requests from a large number of vehicles in a short period of time, focusing on the instantaneous performance and response speed of the cloud platform; the stress load test is mainly used to evaluate the stability and performance of the cloud platform under long-term high load, focusing on the long-term performance and resource utilization of the cloud platform.

[0037] It should be noted that the concurrency test and stress load test of the embodiment of the present application are performed on the same platform, but are divided into two different test interfaces, namely the concurrency test interface and the stress load test interface.

[0038] For example, if you want to perform concurrent testing, select vehicle brand-series-model data, concurrent vehicle data, ECU selection box, upgrade task type selection box, and region selection box in the concurrent testing interface, such as S56 model-2000-TBOX-silent upgrade-A region, and support multiple groups of submissions, you can choose different models; if you want to perform stress load testing, select vehicle brand-series-model data in the load testing interface, the number of vehicles to increase periodically, the cycle time (minutes), the continuous increase time, the initial number of vehicles (can be empty), and for example S56 model-20-1-10-1000, and support the selection of multiple groups of different model data for submission.

[0039] In addition, since the network conditions in different regions may also be different, regional options are added to the test parameters of the embodiments of the present application to fit the actual vehicle-cloud interaction scenario and realize performance scenario testing under different network conditions.

[0040] In step S102, a target test scenario is generated based on a test parameter mapping target scenario library, and a plurality of simulation clients are created based on the test parameters, wherein the simulation clients are used to simulate vehicles.

[0041] It can be understood that the embodiments of the present application can map the target scenario library based on the test parameters set by the user to generate a target test scenario, and create multiple simulation clients based on the test parameters, so as to be used subsequently to simulate the scenario of multiple vehicles interacting with the cloud platform in a real scenario.

[0042] In an embodiment of the present application, a target test scenario is generated based on a target scenario library mapped with test parameters, including: selecting an initial test scenario matching the test parameters from the target scenario library based on the test parameters; and adjusting the initial test scenario based on the test parameters to generate a target test scenario.

[0043] Among them, the target scenario library is a pre-built scenario library that contains a variety of test scenarios. Each test scenario has a set of specific test parameters, such as vehicle brand, car series, model, ECU type, task type, region, etc.

[0044] It is understandable that the embodiments of the present application can select an initial test scene that matches the test parameters from a target scene library based on the test parameters, and adjust the initial test scene according to the test parameters to generate a final target test scene for subsequent testing.

[0045] For example, if the user selects the S56 vehicle model, TBOX ECU, silent upgrade task, and region A, the initial test scenario that matches these parameters is selected from the target scenario library, and based on the concurrent vehicle number of 2000 selected by the user, the concurrent vehicle number is adjusted to 2000 on the basis of the initial test scenario. Through the above adjustments, the final target test scenario is generated. This scenario will be used in the actual test process. The generated target test scenario is the S56 vehicle model, 2000 concurrent vehicles, TBOX ECU, silent upgrade task, and region A.

[0046] In an embodiment of the present application, creating multiple simulated clients based on test parameters includes: creating multiple simulated clients using at least one of multi-threading, multi-process or locust.

[0047] It can be understood that the embodiments of the present application can create multiple simulated clients by using at least one of multi-threading, multi-process or locust.

[0048] Specifically: 1. Select the creation method, such as using Python's threading module to create multiple threads, each thread simulates a client; using Python's multiprocessing module to create multiple processes, each process simulates a client; using the Locust tool to create multiple clients. Locust is an open source tool for load testing that supports distributed testing.

[0049] 2. Generate simulation client configuration, and generate configuration information for each simulation client based on vehicle information, ECU type, task type, region and other parameters in the target test scenario.

[0050] 3. Create simulated client instances. Multithreading: Use threading.Thread to create multiple threads, each thread simulates a client; Multiprocessing: Use multiprocessing.Process to create multiple processes, each process simulates a client; Locust: Write a Locust test script and use Locust's User class to create multiple client instances.

[0051] 4. Set the network conditions of each simulated client instance, such as latency, packet loss rate, etc., according to the region selected by the user.

[0052] In an embodiment of the present application, before generating a target test scenario based on a target scenario library mapped with test parameters, the steps include: writing a target scenario library code, wherein the target scenario library code includes a mapping relationship between vehicle information and ECU, ECU and task type, and task type and task process; and generating a target scenario library based on the target scenario library code.

[0053] It can be understood that the embodiments of the present application can write a target scenario library code, wherein the target scenario library code contains the definition of multiple test scenarios, specifically including the mapping relationship between vehicle information, ECU type, task type and its corresponding task process, and generates a target scenario library based on the target scenario library code.

[0054] In step S103, a target task is generated based on the target test scenario, and the target task is issued to multiple simulation clients, wherein the simulation client sends a task request to the FOTA cloud platform, and executes the corresponding task based on the task instruction issued by the FOTA cloud platform to complete the performance test of the FOTA cloud platform.

[0055] It can be understood that the embodiments of the present application can generate tasks based on the target test scenario and send them to multiple simulated clients. Each simulated client sends a task request to the FOTA cloud platform and executes the corresponding task based on the task instructions issued by the FOTA cloud platform, thereby realizing performance testing of the FOTA cloud platform and realizing multi-client concurrent testing, which can simulate real car-cloud interaction scenarios and realize multiple performance tests for concurrency and stress loads to help improve the stability and reliability of the FOTA cloud platform. No human intervention is required during the test, and the test efficiency is high.

[0056] In an embodiment of the present application, after sending the target task to multiple simulated clients, it also includes: obtaining the task execution results of the multiple simulated clients; and generating performance test results of the FOTA cloud platform based on the task execution results.

[0057] It is understandable that the embodiment of the present application can also obtain the task execution results of multiple simulated clients, such as task execution failure or success, and generate performance test results of the FOTA cloud platform based on the task execution results.

[0058] Specifically, to execute the method of this application, it is necessary to execute under the following conditions: Python operating environment, installation of Python third-party libraries and front-end and back-end codes of this technology, a PC or server with no less than 16G memory, and access to the local intranet, which mainly includes the following parts:

[0059] 1. Based on the MQTT protocol file and protocal file, write the scenario library code to complete the mapping relationship between vehicle model and ECU, ECU and task type, task type and task process. The task process is modularized through the function method. After the vehicle model-ECU-task type is clear, the task process is randomly obtained. For example, a certain vehicle model-a certain ECU-silent upgrade, a random upgrade fails and then rolls back the upgrade package.

[0060] Through the customized scenario library, the corresponding upgrade task can be automatically created after specifying the vehicle model, and different upgrade scenario data under the vehicle model can be randomly selected to cover the actual vehicle-cloud interaction scenario. The real simulation of large-scale upgrade scenarios can achieve performance testing under high coverage of platform services.

[0061] 2. Use the Python third-party library mqtt-phao module and locust module. Combine the multi-process and thread pool modules. Generate clients in different network environments in multiple regions. Write the listening and sending logic, and the sending and receiving data processing logic.

[0062] Set up virtual user groups in different regions, each with different network conditions, to test the interactive performance with the cloud.

[0063] 3. Use vue+Django to write front-end and back-end codes to complete platform integration.

[0064] 4. Use multi-threaded features to monitor the operation of each thread, and throw an exception error log when the thread is abnormal. Count the time of the upstream and downstream gateways to the last abnormal time, as well as the last task end time. Record each time point and abnormal log data.

[0065] The following is a specific example to describe the performance test method of the FOTA cloud platform of the present application. Python+vue3.0+Django is used to develop the front-end and back-end HTML interfaces. The concurrent test process is as follows: Figure 2 As shown, the pressure load test process is as follows Figure 3 shown.

[0066] 1. The page includes cascade filtering of test scenarios, a concurrent test interface, a pressure load test interface, and the concurrent test interface cascades the selection of vehicle brand-car series-model data, concurrent vehicle data, ECU selection box, upgrade task type selection box, and region selection box. For example: S56 model-2000-TBOX-silent upgrade-Nanjing, and supports multiple groups of submissions, and different models can be selected.

[0067] 2. On the load test interface, select vehicle brand-car series-car model data, the number of vehicles to be added periodically, the cycle time (minutes), the continuous increase time, and the initial number of vehicles (can be empty). For example, S56 model-20-1-10-1000, and supports selecting multiple groups of different car model data for submission.

[0068] 3. After selecting the scenario, submit the execution with one click; the background presses the parameters and passes the corresponding class method. Create the model vehicle, create the number of tasks, create a simulated terminal and log in to the mqtx environment, keep the heartbeat, keep monitoring, obtain the task parameters after issuing the task, assemble the task interaction data, and report to the cloud gateway.

[0069] Among them, use Python language to write the scene library scene execution code; obtain the parameters from the interface to the scene library title and map them to the Python component method; Python generates the corresponding proto according to the protocol list, generates the corresponding message format and sends it to the platform server through the MQTT protocol.

[0070] The network simulation implementation method of Mqtt multi-terminal is as follows:

[0071] (1) Introduction to the Python third-party library tool Toxiproxy. The main components of Toxiproxy include a TCP proxy and a client library that communicates with it. The client library can create and manage proxies and manipulate their status. The proxy itself can have multiple "toxicities" (Toxics), such as delay, shutdown, speed limit, etc. These toxicities can be adjusted separately to simulate different network problems.

[0072] (2) Implementation steps: download and install. After downloading Toxiproxy, start the proxy through the command and download and install toxiproxy-python. Use the python client library toxiproxy-python to write the interaction code with the proxy; configure the client proxy. In locust, use the python code to configure the toxiproxy proxy. Locust simulates the user using this proxy. When configuring the proxy, specify the local and target service addresses and ports; configure the network conditions of each client proxy. Add toxic to simulate different network events, such as bandwidth restrictions or packet loss rates, such as proxy.toxics().latency("lantency_downstream", latency = 200.set()) to simulate a 200 millisecond network delay; use the proxy in locust. In the locust task, the message data is sent to the target server through the local proxy's IP and port, so that the request will be affected by the network conditions configured by toxiproxy.

[0073] In addition, the creation of multi-terminal mqtt clients can also be achieved through Python's multi-threading and multi-process. It does not necessarily have to be created with the help of locust. For clients in multi-network situations, it can also be achieved by manually specifying the flag sets of each client. Different flag sets have different reporting delay times. The server defines different response times according to different flag sets. The third-party library has a solution designed specifically for testing and CI environments, providing a deterministic method to interfere with connections, while also supporting random chaos and custom settings. Its core goal is to help prove through testing that the application has no single point of failure.

[0074] 4. Test results show the gateway uplink and downlink time in various scenarios; the number of failed upgrades and successful upgrades. Use Python's paramiko and psutil libraries to remotely connect to the server to monitor service performance indicators, such as CPU utilization, load rate, network IO, disk usage, etc. At the same time, the execution log path is displayed, and the execution log can be downloaded.

[0075] Among them, each test data indicator, log path, etc. are written into the database table to facilitate later data statistics and data analysis.

[0076] 5. Support integrated test reports or Locust's own test reports.

[0077] In summary, the testing method of the embodiment of the present application can:

[0078] 1. Integrate the simulation scenario library of multiple models, and perform multi-scenario mixed stress testing of a single model or a combination of multiple models.

[0079] 2. You can customize the model and number of vehicles to perform concurrent testing. For example, you can concurrently create 5,000 simulated terminals to interact with the cloud.

[0080] 3. You can customize the specified model, select the incremental period, and select the number of new vehicles within the specified period. This allows continuous stress testing and load testing. For example, specify a model to add 20 vehicles per minute, and continue adding vehicles for 10 minutes. Start by creating a FOTA task for the new vehicle. Create a simulation terminal to interact with the cloud.

[0081] 4. It can be combined with the network proxy plug-in function to create a large number of MQTT clients under different network conditions, simulate the vehicle-side upgrade scenarios under different network conditions caused by different regions, and interact with the cloud for data, which is highly practical.

[0082] 5. Integrated test report module, which can make statistical records and analysis on test process data.

[0083] The test method of the embodiment of the present application effectively improves the test efficiency, highly restores the scene under the real OTA peak upgrade, and is conducive to the discovery of business problems and performance problems under high concurrency and high responsibility. At the same time, it is convenient for other tests in the later stage, such as security testing under high concurrency. It does not rely on terminals, generates simulated terminals in batches, and uses real protocol data to perform stress testing in interactive scenarios, which is more comprehensive and detailed, helping to ensure the quality of testing and improve the reliability and stability of the platform.

[0084] According to the performance testing method of the FOTA cloud platform proposed in the embodiment of the present application, a target test scenario can be generated by mapping the target scenario library based on the test parameters corresponding to the target test type, and multiple simulation clients can be created based on the test parameters, and tasks are generated based on the target test scenario and sent to multiple simulation clients. Each simulation client sends a task request to the FOTA cloud platform, and executes the corresponding task based on the task instruction issued by the FOTA cloud platform, thereby realizing performance testing of the FOTA cloud platform, and realizing multi-client concurrent testing, which can simulate real car-cloud interaction scenarios, and realizing multiple performance tests for concurrency and stress loads, so as to help improve the stability and reliability of the FOTA cloud platform, and no manual intervention is required during the test process, and the test efficiency is high.

[0085] Next, the performance testing device of the FOTA cloud platform proposed in accordance with the embodiment of the present application is described with reference to the accompanying drawings.

[0086] Figure 4 It is a block diagram of a performance testing device of a FOTA cloud platform according to an embodiment of the present application.

[0087] like Figure 4 As shown, the performance testing device 10 of the FOTA cloud platform includes: an acquisition module 100 , a creation module 200 and a delivery module 300 .

[0088] Among them, the acquisition module 100 is used to obtain the target test type and the corresponding test parameters, and the test types include concurrent testing and stress load testing; the creation module 200 is used to generate a target test scenario based on the test parameter mapping target scenario library, and create multiple simulation clients based on the test parameters, wherein the simulation client is used to simulate the vehicle; the sending module 300 is used to generate a target task based on the target test scenario, and send the target task to multiple simulation clients, wherein the simulation client sends a task request to the FOTA cloud platform, and executes the corresponding task based on the task instruction issued by the FOTA cloud platform to complete the performance test of the FOTA cloud platform.

[0089] In the embodiment of the present application, the device 10 of the embodiment of the present application further includes: a generating module.

[0090] Among them, the generation module is used to obtain the task execution results of the client after sending the target task to multiple simulated clients; and generate the performance test results of the FOTA cloud platform based on the task execution results.

[0091] In an embodiment of the present application, the test parameters of the concurrent test include at least one of vehicle information, ECU type, task type, number of concurrent vehicles, and region, and the test parameters of the stress load test include at least one of vehicle information, number of vehicles increased in a cycle, cycle time, continuous increase time, and initial number of vehicles.

[0092] In the embodiment of the present application, the creation module 200 is further used to create multiple simulated clients using at least one of multi-threading, multi-process or locust.

[0093] In the embodiment of the present application, the device 10 of the embodiment of the present application further includes: a writing module.

[0094] Among them, a writing module is used to write a target scenario library code before generating a target test scenario based on the test parameter mapping target scenario library, wherein the scenario library code includes the mapping relationship between vehicle information and ECU, ECU and task type, task type and task process; and generate a target scenario library based on the target scenario library code.

[0095] In the embodiment of the present application, the creation module 200 is further used to: select an initial test scenario that matches the test parameters from a target scenario library based on the test parameters; adjust the initial test scenario based on the test parameters to generate a target test scenario.

[0096] It should be noted that the aforementioned explanation of the FOTA cloud platform performance test method embodiment is also applicable to the FOTA cloud platform performance test device of this embodiment, and will not be repeated here.

[0097] According to the performance testing device of the FOTA cloud platform proposed in the embodiment of the present application, a target test scenario can be generated based on the test parameter mapping target scenario library corresponding to the target test type, and multiple simulation clients can be created based on the test parameters, and tasks can be generated based on the target test scenario and sent to multiple simulation clients. Each simulation client sends a task request to the FOTA cloud platform, and executes the corresponding task based on the task instruction issued by the FOTA cloud platform, thereby realizing performance testing of the FOTA cloud platform, and realizing multi-client concurrent testing, which can simulate real car-cloud interaction scenarios, and realizing various performance tests for concurrency and pressure loads, so as to help improve the stability and reliability of the FOTA cloud platform, and no manual intervention is required during the test process, and the test efficiency is high.

[0098] Figure 5 A schematic diagram of the structure of a server provided in an embodiment of the present application. The server may include:

[0099] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0100] When the processor 502 executes the program, the performance testing method of the FOTA cloud platform provided in the above embodiment is implemented.

[0101] Furthermore, the server also includes:

[0102] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0103] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0104] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0105] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0106] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0107] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0108] An embodiment of the present application also provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the performance testing method of the FOTA cloud platform as described above is implemented.

[0109] An embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed, implements the above-mentioned FOTA cloud platform performance testing method.

[0110] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0111] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0112] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0113] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0114] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

Claims

1. A performance testing method for a FOTA cloud platform, characterized in that: The following steps are involved: Obtain target test types and corresponding test parameters, the test types including concurrent testing and stress load testing; Generate a target test scenario based on the test parameter mapping target scenario library, and create a plurality of simulation clients based on the test parameters, wherein the simulation clients are used to simulate vehicles; Generate a target task based on the target test scenario, and issue the target task to the multiple simulation clients, wherein the simulation client sends a task request to the FOTA cloud platform, and executes a corresponding task based on the task instruction issued by the FOTA cloud platform to complete the performance test of the FOTA cloud platform.

2. The performance testing method of the FOTA cloud platform according to claim 1, characterized in that, After issuing the target task to the multiple simulation clients, the method further includes: Obtaining task execution results of the multiple simulated clients; Generate a performance test result of the FOTA cloud platform based on the task execution result.

3. The performance testing method of the FOTA cloud platform according to claim 1, characterized in that, The stress test types include concurrent tests and stress load tests, wherein the test parameters of the concurrent test include at least one of vehicle information, ECU type, task type, number of concurrent vehicles, and region; the test parameters of the stress load test include at least one of vehicle information, number of periodic increase in vehicles, periodic time program, continuous increase time, and initial number of vehicles.

4. The performance testing method of the FOTA cloud platform according to claim 1, characterized in that, The creating multiple clients based on the test parameters includes: Create multiple clients using at least one of multi-threading, multi-processing, or locust.

5. The performance testing method of the FOTA cloud platform according to claim 1, characterized in that, Before generating a target test scenario based on the test parameter mapping target scenario library, the method includes: Writing a target scenario library code, wherein the scenario library code includes a mapping relationship between vehicle information and ECU, ECU and task type, and task type and task process; A target scene library is generated based on the target scene library code.

6. The performance testing method of the FOTA cloud platform according to claim 1, characterized in that, The generating a target test scenario based on the test parameter mapping target scenario library comprises: Selecting an initial test scene matching the test parameters from the target scene library based on the test parameters; The initial test scenario is adjusted based on the test parameters to generate the target test scenario.

7. A performance test device for a FOTA cloud platform, characterized in that: include: An acquisition module is used to acquire a target test type and corresponding test parameters, wherein the test type includes concurrent test and pressure load test; A creation module, used to generate a target test scenario based on the test parameter mapping target scenario library, and create a plurality of simulation clients based on the test parameters, wherein the simulation clients are used to simulate vehicles; A sending module is used to generate a target task based on a target test scenario, and send the target task to the multiple simulation clients, wherein the simulation client sends a task request to the FOTA cloud platform, and executes a corresponding task based on the task instruction sent by the FOTA cloud platform to complete the performance test of the FOTA cloud platform.

8. A server, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the performance testing method of the FOTA cloud platform as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: The computer program or instruction is executed by a processor to implement the performance testing method of the FOTA cloud platform as described in any one of claims 1-6.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed, the performance testing method of the FOTA cloud platform as described in any one of claims 1-6 is implemented.

Images