Data engine service performance test method and device, server and storage medium

Abstract

The invention discloses a data engine service performance test method and device, a server and a storage medium, the method is used for a Locust tool configured by a data engine server, and the method comprises the following steps: acquiring a communication protocol data packet of a to-be-tested data engine server of the data engine server; a rewriting instruction is received, self-defined clients triggering a request event in the Locust tool are rewritten according to the rewriting instruction to obtain simulation clients, the number of the simulation clients is smaller than or equal to the upper limit value of the concurrency amount of the Locust tool, and multiple communication protocols exist in the simulation clients; obtaining performance test data when the simulation client communicates with the to-be-tested data engine server; and determining the data engine service performance of the to-be-tested data engine server according to the performance test data. The data engine performance service method provided by the embodiment of the invention is used for a Locust tool, can support data interaction of complex and diversified communication protocols, and is suitable for complex scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of software performance testing, and in particular to a data engine service performance testing method, device, server and storage medium. Background Art

[0002] With the rapid development of the Internet, the use of data engine services is becoming more and more common. The performance of the data engine server that provides data engine services in the data engine server affects the user experience of using the data engine service. Therefore, performance testing of the data engine server is very important.

[0003] In the related art, the performance of the data engine server is tested using a test tool, which has a fixed single communication protocol and is triggered by threads or processes, so the concurrency is limited. However, for some complex scenarios, the data engine server has complex and diverse communications and high concurrency, and requires a high-concurrency interface based on multiple communication protocols for data interaction. Therefore, the current performance testing method cannot be implemented, or it is necessary to configure various different test tools, which is time-consuming and labor-intensive. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, one of the purposes of the present invention is to propose a data engine service performance testing method, which can provide a simulated client with higher concurrency and diverse communication protocols, so as to simulate complex scenarios requiring high concurrency and complex and diverse communication protocols, and obtain the performance test data of the data engine server in complex scenarios without setting up multiple testing tools, thereby realizing data engine service performance testing, which is simple, fast, time-saving and labor-saving.

[0005] A second objective of the present invention is to provide a data engine service performance testing device.

[0006] A third objective of the present invention is to provide a data engine server.

[0007] A fourth objective of the present invention is to provide a computer-readable storage medium.

[0008] In order to achieve the above-mentioned purpose, a data engine service performance testing method proposed in an embodiment of the first aspect of the present invention is a Locust tool for configuring a data engine server, and the testing method includes: obtaining a communication protocol data packet of a data engine server to be tested of the data engine server; receiving a rewrite instruction input according to the communication protocol data packet, and rewriting a custom client that triggers a request event in the Locust tool according to the rewrite instruction to obtain a simulated client, wherein the number of the simulated clients is less than or equal to the concurrency upper limit of the Locust tool and the communication protocols of the simulated clients include multiple types; obtaining performance test data when the simulated client communicates with the data engine server to be tested; and determining the data engine service performance of the data engine server to be tested according to the performance test data.

[0009] According to the data engine service performance testing method of an embodiment of the present invention, based on the Locust tool as a performance testing tool, the Locust tool can rewrite the custom client that triggers the request event according to the rewrite instruction to obtain a simulated client, and the concurrency that can be set by the Locust tool is very high. The number of simulated clients is less than or equal to the concurrency upper limit of the Locust tool, which means that a relatively high concurrency can be achieved, and the communication protocol of the simulated client includes various, that is, a targeted and customized communication design can be achieved, so that complex scenarios of high concurrency and complex and diverse communication protocols can be simulated, and there is no need to configure multiple testing tools. It is simple, fast, time-saving and labor-saving.

[0010] In some embodiments, obtaining performance test data when the simulated client communicates with the data engine server to be tested includes: obtaining the client concurrency set in the Locust tool; controlling a corresponding number of the simulated clients to communicate with the data engine server to be tested at the same time according to the client concurrency, wherein each of the simulated clients has a unique identification mark; and cyclically obtaining performance test data when the simulated client communicates with the data engine server to be tested according to the identification mark.

[0011] In some embodiments, performance test data of the simulated client when communicating with the data engine server to be tested is cyclically obtained based on the identification mark, including: determining the identity index of all simulated clients communicating with the data engine server to be tested based on the identification mark; obtaining the performance test data of the simulated client when communicating with the data engine server to be tested in turn based on the identity index, and looping a preset number of times.

[0012] In some embodiments, the performance test data includes at least one of the number of requests, data response time, and resource usage rate of the data engine server to be tested.

[0013] In some embodiments, determining the data engine service performance of the data engine server to be tested based on the performance test data includes at least one of the following: determining the query rate per second of the data engine server to be tested based on the number of requests; determining the response time of the data engine server to be tested based on the data response time; determining the resource usage of the data engine server to be tested based on the resource utilization rate.

[0014] In some embodiments of the present invention, the communication protocol data packet includes at least one of a grpc protocol data packet and a websocket protocol data packet; receiving a rewrite instruction input according to the communication protocol data packet, and rewriting the simulated client that triggers a request event in the Locust tool according to the rewrite instruction, including: receiving a first rewrite instruction input according to the grpc protocol data packet, and rewriting the grpc client containing the grpc protocol that triggers the request event in the Locust tool according to the first rewrite instruction; and / or, receiving a second rewrite instruction input according to the websocket protocol data packet, and rewriting the websocket client containing the websocket protocol that triggers the request event in the Locust tool according to the second rewrite instruction.

[0015] In some embodiments of the present invention, the data engine service performance testing method further includes: before performing the data engine service performance test, configuring all the simulated clients to be an online environment that is consistent with the configuration of the data engine server to be tested.

[0016] In order to achieve the above-mentioned purpose, a data engine service performance testing device proposed in an embodiment of the second aspect of the present invention includes: an acquisition module, used to acquire a communication protocol data packet of a data engine server to be tested; a rewrite module, used to receive a rewrite instruction input according to the communication protocol data packet, and rewrite the custom client that triggers a request event in the Locust tool according to the rewrite instruction to obtain a simulated client, wherein the number of the simulated clients is less than or equal to the concurrency upper limit of the Locust tool and the communication protocol of the simulated client includes multiple types; a data acquisition module, used to acquire performance test data when the simulated client communicates with the data engine server to be tested; a determination module, used to determine the data engine service performance of the data engine server to be tested according to the performance test data.

[0017] According to the data engine service performance testing device proposed in the embodiment of the present invention, the Locust tool is selected as the performance testing tool. The custom client that triggers the request event in the Locust tool can be rewritten according to the communication protocol data packet of the data engine server to be tested through the rewriting module to obtain a simulated client. The number of simulated clients can reach the upper limit of the concurrency of the Locust tool, and the communication protocols of the simulated clients are diverse, so that targeted and customized communication design can be achieved, and complex scenarios with high concurrency and complex and diverse communication protocols can be simulated to achieve data engine server performance testing. There is no need to configure multiple testing tools, which is simple, fast, time-saving and labor-saving.

[0018] In order to achieve the above-mentioned purpose, the data engine server proposed in the third aspect of the embodiment of the present invention includes a Locust tool and a data engine server to be tested. The Locust tool communicates with the data engine server to be tested to implement the data engine service performance testing method described in any one of the above items.

[0019] According to the data engine server of the embodiment of the present invention, the Locust tool is used to communicate with the data engine server to be tested as a test tool for the data engine server to be tested. By executing the data engine service performance test method of the above embodiment, the needs of complex and diverse data interactions in complex scenarios are met, and there is no need to configure multiple test tools. It is simple, fast, time-saving and labor-saving.

[0020] In order to achieve the above-mentioned purpose, the fourth aspect of the present invention further proposes a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, it implements the data engine service performance testing method described in any one of the above items.

[0021] The computer-readable storage medium proposed in an embodiment of the present invention stores a computer program thereon. When the computer program is executed by a processor, the data engine service performance testing method of the above embodiment can be implemented. By adopting this method, the needs of complex and diverse data interactions in complex scenarios can be met, and there is no need to configure multiple testing tools. It is simple, fast, and saves time and effort.

[0022] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0024] Figure 1 A block diagram of a data engine service performance testing device according to an embodiment of the present invention;

[0025] Figure 2 is a flow chart of a data engine service performance testing method according to an embodiment of the present invention;

[0026] Figure 3 is a flow chart of a data engine service performance testing method according to another embodiment of the present invention;

[0027] Figure 4 is a flow chart of a data engine service performance testing method according to yet another embodiment of the present invention;

[0028] Figure 5 is a flow chart of a data engine service performance testing method according to yet another embodiment of the present invention;

[0029] Figure 6 FIG. 4 is a block diagram of a data engine server according to an embodiment of the present invention.

[0030] Reference numerals:

[0031] Data engine server 10;

[0032] Locust tool 1, data engine server to be tested 2;

[0033] Data engine service performance testing device 100;

[0034] Acquisition module 101 , rewriting module 102 , data acquisition module 103 , determination module 104 . DETAILED DESCRIPTION

[0035] Embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. Embodiments of the present invention are described in detail below.

[0036] The application scenarios of the data engine service performance testing method according to the embodiment of the present invention are as follows: Figure 1 As shown, the data engine server 10 is configured with the Locust tool 1, and the Locust tool 1 communicates with the data engine server 2 to be tested of the data engine server 10. The data engine server 2 to be tested is the core of the data engine server and can implement data engine services. The data engine service performance of the data engine server 2 to be tested is tested through the Locust tool 1.

[0037] The data engine is used to provide a system based on microservices. The data engine services include basic services such as logging, configuration, authentication, monitoring, etc., as well as basic libraries. The various services provided by the data engine mainly include gateway services, data services, logging services, authentication services, and computing services. Among them, gateway services such as the car-cloud communication of the WS gateway can simultaneously support the online data transmission of millions of vehicles. The authentication service can also support the algorithm service and other upper-layer services to perform authorization authentication concurrently. And by using the grpc interface in the data engine, it can simultaneously support a large number of external services to remotely call data services. The computing service can provide CPU (Central Processing Unit, central processing unit) and GPU (Graphics Processing Unit, graphics processing unit) computing power scheduling support. The business party will encapsulate the execution program into a docker image according to the agreement, and then create a job through the computing service interface to complete specific computing needs, so as to achieve the purpose of shortening the execution time.

[0038] The Locust tool is an easy-to-use distributed user load testing tool. The Locust tool has high single-machine concurrency capabilities and can be used to test the performance of communication interfaces, data services, databases, etc. under concurrent conditions.

[0039] Specifically, the data engine service performance testing method of the embodiment of the present invention can be used in the Locust tool 1 configured in the data engine server to test the data engine service performance of the data engine server 2 to be tested.

[0040] Reference below Figure 2-Figure 5 A data engine service performance testing method according to an embodiment of the present invention is described.

[0041] In some embodiments of the present invention, Figure 2 FIG. 1 is a flow chart of a data engine service performance testing method according to an embodiment of the present invention, wherein the data engine service performance testing method is applied to the Locust tool and at least includes steps S1-S4, which are specifically as follows.

[0042] S1, obtaining a communication protocol data packet of a data engine server to be tested.

[0043] Among them, the data engine server communicates with the service requester through a communication protocol when performing data engine services. Especially in complex scenarios, the data engine server needs to perform complex and diverse data interactions, and therefore needs to have a variety of communication protocol interfaces. In an embodiment of the present invention, in order to achieve performance testing of the data engine server to be tested in complex scenarios, a communication protocol data packet of the data engine server to be tested is obtained, and the communication protocol data packet includes the required communication protocol. Then, the Locust tool can be configured based on this communication protocol data packet to achieve communication with the data engine server to be tested.

[0044] In some embodiments, the communication protocol data packet may include but is not limited to at least one of a grpc protocol data packet and a websocket protocol data packet. After obtaining the grpc protocol data packet and the websocket protocol data packet, support is provided for testing services such as grpc protocol communication or websocket protocol communication according to the grpc protocol data packet or the websocket protocol data packet. Generally, complex scenarios such as tens of thousands of vehicles and other devices such as smart phones, electronic watches, etc. request communication from the server at the same time, and often the communication protocols are diverse. In the embodiments of the present application, other communication protocol data packets can also be obtained to support testing under other communication protocol communication services.

[0045] S2, receiving a rewrite instruction input according to a communication protocol data packet, and rewriting the custom client that triggers a request event in the Locust tool according to the rewrite instruction to obtain a simulated client.

[0046] Among them, the Locust tool is selected as the performance testing tool. Customized code can be written based on the Locust tool to simulate complex scenarios in stress testing. The Locust tool has a custom client function, that is, the information of the custom client can be configured according to needs. The client is equivalent to the user of the data engine server, that is, the simulated client is generated by the Locust tool to simulate the user end communicating with the data engine server to be tested.

[0047] For example, ordinary Python can be used to write user test scenarios, and the custom client can be configured as a simulated client for the required communication needs by rewriting the communication protocol and related parameters of the custom client, such as the operating system version, database version, encoding version, etc. In addition, the number of simulated clients can be less than or equal to the upper limit of the concurrency that can be set by the Locust tool. The Locust tool has a very high concurrency, so the number of simulated clients can also reach a very large number. And the simulated client can be configured with a variety of communication protocols, so that various data engine requesters in complex scenarios can be simulated, and the number of simulated clients can be configured according to demand, and there is no specific restriction from then on.

[0048] Specifically, after the Locust tool obtains the communication protocol data packet of the data engine server to be tested, the communication protocol information required for configuration can be displayed through the display unit matched by the data engine server. The user enters the rewrite instruction in the Locust tool configuration interface according to the required communication protocol information, and then configures the custom client according to the rewrite instruction to obtain the required number and communication protocol of simulated clients. Among them, the rewrite instruction can include the communication protocol, configuration information such as operating system version, database version, etc.

[0049] Furthermore, the Locust tool configures the corresponding custom client according to the input instructions. For example, the Locust tool writes the custom client code based on the official Requsts library, uses Python to call the request event provided by the Requsts library corresponding to the communication protocol data packet, such as the grpc protocol data packet or the websocket protocol data packet, and rewrites the program of the custom client, rewrites its method of sending and receiving messages, thereby obtaining the corresponding simulated client, such as obtaining the grpc client or the websocket client, that is, obtaining the client containing the communication protocol of the corresponding data engine server to be tested. By rewriting the client containing the new communication protocol, it is possible to communicate with the communication protocol interface corresponding to the data engine server to be tested.

[0050] Among them, in the embodiment, for Python, the Requsts library includes a variety of requst request events, and different communication protocol data packets can obtain corresponding requst request events from the Requsts library. And the Locust tool has powerful customization performance, and the custom client can be configured and rewritten. For example, the code program of the custom client can be rewritten to include the timing of sending and receiving data and the data processing method, and the communication protocol for sending and receiving messages can also be reconfigured, so that the required simulation client can be obtained. The simulation client can be configured with different and diverse communication protocols, and the required number of simulation clients can also be configured as needed to achieve high concurrent communication with the data engine server to be tested.

[0051] The simulated client can be used as a data engine service requester, and the data engine server under test can be used as a service provider of engine services. The data engine server under test interacts with the corresponding client through a communication protocol interface to implement data engine services.

[0052] For example, rewriting the communication protocol information of a custom client through the Locust tool can simulate tens of thousands of simulated clients of vehicles, and the simulated client can be configured with a variety of communication protocols, so as to simulate the communication between a large number of vehicle clients and the data engine server to be tested. For another example, rewriting the communication protocol of a custom client through the Locust tool can simulate different clients such as vehicles, home appliances, and mobile terminals, and realize the communication between complex and diverse communication protocol clients and the data engine server to be tested in complex scenarios.

[0053] Therefore, by customizing the configuration of the Locust tool, data communications in complex and diverse scenarios can be simulated to meet the data engine service performance testing requirements.

[0054] S3, obtains performance test data when the simulated client communicates with the data engine server to be tested.

[0055] Specifically, after the Locust tool generates a client containing a new communication protocol, the client can communicate with the communication protocol interface corresponding to the data engine server under test. The client can simulate the requester requesting the data engine service in the real scene to send data to the data engine server under test, and the data engine server under test will respond to the service request and feedback the corresponding data. In the process of simulating the corresponding client communicating with the data engine server under test, the Locust tool can obtain and store the relevant data of the communication protocol interface corresponding to the data engine server under test.

[0056] In some embodiments, the performance test data may include but is not limited to at least one of the number of requests, data response time, and resource usage rate of the data engine server to be tested.

[0057] Specifically, when the client configured by the Locust tool communicates with the data engine server under test, it can record the number of requests sent by the client, calculate the data response time of the data engine server under test in response to the request instruction, and obtain information such as resource utilization rate fed back by the data engine server under test.

[0058] S4, determining the data engine service performance of the data engine server to be tested according to the performance test data.

[0059] Specifically, after the Locust tool obtains the performance test data, it analyzes the performance test data. For example, as the number of configured simulated clients increases, which is equivalent to the increase in the number of users of the simulated data engine data server, how does the data response time of the data engine server to be tested change? For example, as the number of simulated user clients increases, the data response time increases, and it can be preliminarily determined that the request is blocked and queued. It can also be analyzed when the number of simulated user clients reaches a certain value before the data response time increases significantly, and it can also be analyzed how the resource utilization rate of the data engine server to be tested changes with the type and number of simulated user clients, etc. Thus, the data engine service performance of the data engine server to be tested is diagnosed.

[0060] According to the data engine service performance testing method of an embodiment of the present invention, the Locust tool is selected as the performance testing tool. The Locust tool can rewrite the simulated client that triggers the request event according to the rewrite instruction, and the concurrency that can be set by the Locust tool is very high. The number of simulated clients is less than or equal to the concurrency upper limit of the Locust tool, which means that a relatively high concurrency can be achieved, and the communication protocol of the simulated client includes various, that is, a targeted and customized communication design can be achieved, so that complex scenarios of high concurrency and complex and diverse communication protocols can be simulated, and there is no need to configure multiple testing tools. It is simple, fast, time-saving and labor-saving.

[0061] In addition, the Locust tool not only has high concurrency capabilities, but is also not controlled by processes and threads. It has a low threshold for use, a maximum number of concurrent users can reach one million, and good scalability.

[0062] In some embodiments of the present invention, Figure 3 As shown, it is a flow chart of a data engine service performance testing method according to another embodiment of the present invention, wherein a rewrite instruction according to communication protocol data input is received, and a simulated client that triggers a request event in the Locust tool is rewritten according to the rewrite instruction, so that a plurality of communication protocols and a large number of clients can be obtained, that is, the above step S2 may include step S21 and / or step S22, as follows.

[0063] S21, receiving a first rewrite instruction input according to a grpc protocol data packet, and rewriting the grpc client including the grpc protocol that triggers a request event in the Locust tool according to the first rewrite instruction.

[0064] Specifically, after the Locust tool receives a grpc protocol data packet, it can display relevant information of the grpc protocol data packet through a display unit. The user enters a first rewrite instruction in the Locust tool interface according to the displayed information. The Locust tool configures the communication protocol and related parameters of the custom client, such as operating system version, database version, code version, etc., according to the first rewrite instruction, and configures the custom client into a grpc client that includes the grpc protocol. The grpc client can communicate with the grpc protocol interface of the data engine server to be tested.

[0065] S22, receiving a second rewrite instruction input according to a websocket protocol data packet, and rewriting a websocket client including a websocket protocol that triggers a request event in the Locust tool according to the second rewrite instruction.

[0066] Specifically, similar to the above-mentioned process of generating a grpc client, after the Locust tool receives the websocket protocol data packet, it can display the relevant information of the websocket protocol by displaying a single power supply. The user can enter a second rewrite instruction in the Locust tool interface according to the displayed information. The Locust tool configures the communication protocol and related parameters of the custom client, such as database version, operating system version, code version, etc., according to the second rewrite instruction, and generates a websocket client containing the websocket protocol from the custom client. The websocket client can communicate with the websocket protocol interface of the data engine server to be tested.

[0067] The data engine service performance testing method of an embodiment of the present invention is based on the Locust tool to write customized code to realize the simulation of complex scenarios in stress testing, so that the data engine server to be tested supports stress testing of multiple protocols such as grpc protocol and websocket protocol, that is, supports complex business testing.

[0068] Furthermore, the concurrent access volume of the test tools currently used for data engine service performance testing has an upper limit, where concurrent access volume includes the number of concurrent business users, the maximum number of concurrent accesses, the number of system users, and the number of users online at the same time. In other words, the same data engine server can only allow a certain number of users to access or log in at the same time. When more than this number of users log in to the same data engine server at the same time, the data engine server may encounter errors due to the inability to respond to the login requests of all users at the same time. For example, when a large number of users search for the same hot search topic in the Weibo APP at the same time, it may cause the system to crash. Therefore, the communication performance of the data engine server to high-concurrency clients is a very important performance test.

[0069] In response to the above high-concurrency testing issues, the data engine service performance test of the embodiment of the present invention is implemented using the Locust tool. The Locust tool has high concurrency capabilities, and the maximum concurrency can reach one million, so it can simulate millions of clients communicating with the data engine server, and realize the data engine server's test of high-concurrency client communication performance, with good scalability.

[0070] In some embodiments of the present invention, Figure 4 As shown, it is a flowchart of a data engine service performance testing method according to another embodiment of the present invention, wherein performance test data is obtained when a client including a new communication protocol communicates with a data engine server to be tested, that is, the above step S3 specifically includes steps S31-S33.

[0071] S31, obtain the client concurrency set in the Locust tool.

[0072] Among them, the use of the Locust tool is completely based on events, so it is not limited by processes and threads and can support the initiation of a higher number of concurrent requests.

[0073] Specifically, when performing performance testing on the data engine server to be tested, the data engine server to be tested establishes communication with the client configured by the Locust tool, such as the grpc client and the websocket client. At this time, the client acts as a test machine. When performing stress testing on the communication between the data engine server to be tested and the client, the client acts as a stress testing machine.

[0074] For example, for the performance test of the communication between the server and the vehicle in the vehicle integrated dispatching and control system, the server is the data engine server to be tested, and the vehicle is the client, that is, the stress tester. Multiple vehicles will initiate access to the data engine server to be tested at the same time. At this time, the access concurrency of the data engine server to be tested is relatively high. When the performance of the communication between the server and the vehicle is detected based on the Locust tool, the client concurrency can be set in the Locust tool, for example, one million, that is, one million simulated clients are configured to send data requests to the data engine server to be tested, so as to achieve high concurrency testing of the server to be tested.

[0075] S32, controlling a corresponding number of simulated clients to communicate with the data engine server to be tested simultaneously according to the client concurrency, wherein each client has a unique identification mark.

[0076] Specifically, the Locust tool configures a corresponding number of simulated clients according to the set client concurrency, and controls the corresponding number of simulated clients to send requests to the data engine server to be tested at the same time, which is equivalent to simulating a high-concurrency client sending a request to the data engine server. At the same time, the data engine server to be tested generates a corresponding number of communication protocol interfaces, and realizes communication with the corresponding simulated clients through identification marks.

[0077] For example, tens of thousands of vehicles are online at the same time, that is, tens of thousands of vehicles send requests to the data engine server under test at the same time, which means that the data engine server under test needs to generate tens of thousands of communication protocol interfaces at the same time. For example, for the grpc client or websocket client generated by the Locust tool, the grpc protocol interface of the data engine server under test communicates with the grpc client, and the websocket protocol interface of the data engine server under test establishes communication with the websocket client. A unique index can be set for each communication protocol interface. The Locust tool loops to obtain the parameters of each communication protocol interface, that is, the performance test data, and performs performance testing based on the obtained performance test data. The following is a detailed description.

[0078] First, the Locust tool configures a custom client based on the client concurrency settings to generate a number of simulated clients with the same concurrency, so as to test the communication between the simulated client and the data engine server to be tested, that is, it can simulate high concurrency situations in real business scenarios.

[0079] Among them, the Locust tool sets an identification mark (index) for any simulated client, and the identification mark of any simulated client is different from the identification marks of other simulated clients. When the client establishes a connection with the engine server under test, the engine server under test can establish communication with the corresponding simulated client by identifying its identification mark. Therefore, for multiple simulated clients, the engine server under test identifies each simulated client by identifying the identification mark, thereby establishing communication between multiple simulated clients and the data engine server under test.

[0080] S33, cyclically obtaining performance test data of the client concurrency when the client communicates with the data engine server to be tested according to the identification mark.

[0081] Specifically, when the simulated client generated in the Locust tool communicates with the data engine server to be tested, the pressure generator provided by the Locust tool can be used to continuously increase the pressure on the data engine server to be tested, that is, to continuously increase the number of simulated clients that send data to the data engine server to be tested, thereby increasing the client concurrency, such as simulating a scenario where tens of thousands of vehicles are online at the same time. The Locust tool can determine the interaction between the client concurrency and the data engine server to be tested through the client's identification mark, and can collect performance test data.

[0082] In an embodiment, the identity index of all simulated clients communicating with the data engine server to be tested can be determined based on the identification mark of the simulated client; for example, the identification mark of each simulated client is unique, and the identification mark can be configured according to certain rules such as alphabetical order or numerical order or a preset order, so all simulated clients can be sorted according to the identification mark, that is, the identity index of all simulated clients communicating with the data engine server to be tested is obtained, and the identity index can determine the order of the corresponding simulated client in the sorting.

[0083] Furthermore, the performance test data of the simulated client communicating with the data engine server to be tested is obtained in sequence according to the identity index, and the cycle is repeated for a preset number of times, so that a large amount of performance test data of the data engine server to be tested can be obtained, which includes data requested simultaneously, i.e., high-concurrency data, and data communicated under different diverse communication protocols.

[0084] More specifically, the obtained performance test data can be a more critical side test indicator, wherein the performance test data includes at least one of the number of requests, data response time, and resource utilization rate of the data engine server to be tested. Among them, the query rate per second of the data engine server to be tested can be determined according to the number of requests, that is, the number of database connections per unit time can be determined. The response time of the data engine server to be tested can also be determined according to the data response time, and then it can be determined whether there is a CPU queue and a disk queue, and the length of the CPU queue and the length of the disk queue can be determined. And according to the number of requests and the data response time, the number of added clients and the response time of the system when the number of responding clients is increased can be determined, which can reflect the efficiency of the data engine server to be tested in parallel processing of data.

[0085] Furthermore, it may also include: determining the resource usage of the data engine server to be tested based on the resource usage rate. Specifically, by real-time monitoring of the resource usage rate of the data engine server to be tested, such as disk usage rate, CPU usage rate, etc., it can be determined whether the disk, CPU, etc. are in a saturated state.

[0086] Further, in some embodiments, the data engine service performance of the data engine server includes multiple factors, such as: the return size of the stress test interface, the number of database connections, CPU usage, CPU queue length, available memory, page swap frequency, disk usage, disk queue length, network bandwidth, thread pool consumption and connection pool consumption, etc., all of which will affect the test performance of the Locust tool. Among them, CPU usage refers to the ratio of the CPU busy time to the entire refresh cycle within the unit task manager refresh cycle (usually 1s), where when the ratio reaches 70%, it means that the CPU is in a saturated state. CPU queue length refers to the number of threads in the processing queue. For example, when the number of threads is greater than 2*the number of CPU cores, it indicates that there is a processor blocking situation. The number of available memory indicates the available memory size in megabytes. The page swap frequency is the frequency of data exchange between the internal memory and the virtual memory (hard disk), where the lower the page swap frequency, the better. Therefore, when the amount of available memory is sufficient, the cache can be increased to reduce the value of the page swap frequency. Disk usage is similar to CPU usage, indicating the proportion of the hard disk in reading and writing working states. When the disk usage reaches 70%, it means that the internal disk is saturated. The disk queue length is similar to the CPU queue length. When the disk is fully loaded, a read and write queue will be generated. The disk queue length is generally in the single digits, and it is normal to have a momentary disk queue. To evaluate whether the network bandwidth is sufficient, it is usually judged by the traffic received per second in "Bytes Received / Sec" and the traffic sent per second in "Bytes Send / Sec". The connection pool is mainly used for the connection between the client and the server, and the connection between the server and the database. The connection pool consumption is also a very important indicator for the consumption of the connection.

[0087] In some embodiments of the present invention, Figure 5 As shown, it is a flowchart of a data engine service performance testing method according to another embodiment of the present invention, wherein, before performing the data engine service performance testing in the above step S4, the data engine service performance testing method further includes step S40, which is specifically as follows.

[0088] S40, configuring all simulated clients to be an online environment consistent with the configuration of the data engine server to be tested.

[0089] Before testing, you first need to synchronize the Locust configuration file to all generated clients. For example, before testing, copy the Locust configuration file to each client to ensure that the configurations of the data engine server and client to be tested are consistent, and then perform the data engine service performance test.

[0090] Specifically, in some embodiments, an online environment is configured that is consistent with the configuration of the data engine server to be tested, including: the operating system version, database version, code version and etcd version of the online environment are consistent with those of the data engine server to be tested.

[0091] In summary, the data engine service performance testing method of the embodiment of the present invention is implemented using the Locust tool. The Locust tool can rewrite the simulated client that triggers the request event according to the rewrite instruction, and can obtain tens of thousands of simulated clients with diverse communication protocols, thereby realizing targeted and customized communication design, meeting the communication requirements of the data engine server communication protocol data packets, and thus meeting the needs of complex and diverse data interactions in complex scenarios. In addition, the Locust tool has powerful concurrency and can write customized code to realize the simulation of complex scenarios in stress testing, which is more flexible, simple and quick to use.

[0092] In some embodiments of the present invention, Figure 6 , which is a block diagram of a data engine service performance testing device according to an embodiment of the present invention, wherein the data engine service performance testing device 100 includes an acquisition module 101 , a rewriting module 102 , a data acquisition module 103 and a determination module 104 .

[0093] The acquisition module 101 is used to acquire the communication protocol data packet of the data engine server to be tested. In the embodiment, the communication protocol data packet may include but is not limited to at least one of a grpc protocol data packet and a websocket protocol data packet. After acquiring the grpc protocol data packet and the websocket protocol data packet, the test of services such as the grpc protocol or the websocket protocol can be implemented according to the grpc protocol data packet or the websocket protocol data packet.

[0094] The rewrite module 102 is used to receive a rewrite instruction input according to the communication protocol data, and rewrite the custom client that triggers the request event in the Locust tool according to the rewrite instruction to obtain a simulated client, wherein the number of simulated clients is less than or equal to the concurrency upper limit of the Locust tool and the communication protocols of the simulated clients include multiple types, thereby simulating complex communication scenarios with high concurrency and multiple communication protocols.

[0095] The data acquisition module 103 is used to obtain performance test data when the simulated client communicates with the data engine server to be tested. After the client program is rewritten based on the Locust tool, the data engine server to be tested establishes a communication connection with the simulated client. When the data engine server to be tested communicates with the new client, the Locust tool can collect data related to the communication protocol interface of the data engine server to be tested as performance test data, wherein, in some embodiments, the performance test data may include but is not limited to at least one of the number of requests, data response time, and resource utilization rate of the data engine server to be tested. Among them, the query rate per second of the data engine server to be tested can be determined according to the number of requests, that is, the number of database connections per unit time can be determined. The response time of the data engine server to be tested can also be determined according to the data response time, and then it can be determined whether there is a CPU queue and a disk queue, and the length of the CPU queue and the length of the disk queue can be determined. And according to the number of requests and the data response time, the number of clients to be added and the response time of the system when the number of clients responding can be increased, which can reflect the efficiency of the data engine server to be tested in parallel processing of data. The resource usage of the data engine server to be tested can be determined based on the resource usage rate. Specifically, by real-time monitoring of the resource usage rate of the data engine server to be tested, such as disk usage rate, CPU usage rate, etc., it can be determined whether the disk, CPU, etc. are in a saturated state.

[0096] The determination module 104 is used to determine the data engine service performance of the data engine server to be tested based on the performance test data. When the Locust tool is testing the data engine service performance of the data engine server to be tested, the Locust tool can continuously apply pressure to the data engine server to be tested through the pressure generator provided by the Locust tool, so that its concurrency continues to increase, while observing the resource usage of the system under test and indicators such as QPS and response time. When testing the data engine service performance of the data engine server to be tested, the embodiment of the present invention selects the Locust tool as the performance testing tool, and by determining the data engine service performance of the data engine server to be tested, it can then conduct a survey of various performance indicators of the data engine service performance, so as to provide a basis for future operation and maintenance to upgrade the software resources and hardware resources of the service.

[0097] According to the data engine service performance testing device 100 proposed in the embodiment of the present invention, the Locust tool is selected as the performance testing tool. The rewriting module 102 can rewrite the simulated client that triggers the request event in the Locust tool according to the communication protocol data packet of the data engine server to be tested obtained by the acquisition module 101. For example, the program of the original grpc client and / or the original websocket client is rewritten according to the grpc protocol data packet and / or the websocket protocol data packet, and its sending and receiving message methods are rewritten, so as to regenerate the corresponding client, i.e., the grpc client or the websocket client. The data acquisition module 103 can obtain performance test data when the data engine server to be tested communicates with the simulated client, and the determination module 104 further determines the data engine service performance of the data engine server to be tested according to the performance test data.

[0098] The data engine service performance testing device 100 proposed in the embodiment of the present invention can obtain a large number of simulated clients with diverse communication protocols by rewriting the custom client through the Locust tool, and the concurrency that can be set by the Locust tool is very high. The number of simulated clients is less than or equal to the concurrency upper limit of the Locust tool, which means that a relatively high concurrency can be achieved, and the communication protocols of the simulated clients are diverse, which means that a targeted and customized communication design can be achieved, thereby simulating complex scenarios of high concurrency and complex and diverse communication protocols, and there is no need to configure multiple testing tools. It is simple, fast, time-saving and labor-saving.

[0099] In addition, the Locust tool with high concurrency capability is selected as the performance testing tool. It is not controlled by processes and threads, supports distributed stress testing, has a low threshold for use, has a maximum concurrency of up to one million, and has good scalability.

[0100] In some embodiments of the present invention, Figure 1 As shown, it is a block diagram of a data engine server according to an embodiment of the present invention, wherein the data engine server 10 includes a Locust tool 1 and a data engine server 2 to be tested, and the Locust tool 1 communicates with the data engine server 2 to be tested to implement any of the above data engine service performance testing methods.

[0101] According to the data engine server 10 of the embodiment of the present invention, the Locust tool 1 is used to communicate with the data engine server 2 to be tested as a test tool for the data engine server 2 to be tested. By executing the data engine service performance test method of the above embodiment, the simulation of complex scenarios in stress testing can be realized, thereby enabling the data engine server 2 to be tested to support complex business testing. In addition, the Locust tool 1 is used as a performance testing tool to support distributed stress testing, with high concurrency and a maximum concurrency of one million, and good scalability.

[0102] In some embodiments of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored, and when the computer program is executed, any of the above data engine service performance testing methods is implemented.

[0103] According to the computer-readable storage medium proposed in an embodiment of the present invention, a computer program is stored thereon. When the computer program is executed by a processor, the data engine service performance testing method of the above embodiment can be implemented. By adopting this method, the concurrency capability of a single machine can be improved, the scalability is good, complex business testing is supported, and distributed stress testing is supported.

[0104] Other structures and operations of the data engine server 10 and the like according to the embodiment of the present invention are well known to those skilled in the art and will not be described in detail here.

[0105] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example.

[0106] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A data engine service performance testing method, characterized in that: Locust tool for data engine server configuration, test methods include: Obtaining a communication protocol data packet of the data engine server to be tested; Receiving a rewrite instruction input according to the communication protocol data packet, rewriting the custom client that triggers the request event in the Locust tool according to the rewrite instruction to obtain a simulated client, wherein the number of the simulated clients is less than or equal to the upper limit of the concurrency of the Locust tool and the communication protocols of the simulated clients include multiple types; Acquire performance test data when the simulated client communicates with the data engine server to be tested; The data engine service performance of the data engine server to be tested is determined according to the performance test data.

2. The data engine service performance testing method according to claim 1, characterized in that: Acquiring performance test data when the simulated client communicates with the data engine server to be tested, including: Get the client concurrency set in the Locust tool; Control a corresponding number of the simulated clients to communicate with the data engine server under test at the same time according to the concurrent client volume, wherein each of the simulated clients has a unique identification mark; The performance test data when the simulated client communicates with the data engine server to be tested is cyclically obtained according to the identification mark.

3. The data engine service performance testing method according to claim 2, characterized in that: The performance test data when the simulated client communicates with the data engine server to be tested is cyclically obtained according to the identification mark, including: Determine the identity indexes of all simulated clients communicating with the data engine server to be tested according to the identification mark; The performance test data when the simulated client communicates with the data engine server to be tested is obtained in sequence according to the identity index, and the cycle is repeated for a preset number of times.

4. The data engine service performance testing method according to claim 3, characterized in that: The performance test data includes at least one of the number of requests, data response time, and resource usage rate of the data engine server to be tested.

5. The data engine service performance testing method according to claim 4, characterized in that: Determining the data engine service performance of the data engine server to be tested according to the performance test data includes at least one of the following: Determine the query rate per second of the data engine server to be tested according to the number of requests; Determine the response time of the data engine server to be tested according to the data response time; The resource usage of the data engine server to be tested is determined according to the resource usage rate.

6. The data engine service performance testing method according to claim 1, characterized in that: The communication protocol data packet includes at least one of a grpc protocol data packet and a websocket protocol data packet; Receiving a rewrite instruction input according to the communication protocol data packet, and rewriting the simulated client triggering the request event in the Locust tool according to the rewrite instruction, including: Receiving a first rewrite instruction input according to the grpc protocol data packet, and rewriting the grpc client including the grpc protocol that triggers the request event in the Locust tool according to the first rewrite instruction; And / or, receiving a second rewrite instruction input according to the websocket protocol data packet, rewriting the websocket client containing the websocket protocol that triggers the request event in the Locust tool according to the second rewrite instruction.

7. The data engine service performance testing method according to any one of claims 1 to 6, characterized in that: The data engine service performance testing method also includes: Before performing a data engine service performance test, all of the simulated clients are configured as an online environment that is consistent with the configuration of the data engine server to be tested.

8. A data engine service performance testing device, characterized in that: include: The acquisition module is used to obtain the communication protocol data packet of the data engine server under test; a rewrite module, configured to receive a rewrite instruction input according to the communication protocol data packet, and rewrite the custom client that triggers the request event in the Locust tool according to the rewrite instruction to obtain a simulated client, wherein the number of the simulated clients is less than or equal to the upper limit of the concurrency of the Locust tool and the communication protocols of the simulated clients include multiple types; A data acquisition module, used to obtain performance test data when the simulated client communicates with the data engine server to be tested; The determination module is used to determine the data engine service performance of the data engine server to be tested according to the performance test data.

9. A data engine server, characterized in that: It comprises a Locust tool and a data engine server to be tested, wherein the Locust tool communicates with the data engine server to be tested to implement the data engine service performance testing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the data engine service performance testing method described in any one of claims 1-7 is implemented.

Images