Performance test method and platform for industrial internet platform

Abstract

The invention discloses a performance testing method of an industrial internet platform, and relates to the technical field of performance testing of the industrial internet platform. The industrial internet performance test method provided by the invention can cover a multi-level framework, and is specifically embodied in that an industrial PaaS layer and a general PaaS layer mainly carry out platform Web performance test, an industrial DaaS layer mainly carries out data service performance test, and an edge layer mainly utilizes a device simulator to carry out edge layer performance test. According to the method, the problem that a performance testing method capable of covering a multi-level framework and forming a standard system is lacked in the industry at present is solved, the coverage rate of performance testing is increased, and the increasing testing requirements of customers are met.

Description

Technical Field

[0001] The present invention relates to the field of new generation information technology, and in particular to a performance testing method for an industrial Internet platform, which covers a multi-level architecture of a working Internet platform. Background Art

[0002] The industrial Internet platform system is becoming more mature and showing significant differences from general software. These differences are mainly reflected in their application areas, functional characteristics, technical architecture and development goals. The industrial Internet platform usually adopts a multi-level architecture (edge ​​layer, industrial DaaS layer, industrial PaaS layer, etc.) to run various industrial applications on the platform in a SaaS manner. This architecture enables the platform to flexibly integrate various industrial apps and realize modularization and customization of functions.

[0003] Industrial Internet platforms usually need to process a large amount of data and requests, especially in production environments, where the load across all levels of the system platform architecture may be very high. From the massive industrial data collection and processing at the edge layer, to the storage and transmission of heterogeneous data at the industrial DaaS layer, to the multimodal data resource access at the industrial PaaS layer, performance testing can simulate these high-load conditions, verify the system's processing capacity, responsiveness, and stability under high load, and ensure that the system will not crash or performance degrade due to excessive load. All of these put forward newer and higher requirements for the comprehensive performance level of the industrial Internet platform, and are also a severe test for testing technology methods.

[0004] At present, the industry's testing of industrial Internet platforms is mostly based on the requirements of ready-to-use software products (RUSP). The technical verification capabilities of industrial Internet platform features often remain at the stage of manufacturer demonstration and expert inquiry, lacking standard testing methods and customized tools. The test using RUSP software only focuses on the performance efficiency at the software interface level, and the demonstration inquiry cannot fully measure the platform performance indicators, and cannot fully demonstrate the performance of each level of the industrial Internet platform architecture, such as the concurrent access capability of edge nodes, edge node data transmission latency, etc., which is not conducive to highlighting the comprehensive evaluation of the performance characteristics of the industrial Internet platform; and there are also the following problems:

[0005] 1. There is no "layering" for the industrial Internet platform in the software performance testing method steps.

[0006] 2. Software performance testing methods have limitations and cannot accurately and comprehensively measure the performance indicators of industrial Internet platforms.

[0007] 3. There is no "grading" for the industrial Internet platform architecture in the software performance testing method steps. Summary of the invention

[0008] In order to overcome the current lack of performance testing methods in the industry that can cover the multi-level architecture of the industrial Internet platform and form a standard system, the present invention provides a performance testing method for the industrial Internet platform, which can cover the multi-level performance capability verification of the working Internet platform, and can comprehensively detect the platform Web performance, data service performance, edge layer performance, etc. of the industrial Internet platform, meet the growing testing needs of customers, and support the high-quality development of the industrial Internet platform. In addition, the performance testing method of the Gonglian Internet platform proposed in this application is standardized, can cover the multi-level performance capability verification of the industrial Internet platform, and can comprehensively detect the performance of the platform.

[0009] Specifically, the present invention implements the performance test of the above three-level architecture through the following technical solutions. One of the purposes of the present invention is achieved by the following technical solutions:

[0010] See also Figure 1 , a performance testing method for an industrial Internet platform, comprising the following steps:

[0011] S1. Test the service performance of the edge layer of the industrial Internet;

[0012] S2. Test the service performance of the Industrial Internet DaaS layer;

[0013] S3, testing the service performance of the Industrial Internet PaaS layer;

[0014] S4. Test the service performance of the Industrial Internet SaaS layer;

[0015] S5. Classify the performance of industrial Internet platforms.

[0016] Furthermore, in S1, the specific steps of the performance testing method of the edge layer include: determining the edge data collection scope, defining the edge gateway protocol type, clarifying the edge device locations, designing a specific performance testing script R4 for simulating edge layer data collection, data processing and analysis, and data access, and by executing the performance testing script R4, recording and analyzing the edge node concurrent access capability, data transmission delay, and packet loss rate.

[0017] Furthermore, in S2, the specific steps of the performance testing method of the DaaS layer include: determining the performance testing application scenario, designing a specific execution script R3, simulating the industrial DaaS layer data application, monitoring the industrial database operation indicators, and recording and analyzing the data storage throughput, storage delay, and query delay by executing the comprehensive performance testing script R3.

[0018] Furthermore, in S3, the specific steps of the performance testing method of the PaaS layer include: by determining the performance testing objectives and scope of the industrial PaaS layer, designing a specific execution script R2 to simulate the continuous release and deployment of the industrial PaaS layer, and monitoring resource changes according to scenario characteristics, and by executing the comprehensive performance testing script R2, recording and analyzing the number of developers' concurrency, resource usage, and service availability.

[0019] Furthermore, the service performance testing method steps of the PaaS layer also specifically include: conducting developer concurrency, resource occupancy, and service availability tests, and recording and analyzing developer concurrency, resource occupancy, and service availability test results.

[0020] Furthermore, in S4, the specific steps of the performance testing method of the SaaS layer include: determining the performance testing benchmark, and then designing a specific execution script R1 to simulate the actual operation of industrial SaaS layer users, and testing the single sign-on performance in series according to the scenario characteristics, and generating and recording the response time, throughput, and request success rate by executing the comprehensive performance testing script R1.

[0021] Furthermore, the throughput is calculated as follows:

[0022]

[0023] Furthermore, in S5, the overall performance is evaluated by normalization processing and weighted calculation, and the overall performance score S can be calculated by the following formula:

[0024] S=wI×(wI1×I1+wI2×I2+wI3×I3)+wP×(wP1×P1+wP2×P2+wP3×P3)+

[0025] wA×(wA1×A1+wA2×A2+wA3×A3)+

[0026] wD×(wD1×D1+wD2×D2+wD3×D3)

[0027] Among them, I1, I2, I3 are the normalized values ​​of the edge layer indicators, P1, P2, P3 are the normalized values ​​of the PaaS layer indicators, A1, A2, A3 are the normalized values ​​of the SaaS layer indicators, D1, D2, D3 are the normalized values ​​of the DaaS layer indicators, wI1, wI2, wI3 are the internal weights of the edge layer, wP1, wP2, wP3 are the internal weights of the PaaS layer, wA1, wA2, wA3 are the internal weights of the SaaS layer, wD1, wD2, wD3 are the internal weights of the DaaS layer, wI, wP, wA and wD are the weights of the edge layer, PaaS layer, SaaS layer and DaaS layer respectively.

[0028] The second object of the present invention is achieved by adopting the following technical solution:

[0029] A performance testing platform for an industrial Internet platform is applied to a performance testing method for an industrial Internet platform as one of the purposes of the present invention. The performance testing platform for an industrial Internet platform includes: a processor and a memory, the processor is communicatively connected to the memory, the memory stores a computer program, and the processor executes the performance testing method for the industrial Internet platform described above according to the computer program.

[0030] The third object of the present invention is achieved by adopting the following technical solution:

[0031] A computer-readable storage medium stores program data, and the program data is used to execute the performance testing method of the industrial Internet platform described above.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. According to the hierarchical architecture characteristics of the industrial Internet platform, a "layered" performance testing approach is provided, making the performance testing method of the industrial Internet platform scientific and useful.

[0034] 2. For each layer of the industrial Internet platform, formulate a performance testing method that meets the specific technical requirements of each layer of the industrial Internet platform to avoid single and incomplete performance testing methods.

[0035] 3. Through normalization and weighted calculation of the performance test index results of different dimensions at each level, the quantifiable overall performance results of the industrial Internet platform are obtained. Based on 1 and 2, the performance test results are more valuable and more meaningful.

[0036] 4. Based on the performance results of the industrial Internet platform, define the classification rules and implement the horizontal comparison method of the comprehensive performance indicators of different industrial Internet platforms. So far, a full range of "vertical" and "horizontal" performance testing has been achieved.

[0037] 5. In summary, the limitation of software performance testing methods and the inability to correctly, scientifically and comprehensively measure the performance indicators of industrial Internet platforms have been solved. At the same time, a "layered" and "graded" performance testing method for industrial Internet platforms has been implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the performance testing method of the multi-level architecture of the Industrial Internet platform.

[0039] Figure 2 It is a schematic diagram of the performance testing method of the industrial SaaS layer.

[0040] Figure 3 It is a schematic diagram of the performance testing method of the industrial PaaS layer.

[0041] Figure 4 Schematic diagram of the performance test method of the industrial DaaS layer

[0042] Figure 5 Schematic diagram of edge layer performance testing method. DETAILED DESCRIPTION

[0043] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0044] Embodiment 1

[0045] The present invention provides a preferred embodiment, a performance testing method for an industrial Internet platform, such as Figure 1 As shown in the figure, there is no "layering" problem for the Industrial Internet platform in the steps of solving the software performance testing method; there are limitations in solving the problem that the software performance testing method cannot correctly and comprehensively measure the performance indicators of the Industrial Internet platform; there is no "grading" problem for the Industrial Internet platform architecture in the steps of solving the software performance testing method.

[0046] One of the purposes of the present invention is achieved by the following technical solution:

[0047] A performance testing method for an industrial Internet platform comprises the following steps:

[0048] S1. Test the service performance of the edge layer of the industrial Internet;

[0049] S2. Test the service performance of the Industrial Internet DaaS layer;

[0050] S3, testing the service performance of the Industrial Internet PaaS layer;

[0051] S4. Test the service performance of the Industrial Internet SaaS layer;

[0052] S5. Classify the performance of industrial Internet platforms.

[0053] Furthermore, in S1, the specific steps of the performance testing method of the edge layer include: determining the edge data collection scope, defining the edge gateway protocol type, clarifying the edge device locations, designing a specific performance testing script R4 for simulating edge layer data collection, data processing and analysis, and data access, and by executing the performance testing script R4, recording and analyzing the edge node concurrent access capability, data transmission delay, and packet loss rate.

[0054] In this example, see Figure 5 ,For the edge layer, the edge node concurrent access capability, edge node data transmission delay, and edge node packet loss rate tests are ,performed, and the test results of the edge node concurrent access ,capability, edge node data transmission delay, and edge node packet loss rate are ,recorded and analyzed for subsequent overall performance evaluation.

[0055] Furthermore, in S2, the specific steps of the performance testing method of the DaaS layer include: determining the performance testing application scenario, designing a specific execution script R3, simulating the industrial DaaS layer data application, monitoring the industrial database operation indicators, and recording and analyzing the data storage throughput, storage delay, and query delay by executing the comprehensive performance testing script R3.

[0056] In this example, see Figure 4 , formulate a performance testing method that meets the specific technical requirements of the industrial DaaS layer of the industrial Internet platform: For the industrial DaaS layer, conduct data entry throughput, data entry latency, and data query latency tests on the industrial database, and record and analyze the test results of data entry throughput, data entry latency, and data query latency for subsequent overall performance evaluation.

[0057] Further, in S3, please refer to Figure 3 ,The specific steps of the performance testing method for the PaaS layer include ,determining the performance testing objectives and scope of the ,industrial PaaS layer, designing a specific execution script R2 to ,simulate the continuous release and deployment of the industrial PaaS layer, and ,monitoring resource changes according to scenario characteristics, and ,recording and analyzing the number of developers’ concurrency, resource occupancy, and service availability by executing ,the comprehensive performance test script R2.

[0058] Furthermore, the service performance testing method steps of the PaaS layer also specifically include: conducting developer concurrency, resource occupancy, and service availability tests, and recording and analyzing developer concurrency, resource occupancy, and service availability test results for subsequent overall performance evaluation.

[0059] For further information, see Figure 2 In S4, the specific steps of the performance test method of the SaaS layer include: determining the performance test benchmark, then designing a specific execution script R1, simulating the actual operation of industrial SaaS layer users, and testing the single sign-on performance in series according to the scene characteristics, and generating and recording the response time, throughput, and request success rate by executing the comprehensive performance test script R1. Furthermore, the throughput is calculated as follows:

[0060]

[0061] In this implementation, the corresponding Figure 2 ,determine the performance test benchmark, and then design a specific execution script R1 to simulate the actual operation of industrial SaaS layer users, and test the single sign-on performance in series according to the scenario characteristics. By executing the comprehensive performance test script R1, the response time, throughput, and request success rate are generated and recorded for subsequent overall performance evaluation.

[0062] Furthermore, in S5, the overall performance is evaluated by normalization processing and weighted calculation, and the overall performance score S can be calculated by the following formula:

[0063] S=wI×(wI1×I1+wI2×I2+wI3×I3)+wP×(wP1×P1+wP2×P2+wP3×P3)+

[0064] wA×(wA1×A1+wA2×A2+wA3×A3)+

[0065] wD×(wD1×D1+wD2×D2+wD3×D3)

[0066] Among them, I1, I2, I3 are the normalized values ​​of the edge layer indicators, P1, P2, P3 are the normalized values ​​of the PaaS layer indicators, A1, A2, A3 are the normalized values ​​of the SaaS layer indicators, D1, D2, D3 are the normalized values ​​of the DaaS layer indicators, wI1, wI2, wI3 are the internal weights of the edge layer, wP1, wP2, wP3 are the internal weights of the PaaS layer, wA1, wA2, wA3 are the internal weights of the SaaS layer, wD1, wD2, wD3 are the internal weights of the DaaS layer, wI, wP, wA and wD are the weights of the edge layer, PaaS layer, SaaS layer and DaaS layer respectively.

[0067] In this embodiment, since the dimensions of different measurement values ​​of each layer are different and the differences are large, the present invention adopts the Z-Score normalization method for normalization processing. The data is converted into a distribution with a mean of 0 and a standard deviation of 1, and the formula is: x'=(x-μ) / σ, where x is the original data, μ is the mean of the data, σ is the standard deviation of the data, and x' is the normalized data.

[0068] Based on the test results of the edge node concurrent access capability, edge node data transmission delay, and edge node packet loss rate of the edge layer (I), the developer concurrency, resource usage, and service availability test results of the industrial PaaS layer (P), the response time, throughput, and request success rate test results of the industrial SaaS layer (A), and the data storage throughput, data storage delay, and data query delay test results of the industrial DaaS layer (D), the mean μ and standard deviation σ of the 9 original data xi (x1 to x9) are calculated, and then the normalized values ​​x of the 9 values ​​are calculated one by one. i (x1 to x9), and assign them to I1, I2, I3, P1, P2, P3, A1, A2, A3, D1, D2, D3 one by one.

[0069] Weight assignment: Through the analytic hierarchy process (AHP), weights are assigned to indicators at each level. Within each level, the importance of different indicators may also be different. We assign weights to each level, and then assign weights to each specific indicator. The weight of the edge layer (I) is wI, the weight of the industrial PaaS layer (P) is wP, the weight of the industrial SaaS layer (A) is wA, and the weight of the industrial DaaS layer (D) is wD. Inside the edge layer (I): wI1, wI2, wI3; inside the industrial PaaS layer (P): wP1, wP2, wP3; inside the industrial SaaS layer (A): wA1, wA2

[0070] ,wA3; within the industrial DaaS layer (D): wD1,wD2,wD3.

[0071] Then, based on the comprehensive score, the performance of the industrial Internet platform can be divided into different levels. The higher the score, the higher the star rating, indicating better performance, as follows:

[0072] 5 stars: 90-100 points;

[0073] 4 stars: 80-89 points;

[0074] 3 stars: 70-79 points;

[0075] 2 stars: 60-69 points;

[0076] 1 star: 59 points and below.

[0077] Embodiment 2

[0078] The second object of the present invention is achieved by adopting the following technical solution:

[0079] A performance testing platform for an industrial Internet platform is applied to a performance testing method for an industrial Internet platform as one of the purposes of the present invention. The performance testing platform for an industrial Internet platform includes: a processor and a memory, the processor is communicatively connected to the memory, the memory stores a computer program, and the processor executes the performance testing method for the industrial Internet platform described above according to the computer program.

[0080] Embodiment 3

[0081] The third object of the present invention is achieved by adopting the following technical solution:

[0082] A computer-readable storage medium stores program data, and the program data is used to execute the performance testing method of the industrial Internet platform described above.

[0083] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A performance testing method for an industrial Internet platform, characterized in that: The following steps are involved: S1. Test the service performance of the edge layer of the industrial Internet; S2. Test the service performance of the Industrial Internet DaaS layer; S3, testing the service performance of the Industrial Internet PaaS layer; S4. Test the service performance of the Industrial Internet SaaS layer; S5. Classify the performance of industrial Internet platforms.

2. The performance testing method of an industrial Internet platform according to claim 1 is characterized in that: In S1, the specific steps of the performance testing method of the edge layer include: determining the edge data collection scope, defining the edge gateway protocol type, clarifying the edge device locations, designing a specific performance testing script R4 for simulating edge layer data collection, data processing and analysis, and data access, and by executing the performance testing script R4, recording and analyzing the edge node concurrent access capability, data transmission delay, and packet loss rate.

3. The performance testing method of an industrial Internet platform according to claim 1 is characterized in that: In S2, the specific steps of the performance testing method of the DaaS layer include: determining the performance testing application scenario, designing a specific execution script R3, simulating the industrial DaaS layer data application, monitoring the industrial database operation indicators, and recording and analyzing the data storage throughput, storage delay, and query delay by executing the comprehensive performance testing script R3.

4. The performance testing method of an industrial Internet platform according to claim 1 is characterized in that: In S3, the specific steps of the performance testing method of the PaaS layer include: by determining the performance testing objectives and scope of the industrial PaaS layer, designing a specific execution script R2 to simulate the continuous release and deployment of the industrial PaaS layer, and monitoring resource changes according to scenario characteristics, and by executing the comprehensive performance testing script R2, recording and analyzing the number of developers' concurrency, resource usage, and service availability.

5. The performance testing method of an industrial Internet platform according to claim 4 is characterized in that: The service performance testing method steps of the PaaS layer also specifically include: conducting developer concurrency, resource occupancy, and service availability tests, and recording and analyzing developer concurrency, resource occupancy, and service availability test results.

6. The performance testing method of an industrial Internet platform according to claim 1 is characterized in that: In S4, the specific steps of the performance testing method of the SaaS layer include: determining the performance testing benchmark, then designing a specific execution script R1, simulating the actual operation of industrial SaaS layer users, and testing the single sign-on performance in series according to the scenario characteristics, and generating and recording the response time, throughput, and request success rate by executing the comprehensive performance testing script R1.

7. The performance testing method of an industrial Internet platform according to claim 6 is characterized in that: The throughput is calculated as follows:

8. The performance testing method of an industrial Internet platform according to claim 1 is characterized in that: In S5, the overall performance is evaluated through normalization processing and weighted calculation, and the overall performance score S is calculated by the following formula: S=wI×(wI1×I1+wI2×I2+wI3×I3)+wP×(wP1×P1+wP2×P2+wP3×P3)+ wA×(wA1×A1+wA2×A2+wA3×A3)+ wD×(wD1×D1+wD2×D2+wD3×D3) Among them, I1, I2, I3 are the normalized values ​​of the edge layer indicators, P1, P2, P3 are the normalized values ​​of the PaaS layer indicators, A1, A2, A3 are the normalized values ​​of the SaaS layer indicators, D1, D2, D3 are the normalized values ​​of the DaaS layer indicators, wI1, wI2, wI3 are the internal weights of the edge layer, wP1, wP2, wP3 are the internal weights of the PaaS layer, wA1, wA2, wA3 are the internal weights of the SaaS layer, wD1, wD2, wD3 are the internal weights of the DaaS layer, wI, wP, wA and wD are the weights of the edge layer, PaaS layer, SaaS layer and DaaS layer respectively.

9. A performance testing platform for an industrial Internet platform, characterized in that: The performance testing platform of the industrial Internet platform includes a processor and a memory. The processor is communicatively connected to the memory. The memory stores a computer program. The processor executes the performance testing method of the industrial Internet platform as described in any one of claims 1 to 8 according to the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program data, and the program data is used to execute the performance testing method of the industrial Internet platform as described in any one of claims 1-8.

Images