Data processing method, platform, terminal and medium

Abstract

The present application relates to the technical field of computer, and discloses a data processing method, platform, terminal and medium. The present application reads in a pressure test script and a parameterized file corresponding to the pressure test script, splits the parameterized file according to a configured number of pressure machines when the parameterized file is determined to be used for online pressure test, generates a plurality of corresponding split files, distributes the plurality of split files to the pressure machines for pressure test, and marks the number of read file lines in the split files during the pressure test. The present application removes the consumptive data in the read split files through the pressure machines, obtains corresponding residual data, generates new files from the residual data, combines the new files generated in the pressure machines to generate a new parameterized file, and thus solves the problem of repeated conflict of parameterized file data used by scripts under distributed pressure test.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data processing method, platform, terminal, and medium. Background Technology

[0002] With the explosive growth of internet businesses, business systems are experiencing increasing distributed pressure. Verifying performance and identifying performance issues early has become a crucial step before system deployment. Traditional single-machine load testing and small-machine joint load testing methods are limited by the resources of the load machines, resulting in performance bottlenecks on the load-generating end. Distributed load testing platforms, as an alternative, solve the problem of single-point performance bottlenecks in machines, tools, and databases. Compared to single-machine load testing, in distributed load testing scenarios, parameterized files used by load testing scripts, such as session IDs, order numbers, and transaction serial numbers, suffer from data conflicts and duplicate data issues. Users face the following pain points in handling these issues:

[0003] 1) Consumable data problem: For parameterized data that is only used once, such as the original serial number of a refund, users are required to mark the data used by each machine after each load test and then clean up the data in the original file, which is inefficient and costly.

[0004] 2) Data conflict issue: The current file reading component reads data in a polling manner using multiple threads. Inevitably, some threads execute faster and start reading the file from the beginning, causing conflicts with the data lines read by the previous threads. Therefore, users have a need for data to be used exclusively in blocks by thread.

[0005] 3) Existing parameterized data file splitting and merging methods cannot adapt to solve the above two points. Summary of the Invention

[0006] The main objective of this invention is to propose a data processing method, platform, terminal, and medium, which aims to solve the problem of duplicate conflict in parameterized file data used by scripts under distributed load testing, meet the needs of processing consumable parameter data, and improve performance testing efficiency.

[0007] To achieve the above objectives, the present invention provides a data processing method applied to a distributed load testing platform, the data processing method comprising the following steps:

[0008] The load testing script and the corresponding parameterized file are read in. When the parameterized file is determined to be used for online load testing, the parameterized file is divided according to the configured number of load machines to generate multiple corresponding split files.

[0009] The multiple segmented files are assigned to various pressure machines for pressure testing, and the number of lines read in each segmented file is marked during the pressure testing process;

[0010] Consumable data in the read segmented files is removed by each of the presses to obtain the corresponding remaining data, and the remaining data is used to generate a new file;

[0011] The new files generated in each of the presses are merged to generate a new parameterized file.

[0012] Preferably, the step of reading the load testing script and the corresponding parameterized file, and when determining that the parameterized file is to be used for online load testing, splitting the parameterized file according to the configured number of load machines to generate multiple corresponding split files includes:

[0013] Read in the load testing script and the corresponding parameterized file, and select to perform online debugging or load testing on the parameterized file;

[0014] When determining the parameterized file for online load testing, the parameterized file is read in through the load testing core module, and the parameterized file is split into multiple segmented files according to the configured number of load machines based on the load machine configuration mechanism, and the order of the multiple segmented files is marked.

[0015] Preferably, after the step of reading the load testing script and the parameterized file corresponding to the load testing script, the method further includes:

[0016] In response to an online debugging operation on the parameterized file, the parameterized file is debugged via a debugging machine, and the number of second file lines read in the parameterized file is marked.

[0017] After the parameterized file is debugged, the unused data in the parameterized file is determined by the debugging machine based on the number of lines in the second file, and the unused data is used to generate a second parameterized file.

[0018] The parameterized file is replaced with the second parameterized file by the stress testing core module.

[0019] Preferably, the step of distributing the plurality of segmented files to various pressure machines for pressure testing, and marking the number of lines read in the segmented files during the pressure testing process, includes:

[0020] The multiple segmented files are allocated to each pressure machine through the core template of the pressure test, and the pressure test is performed by each pressure machine using the allocated segmented files.

[0021] During the pressure test performed by each of the presses, the allocated segmented files in each press are divided equally by thread, so that each thread reads a fixed number of regions in the segmented files in an iterative order.

[0022] Based on the configured thread-independent read switch, the start and end positions of each thread reading the file are calculated to mark the number of lines read in the segmented file; wherein, each thread stores the start and end position pointers of reading the segmented file.

[0023] Preferably, the step of removing consumable data from the read segmented files using each of the presses to obtain the corresponding remaining data includes:

[0024] After the load test of the segmented file is completed, the file lines that have been read in the segmented file are queried, and the consumable data in the file lines that have been read in the segmented file are removed by each of the load testers to obtain the corresponding sub-remaining data;

[0025] Based on the number of lines read in the segmented file, the unused data in the segmented file is determined by each of the pressure machines, and the sub-remaining data and the unused data are determined as remaining data.

[0026] Preferably, the step of merging the new files generated in each of the presses to generate a new parameterized file includes:

[0027] The core module of pressure testing collects the new files generated in each of the presses, and iterates through the new files generated in each of the presses, assembling each new file into a corresponding new parameterized file in a preset order.

[0028] Preferably, after the step of merging the new files generated in each of the presses to generate a new parameterized file, the method further includes:

[0029] The parameterized file is replaced with the new parameterized file through the stress testing core module;

[0030] In response to the load testing operation for the new parameterized file, the load testing module divides the new parameterized file according to the configured number of load testers to obtain multiple corresponding block files, and distributes the multiple block files to each load tester for load testing, and marks the number of file lines read in the block files during the load testing process.

[0031] Furthermore, to achieve the above objectives, the present invention also provides a distributed load testing platform, the distributed load testing platform comprising:

[0032] The splitting module is used to read in the load testing script and the parameterized file corresponding to the load testing script. When it is determined that the parameterized file will be used for online load testing, the parameterized file is split according to the configured number of load machines to generate multiple corresponding split files.

[0033] The reading module is used to distribute the multiple segmented files to various pressure machines for stress testing, and to mark the number of file lines read in the segmented files during the stress testing process.

[0034] The elimination module is used to eliminate consumable data from the read segmented files through each of the pressure machines, obtain the corresponding remaining data, and generate a new file from the remaining data;

[0035] The generation module is used to merge the new files generated in each of the presses to generate a new parameterized file.

[0036] In addition, to achieve the above objectives, the present invention also provides a terminal, the terminal comprising: a memory, a processor, and a data processing program stored in the memory and executable on the processor, wherein the data processing program, when executed by the processor, implements the steps of the data processing method described above.

[0037] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a data processing program, which, when executed by a processor, implements the steps of the data processing method described above.

[0038] This invention proposes a data processing method, platform, terminal, and medium; the data processing method is applied to a distributed load testing platform. The invention first reads the load testing script and its corresponding parameterized file. When determining the parameterized file for online load testing, the parameterized file is split according to the configured number of load machines, generating multiple split files. These split files are then distributed to various load machines for load testing, and the number of lines read from each split file is marked during the testing process. After load testing of the split files, consumable data is removed from the read split files by each load machine to obtain the remaining data, which is then used to generate new files. The new files generated by each load machine are merged to generate a new parameterized file, thereby resolving the problem of duplicate and conflicting parameterized file data used by the script under distributed load testing, meeting the requirements for processing consumable parameter data, and improving performance testing efficiency. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention;

[0040] Figure 2 This is a flowchart illustrating the first embodiment of the data processing method of the present invention;

[0041] Figure 3 This is a schematic diagram of the basic architecture of the distributed load testing platform for the data processing method of the present invention;

[0042] Figure 4This is a flowchart illustrating the stress testing and debugging process of the parameterized file in the data processing method of the present invention.

[0043] Figure 5 This is a schematic diagram illustrating the processing of the parameterized file before and after stress testing in the data processing method of the present invention.

[0044] Figure 6 This is a schematic diagram of a sub-process of the first embodiment of the data processing method of the present invention;

[0045] Figure 7 This is a flowchart illustrating the second embodiment of the data processing method of the present invention;

[0046] Figure 8 This is a flowchart illustrating the third embodiment of the data processing method of the present invention;

[0047] Figure 9 This is a flowchart illustrating the fourth embodiment of the data processing method of the present invention;

[0048] Figure 10 This is a schematic diagram of the thread-independent read switch for the data processing method of the present invention;

[0049] Figure 11 This is a flowchart illustrating the fifth embodiment of the data processing method of the present invention;

[0050] Figure 12 This is a schematic diagram of the functional modules of the first embodiment of the data processing device of the present invention.

[0051] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0053] like Figure 1 As shown, Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0054] In this embodiment of the invention, the terminal device can be a mobile terminal or a server device.

[0055] like Figure 1As shown, the device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0056] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0057] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a data processing program.

[0058] The operating system is a program that manages and controls device and software resources, and supports the operation of the network communication module, user interface module, data processing program, and other programs or software; the network communication module is used to manage and control the network interface 1002; and the user interface module is used to manage and control the user interface 1003.

[0059] exist Figure 1 In the terminal device shown, the terminal device calls the data processing program stored in the memory 1005 through the processor 1001 and executes the operations in the various embodiments of the data processing method described below.

[0060] Based on the above hardware structure, an embodiment of the data processing method of the present invention is proposed.

[0061] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the data processing method of the present invention, the data processing method comprising:

[0062] Step S10: Read in the load testing script and the parameterized file corresponding to the load testing script. When it is determined that the parameterized file is to be used for online load testing, the parameterized file is divided according to the configured number of load machines to generate multiple corresponding segmented files.

[0063] Step S20: Distribute the multiple segmented files to each pressure machine for pressure testing, and mark the number of file lines read in the segmented files during the pressure testing process;

[0064] Step S30: The consumable data in the read segmented files is removed by each of the presses to obtain the corresponding remaining data, and the remaining data is used to generate a new file;

[0065] Step S40: Merge the new files generated in each of the presses to generate a new parameterized file.

[0066] This embodiment reads the load testing script and its corresponding parameterized file. When online load testing is performed on the parameterized file, it splits the file according to the configured number of load machines, generating multiple split files. These split files are then distributed to various load machines for load testing, and the number of lines read from each split file is marked during the testing process. After load testing, each load machine removes consumable data from the read split files to obtain the remaining data, which is then used to generate new files. The new files generated by each load machine are then merged to create a new parameterized file. This resolves the issue of duplicate and conflicting parameterized file data used by the script in distributed load testing, meets the requirements for processing consumable parameter data, and improves performance testing efficiency.

[0067] The following will provide a detailed explanation of each step:

[0068] Step S10: Read in the load testing script and the parameterized file corresponding to the load testing script. When determining that the parameterized file is to be used for online load testing, the parameterized file is divided according to the configured number of load machines to generate multiple corresponding segmented files.

[0069] In this embodiment, the data processing method is applied to a distributed load testing platform. For the processing solution of consumable data requirements, in the distributed load testing environment, before the load test, all parameterized files involved in a load testing script are read from the specified script, the block size is calculated for each file, and then the files are evenly split and distributed to the load machines associated with this load test. After the load test, each load machine checks whether the data is consumable according to the user's configuration requirements. If so, the used data rows are deleted, new files are regenerated, and finally, the files from each load machine are automatically merged into a new file to replace the old files. This process is repeated for all used parameterized files. Therefore, the data processing method of this invention solves the problems of data duplication and conflict in parameterized files and consumable data requirements (data used only once), replacing the user's manual operation and data preparation work. (Refer to...) Figure 3 , Figure 3This is a schematic diagram of the basic architecture of a distributed load testing platform. The basic architecture of the distributed load testing platform consists of a main control unit load testing core module (hereinafter referred to as "main control") that controls and manages one or more load machines.

[0070] Users upload load testing scripts (also known as "jmx files") and corresponding parameterized files to the distributed load testing platform, and the platform determines whether to perform online debugging or online load testing on the parameterized file.

[0071] Reference Figure 4 , Figure 4 This is a flowchart illustrating the stress testing and debugging process for a parameterized file. When determining whether to perform online stress testing on the parameterized file, that is, before stress testing the source parameter file, the stress testing core module splits the parameterized file according to the configured number of pressure machines. After splitting, multiple corresponding split files are generated, and the file order is marked for each pressure machine. (Refer to...) Figure 5 , Figure 5 The diagram shows the processing of the parameterized file before and after load testing (left image).

[0072] Among them, stress testing refers to examining the maximum load that the system under test can withstand and helping to identify the bottleneck of the system under test; at the same time, it is to maintain the online processing capacity and stability of the system under test within a standard range.

[0073] Step S20: Distribute the multiple segmented files to each pressure machine for pressure testing, and mark the number of file lines read in the segmented files during the pressure testing process.

[0074] In this embodiment, the load testing core module distributes the load testing script and multiple segmented files to various load testers for load testing. After the load test starts, each load tester calls the load testing script and the assigned segmented files to perform load testing, and marks the number of lines read in each segmented file during the load test. This allows for querying the number of lines read in each segmented file after the load test is completed, enabling further processing of the segmented files. By having each load tester call the load testing script and the assigned segmented files to perform load testing, the serious data conflicts and duplication problems caused by multiple load testers using the same parameterized files in existing technologies are resolved.

[0075] Step S30: Consumable data in the read segmented files is removed by each of the presses to obtain the corresponding remaining data, and the remaining data is used to generate a new file.

[0076] In this embodiment, after the segmented files assigned to each pressure tester undergo load testing, based on the number of lines read from the segmented files, each pressure tester removes consumable data from the read segmented files to obtain the remaining data corresponding to each segmented file. This remaining data is then used by each pressure tester to generate a new file. Consumable data refers to data in the parameterized file that is used only once; for example, user IDs, order numbers, and refund serial numbers derived from the file. This automatically clears used consumable data after the load test, eliminating the need for manual cleanup after each load test, thus improving data processing efficiency and reducing labor costs.

[0077] Step S40: Merge the new files generated in each of the presses to generate a new parameterized file.

[0078] In this embodiment, refer to Figure 5 , Figure 5 The diagram on the right shows the processing of the parameterized file before and after the pressure test. After each pressure machine generates a new file, the core module of the pressure test collects the new files generated in each pressure machine and merges them in the order of the previously marked files to generate a new parameterized file, so that the new parameterized file is as close as possible to the original parameterized file.

[0079] Further, step S40 includes: collecting the new files generated in each of the presses through the pressure testing core module, and traversing the new files generated in each of the presses to assemble each new file into a corresponding new parameterized file in a preset order.

[0080] In this embodiment, after each press generates a new file from the remaining data, the core module of the pressure test collects the new files generated by each press and iterates through the new files generated by each press. Then, each new file is concatenated according to the previously marked file order to generate a corresponding new parameterized file, so that the new parameterized file is as close as possible to the original parameterized file; thereby ensuring that the data order of the new parameterized file is consistent with the data order of the parameterized file.

[0081] Furthermore, referring to Figure 6 After step S40, the data processing method further includes:

[0082] Step A10: Replace the parameterized file with the new parameterized file through the stress testing core module;

[0083] Step A20: In response to the load testing operation for the new parameterized file, the load testing module divides the new parameterized file according to the configured number of load testers to obtain multiple corresponding block files, and distributes the multiple block files to each load tester for load testing, and marks the number of file lines read in the block files during the load testing process.

[0084] In this embodiment, after the load testing core module regenerates a new parameterized file, the parameterized file in the distributed load testing platform is replaced with the new parameterized file. When a load testing operation targeting the new parameterized file is detected, the load testing module divides the new parameterized file according to the configured number of load machines to obtain multiple block files corresponding to the new parameterized file. These multiple block files are then distributed to various load machines for load testing, and the number of lines read in each block file is marked during the load testing process. Thus, in a distributed multi-load machine environment, the new parameterized file is split into multiple block files according to the load machine configuration strategy for each load machine to use. After the load testing is completed, the used consumable data is automatically cleared, thereby ensuring that the parameterized file can be reused repeatedly.

[0085] This embodiment reads the load testing script and its corresponding parameterized file. When online load testing is performed on the parameterized file, it splits the file according to the configured number of load machines, generating multiple split files. These split files are then distributed to various load machines for load testing, and the number of lines read from each split file is marked during the testing process. After load testing, each load machine removes consumable data from the read split files to obtain the remaining data, which is then used to generate new files. The new files generated by each load machine are then merged to create a new parameterized file. This resolves the issue of duplicate and conflicting parameterized file data used by the script in distributed load testing, meets the requirements for processing consumable parameter data, and improves performance testing efficiency.

[0086] Furthermore, based on the first embodiment of the data processing method of the present invention, a second embodiment of the data processing method of the present invention is proposed.

[0087] The difference between the second embodiment of the data processing method and the first embodiment is that in this embodiment, step S10, which involves reading the load testing script and the corresponding parameterized file, and determining the parameterized file for online load testing, involves segmenting the parameterized file according to the configured number of load machines to generate multiple corresponding segmented files. Figure 7 This step specifically includes:

[0088] Step S11: Read in the load testing script and the parameterized file corresponding to the load testing script, and select to perform online debugging or load testing on the parameterized file;

[0089] Step S12: When determining the parameterized file for online load testing, the parameterized file is read in through the load testing core module, and the parameterized file is split into multiple segmented files according to the configured number of load machines based on the load machine configuration mechanism, and the order of the multiple segmented files is marked.

[0090] In this embodiment, the load testing script and its corresponding parameterized file are read in, and the parameterized file is selected for online debugging or load testing. When it is determined that the parameterized file will be used for online load testing, the load testing core module reads the parameterized file and splits it into multiple segments according to the configured number of load machines based on the load machine configuration mechanism, and marks the order of the multiple segments. By dividing the parameterized file into blocks according to the user's load testing configuration mechanism and marking the order of the split files, the serious data conflict and duplication problem caused by multiple load machines using the same parameterized file during the load testing process in the prior art is solved.

[0091] The following will provide a detailed explanation of each step:

[0092] Step S11: Read in the load testing script and the parameterized file corresponding to the load testing script, and select to perform online debugging or load testing on the parameterized file.

[0093] In this embodiment, the user uploads the load testing script and the corresponding parameterized file to the distributed load testing platform, and chooses to perform online debugging or online load testing on the parameterized file.

[0094] Among these steps, when determining to perform online load testing on the parameterized file, the load testing script is operated on the operation page of the distributed load testing platform, and the parameterized file needs to be split when executing the load testing operation;

[0095] Specifically, when determining to perform online debugging of parameterized files, the debugging script is operated on the operation page of the distributed load testing platform, and it is not necessary to split the parameterized files when performing debugging operations.

[0096] Step S12: When determining the parameterized file for online load testing, the parameterized file is read in through the load testing core module, and the parameterized file is split into multiple segmented files according to the configured number of load machines based on the load machine configuration mechanism, and the order of the multiple segmented files is marked.

[0097] In this embodiment, when selecting a parameterized file for online load testing, the load testing core module reads the parameterized file and calculates the total number of lines N. Based on the number of load machines configured for this load test according to the user's settings, the number of lines S to be split is calculated as: (Total number of lines N / Number of load machines). The parameterized file is then split according to the calculated number of lines S, resulting in multiple split files corresponding to the parameterized file, and the order of these split files is marked. Specifically, split file 1 consists of lines 1 to S, split file 2 consists of lines S+1 to 2S, and the last split file contains all lines up to the end of the filename. The load testing core module distributes these split files to various load machines for their dedicated use. The load machine configuration mechanism is used to characterize the splitting of the parameterized file according to the number of load machines configured for this load test according to the user's settings.

[0098] In this embodiment, the load testing script and its corresponding parameterized file are read in, and the parameterized file is selected for online debugging or load testing. When it is determined that the parameterized file will be used for online load testing, the parameterized file is read in through the load testing core module, and based on the load machine configuration mechanism, the parameterized file is split into multiple segmented files according to the configured number of load machines, and the order of the multiple segmented files is marked. By dividing the parameterized file into blocks according to the user's load testing configuration mechanism and marking the order of the segmented files, the serious data conflict and duplication problem caused by multiple load machines using the same parameterized file during the load testing process in the prior art is solved.

[0099] Furthermore, based on the first and second embodiments of the data processing method of the present invention, a third embodiment of the data processing method of the present invention is proposed.

[0100] The difference between the third embodiment of the data processing method and the first and second embodiments is that this embodiment follows step S10, which involves reading the load testing script and the parameterized file corresponding to the load testing script, and then refers to... Figure 8 The data processing method further includes:

[0101] Step B10: In response to the online debugging operation for the parameterized file, the parameterized file is debugged via a debugging machine, and the number of second file lines read in the parameterized file is marked.

[0102] Step B20: After debugging the parameterized file, the unused data in the parameterized file is determined by the debugging machine based on the number of lines in the second file, and the unused data is used to generate a second parameterized file.

[0103] Step B30: Replace the parameterized file with the second parameterized file through the stress testing core module.

[0104] In this embodiment, after reading the load testing script and the corresponding parameterized file, when it is determined that the parameterized file will be debugged online, the parameterized file is debugged by a debugging machine, and the number of second file lines read in the parameterized file is marked. After the parameterized file is debugged, the unused data in the parameterized file is determined by the debugging machine according to the number of second file lines, and the unused data is used to generate a second parameterized file. The load testing core module replaces the parameterized file with the second parameterized file. Thus, the distributed load testing platform is verified by online debugging of the parameterized file.

[0105] The following will provide a detailed explanation of each step:

[0106] Step B10: In response to the online debugging operation for the parameterized file, the parameterized file is debugged via a debugging machine, and the number of second file lines read in the parameterized file is marked.

[0107] In this embodiment, when an online debugging operation targeting a parameterized file is detected, the debugging machine executes a script using JMeter to debug the parameterized file according to the number of debugging attempts configured by the user. During the execution, the parameterized file is read and the number of second file lines read in the parameterized file is recorded. The number of second file lines is the number of file lines read when the parameterized file is debugged online.

[0108] Step B20: After debugging the parameterized file, the unused data in the parameterized file is determined by the debugging machine based on the number of lines in the second file, and the unused data is used to generate a second parameterized file.

[0109] In this embodiment, after debugging the parameterized file, if the data in the parameterized file that has been read is consumable data, the used data is removed; and based on the second file line number mentioned above, the debugger determines the unused data in the parameterized file, and generates a second parameterized file from the unused data.

[0110] Step B30: Replace the parameterized file with the second parameterized file through the stress testing core module.

[0111] In this embodiment, after generating the second parameterized file, the load testing core module replaces the parameterized file in the distributed load testing platform with the second parameterized file. Then, when a load testing operation targeting the second parameterized file is detected, the load testing module segments the second parameterized file according to the configured load level to obtain multiple block files corresponding to the second parameterized file. These block files are then distributed to various load machines for load testing. After the load testing is completed, used consumable data is automatically cleared, thus ensuring that the parameterized file can be reused repeatedly. This allows for online debugging of the parameterized file to verify the distributed load testing platform and ensure its stability.

[0112] In this embodiment, after reading the load testing script and the corresponding parameterized file, when it is determined that the parameterized file will be debugged online, the parameterized file is debugged by a debugging machine, and the number of second file lines read in the parameterized file is marked. After the parameterized file is debugged, the unused data in the parameterized file is determined by the debugging machine according to the number of second file lines, and the unused data is used to generate a second parameterized file. The load testing core module replaces the parameterized file with the second parameterized file. Thus, the distributed load testing platform is verified by online debugging of the parameterized file.

[0113] Furthermore, based on the first, second, and third embodiments of the data processing method of the present invention, a fourth embodiment of the data processing method of the present invention is proposed.

[0114] The fourth embodiment of the data processing method differs from the first, second, and third embodiments in that this embodiment refines step S20, which involves distributing the multiple segmented files to various pressure machines for stress testing, and marking the number of lines read from the segmented files during the stress testing process. Figure 9 This step specifically includes:

[0115] Step S21: The multiple segmented files are allocated to each pressure machine through the core template of the pressure test, and the pressure test is performed by each pressure machine using the allocated segmented files.

[0116] Step S22: During the pressure test performed by each of the presses, the allocated segmented files in each press are divided equally by thread, so that each thread reads a fixed number of regions in the segmented files in an iterative order.

[0117] Step S23: Calculate the start and end positions of each thread's file reading according to the configured thread-independent read switch, to mark the number of lines read in the segmented file; wherein, each thread stores the start and end position pointers of reading the segmented file.

[0118] In this embodiment, multiple segmented files are allocated to various load testers using a core load test template. Each load tester then performs load testing using its assigned segmented files. During the load test, the allocated segmented files are evenly divided across each load tester using threads, ensuring that each thread reads a fixed number of regions within the segmented files in an iterative order. Based on a configured thread-independent read switch, the start and end positions of each thread's file reading are calculated to mark the number of lines read from the segmented files. Each thread stores its start and end position pointers for reading the segmented files. This allows each load tester to record the number of lines used during the load test, enabling the removal of consumable data after load test debugging.

[0119] The following will provide a detailed explanation of each step:

[0120] Step S21: The multiple segmented files are allocated to each pressure machine through the core template of the pressure test, and the pressure test is performed by each pressure machine using the allocated segmented files.

[0121] In this embodiment, in a distributed multi-pressure machine environment, multiple segmented files are allocated to each pressure machine according to the number of pressure machines configured by the user, so that each pressure machine can use them for its own purposes; each pressure machine uses the allocated segmented files and pressure test scripts to perform pressure test processing.

[0122] Step S22: During the pressure test performed by each of the presses, the allocated segmented files in each press are divided equally by thread, so that each thread reads a fixed number of regions in the segmented files in an iterative order.

[0123] In this embodiment, during the pressure test performed by each pressure machine, the allocated segmented files in each pressure machine are divided equally by thread, so that each thread reads a fixed number of regions in the segmented files in an iterative order; thereby resolving the data conflict problem that occurs when each thread reads data.

[0124] Step S23: Calculate the start and end positions of each thread's file reading according to the configured thread-independent read switch, to mark the number of lines read in the segmented file; wherein, each thread stores the start and end position pointers of reading the segmented file.

[0125] In this embodiment, refer to Figure 10 , Figure 10This diagram illustrates the thread-independent read switch. The configured thread-independent read switch is turned on, and based on this switch, the start and end positions of each thread's file reading are calculated to mark the number of lines read in the segmented file. Each thread stores a pointer to the start and end positions of the segmented file. Based on these pointers, the number of lines read in the segmented file is determined. Furthermore, each load tester records the number of lines used during the load test to remove consumable data after load testing and debugging.

[0126] In this embodiment, multiple segmented files are allocated to various load testers using a core load test template. Each load tester then performs load testing using its assigned segmented files. During the load test, the allocated segmented files are evenly divided across each load tester using threads, ensuring that each thread reads a fixed number of regions within the segmented files in an iterative order. Based on a configured thread-independent read switch, the start and end positions of each thread's file reading are calculated to mark the number of lines read from the segmented files. Each thread stores its start and end position pointers for reading the segmented files. This allows each load tester to record the number of lines used during the load test, enabling the removal of consumable data after load test debugging.

[0127] Furthermore, based on the first, second, third, and fourth embodiments of the data processing method of the present invention, a fifth embodiment of the data processing method of the present invention is proposed.

[0128] The fifth embodiment of the data processing method differs from the first, second, third, and fourth embodiments in that this embodiment refines step S30, which involves using various pressure machines to remove consumable data from the read segmented files to obtain the corresponding remaining data. (Refer to...) Figure 11 This step specifically includes:

[0129] Step S31: After the load test of the segmented file is completed, query the file lines that have been read in the segmented file, and remove the consumable data in the file lines that have been read in the parameterized file through each of the load testers to obtain the corresponding sub-remaining data;

[0130] Step S32: Based on the number of file lines read in the segmented file, determine the unused data in the segmented file through each of the pressure machines, and determine the sub-remaining data and the unused data as remaining data.

[0131] This embodiment queries the lines of file that have been read in the segmented file after the load test is completed, and removes the consumable data from the lines of file that have been read in the parameterized file through each load tester to obtain the corresponding sub-remaining data; based on the number of lines of file that have been read in the segmented file, the unused data in the segmented file is determined by each load tester, and the sub-remaining data and unused data are determined as remaining data; thus, the used consumable data is automatically cleared after the load test is completed, ensuring that the parameterized file can be reused repeatedly.

[0132] The following will provide a detailed explanation of each step:

[0133] Step S31: After the load test of the segmented file is completed, query the file lines that have been read in the segmented file, and remove the consumable data in the file lines that have been read in the parameterized file through each of the load testers to obtain the corresponding sub-remaining data.

[0134] In this embodiment, after the load test of the segmented file is completed, the file lines that have been read in the segmented file are queried; it is detected whether there is consumable data in the file lines that have been read in the segmented file. If it is determined that there is consumable data in the file lines that have been read in the segmented file, the consumable data in the file lines that have been read in the segmented file is removed by each load tester to obtain the corresponding sub-remaining data.

[0135] Step S32: Based on the number of file lines read in the segmented file, determine the unused data in the segmented file through each of the pressure machines, and determine the sub-remaining data and the unused data as remaining data.

[0136] In this embodiment, based on the number of lines read in the segmented file, unused data in the segmented file is determined by each load tester, and the remaining data is determined by merging the sub-remaining data and the unused data. This automatically clears used consumable data after the load test, ensuring that the parameterized file can be reused.

[0137] In this embodiment, after the load test of the segmented file is completed, the file lines that have been read in the segmented file are queried, and the consumable data in the file lines that have been read in the parameterized file are removed by each load tester to obtain the corresponding sub-remaining data; based on the number of file lines that have been read in the segmented file, the unused data in the segmented file is determined by each load tester, and the sub-remaining data and the unused data are determined as remaining data; thus, the used consumable data is automatically cleared after the load test is completed, ensuring that the parameterized file can be reused repeatedly.

[0138] This invention also provides a distributed load testing platform. (See reference...) Figure 12 The distributed load testing platform of the present invention includes:

[0139] The splitting module 10 is used to read in the load testing script and the parameterized file corresponding to the load testing script. When it is determined that the parameterized file is to be used for online load testing, the parameterized file is split according to the configured number of load machines to generate multiple corresponding split files.

[0140] The reading module 20 is used to distribute the multiple segmented files to various pressure machines for pressure testing, and to mark the number of file lines read in the segmented files during the pressure testing process.

[0141] The elimination module 30 is used to eliminate consumable data in the read segmented files through each of the presses, obtain the corresponding remaining data, and generate a new file from the remaining data;

[0142] The generation module 40 is used to merge the new files generated in each of the presses to generate a new parameterized file.

[0143] In addition, the present invention provides a computer-readable storage medium having a data processing program stored thereon, which, when executed by a processor, implements the steps of the data processing method described above.

[0144] The method implemented when the data processing program running on the processor is executed can be referred to in various embodiments of the data processing method of the present invention, and will not be repeated here.

[0145] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0146] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0148] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A data processing method, characterized in that, The data processing method is applied to a distributed load testing platform, and the data processing method includes the following steps: The load testing script and the corresponding parameterized file are read in. When the parameterized file is determined to be used for online load testing, the parameterized file is divided according to the configured number of load machines to generate multiple corresponding split files. The multiple segmented files are assigned to various pressure machines for pressure testing, and the number of lines read in each segmented file is marked during the pressure testing process; Consumable data in the read segmented files is removed by each of the presses to obtain the corresponding remaining data, and the remaining data is used to generate a new file; The new files generated in each of the presses are merged to generate a new parameterized file; After the step of reading the load testing script and the corresponding parameterized file, the method further includes: In response to the online debugging operation of the parameterized file, the parameterized file is debugged using a debugging machine, and the number of second file lines read in the parameterized file is marked; after the parameterized file is debugged, the unused data in the parameterized file is determined by the debugging machine based on the second file line number, and the unused data is used to generate a second parameterized file; the parameterized file is replaced with the second parameterized file by the load testing core module; After the step of merging the new files generated in each of the presses to generate a new parameterized file, the method further includes: The parameterized file is replaced with the new parameterized file by the load testing core module; in response to the load testing operation for the new parameterized file, the load testing module divides the new parameterized file according to the configured number of load testers to obtain multiple corresponding block files, and distributes the multiple block files to each load tester for load testing, and marks the number of file lines read in the block files during the load testing process.

2. The data processing method as described in claim 1, characterized in that, The steps of reading the load testing script and the corresponding parameterized file, and dividing the parameterized file into multiple segmented files according to the configured number of load machines when determining that the parameterized file is to be used for online load testing, include: Read in the load testing script and the corresponding parameterized file, and select to perform online debugging or load testing on the parameterized file; When determining the parameterized file for online load testing, the parameterized file is read in through the load testing core module, and the parameterized file is split into multiple segmented files according to the configured number of load machines based on the load machine configuration mechanism, and the order of the multiple segmented files is marked.

3. The data processing method as described in claim 1, characterized in that, The step of distributing the multiple segmented files to various pressure machines for pressure testing, and marking the number of lines read from the segmented files during the pressure testing process, includes: The multiple segmented files are allocated to each pressure machine through the core template of the pressure test, and the pressure test is performed by each pressure machine using the allocated segmented files. During the pressure test performed by each of the presses, the allocated segmented files in each press are divided equally by thread, so that each thread reads a fixed number of regions in the segmented files in an iterative order. Based on the configured thread-independent read switch, the start and end positions of each thread reading the file are calculated to mark the number of lines read in the segmented file; wherein, each thread stores the start and end position pointers of reading the segmented file.

4. The data processing method as described in claim 1, characterized in that, The step of removing consumable data from the read segmented files using each of the presses to obtain the corresponding remaining data includes: After the load test of the segmented file is completed, the file lines that have been read in the segmented file are queried, and the consumable data in the file lines that have been read in the segmented file are removed by each of the load testers to obtain the corresponding sub-remaining data; Based on the number of lines read in the segmented file, the unused data in the segmented file is determined by each of the pressure machines, and the sub-remaining data and the unused data are determined as remaining data.

5. The data processing method as described in claim 1, characterized in that, The step of merging the new files generated in each of the presses to generate a new parameterized file includes: The core module of pressure testing collects the new files generated in each of the presses, and iterates through the new files generated in each of the presses, assembling each new file into a corresponding new parameterized file in a preset order.

6. A distributed load testing platform, characterized in that, The distributed load testing platform includes: The splitting module is used to read in the load testing script and the parameterized file corresponding to the load testing script. When it is determined that the parameterized file will be used for online load testing, the parameterized file is split according to the configured number of load machines to generate multiple corresponding split files. The reading module is used to distribute the multiple segmented files to various pressure machines for stress testing, and to mark the number of file lines read in the segmented files during the stress testing process. The elimination module is used to eliminate consumable data from the read segmented files through each of the pressure machines, obtain the corresponding remaining data, and generate a new file from the remaining data; The generation module is used to merge the new files generated in each of the presses to generate a new parameterized file; The debugging module is used to respond to online debugging operations on the parameterized file, perform debugging on the parameterized file through a debugging machine, and mark the number of second file lines read in the parameterized file; after the parameterized file is debugged, based on the second file line number, the debugging machine determines the unused data in the parameterized file, and generates a second parameterized file from the unused data; the load testing core module replaces the parameterized file with the second parameterized file; The segmentation module is used to replace the parameterized file with the new parameterized file through the load testing core module; in response to the load testing operation for the new parameterized file, the load testing module segments the new parameterized file according to the configured number of load machines to obtain multiple corresponding block files, and distributes the multiple block files to each load machine for load testing, and marks the number of file lines read in the block files during the load testing process.

7. A terminal, characterized in that, The terminal includes: a memory, a processor, and a data processing program stored in the memory and executable on the processor, wherein the data processing program, when executed by the processor, implements the steps of the data processing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a data processing program, which, when executed by a processor, implements the steps of the data processing method as described in any one of claims 1 to 5.

Images